Public Access
88 lines
2.1 KiB
Markdown
88 lines
2.1 KiB
Markdown
# LiteLLM Config — Alcozaky Cookies
|
|
|
|
Konfigurasi LiteLLM proxy untuk Hermes Agent & AI tools.
|
|
|
|
## Struktur
|
|
|
|
```
|
|
├── docker-compose.yml ← Cara jalanin LiteLLM + DB + Redis
|
|
├── config.yaml ← Daftar model AI & load balancing
|
|
├── .env.example ← Template API key (isi sendiri)
|
|
├── .gitignore
|
|
├── scripts/ ← Helper Python (add/remove key, proxy, etc)
|
|
└── README.md
|
|
```
|
|
|
|
## Instalasi dari Awal
|
|
|
|
### 1. Clone repo di server tujuan
|
|
|
|
Clone:
|
|
```bash
|
|
sudo git clone https://git.alcozaky.my.id/sakhiy/litellm-config.git /opt/litellm
|
|
cd /opt/litellm
|
|
```
|
|
|
|
### 2. Buat file .env
|
|
|
|
```bash
|
|
cp .env.example .env
|
|
nano .env # ← isi semua API key & password
|
|
```
|
|
|
|
### 3. Jalanin Docker Compose
|
|
|
|
```bash
|
|
docker compose up -d
|
|
```
|
|
|
|
Cek status:
|
|
```bash
|
|
docker compose ps
|
|
docker compose logs -f litellm
|
|
```
|
|
|
|
### 4. Test proxy
|
|
|
|
```bash
|
|
curl http://localhost:4000/health
|
|
|
|
# Test pake model
|
|
curl http://localhost:4000/v1/chat/completions \
|
|
-H "Content-Type: application/json" \
|
|
-H "Authorization: Bearer $(grep LITELLM_MASTER_KEY .env | cut -d= -f2)" \
|
|
-d '{
|
|
"model": "primary",
|
|
"messages": [{"role": "user", "content": "halo"}]
|
|
}'
|
|
```
|
|
|
|
## Model Tiers
|
|
|
|
| Tier | Model Name | Provider | Akun | Fungsi |
|
|
|------|-----------|----------|------|--------|
|
|
| 1 | `primary` | GPT-4o Mini (OpenRouter) | 3 | Main — load balanced |
|
|
| 2 | `secondary` | Llama 3.3 70B (Groq) | 3 | Fallback — load balanced |
|
|
| 3 | `tertiary` | GPT-4o Mini (GitHub) | 3 | Fallback terakhir — load balanced |
|
|
|
|
## Port
|
|
|
|
| Service | Port |
|
|
|---------|------|
|
|
| LiteLLM proxy | 4000 |
|
|
| PostgreSQL | 5432 (internal) |
|
|
| Redis | 6379 (internal) |
|
|
|
|
## Scripts
|
|
|
|
| Script | Fungsi |
|
|
|--------|--------|
|
|
| `scripts/proxy.py` | AI proxy utama (alternative ke LiteLLM) |
|
|
| `scripts/add-keys.py` | Nambah API key ke proxy |
|
|
| `scripts/copy-keys.py` | Copy key antar config |
|
|
| `scripts/extract-keys.py` | Extract key dari env |
|
|
| `scripts/add-groq3.py` | Nambah Groq akun ke-3 |
|
|
| `scripts/rm-gemini.py` | Hapus Gemini key tertentu |
|
|
| `scripts/setup-github.py` | Setup GitHub token ke proxy |
|
|
| `scripts/test.py` | Test koneksi proxy |
|