LiteLLM Config — Alcozaky Cookies

Konfigurasi LiteLLM proxy untuk Hermes Agent & AI tools.

Struktur

├── docker-compose.yml    ← Cara jalanin LiteLLM + DB + Redis
├── config.yaml           ← Daftar model AI & load balancing
├── .env.example          ← Template API key (isi sendiri)
├── .gitignore
├── scripts/              ← Helper Python (add/remove key, proxy, etc)
└── README.md

Instalasi dari Awal

1. Clone repo di server tujuan

Pertama bikin token dulu di https://git.alcozaky.my.id/user/settings/applications → Generate New Token (centang repo & read aja).

Terus clone:

git clone https://sakhiy:TOKEN_LO@git.alcozaky.my.id/sakhiy/litellm-config.git /opt/litellm
cd /opt/litellm

Atau kalo clone dari internal network (LXC lain di 192.168.1.x):

git clone http://192.168.1.77:3000/sakhiy/litellm-config.git /opt/litellm
cd /opt/litellm

2. Buat file .env

cp .env.example .env
nano .env   # ← isi semua API key & password

3. Jalanin Docker Compose

docker compose up -d

Cek status:

docker compose ps
docker compose logs -f litellm

4. Test proxy

curl http://localhost:4000/health

# Test pake model
curl http://localhost:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $(grep LITELLM_MASTER_KEY .env | cut -d= -f2)" \
  -d '{
    "model": "primary",
    "messages": [{"role": "user", "content": "halo"}]
  }'

Model Tiers

Tier	Model Name	Provider	Akun	Fungsi
1	`primary`	GPT-4o Mini (OpenRouter)	3	Main — load balanced
2	`secondary`	Llama 3.3 70B (Groq)	3	Fallback — load balanced
3	`tertiary`	GPT-4o Mini (GitHub)	3	Fallback terakhir — load balanced

Port

Service	Port
LiteLLM proxy	4000
PostgreSQL	5432 (internal)
Redis	6379 (internal)

Scripts

Script	Fungsi
`scripts/proxy.py`	AI proxy utama (alternative ke LiteLLM)
`scripts/add-keys.py`	Nambah API key ke proxy
`scripts/copy-keys.py`	Copy key antar config
`scripts/extract-keys.py`	Extract key dari env
`scripts/add-groq3.py`	Nambah Groq akun ke-3
`scripts/rm-gemini.py`	Hapus Gemini key tertentu
`scripts/setup-github.py`	Setup GitHub token ke proxy
`scripts/test.py`	Test koneksi proxy

2.4 KiB Raw Blame History