LlamaCpp

运行 llama.cpp 服务器二进制文件以启动 API 服务器。如果是在远程服务器上运行，请务必将主机设置为 0.0.0.0

.\server.exe -c 4096 --host 0.0.0.0 -t 16 --mlock -m models\meta\llama\codellama-7b-instruct.Q8_0.gguf

启动并运行后，修改 ~/.continue/config.json 使其看起来像这样

YAML

config.yaml

models:
  - name: Llama CPP
    provider: llama.cpp
    model: MODEL_NAME
    apiBase: http://localhost:8080

JSON

config.json

{
  "models": [
    {
      "title": "Llama CPP",
      "provider": "llama.cpp",
      "model": "MODEL_NAME",
      "apiBase": "http://localhost:8080"
    }
  ]
}

查看源代码