HuggingFace Inference Endpoints

Hugging Face Inference Endpoints 是一种在任何云上设置开源语言模型实例的简单方法。此处注册账户并添加账单，此处访问 Inference Endpoints，点击“New endpoint”，填写表格（例如选择一个模型，如 WizardCoder-Python-34B-V1.0），然后点击“Create Endpoint”部署您的模型。将 ~/.continue/config.json 修改成如下所示

YAML

config.yaml

models:
  - name: Hugging Face Inference API
    provider: huggingface-inference-api
    model: MODEL_NAME
    apiKey: <YOUR_HF_TOKEN>
    apiBase: <YOUR_HF_INFERENCE_API_ENDPOINT_URL>

JSON

config.json

{
  "models": [
    {
      "title": "Hugging Face Inference API",
      "provider": "huggingface-inference-api",
      "model": "MODEL_NAME",
      "apiKey": "<YOUR_HF_TOKEN>",
      "apiBase": "<YOUR_HF_INFERENCE_API_ENDPOINT_URL>"
    }
  ]
}

查看源码