AWS SageMaker

SageMaker 可用于聊天模型和嵌入模型。使用 LMI 部署的终端节点支持聊天模型，使用 HuggingFace TEI 部署的终端节点支持嵌入模型。

以下是 SageMaker 配置示例

YAML

config.yaml

models:
  - name: deepseek-6.7b-instruct
    provider: sagemaker
    model: lmi-model-deepseek-coder-xxxxxxx
    region: us-west-2
    roles:
      - chat
  - name: mxbai-embed
    provider: sagemaker
    model: mxbai-embed-large-v1-endpoint
    roles:
      - embed

JSON

config.json

{
  "models": [
    {
      "title": "deepseek-6.7b-instruct",
      "provider": "sagemaker",
      "model": "lmi-model-deepseek-coder-xxxxxxx",
      "region": "us-west-2"
    }
  ],
  "embeddingsProvider": {
    "provider": "sagemaker",
    "model": "mxbai-embed-large-v1-endpoint"
  }
}

model 中的值应为您部署的 SageMaker 终端节点名称。

认证将通过 ~/.aws/credentials 文件中名为 "sagemaker" 的配置文件下的临时或长期凭证进行。

[sagemaker]
aws_access_key_id = abcdefg
aws_secret_access_key = hijklmno
aws_session_token = pqrstuvwxyz # Optional: means short term creds.