Using environment variables for model providers

Many model providers support setting credentials and other configuration options through environment variables. This is useful for self-hosted deployments where you want to avoid hardcoding sensitive information in your code or configuration files. In LangSmith, most model interactions are done through the playground service, which allows you to configure many of those environment variables directly on the pod itself. This can be useful to avoid having to set credentials in the UI.

Requirements

• A self-hosted LangSmith instance with the playground service running.
• The provider you want to configure must support environment variables for configuration. Check the provider's Chat Model documentation for more information.
• The secrets/roles you may want to attach to the playground service.
  • Note that for IRSA you may need to grant the langsmith-playground service account the necessary permissions to access the secrets or roles in your cloud provider.

Configuration

With the parameters from above, you can configure your LangSmith instance to use environment variables for model providers. You can do this by modifying the langsmith_config.yaml file for your LangSmith Helm Chart installation or the docker-compose.yaml file for your Docker installation.

Helm

playground:
  deployment:
    extraEnv:
      - name: OPENAI_BASE_URL
        value: https://<my_proxy_url>
      - name: OPENAI_API_KEY
        valueFrom:
          secretKeyRef:
            name: <your_secret_name>
            key: api_key
   serviceAccount: # Can be useful if you want to use IRSA or workload identity
     annotations:
       eks.amazonaws.com/role-arn: <your_role_arn>

Docker

# In your docker-compose.yaml file
langchain-playground:
  environment:
    .. # Other environment variables
    - OPENAI_BASE_URL=https://<my_proxy_url>
    - OPENAI_API_KEY=<your_key> # This will be set in the .env file