Skip to content

Provider Configuration

Configure multiple providers to switch by model prefix without rebuilding the gateway. DeepIntShield registers 29 built-in provider identities, including the following newly documented slices:

  • DeepSeek: text, chat, Responses, discovery, and SSE.
  • Amazon Bedrock Mantle: regional OpenAI-compatible chat, Responses lifecycle, compaction, embeddings, files, and token counting.
  • Sarvam AI: multilingual chat, TTS, streaming speech, and transcription.
  • Wafer: live per-model capabilities/pricing, chat, Responses, text, and bounded upload.

Provider Configuration Interface

  1. Go to https://app.deepintshield.com
  2. Navigate to “Model Hub” → “AI Providers” in the sidebar
  3. Select a provider (e.g. OpenAI, Anthropic, Mistral) and add its API keys
  4. To add a self-hosted OpenAI-compatible server (e.g. vLLM), click Custom to add a custom provider, set its Base URL, and choose OpenAI as the Base Format

Once providers are configured, you can make requests to any specific provider. This example shows how to send a request directly to OpenAI’s GPT-4o Mini model. DeepIntShield handles the provider-specific API formatting automatically.

Terminal window
curl --location 'https://app.deepintshield.com/v1/chat/completions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer sk-ds-your-virtual-key' \
--data '{
"model": "openai/gpt-4o-mini",
"messages": [
{"role": "user", "content": "Hello!"}
]
}'

Bare model names (automatic provider resolution)

Section titled “Bare model names (automatic provider resolution)”

The provider/ prefix is optional. If you send a bare model name, DeepIntShield resolves the provider from the model catalog automatically - so "model": "gpt-4o-mini" routes to OpenAI, "gemini-2.5-flash" routes to Gemini, and so on. When a model is served by more than one provider the choice is deterministic (OpenAI is preferred on the OpenAI-compatible endpoints, otherwise the alphabetically-first match), and an explicit provider/model prefix always takes precedence. If the catalog has no match the request is rejected, so an unknown model never silently mis-routes.

Terminal window
# Same call, no provider prefix - resolved via the catalog
--data '{ "model": "gpt-4o-mini", "messages": [{"role": "user", "content": "Hello!"}] }'

Set up your API keys for the providers you want to use. DeepIntShield supports both direct key values and environment variable references with the env. prefix:

Terminal window
export OPENAI_API_KEY="your-openai-api-key"
export ANTHROPIC_API_KEY="your-anthropic-api-key"
export MISTRAL_API_KEY="your-mistral-api-key"
export CEREBRAS_API_KEY="your-cerebras-api-key"
export GROQ_API_KEY="your-groq-api-key"
export COHERE_API_KEY="your-cohere-api-key"
export DEEPSEEK_API_KEY="your-deepseek-api-key"
export AWS_BEARER_TOKEN_BEDROCK="your-short-lived-bedrock-api-key"
export SARVAM_API_KEY="your-sarvam-api-key"
export WAFER_API_KEY="your-wafer-api-key"

Environment Variable Handling:

  • Use "value": "env.VARIABLE_NAME" to reference environment variables
  • Use "value": "sk-proj-xxxxxxxxx" to pass keys directly
  • All sensitive data is automatically redacted in GET requests and UI responses for security

Per-workspace & live: the provider network settings below - base URL, request timeout, retries, concurrency, proxy, TLS, and headers - are scoped to your workspace and applied per request. Saving a change takes effect on your next request with no restart, and one workspace’s settings never affect another.

Distribute requests across multiple API keys or providers based on custom weights. This example shows how to split traffic 70/30 between two OpenAI keys, useful for managing rate limits or costs across different accounts.

AI Providers page with OpenAI selected and its configured key, weight, and Add new key button

  1. Navigate to “Model Hub” → “AI Providers” → “OpenAI”
  2. Click “Add new key” to add multiple keys
  3. Set weight values (0.7 and 0.3)
  4. Save configuration

Use different API keys for specific models, allowing you to manage access controls and billing separately. This example uses a premium key for advanced reasoning models (o1-preview, o1-mini) and a standard key for regular GPT models.

AI Providers page with OpenAI selected and its configured key

  1. Navigate to “Model Hub” → “AI Providers” → “OpenAI”
  2. Add first key with models: ["gpt-4o", "gpt-4o-mini"]
  3. Add premium key with models: ["o1-preview", "o1-mini"]
  4. Save configuration

Override the default API endpoint for a provider. This is useful for connecting to self-hosted models, local development servers, or OpenAI-compatible APIs like vLLM, Ollama, or LiteLLM.

Base URL Configuration Interface

  1. Navigate to “Model Hub” → “AI Providers” → “OpenAI” → “Edit Provider Config” → “Network”
  2. Set Base URL: https://your-custom-endpoint.example.com/v1
  3. Save configuration

Amazon Bedrock Mantle is regional. It defaults to https://bedrock-mantle.us-east-1.api.aws; set the regional Mantle endpoint for any other AWS Region. Do not configure the Bedrock Runtime hostname for the bedrock-mantle provider identity.

Configure retry behavior for handling temporary failures and rate limits. This example sets up exponential backoff with up to 5 retries, starting with 1ms delay and capping at 10 seconds - ideal for handling transient network issues.

Retry Configuration Interface

  1. Navigate to “Model Hub” → “AI Providers” → “OpenAI” → “Edit Provider Config” → “Network”
  2. Set Max Retries: 5
  3. Set Initial Backoff: 1 ms
  4. Set Max Backoff: 10000 ms
  5. Save configuration

Fine-tune performance by adjusting worker concurrency and queue sizes per provider (defaults are 1000 workers and 5000 queue size). This example gives OpenAI higher limits (100 workers, 500 queue) for high throughput, while Anthropic gets conservative limits to respect their rate limits.

Concurrency Configuration Interface

  1. Navigate to “Model Hub” → “AI Providers” → {Provider} → “Edit Provider Config” → “Performance”
  2. Set Concurrency: Worker count (100 for OpenAI, 25 for Anthropic)
  3. Set Buffer Size: Queue size (500 for OpenAI, 100 for Anthropic)
  4. Save configuration

DeepIntShield supports two ways to add custom headers to provider requests: static headers configured at the provider level, and dynamic headers passed per-request.

Configure headers that are automatically included in every request to a specific provider. This is useful for provider-specific requirements, API versioning, or organizational metadata.

Extra Headers Configuration Interface

  1. Navigate to “Model Hub” → “AI Providers” → “OpenAI” → “Edit Provider Config” → “Network”
  2. Add headers in the “Extra Headers” section (e.g. x-custom-org: my-organization, x-environment: production)
  3. Save configuration

Send custom headers with individual requests using the x-deepintshield-eh-* prefix. Headers are automatically propagated to the provider after stripping the prefix. This is useful for request-specific metadata, user identification, or custom tracking information.

Terminal window
curl --location 'https://app.deepintshield.com/v1/chat/completions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer sk-ds-your-virtual-key' \
--header 'x-deepintshield-eh-user-id: user-123' \
--header 'x-deepintshield-eh-tracking-id: trace-456' \
--data '{
"model": "openai/gpt-4o-mini",
"messages": [
{"role": "user", "content": "Hello!"}
]
}'

The x-deepintshield-eh- prefix is stripped before forwarding, so x-deepintshield-eh-user-id becomes user-id in the request to the provider.

Example use cases:

  • User identification: x-deepintshield-eh-user-id, x-deepintshield-eh-tenant-id
  • Request tracking: x-deepintshield-eh-correlation-id, x-deepintshield-eh-trace-id
  • Custom metadata: x-deepintshield-eh-department, x-deepintshield-eh-cost-center
  • A/B testing: x-deepintshield-eh-experiment-id, x-deepintshield-eh-variant

For security, DeepIntShield never forwards the following headers to providers, regardless of configuration:

  • proxy-authorization
  • cookie
  • host
  • content-length
  • connection
  • transfer-encoding
  • x-api-key
  • x-goog-api-key
  • x-deepintshield-api-key
  • x-deepintshield-api-key-id
  • x-deepintshield-vk

This denylist is applied to both static and dynamic headers (and blocks attempts to override auth or virtual keys via the x-deepintshield-eh-* prefix) to prevent security vulnerabilities.

Route requests through proxies for compliance, security, or geographic requirements. This example shows both HTTP proxy for OpenAI and authenticated SOCKS5 proxy for Anthropic, useful for corporate environments or regional access.

Proxy Configuration Interface

  1. Navigate to “Model Hub” → “AI Providers” → {Provider} → “Edit Provider Config” → “Proxy”
  2. Select Proxy Type: HTTP or SOCKS5
  3. Set Proxy URL: https://your-proxy.example.com
  4. Add credentials if needed (username/password)
  5. Save configuration

Include the original provider response alongside DeepIntShield’s standardized response format. Useful for debugging and accessing provider-specific metadata.

Raw Response Configuration Interface

  1. Navigate to “Model Hub” → “AI Providers” → {Provider} → “Edit Provider Config” → “Debugging”
  2. Toggle “Send Back Raw Response” to enabled
  3. Save configuration

When enabled, the raw provider response appears in extra_fields.raw_response:

{
"choices": [...],
"usage": {...},
"extra_fields": {
"provider": "openai",
"raw_response": {
// Original OpenAI response here
}
}
}

Include the original request sent to the provider alongside DeepIntShield’s response. Useful for debugging request transformations and verifying what was actually sent to the provider.

Raw Request Configuration Interface

  1. Navigate to “Model Hub” → “AI Providers” → {Provider} → “Edit Provider Config” → “Debugging”
  2. Toggle “Send Back Raw Request” to enabled
  3. Save configuration

When enabled, the raw provider request appears in extra_fields.raw_request:

{
"choices": [...],
"usage": {...},
"extra_fields": {
"provider": "openai",
"raw_request": {
// Original request sent to OpenAI here
}
}
}

Enable passthrough mode for extra parameters. When enabled, any parameters in the extra_params field (or provider-specific extra parameter fields) will be merged directly into the request sent to the provider, bypassing DeepIntShield’s parameter filtering.

Terminal window
curl --location 'https://app.deepintshield.com/v1/chat/completions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer sk-ds-your-virtual-key' \
--header 'x-deepintshield-passthrough-extra-params: true' \
--data '{
"model": "openai/gpt-4o-mini",
"messages": [
{"role": "user", "content": "Hello!"}
],
"extra_params": {
"custom_param": "value",
"another_param": 123,
"nested_param": {
"nested_key": "nested_value"
}
}
}'

When enabled, the extra parameters are merged into the JSON request body sent to the provider. This allows you to pass provider-specific parameters that DeepIntShield doesn’t natively support.

Provider configuration also controls the account and key used by stored Responses, cached content, Realtime sessions, and OCR. Resource IDs stay bound to the provider account that created them, so multi-key deployments must use an explicit configured-key selector on creation and follow-up calls. Review Protocol operations before enabling these routes in an application.

Enterprise cloud providers require additional configuration beyond API keys. Configure Azure, AWS Bedrock, and Google Vertex with platform-specific authentication details.

Azure supports three authentication methods: Managed Identity (the Azure SDK default credential chain), Entra ID (Service Principal), and Direct (API Key).

Managed Identity / Azure default credential chain

Section titled “Managed Identity / Azure default credential chain”

Leave API key and Entra ID credentials empty. DeepIntShield uses the Azure SDK default credential chain, which auto-detects managed identity on Azure VMs, App Service, AKS, and similar environments. Provide only endpoint, deployments, and optionally api_version.

Azure Add new key panel with the Entra ID (Service Principal) authentication method

  1. Navigate to “Model Hub” → “AI Providers” → “Azure” and click “Add new key”
  2. Under Authentication Method, select Entra ID (Service Principal)
  3. Set Client ID: Your Azure Entra ID client ID
  4. Set Client Secret: Your Azure Entra ID client secret
  5. Set Tenant ID: Your Azure Entra ID tenant ID
  6. Set Endpoint: Your Azure endpoint URL
  7. Configure Deployments: Map model names to deployment names
  8. Set API Version: e.g., 2024-08-01-preview
  9. Save configuration

For simpler use cases, provide the authentication credential directly in the value field:

Azure Add new key panel with the API Key authentication method

  1. Navigate to “Model Hub” → “AI Providers” → “Azure” and click “Add new key”
  2. Under Authentication Method, select API Key, then set API Key: Your Azure API key
  3. Set Endpoint: Your Azure endpoint URL
  4. Configure Deployments: Map model names to deployment names
  5. Set API Version: e.g., 2024-08-01-preview
  6. Save configuration

AWS Bedrock supports both explicit credentials and IAM role authentication:

AWS Bedrock Configuration Interface

  1. Navigate to “Model Hub” → “AI Providers” → “AWS Bedrock” and click “Add new key”
  2. Under Authentication Method, select IAM Role (Inherited), Explicit Credentials, or API Key
  3. For Explicit Credentials, set Access Key and Secret Key: your AWS Access Key ID and Secret Access Key
  4. For API Key, set API Key: your AWS API key
  5. Set Region: e.g., us-east-1
  6. Configure Deployments: Map model names to inference profiles
  7. Set ARN: Required for deployments mapping
  8. Save configuration

Notes:

  • If using API Key authentication, set value field to the API key, else leave it empty for IAM role authentication.
  • In IAM role authentication, if both access_key and secret_key are empty, DeepIntShield uses IAM role authentication from the environment.
  • arn is required for URL formation - deployments mapping is ignored without it.
  • When using arn + deployments, DeepIntShield uses model profiles; otherwise forms path with incoming model name directly.
  • ARN vs deployments: Put the ARN prefix in arn and the model/inference profile resource ID only in deployments - never the full ARN in deployments. See How to Use ARNs and Application Inference Profiles for details.

Google Vertex requires project configuration and authentication credentials:

Google Vertex Configuration Interface

  1. Navigate to “Model Hub” → “AI Providers” → “Vertex AI” and click “Add new key”
  2. Set API Key: Your Vertex API key
  3. Set Project ID: Your Google Cloud project ID
  4. Set Region: e.g., us-central1
  5. Set Auth Credentials: Service account credentials JSON
  6. Save configuration

Notes:

  • You can leave both API Key and Auth Credentials empty to use service account authentication from the environment.
  • You must set Project Number in Key config if using fine-tuned models.
  • API Key Authentication is only supported for Gemini and fine-tuned models.
  • You can use custom fine-tuned models by passing vertex/<your-fine-tuned-model-id> or vertex/<model-deployment-alias> if you have set the deployments in the key config.

Now that you understand provider configuration, explore these related topics: