Microsoft Foundry Chat Models integration

Integrate with the AzureAIOpenAIApiChatModel chat model using LangChain Python.

This will help you get started with AzureAIOpenAIApiChatModel chat models.

The AzureAIOpenAIApiChatModel class uses the OpenAI-compatible API available in Azure AI Foundry. AI Foundry has several chat models, including AzureOpenAI, Cohere, Llama, Phi-3/4, and DeepSeek-R1, among others. You can find information about their latest models and their costs, context windows, and supported input types in the Azure docs.

Overview

Integration details

Class	Package	Serializable	JS support	Downloads	Version
ChatOpenAI	langchain-openai	✅	✅
ChatAnthropic	langchain-anthropic	✅	✅
AzureAIOpenAIApiChatModel	langchain-azure-ai	✅	✅

Model features

Tool calling	Structured output	Image input	Audio input	Video input	Token-level streaming	Native async	Token usage	Logprobs
✅	✅	✅	❌	❌	✅	✅	✅	✅

Setup

To access AzureAIOpenAIApiChatModel models, you’ll need to create an Azure account, get an API key, and install the langchain-azure-ai integration package.

Credentials

Head to the Azure docs to see how to create your deployment and generate an API key. Once your model is deployed, you click the ‘get endpoint’ button in AI Foundry. This will show you your endpoint and api key. Once you’ve done this, set the environment variables:

import getpass
import os

if not os.getenv("AZURE_AI_PROJECT_ENDPOINT"):
    os.environ["AZURE_AI_PROJECT_ENDPOINT"] = getpass.getpass(
        "Enter your Azure AI project endpoint: "
    )

If you want to get automated tracing of your model calls, you can also set your LangSmith API key by uncommenting below:

os.environ["LANGSMITH_TRACING"] = "true"
os.environ["LANGSMITH_API_KEY"] = getpass.getpass("Enter your LangSmith API key: ")

Installation

The LangChain AzureAIOpenAIApiChatModel integration lives in the langchain-azure-ai package:

pip install -qU langchain-azure-ai

Instantiation

Now we can instantiate our model object and generate chat completions:

from langchain_azure_ai.chat_models import AzureAIOpenAIApiChatModel
from azure.identity import DefaultAzureCredential

llm = AzureAIOpenAIApiChatModel(
    model="gpt-4o",
    credential=DefaultAzureCredential(),
    temperature=0,
    max_tokens=None,
    max_retries=2,
)

Invocation

messages = [
    (
        "system",
        "You are a helpful assistant that translates English to French. Translate the user sentence.",
    ),
    ("human", "I love programming."),
]
ai_msg = llm.invoke(messages)
ai_msg

AIMessage(content="J'adore programmer.", additional_kwargs={}, response_metadata={'model': 'gpt-4o-2024-05-13', 'token_usage': {'input_tokens': 31, 'output_tokens': 4, 'total_tokens': 35}, 'finish_reason': 'stop'}, id='run-c082dffd-b1de-4b3f-943f-863836663ddb-0', usage_metadata={'input_tokens': 31, 'output_tokens': 4, 'total_tokens': 35})

print(ai_msg.content)

J'adore programmer.

---

Connect these docs to Claude, VSCode, and more via MCP for real-time answers.

Edit this page on GitHub or file an issue.