Use admin-connected models in cloud evaluations

Admin-connected models are models hosted behind an enterprise AI gateway, such as Azure API Management or a non-Azure AI model gateway, that an administrator connects to your Foundry project. You can use an admin-connected model for cloud evaluation scenarios that accept a model deployment.

Foundry resolves the connection endpoint and authentication, including API key, managed identity, or OAuth 2.0 authentication. Your evaluation request references the connection and deployment, not the gateway endpoint or its credentials.

Prerequisites

• A Foundry project.
• Foundry User role on the Foundry project.
• An administrator has created an Azure API Management or non-Azure AI model gateway connection in your Foundry project and added the model on the Admin-connected models tab. For setup instructions, see Bring your own model to Foundry Agent Service.
• The connection name and deployment name for a model that supports the OpenAI Chat Completions API.

Reference an admin-connected model

Use the following format anywhere a supported evaluation scenario accepts a model deployment:

<connection-name>/<deployment-name>

The following table shows common evaluation surfaces and the field that accepts the reference:

Use an admin-connected model as an evaluator judge model

Set initialization_parameters.model when you configure an AI-assisted evaluator. This example uses the admin-connected model as the judge for the coherence evaluator:

from azure.ai.projects.models import TestingCriterionAzureAIEvaluator

admin_connected_model = "my-apim-connection/gpt-4o"

testing_criteria = [
    TestingCriterionAzureAIEvaluator(
        type="azure_ai_evaluator",
        name="coherence",
        evaluator_name="builtin.coherence",
        initialization_parameters={"model": admin_connected_model},
        data_mapping={
            "query": "{{item.query}}",
            "response": "{{item.response}}",
        },
    ),
]

Use an admin-connected model as a target

Set target.model to send each evaluation input to the admin-connected model:

admin_connected_model = "my-apim-connection/gpt-4o"

target = {
    "type": "azure_ai_model",
    "model": admin_connected_model,
    "sampling_params": {
        "top_p": 1.0,
        "max_completion_tokens": 2048,
    },
}

Use this target with the model target evaluation flow described in Run evaluations in the cloud.

Use other evaluation scenarios

Other model-based scenarios in Run evaluations in the cloud by using the Microsoft Foundry SDK work similarly with admin-connected models. Wherever the scenario accepts a supported model deployment, replace the deployment name with <connection-name>/<deployment-name>. For example, conversation simulation accepts this reference in item_generation_params.model.

Keep the Foundry project endpoint unchanged, and don't add the gateway endpoint or credentials to the evaluation request. Foundry resolves those values from the admin-connected model connection.

Related content

• Run evaluations in the cloud by using the Microsoft Foundry SDK
• Bring your own model to Foundry Agent Service