Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
[This article is prerelease documentation and is subject to change.]
After you create a test set with conversations, run an evaluation to measure your agent's performance. The evaluation processes each conversation and produces scored results based on the selected test method.
Important
- This is a production-ready preview feature.
- Production-ready previews are subject to supplemental terms of use.
Prerequisites
- An agent created and saved with the GitHub Copilot harness. See Create an agent (preview).
- At least one evaluation with conversations. See Create a test set for an agent (preview).
To run an evaluation for an agent:
- Open your agent in Copilot Studio.
- Select the Evaluate tab.
- In the Evaluation dropdown, select the evaluation you want to run.
- Add at least one conversation to the test set if you haven't already.
- In the Configure test set panel, verify:
- The evaluation Name is set.
- The Test method is configured (for example, General quality).
- Select Evaluate to start the evaluation. The evaluation processes each conversation in the test set. Depending on the number of conversations, this process might take several minutes.
Tip
Run the same evaluation multiple times. Each run is saved separately, so you can compare results across runs and see how changes to your agent affect quality.
Re-run an evaluation
After you change your agent's instructions, knowledge, or tools, re-run the evaluation to measure the impact:
- On the Evaluate tab, select the evaluation you want to re-run from the Evaluation dropdown.
- Under Recent results, select Evaluate test set again or select the Evaluate test set icon
within the test set to start a new run. - Compare the new run's results with previous runs to see if the changes improved or degraded performance.