Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
Windows ML accelerates inference across NPUs, GPUs, and CPUs by pairing the ONNX Runtime with hardware-tuned execution providers (EPs). To learn more about execution providers, see the ONNX Runtime docs.
Tip
When your app targets an EP, Windows ML highly recommends pre-compiling your model and caching the result. Pre-compiling turns a multi-second (or even multi-minute) cold start into a one-time cost. See Model compilation and caching for guidance on when to pre-compile, how to pre-compile, and where compiled artifacts are cached per device.
Note
You're still responsible for optimizing your models for different hardware. Windows ML handles execution provider distribution, not model optimization. See AI Toolkit and the ONNX Runtime Tutorials for more info on optimization.
What is an execution provider?
An execution provider (EP) is a component that enables hardware-specific optimizations for machine learning (ML) operations. Execution providers abstract different compute backends (NPU, GPU, and CPU) and provide a unified interface for graph partitioning, kernel registration, and operator execution. To learn more, see the ONNX Runtime docs.
Two ways to get EPs
Windows ML EPs: Use the ExecutionProviderCatalog APIs to acquire Windows-certified EPs that go through a rigorous certification and regression testing process, and are automatically updated. See Windows ML EPs to learn more.
Bring your own: Obtain and reference EP binaries yourself, enabling support for offline environments, managed devices, or strict version-pinning requirements. See Bring your own EPs to learn more.
See Windows ML EPs vs. bring-your-own for tradeoffs.
Silicon-to-EP mapping
| Silicon | Execution providers | Typical use case |
|---|---|---|
| NPU | OpenVINO (Intel) QNN (Qualcomm) VitisAI (AMD) |
Battery-efficient, sustained on-device inference on Copilot+ PCs |
| GPU | MIGraphX (AMD) NvTensorRtRtx (NVIDIA) OpenVINO (Intel) QNN (Qualcomm) DirectML (included - legacy) |
High-throughput image/video/GenAI workloads |
| CPU | OpenVINO (Intel) ORT CPU EP (included) |
Universal fallback; low-latency for small models |
See also
- Windows ML execution providers — EPs available through Windows ML
- Model compilation and caching — pre-compile and per-device caching
- Windows ML EPs vs. bring-your-own — choose the right EP sourcing strategy
- Install Windows ML EPs
- Register Windows ML EPs
- Select execution providers