ML.NET model accuracy drops after deployment to production

Thank you for reaching out.

PredictionEngine is not designed to handle multiple requests at the same time. It is meant for simple or single‑prediction scenarios, and it is not thread-safe.

In a production ASP.NET app, many requests can happen in parallel, and if predictions are made at the same time using this type, it can lead to inconsistent or unreliable outputs. That can look like the model accuracy has dropped, even though the model itself hasn’t changed.

To avoid this, the recommended approach in web applications is to use PredictionEnginePool, which is built to handle concurrent requests safely by managing multiple instances internally.

At the same time, it is also important to make sure the data being passed in production is the same format and processed the same way as during training, because differences there can also affect prediction results.

Please let us know if you require any further assistance, we’re happy to help. If you found this information useful, kindly mark this as "Accept Answer". So that others in the community facing similar issues can easily find the solution. Your contribution is highly appreciated.

PredictionEngine is not safe for concurrent ASP.NET requests.

PredictionEngine is explicitly documented as:

• A convenience API for single predictions.
• Not thread-safe.
• Acceptable only in single-threaded or prototype environments.

In a production ASP.NET Core application handling concurrent requests, using a shared PredictionEngine instance can cause race conditions and corrupt internal state, which can manifest as unstable or degraded prediction accuracy after deployment.

For production and concurrent scenarios, use PredictionEnginePool instead. PredictionEnginePool:

• Creates an ObjectPool of PredictionEngine instances.
• Provides improved performance and thread safety.
• Can automatically reload models when the underlying file changes (when configured with watchForChanges: true).

In an ASP.NET Core app, register it in Program.cs like this:

builder.Services.AddPredictionEnginePool<ModelInput, ModelOutput>()
    .FromFile(modelName: "SentimentAnalysisModel", filePath: "sentiment_model.zip", watchForChanges: true);

Then inject and use the pool in controllers/services instead of a single PredictionEngine instance.

Using PredictionEnginePool is the recommended approach for production, concurrent ASP.NET workloads and can help prevent the kind of reliability issues observed after deployment.

References:

• Deploy a model in an ASP.NET Core Web API
• Make predictions with a trained model
• Tutorial: Categorize support issues using multiclass classification with ML.NET
• Tutorial: Analyze sentiment of website comments with binary classification in ML.NET
• Tutorial: Train an ML.NET classification model to categorize images
• Tutorial: Categorize iris flowers using k-means clustering with ML.NET
• Tutorial: Build a movie recommender using matrix factorization with ML.NET