ML Model Deployment Strategies
January 20, 2025•257 words
(Part I of the study notes for the MLOps Concepts course on DataCamp (link))
Say, you have updated an ML model and want to deploy it in production to test its performance on real, unseen data. There are three possible strategy: Basic, Shadow, and Canary.
- Basic Deployment: Retire the old model immediately, and direct all production data to the new model.
- Shadow Deployment: Keep the old model running, and pass production data to both the old and new model. Real data are still processed by the old model. Meanwhile, collect and analyze the predictions of both models. Retire the old model only after checking everything is working as expected for the new model.
- Canary Deployment: Use both the old and new models in production, but only direct a small portion of real data to the new model.
After a bit of searching, I think these strategies/concepts actually originate from software development. Here is a useful blogpost that dives into the same topic from a software perspective.
DataCamp also provided the following table that briefly compares these strateties.
| Strategy | Pros | Cons |
|---|---|---|
| Basic | Straightforward, easy to implement, low resources | High risk if the model does not work as expected |
| Shadow | Easy to implement, no risk of the model does not work as expected | Double resources |
| Canary | Small risk if model does not work as expected | Slightly harder to implement, medium amount of resources |