In this post, we explore two terms which are becoming relatively common in professional machine learning applications – MLOps and DevOps
The term MLOps refers to a set of techniques and practises for data scientists to collaborate operations professionals.. MLOps aims to manage deployment of machine learning and deep learning models in large-scale production environments.
The term DevOps comes from the software engineering world and is concerned with developing and operating large-scale software systems. DevOps introduces two concepts: Continuous Integration (CI) and Continuous Delivery (CD). DevOps aims to shorten development cycles, increase deployment velocity and create dependable releases.
Since, an ML system is a software system, DevOps principles also apply to MLOps.
However, there are differences between the two.
Before we explore the differences between MLOps and DevOps, let us look at the overall flow of deploying an ML model intro production.
The overall steps for deploying an ML/DL model in production are:
- Data extraction
- Data analysis
- Data preparation
- Model training
- Model evaluation
- Model serving
- Model monitoring
With this background, here are the differences between MLOps and DevOps
- Unlike DevOps, MLOps is much more experimental in nature. Data scientists try different features, parameters, models. In all these iterations, they must manage the code base and create reproducible results.
- Performance degradation of the system due to changing data profiles: ML and DL systems are impacted by changing data profiles. This is not the case in a traditional IT system. Hence, the model has to be refreshed even if it ‘works’ currently – leading to more iterations in the pipeline
- Model monitoring: Models in production need to be monitored. Similarly, the summary statistics of data that built the model needs to be monitored so that we can refresh models when needed.
- Automation of steps before model building: There are many steps before a model is built that may need to be automated.
- Team composition: The team needed to build and deploy models in production may not always comprise of software engineers. For example, the members working with exploratory data analysis maybe business experts as opposed to software engineers.
- Testing: Testing an ML system involves model validation, model training etc – in addition to the software tests such as unit testing and integration testing.
To conclude:
- Continuous integration (CI) for MLOps also involves validating the data and the schema in addition to testing code.
- Continuous deployment(CD) validating the performance of models in production – including the ability to deploy new models and rollback changes from a model.
- We need a new term –Continuous testing (CT) to retrain and serve models.
Image: represents each step in model deployment. Image source – Google
References: continuous delivery and automation pipelines in machine learning
