You Need ModelOps To Scale

You Need ModelOps To Scale

As we scale up our data science pipeline and enterprise AI, we are in a world where DevOps is not enough.

As companies, particularly large organizations, scale up their models as a part of building an enterprise-wide pipeline, there’s an increasing need to operationalize the model development process. Similar to DevOps, models need to be developed, integrated, deployed and monitored. Often, with Enterprise AI initiatives, there are a host of governance considerations such as data integrity, change management, regulatory concerns, etc..

You want to be able to connect your data science pipeline to the IT organization. You want our IT organization to maintain and upgrade your data science pipeline as needed.

In other words, you want your pipeline to be fully functional across your organization, in real-time, and in line with your industry’s regulation.

According to VentureBeat , 87% of data science projects never make it to production. Don’t let that be your project.

The Critical Nature of Data Science Pipelines and Enterprise AI

Organizations routinely invest millions of dollars into building out data science pipelines and AI algorithms. This is because these pipelines and enterprise AI will drive the bulk of future profits. They also differentiate the company from marketplace competitors. They allow the company to shift from survival mode to thrive in the marketplace.

At the same time, we are moving away from one model per business problem. The marketplace is dynamic. Many companies in industries like the financial industry are using many models to account for market conditions to solve one business problem. The need to dynamically switch models on the fly is important.

As our systems become increasingly complex, two issues arise, data and logic. The explainability of AI models is one good example of regulatory concern. Data governance has been the critical issue of the past decade as data mining in all shapes and forms strive to make it easier for different users to run analytics on enterprise data without compromising security. Adherence to regulatory rules in certain industries means consistency.

Information Technology is built on the concept of consistency. Developers and data scientists often get the bulk of accolades when it comes to innovation, but if systems are not productionized, they are the same as scrap metal. Citizen data scientists (user-driven data science) may yield good outcomes, but these models do not impact the bottom line if they can not be productionized and scaled.

Productionsizing systems that involve full-scale data science pipeline and enterprise-level artificial intelligence means consistent handling of integration, test, changes, deployment, and monitoring.

The Beauty of ModelOps

Model Ops is a new discipline that will change the day-to-day interactions between data science teams, application development teams, information technology teams, and user teams. Just like how DevOps changed development and allowed software engineers to focus on developing software, model ops will allow data scientists, software engineers, and even users to focus on their own day to day work related to a data science project.

Image by Jun Wu

Data science projects are not like software engineering projects, that can be more silo-ed. Data science projects are often multidisciplinary. They are often enterprise-wide initiatives encompassing multiple teams. This also goes for Enterprise AI initiatives. Not only are enterprise AI initiatives company-wide, they often change the business and its workflows intrinsically.

Image by Jun Wu

ModelOps is a process that ties together DataOps and DevOps to help the organization integrate its data science initiative into its organization through streamlined integration, deployment, security, and monitoring, etc..

Companies such as SAS, ModelOp, RapidMiner, and IBM put emphasis on ModelOps because it allows synchronization between DevOps, DataOps, and ModelOps. This means that overlaps between development, data, and model can be better managed by the various teams.

It also means that there’s a streamlined process, often in conjunction with a host of tools to perform the following:

1. Model Building
2. Model Testing
3. Model Inventory
4. Model Training
5. Model Integration
6. Model Deployment
7. Model Security
8. Model Change Mangement
9. Model Monitoring

Image by Jun Wu

Teamwork Opportunities

By having a streamlined way to manage your models, there are far more teamwork opportunities that can be created to fully utilize all of your resources to build much more intelligent systems.

• Your citizen data scientists (from business units) can work with data scientists to come up with real business problems and solutions. These citizen data scientists have the business savvy. At the same time, data scientists can bridge the gap between creating a model using a tool versus scaling that model into the production environment. The data scientist can work with ModelOps and application developers to integrate the model and put it into full use.
• Your product managers for specific applications can work with business units and product designers to figure out how to handle specific use cases in application features that are complicated by more intelligent logic inside models. Product managers can work with ModelOps and application developers to bridge the gap of communication between handling model inputs, data, and application features.
• Your application testing engineers can work with ModelOps, DevOps, and DataOps to make sure proper testing use cases are defined so that change can be handled seamlessly between integration and production.
• Your ITSM team will be able to monitor models 24/7 in production. As we know that data can complicate models in the production environment. Having DataOps work closely with ModelOps will give DevOps a lot more information to investigate issues.