As a Data Scientist who works on machine learning at scale, I’m constantly on the lookout for best practices and resources to help me become a better machine learning practitioner. Although I often come across very helpful information in the form of blog posts and tutorials, this content is sparse compared to the number of posts about training neural networks or building your first sentiment analysis classifier. As a practitioner, I can tell you that there is a huge disconnect between the content that’s out there, and what people working on machine learning actually do.
That’s why I started the ML in Production newsletter.
Each week I send out an email containing tutorials, blog posts, open source software, and more, dedicated to running machine learning systems in production. Below I’ve posted a copy of the first issue of the newsletter. If you think this would be valuable for you, I’d love for you to subscribe.
ML in Production – Issue 01
Building and deploying machine learning systems is hard. While training performant models is really important, that’s just the tip of the iceberg when it comes to putting your models in production and making sure they have the positive impact you care about. Aside from model training, you also need to think about monitoring, model staleness (i.e. performance degradation over time), versioning, experiment tracking and more. My goal is to dig into each aspect of production machine learning systems.
Here’s what I’ve been reading/watching/listening to recently:
- Clipper is a general purpose, low latency prediction serving system developed at UC Berkeley. Its goal is to render predictions that meet latency requirements of user facing applications while allowing data scientists to build models in the framework of their choice. Definitely check it out if you’re looking for an open source framework to serve your models to users.
- Scaling Deep Learning on Kubernetes – In this episode of the TWiML & AI podcast, host Sam Charrington spoke with Christopher Berner, Head of Infrastructure at OpenAI. They go into depth about how OpenAI is leveraging Kubernetes to scale their deep learning research, including their hybrid setup which includes cloud and bare metal clusters. They also dig into how they set up Kubernetes to ensure that researchers have the resources they need for experiments. If you’re thinking about leveraging Kubernetes for your machine learning workloads, give the show a listen. FYI, I’m a huge fan of the TWiML podcast.
- What’s your ML Test Score – If you’re running an ML system in production, do you wonder if you need more monitoring? Or maybe you’re just about to launch and wondering what you can do to sure up the codebase? This paper, based on best practices from running ML at Google, provides a set of tests to quantitatively determine how ready for production your system is. What’s your score? ; )
- My Best Tips for Agile Data Science Research – A cool piece on data science process. Rather than just go off and work on a model for months, the author mentions setting project goals, starting with a simple model, and moving to production as soon as possible. It’s interesting to watch as best practices from traditional software development are ported over to the data science world.
- Meet Michelangelo: Uber’s Machine Learning Platform – If you haven’t yet come across Uber’s ML-as-a-service platform, Michelangelo, you’re overdo. In its introductory blog post on the tool, Uber’s team describes the motivation and architecture of the end-to-end system and how it powers the models behind features like UberEATS estimated time of delivery. A really comprehensive post that’s worth a read. And while you’re at it, check out their follow-up piece, Scaling Machine Learning at Uber with Michelangelo, as well.
That does it for this week’s issue. If you’d like to chat about these links or anything else data science or machine learning related, you can find me on Twitter at @MLinProduction.
***
If you enjoyed reading this first issue, sign up for my newsletter below!