RAPIDS 2018 Talk - Machine Learning in Production - A practical approach to continuous deployment of Machine Learning pipelines
Full RAPIDS 2018 schedule and tickets available here
Machine Learning in Production - A practical approach to continuous deployment of Machine Learning pipelines - Luca Palmieri, Machine Learning & Data Scientist and Christos Dimitroulas, Fullstack developer & DevOps - Headstart.io
We take reproducibility very seriously.
Our system needs to be fully auditable: our “match score” is a crucial element for candidate selection. At any point in time we need to be able to:
- Access the models that were being used in production when the match score was computed;
- Examine their code (including all upstream ETL/preprocessing pipelines);
- Examine the data they were trained on;
- Be able to deserialize the models and run diagnostics/tests on them.
To support our requirements, we developed our own internal model versioning system using Git, Docker, CircleCI, AWS S3 and Pipenv.
This presentation will share the design, implementation and functionalities of our versioning system, with a detailed walkthrough using our skill recommendation engine as a streamlined running example.
Luca Palmieri is a Machine Learning and Data Scientist at Headstart, a tech startup providing diversity recruiting software. Luca develops data-driven strategies to identify the best matches between companies and candidates in the shortest amount of time.
Luca’s background is in Theoretical and Applied Mathematics. He holds a M.Sc in Mathematics jointly awarded by the International School for Advanced Studies (SISSA) and the University of Trieste and a B.Sc. in Mathematics from the University of Rome “La Sapienza”.
His Master’s thesis explored the use of deep learning techniques to speed up the state-of-the-art computational models currently available to perform cell simulations in healthy and cancer tissues.
Christos Dimitroulas is a Full-Stack Developer & DevOps Engineer at Headstart. He manages the company’s cloud infrastructure, continuous integration & deployment pipelines and works on client applications (both backend and frontend).
Christos graduated with a Master’s in Mechanical Engineering from the University of Bristol and worked for two years in the nuclear power industry before making a wild career change into software development.
Get your free ticket
Code of Conduct
Please note that by attending the conference you agree to the following code of conduct .
- Sign up for Dothub for free.
- Try it via Katacoda, or the hello dotmesh tutorial.
- Check it out on Github.
- Give us feedback on Slack or get in touch via email.
- Learn more about what a datadot is.
- Browse the tutorials here.