This article is part of a Cookiecutter series - we’ll be posting “How-tos” for Cookiecutter and common pitfalls that we’ve encountered.
Over the last few years, the tech industry has seen the shift away from huge monolithic applications to more agile, scalable microservices that better represent the teams that own them. If you’ve made the transition, you’ve probably spent more time than you’d like just setting up repositories, copy pasting boilerplate, and building the same plumbing over and over. For each service that you create, you need to spend hours ensuring that you’re following standards and agreements and have all the usual configurations. This can sometimes include a long list of items:
Logging & Metrics
Upstream Dependency Configurations
Documentation
CI / CD Pipelines
Testing & Code Quality Infrastructure
… and much more!
Doing all this manually can be time-consuming, cumbersome, and worse, can lead to a proliferation of varying service standards & service quality. Let’s be honest, no engineer wants to copy-paste files from one repo to another. We’d rather be coding exciting new features. However, getting the structure of your new project correct from the beginning is critical, as it can save you loads of time and operational overhead in the long run. (We recently wrote about the benefits of service standardization, if you’re interested in reading more about why it’s so important!).
The solution is to remove humans from this manual process, and automate the creation of new services and projects as much as possible, ideally using a flexible templating engine. One engine that is simple and intuitive to use is Cookiecutter, which is widely adopted as an industry standard. It’s a CLI tool that allows you to easily bootstrap a new project from a standard template. It uses Jinja2 to go through and replace all the variables that are surrounded by templating tags, giving users the power to build templates that handle every use case developers can throw at it. It’s easy to install and get started, just pick from the variety of available templates:
… and many more!
or create your own templates!
With these Cookiecutter templates, your team can stop reinventing the wheel and focus on developing new features for your users. Moreover, the templates enable your team to:
Enforce standards
Ensure consistent structure in repositories
Lower the entry-level to new collaborators
Accelerate developer velocity
There’s a handful of case studies of organizations using Cookiecutter and the benefits that they’ve seen. Our personal favorite is Her Majesty’s cookiecutter template for data science projects. The Crown recognizes the importance of setting up standardized templates that guarantee the security & quality of every new project.
Using CookieCutter, the government has defined the following standards for any new data science projects:
Project Setup: The base repository includes the entire production-ready project structure (docs, README, Jupyter notebooks, Makefiles etc.) to get you going on your data science project.
Security of the data & secrets: Using pre-commit hooks, the template ensures that secrets like credentials, API Tokens are not accidentally checked into your VCS. Since this is a Python + Jupyter project, another hook ensures that your notebooks are sanitized before checking them in.
Testing & Code Coverage: Using a combination of pytest & coverage.py, the project repo is set up in a way that includes the necessary packages & files to run your first test.
To learn more about how Cortex is revolutionizing Service Creation using CookieCutter, head over to Cortex.io.
