Posts

In the world of microservices, having secure, efficient, and compact Docker images is crucial for successful deployments. This blog will explore the key factors that contribute to building such images, including the importance of not running images as the root user, updating and upgrading packages when building the image, considering Docker’s layer architecture while writing Dockerfile instructions, and utilizing multistage builds to decrease your image size. Parts of this blog were inspired by a great talk that I recently from Matthijs Brouns, which you can view here....

Dbt is a great tool for data transformations. The paid version, called dbt Cloud, offers Slim CI. Slim CI makes it easy to test new and modified dbt models before deploying them into production. You can configure jobs to run when opening a Pull Request in your repository. By surfacing the job’s most recent successful run artifacts, dbt Cloud determines the set of new and modified resources. You can choose to only run the dbt models for which any code has changed....

Using a pull request template, labeler and a semantics file You have worked hard on a new feature or on a bug, and it is time to open a pull request to notify your team members that the feature or fix that you worked on, is ready. It’s the reviewers job to review your code and thoroughly discuss the implementation of a feature before approving the merge to main. But what about a closer look at the pull request itself?...

Within the software engineering industry most developers will be familiar with unit testing. A unit test aims to check whether a part of your code operates in the intended way. Writing them has the following benefits: Reduces bugs when developing new features or when changing the existing functionality Prevents unexpected output Helps detecting edge cases Tests can serve as documentation All the benefits of unit testing for software engineering projects apply to data science projects as well...