CircleCI Blog

What follows is a letter to potential CircleCI employees, from our Head of Human Resources, David Mann.

Dear CircleCI Candidates,

I wanted to share our philosophy and standards around the hiring process to give you more clarity about what you should expect from us, and what we’ll look for from you.

Overall, know this: we respect you. And we’re honored you’re interested in joining our team.

To make the interview flow go smoothly, we have some standards we live by. We’re not perfect, but we do our best to make the getting-to-know-each-other process as enjoyable as possible. And as you consider joining our team, I want to share with you the thinking behind our methodology.

The importance of maintaining version dependencies for your microservice architecture

Scaling a microservice architecture has a number of challenges. You’ll hear people talk about having “thousands of services” where they get to “have the right tools for the job” all the way down to their languages. This is all well and good, but thousands of things are not always great.

For instance, things I would rather not have thousands of:

• Log formats
• Spider-man reboots
• Database access patterns
• Metrics dashboards I need to maintain
• Stats publishing formats
• Recipes including the words “Jello” and “Salad” (no, this was not okay, 1950s. You gave us the birth of NASA, but this almost cancels it out)
• Measurement standards
• Health check endpoints
• RabbitMQ access patterns

And at the top of the list: Java Maven Dependencies

In the previous installment, we saw how Docker images add power and customization to the build process. In this installment we’ll show you how to amplify that power by using the CircleCI 2.0 Workflows feature.

Workflows in detail

Simply stated, Workflows adds a simple coordination layer between jobs. Let’s start by visualizing a simple workflow:

Workflows DAG

This week, leading research firm Forrester released their first-ever Forrester Wave for Continuous Integration. Our team at CircleCI spent a considerable amount of time working with the research team at Forrester, and we’re proud to be named a leader in the space. You can go read the report here, but there’s a larger point we think is worth exploring: why is there so much attention on continuous integration right now? Why are Forrester’s clients asking them to provide research and analysis on which CI tools and companies are worth paying attention to?

This post is a follow-on of our overview of persisting data in Workflows. To learn all about how to best use workspaces, caching and artifacts, read our introductory post here.

Workspaces are a feature of Workflows and are used to move data from a job in a Workflow to subsequent jobs.

The name “workspace” may conjure up an image of a single storage location which jobs can add to and remove from at will. However, Workflows can introduce a lot of concurrency into a build process, and there is no defined order that concurrent jobs run in. In these circumstances a single mutable storage location leads to jobs flip-flopping from pass to fail in different workflow runs since they never get a consistent view of the workspace contents. This significantly impacts your ability to get a repeatable build process if your workflow has concurrent jobs.

CircleCI 2.0 provides a number of different ways to move data into and out of jobs, persist data, and with the introduction of Workspaces, move data between jobs. Using the right feature for the right task will help speed up your builds, improve repeatability, and improve efficiency.

The benefit of faster CI runs will be clear to anybody who’s waited for the CI to go green.

Repeatability is also important. A repeatable CI process means that if you run the same process again against the same SHA from your repo, you will get the same result. When a CI process isn’t repeatable you’ll find yourself wasting time re-running jobs to get them to go green.

The Perks of Going Distributed

Since our early days, CircleCI has purposely embraced building a global workforce. This simple choice has given us many advantages. First, we have access to the best talent on Planet Earth. The great variety of viewpoints we have access to helps us make better decisions. Equally important: it allows us to continuously deliver value to our customers 24 hours a day. Having team members all over the world brings many advantages for us as a company.

What’s even more valuable, though, is knowing that the flexibility to work remotely confers some pretty great benefits on our global employees as well.

As a Success Engineer, I often see a lot of confusion around using Docker on CircleCI. Before the release of CircleCI 2.0, it was possible to build and test applications without ever having to think about Docker: all our builds ran in LXC containers, configured by our inference engine to include any necessary tools and dependencies in an oftentimes fairly automatic way.

With the release of CircleCI 2.0, however, we’ve placed Docker front-and-center in our build process: any Docker image, public or private (we now support private images hosted on Amazon’s EC2 Container Registry, too!), can be the primary container for a build. This adds power and customization to the build process, especially when combined with Workflows, which allows for sets of jobs (build, test, deploy, etc.) to be strung together, sometimes running in parallel, each with its own base Docker image. To aid the transition from CircleCI 1.0 to 2.0, we’ve even provided a wide range of convenience images to meet common language and dependency needs, tweaked to maximize compatibility with the CircleCI platform.

However, as they say, with great power comes great responsibility. Using Docker can be intimidating. For some developers, it may seem to have a high learning curve. Others worry about maintenance or upkeep for any images they’ve created. And then there are privacy concerns—after all, many of the images on the main Docker registry, Docker Hub, are completely public.

Workflows has led many people to get “job happy” in their config files. One job to ‘build’, another to ‘test’, and another to ‘deploy’? Maybe, that’s not too bad. How about a fan-out, fan-in workflow that comes in at a total of 5 jobs where each uses the same Docker image? There’s going to be redundancy in that config.yml file. We can reduce it, and make YAML do the work for us.

Our Outrageous Decision to Orchestrate Nomad with Kubernetes

“Write less code.”

Rob Zuber, CTO of CircleCI, stepped back to observe this directive, written in faded scarlet on a whiteboard. Like all important directives, this one was pithy and counterintuitive; how could we possibly hope to build CircleCI 2.0 by doing less?

If you’ve read this blog post about our release philosophy, you’ll remember that replatforming was a pretty frightening prospect for us. One of the reasons it was so frightening was that we were starting from scratch; there’s a real danger in overdesigning a product when you do that. But there’s also an opportunity to take stock of all the new options — the great work that’s been done to advance the toolset.