IDG Contributor Network: Why are containers so short-lived?

A recent study by my employer, Sysdig, on production container deployments shows that 74 percent of containers live less than an hour and 85 percent live less than a day, which, on the face of it, makes it seem like containers are too ephemeral to do any heavy lifting.

That isn’t the case, but it is interesting to dig deeper into the short, happy lives of containers. There are a few reasons containers come and go so frequently, and some evidence that, over time, the lifespan of containers may lengthen as the technology matures.

That is not to say containers will ever stick around as long as virtual machines, which typically live for a month. Containers are a different breed, and their unique attributes give rise to unique use cases and, frankly, unique challenges, especially on the management and security fronts.

Test and kill

One reason containers are so short-lived is the technology was first used in dev/test environments. Instead of taking three days to set up a test environment for your application, you can use containers to set it up in a flash, run the test, and then dump the container.

This is still a prominent use case today, especially in large companies with continuous integration/continuous delivery (CI/CD) environments. One company I know spins up a container for every job it creates in its Jenkins application build system, tests the change, and then shuts down the container. That accounts for thousands of containers coming and going each day.

And let’s face it: The last thing companies are going to containerize are things like databases. Containers are perfect for batch jobs, say payroll processing, or tasks that run on a scheduled basis. Every day you need to do some analysis and send a report. So you spin up a container, run it, get the report, and then kill the container. The quick jobs, maybe a maintenance check or a backup, are the kinds of things people containerize first, and most result in short-lived containers.

I have seen, however, increased usage of database solutions such as PostgreSQL and MongoDB, running in containers. As the container ecosystem matures, as with the evolution of stateful sets and persistent storage, it is likely that the lifetime of containers will increase in the future.

Containers behind the curtain

That fact, however, may be obscured by the other major reason containers die young: Orchestration tools such as Kubernetes abstract the container infrastructure.

Orchestrators make it possible to look at things at the service level, meaning the underlying infrastructure ceases to matter. You specify how much CPU and memory a job or task needs, and the orchestrator assumes the responsibility of providing the container resources to meet the job requirements. If there is a hiccup along the way, a new container may be spun up and the problematic one killed off.

With the orchestrator acting as a control plane, containers can be added or deleted on the fly as demand dictates, contributing to container churn. In microservices architecture environments, where autoscaling is the inherent nature of the application, application components may be spread across infrastructure, and the result is even more exaggerated.

The progression of this trend is the march to serverless environments (such as AWS Lambda), where you don’t even have access to the infrastructure. Your code runs as a function and underneath the hood containers are coming and going to support those functions.

Coping with the new realities

While orchestration tools facilitate use of containers, they don’t remove the need for IT to be able to see under the covers. For example, Sysdig Monitor can ascertain the performance of a service, which is key, but it can also reveal the health of containers, revealing, for example, some that are churning too much and why.

As container environments mature it is likely some containers live longer than their young cousins today, but the unique attributes of containers and the unique way they are being used means containers will never be used like the heavier, harder-to-manage VMs prevalent today. That makes it imperative to get out in front of the new management and security challenges that come with burgeoning container environments.

This article is published as part of the IDG Contributor Network.Want to Join?