A few months ago Azure announced a new service - Azure Container Instances. With a relatively simple API to work with I wanted to see how difficult it would be to spin up containers in order to run once off tasks.
TLDR;
Azure Container Instances are awesome. But read the docs carefully. The sample code is on Github
The plan
I have a worker process that currently runs on an Azure VM. It runs between 20 - 30 times a day in response to a user action. It typically runs for around 30-60 seconds at a time. This means that we're using less than 1 hour of compute time a day and paying for 24. That seems wasteful. I wanted to experiment with spinning up containers when a queue message needs to be processed and then destroying the container when done.
Ordinarily I would use Azure Functions for this but there are occasions where this worker process needs to run for longer than 10 minutes and thus Azure Functions is not suitable.
What I did
Create an Azure Function that
- Receives a message from somewhere (in this case it was http, but it could be a queue or blob storage or anywhere really)
- Creates a container instance and passes the data from the message to the container via an ENV var
The container code does the following:
- Does the actual work
- Posts the results to another queue
Create a second Azure Function that
- Receives the result
- Destroys the container instance that did the work
- (Optionally) Posts the result to queue for another application to receive
It worked, but with some caveats. The most annoying is that the containers will automatically restart when they exit. Which means the worker has to block until the Azure Function destroys the container.
Overcomplicating life (for the 4 millionth time)
I then tried a few ways of resolving this (all the while wishing for a restart policy like in docker compose). I tried shutting down the container instance from within it. Which worked but didn't exactly feel very clean. I almost started writing a simple container orchestrator to track which containers were busy and which were not. And then...
Reading the docs
It turns out that ACI does support specifying a restart policy, but the dotnet library for Azure Resource Management does not expose this. I created a bug report for this but didn't want to wait for that to be fixed before trying this out.
Mirror mirror on the wall...
Enter System.Reflection. I was using the Fluent library which is a wrapper around the non-fluent library. It turns out the regular library supports setting a restart policy so setting the appropriate property via reflection was pretty simple. I also needed to set the rest api-version to a newer version via reflection. I'm pretty sure this will break some other part of the Fluent library so it probably isn't a great idea to do in production.
Let's do it
After all of this I ended up with something that I'm pretty happy with. I can spin up 20 containers in around 10 seconds and they'll be ready to start work within around 20-30 seconds in my tests with a simple .NET core console app running in the container.
In Conclusion
ACI is pretty cool, but requires a little bit of research and effort. It definitely turns out more cost effective than a dedicated VM for infrequent tasks.
Cost
| Virtual Machine | Container Instances |
|---|---|
| A1 v2 - $0.041/hour * 744 | 800 create * $0.0025 = $2 |
| 60s * 800 create * 2GB * $0.0000125 (mem) = $1.20 | |
| 60s * 800 create * 1 core * $0.0000125 (cpu) = $0.60 | |
| $30.51 | $3.80 |
I've taken a fairly extreme estimate of the usage for the month that I would see. Even with this rather extreme estimate I see cost savings of around 85% and a big increase in potential parallelism. Pretty impressive.
Code
If you missed the link above all the code I wrote for this is on Github.
Tell me how to do it better
If you want to tell how to do this better (or you just like following people on twitter) then come find me on twitter.