As promised in our previous blog post, we’re continuing our discussion about the components of Zenoss as a Service (ZaaS) that help us deliver an incredibly robust and reliable monitoring service. First up is AWS CloudFormation (CF).
For those who may be unfamiliar with the AWS service arsenal, CF lets you create and manage a collection of related AWS resources, provisioning and updating them in an orderly and predictable fashion. You can deploy and update a CF template and its associated collection of resources (called a stack) via the AWS Management Console. This template-ized approach makes update and deploy services a heck of a lot easier and quicker.
When we set out to build ZaaS, we didn’t start off with a plan to leverage CF. We were actually busy writing a workflow controller in Celery. Celery worked fine, but it presented our team with one more ‘thing’ to become experts at. It also required its own infrastructure for us to manage. Those were investments we felt would take away from the core mission in delivering ZaaS functionality. So in April of 2013, the team and I decided to dump weeks of work and adopt CF in order to manage how we deploy, update, and recover from problems.
There were three key reasons that made the leap to CF:
-
IaaS. In delivering “monitoring as a service” with ZaaS, it makes sense to operate on a model that allows us to define the product as version controlled code. The IaaS agile development methodology lets us create, destroy, and change infrastructure on the fly, which improves the speed and quality of our design, deploy, test efforts. This has enabled us to, roll out 8-15 new features a week instead of the typical one or two. Our CF templates are hosted in a private Github repository so we can collaboratively work on them no differently than any other piece of source code. We are currently working on a public repository to share our standard infrastructure CF templates and Chef recipes with others when it makes sense.
-
Encapsulation. The ability to manage groups of resources as an individual entity is incredibly useful. This provides two benefits. First, it allows us to update internal pieces of the infrastructure without affecting others. That means we can compartmentalize changes on a customer by customer basis. Second, it gives us the flexibility to try new things in development stacks without adversely affecting anything actively running in production, but then easily port CF templates into production when ready. This dramatically reduces the time to market for fixes and new functionality.
-
Customization. While CF is an extremely important component of AWS and is frequently updated to include new functionality, you’re always going to run into a situation where you can’t do what you want with the standard offering. That is why it was so important that AWS included the ability for CF to call out, during execution, to third party services (which they refer to as “Custom Resources”). Any time we run into requirements not met by CF, the ability to develop Custom Resources as a work around is critical to meet customer demands. In addition, WaitConditions allow us to interact with CF from inside of the instance boot and software configuration process to trigger failures at a cluster level in response to OS level issues.
More than anything, CF has helped us improve efficiency and customization for ZaaS customers. ZaaS is ensuring the availability and performance of your applications and services, and CF helps us give you the quickest, most effective updates to your specific monitoring environment. For more information on CF, you can visit the AWS site. If you want to talk further about how the ZaaS team works with CF, feel free to contact us directly.
