Cloud Culture Shock: Insights From Developing and Managing ZaaS

MultiCloud Computing

When we set out to create Zenoss as a Service (ZaaS), we wanted to deliver an incredibly robust service that exactly mirrored the resource monitoring and service impact capabilities that we offer in our on-premises Zenoss Service Dynamics solution. One of the central requirements was that our customers have the same unified view of operations across their physical, virtual, or cloud environments, regardless of whether or not it was in a cloud “package.” But as we started down the cloud path, the unfamiliarity of the new ecosystem caused many of us on the team to experience significant culture shock -- much like what can happen when you travel to a foreign country. We didn’t necessarily know the dialect, the customs seemed a bit strange, and we couldn’t take advantage of “shortcuts” like the locals.

Our development journey

That there will be some level of culture shock with developing cloud-based services isn’t terribly surprising, but experiencing it first hand was certainly eye opening. Because we wanted to use the proven Zenoss Enterprise codebase, we had to really think a lot more about how the cloud deployment design would differ from traditional on-premise designs. To get the full value of the AWS platform – beyond what is available in Amazon Elastic Cloud Compute (Amazon EC2) – we really had to stretch the limits of what we would normally recommend customers do themselves on-premise.

According to some quick (but highly scientific) online research, there are four stages of culture shock– honeymoon, disillusionment, adjustment, and acclimation– and we definitely made our stop at each of them along our development journey. I’ve tried to capture a bit of what this looked like below:


All of these new experiences are awesome!

When we started digging in and learned to trust the capabilities that AWS provided, we were really excited about what we could do.

  • Amazon RDS let us take advantage of the database availability and resiliency benefits of multi-AZ deployment for close to zero administrative effort.
  • Amazon RDS also handled multi-AZ MySQL replication for us, which was a huge time saver.
  • The AWS SDK for Python (Boto) took a lot of complexity out of the coding process.



This is really messed up!

As we continued to work with AWS, we started hitting some snags that introduced understandable frustration.

  • Amazon Linux didn’t support all the packages that Zenoss wanted.
  • Amazon RDS didn’t give us the “real” root access we were accustomed to having at our fingertips.
  • We didn’t get a real global routed network in Amazon VPC. We had to build that ourselves.
  • We had to teach some developers that there is more to networking than Class C/24 networks.

I guess I understand why they do things differently.

Once we had worked with the platform for a while, we started to understand how things connected and see what workarounds we could come up with to get to our final goal.

  • We were now really comfortable with Amazon Linux. We just had to work with the Zenoss packaging team to identify more flexibility in dependent packages and alternate packages.
  • Amazon RDS, similarly, became much easier for us to work with – we found a set of permissions in RDS that works great. We also recently moved to MySQL 5.6!

OK, I’m one of you guys now

We now consider ourselves as fully integrated members of the AWS development community.  The platform is allowing us to deliver a truly dynamic, robust, and fully enterprise-ready solution for customers.

  • We have fast Cloudformation-based deployments (we’ll be covering this in more detail in an upcoming blog series)
  • We have an incredibly flexible model-based deployments and configurations using Chef.
  • We look for ways to push functionality into AWS so that we don’t have to maintain those skills or the operational burden in house.

The process was difficult at times, but we came out of it with an incredibly flexible, scalable solution that delivers advanced service assurance capabilities for our customers - ZaaS.

Our ZaaS service provides

  • Unified view across physical, cloud and hybrid environments.
  • Automatic discovery, modeling, and tracking of resources and configurations.
  • Real-time service impact analysis and root cause analysis work to speed problem resolution.

But we weren’t done quite yet – we found there was additional learning to do when it came to operations.

Our operations journey

We experienced another bout of culture shock when it came to managing the ZaaS service. When there was a hiccup in availability or performance, we found it was difficult to distinguish whether a problem was occurring in the cloud infrastructure or in our service running on the cloud. All of the likely suspects for availability or performance issues – the server, the VM instance, the router, the firewalls, the switches– weren’t under our direct control. We just didn’t have the visibility we were used to with our on-premise solution.

Luckily for us, we had the entire Zenoss development team on our side. Based on our experiences, we went to the solutions team and asked for changes to our ZenPack for AWS that would give us the additional visibility into the back end operations we needed. The original AWS Zenpack we had been using was oriented to monitor the ‘instances’ that are inside Amazon Classic EC2. Since it was instance oriented, the Zenpack didn’t look at the status of the AWS services from the cloud provider’s point of view. Adding to that issue was the fact that Amazon VPC didn’t exist when the original Zenpack was written, creating some blind spots around networking. The team re-wrote the Zenpack to add these new critical capabilities, which gave us the ability to compare the cloud provider’s point of view with the “instances’” point of view and reconcile any differences. We were back in business, able to deliver the robust, reliable service we had set out to create.

While you probably don’t have an entire ZenPack team at your disposal to help you through similar operations challenges, not to worry. All of our hard work is available as a free download! The AWS ZenPack (including all the new functionality we requested) can be used with either Zenoss Core (our free open source offering) or Zenoss Service Dynamics to bring additional visibility to your own cloud environments.

We hope that sharing our cloud journey can help lessen the culture shock you experience as you undertake your own cloud development and operations efforts. If you are interested in understanding more about the work we did in developing the ZaaS offering, stay tuned! Over the next few weeks we’re going to be rolling out additional articles on our methodology for building ZaaS on AWS, and some of the individual technologies we leverage to make it as robust and reliable as its on-premise equivalent. In the meantime, you can join our Cloud Forum to get answers to any questions on ZaaS and the AWS ZenPack.



Enter your email address in the box below to subscribe to our blog.

Zenoss Cloud Product Overview: Intelligent Application & Service Monitoring
Analyst Report
451 Research: New Monitoring Needs Are Compounding Challenges Related to Tool Sprawl

Enabling IT to Move at the Speed of Business

Zenoss is built for modern IT infrastructures. Let's discuss how we can work together.

Schedule a Demo

Want to see us in action? Schedule a demo today.