Software development has evolved a lot in the last few years and it will continue to do so with inculcation of cloud based technologies rising to an all time high. While this is happening, clients often ask what would it take to spin up new environments, how much time and money will this cost? Well, this is where DevOps has come into picture. Clients are looking for their infrastructure as source code which they can take and spin up new environments quickly and with ease. In Paxcel, we have setup a dedicated team which is looking after this domain and are using various tools in domains like Continuous Integration, Continuous Delivery, Log Management and Repository Management.
I will bring up a case scenario wherein I will first define the problem and then the solution we designed for our client Chaikin Analytics using various tools like Vagrant, Puppet, Docker, Sonatype Nexus, Jenkins, Maven, GIT and Ruby Migrations.
Problem: Client wanted to transform their whole infrastructure into code which can then be used to quickly spin up new environments with latest builds. In nutshell they wanted to automate their build deployment process
Solution – (a): To cater to these requirements we opted for Jenkins and Sonatype Nexus.
Developers write code and commit to GitHub. Post-commit hook at the Github repo then triggers Jenkins to create build and push to Sonatype Nexus with help of a maven plugin. From here on, actual VM, where this build has to be deployed, uses a yum plugin, which makes the process much easier. We have configured this plugin in Nexus machine and web servers fetch the build by running command like ‘yum ensure latest’ through Puppet module. This works flawlessly, developers commit code and build is deployed in no time…cool !
Nexus not only helped us in achieving the automation, but also helped in hosting all third party jars in it so as to give more control to admin on which version of jars developers should be using. Along with this, placeholders feature for previous deployments was a big plus in case builds required a roll back.
Solution – (b): With first target of build automation achieved, next was to port whole infrastructure as source code. For this we used tools like Vagrant, Puppet, Docker, Ruby Migrations, ElasticSearch/Logstash.
Lets dig in more detail about how we are using them.
We first opted for creating linux containers due to advantages they offer over a Virtual Machine. For creating those containers we opted for Vagrant, which helps us define the configurations of docker containers in single file. Once this is done, user can take this file and run command ‘vagrant up’ which creates containers in no time. Next was configuring those containers with necessary RPMs and other settings, for this we opted for Puppet. We wrote Puppet module for each configuration which needs to be applied to a container. Vagrant has build in support for Puppet, which works flawlessly. ‘vagrant up’ command now apart from creating containers, also puppet provisioned them with necessary RPMs.
Till this point, we solved most of the problem areas. Now user needs to take a clone of repo and fire command ‘vagrant up’ which will create necessary containers and provision them by Puppet where one of these modules would communicate with Sonatype Nexus for ensuring latest version of rpm is installed on machine.
One last thing we were left with were database changes. For eg. if we had changes in code, which required schema change as well, then how to automate that. For this we opted for Ruby Migrations, so developer will write migration scripts for whatever change they want to make in database and push it to the repo alongwith the code. Puppet module will take care of applying those migrations into your database, so it works same way as it worked for builds.
With this done, all problem areas were addressed. All these wonderful tools have really helped us to represent true case of infrastructure as code and this workflow will only improve further with lot more promised on roadmaps of these tools.