SaltyVagrant7.1 Less Talk, More Do!
Complexity ahead!
Although the steps to setup this ‘first server’ are trivial the material explaining how this setup works is more complex and you may need to come back to it after reading more of the book. I have tried to keep is simple (both in implementation and description) but some of you may feel overwhelmed. DON’T PANIC! If you have trouble just skip ahead and come back when you’ve learned more.
Setup to Follow Along
Although this may take a while to run I think we can agree that it’s simple enough2?
You have just created a web server with simple static website, set up a basic firewall, and run some smoke tests on the server to ensure basic functionality. Furthermore, the system you end up with will be functionally the same3 as mine. We have a repeatable process.
On your host computer, open a Web browser and access http://localhost:35555. This should display the home page of our sample website.
That process illustrates just part of the power of infrastructure as code.
Let’s take a look at this example.
Before we take that closer look…
- This is a simple example. Only one server is created the ‘system’ it implements is simple and the tests are all run locally.
- This is a very simple configuration, no attempt has been made to make this ‘production ready’.
- Tests are far from exhaustive and intended as a simple illustration.
- In the following discussion I am not presenting all of the details, just the highlights. The rest of this book will flesh out the details, how the various tools are used, and more importantly it will offer guidance on how to approach solving some of the problems we encounter in DevOps.
7.1.1 The source
The first thing to note is that the entire ‘specification’ for this system is held under version control in a Git repository. Any problems detected with the system would be corrected in this repository and then redeployed from there (more on this later). No modification would be made to this system directly (by, for example, someone logging on to the system directly)4.
This idea that our servers should be ‘untouched by human hands’ is perhaps the most foreign idea for most people. We are so used to logging on to servers to diagnose and fix problems that being told this is no longer the way to work is often greeted with ‘how can we possibly support our customers this way?’ Not only is this possible it is essential.
7.1.2 The Vagrantfile
This file tells the vagrant software how to create and initialise our server.
This Vagrantfile is the highest level of our local system specification but contains very little of the actual server configuration. This is deliberate. We want as much of the configuration to be reusable in different contexts; building a physical server, building a cloud server, as well as building a local Vagrant server.
To this end we include in the Vagrantfile only those elements peculiar to setting up a Vagrant server. Notably, the line that create a grains file for the salt configuration:
15config.vm.provision "shell", inline: "mkdir -p /etc/salt; [ -f /etc/salt/grains ] || echo 'is_vagrant: True' > /etc/salt/grains ; sed -i -e '/^is_vagrant/{s/is_vagrant:.*/is_vagrant: True/;:a;n;ba;q}' -e '$ais_vagrant: True' /etc/salt/grains"
This is broken into two parts. The first creates the grains file if it does not exist:
1mkdir -p /etc/salt; [ -f /etc/salt/grains ] || echo 'is_vagrant: True' > /etc/salt/grains ;
The second is redundant when the provisioner is first run, it is included in case the provisioners are re-run. This second line ensures that the is_vagrant grain is set correctly.
1sed -i -e '/^is_vagrant/{s/is_vagrant:.*/is_vagrant: True/;:a;n;ba;q}' -e '$ais_vagrant: True' /etc/salt/grains
This line tells Salt that this server is a Vagrant managed server. This is required so we can make adjustments to the server configuration. In this instance we ensure that the firewall allows ssh connections. We could have used other features of the server (such as the existence of the vagrant user account) but attempting to ‘fingerprint’ the server like this is prone to all manner of difficulties (what if someone creates a server with a vagrant account). Being explicit using a grain like this is better.
7.1.2.1 Salt provisioner
Given that the Vagrant Salt provisioner can be used to perform a similar function to the provisioning script described in §7.1.3, why not use it? In general I want my provisioning method to be portable so that I can use the same provisioner to build a ‘real’ server (physical or virtual) that I use to build the development and test server. This flexibility is denied us if we lock ourselves in using a Vagrant specific provisioner configuration.
Does this mean I would never use the Vagrant provided salt provisioner? Not at all. As I said, ‘it depends’. I would use it if I knew that the machine being specified was uniquely a Vagrant machine, or when I knew that everything in my specification can be provided by a salt state.highstate. I’ll describe some of the issues I see with the current setup in §7.1.3.
7.1.3 The provisioning script
The provisioning/build script is a simple beast that replicates much of the Vagrant salt provisioner.
Installing git allows us to pull the formulas for iptables and nginx. I could have installed the salt configuration and then invoked a state that installed the formulas before using state.highstate to complete the machine setup, but that seems unnecessary and does not eliminate the need for a script5.
Issue: I have not specified the versions of these formula to be used. Instead we simply pull the ‘latest’ available. For a non-critical system this is fine but as we will see this may not be appropriate to your project.
Next the script installs salt using the bootstrap-script provided by SaltStack. This is essentially what the Vagrant Salt provisioner would do too.
Then, the salt configuration is put in place and the Salt minion configuration (again, Vagrant will do these tasks too but relying on the Vagrant provisioner locks the solution into Vagrant while this script can be run on any Debian base system).
Finally we run salt to assert the highstate, this completes the setup of the machine.
Having set up the server we now deploy the website. To keep things simple this is a static web site and is controlled as source in a Git repository. The website is not deployed using salt but it could be.
Now that everything is deployed we can run some smoke tests. To do this I simply run the infratests held alongside the configuration.
Issue: I prefer to have tests hosted on an external server even if they are run locally but in a single server system such as we’ve just created that is not practicable.
7.1.4 Salt configuration
The majority of the ‘hard work’ of setting up the server is done by salt. We cover Salt configuration in more detail as we work through this book (and in much more detail in Saltstack from Scratch[Boo20d]), here I will just outline briefly the components used in this example.
Salt’s configuration is provided in two parts:
- states
- These specify the desired state of our server, they are held under /srv/salt (custom states) and /srv/formula (standardised states)6.
- pillar
- Files under /srv/pillar provide data which is subsequently used when processing states.
This configuration is made of of three parts:
- Standard tools
- Firewall
- Nginx
Standard tools installs Git and Tree. We have installed Git already in the provisioning script, but this install specifies the version of Git that we consider correct. This ensures our server has a consistent configuration.
The firewall is set up using the Linux net filter using the iptables tool.
Nginx is the tool that will server our website.
2If you have something already occupying port 35555 on your host computer you will get an error message from Vagrant that the port cannot be mapped. In this case edit Vagrantfile and change the line website_port=35555, changing 35555 to a free port
3I say ‘functionally’ because there may be some variation in specific versions of some packages installed. For example, although we have fixed the version of the Python pytest package this may in turn install dependencies and these may be limited to a range of versions rather than one specific version. Whether this lack of precision is important is context dependent.
4Strictly, of course, in this simple example we are logging on the server to run the salt and git commands, but that’s all and we could run these remotely if we wanted
5I could also have set up a Git file system—discussed in detail in [Boo20d]—but I think the current approach is simpler for new users.
6The custom/standard distinction is my own, but I think characterises the difference well.