Chapter 4
DevOps from 20,000 feet

This is the obligatory ‘what is DevOps’ chapter. The problem is DevOps has been so abused as a term that it is largely useless but here goes.

The core idea of DevOps pre-dates the term DevOps, indeed at the start of computing there was no significant distinction between user, developers, or operators as users were also the developers and the operators. As computers matured from academic and military environments into more commercial settings users split off into a clearly distinct group1. The people developing systems also started to separate from the people operating the systems, particularly in the mainframe days where these vast machines required the constant attention of operators who loaded punched cards and paper tape, and removed printouts for delivery to developers or users. The advent of Teletype terminals and subsequently video display terminals made it possible for user to interact with systems blissfully ignorant of the operators behind the scenes. Similarly developers became increasingly independent of the operators, able to write, compile, and run their creations independent of the operators keeping things running behind the scenes.

Once developers had completed their work they would hand over the finished product to operators who would then take over the loading and operation of the system on behalf of users. This generally worked well in the early days simply because both developers and operators worked within the same organisation. The increased commodification of software though increased the distance between developer and operator. Indeed operators would often take a software product and install it knowing only that it was an IBM product, never knowing or interacting with the developer. Because this gap developed between commercial suppliers of software and those who operated the systems delivering value to customers so too there tended to be an increasing gap between developers and operators even within a single organisation; why have one rule for operators when working with an external supplier and another when working with internal developers?

This divorce between developers and operators made a certain sense when delivering packages software, that is software that was a simple stand-alone system. Developers had little concern about working well with others. The operating system ensured the various programs running on the computer were separated from one another so operators had few concerns. Products tended to go through long life-cycles and this allowed for extensive testing (and testing was somewhat simpler as the software operated largely on its own).

This situation did not last long though. Increased complexity meant increased specialisation and consequently increased interaction between components of a system. No longer would a development team write a system largely from scratch but they would take various commercial products and build them into a system. Each building block would be treated as a black-box and development teams where hostage to the delivery cycle of the vendor to fix problems (resulting in code being developed by these teams to ‘work around’ perceived deficiencies in the commercial packages).

Long story short, the increased complexity of systems also increased the complexity of delivering systems from development teams into operation. No longer a simple ‘here it is’ the delivery became a complex set of operations the install various off-the-shelf systems, configure them, then install the custom components, test they worked, etc. All this complexity resulted in either longer delivery times (and wicked documents describing—hopefully—how the installation should be performed), or more commonly development projects would build out the first operational system and deliver the entire system to the operations team (saving the complexity of delivering build instructions to operators). This second approach often became the norm.

Maintaining development teams after the initial system was delivered starts to become expensive as more and more systems are delivered. Consequently it is common for operations teams to take over maintenance of systems. Problems arise when knowledge held by the developers evaporates with the development team, seldom being effectively communicated to the operations team. This leads to poor understanding of the system and consequently a struggle to diagnose and correct problems.

This problem is exacerbated by the advent of systems supporting web based products and services. The rapid development and delivery of these systems means constant rapid change.

Enter DevOps, a call to return to a culture in which developer and operators work closely together in both developing and maintaining systems. DevOps philosophy complements the Agile movement which encourages close work between users and developers.

DevOps is about creating as frictionless a cycle as possible for the development, validation, delivery, and monitoring of systems.

1Yes, I am massively over simplifying things here, but I think the point stands.