Jenkins has seen some major changes in the last few years, one of the largest of which is a scripted pipeline, or pipeline as code: the ability to describe your continuous integration/continuous delivery (CI/CD) process programmatically.
It appears on the surface that a scripted pipeline is bringing things back to a text editor. But the reality is that the Jenkins UI is no longer powerful enough to handle the needs of a modern CI/CD process.
The power of being able to use a high-level language to specify what needs to happen during CI was a step in the right direction that has helped tame some of the complexity when rapidly building new software.
While many of the steps in a CI/CD pipeline are automated, management of the pipeline itself remains a largely manual process that slows delivery. It doesn't have to be that way.
You can use pipeline as code to give your teams the ability to define and manage an entire DevOps CI/CD pipeline. In this way you can store pipeline configurations in source control, version them, and independently test them. Here's why, plus how to get started.
What’s wrong with the old approach
Jenkins was a product of its time. In the early 2000s everything needed a web interface, so a web-based visual UI for your software automation just made sense.
A browser-based UI was an interactive improvement over building at the command line. In addition to being more user-friendly, the web-based interface provided a layer of separation from the build and the underlying machine. This allowed for increased configuration management, keeping the build environment neutral and versioned.
Exposing builds as a service had the advantage of allowing an automated tie-in with your software configuration management (SCM) provider, the ability to catch and control configuration management changes, and a dashboard and notification system that would actively tell developers about issues, rather than require developers to constantly poll the command line.
Overall, the advantages Jenkins brought to the table were a marked improvement from typing at at the command line and checking back every 15 minutes.
[ Special Coverage: Agile + DevOps West Conference ]
How to change the tires while the car is moving
But being entirely UI-driven had several downsides, the first of which was the configuration management of the build job itself. Since everything is configurable only through the UI, you need to jump through some pretty extraordinary hoops to track changes and back things up, from using plugins to putting SCM on the Jenkins home directory.
In addition, a CI job could realistically handle only one CI scenario. Having only one path to build and deliver a viable artifact can significantly slow down the development process. With philosophies such as Gitflow giving developers the theoretical tools to collaborate in parallel, the one-size-fits-all build steps implicitly enforced by Jenkins became an impediment to quickly delivering software value.
That's where the multi-branch scripted pipeline, a corollary to pipeline as code, came in. With a multi-branch pipeline, you can keep a copy of the build steps married to the code in your SCM repository. When the code requires a new component, or build step, you can invoke only the code that's ready to handle it.
This removes the complexity of a zillion "if" statements in a centrally scripted pipeline. Developers can now make build changes as needed, and affect only the code they are working on, rather than the entire CI/CD process.
In essence, each feature branch can have its own build process, and you only change the master branch when ready. This speeds code development, pipeline enhancements, and improvements by offering an isolated environment in which to test these changes.
Scripted versus declarative
Open source is about choices; it gives you enough rope to hang yourself, with a few extra feet for good measure. Jenkinsfile, the file that holds your pipeline script, comes in two flavors. The first is scripted. A scripted pipeline is raw Apache Groovy with a little bit of Jenkins flavor. The second is declarative, which is structured with slightly more syntax that can still fall back to the scripted syntax if necessary.
There are almost no limits to what a scripted pipeline can do programmatically. As long as someone has created the proper Jenkins functionality via a plugin or Jenkins binding, you can use and abuse it in a pipeline step.
There are two major disadvantages to a scripted pipeline: structure and the UI. With a scripted pipeline you aren't beholden to a particular program structure or flow; the Groovy syntax is your only limitation. This can create some pretty unreadable pipeline code and cause a maintenance headache.
Additionally, not everything you do in a scripted pipeline will render well in the Jenkins UI. With pipelines becoming part of the information radiators needed to implement DevOps, visual ingestion of the code's state in the CI/CD process is becoming more and more critical.
The declarative flavor of a Jenkins file should produce a more maintainable and structured pipeline file. Additionally, because of the tighter structure, the declarative code maps more tightly to the Jenkins UI rendering of the pipeline. You will have a clean visual representation of where the code is in the pipeline.
The disadvantages are functionality and flexibility. Where the scripted pipeline has access to everything, the declarative pipeline can access only the functionality that has clearly been mapped to the UI elements.
This means that in terms of new features, the declarative pipeline will almost always lag behind the scripted pipeline. Additionally, being structured, there are just some things that the messy world of building software will require that only the flexibility of the scripted syntax can provide.
How to get started
Write a line of code, then two, then three. The best way to learn how the scripted pipeline can help you is to download Jenkins, run it in a container, and run through several of the practical examples.
The best references for help and syntax are the Jenkins website and the dynamic code generator inside Jenkins. If you want tangible results, start by re-implementing a traditional UI-created pipeline currently under use so you have a reference for expected behavior and grow the implementation from there.
I also recommend doing some reading online for pointers and some real-world experience. But the best way to make informed decisions on how to implement this new way of doing CI/CD is to get some training.
Shift your pipeline left
Scripted pipelines are the future of CI/CD. From the YAML syntax of AWS' code build to GitLab CI and the Groovy implementation of Jenkins, this trend is being widely adopted.
While the web-based Jenkins UI was a huge advantage, it was limiting in that all changes needed to be made in the UI. The programmatic specification of your pipeline is now a necessary tool to keep up with development in parallel and the rapidly changing demands this will cause in your pipeline.
Shift your pipeline development left with testing and other beneficial software practices and you'll ensure that the your CI/CD process is never a bottleneck to quicker value delivery.
For more on implementing pipeline as code using Jenkins, come to my tutorial at AgileDev + DevOps West in Las Vegas. The conference runs from June 7-9.
Keep learning
Take a deep dive into the state of quality with TechBeacon's Guide. Plus: Download the free World Quality Report 2022-23.
Put performance engineering into practice with these top 10 performance engineering techniques that work.
Find to tools you need with TechBeacon's Buyer's Guide for Selecting Software Test Automation Tools.
Discover best practices for reducing software defects with TechBeacon's Guide.
- Take your testing career to the next level. TechBeacon's Careers Topic Center provides expert advice to prepare you for your next move.
