MONTH START OFFER: Flat 15% Off with Free Self Learning Course | Use Coupon MONTH15

- DevOps Blogs -

The Jenkins Pipeline Is So Famous, But Why?



Introduction

As we discussed in the last blog about the Jenkins download and installation(mainly on Linux CentOS machine). We started a discussion about Jenkins dashboard also, but that was too limited. Let's take that Jenkins installation one step further.

For now, let’s start creating our first job. If you click on “New Item” in your Jenkins dashboard, you will get different options of which type of job is required:

Jenkins Pipeline

Jenkins Pipeline

“Enter an item name” and select type of item (Jenkins job/project). Each project has its purpose defined & as per your requirement, click on the appropriate and press “OK”. It will navigate you to the job’s configuration page. Click on “Save”. Dummy project is ready !.

Let's together create a project for this blog. As mentioned in detail, we will discuss pipeline concepts and will create a dummy pipeline project. Now, we have taken this type because of its wide usage across many organizations. Other types are based on need but “Pipeline” is an idea finding its ways automatically in achieving CI/CD nature.

The first question will be - 

What is Jenkinsfile ?

It can be defined based on following two methods(or known as Jenkinsfile syntax):

  • Declarative  Jenkinsfile syntax and 
  • Scripted Jenkinsfile syntax

Declarative pipelines are much more popular than Scripted, because of its abundant syntactical attributes and much better design which result in effortless reading/writing of Pipeline code. General snippet of Declarative  Jenkinsfile syntax is as follows:

pipeline { 
     agent any 

     stages { 
         stage('Checkout') { 

             steps { 

                 //Checkout  

             } 

         } 

         stage('Compile') { 

             steps { 

                 //Compile Stage 
             } 

         } 

         …… 

   … 

     } 
 } 

 Corresponding Scripted Jenkinsfile syntax: 

 node { 

     stage('Checkout') { 

         //Checkout  

     } 

     stage('Compile') { 

         //Compile  

     } 

 }

Jenkinsfile is defined in the parent directory of the source code project. For this blog, we will go with the most popular one that is, Descriptive. Jenkinsfile code is defined using Groovy DSL. Various characteristics of  Jenkinsfile syntax:

  • “pipeline” -- initial position defines all directives required for execution
  • “agent” -- instructs Jenkins to allot an executor on that node 
  • “stages” -- defines all stages/phases.
  • “stage” -- consists of sub-task constituting one single block 'stages'.
  • “steps” -- series of steps to be performed for each phase.

Why Use the Jenkins Pipeline?

Jenkins Pipeline attaches a powerful stack of automation tools that will support every kind of scenario from plain continuous integration of third-party tools to complex continuous delivery of final software. In a nutshell, the following are some of the key aspects provided by pipelines:

  • Worked on the model of “PaaC” where various developers can contribute for smooth execution of each and every stage and process
  • Whenever any server crash happens, pipeline jobs will be easy to restore as the same steps will be running on code present in your version control (whether Github, Gitlab, Bitbucket, etc.)
  • A pipeline can be forced to wait for some condition to be true e.g. user input, upstream/downstream build to complete, etc.
  • Multiple jobs can be run in parallel using pipelines. Various stages of various projects can be called/managed or we can use it in the loop as well.

With the help of Jenkins pipeline, a CD can be achieved in a much faster way and frequent releases help in catching the failures/errors in the very early stage. The ongoing build will act as a base or provide feedback for subsequent future releases. Now, let’s go through Jenkinsfile examples.

Pipeline Script DEMO

1. Install the “Pipeline” plugin(if not installed). Go to Manage Jenkins → Manage Plugins

Jenkins Pipeline 

Jenkins Pipeline

2. There are 4 tabs available, Go to Available(if not installed), otherwise, it will be present in the “Installed” tab.

Jenkins Pipeline

3. Select Pipeline plugin and “Install without restart”. The plugin will be available in “Installed” tab after this

Jenkins Pipeline

4. Go to Jenkins Dashboard, select New Item → Enter name → Select Pipeline → Click Ok

You will be moved to Job’s configuration page

Jenkins Pipeline

5. In this Jenkinsfile examples, the Pipeline option, there will two types of definition available:

  1. Pipeline Script

Here you will create your pipeline script in the below given text box with all stages defined. It’s like you were creating a Jenkins file in that Script box.

6.2. Pipeline Script from SCM

Jenkins Pipeline

This means that you have to pick required Jenkinsfile from SCM(Source Code Management). Default options are “None”, “Git” or “Subversion”

Script Path → Path of your Jenkinsfile. If in the root directory(default location), simply write “Jenkinsfile”.

DevOps Training & Certification Course

  • No cost for a Demo Class
  • Industry Expert as your Trainer
  • Available as per your schedule
  • Customer Support Available

We will try both options one by one. Let’s start with “Pipeline Script’ first to have a basic idea of flow from one stage to another.

6. In the backend, Jenkinsfile is based on Groovy syntax. Write the Jenkinsfile in script box and click ‘Save’

Jenkins Pipeline

7. Click on “Build Now” and you will see different stages as blocks in the Job dashboard. See Console Output

And stage view:

Jenkins Pipeline

It will show all stages and time taken to complete each stage. When you hover an arrow on each stage, you will see the “Logs” option. Clicking that “Logs” will show you the steps executed in that stage.

unit test

Pipeline Script from SCM - DEMO

  1. As we have seen above, when selecting “Pipeline Script from SCM” for these Jenkinsfile examples, it is giving us two options. SCM and Script Path. 
  2. Now, we will create one git repository, place source code(e.g. Maven), create Jenkinsfile, and then will call from Jenkins. This will help us in understanding CI/CD flow in a much better way.
  3. Let’s reconfigure the job. Click on “Configure”. 

Jenkins Pipeline

  • SCM -- Git
  • Repository URL -- GIT repository(preferably, https)
  • Credentials -- To connect to GitLab repository. Credentials can be added by clicking on “Add” → “Jenkins” → Select “Username with Password” and provide credentials. It will show in the drop down of the “Credentials” box. Select required one and it’s done otherwise you will get an error like “Failed to Connect”
  • Branches to build -- Branch on which you want to perform the build. You can provide regex also
  • Script Path -- Jenkinsfile as it is present at Project root location.

4. In the Jenkinsfile, we are using two tools Maven and JAVA, so we have to configure these tools as well so that our Jenkinsfile can execute without any issue

  1. Go to Manage Jenkins → Global Tool Configuration → Add JDK and Maven Installation like below:

Jenkins Pipeline

I checked both as “Install Automatically”. If it is already installed in your machine, just provide the executable path instead of asking Jenkins to install automatically.

6. Click Save and configuration Done.

7. Click on Build Now. Now, from the console output, you will get the view that first it will download JDK8 and M3. Then proceed to each and every stage

Jenkins Pipeline

8. Now, here we did build, unit test and do some post actions like “cleaning up workspace”. Post block will come after “stages”

9. Now, you can achieve CI/CD by adding different stages like

  • Build
  • Unit Test
  • Integration Test
  • Sonarqube 
  • Artifact upload to Nexus/Artifactory repository
  • Deploy to dev environment

Above will be basic stages that will cater to the whole software cycle process. Similarly, post actions can be utilized in multiple ways:-

  • Cleaning up workspace
  • Drop mail for every failure, success or unstable build to set of recipients
  • Attach logs etc.

10.So, a variety of tasks and integration can be performed using Jenkinsfile.

DevOps Training & Certification Course

  • Personalized Free Consultation
  • Access to Our Learning Management System
  • Access to Our Course Curriculum
  • Be a Part of Our Free Demo Class

Jenkins Plugins to simulate Jenkinsfile pipeline examples

Some of the plugins that can enhance the view of Jenkins stage flow are:

  • Build PIpeline

  1. Install “Build Pipeline plugin” from Manage Jenkins → Manage Plugins

Jenkins Pipeline

This will connect different jobs to form a build pipeline.

  1. Go to Jenkins Dashboard, Click on new view “+”

  1. Select “Build Pipeline View”

Jenkins Pipeline

  1. Select Initial Job and let all be as Default. Click Save

Jenkins Pipeline

  1. Click on Run

Jenkins Pipeline

  1. Now, we will create one more job and add it to this pipeline. Go to previously create “Test-Pipeline” jenkins pipeline example job → configure and add 

Jenkins Pipeline

  1. When you rerun “Build” job, it will trigger “Test-pipeline” job and you can create more complex pipelines like this:

Jenkins Pipeline

  • Blue Ocean

  1. Install “Blue Ocean” Plugin.

  2. Once Installed, it will be visible in left side of dashboard

Jenkins Pipeline

  1. Click on “Open Blue Ocean”, Then you will see job list in a completely different view:

Jenkins Pipeline

  1. Click on “Test-Pipeline” and build whose status you want to see. YOu will get stage view like this:

Jenkins Pipeline

As you can see, this beautifies the pipeline dashboard and gives various options like how build flows from Start to End, how much time, each build took, total time etc. In my personal opinion, this is the best dashboard that can be shared with upper management to give a much better understanding of the whole process.

DevOps Training & Certification Course

  • Detailed Coverage
  • Best-in-class Content
  • Prepared by Industry leaders
  • Latest Technology Covered

Conclusion

We discussed a lot about pipelines in this blog: Why pipeline, What is Pipeline, How to create pipeline, different views of pipeline etc. Try creating your own first pipeline and then discuss your success stories. After this, next, we will discuss JENKINS management. 

See you soon !!


    Vipin Bansal

    As an experienced DevOps professional, I am having a good understanding of Change and Configuration Management as well. I like to meet new technical challenges and finding effective solutions to meet the needs of the project. I believe in Sharing is Learning.


Comments

Trending Courses

AWS

  • AWS & Fundamentals of Linux
  • Amazon Simple Storage Service
  • Elastic Compute Cloud
  • Databases Overview & Amazon Route 53

Upcoming Class

-1 day 04 Dec 2020

DevOps

  • Intro to DevOps
  • GIT and Maven
  • Jenkins & Ansible
  • Docker and Cloud Computing

Upcoming Class

7 days 12 Dec 2020

Data Science

  • Data Science Introduction
  • Hadoop and Spark Overview
  • Python & Intro to R Programming
  • Machine Learning

Upcoming Class

4 days 09 Dec 2020

Hadoop

  • Architecture, HDFS & MapReduce
  • Unix Shell & Apache Pig Installation
  • HIVE Installation & User-Defined Functions
  • SQOOP & Hbase Installation

Upcoming Class

6 days 11 Dec 2020

Salesforce

  • Salesforce Configuration Introduction
  • Security & Automation Process
  • Sales & Service Cloud
  • Apex Programming, SOQL & SOSL

Upcoming Class

-1 day 04 Dec 2020

QA

  • Introduction and Software Testing
  • Software Test Life Cycle
  • Automation Testing and API Testing
  • Selenium framework development using Testing

Upcoming Class

0 day 05 Dec 2020

Business Analyst

  • BA & Stakeholders Overview
  • BPMN, Requirement Elicitation
  • BA Tools & Design Documents
  • Enterprise Analysis, Agile & Scrum

Upcoming Class

-1 day 04 Dec 2020

MS SQL Server

  • Introduction & Database Query
  • Programming, Indexes & System Functions
  • SSIS Package Development Procedures
  • SSRS Report Design

Upcoming Class

-1 day 04 Dec 2020

Python

  • Features of Python
  • Python Editors and IDEs
  • Data types and Variables
  • Python File Operation

Upcoming Class

-1 day 04 Dec 2020

Artificial Intelligence

  • Components of AI
  • Categories of Machine Learning
  • Recurrent Neural Networks
  • Recurrent Neural Networks

Upcoming Class

0 day 05 Dec 2020

Machine Learning

  • Introduction to Machine Learning & Python
  • Machine Learning: Supervised Learning
  • Machine Learning: Unsupervised Learning

Upcoming Class

15 days 20 Dec 2020

Tableau

  • Introduction to Tableau Desktop
  • Data Transformation Methods
  • Configuring tableau server
  • Integration with R & Hadoop

Upcoming Class

14 days 19 Dec 2020

Search Posts

Reset

Receive Latest Materials and Offers on DevOps Course

Interviews