Jenkins: Declarative Pipelines, Distributed Agents, and Shared Libraries

Jenkins is an open-source automation server supporting distributed build orchestration. This guide covers modern Declarative Pipeline (Jenkinsfile) syntax, controller-agent architecture, containerized dynamic build agents on Kubernetes, credential binding, and shared pipeline libraries.

⚡ Quick Dive

Declarative Pipeline Syntax Cheat Sheet

Directive Purpose Example
pipeline { ... } Root wrapper for declarative pipeline Top-level block
agent Defines where pipeline executes agent { docker { image 'golang:1.22' } }
stages Container for sequence of stages stages { stage('Build') { ... } }
environment Defines environment variables or credentials environment { DB_PASS = credentials('db-secret') }
when Conditional execution of a stage when { branch 'main' }
parallel Runs multiple stages concurrently parallel { stage('Test 1') {...} stage('Test 2') {...} }
post Actions executed after stages/pipeline post { failure { mail to: 'ops@company.com' } }
options Pipeline-wide configuration options { timeout(time: 1, unit: 'HOURS') }

Production Jenkinsfile (Declarative)

pipeline {
    agent {
        kubernetes {
            yaml '''
apiVersion: v1
kind: Pod
metadata:
  labels:
    jenkins-agent: build-pod
spec:
  containers:
  - name: golang
    image: golang:1.22-alpine
    command: ['sleep']
    args: ['99d']
  - name: docker
    image: docker:26-dind
    securityContext:
      privileged: true
'''
        }
    }

    options {
        timeout(time: 30, unit: 'MINUTES')
        disableConcurrentBuilds()
        buildDiscarder(logRotator(numToKeepStr: '20'))
    }

    environment {
        APP_NAME = 'payment-service'
        REGISTRY = 'registry.internal.company.com'
        DOCKER_CREDS = credentials('docker-registry-credentials')
    }

    stages {
        stage('Lint & Test') {
            steps {
                container('golang') {
                    sh 'go test -v -race ./...'
                }
            }
        }

        stage('Build & Push Container') {
            when {
                branch 'main'
            }
            steps {
                container('docker') {
                    sh '''
                        docker login -u ${DOCKER_CREDS_USR} -p ${DOCKER_CREDS_PSW} ${REGISTRY}
                        docker build -t ${REGISTRY}/${APP_NAME}:${BUILD_NUMBER} .
                        docker push ${REGISTRY}/${APP_NAME}:${BUILD_NUMBER}
                    '''
                }
            }
        }
    }

    post {
        always {
            cleanWs()
        }
        failure {
            echo "Pipeline failed! Alerting incident channel..."
        }
    }
}

📖 Extended Guide

1. Modern Controller-Agent Architecture

                       ┌────────────────────────────┐
                       │ Jenkins Master Controller  │
                       │ (Web UI, Job Scheduler)    │
                       └─────────────┬──────────────┘
                                     │ (Dispatches over JNLP / SSH)
         ┌───────────────────────────┼───────────────────────────┐
         ▼                           ▼                           ▼
┌──────────────────┐        ┌──────────────────┐        ┌──────────────────┐
│ Static Agent VM  │        │ Dynamic K8s Pod  │        │ Docker Container │
│ (Legacy / macOS) │        │ (Auto-scaled)    │        │ (Local Worker)   │
└──────────────────┘        └──────────────────┘        └──────────────────┘
  • Best Practice: Never execute builds on the Jenkins Master Controller. Always offload workloads to dynamic, ephemeral worker agents (e.g. Kubernetes Pods) that terminate when the build completes.

2. Declarative vs. Scripted Pipelines

  • Declarative Pipeline (pipeline { ... }): Strict, opinionated syntax with built-in validation, structured error handling, and visual stage rendering in Jenkins Blue Ocean. Recommended for 95% of use cases.
  • Scripted Pipeline (node { ... }): Imperative Groovy-based syntax providing unlimited programming flexibility at the cost of readability and maintenance complexity.

3. Enterprise Jenkins Shared Libraries

Centralize and standardize deployment logic across hundreds of microservices using Jenkins Shared Libraries:

// In shared library repository: vars/standardMicroservicePipeline.groovy
def call(Map config = [:]) {
    pipeline {
        agent any
        stages {
            stage('Standard Build') {
                steps {
                    echo "Building ${config.appName} with version ${config.version}"
                }
            }
        }
    }
}

// In individual microservice Jenkinsfile:
@Library('company-shared-lib@v2.1.0') _
standardMicroservicePipeline(appName: 'order-api', version: '1.0.0')
⌂ All Guides