# Configure Jenkins to run on K8S

Jenkins An open source automation server which helps people to build, deploy and automate their tasks. I am using it for the following purposes:

* Treat as the cron job which runs some tasks periodically
* Treat as a Pipeline which runs several long time integration test

Jenkins is based on a master-slave structure, master runs as the task scheduler, and slaves run as the task agent. If you setup Jenkins on a single node cluster, master and salves are on the same node. In my case, I deploy Jenkins on a K8s cluster. By doing so, you could expect the master runs as a pod on a node which is scheduled by K8s scheduler, and Jenkins jobs would be running as pods as well.

## Pre-requisition

* Setup A running k8s cluster
* Deploy Jenkins with the following YAMLs

{% code title="00-ns.yaml" %}

```
apiVersion: v1
kind: Namespace
metadata:
  name: pks-testbed
```

{% endcode %}

{% code title="01-service-account.yaml" %}

```
apiVersion: v1
kind: ServiceAccount
metadata:
  name: jenkins-service-account
  namespace: pks-testbed
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  annotations:
    rbac.authorization.kubernetes.io/autoupdate: "true"
  creationTimestamp: null
  labels:
    kubernetes.io/bootstrapping: rbac-defaults
  name: jenkins
rules:
- apiGroups: ['']
  resources: ['events',
              'namespaces',
              'nodes',
              'nodes/stats',
              'pods',
              'services']
  verbs: ['get', 'list', 'watch', 'create', 'delete']
- apiGroups: ['extensions']
  resources: ['deployments']
  verbs: ['get', 'list', 'watch']
- nonResourceURLs: ["/metrics"]
  verbs:
  - get
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: jenkins
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: jenkins
subjects:
- kind: ServiceAccount
  name: jenkins-service-account
  namespace: pks-testbed
```

{% endcode %}

{% code title="02-deployment.yaml" %}

```
apiVersion: apps/v1
kind: Deployment
metadata:
  name: jenkins-deployment
  namespace: pks-testbed
spec:
  selector:
    matchLabels:
      app: jenkins
  replicas: 1
  template:
    metadata:
      labels:
        app: jenkins
    spec:
      serviceAccountName: jenkins-service-account
      containers:
      - name: jenkins
        image: jenkins/jenkins:lts
        imagePullPolicy: Always
        ports:
        - name: http-port
          containerPort: 8080
        - name: jnlp-port
          containerPort: 50000

```

{% endcode %}

{% code title="03-service.yaml" %}

```
apiVersion: v1
kind: Service
metadata:
  name: jenkins-np
  namespace: pks-testbed
spec:
  type: NodePort
  selector:
    app: jenkins
  ports:
  - name: http
    port: 80
    targetPort: 8080
    nodePort: 30036
    protocol: TCP
---
apiVersion: v1
kind: Service
metadata:
  name: jenkins-lb
  namespace: pks-testbed
spec:
  type: LoadBalancer
  selector:
    app: jenkins
  ports:
  - name: http
    port: 80
    targetPort: 8080
    protocol: TCPs

```

{% endcode %}

After deploy the Jenkins, you should be able to access it from your node IP or external IP from your load balancer.

## Configurations

* Install the kubernetes plugin
* Go to "Manage Jenkins" -> "Configure System" -> "Add new cloud", choose "Kubernetes"
* Run "kubectl cluster-info" to get the "Kubernetes URL"
* Get the IP of the pod on which Jenkins master is running on:

```
kubo@jumper:~/jenkins$ kubectl get pods -n pks-testbed
NAME                                 READY   STATUS    RESTARTS   AGE
jenkins-deployment-cb7bfb86c-mvkl6   1/1     Running   0          45h

kubo@jumper:~/jenkins$ kubectl describe pod jenkins-deployment-cb7bfb86c-mvkl6 -n pks-testbed
Name:               jenkins-deployment-cb7bfb86c-mvkl6
Namespace:          pks-testbed
Priority:           0
PriorityClassName:  <none>
Node:               ec78dee2-3652-4c45-8621-1877e66856bc/30.0.3.4
Start Time:         Fri, 29 Mar 2019 01:22:55 -0700
Labels:             app=jenkins
                    pod-template-hash=cb7bfb86c
Annotations:        <none>
Status:             Running
IP:                 40.0.0.2
Controlled By:      ReplicaSet/jenkins-deployment-cb7bfb86c
......
```

![](https://4165656130-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-Lb7SAstgE6Ae0BisAes%2F-LbHIHMv2-BTGumMUuy3%2F-LbHIRhu0uCuHo6jXIWU%2FScreen%20Shot%202019-03-30%20at%2010.35.58%20PM.png?alt=media\&token=8052005b-acb3-457d-a99c-4e07ed53b11d)

### Use freestyle project

* Adding pod template here is required for Jenkins freestyle projects to be run on a pod.&#x20;
  * Under the "Image" section, Click on "Add Pod Template"&#x20;
  * Add "Containers" is a little bit tricky. By default Jenkins will use a container named "**jnlp"** to run your job even you don't add any "Containers" in configurations. If you add another container whose name is different from "**jnlp**", Jenkins will bring up a pod with two containers(jnlp + the\_other\_one), and the script you put in your "**Execute Shell**" section will only be executed in "**jnlp**". If you want to run your shell script in your own container, you have to [build you own slave image](https://daniel-guo.gitbook.io/notebook/cicd-related/broken-reference), and add the container as "**jnlp**". By doing this, the default jnlp image from Jenkins will be replaced.&#x20;

![](https://4165656130-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-Lb7SAstgE6Ae0BisAes%2F-LbHIHMv2-BTGumMUuy3%2F-LbHIafFuDMkHuMJFvkF%2FScreen%20Shot%202019-03-30%20at%2010.52.20%20PM.png?alt=media\&token=70e1b5ca-0c19-406c-a96f-c7a1075dbb8a)

### Use pipeline

The more flexible solution to use different docker images for different job is to use Jenkins Pipeline.

* Install "[Pipeline](https://wiki.jenkins-ci.org/display/JENKINS/Pipeline+Plugin)" plugin, then you will be able to add a new item as pipeline. ([Official Doc](https://jenkins.io/doc/book/pipeline/))
* In your pipeline script, you could run any shell script in a specific container

```
def label = "mypod-${UUID.randomUUID().toString()}"
podTemplate(label: label) {
  node(label) {
    stage('Run shell') {
      container('mycontainer') {
        sh 'echo hello world'
      }
    }
  }
}
```

{% code title="Advanced Example" %}

````
def pod_name = "${JOB_NAME}-${UUID.randomUUID().toString()}"
def pod_label = "${JOB_NAME}"
def docker_image = params.docker_image

podTemplate(
        name: pod_name,
        label: pod_label,
        containers: [
                containerTemplate(name: 'hello-jenkins',
                        image: docker_image,
                        alwaysPullImage: true,
                        ttyEnabled: true,
                        command: 'cat')
        ]
)
{
    node(pod_label) {
        stage('Provision') {
            container('hello-jenkins') {
              sh ```
              echo hello-world
              ```  
            }
        }
    }
}

````

{% endcode %}

{% hint style="info" %}
If use pipeline, you could configure the pot template, container template in pipeline script instead of doing it from "Manage Jenkins" UI portal.
{% endhint %}
