Skip to content

dockerExecuteOnKubernetes

Description

Executes a closure inside a container in a kubernetes pod. Proxy environment variables defined on the Jenkins machine are also available in the container.

By default jnlp agent defined for kubernetes-plugin will be used (see https://github.com/jenkinsci/kubernetes-plugin#pipeline-support).

It is possible to define a custom jnlp agent image by

  1. Defining the jnlp image via environment variable JENKINS_JNLP_IMAGE in the Kubernetes landscape
  2. Defining the image via config (jenkinsKubernetes.jnlpAgent)

Option 1 will take precedence over option 2.

Prerequisites

  • The Jenkins should be running on kubernetes.
  • An environment variable ON_K8S should be created on Jenkins and initialized to true. This could for example be done via Jenkins - Manage Jenkins - Configure System - Global properties - Environment variables

Jenkins environment variable configuration

Parameters

name mandatory default possible values
additionalPodProperties no
containerCommand no
containerCommands no
containerEnvVars no
containerMap no
containerName no
containerPortMappings no
containerPullImageFlags no
containerShell no
containerWorkspaces no
dockerEnvVars no
dockerImage no
dockerPullImage no true
dockerWorkspace no
jenkinsKubernetes/inheritFrom no
jenkinsKubernetes/jnlpAgent no ppiper/jenkins-agent-k8s:v8
jenkinsKubernetes/namespace no
nodeSelector no
resources no
script yes
securityContext no
sidecarEnvVars no
sidecarImage no
sidecarName no
sidecarPullImage no true
sidecarReadyCommand no
sidecarWorkspace no
stashContent no []
stashExcludes no [workspace:nohup.out]
stashIncludes no [workspace:**/*]
stashNoDefaultExcludes no true, false
verbose no true, false
  • additionalPodProperties - Additional pod specific configuration. Map with the properties names as key and the corresponding value as value. The value can also be a nested structure. The properties will be added to the pod spec inside node spec at the same level like e.g. containers. This property provides some kind of an expert mode. Any property which is not handled otherwise by the step can be set. It is not possible to overwrite e.g. the containers property or to overwrite the securityContext property. Alternate way for providing additionalPodProperties is via general/jenkinsKubernetes/additionalPodProperties in the project configuration. Providing the resources map as parameter to the step call takes precedence. This freedom comes with great responsibility. The property additionalPodProperties should only be used in case you really know what you are doing.
  • containerCommand - Allows to specify start command for container created with dockerImage parameter to overwrite Piper default (/usr/bin/tail -f /dev/null).
  • containerCommands - Specifies start command for containers to overwrite Piper default (/usr/bin/tail -f /dev/null). If container's defaultstart command should be used provide empty string like: ['selenium/standalone-chrome': ''].
  • containerEnvVars - Specifies environment variables per container. If not provided dockerEnvVars will be used.
  • containerMap - A map of docker image to the name of the container. The pod will be created with all the images from this map and they are labelled based on the value field of each map entry. Example: ['maven:3.5-jdk-8-alpine': 'mavenExecute', 'selenium/standalone-chrome': 'selenium', 'famiko/jmeter-base': 'checkJMeter', 'ppiper/cf-cli': 'cloudfoundry']
  • containerName - Optional configuration in combination with containerMap to define the container where the commands should be executed in.
  • containerPortMappings - Map which defines per docker image the port mappings, e.g. containerPortMappings: ['selenium/standalone-chrome': [[name: 'selPort', containerPort: 4444, hostPort: 4444]]].
  • containerPullImageFlags - Specifies the pullImage flag per container.
  • containerShell - Allows to specify the shell to be executed for container with containerName.
  • containerWorkspaces - Specifies a dedicated user home directory per container which will be passed as value for environment variable HOME. If not provided dockerWorkspace will be used.
  • dockerEnvVars - Environment variables to set in the container, e.g. [http_proxy:'proxy:8080'].
  • dockerImage - Optional name of the docker image that should be used. If no docker image is provided, the closure will be executed in the jnlp agent container.
  • dockerPullImage - Set this to 'false' to bypass a docker image pull. Useful during development process. Allows testing of images which are available in the local registry only.
  • dockerWorkspace - Specifies a dedicated user home directory for the container which will be passed as value for environment variable HOME.
  • jenkinsKubernetes/inheritFrom - Name of the pod template that should be inherited from. The pod template can be defined in the Jenkins UI
  • jenkinsKubernetes/jnlpAgent - Jnlp agent Docker images which should be used to create new pods.
  • jenkinsKubernetes/namespace - Namespace that should be used to create a new pod
  • nodeSelector - Defines the Kubernetes nodeSelector as per https://github.com/jenkinsci/kubernetes-plugin.
  • resources - A map containing the resources per container. The key is the container name. The value is a map defining valid resources. An entry with key DEFAULT can be used for defining resources for all containers which does not have resources specified otherwise. Alternate way for providing resources is via general/jenkinsKubernetes/resources in the project configuration. Providing the resources map as parameter to the step call takes precedence.
  • script - The common script environment of the Jenkinsfile running. Typically the reference to the script calling the pipeline step is provided with the this parameter, as in script: this. This allows the function to access the commonPipelineEnvironment for retrieving, e.g. configuration parameters.
  • securityContext - Kubernetes Security Context used for the pod. Can be used to specify uid and fsGroup. See: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/
  • sidecarEnvVars - as dockerEnvVars for the sidecar container
  • sidecarImage - as dockerImage for the sidecar container
  • sidecarName - SideCar only: Name of the container in local network.
  • sidecarPullImage - Set this to 'false' to bypass a docker image pull. Useful during development process. Allows testing of images which are available in the local registry only.
  • sidecarReadyCommand - Command executed inside the container which returns exit code 0 when the container is ready to be used.
  • sidecarWorkspace - as dockerWorkspace for the sidecar container
  • stashContent - Specific stashes that should be considered for the step execution.
  • stashExcludes - In the Kubernetes case the workspace is only available to the respective Jenkins slave but not to the containers running inside the pod.
    This configuration defines exclude pattern for stashing from Jenkins workspace to working directory in container and back. Following excludes can be set: * workspace: Pattern for stashing towards container * stashBack: Pattern for bringing data from container back to Jenkins workspace. If not set: defaults to setting for workspace.
  • stashIncludes - In the Kubernetes case the workspace is only available to the respective Jenkins slave but not to the containers running inside the pod.
    This configuration defines include pattern for stashing from Jenkins workspace to working directory in container and back. Following includes can be set: * workspace: Pattern for stashing towards container * stashBack: Pattern for bringing data from container back to Jenkins workspace. If not set: defaults to setting for workspace.
  • stashNoDefaultExcludes - In the Kubernetes case the workspace is only available to the respective Jenkins slave but not to the containers running inside the pod.
    This configuration defines include pattern for stashing from Jenkins workspace to working directory in container and back. This flag controls whether the stashing does not use the default exclude patterns in addition to the patterns provided in stashExcludes.
  • verbose - Print more detailed information into the log.

Step configuration

We recommend to define values of step parameters via config.yml file.

In following sections of the config.yml the configuration is possible:

parameter general step/stage
additionalPodProperties X X
containerCommand X
containerCommands X
containerEnvVars X
containerMap X
containerName X
containerPortMappings X
containerPullImageFlags X
containerShell X
containerWorkspaces X
dockerEnvVars X
dockerImage X
dockerPullImage X
dockerWorkspace X
jenkinsKubernetes/inheritFrom X X
jenkinsKubernetes/jnlpAgent X X
jenkinsKubernetes/namespace X X
nodeSelector X
resources X X
script
securityContext X
sidecarEnvVars X
sidecarImage X
sidecarName X
sidecarPullImage X
sidecarReadyCommand X
sidecarWorkspace X
stashContent X
stashExcludes X
stashIncludes X
stashNoDefaultExcludes X
verbose X X

Dependencies

The step depends on the following Jenkins plugins

The kubernetes plugin is only used if running in a kubernetes environment. Transitive dependencies are omitted.

The list might be incomplete.

Consider using the ppiper/jenkins-master docker image. This images comes with preinstalled plugins.

Side effects

none

Exceptions

none

Example 1: Run a closure in a single container pod

# set environment variable
export ON_K8S=true"
dockerExecuteOnKubernetes(script: script, dockerImage: 'maven:3.5-jdk-7'){
    sh "mvn clean install"
}

In the above example, a pod will be created with a docker container of image maven:3.5-jdk-7. The closure will be then executed inside the container.

Example 2: Run a closure in a multi-container pod

# set environment variable
export ON_K8S=true"
dockerExecuteOnKubernetes(script: script, containerMap: ['maven:3.5-jdk-8-alpine': 'maven', 'ppiper/cf-cli': 'cfcli']){
    container('maven'){
        sh "mvn clean install"
    }
    container('cfcli'){
        sh "cf plugins"
    }
}

In the above example, a pod will be created with multiple Docker containers that are passed as a containerMap. The containers can be chosen for executing by referring their labels as shown in the example.

Example 3: Running a closure in a dedicated container of a multi-container pod

# set environment variable
export ON_K8S=true"
dockerExecuteOnKubernetes(
  script: script,
  containerCommands: ['selenium/standalone-chrome': ''],
  containerMap: ['maven:3.5-jdk-8-alpine': 'maven', 'selenium/standalone-chrome': 'selenium'],
  containerName: 'maven',
  containerPortMappings: ['selenium/standalone-chrome': [containerPort: 4444, hostPort: 4444]]
  containerWorkspaces: ['selenium/standalone-chrome': '']
){
  echo "Executing inside a Kubernetes Pod inside 'maven' container to run Selenium tests"
  sh "mvn clean install"
}