Schedule parametrized Jenkins jobs from another with properties files

jenkins Feb 24, 2016

Scheduling Jenkins job from another one is quite straightforward task. We can add dedicated post-build action and specify all required parameters. However, problem is a bit more complicated when we want to schedule parametrized jobs dynamically. In this case we don't know in advance:

how many tasks should be scheduled (if any)?
what parameter values assign to each task?

Let's start with a real life example - scheduling tests suite executing job on every changed branch. You can achieve that using Multi-Branch Project Plugin which creates separate task for each of the existing branches. The downside of this approach is that for each of the created tasks we must store separate copy of our project even if it's size is quite significant. It's also possible to schedule jobs using REST API with tools like curl but you have to pass the authorization data each time (which may not be too handy with fully automated approach).

Luckily there is another solution using Parametrized Trigger Plugin - we can generate separate .properties file for each task we want to schedule with all it's parameter values (branch name) and save them into job's workspace with some custom script (eg. bash). Then, using plugin's feature called Parameter factory we can schedule separate job for each of the files.

Enough theory - let's check it on the real environment but in a bit simplier form. The easiest way to set up Jenkins for test purposes is probably to grab it's docker image and run it localy:

docker pull jenkins
docker run -p 8080:8080 -p 50000:50000 jenkins

Jenkins web interface should be now available at http://localhost:8080/.

Firstly, we have to create a job (Freestyle project) to be scheduled:

Our job should have a single String parameter called param:

As the only build step we will execute the following script (printing parameter value):

#!/bin/bash
echo "Hello, $param"

Now we need another job (let's call it job-trigger) that will be scheduling example-job runs. Let's create similar Freestyle project but this time without parameters. As the first build step we will use the following script, which creates separate .properties file for each of given names:

#!/bin/bash
# static value for test purposes
names=("First" "Second" "Third")
i=0
for name in "${names[@]}"
do
   echo "param=$name" > "job_config_$i.properties"
   i=$((i+1))
done
ls -la

All we got to do now is to create the second build step with a regexp expression matching all of the created .properties files. During execution stage, Jenkins will trigger separate jobs for each file passing it's content as key-value parameters.