Jenkins 2.0 and Multi-branch pipeline builds for iOS apps with Fastlane
Jenkins 2.0 beta final is now out. We’ve been using Jenkins for a while to run our automated build system for our iOS and Android app builds. I’ve written about this before:
- iOS Deployment and Provisioning - Part I - Terminology
- I Love our Build System
- iOS Deployment and Provisioning - Part II - Rational
- iOS Deployment and Provisioning - Part III - Tools in Detail - Fastlane
One of the areas I’ve not gone into too much detail on is the Jenkins config itself. That was intended for Part IV of the series above, but I never got around to writing it. But in short we were just using Jenkins to run the Fastlane build when a change occurred on Github. Fastlane was responsible for all the actual heavy work.
We had three Jenkins jobs set up for each app:
- Production build and release
- Alpha build from the develop/ branch
- Alpha builds from the feature/* branches
The latter allowed us to have automated builds from any of the feature branches we had built automatically when code was pushed. The problem was that all of the different feature branches were built by that one single job. That meant that when a branch test/build failed it would mark the Jenkins job red, even though only a single branch had failed. It was then impossible to manually re-run a specific branch build. You had to push a trivial commit to Github on that branch to trigger it.
Jenkins 2.0 comes with the Multibranch Pipeline plugin. It was as though the Jenkins developers were reading my mind, as this is what they had to say about the inclusion of pipelines in Jenkins 2.0:
Problem
As organisations of all types seek to deliver high quality software faster, their use of Jenkins is extending beyond just continuous integration (CI) to continuous delivery (CD). In order to implement continuous delivery, teams need a flexible way to model, orchestrate and visualise their entire delivery pipeline.
Solution
Jenkins 2.0 supports delivery pipelines as a first-class entity. The Pipeline plugin introduces a domain-specific language (DSL) that helps Jenkins users to model their software delivery pipelines as code, which can be checked in and version-controlled along with the rest of their project’s source code.
So I thought I’d give it a try and see how we get on with it. One of the main changes is that the actual pipeline is written in a DSL in Groovy. I have a bit of a hate of DSLs for automation… they are a necessary evil, but can make things very difficult to understand to newcomers to the tool of they don’t already know the language inside out. I’ve battled Fastlane’s Ruby DSL and now I’ve battled Jenkin’s Groovy DSL.
To make it even harder the docs on the Jenkins Pipeline DSL are sparse and not very accurate. This is the curse of any fast-moving early-access project, so something that I hope will improve (and I’ll try and contribute back where I can).
Another issue is working out the best way to split the work between Fastlane and Jenkins. I’ve had to refactor our Fastlane config into a larger number of separate lanes in order to have each lane called as a separate pipeline stage. The idea ultimately is to be able to increase parallelism in the build process, and to allow the builds to be restarted at a particular point — e.g. due to something like a transient network error. Quite often the build of the app itself has succeeded but the upload to Hockey or TestFlight has failed at the end or timed out. I currently have to re-run the entire job (and hence it gets re-built and re-tested).
# Customise this file, documentation can be found here:
# https://github.com/KrauseFx/fastlane/tree/master/docs
# vi: ft=ruby
$:.unshift File.dirname(__FILE__)
require 'lib/utils.rb'
fastlane_version "1.3.0"
default_platform :iOS
platform :iOS do
def run_tests(scheme)
scan(
scheme: scheme,
)
File.rename "../build/reports/report.junit", "../build/reports/#{scheme}-tests.xml"
end
before_all do
ENV['DELIVER_WHAT_TO_TEST'] = git_commit_log
ENV['SIGH_CERTIFICATE_ID'] = '9Q5433VBYW'
GIT_BRANCH = ENV['GIT_BRANCH'] || 'unknown'
GIT_BRANCH_ID = GIT_BRANCH.dup
GIT_BRANCH_ID.gsub! '/', '.'
GIT_BRANCH_ID.gsub! /[^A-Za-z0-9\-.]/, "-"
end
lane :pods do
cocoapods(
pod file: './Nutrition'
)
end
lane :tests do
run_tests('Nutrition')
end
lane :build do |options|
project = 'Nutrition/Nutrition.xcodeproj'
increment_build_number(
xcodeproj: project,
build_number: ENV['BUILD_ID']
)
increment_version_number(
xcodeproj: project,
version_number: '1.4.1',
)
if options[:release]
app_identifier = 'com.enquos.nutrition'
display_name = 'enquos nutrition'
# Disable API server selection in settings app
set_info_plist_value(
path: './Nutrition/Nutrition/Environment.plist',
key: 'isTest',
value: false,
)
# Remove ability to load arbitrary TLS connections (may be added by developers)
set_info_plist_value(
path: './Nutrition/Nutrition/Info.plist',
key: 'NSAppTransportSecurity',
value: { 'NSAllowsArbitraryLoads' => false }
)
sigh(
app_identifier: app_identifier,
)
else
app_identifier = 'com.enquos.Nutrition.alpha.' + GIT_BRANCH_ID
display_name = 'enquos nutrition ' + GIT_BRANCH
sigh(
app_identifier: '*',
adhoc: '1',
)
end
update_app_identifier(
app_identifier: app_identifier,
plist_path: 'Nutrition/Info.plist',
xcodeproj: 'Nutrition/Nutrition.xcodeproj',
)
update_info_plist(
display_name: display_name,
plist_path: 'Nutrition/Info.plist',
xcodeproj: 'Nutrition/Nutrition.xcodeproj',
)
ENV["PROFILE_UUID"] = lane_context[SharedValues::SIGH_UDID]
gym(
scheme: options[:release] ? "Nutrition Release" : "Nutrition",
configuration: options[:release] ? "Release" : "Beta",
workspace: "Nutrition/Nutrition.xcworkspace",
cargos: '-derivedDataPath ./build',
clean: false,
use_legacy_build_api: true,
)
end
lane :alpha do
pods
tests
build_alpha
deploy_alpha
end
lane :build_alpha do
build(release: false)
end
lane :deploy_alpha do
Actions.lane_context[SharedValues::IPA_OUTPUT_PATH] = './Nutrition.ipa'
Actions.lane_context[SharedValues::DSYM_OUTPUT_PATH] = './Nutrition.app.dSYM.zip'
ENV['BUILD_ID'] = get_info_plist_value({ key: "CFBundleVersion", path: "Nutrition/Nutrition/Info.plist"})
hockey(
api_token: 'deadbeef',
notes: git_commit_log,
notify: '0', # Means do not notify
status: '2',
release_type: '2',
)
if ENV["SLACK_URL"]
slack(
message: "New alpha build available for download",
success: true,
payload: {
'Build number' => ENV['BUILD_ID'],
'Git branch' => ENV['GIT_BRANCH'],
'Download URL' => Actions.lane_context[ Actions::SharedValues::HOCKEY_DOWNLOAD_LINK ],
'What\'s new' => git_commit_log,
},
default_payloads: [],
)
end
end
lane :build_release do
tests
build(release: true)
end
lane :deploy_release do
Actions.lane_context[SharedValues::IPA_OUTPUT_PATH] = './Nutrition.ipa'
Actions.lane_context[SharedValues::DSYM_OUTPUT_PATH] = './Nutrition.app.dSYM.zip'
ENV['BUILD_ID'] = get_info_plist_value({ key: "CFBundleVersion", path: "Nutrition/Nutrition/Info.plist"})
hockey(
api_token: 'deadbeef',
notes: git_commit_log,
notify: '0', # Means do not notify
status: '1', # Means do not make available for download
release_type: '1',
public_identifier: 'deadbeef',
)
pilot
if ENV["SLACK_URL"]
slack({
message: "Nutrition Release build #{ENV['BUILD_ID']} uploaded to TestFlight"
})
end
end
end
And then we need to create a Jenkinsfile which contains the description of the pipeline itself:
env.PATH = '/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Applications/Server.app/Contents/ServerRoot/usr/bin:/Applications/Server.app/Contents/ServerRoot/usr/sbin'
env.HOME = '/Users/iosbuilds'
env.USER = 'iosbuilds'
// backwards compat with old branch variable
env.GIT_BRANCH = env.BRANCH_NAME
def getWorkspace() {
pwd().replace("%2F", "_")
}
def wipeWorkspace(String workspace) {
if (workspace) {
sh "find ${workspace} -mindepth 4 -depth -delete"
}
}
def isRelease() {
if (env.RELEASE == "true") {
return true
}
}
node('xcode7') {
try {
// Manually set the workspace to deal with clang
// choking on %2f in the directory
ws(getWorkspace()) {
wrap([$class: 'AnsiColorBuildWrapper', 'colorMapName': 'XTerm', 'defaultFg': 1, 'defaultBg': 2]) {
workspace = pwd()
// Wipe the workspace so we are building completely clean
wipeWorkspace(workspace)
// Mark the code checkout 'stage'....
stage 'Checkout'
// Checkout code from repository
checkout scm
// Copy nutrition database into place
sh "cp ${workspace}/../nutrition.db ${workspace}/Nutrition/Nutrition/nutrition.db"
// Mark the cocoapods 'stage'....
stage 'Cocoapods Install'
sh "fastlane pods"
// Mark the code unit tests 'stage'....
stage 'Tests'
// reset the simulators before running tests
sh "killall Simulator || true"
sh "SNAPSHOT_FORCE_DELETE=yes snapshot reset_simulators"
sh "fastlane tests"
step([$class: 'JUnitResultArchiver', testResults: 'build/reports/*.xml'])
// Mark the code build 'stage'....
stage 'Build'
sh "security list-keychains -s ~/Library/Keychains/iosbuilds.keychain"
sh "security unlock-keychain -p ${env.KEYCHAIN_PASSWORD} /Users/iosbuilds/Library/Keychains/iosbuilds.keychain"
if (isRelease()) {
sh "fastlane build_release"
} else {
sh "fastlane build_alpha"
}
// Mark the code deploy 'stage'....
stage 'Deploy'
if (isRelease()) {
slackSend channel: '#ios', color: 'warning', message: "${env.JOB_NAME} (${env.BUILD_NUMBER}) waiting for confirmation to upload to Testflight.\n${env.BUILD_URL}"
input 'Please confirm OK to deploy to Testflight?'
sh "fastlane deploy_release"
} else {
sh "fastlane deploy_alpha"
}
}
}
} catch (e) {
slackSend channel: '#ios', color: 'danger', message: ":dizzy_face: Build failed ${env.JOB_NAME} (${env.BUILD_NUMBER})\n${env.BUILD_URL}"
throw e
}
}
Some notes on above:
-
Due to our branches having slashes in the names (e.g.
feature/foobar) Jenkins will by default attempt to escape the slashes by replacing them with%2f. This actually confuses a lot of the build tools who un-escape the%2fget a slash back and then treat it as a path separator. So we have a customgetWorkspacemethod that replaces the slashes with underscores so that they don't cause problems later on. -
In order to get a clean build we need to wipe the workspace before the build happens. On 'old' Jenkins jobs there was a checkbox something like 'Wipe workspace before build' this is not present in Jenkins 2.0's pipeline builds. So we have a manual method that attempts to wipe the workspace. As this method takes input in from an essentially untrusted source (Don't let little Bobby Tables create branch names in Github for you), I have tried to make it as safe as can be. I use
findand the-mindepthargument, which should at least limit the potential damage that could occur. -
The pipeline is broken into a number of stages. The idea was that this might increase concurrency. On the Mac Minis running the iOS tests on the simulator, you can only run one set of tests at a time. But in theory other pipeline stages from other jobs could be executing. In practise I've not quite got this working yet... so needs further investigation.
-
When deploying a release we pause and ask for input to confirm to proceed. This is so that an accidental commit to master doesn't cause a new release to be deployed to our beta testers unless we specifically approve it.
The end result:
Looks great! Now to tune the concurrency, which is a job for a later date.