Jenkins 2.0 and Multi-branch pipeline builds for iOS apps with Fastlane

27 April 2016

// under technology jenkins ios fastlane ci pipelines github

Jenkins 2.0 ~~beta~~ final is now out. We’ve been using Jenkins for a while to run our automated build system for our iOS and Android app builds. I’ve written about this before:

One of the areas I’ve not gone into too much detail on is the Jenkins config itself. That was intended for Part IV of the series above, but I never got around to writing it. But in short we were just using Jenkins to run the Fastlane build when a change occurred on Github. Fastlane was responsible for all the actual heavy work.

We had three Jenkins jobs set up for each app:

Production build and release
Alpha build from the develop/ branch
Alpha builds from the feature/* branches

The latter allowed us to have automated builds from any of the feature branches we had built automatically when code was pushed. The problem was that all of the different feature branches were built by that one single job. That meant that when a branch test/build failed it would mark the Jenkins job red, even though only a single branch had failed. It was then impossible to manually re-run a specific branch build. You had to push a trivial commit to Github on that branch to trigger it.

Jenkins 2.0 comes with the Multibranch Pipeline plugin. It was as though the Jenkins developers were reading my mind, as this is what they had to say about the inclusion of pipelines in Jenkins 2.0:

Problem

As organisations of all types seek to deliver high quality software faster, their use of Jenkins is extending beyond just continuous integration (CI) to continuous delivery (CD). In order to implement continuous delivery, teams need a flexible way to model, orchestrate and visualise their entire delivery pipeline.

Solution

Jenkins 2.0 supports delivery pipelines as a first-class entity. The Pipeline plugin introduces a domain-specific language (DSL) that helps Jenkins users to model their software delivery pipelines as code, which can be checked in and version-controlled along with the rest of their project’s source code.

So I thought I’d give it a try and see how we get on with it. One of the main changes is that the actual pipeline is written in a DSL in Groovy. I have a bit of a hate of DSLs for automation… they are a necessary evil, but can make things very difficult to understand to newcomers to the tool of they don’t already know the language inside out. I’ve battled Fastlane’s Ruby DSL and now I’ve battled Jenkin’s Groovy DSL.

To make it even harder the docs on the Jenkins Pipeline DSL are sparse and not very accurate. This is the curse of any fast-moving early-access project, so something that I hope will improve (and I’ll try and contribute back where I can).

Another issue is working out the best way to split the work between Fastlane and Jenkins. I’ve had to refactor our Fastlane config into a larger number of separate lanes in order to have each lane called as a separate pipeline stage. The idea ultimately is to be able to increase parallelism in the build process, and to allow the builds to be restarted at a particular point — e.g. due to something like a transient network error. Quite often the build of the app itself has succeeded but the upload to Hockey or TestFlight has failed at the end or timed out. I currently have to re-run the entire job (and hence it gets re-built and re-tested).

# Customise this file, documentation can be found here:
# https://github.com/KrauseFx/fastlane/tree/master/docs
# vi: ft=ruby

$:.unshift File.dirname(__FILE__)
require 'lib/utils.rb'

fastlane_version "1.3.0"

default_platform :iOS

platform :iOS do

  def run_tests(scheme)
    scan(
        scheme: scheme,
      )
      File.rename "../build/reports/report.junit", "../build/reports/#{scheme}-tests.xml"
  end

  before_all do
    ENV['DELIVER_WHAT_TO_TEST'] = git_commit_log
    ENV['SIGH_CERTIFICATE_ID'] = '9Q5433VBYW'
    GIT_BRANCH = ENV['GIT_BRANCH'] || 'unknown'

    GIT_BRANCH_ID = GIT_BRANCH.dup
    GIT_BRANCH_ID.gsub! '/', '.'
    GIT_BRANCH_ID.gsub! /[^A-Za-z0-9\-.]/, "-"

  end

  lane :pods do
    cocoapods(
      pod file: './Nutrition'
    )
  end

  lane :tests do
    run_tests('Nutrition')
  end

  lane :build do |options|

      project = 'Nutrition/Nutrition.xcodeproj'

      increment_build_number(
        xcodeproj: project,
        build_number: ENV['BUILD_ID']
      )

      increment_version_number(
        xcodeproj: project,
        version_number: '1.4.1',
      )

      if options[:release]
        app_identifier = 'com.enquos.nutrition'
        display_name = 'enquos nutrition'
        # Disable API server selection in settings app
        set_info_plist_value(
          path: './Nutrition/Nutrition/Environment.plist',
          key: 'isTest',
          value: false,
        )
        # Remove ability to load arbitrary TLS connections (may be added by developers)
        set_info_plist_value(
          path: './Nutrition/Nutrition/Info.plist',
          key: 'NSAppTransportSecurity',
          value:  { 'NSAllowsArbitraryLoads' => false }
        )
        sigh(
          app_identifier: app_identifier,
        )
      else
        app_identifier = 'com.enquos.Nutrition.alpha.' + GIT_BRANCH_ID
        display_name = 'enquos nutrition ' + GIT_BRANCH
        sigh(
          app_identifier: '*',
          adhoc: '1',
        )
      end

      update_app_identifier(
        app_identifier: app_identifier,
        plist_path: 'Nutrition/Info.plist',
        xcodeproj: 'Nutrition/Nutrition.xcodeproj',
      )

      update_info_plist(
        display_name: display_name,
        plist_path: 'Nutrition/Info.plist',
        xcodeproj: 'Nutrition/Nutrition.xcodeproj',
      )

      ENV["PROFILE_UUID"] = lane_context[SharedValues::SIGH_UDID]

      gym(
        scheme: options[:release] ? "Nutrition Release" : "Nutrition",
        configuration: options[:release] ? "Release" : "Beta",
        workspace: "Nutrition/Nutrition.xcworkspace",
        cargos: '-derivedDataPath ./build',
        clean: false,
        use_legacy_build_api: true,
      )

  end

  lane :alpha do
    pods
    tests
    build_alpha
    deploy_alpha
  end

  lane :build_alpha do
    build(release: false)
  end

  lane :deploy_alpha do

    Actions.lane_context[SharedValues::IPA_OUTPUT_PATH] = './Nutrition.ipa'
    Actions.lane_context[SharedValues::DSYM_OUTPUT_PATH] = './Nutrition.app.dSYM.zip'
    ENV['BUILD_ID'] = get_info_plist_value({ key: "CFBundleVersion", path: "Nutrition/Nutrition/Info.plist"})

    hockey(
      api_token: 'deadbeef',
      notes: git_commit_log,
      notify: '0', # Means do not notify
      status: '2',
      release_type: '2',
    )

    if ENV["SLACK_URL"]
      slack(
        message: "New alpha build available for download",
        success: true,
        payload: {
          'Build number' => ENV['BUILD_ID'],
          'Git branch' => ENV['GIT_BRANCH'],
          'Download URL' => Actions.lane_context[ Actions::SharedValues::HOCKEY_DOWNLOAD_LINK ],
          'What\'s new' => git_commit_log,
          },
        default_payloads: [],
     )
    end
  end

  lane :build_release do
    tests
    build(release: true)
  end

  lane :deploy_release do

    Actions.lane_context[SharedValues::IPA_OUTPUT_PATH] = './Nutrition.ipa'
    Actions.lane_context[SharedValues::DSYM_OUTPUT_PATH] = './Nutrition.app.dSYM.zip'
    ENV['BUILD_ID'] = get_info_plist_value({ key: "CFBundleVersion", path: "Nutrition/Nutrition/Info.plist"})

    hockey(
      api_token: 'deadbeef',
      notes: git_commit_log,
      notify: '0', # Means do not notify
      status: '1', # Means do not make available for download
      release_type: '1',
      public_identifier: 'deadbeef',
    )

    pilot

    if ENV["SLACK_URL"]
      slack({
        message: "Nutrition Release build #{ENV['BUILD_ID']} uploaded to TestFlight"
      })
    end
  end

end

And then we need to create a Jenkinsfile which contains the description of the pipeline itself:

env.PATH = '/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Applications/Server.app/Contents/ServerRoot/usr/bin:/Applications/Server.app/Contents/ServerRoot/usr/sbin'
env.HOME = '/Users/iosbuilds'
env.USER = 'iosbuilds'
// backwards compat with old branch variable
env.GIT_BRANCH = env.BRANCH_NAME


def getWorkspace() {
    pwd().replace("%2F", "_")
}

def wipeWorkspace(String workspace) {
    if (workspace) {
        sh "find ${workspace} -mindepth 4 -depth -delete"
    }
}

def isRelease() {
    if (env.RELEASE == "true") {
      return true
    }
}

node('xcode7') {
   try {
     // Manually set the workspace to deal with clang 
     // choking on %2f in the directory
     ws(getWorkspace()) {
       wrap([$class: 'AnsiColorBuildWrapper', 'colorMapName': 'XTerm', 'defaultFg': 1, 'defaultBg': 2]) {
         workspace = pwd()
         // Wipe the workspace so we are building completely clean
         wipeWorkspace(workspace) 
         // Mark the code checkout 'stage'....
         stage 'Checkout'
         // Checkout code from repository
         checkout scm
         // Copy nutrition database into place
         sh "cp ${workspace}/../nutrition.db ${workspace}/Nutrition/Nutrition/nutrition.db"

         // Mark the cocoapods 'stage'....
         stage 'Cocoapods Install'
         sh "fastlane pods"   

         // Mark the code unit tests 'stage'....
         stage 'Tests'
         // reset the simulators before running tests
         sh "killall Simulator || true"
         sh "SNAPSHOT_FORCE_DELETE=yes snapshot reset_simulators"
         sh "fastlane tests"   

         step([$class: 'JUnitResultArchiver', testResults: 'build/reports/*.xml'])

         // Mark the code build 'stage'....
         stage 'Build'
         sh "security list-keychains -s ~/Library/Keychains/iosbuilds.keychain"
         sh "security unlock-keychain -p ${env.KEYCHAIN_PASSWORD} /Users/iosbuilds/Library/Keychains/iosbuilds.keychain"
         if (isRelease()) {
           sh "fastlane build_release"
         } else {
           sh "fastlane build_alpha"
         }

         // Mark the code deploy 'stage'....
         stage 'Deploy'
         if (isRelease()) {
           slackSend channel: '#ios', color: 'warning', message: "${env.JOB_NAME} (${env.BUILD_NUMBER}) waiting for confirmation to upload to Testflight.\n${env.BUILD_URL}"
           input 'Please confirm OK to deploy to Testflight?'
           sh "fastlane deploy_release"
         } else {
           sh "fastlane deploy_alpha"
         }
      }
    }
  } catch (e) {
    slackSend channel: '#ios', color: 'danger', message: ":dizzy_face: Build failed ${env.JOB_NAME} (${env.BUILD_NUMBER})\n${env.BUILD_URL}"
    throw e
  }
}

Some notes on above:

Due to our branches having slashes in the names (e.g. feature/foobar) Jenkins will by default attempt to escape the slashes by replacing them with %2f. This actually confuses a lot of the build tools who un-escape the %2f get a slash back and then treat it as a path separator. So we have a custom getWorkspace method that replaces the slashes with underscores so that they don't cause problems later on.
In order to get a clean build we need to wipe the workspace before the build happens. On 'old' Jenkins jobs there was a checkbox something like 'Wipe workspace before build' this is not present in Jenkins 2.0's pipeline builds. So we have a manual method that attempts to wipe the workspace. As this method takes input in from an essentially untrusted source (Don't let little Bobby Tables create branch names in Github for you), I have tried to make it as safe as can be. I use find and the -mindepth argument, which should at least limit the potential damage that could occur.
The pipeline is broken into a number of stages. The idea was that this might increase concurrency. On the Mac Minis running the iOS tests on the simulator, you can only run one set of tests at a time. But in theory other pipeline stages from other jobs could be executing. In practise I've not quite got this working yet... so needs further investigation.
When deploying a release we pause and ask for input to confirm to proceed. This is so that an accidental commit to master doesn't cause a new release to be deployed to our beta testers unless we specifically approve it.

The end result:

Jenkins 2.0 Pipeline view

Looks great! Now to tune the concurrency, which is a job for a later date.

Go Top