Caching Carthage con CircleCI

Datetime:2016-08-23 01:40:04          Topic:          Share

I hope we can agree that Using a CI system for automating your tests is important. Unfortunately, continuous integration for iOS apps hasn’t always been a great experience. Here at thoughtbot, we’ve run the gamut of services as I’m sure most have. Jenkins , Travis , Xcode bots , Circle . We’ve tried them all. We ended up choosing Circle for a number of reasons. A big reason was Circle’s ability to cache directories, which improves build times immensely.

By default, Circle already supports caching builds with CocoaPods . This is fantastic, and for a lot of users will be more than enough. However, we’ve been using Carthage more and more internally, due to the lightweight nature of the tooling, and the ability to use binary frameworks.

Unfortunately, trying to use Carthage caused problems with CI. Of note, the need to code-sign frameworks as they were built meant that we’d need to share signing certificates with our CI service. Additionally, Carthage is itself written in Swift. That means that it needs to be built with a specific version of Xcode, which complicates the build process using Homebrew .

Thankfully, these issues have been fixed! Carthage has been updated to Swift 2.0, which means that Homebrew can build with the latest stable version of Xcode. In addition, since Carthage version 0.11 , it no longer requires code-signing when building frameworks, so there’s no need to pass around signing certificates. Even better, Circle recently started pre-installing Carthage on their Mac build systems so we no longer have to worry about manually installing it.

So where does all of this leave us? We can now build iOS projects on Circle, and we can use Carthage as our dependency management tool of choice. But there’s a problem: Circle doesn’t know how to cache Carthage dependencies by default, and as a result, our builds are really really slow. Since Carthage has to re-build all of our dependencies every time, it could easily take over 10 minutes to build a project and all of its dependencies.

Luckily for us, Circle’s build process is easily customized in way that will let us work around this issue.

Note that we’re assuming that you have added your Carthage/ directory to your .gitignore for the purposes of this post. If you are checking in Carthage/Checkouts or Carthage/Build , this post might not be super useful for you.

Improving Carthage

To begin, we need to look at adding some smarts around Carthage. By design, Carthage is a simple tool, but that same simplicity will let us write a thin wrapper to add custom behavior. In this case, what we’d like to do is to only run carthage bootstrap if the dependencies have changed. Carthage doesn’t do this by default, but it’s fairly trivial to handle ourselves.

Carthage generates a file named Cartfile.resolved that declares the exact dependencies it expects to be installed, similar to Podfile.lock or Gemfile.lock . We can use that to determine which dependencies we have locally and which ones we expect based on the current state of the repo.

To do this, we will first wrap the carthage bootstrap command to perform an additional action. We’ll save this as bin/bootstrap :

#!/bin/sh

carthage bootstrap
cp Cartfile.resolved Carthage

Don’t forget to chmod +x bin/bootstrap so that this becomes executable.

This small script will run carthage bootstrap , and then copy the Cartfile.resolved into our (gitignored) Carthage/ directory. This means that Carthage/Cartfile.resolved will always reflect the currently downloaded/built dependencies, while ./Cartfile.resolved will reflect the dependencies that the project expects.

Now we can write another small script that will use this new Cartfile.resolved file to determine if it needs to update the dependencies. We’ll save this as bin/bootstrap-if-needed :

#!/bin/sh

if ! cmp -s Cartfile.resolved Carthage/Cartfile.resolved; then
  bin/bootstrap
fi

Don’t forget to chmod +x bin/bootstrap-if-needed so that this becomes executable.

This script will compare ./Cartfile.resolved and Carthage/Cartfile.resolved . If they are different (or if Carthage/Cartfile.resolved doesn’t exist), we will run our bin/bootstrap script, which will in turn update the dependencies and move the new Cartfile.resolved into place.

You can now test this. Running bin/bootstrap-if-needed should update your dependencies the first time, but running it a second time should become a no-op. Updating the dependencies (or deleting Carthage/Cartfile.resolved ) should also result in the script re-installing the dependencies.

Caching with Circle

So now comes the fun part. Since we’re now able to determine if we need to update our dependencies, we can leverage Circle’s built in (and fantastic) support for caching to speed up our builds.

To accomplish this, we’re going to add some config to our circle.yml file . Specifically, we’re going to override the dependencies key:

dependencies:
  override:
    - bin/bootstrap-if-needed
  cache_directories:
    - "Carthage"

Seriously, that’s it. The override key tells Circle what command to run when installing dependencies. We tell it to use our smarter version of carthage bootsrap . Then, we tell it to cache the entire Carthage directory. Circle automatically moves the cache into place before the dependency step, and saves it at the end. By the time we run bin/bootstrap-if-needed , everything should be in place.

Since we’re caching the entire directory, that means we’re also caching the Carthage/Cartfile.resolved file we create with bin/bootstrap . So when Circle runs bin/bootstrap-if-needed , it will only build our dependencies if the cached dependencies are out of date.

And there you have it. Once you push this up and Circle starts to use the new config, you should see a dramatic decrease in build times. We saw our times drop from 14 minutes to under 2 minutes.