Continuous Integration of Positron with TaskCluster

Datetime:2016-08-23 02:32:00          Topic: Continuous Integration           Share

I recently enabled continuous integration of Positron using TaskCluster’s support for GitHub repositories . I’ve used Travis CI to integrate other GitHub projects, but Positron is too large for Travis to build without timing out, since it builds all of Gecko. So I needed an alternative.

Caveat: TaskCluster only supports GitHub repositories in the mozilla organization or another org that is explicitly configured for it, as described by Setting up Taskcluster with a Github Organization . So it won’t work with your personal fork.

Also, while the Positron repository is based on gecko-dev , I didn’t use GitHub’s “fork” feature to create it, since GitHub won’t let you fork a repo to another repo in the same organization. Instead, I created a new repository and then copied gecko-dev’s commits to it, as described in Copy/fork a git repo on github into same organization .

(I also had to push the copied commits to GitHub in batches, as described in Github remote push pack size exceeded . And I had to push the first batch via special syntax, per Pushing a large github repo fails with “unable to push to unqualified destination: master” .)

To enable CI, I first created this Docker image , based on Ubuntu 14.04 (Trusty), that includes the Mozilla build toolchain and dependencies installed by Gecko’s bootstrap.py script . The image also includes the taskcluster-vcs utility , which caches gecko-dev and clones from the cache so that TaskCluster pulls from GitHub are lighter-weight.

Here’s the Dockerfile for that image:

FROM ubuntu:trusty

RUN apt-get update -y && \
    apt-get install -y \
      curl \
      git \
      nodejs \
      nodejs-legacy \
      npm \
      python \
      wget

RUN npm install -g taskcluster-vcs

RUN wget -O bootstrap.py https://raw.githubusercontent.com/mozilla/positron/master/python/mozboot/bin/bootstrap.py && \
    python bootstrap.py --no-interactive --application-choice=browser

Then I replaced the upstream .taskcluster.yml file with one that clones and builds Positron when a developer submits a pull request or lands a change. .taskcluster.yml is like .travis.yml , but for TaskCluster. Here’s that file:

version: 0
metadata:
    name: "Positron"
    description: "continuous integration for Positron"
    owner: "{{ event.head.user.email }}"
    source: "{{ event.head.repo.url }}"
tasks:
    - provisionerId: "{{ taskcluster.docker.provisionerId }}"
      workerType: "{{ taskcluster.docker.workerType }}"
      extra:
          github:
              env: true
              events:
                  - pull_request.opened
                  - pull_request.synchronize
                  - pull_request.reopened
                  - push
      payload:
          maxRunTime: 7200 # seconds (i.e. two hours)
          image: "mykmelez/docker-build-positron:latest"
          command:
              - "/bin/bash"
              - "--login"
              - "-c"
              - "tc-vcs checkout repo https://github.com/mozilla/gecko-dev $GITHUB_HEAD_REPO_URL $GITHUB_HEAD_BRANCH &&
                 cd repo/ && SHELL=/bin/bash ./mach build"
      metadata:
          name: "checkoutandbuild"
          description: "check out and build the repository"
          owner: "{{ event.head.user.email }}"
          source: "{{ event.head.repo.url }}"

Finally, I added TaskClusterRobot to the list of collaborators in the repo’s Settings > Collaborators & teams list, giving it Write permissions, so it can annotate pull requests with icons representing the state of the job, just like Travis does:

Et voilà, continuous integration for Positron!

(Once Positron starts running Node and Electron tests, I’ll add tasks to run those tests and report their results.)





About List