These are some notes for the development of nightly fennec on TaskCluster.

Funsize-Balrog scripworker is a worker that will move artifacts from a TaskCluster location into an S3 bucket and notify a balrog service with a specified manifest. This is the last portion of the nightly pipeline which will need to be intergrated in order to release the builds that have been signed.

This short guide will walk you through the process of setting up a local instance of Balrog and BalrogWorker which will involve docker and the use of TaskCluster authentication.

You must have docker and docker-compose installed on your machine. First clone both repositories locally.

$ git clone https://github.com/mozilla-releng/funsize-balrogworker
$ git clone --recursive https://github.com/mozilla/balrog

The balrog setup is straightforward. Make sure to initialize the admin-ui submodule before you instantiate the local instance.

$ docker-compose up

The setup for the scripworker is a little less straightforward. The important files to keep note of are the config.json and the Dockerfile.

Follow the provided instructions to update the public keys. This will download the release, nightly, and dep keys from mozilla-central.

$ make update pub_keys

Then generate the docker build.

$ docker build --tag balrogworker .

In order for the worker to do anything useful within the context of TaskCluster, it will need a way to authenticate against the system and obtain the appropriate scopes. We can do this by generating a new client for our local balrogworker.

First you should obtain the project:taskcluster:worker-test-scopes role by joining the taskcluster-contributors Mozillians group. Make sure that you have an associated @mozilla.com email address associated with your account. Once you have been approved, you will have the ability to generate a client that can use these scopes on your behalf.

Visit the Client Manager on the TaskCluster tools site and generate a new client. In the Client Scopes field, enter the following.

assume:project:taskcluster:worker-test-scopes

You should now have a client-id in the form mozilla-ldap/<your-name>@mozilla.com/<client-name> along with an access token. You will need to generate a new access token if you lose it.

Balrogworker is configured through environmental variables. Balrogworker will copy artifacts to a S3 bucket before notifying balrog of the new changes if S3 credentials are specified. We will be omitting the S3 step and pass TaskCluster urls directly to Balrog.

Provided is a script to launch the balrogworker instance.

#!/bin/bash

# Taskcluster client information. This should assume worker-test-scopes.
export TASKCLUSTER_CLIENT_ID="mozilla-ldap/<your-name>@mozilla.com/<client-name>"
export TASKCLUSTER_ACCESS_TOKEN="<access-token>"

# This should match `dummy-worker-*` allowed by 
# `project:taskcluster:worker-test-scopes`, as well as the worker name 
# specified in `config.json`. This should be unique so there are no name 
# conflicts, which could lead to mutliple unrelated workers racing for the 
# same task definition.
export SCRIPTWORKER_WORKER_ID="dummy-worker-balrog"

# Point to the local balrog instance
export BALROG_API_ROOT="127.0.0.1:8080"
export BALROG_USERNAME="balrogadmin"

# Run the docker instance with the above environmental variables. This does 
# not include S3 credentials. Refer to the README for more details.
docker run -ti \
        -e TASKCLUSTER_CLIENT_ID=$TASKCLUSTER_CLIENT_ID \
        -e TASKCLUSTER_ACCESS_TOKEN=$TASKCLUSTER_ACCESS_TOKEN \
        -e SCRIPTWORKER_WORKER_ID=$SCRIPTWORKER_WORKER_ID \
        -e BALROG_API_ROOT=$BALROG_API_ROOT \
        -e BALROG_USERNAME=$BALROG_USERNAME \
        -e BALROG_PASSWORD="wootwoot" \
        balrogworker

The worker should now be polling the task queue for messages to consume.

• Provide a way to validate that your setup is correct
• Add more context to balrogworker in the nightly fennec graph
• Broad overview of how balrogworker works

• mozilla-releng/funsize-balrogworker
• mozilla-balrog

Periodic scheduling is an important aspect of generating a nightly Fennec build. This will provide an overview of how nightly Fennec is scheduled, the tools available in TaskCluster for periodic task execution, and ideas on expanding and improving existing solutions.

launching task graphs. Taskgraph is primarily used in the decision task, a task that will generate selected builds and tests specified by a try commit message. This tool generates task definitions from configuration files defined in tree, finds all the neccessary tasks to fulfill the target set of tasks, and inserts them into the queue with the proper dependencies.

The taskgraph for nightly fennec is specified a nightly target set, which includes tasks for the build, signing, and upload of nightly fennec binaries. We can now specify a triggered-by argument in taskgraph to specify that we want to target a nightly graph. This acts as the entrypoint where we can generate the nightly fennec graph.

TaskCluster Hooks is a service that will execute tasks on a periodic basis. You can register a task definition and associate it with a cron schedule. You can also take advantage of the authorization model of TaskCluster to limit access to editing and manual triggering of hooks.

Mozilla-TaskCluster is a similar service that creates decision tasks by polling the pushlog on the Try-Server and filling in a task template with the necessary details. The Fennec nightly hook takes the decision definition and adds a few modifications. We want to generate decision tasks on a periodic basis where the triggered-by argument is set to nightly. We also want to generate nightlies from the TIP revision every night, so we leave that as a constant. This gives us a hook that generates a decision task every night.

We appropriate the following scopes for the hook in order for it to work properly.

assume:repo:hg.mozilla.org/try:*
project:releng:signing:cert:dep-signing
project:releng:signing:format:jar
queue:create-task:scriptworker-prov-v1/signing-linux-v1

The repo scope allows tasks to use the caches for the try server. The next three scopes deal with permissions that are required by the signing worker in order to sign binaries.

Hooks that rely on changing revisions are hard to debug. Because of this, I use a workflow that relies on information from your local environment. When you make modifications to any of the task configurations, you push the changes to try. I use a script to then figure out which revision to use based on the head revision in the local repository, update a task template, and launch the resulting task with a client-id that assumes the hook-id role.

Refer to documentation on how to launch task definitions locally if you would like to reproduce my local workflow.