cc @jgraham

We are certain that Chrome is failing to start in Docker:

Failed to move to new namespace: PID namespaces supported, Network namespace supported, but failed: errno = Operation not permitted

which is a rather common problem, c.f. Puppeteer troubleshooting doc.

We have a few workarounds on the Docker side as seen here:

Run the container with: --privileged
Run with: --cap-add SYS_ADMIN
Run with: --security-opt seccomp:unconfined
Run with: --security-opt seccomp:chrome.json

Our understanding is that they have increasing security levels. The first one is easy to do on Community-TC, but particularly bad security-wise, because it gives Docker containers RW access to /sys and /dev on the host, allowing a container to pollute the host easily (e.g. using sysctl or dd to /dev). The hosts are NOT reset between tests on Taskcluster. A good long-term solution is to use a custom seccomp profile, which may require building our own worker image as it is not configurable otherwise, and currently there is no self served way to build a worker image.

Alternatively, we could disable Chrome sandboxing. This is in fact better security-wise: we have password-less sudo inside Docker, so anyone can get root in a PR already. By disabling Chrome sandboxing, we are not loosening the container isolation at all, so we are not opening up new vulnerabilities. However, it is unclear whether it'd affect web-observable behaviours (I'm checking with the Chrome team). If not, I think this might be the best tradeoff.

None of the aforementioned workarounds were used previously. We are not sure why it used to work, but have a theory that a different sandboxing mechanism was used because of old kernel version.

On the old taskcluster.net instance, github-worker runs a 14.04-based image. According to Chromium docs, user namespace sandbox is only used on kernel >= 3.8. Yet 14.04 shipped 3.13, so we are not sure...

We have two immediate fix candidates:

• [taskcluster] Use priviledged mode on Community-TC #20134 turn on priviledged mode
• [taskcluster] Disable Chrome sandboxing in Docker #20136 disable Chrome sandboxing

We believe that bc83451 should resolve this, by disabling the Chrome sandbox when running under the new taskcluster instances. Long term, we should investigate adding support to TaskCluster for custom seccomp profiles, but hopefully this puts out the fire for now.

🔥 <-- 🚒