Any feedback on this one? I am happy to pursue a different approach if this change is not wanted / should be implemented in a different way.

I like this!

Can this produce JSON like api in #440? If not, do you know how difficult it would be to support that?

Hardly a barrier for entry but such an endpoint seems especially compelling when talking about thousands of builds -- many more than a human would ever issue requests for individually.

@dtzWill yes the format should be the same as that API. It should return exactly the same as querying the builds of the evaluation and then getting the build details for each build with the JSON api. This new endpoint is only necessary to avoid making 40k requests to get the info for a single evaluation.

If this would result in too much load on the hydra server, we could also provide a cached version of this is part of a channel (just like store-paths)

cc @edolstra

@bennofs mhm, if we cannot get this in, maybe a travis cron job in nix-index would do.

Uploading built index files to the github release tab is not too hard: https://github.com/Mic92/cntr/blob/master/.travis.yml#L39

The way this works right now is not wonderful for hydra.nixos.org, taking almost 9 minutes and producing 80M of data:

[grahamc@Morbo:~]$ curl --header "Accept:application/json" https://hydra.nixos.org/eval/.../builds > latest-eval-builds
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 75.9M  100 75.9M    0     0   149k      0  0:08:40  0:08:40 --:--:-- 8008k

I'm not sure what to do about that, but just for what its worth.

Also the data includes magic numbers, like build status numbers: https://github.com/NixOS/hydra/blob/master/src/sql/hydra.sql#L202-L215

What information is actually needed from builds for nix-index to work?
Compression would also save 90%

$ du latest-eval-builds.xz
7.0M    latest-eval-builds.xz

I'm not sure what to do about that, but just for what its worth.

The information is very similar to other queries that are fast. Running this with debug produces an extremely long and convoluted output. I suspect this can be orders of magnitude faster with a purpose built query.