@HackerFoo After each CI run on master/PR an artifact is produced with the dataframe.json. An example is here: https://storage.cloud.google.com/fpga-tool-perf/artifacts/prod/foss-fpga-tools/fpga-tool-perf/continuous/47/20200617-104132/github/fpga-tool-perf/build/dataframe.json

Sign in - Google Accounts

@acomodi Is there a good way to programmatically get the latest link? I'm guessing it's https://.../47/${date}-${hash-or-sequential-id}/.../dataframe.json, but I'm not sure how to get hash-or-sequential-id.

@HackerFoo I think that is a question for @mithro as I am unsure on how the kokoro instance does upload the artifacts on GCS. If there was the possibility to have a latest link, it would be great.

@HackerFoo Are the columns in the example sheet all the important metrics? Do the metrics you want to display vary between runs?

@TypingKoala those are a good start, but the design should be flexible enough to add more metrics if needed. Once we have a good overview of the most important metrics, we can create issues to track improvements (such as tracking trends over time.)

@HackerFoo I've created a proof-of-concept here that ingests from Hydra or GCS, cleans, normalizes, and outputs the information as a CSV that can be exported directly to Google Sheets. Is this along the lines of what you are looking for? Is there any color coding that would be helpful in the output?

https://colab.research.google.com/drive/1NXLYCYtY4YOslg0lbGryRfGdjKIeYZ1V?usp=sharing

https://docs.google.com/spreadsheets/d/1woQR-Ag2jQCDWpQE5ko89d5jdvkZswYSxb4DmHlBE5o/edit?usp=sharing

Google Colaboratory

Google Sheets - create and edit spreadsheets online, for free.

Create a new spreadsheet and edit with others at the same time -- from your computer, phone or tablet. Get stuff done with or without an internet connection. Use Sheets to edit Excel files. Free from Google.

@TypingKoala Could you embed a table into the Colab instead of generating CSV? Like this.

Color coding might be nice, with the vendor toolchain (e.g. Vivado) as the baseline, red for worse, green for better.

@HackerFoo I've implemented those changes. Please take a look at the colab now. I've added an embedded table with styling to reflect the relative performance compared to the Vivado toolchain for every project.

Development is in progress in a separate repo. Once issue #10 is resolved in that repo, this issue can be closed.