We have a https://github.com/SymbiFlow/fpga-tool-perf which is a tool for understanding the performance of various toolchains, where performance means multiple things;

• If a benchmark actually works
• The resulting highest fmax achievable for the design
• Resources taken (like CPU + memory) to produce the bitstream
• etc

The focus is on real world designs (for example LiteX SoCs, PicoSoC and OpenTitan) that people are using rather than synthetic benchmarks and wants to make sure that the output result could be usable.

We have a Google doc at https://j.mp/fpga-tool-perf-spec which includes various information about what we are targetting. Currently it is mostly targeted at the Xilinx 7 series as that is the flow we have the most options to compare around (We have 3 options for place around! Vivado, nextpnr and vpr!).

It would be nice to have support for ECP5, iCE40 and anything else in the future too but they are not a high priority.

GitHub

SymbiFlow/fpga-tool-perf

FPGA tool performance profiling. Contribute to SymbiFlow/fpga-tool-perf development by creating an account on GitHub.

Google Docs

SymbiFlow FPGA Tool Performance (Xilinx Performance)

SymbiFlow FPGA Tool Performance (for Xilinx Devices) This doc → https://j.mp/fpga-tool-perf-spec GitHub repo → https://github.com/SymbiFlow/fpga-tool-perf Modes Fixed fmax -- Looking at run time and memory usage Find fastest fmax -- Try and find the fastest possible Toolchains Synthe...

If you are more interested in benchmarks which aim to exercise a CPU (IE The fact the benchmark is an FPGA toolchain is mostly inconsequential) Verilog to Routing also has a pretty CPU intensive benchmarks. See https://docs.verilogtorouting.org/en/latest/tutorials/titan_benchmarks/ -- these range from O(10 minutes) to O(2 days) run time. See also https://docs.verilogtorouting.org/en/latest/dev/developing/#running-tests and https://docs.verilogtorouting.org/en/latest/dev/developing/#evaluating-quality-of-result-qor-changes

Running the Titan Benchmarks — Verilog-to-Routing 8.1.0-dev documentation

Commit Procedures — Verilog-to-Routing 8.1.0-dev documentation

Commit Procedures — Verilog-to-Routing 8.1.0-dev documentation

Sweet! Thanks for the resources. Didn't know some of this stuff already existed.