I've spent the day profiling the rr graph load and identified a couple low hanging fruit. Around 40 seconds is wasted doing the following:

• ~12 seconds in check rr graph, which generates 4 millions lines of warnings we ignore
• ~27 in process edges, the majority of which is spent dealing with edge metadata which is only used during genfasm, so it can be safely avoid

After commenting out check_rr_graph and disabling the reading of edge metadata, the rr graph creation time dropped ~24 seconds. The vast majority of that time was spent in pugixml. Based on some experiments, those 24 seconds will become ~2-3 seconds if we switch to using the mmap based capnp rr graph.

In addition, if a mmap based capnp rr graph is used, the edge metadata could be proxied to from the capnp rr graph, avoiding the double memory usage and memcpy for the edge metadata strings.

FYI - @duck2

Disabling the reading of metadata except for genfasm is a good idea, given that it's not used in the other stages of vpr.

I'm not sure how to get rid of copying the capnp structure into VPR, though. Can we copy that much data in 2-3 seconds? Maybe a RRGraph object-like structure but with accessors to capnp instead of a create_rr_graph?

For genfasm, as we have discussed in IRC, optimizing the std::unordered_map<std::string, std::vector<std::string>> will go a long way.

First cut implementation of the uxsdcxx capnp based reader:

## Loading routing resource graph

Warning 134: This architecture version is for VPR 8.0.0-rc1+3020d7146-dirty while your current VPR version is 8.0.0-rc1+92e57c2d0 compatability issues may arise

## Loading routing resource graph took 9.34 seconds (max_rss 3563.2 MiB, delta_rss +3469.8 MiB)
  RR Graph Nodes: 3539499
  RR Graph Edges: 22966465
# Create Device took 9.88 seconds (max_rss 3563.2 MiB, delta_rss +3469.8 MiB)

versus the uxsdcxx XML based reader:

## Loading routing resource graph

Warning 134: This architecture version is for VPR 8.0.0-rc1+3020d7146-dirty while your current VPR version is 8.0.0-rc1+92e57c2d0 compatability issues may arise

## Loading routing resource graph took 18.58 seconds (max_rss 7307.4 MiB, delta_rss +7216.5 MiB)
  RR Graph Nodes: 3539499
  RR Graph Edges: 22966465
# Create Device took 19.12 seconds (max_rss 7307.4 MiB, delta_rss +7216.5 MiB)

So not only was the capnp reader faster, but it consumes half the memory! Not bad at all!

9 seconds for the whole 50T graph is impressive! I think this can be made faster if we pool the edges instead of allocating a new unique_ptr for each node.edges.

9 seconds for the whole 50T graph is impressive! I think this can be made faster if we pool the edges instead of allocating a new unique_ptr for each node.edges.

I had a similar thought. However, the next step is to complete the integration of the reader into the master+wip, along with a XML -> capnp writer tool, so that the improved memory usage and runtime can be used.

The changes metioned in this issue have been commited!