Clock signals routed through INT tiles #1153

acomodi · 2019-11-14T14:49:21Z

As explained in this comment, VPR is able to route clock signals through INT tiles, violating the clock regions boundaries. This can result in non-working bitstreams, as well as an impossibility for Vivado to correctly place and route the designs produced with fasm2bels.

Failure example of Vivado (got from https://source.cloud.google.com/results/invocations/ab9206f5-759f-4c0f-adec-69dc24e943e1/targets/foss-fpga-tools%2Fsymbiflow-arch-defs%2Fpresubmit%2Fxc7_vendor/log):

ERROR: [Place 30-123] Unroutable Placement! A BUFHCE / MMCM component pair is not placed in a routable site pair. The BUFHCE component can use the dedicated path between the BUFHCE and the MMCM if both are placed in the same clock region. If this sub optimal condition is acceptable for this design, you may use the CLOCK_DEDICATED_ROUTE constraint in the .xdc file to demote this message to a WARNING. However, the use of this override is highly discouraged. These examples can be used directly in the .xdc file to override this clock rule.
	< set_property CLOCK_DEDICATED_ROUTE FALSE [get_nets CLK_HROW_TOP_R_X60Y78_BUFHCE_X0Y21_O] >

	CLK_HROW_TOP_R_X60Y78_BUFHCE_X0Y21_BUFHCE (BUFHCE.O) is locked to BUFHCE_X0Y21
	CLK_BUFG_BOT_R_X60Y48_BUFGCTRL_X0Y1_BUFGCTRL (BUFGCTRL.I0) is locked to BUFGCTRL_X0Y1

	The above error could possibly be related to other connected instances. Following is a list of
	all the related clock rules and their respective instances.

	Clock Rule: rule_bufh_bufr_ramb
	Status: PASS
	Rule Description: Reginal buffers in the same clock region must drive a total number of brams less
	than the capacity of the region
	CLK_HROW_TOP_R_X60Y78_BUFHCE_X0Y22_BUFHCE (BUFHCE.O) is locked to BUFHCE_X0Y22
...

This happens because some clocks traverse the clock regions through the INT tiles, instead of using the dedicated clock net.

Another behavior I have seen is that the clock source driven from the W5 IOPAD is not being routed through the HCLK row, but reaches the BUFG through a series of jumps between INT tiles as well.

The text was updated successfully, but these errors were encountered:

litghost · 2019-11-14T18:05:39Z

VPR has a two phase router that is an open PR here: verilog-to-routing/vtr-verilog-to-routing#928

mkurc-ant · 2019-11-19T16:57:27Z

I've encountered the same problem with PLL. I think that it may be a good idea to add to fasm2bels setting the `CLOCK_DEDICATED_ROUTE' to FALSE on those nets somehow. Won't solve the routing problem but will ease debugging.

litghost · 2019-11-19T17:00:01Z

The PLL has a simplier explain, in that it was missing some BUFG instances, see: 5852cd2

One thing to note is that right now BUFG must be instanced explicitly. Vivado has implicit BUFG placement enabled, but we have it disabled for now.

acomodi · 2019-12-06T12:01:58Z

@litghost Two stage clock routing has been merged

acomodi · 2020-01-08T13:56:18Z

I could reproduce locally the issues related to the Vendor Tool Tests CI failure. The test though is the DDR litex one.

In the figure above, the highlighted wire (white) is the output of the BUFHCE_X1Y20.
The horizontal purple line, instead, divides the fabric in two clock regions. We can see that the highlighted clock is illegally crossing the clock region through the INT tiles, resulting in the error listed below:

ERROR: [Place 30-176] Unroutable Placement! The following clock source instance is driving the following locked load instances. The clock source instance is placed too far away from its load instance to be routable.
        CLK_HROW_TOP_R_X60Y78_BUFHCE_X1Y20_BUFHCE (BUFHCE.O) is locked to BUFHCE_X1Y20
        The loads are distributed to 1 user pblock constraints. In addition, there are 20 loads not in user pblock constraints.

        Displaying only the first 20 or fewer instances under each constraint as list of loads is too long

        Displaying first 20 loads not in user pblock constraint:
        $auto$simplemap.cc:442:simplemap_dffe$22079CLBLM_L_X32Y60_SLICE_X50Y60_D5_FDRE (FDRE.C) is locked to SLICE_X50Y60
        CLBLM_L_X32Y60_SLICE_X50Y60_B_FDRE (FDRE.C) is locked to SLICE_X50Y60
        $auto$simplemap.cc:442:simplemap_dffe$22078CLBLM_L_X32Y60_SLICE_X50Y60_A_FDRE (FDRE.C) is locked to SLICE_X50Y60
        $auto$simplemap.cc:442:simplemap_dffe$23899CLBLL_L_X28Y99_SLICE_X45Y99_C5_FDRE (FDRE.C) is locked to SLICE_X45Y99
        CLBLM_R_X29Y56_SLICE_X46Y56_C5_FDRE (FDRE.C) is locked to SLICE_X46Y56
        CLBLM_L_X32Y56_SLICE_X50Y56_RAM64M/RAMD (RAMD64E.CLK) is locked to SLICE_X50Y56
        BRAM_L_X30Y95_RAMB36_X1Y19_RAMB36E1 (RAMB36E1.CLKARDCLK) is locked to RAMB36_X1Y19
        CLBLM_R_X27Y99_SLICE_X42Y99_D_FDRE (FDRE.C) is locked to SLICE_X42Y99
...

This is unrelated to the placement of the BUFG or PLL, as it interests only the routing phase, as the router chooses a wrong BUFH output to drive some FF.

I suppose that a temporary fix to this issue could be to add an increased cost to those wires that can get the CLK signal to get out of the INT tile and jump across non-dedicated routes. Namely wires that, for instance, connect the GCLK ppip (of the INT tile) to GFAN (of the same INT tile). This way, VPR should be able to choose the dedicated clock routes over the non-dedicated routes.

Does this make sense @mkurc-ant @litghost ?

acomodi · 2020-01-08T14:27:20Z

I think that a possible way to add the cost of the wrong paths, is to add yet another switch, opposite to the delayless one, as it will have a higher fake delay, that will act as a blocker for VPR to use a route instead of another.

litghost · 2020-01-08T15:10:38Z

Can you add the bitstream fasm, fasm2bels vivado verilog, fasm2bels vivado tcl, and vivado checkpoint (dcp) in a zip?

acomodi · 2020-01-08T15:41:15Z

@litghost Sure, here it is:
https://drive.google.com/open?id=1b_YNFKU3fng0JHvjl6OOgVvHS-hS12TI

Meet Google Drive – One place for all your files
Google Drive is a free way to keep your files backed up and easy to reach from any phone, tablet, or computer. Start with 15GB of Google storage – free.

litghost · 2020-01-08T18:25:40Z

So the bad route is:

[list INT_L_X32Y50/GCLK_L_B9_EAST [list INT_R_X33Y50/GFAN0 INT_R_X33Y50/FAN_ALT0 INT_R_X33Y50/FAN_BOUNCE0 INT_R_X33Y49/FAN_ALT3 INT_R_X33Y49/FAN_BOUNCE3 INT_R_X33Y49/FAN_ALT7 INT_R_X33Y49/FAN_BOUNCE7 INT_R_X33Y49/FAN_ALT4 INT_R_X33Y49/FAN_BOUNCE4 INT_R_X33Y48/FAN_ALT1 INT_R_X33Y48/FAN_BOUNCE1 INT_R_X33Y48/FAN_ALT5 INT_R_X33Y48/FAN_BOUNCE5 [list INT_R_X33Y48/FAN_ALT2 INT_R_X33Y48/FAN_BOUNCE2 INT_R_X33Y47/FAN_ALT3 INT_R_X33Y47/FAN_BOUNCE3 INT_R_X33Y47/FAN_ALT7 INT_R_X33Y47/FAN_BOUNCE7 INT_R_X33Y47/FAN_ALT4 INT_R_X33Y47/FAN_BOUNCE4 INT_R_X33Y46/FAN_ALT1 INT_R_X33Y46/FAN_BOUNCE1 INT_R_X33Y46/FAN_ALT5 INT_R_X33Y46/FAN_BOUNCE5 INT_R_X33Y46/CLK1 CLBLM_R_X33Y46/CLBLM_M_CLK  {} ]

This particular route is actually illegal if the relevant FAN_ALTx are not consuming the signal. If we fix #1019 , then this particular issue will go away.

The fix is suggested here: #1019 (comment)

I've started the first part of the fix here: #1151

But that PR is stale, and I believe I had to modify VPR to make it work. @acomodi, if you make that PR work, then fixing part 2 from #1019 (comment) is pretty straight forward, and would fix this issue and fix another fasm2bels failure we've been getting intermittently.

acomodi · 2020-01-10T10:54:38Z

Ok, I will look into that and get the PR to work.
I have tried to test it locally and got a VPR error related to a Cinternal issue. Probably the VPR fix was related to that.

litghost · 2020-01-13T21:22:20Z

@acomodi FYI verilog-to-routing/vtr-verilog-to-routing#1044

acomodi · 2020-01-14T11:14:38Z

I am worried that this patch is not enough (and also disabling check route). I get past the check of the rr_graph, but the problem is now that an assertion fails during rr_graph load:

/vtr-symbiflow/vpr/src/route/rr_graph.cpp:3209 create_sets: Assertion 'node_to_node_set[edge.first] == node_to_node_set[edge.second]' failed.

I have tried disabling that, to see how far it would get, but apparently it is not enough as I encountered an issue during routing:

---- ------ ------- ---- ------- ------- ------- ----------------- --------------- -------- ---------- ---------- ---------- ---------- --------
Iter   Time    pres  BBs    Heap  Re-Rtd  Re-Rtd Overused RR Nodes      Wirelength      CPD       sTNS       sWNS       hTNS       hWNS Est Succ
      (sec)     fac Updt    push    Nets   Conns                                       (ns)       (ns)       (ns)       (ns)       (ns)     Iter
---- ------ ------- ---- ------- ------- ------- ----------------- --------------- -------- ---------- ---------- ---------- ---------- --------
Warning 122: No routing path for connection to sink_rr 449652, retrying with full device bounding box
Cannot route from BLK-TL-CLK_BUFG_BOT_R.CLK_BUFG_R_BUFGCTRL10_CE0[0] (RR node: 2019486 type: SOURCE location: (79,111) class: 96 capacity: 1) to BLK-TL-SLICEL.CLK[0] (RR node: 449652 type: SINK location: (54,45) class: 44 capacity: 1) -- no possible path
Failed to route:
.../vtr-symbiflow/vpr/src/route/route_common.cpp:1290 print_route: Assertion 'itr != route_ctx.rr_net_map.end()' failed.

HackerFoo · 2020-01-15T21:48:52Z

I think this command should find the problematic clock connections in the .fasm:

grep -E 'GFAN.*\.GCLK|CLK.*\.FAN_BOUNCE' top.bit.fasm

HackerFoo · 2020-01-17T01:52:19Z

murax_basys_full also has GCLK -> GFAN & FAN -> CLK connections in toplevel.genfasm.fasm.

HackerFoo · 2020-01-21T18:31:15Z

I think the regex should be just GFAN.*\.GCLK, which only finds minilitex_arty.

make all_xc7
find build -iname \*.fasm | xargs grep -E 'GFAN.*\.GCLK' -l

litghost · 2020-03-02T15:07:10Z

I believe this issue has been resolved with:

acomodi mentioned this issue Nov 15, 2019

Add OSERDES support #1102

Merged

acomodi mentioned this issue Dec 4, 2019

Flaky xc7 vendor tests #1196

Closed

acomodi mentioned this issue Dec 23, 2019

xc7: added RL_IOI_SING tile pips correct fasm_feature #1242

Merged

acomodi mentioned this issue Jan 17, 2020

Add penalty cost switch to avoid wrong routes #1267

Merged

litghost closed this as completed Mar 2, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Clock signals routed through INT tiles #1153

Clock signals routed through INT tiles #1153

acomodi commented Nov 14, 2019

litghost commented Nov 14, 2019

mkurc-ant commented Nov 19, 2019

litghost commented Nov 19, 2019

acomodi commented Dec 6, 2019

acomodi commented Jan 8, 2020

acomodi commented Jan 8, 2020 •

edited

litghost commented Jan 8, 2020 •

edited

acomodi commented Jan 8, 2020 •

edited by unfurl-links bot

litghost commented Jan 8, 2020

acomodi commented Jan 10, 2020

litghost commented Jan 13, 2020

acomodi commented Jan 14, 2020

HackerFoo commented Jan 15, 2020

HackerFoo commented Jan 17, 2020

HackerFoo commented Jan 21, 2020 •

edited

litghost commented Mar 2, 2020

Clock signals routed through INT tiles #1153

Clock signals routed through INT tiles #1153

Comments

acomodi commented Nov 14, 2019

litghost commented Nov 14, 2019

mkurc-ant commented Nov 19, 2019

litghost commented Nov 19, 2019

acomodi commented Dec 6, 2019

acomodi commented Jan 8, 2020

acomodi commented Jan 8, 2020 • edited

litghost commented Jan 8, 2020 • edited

acomodi commented Jan 8, 2020 • edited by unfurl-links bot

litghost commented Jan 8, 2020

acomodi commented Jan 10, 2020

litghost commented Jan 13, 2020

acomodi commented Jan 14, 2020

HackerFoo commented Jan 15, 2020

HackerFoo commented Jan 17, 2020

HackerFoo commented Jan 21, 2020 • edited

litghost commented Mar 2, 2020

acomodi commented Jan 8, 2020 •

edited

litghost commented Jan 8, 2020 •

edited

acomodi commented Jan 8, 2020 •

edited by unfurl-links bot

HackerFoo commented Jan 21, 2020 •

edited