Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve out-of-memory error during fasm generation (seen in nexys_video counter_test example) #119

Open
tcal-x opened this issue Jan 24, 2021 · 7 comments
Open

Improve out-of-memory error during fasm generation (seen in nexys_video counter_test example) #119

tcal-x opened this issue Jan 24, 2021 · 7 comments
Assignees
@tcal-x

Comments

@tcal-x
Copy link
Contributor

tcal-x commented Jan 24, 2021 •
edited

My laptop has 10GB available according to /proc/meminfo. I see this error running the counter_test example with TARGET=nexys_video:

Serial number (magic cookie) for the routing is: 416094740
Circuit successfully routed with a channel width factor of 500.
Incr Slack updates 1 in 1.4106e-05 sec
Full Max Req/Worst Slack updates 1 in 4.114e-06 sec
Incr Max Req/Worst Slack updates 0 in 0 sec
Incr Criticality updates 0 in 0 sec
Full Criticality updates 1 in 1.8798e-05 sec
Writing Implementation FASM: top.fasm
/media/tim/GIT/opt/symbiflow/xc7/install/bin/vpr_common: line 116: 2752047 Killed                  genfasm ${ARCH_DEF} ${EBLIF} --device ${DEVICE_NAME} ${VPR_OPTIONS} --read_rr_graph ${RR_GRAPH} $@
make: *** [Makefile:54: build/nexys_video/top.fasm] Error 137

The file build/nexyx_video/top.fasm is left behind. Is it partially written? If I run the make command again, top.bit is generated from top.fasm. Is the top.bit file correct?

I have two concerns here:

  • The user should get a clear "out of memory" error if that is indeed the case.
  • When an error is encountered while generating top.fasm, that file should be removed, or else a subsequent make will use it and possibly write an incorrect top.bit file.

Note: If I close other applications to get 15GM of available memory, the flow completes without error, although I haven't had a chance to test the generated bitstream on the Nexys Video board yet.
@mithro
Copy link
Contributor

mithro commented Jan 25, 2021 •
edited

@litghost - Ideas?

@mithro
Copy link
Contributor

mithro commented Jan 25, 2021

Makefiles do have a remove file on failure option?

@litghost
Copy link
Contributor

litghost commented Jan 25, 2021 •
edited

Makefiles do have a remove file on failure option?

We need to add .DELETE_ON_ERROR to the makefile. @tcal-x, can you please retest your case with .DELETE_ON_ERROR in the makefile?

@litghost litghost assigned tcal-x and unassigned litghost Jan 25, 2021
@litghost
Copy link
Contributor

litghost commented Jan 25, 2021 •
edited by unfurl-links bot

Link to relevant documentation: https://www.gnu.org/software/make/manual/html_node/Errors.html#Errors

Errors (GNU make)

@tcal-x
Copy link
Contributor Author

tcal-x commented Jan 25, 2021

@litghost , yep, that works. I'll open a PR with the change.

I could also add a message on failure, "If you see 'Killed...' above, the process may have been unable to allocate sufficient memory".

tcal-x added a commit to tcal-x/symbiflow-examples that referenced this issue Jan 25, 2021
@tcal-x
Add .DELETE_ON_ERROR to Makefiles (issue chipsalliance#119).
fd078c9 
Signed-off-by: Tim Callahan <tcal@google.com>
tcal-x added a commit that referenced this issue Jan 26, 2021
@tcal-x
Merge pull request #121 from tcal-x/del_on_fail
b0ee6d2 
Add .DELETE_ON_ERROR to Makefiles (issue #119).
@andrewb1999
Copy link

Just as an FYI, I run into this same issue in yosys when running very large designs and I have another memory intensive application open in the background. It would be helpful to add some warning there as well.

@tcal-x
Copy link
Contributor Author

tcal-x commented Apr 15, 2021

@kgugala @mithro I'd like to add this to the cleanup list. It still dies with just "Error 137", no mention of out-of-memory. This is running the counter example; I have 11GB free but it's not enough:

Serial number (magic cookie) for the routing is: 481366680
Circuit successfully routed with a channel width factor of 500.
Incr Slack updates 1 in 1.5997e-05 sec
Full Max Req/Worst Slack updates 1 in 5.446e-06 sec
Incr Max Req/Worst Slack updates 0 in 0 sec
Incr Criticality updates 0 in 0 sec
Full Criticality updates 1 in 1.992e-05 sec
Writing Implementation FASM: top.fasm
/media/tim/GIT/opt/sf2/xc7/install/bin/vpr_common: line 116: 1737994 Killed                  genfasm ${ARCH_DEF} ${EBLIF} --device ${DEVICE_NAME} ${VPR_OPTIONS} --read_rr_graph ${RR_GRAPH} $@
make: *** [Makefile:57: build/nexys_video/top.fasm] Error 137
make: *** Deleting file 'build/nexys_video/top.fasm'

@tcal-x tcal-x mentioned this issue May 16, 2021
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tcal-x tcal-x

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@mithro @litghost @andrewb1999 @tcal-x