Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added compile progress tracker #4182

Merged
merged 2 commits into from May 19, 2017
Merged

Conversation

RX14
Copy link
Contributor

@RX14 RX14 commented Mar 24, 2017

This PR adds a compile progress indicator to crystal. A new ProgressTracker class has been added which controls both stats and progress options. This is WIP, as there are a few things left to figure out such as the current assumption of the number of compiler stages run.

Demo:

@kostya
Copy link
Contributor

kostya commented Mar 24, 2017

just tested, this is nice, but in release mode longest is: [12/13] [0/1] Codegen (bc+obj) - 5 minutes, is there possible to create progress bar for it? may be even approximated, based on main obj file size.

@makenowjust
Copy link
Contributor

It is great feature, however I would like to disable it by default because the compiler is quiet a long time ago. For instance, gcc (and I wish other compilers too) says nothing without warnings and errors on the default. This opinion comes from just a tradition, and I don't consider a usefulness for now. I agree of it to be enabled on shards by default because a build system is noisy a long time ago.

@bew
Copy link
Contributor

bew commented Mar 24, 2017

I think this should be enable on --verbose and when the stdout is an interactive tty

@refi64
Copy link
Contributor

refi64 commented Mar 24, 2017

Oooh, nice!

@RX14
Copy link
Contributor Author

RX14 commented Apr 10, 2017

@asterite Would you consider adding a progress tracker to crystal? If so, when should it be displayed?

@asterite
Copy link
Member

@RX14 It does look right, but usually compilers and command line tools don't emit progress output unless requested.

I think the compiler should be faster so that a progress bar is not needed.

@RX14
Copy link
Contributor Author

RX14 commented Apr 11, 2017

@asterite I'll change it to default to progress off.

Regarding neccesity, I doubt LLVM is going to get 10x faster, and LLVM makes up >50% of compile times for new builds (more for release, but progress tracking isn't useful there). While progress hopefully won't be necessary for incremental builds, it will be neccesary for initial builds I think. Regardless, this PR cleans up the statistics code a bit, and I don't think a progress indicator can be harmful.

@@ -480,8 +478,8 @@ class Crystal::Command
opts.on("-s", "--stats", "Enable statistics output") do
compiler.progress_tracker.stats = true
end
opts.on("--no-progress", "Disable progress output") do
compiler.progress_tracker.progress = false
opts.on("-p", "--progress", "Enable progress output") do
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only other crystal command using -p is crystal play (for port), which doesn't take compiler options, so this is safe.

Copy link
Contributor Author

@RX14 RX14 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've updated this PR so that progress is off by default, and added a progress option to the makefile.

@RX14
Copy link
Contributor Author

RX14 commented Apr 14, 2017

Looks like a weird failure from osx.

@RX14
Copy link
Contributor Author

RX14 commented Apr 21, 2017

Would have been nice to have this in 0.22.0, is there anything left to do?

@RX14
Copy link
Contributor Author

RX14 commented May 7, 2017

Bump again :/
I'm sad I feel I have to do this. It's hard as a pull request author not knowing what the status of your PR is.

@bararchy
Copy link
Contributor

bararchy commented May 7, 2017

@RX14 Nice work :)
It's a really cool feature

@mverzilli
Copy link

@RX14 sorry for the delay with this. I think most of us simply assumed this opinion was enough feedback:

#4182 (comment)

Personally, I don't see any benefit to running with --progress as opposed to --stats.
In which scenario would you prefer running with --progress over --stats?

@RX14
Copy link
Contributor Author

RX14 commented May 9, 2017

@mverzilli --progress has a cleaner output than --stats, with less profiling info (memory, exact times) and more progress info. For example, --progress shows the total number of stages before completion, whereas --stats does not. --progress also shows progress through the LLVM stage. It also updates the progress on the current line so as to use a single line of output instead of multiple, this makes your console history less messy as you don't see the progress output after the completion of the compiler.

Overall, --progress is useful for seeing progress while compiling, whereas --stats is useful for profiling the compiler.

@mverzilli
Copy link

I get the differences but still, if you want to see progress while compiling, --stats gets the job done.

Maybe we could add the total number of stages before completion to --stats, to give us a better idea of where we're standing?

Anyway, just my opinion!

@RX14
Copy link
Contributor Author

RX14 commented May 9, 2017

@mverzilli --stats gets the job done (this PR also refactors stats to be cleaner), but it's not that much extra code to add this, and I can't see the maintenance cost being all too high either. There seems to be a lot of support for this (8 hearts!) and you can continue to use --stats. Why not merge it?

@refi64
Copy link
Contributor

refi64 commented May 9, 2017

There seems to be a lot of support for this (8 hearts!)

Make that nine hearts!!

Personally, I would love this feature. --stats's output is kind of ugly for normal use, and, although most compilers are silent, usually I enjoy seeing the progress (helps a bit with my lack of patience!).

+1000000000 from me!

@bew
Copy link
Contributor

bew commented May 9, 2017

I'd simply say that --progress should stay simple (one-line, show progress only, no internal datas like memory or anything), so impatients like @kirbyfan64 (:smile:) will be able to keep track of the progression of the compilation for normal use, without having an ugly output (I agree it's really opinionated).

And --stat should be a "debug" for the compilation, with internal datas output (time, memory usage.. and there is room for other things too!), for thoses who wants precise informations about each compilation steps..

WDYT?

@RX14
Copy link
Contributor Author

RX14 commented May 18, 2017

A lot of good arguments for this feature have been already covered, but I just want to point out that it's an easy feature to remove (it has no API surface) if for whatever reason it turns out to be a bad idea.

@spalladino
Copy link
Contributor

Given all the thumbs up for the feature, and since it is not enabled by default, and, as RX14 points out, it is easy to remove should it cause any troubles, I say we merge.

@mverzilli mverzilli added this to the Next milestone May 26, 2017
@asterite
Copy link
Member

asterite commented Jun 3, 2017

@RX14 This broke the current stats. Previously it would show, for example, "Codegen (crystal)" and then after some time print the time. This was useful to know in which stage the compiler were. But right now that's lost (it's printed all at once). Could you please fix it? :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

9 participants