Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Travis-CI #1152

Closed
wants to merge 1 commit into from
Closed

Add Travis-CI #1152

wants to merge 1 commit into from

Conversation

zimbatm
Copy link
Member

@zimbatm zimbatm commented Dec 10, 2016

This is a work-in-progress so PRs can be tested for compilation success. Once that's working we can start thinking of adding static analysis tools like coverity to the mix.

Right now the issue is that Travis only provides GCC 4.8.4 and Clang 3.5.0. Both fail on the std::regex_replace usage as it's probably too new1. I think it's important for nix to be compatible with older C++ implementations as it's necessary to bootstrap on older linux distributions.

@zimbatm
Copy link
Member Author

zimbatm commented Dec 10, 2016

What is the best replacement for std::regexp_replace that is also backward-compatible?

@zimbatm
Copy link
Member Author

zimbatm commented Dec 10, 2016

boost::regexp might be a good replacement. http://stackoverflow.com/questions/8561850/compile-stdregex-iterator-with-gcc

@domenkozar
Copy link
Member

domenkozar commented Dec 11, 2016

We already have a travis PR, see #871 for discussion.

We also build Nix in a bunch of popular distributions: http://hydra.nixos.org/jobset/nix/master#tabs-jobs

@zimbatm
Copy link
Member Author

zimbatm commented Dec 11, 2016

Oops, I should have searched for it.

If we want to keep bootstrapping simple I think nix should work on normal distros out of the box. Right now it depends on a modern C++ compiler and Flex version that's not available in most Linux environments. I think that's a side-effect of being able to use the latest packages in nixpkgs. Having Travis-CI run the build against Ubuntu would help make sure we don't have that kind of regression and make adoption easier.

veprbl referenced this pull request Jan 17, 2017
Also, switch to C++14 for std::make_unique.
@zimbatm zimbatm changed the title WIP: Add Travis-CI Add Travis-CI Mar 4, 2017
@zimbatm
Copy link
Member Author

zimbatm commented Mar 4, 2017

This is good to be merged now. It supersedes #871 because it uses the Travis nix integration.

Having PRs automatically tested will help avoid having unnecessary regressions. Once we have Hydra PR tests this can be replaced.

@zimbatm zimbatm mentioned this pull request Mar 4, 2017
@domenkozar
Copy link
Member

@edolstra I would maintain this if you're ok with that.

@zimbatm zimbatm mentioned this pull request Mar 5, 2017
@gilligan
Copy link
Contributor

gilligan commented Nov 1, 2017

@edolstra i was wondering why there is no travis integration and then found this PR. What reason is there for not merging this ? The change is minimal, i can only think of advantages for having travis builds!

@edolstra
Copy link
Member

edolstra commented Nov 2, 2017

But I don't want to have to deal with multiple CI systems. One is more than enough.

@copumpkin
Copy link
Member

But Hydra still doesn't build PRs, and Travis does. It would probably help scale reviews on Nix if there were more automation helping reviews, since currently PRs can languish for months (and in doing so bitrot, which generally discourages further contributions)

@gilligan
Copy link
Contributor

gilligan commented Nov 2, 2017

I wanted to go through some of the issues and PRs to try to help triage issues and figure out which PRs with minimal but valuable changes might have been forgotten - I am convinced this would be much more useful if there was any feedback on whether or not the changes even compile at all.

It seems obvious to me that this should help with maintaining Nix? It means that those with merge/commit access don't have to clone/build/test everything locally. That definitely seems like a time saver to me.

What am I missing?

@zimbatm
Copy link
Member Author

zimbatm commented Nov 2, 2017

Fixed the merge conflict ;)

@bhipple
Copy link
Contributor

bhipple commented Dec 9, 2017

It would be really nice to get some very basic PR validation going for nix. Even some very basic sanity-checking would be nice to help prevent build regressions like this one, which have left master in a broken state until we merge the fix.

As @copumpkin and @gilligan mentioned, this could help make it easier to accept trivial improvements (like the above PR) with less risk of breaking master and less maintainer burden doing manual testing.

@zimbatm zimbatm force-pushed the add-travis branch 3 times, most recently from dc67f4e to 1df5ded Compare January 31, 2018 20:35
Copy link
Member

@peti peti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.

@zimbatm
Copy link
Member Author

zimbatm commented Feb 1, 2018

release.nix is using builtins.fetchGit which isn't supported in the nix version that is installed in Travis CI (1.11.16).

One thing we could do is to test the installer first and install the latest version of Nix.
Or avoid using too recent constructs to make bootstrapping easier.

@copumpkin
Copy link
Member

As I've suggested here and elsewhere, I think this will be a big win, especially when it comes to testing installers on macOS (#1739). But @edolstra even if I merged this, we'd still need you (as an owner) to flip the Travis CI switch on this repo. At the least, it seems to not cost much to try, and if it proves to be too noisy, we can adjust what it runs or turn it off. Does that seem okay?

@zimbatm zimbatm force-pushed the add-travis branch 3 times, most recently from bcd748e to 5b07878 Compare April 2, 2018 18:21
@zimbatm
Copy link
Member Author

zimbatm commented Apr 2, 2018

Travis CI has been updated to Nix 2.0 so I could drop the fetchGit changes. There are some remaining issues to fix before merging:

  • I don't know if the libstore patch is still necessary, can it be cherry-pick if yes?

Darwin:

geninfo: ERROR: need tool gcov!

Linux:

error: cloning builder process: Operation not permitted
error: unable to start build process

@dtzWill
Copy link
Member

dtzWill commented Apr 3, 2018

So I'm very new to Travis (especially w/Nix), but I believe adding sudo: required was sufficient to resolve problems enabling "sandbox" in the nix.conf used by my Travis CI.

There may be a better way, but this seems to do the trick for me.

The Travis CI documentation says to cc these folks regarding Nix support, so mentioning them here in case they have anything to add or can otherwise help:
@domenkozar @garbas and @matthewbauer (thanks guys!)

@matthewbauer
Copy link
Member

matthewbauer commented Apr 3, 2018

Interesting... Is the installer now doing multi-user Linux by default? The "Operation not permitted" looks like an issue with the travis machines:

https://docs.travis-ci.com/user/reference/overview/#Virtualisation-Environment-vs-Operating-System

When we put "sudo: false", it implies the newer container Travis. The builds are much faster with containers though obviously you can't use sudo. But maybe sandboxing does not work in a Travis container? I can look into what type of container system Travis uses but maybe it is also reproducible on other EC2 machines?

@dtzWill
Copy link
Member

dtzWill commented Apr 3, 2018

I don't believe sandbox works in typical containers, unless they are "privileged" in Docker terminology.

@matthewbauer
Copy link
Member

matthewbauer commented Apr 3, 2018

Ironic that you can't create a container in a container! We had a similar problem with the macOS sandbox (1888f78). Maybe Nix should be able to detect when this happens?

@zimbatm
Copy link
Member Author

zimbatm commented Apr 3, 2018

Okay so Linux is fixed and I still have the geninfo: ERROR: need tool gcov! error on Darwin.

Current build time: 33min

@zimbatm
Copy link
Member Author

zimbatm commented Jun 8, 2018

#2193 could have been avoided if the Travis CI installer test had been running

@zimbatm
Copy link
Member Author

zimbatm commented Jun 8, 2018

I'm happy to put more effort into this but only if it's going to be enabled

@copumpkin
Copy link
Member

I'm still in favor of this, but don't have the power to actually flip the switch to turn it on. @edolstra I think is the only one who can.

@zimbatm
Copy link
Member Author

zimbatm commented Jun 8, 2018

It's up to the @NixOS/nix-core team as a whole I think. Can we get a vote please?

@shlevy
Copy link
Member

shlevy commented Jun 8, 2018

Before a vote, let's figure out who on the team wants to participate in the discussion and make sure they've all had a chance to hash out their perspective, so we can try for consensus first. @NixOS/nix-core can you 👍 this comment if you're interested in participating and, if need be, voting?

@shlevy
Copy link
Member

shlevy commented Jun 8, 2018

As for me: I think we definitely should have CI for PRs. I'm sympathetic to the idea of not using multiple CI systems, but hydra has had the basic capability to support building PRs for quite some time now and it's not being used so I'm kind of skeptical we'll see that in action soon. Given that, if Travis is able to give us reliable CI without false negatives or significant ongoing maintenance then I think we should go ahead with this, and if we want to move to something else later we can do so when it's implemented. I'm happy to trust @zimbatm on the technical question of whether Travis will be reliable enough.

@peti
Copy link
Member

peti commented Jun 9, 2018

I can't think of any downsides to using travis-ci, especially considering that @zimbatm has already said that he'd be willing to maintain the CI build scripts.

@copumpkin
Copy link
Member

I also think Travis has something to offer in excess of what Hydra can do for us right now, which is to give us pristine boxes of different OSes (including macOS!) upon which to test the Nix installer. Running the test suite is great, but being able to automatically test the installers I think would really take some of the pain out of improving them.

@shlevy
Copy link
Member

shlevy commented Jun 23, 2018

OK, looks like the three of us currently engaged are in favor of this... @edolstra do you want to chime in?

Improve the test coverage during PR
@zimbatm
Copy link
Member Author

zimbatm commented Jul 9, 2018

Just rebased and now it's failing on both platforms with:

src/libexpr/eval.cc:1728:5: error: unknown type name 'GC_prof_stats_s'
    GC_prof_stats_s gc;
    ^
1 error generated.
make: *** [mk/patterns.mk:3: src/libexpr/eval.o] Error 1

This demonstrates pretty well what happens when there is no CI in place

@edolstra
Copy link
Member

edolstra commented Jul 9, 2018

We don't need Travis for that, because Hydra is showing the exact same problem: https://hydra.nixos.org/build/77124123

@copumpkin
Copy link
Member

But as we've said before, it isn't checking PRs (which it could, but it isn't) or testing the macOS installer

@dtzWill
Copy link
Member

dtzWill commented Jul 9, 2018

Hydra does more than I think is generally perceived, perhaps badges would help increase visibility for this sort of thing?

Also this particular error is addressed by #2282 .
(Although indirectly since I think the real problem is we're not providing libgc to the builds? Investigating presently...)
(EDIT: nevermind, although insights as to why we don't build with GC for ubuntu/fedora would be appreciated)

@zimbatm zimbatm closed this Aug 13, 2019
@zimbatm zimbatm deleted the add-travis branch August 13, 2019 07:35
@zimbatm
Copy link
Member Author

zimbatm commented Aug 13, 2019

Not interested in pursuing that further.

Hydra is able to build pull-requests so that's probably a better route to take.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

10 participants