Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

JRuby 9000 + TravisCI + Coveralls Erroneous Failure #3256

Closed
jdantonio opened this issue Aug 15, 2015 · 5 comments
Closed

JRuby 9000 + TravisCI + Coveralls Erroneous Failure #3256

jdantonio opened this issue Aug 15, 2015 · 5 comments
Labels
JRuby 9000
Milestone
Non-Release

Comments

@jdantonio
Copy link

When running RSpec tests on Travis CI, JRuby 9000 will report 0.0% Coveralls code coverage. This causes Travis to register a failing build. An example can be seen here. This seems to have started happening this weekend.
@rtyler
Copy link

rtyler commented Aug 17, 2015

@jdantonio tangential to this bug report, in the future it's useful to gist some build output or embed the build output and relevant details into the GitHub issue itself.

When triaging some old bugs (>1 year) weekend before last I ran into a lot of Travis build links that had since expired, which was most frustrating.

@jdantonio
Copy link
Author

@rtyler Thanks for the suggestion. I didn't think of that when I created this issue, but I can see why it's important for you. For completeness, I've post the relevant output here:

Finished in 5 minutes 8 seconds (files took 9.17 seconds to load)
2198 examples, 0 failures, 3 pending
Randomized with seed 1
Coverage report generated for RSpec to /home/travis/build/ruby-concurrency/concurrent-ruby/coverage. 0 / 6 LOC (0.0%) covered.
[Coveralls] Submitting with config:
{
  "environment": {
    "pwd": "/home/travis/build/ruby-concurrency/concurrent-ruby",
    "rails_root": null,
    "simplecov_root": "/home/travis/build/ruby-concurrency/concurrent-ruby",
    "gem_version": "0.8.2",
    "travis_job_id": "75747464",
    "travis_pull_request": "false"
  },
  "git": {
    "head": {
      "id": "d58679c325cd370286bde049636c29c82bb35585",
      "author_name": "Jerry D'Antonio",
      "author_email": "stumpjumper@gmail.com",
      "committer_name": "Jerry D'Antonio",
      "committer_email": "stumpjumper@gmail.com",
      "message": "Merge pull request #386 from ruby-concurrency/thread_safe"
    },
    "branch": "HEAD\n",
    "remotes": [
      {
        "name": "origin",
        "url": "git://github.com/ruby-concurrency/concurrent-ruby.git"
      }
    ]
  },
  "configuration": {
    "repo_token": "[secure]"
  },
  "repo_token": "[secure]",
  "service_job_id": "75747464",
  "service_name": "travis-ci",
  "service_number": null,
  "service_build_url": null,
  "service_branch": null,
  "service_pull_request": null
}
[Coveralls] Submitting to https://coveralls.io/api/v1
[Coveralls] Job #1687.8
[Coveralls] https://coveralls.io/jobs/7672889
Coverage is at 0.0%.
Coverage report sent to Coveralls.
/home/travis/.rvm/rubies/jruby-9.0.0.0.pre1/bin/jruby -I/home/travis/.rvm/gems/jruby-9.0.0.0.pre1/gems/rspec-core-3.3.2/lib:/home/travis/.rvm/gems/jruby-9.0.0.0.pre1/gems/rspec-support-3.3.0/lib /home/travis/.rvm/gems/jruby-9.0.0.0.pre1/gems/rspec-core-3.3.2/exe/rspec --pattern spec/\*\*\{,/\*/\*\*\}/\*_spec.rb --color --backtrace --tag ~unfinished --seed 1 --format documentation --tag ~notravis failed

The command "RUBYOPT=-w bundle exec rake ci" exited with 1.
Done. Your build exited with 1.

@deivid-rodriguez
Copy link
Contributor

I haven't seen reliable coverage reports in 9.0.0.0 yet, there's this open issue, so it seems like a work in progress.

In any case, two comments:

  • The sample build seems to be using an unreleased 9.0.0.0 ruby.
  • I think you're supposed to use the --debug flag to get coverage in jruby or at least that's was simplecov tells you.

Cheers!

@jdantonio
Copy link
Author

@deivid-rodriguez Thank you for the feedback, but I don't believe I've clearly explained the issue. Please let me clarify. The problem isn't the code coverage value--it's the return code of 1. This causes the build to register as a failure even though all the tests passed. This wasn't a problem a few days ago. Up until last weekend the exit code was 0 and the build passed even when the code coverage value was completely wrong. This build here is an example (details below).

Having a wrong code coverage value is minor. Having the build erroneously fail is a much bigger issue.

For now I've setup my CI rake task to not report to Coveralls when running under JRuby 9000. That's the only way I can keep the build green.

Thanks again!

Finished in 3 minutes 30.4 seconds (files took 8.99 seconds to load)
2140 examples, 0 failures, 3 pending
Randomized with seed 1
Coverage report generated for RSpec to /home/travis/build/ruby-concurrency/concurrent-ruby/coverage. 0 / 6 LOC (0.0%) covered.
[Coveralls] Submitting with config:
{
  "environment": {
    "pwd": "/home/travis/build/ruby-concurrency/concurrent-ruby",
    "rails_root": null,
    "simplecov_root": "/home/travis/build/ruby-concurrency/concurrent-ruby",
    "gem_version": "0.8.2",
    "travis_job_id": "74924493",
    "travis_pull_request": "false"
  },
  "git": {
    "head": {
      "id": "1fbf9d594801d2c7a2d1b12c24338711d1ffac3f",
      "author_name": "Jerry D'Antonio",
      "author_email": "stumpjumper@gmail.com",
      "committer_name": "Jerry D'Antonio",
      "committer_email": "stumpjumper@gmail.com",
      "message": "Merge pull request #391 from ruby-concurrency/one-point-zero-pre-1"
    },
    "branch": "HEAD\n",
    "remotes": [
      {
        "name": "origin",
        "url": "git://github.com/ruby-concurrency/concurrent-ruby.git"
      }
    ]
  },
  "configuration": {
    "repo_token": "[secure]"
  },
  "repo_token": "[secure]",
  "service_job_id": "74924493",
  "service_name": "travis-ci",
  "service_number": null,
  "service_build_url": null,
  "service_branch": null,
  "service_pull_request": null
}
[Coveralls] Submitting to https://coveralls.io/api/v1
[Coveralls] Job #1665.8
[Coveralls] https://coveralls.io/jobs/7558295
Coverage is at 0.0%.
Coverage report sent to Coveralls.
The command "RUBYOPT=-w bundle exec rake ci" exited with 0.
Done. Your build exited with 0.

@jdantonio jdantonio changed the title JRuby 9000 reports 0% Coveralls coverage JRuby 9000 + TravisCI + Coveralls Erroneous Failure Aug 18, 2015
@deivid-rodriguez
Copy link
Contributor

Oh, I see. Sorry for the noise then!

@enebo enebo added this to the Non-Release milestone Oct 13, 2015
tfausak added a commit to bolshakov/stoplight that referenced this issue Feb 5, 2016
@tfausak
Fix JRuby coverage by setting debug flag
e8f6e1d 
jruby/jruby#3256
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
JRuby 9000
Projects
None yet
Milestone
Non-Release
Development

No branches or pull requests
4 participants
@enebo @rtyler @jdantonio @deivid-rodriguez