Ok, interesting. I'll have a look.

Ok, the problem here is that the new logic for process spawning does not support chdir in the same way as MRI.

In MRI, if you specify chdir when calling spawn or system or whatever, it will actually do a native chdir before launching the child process. It does this either after forking or temporarily around the call to spawn(2) (setting it back after launching the process).

MRI also actually does a real native chdir when doing Dir.chdir.

We have never been comfortable changing the directory the JVM is running in, because we worry that will at least confuse the JVM and at worst interfere with other threads or applications running in the same process. However, we have never tested real chdir to see if anything bad happens.

A short term fix for this in JRuby would be to fall back on the old JVM process logic when we need to chdir, since that logic uses fork+exec similar to MRI. This might be ok for system, since all it does is capture all child process output. However, it would be too great a change for e.g. spawn since the JVM logic can't support the many options spawn requires.

It's worth pointing out that MRI will behave the same as us even without chdir if the command given looks like a command line. For example, if I change

sh RUBY, 'check_no_expansion.rb', '$RAKE_TEST_SH', 'someval'

to

sh RUBY + ' check_no_expansion.rb $RAKE_TEST_SH someval'

MRI will also expand the env var. So it's a bit fragile to build commands that have loose env var references and expect them not to expand.

This is fixed, but it's still an issue for other spawn forms. However, I still feel like it's asking for trouble spawning subprocesses with env vars and expecting them not to expand.

I still feel like it's asking for trouble spawning subprocesses with env vars and expecting them not to expand.

I agreed. Rake has expantion and no expantion tests at https://github.com/ruby/rake/blob/master/test/test_rake_file_utils.rb#L140 . I didn't know why CRuby shows this behavior.

I will discuss this to CRuby core team.

@headius @hsbt I think there's still and issue in 9.0.4.0 and Rake 11.0/11.1.

(Not sure right place to post or make issue, so thought I'd start here with a comment.)

See rails-api/active_model_serializers#1585

Excerpt for convenience:

https://travis-ci.org/rails-api/active_model_serializers/jobs/115622797

Coverage may be inaccurate; set "cli.debug=true" ("-Xcli.debug=true") in your .jrubyrc or do JRUBY_OPTS="-d"
/home/travis/build/rails-api/active_model_serializers/.simplecov:48: warning: tracing (e.g. set_trace_func) will not capture all events without --debug flag
/home/travis/.rvm/rubies/jruby-9.0.4.0/bin/jruby --verbose -w -I"lib:test" -r./test/test_helper.rb -I"/home/travis/build/rails-api/active_model_serializers/vendor/bundle/jruby/2.2.0/gems/rake-11.1.0/lib" "/home/travis/build/rails-api/active_model_serializers/vendor/bundle/jruby/2.2.0/gems/rake-11.1.0/lib/rake/rake_test_loader.rb" "test/action_controller/adapter_selector_test.rb" "test/action_controller/explicit_serializer_test.rb" "test/action_controller/json/include_test.rb" "test/action_controller/json_api/deserialization_test.rb" "test/action_controller/json_api/errors_test.rb" "test/action_controller/json_api/linked_test.rb" "test/action_controller/json_api/pagination_test.rb" "test/action_controller/serialization_scope_name_test.rb" "test/action_controller/serialization_test.rb" "test/active_model_serializers/adapter_for_test.rb" "test/active_model_serializers/json_pointer_test.rb" "test/active_model_serializers/logging_test.rb" "test/active_model_serializers/model_test.rb" "test/active_model_serializers/test/schema_test.rb" "test/active_model_serializers/test/serializer_test.rb" "test/active_record_test.rb" "test/adapter/deprecation_test.rb" "test/adapter/fragment_cache_test.rb" "test/adapter/json/belongs_to_test.rb" "test/adapter/json/collection_test.rb" "test/adapter/json/has_many_test.rb" "test/adapter/json_api/belongs_to_test.rb" "test/adapter/json_api/collection_test.rb" "test/adapter/json_api/errors_test.rb" "test/adapter/json_api/fields_test.rb" "test/adapter/json_api/has_many_embed_ids_test.rb" "test/adapter/json_api/has_many_explicit_serializer_test.rb" "test/adapter/json_api/has_many_test.rb" "test/adapter/json_api/has_one_test.rb" "test/adapter/json_api/json_api_test.rb" "test/adapter/json_api/linked_test.rb" "test/adapter/json_api/links_test.rb" "test/adapter/json_api/pagination_links_test.rb" "test/adapter/json_api/parse_test.rb" "test/adapter/json_api/relationship_test.rb" "test/adapter/json_api/relationships_test.rb" "test/adapter/json_api/resource_identifier_test.rb" "test/adapter/json_api/resource_meta_test.rb" "test/adapter/json_api/toplevel_jsonapi_test.rb" "test/adapter/json_api/type_test.rb" "test/adapter/json_test.rb" "test/adapter/null_test.rb" "test/adapter_test.rb" "test/array_serializer_test.rb" "test/collection_serializer_test.rb" "test/generators/scaffold_controller_generator_test.rb" "test/generators/serializer_generator_test.rb" "test/grape_test.rb" "test/include_tree/from_include_args_test.rb" "test/include_tree/from_string_test.rb" "test/include_tree/include_args_to_hash_test.rb" "test/lint_test.rb" "test/logger_test.rb" "test/poro_test.rb" "test/serializable_resource_test.rb" "test/serializers/association_macros_test.rb" "test/serializers/associations_test.rb" "test/serializers/attribute_test.rb" "test/serializers/attributes_test.rb" "test/serializers/cache_test.rb" "test/serializers/cached_serializer_test.rb" "test/serializers/configuration_test.rb" "test/serializers/fieldset_test.rb" "test/serializers/meta_test.rb" "test/serializers/options_test.rb" "test/serializers/root_test.rb" "test/serializers/serializer_for_test.rb" 
jruby: unknown option --verbose
rake aborted!
Command failed with status (1): [ruby --verbose -w -I"lib:test" -r./test/test_helper.rb -I"/home/travis/build/rails-api/active_model_serializers/vendor/bundle/jruby/2.2.0/gems/rake-11.1.0/lib" "/home/travis/build/rails-api/active_model_serializers/vendor/bundle/jruby/2.2.0/gems/rake-11.1.0/lib/rake/rake_test_loader.rb" "test/action_controller/adapter_selector_test.rb" 

@bf4 It's not related this issue. It's regression of rake behavior. Can you submit this issue to ruby/rake ?

I will investigate next week.

@hsbt Thanks. So, JRuby isn't supposed to have a --verbose flag?

Yes. I will fix your issue at ruby/rake#120 and release 11.1.1.

@bf4 It appears MRI does have a --verbose flag and we don't. So I'll fix that here.