Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: jruby/jruby
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: 3528613cc58c
Choose a base ref
...
head repository: jruby/jruby
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: 294bf2765262
Choose a head ref
  • 18 commits
  • 23 files changed
  • 5 contributors

Commits on Jun 5, 2015

  1. Copy the full SHA
    410cdee View commit details
  2. Don't call modify() on RubyStrings so eagerly when freezing them

    This fixes #3019, where in certain cases it's possible to end up with
    RubyString instances backed by very large unique ByteLists where only
    a tiny portion of the bytes in the ByteList are actually needed.
    
    What was happening is `modify` was being called on RubyString
    instances too eagerly, resulting in unnecessary duplication of their
    underlying ByteList instances instead of sharing when possible.
    
    The two changes here are in `RubyString#newFrozen` and
    `RubyString#resize` and I believe are more correct based on the C
    implementation of these methods.
    
    For `RubyString#newFrozen`, which roughly corresponds to
    `rb_str_new_frozen` in
    C (https://github.com/ruby/ruby/blob/v2_2_2/string.c#L932), the logic
    looks to me to return the shared string and I don't see it calling
    `str_make_independent`, which is roughly the same thing as
    `RubyString#modify`. So, I just removed the call to `modify`.
    
    For `RubyString#resize`, which roughly corresponds to to
    `rb_str_resize` in C, shared strings are only made independent if
    their length
    differ (https://github.com/ruby/ruby/blob/v2_2_2/string.c#L2155). Thus,
    instead of unconditionally calling `modify` here, I moved it into the
    conditions that are true when the length differs.
    bbrowning committed Jun 5, 2015

    Verified

    This commit was signed with the committer’s verified signature.
    headius Charles Oliver Nutter
    Copy the full SHA
    4fb81b7 View commit details
  3. Merge pull request #3022 from bbrowning/string-cow-oom

    Don't call modify() on RubyStrings so eagerly when freezing them
    enebo committed Jun 5, 2015

    Verified

    This commit was signed with the committer’s verified signature.
    headius Charles Oliver Nutter
    Copy the full SHA
    f4ede91 View commit details
  4. 1
    Copy the full SHA
    8f4ec6b View commit details
  5. Copy the full SHA
    f4a988b View commit details
  6. Copy the full SHA
    c4249fa View commit details
  7. Fix the slow test indexes so test:slow_suites runs tests again

    This has been broken since July 2014 when some tests were moved from
    test/ to test/jruby/. Thus, once reenabled some of these tests have
    failures.
    bbrowning committed Jun 5, 2015
    Copy the full SHA
    3f59de2 View commit details
  8. Revert "[Truffle] Remove use of Java 8 method."

    This reverts commit c4249fa.
    chrisseaton committed Jun 5, 2015
    Copy the full SHA
    cb97f5a View commit details
  9. Revert "Merge branch 'truffle-symbol-refactor'"

    This reverts commit f4a988b, reversing
    changes made to 8f4ec6b.
    chrisseaton committed Jun 5, 2015
    Copy the full SHA
    bbdd0ab View commit details
  10. Merge pull request #3023 from bbrowning/slow_suites

    Fix the slow test indexes so test:slow_suites runs tests again
    headius committed Jun 5, 2015
    Copy the full SHA
    828fad3 View commit details

Commits on Jun 8, 2015

  1. Update ripper parser to line up with mainline parser. Fix lineno to p…

    …ass all but 8 specs on Yard.
    enebo committed Jun 8, 2015
    Copy the full SHA
    86dfb5d View commit details
  2. Ripper on_params handling of kw were using hash and not array and cra…

    …shing for reqd kwargs
    enebo committed Jun 8, 2015
    Copy the full SHA
    1d1e883 View commit details

Commits on Jun 9, 2015

  1. Copy the full SHA
    b81d378 View commit details
  2. Update for released deps

    enebo committed Jun 9, 2015
    Copy the full SHA
    1baad69 View commit details
  3. Copy the full SHA
    b96813f View commit details
  4. Fix new id values for true,false,nil. Workaround #3028 so we can test…

    … the behavior of finalization in ObjectSpace
    enebo committed Jun 9, 2015
    Copy the full SHA
    9f76735 View commit details
  5. Revert "Revert "Merge branch 'truffle-symbol-refactor'""

    This reverts commit bbdd0ab.
    chrisseaton committed Jun 9, 2015
    Copy the full SHA
    eb4f675 View commit details
  6. Copy the full SHA
    294bf27 View commit details
4 changes: 2 additions & 2 deletions core/pom.rb
Original file line number Diff line number Diff line change
@@ -44,8 +44,8 @@
jar 'com.github.jnr:jnr-enxio:0.9'
jar 'com.github.jnr:jnr-x86asm:1.0.2'
jar 'com.github.jnr:jnr-unixsocket:0.8'
jar 'com.github.jnr:jnr-posix:3.0.13-SNAPSHOT'
jar 'com.github.jnr:jnr-constants:0.8.7'
jar 'com.github.jnr:jnr-posix:3.0.14'
jar 'com.github.jnr:jnr-constants:0.8.8'
jar 'com.github.jnr:jnr-ffi:2.0.3'
jar 'com.github.jnr:jffi:${jffi.version}'
jar 'com.github.jnr:jffi:${jffi.version}:native'
4 changes: 2 additions & 2 deletions core/pom.xml
Original file line number Diff line number Diff line change
@@ -105,12 +105,12 @@
<dependency>
<groupId>com.github.jnr</groupId>
<artifactId>jnr-posix</artifactId>
<version>3.0.13-SNAPSHOT</version>
<version>3.0.14</version>
</dependency>
<dependency>
<groupId>com.github.jnr</groupId>
<artifactId>jnr-constants</artifactId>
<version>0.8.7</version>
<version>0.8.8</version>
</dependency>
<dependency>
<groupId>com.github.jnr</groupId>
2 changes: 2 additions & 0 deletions core/src/main/java/org/jruby/Ruby.java
Original file line number Diff line number Diff line change
@@ -46,6 +46,7 @@
import org.jruby.ast.FCallNode;
import org.jruby.ast.GlobalAsgnNode;
import org.jruby.ast.GlobalVarNode;
import org.jruby.ast.StrNode;
import org.jruby.ast.VCallNode;
import org.jruby.ast.WhileNode;
import org.jruby.compiler.Constantizable;
@@ -702,6 +703,7 @@ private RootNode addGetsLoop(RootNode oldRoot, boolean printing, boolean process
whileBody.add(oldRoot.getBodyNode());
}

newBody.prepend(new GlobalAsgnNode(pos, "$/", new StrNode(pos, new ByteList(getInstanceConfig().getRecordSeparator().getBytes()))));
GlobalVarNode dollarUnderscore = new GlobalVarNode(pos, "$_");

newBody.add(new WhileNode(pos, new VCallNode(pos, "gets"), whileBody));
4 changes: 2 additions & 2 deletions core/src/main/java/org/jruby/RubyString.java
Original file line number Diff line number Diff line change
@@ -965,18 +965,18 @@ public RubyString newFrozen() {
klass = getMetaClass();
str = strDup(klass.getClassRuntime());
str.setCodeRange(getCodeRange());
str.modify();
str.setFrozen(true);
return str;
}

/** rb_str_resize
*/
public final void resize(int length) {
modify();
if (value.getRealSize() > length) {
modify();
value.setRealSize(length);
} else if (value.length() < length) {
modify();
value.length(length);
}
}
393 changes: 213 additions & 180 deletions core/src/main/java/org/jruby/ext/ripper/RipperLexer.java

Large diffs are not rendered by default.

1,569 changes: 815 additions & 754 deletions core/src/main/java/org/jruby/ext/ripper/RipperParser.java

Large diffs are not rendered by default.

43 changes: 32 additions & 11 deletions core/src/main/java/org/jruby/ext/ripper/RipperParser.y
Original file line number Diff line number Diff line change
@@ -887,8 +887,12 @@ arg : lhs '=' arg {
| kDEFINED opt_nl arg {
$$ = p.dispatch("on_defined", $3);
}
| arg '?' arg opt_nl ':' arg {
$$ = p.dispatch("on_ifop", $1, $3, $6);
| arg '?' {
p.getConditionState().begin();
} arg opt_nl ':' {
p.getConditionState().end();
} arg {
$$ = p.dispatch("on_ifop", $1, $4, $8);
}
| primary {
$$ = $1;
@@ -1024,19 +1028,27 @@ primary : literal
| tFID {
$$ = p.dispatch("on_method_add_arg", p.dispatch("on_fcall", $1), p.dispatch("on_args_new"));
}
| kBEGIN bodystmt kEND {
$$ = p.dispatch("on_begin", $2);
| kBEGIN {
$$ = p.getCmdArgumentState().getStack();
p.getCmdArgumentState().reset();
} bodystmt kEND {
p.getCmdArgumentState().reset($<Long>2.longValue());
$$ = p.dispatch("on_begin", $3);
}
| tLPAREN_ARG {
p.setState(LexState.EXPR_ENDARG);
} rparen {
$$ = p.dispatch("on_paren", null);
}
| tLPAREN_ARG expr {
| tLPAREN_ARG {
$$ = p.getCmdArgumentState().getStack();
p.getCmdArgumentState().reset();
} expr {
p.setState(LexState.EXPR_ENDARG);
} rparen {
p.getCmdArgumentState().reset($<Long>2.longValue());
p.warning("(...) interpreted as grouped expression");
$$ = p.dispatch("on_paren", $2);
$$ = p.dispatch("on_paren", $3);
}
| tLPAREN compstmt tRPAREN {
$$ = p.dispatch("on_paren", $2);
@@ -1611,15 +1623,24 @@ string_content : tSTRING_CONTENT
}
| tSTRING_DBEG {
$$ = p.getStrTerm();
p.getConditionState().stop();
p.getCmdArgumentState().stop();
p.setStrTerm(null);
p.getConditionState().stop();
} {
$$ = p.getCmdArgumentState().getStack();
p.getCmdArgumentState().reset();
} {
$$ = p.getState();
p.setState(LexState.EXPR_BEG);
} compstmt tRCURLY {
} {
$$ = p.getBraceNest();
p.setBraceNest(0);
} compstmt tSTRING_DEND {
p.getConditionState().restart();
p.getCmdArgumentState().restart();
p.setStrTerm($<StrTerm>2);
$$ = p.dispatch("on_string_embexpr", $3);
p.getCmdArgumentState().reset($<Long>3.longValue());
p.setState($<LexState>4);
p.setBraceNest($<Integer>5);
$$ = p.dispatch("on_string_embexpr", $6);
}

string_dvar : tGVAR {
23 changes: 20 additions & 3 deletions core/src/main/java/org/jruby/ext/ripper/RipperParserBase.java
Original file line number Diff line number Diff line change
@@ -210,11 +210,16 @@ public IRubyObject method_optarg(IRubyObject method, IRubyObject arg) {
}

public IRubyObject keyword_arg(IRubyObject key, IRubyObject value) {
RubyHash hash = RubyHash.newHash(context.runtime);
RubyArray array = RubyArray.newArray(context.runtime, 2);

hash.fastASet(key, value);
array.append(key);
if (value != null) {
array.append(value);
} else {
array.append(context.nil);
}

return hash;
return array;
}

public IRubyObject new_args(IRubyObject f, IRubyObject o, IRubyObject r, IRubyObject p, ArgsTailHolder tail) {
@@ -351,6 +356,18 @@ public int getInSingle() {
return inSingleton;
}

public int getBraceNest() {
return lexer.getBraceNest();
}

public LexState getState() {
return lexer.getState();
}

public void setBraceNest(int braceNest) {
lexer.setBraceNest(braceNest);
}

public void setState(LexState lexState) {
lexer.setState(lexState);
}
8,178 changes: 4,159 additions & 4,019 deletions core/src/main/java/org/jruby/ext/ripper/YyTables.java

Large diffs are not rendered by default.

5 changes: 2 additions & 3 deletions core/src/main/java/org/jruby/lexer/yacc/RubyLexer.java
Original file line number Diff line number Diff line change
@@ -1941,13 +1941,12 @@ private int identifier(int c, boolean commandState) throws IOException {
String tempVal = createTokenString();

if (isLabelPossible(commandState)) {
int c2 = nextc();
if (c2 == ':' && !peek(':')) {
if (isLabelSuffix()) {
setState(LexState.EXPR_LABELARG);
nextc();
yaccValue = tempVal.intern();
return Tokens.tLABEL;
}
pushback(c2);
}

if (lex_state != LexState.EXPR_DOT) {
4 changes: 4 additions & 0 deletions core/src/main/java/org/jruby/lexer/yacc/StackState.java
Original file line number Diff line number Diff line change
@@ -43,6 +43,10 @@ public void reset(long backup) {
stack = backup;
}

public long getStack() {
return stack;
}

// PUSH(1)
public long begin() {
long old = stack;
1,460 changes: 745 additions & 715 deletions core/src/main/java/org/jruby/parser/RubyParser.java

Large diffs are not rendered by default.

28 changes: 19 additions & 9 deletions core/src/main/java/org/jruby/parser/RubyParser.y
Original file line number Diff line number Diff line change
@@ -1331,21 +1331,29 @@ primary : literal
| tFID {
$$ = support.new_fcall($1);
}
| kBEGIN bodystmt kEND {
$$ = new BeginNode($1, $2 == null ? NilImplicitNode.NIL : $2);
| kBEGIN {
$$ = lexer.getCmdArgumentState().getStack();
lexer.getCmdArgumentState().reset();
} bodystmt kEND {
lexer.getCmdArgumentState().reset($<Long>2.longValue());
$$ = new BeginNode($1, $3 == null ? NilImplicitNode.NIL : $3);
}
| tLPAREN_ARG {
lexer.setState(LexState.EXPR_ENDARG);
} rparen {
$$ = null; //FIXME: Should be implicit nil?
}
| tLPAREN_ARG expr {
| tLPAREN_ARG {
$$ = lexer.getCmdArgumentState().getStack();
lexer.getCmdArgumentState().reset();
} expr {
lexer.setState(LexState.EXPR_ENDARG);
} rparen {
lexer.getCmdArgumentState().reset($<Long>2.longValue());
if (Options.PARSER_WARN_GROUPED_EXPRESSIONS.load()) {
support.warning(ID.GROUPED_EXPRESSION, $1, "(...) interpreted as grouped expression");
}
$$ = $2;
$$ = $3;
}
| tLPAREN compstmt tRPAREN {
if ($2 != null) {
@@ -2011,7 +2019,9 @@ string_content : tSTRING_CONTENT {
$$ = lexer.getStrTerm();
lexer.setStrTerm(null);
lexer.getConditionState().stop();
lexer.getCmdArgumentState().stop();
} {
$$ = lexer.getCmdArgumentState().getStack();
lexer.getCmdArgumentState().reset();
} {
$$ = lexer.getState();
lexer.setState(LexState.EXPR_BEG);
@@ -2020,12 +2030,12 @@ string_content : tSTRING_CONTENT {
lexer.setBraceNest(0);
} compstmt tSTRING_DEND {
lexer.getConditionState().restart();
lexer.getCmdArgumentState().restart();
lexer.setStrTerm($<StrTerm>2);
lexer.setState($<LexState>3);
lexer.setBraceNest($<Integer>4);
lexer.getCmdArgumentState().reset($<Long>3.longValue());
lexer.setState($<LexState>4);
lexer.setBraceNest($<Integer>5);

$$ = support.newEvStrNode(support.getPosition($5), $5);
$$ = support.newEvStrNode(support.getPosition($6), $6);
}

string_dvar : tGVAR {
6,278 changes: 3,147 additions & 3,131 deletions core/src/main/java/org/jruby/parser/YyTables.java

Large diffs are not rendered by default.

37 changes: 37 additions & 0 deletions spec/regression/parser_regressions_spec.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
require 'rspec'

describe "The Parsing experience" do
it "parses a cond which used to get confused as a tLABEL" do
cond = true
x = '('
a = cond ? x + ')': ''
a.should eq('()')
end

it "parse the stabby lambda even if args params separated by space" do
a = -> () { 1 }
a.call.should eq(1)
end

it "parses a jumbled mess of nestng" do
self.class.send(:define_method, :foo) { |a| a[:key].call[:key] }
var = 1
res = foo key: (proc do
{
key: "#{var}#{var}",
other_key: proc do
end
}
end)

res.should eq("11")
end

it "parsers a block do inside of a call arg list" do
self.class.send(:define_method, :foo) { |a| a }
res = foo (10.times.to_a.map do |i|
7 + i
end)
res.should eq([7,8,9,10,11,12,13,14,15,16])
end
end
43 changes: 24 additions & 19 deletions test/jruby/test_objectspace.rb
Original file line number Diff line number Diff line change
@@ -35,12 +35,12 @@ def test_id2ref
assert_equal(-19, -10.object_id)

assert_equal(0, false.object_id)
assert_equal(2, true.object_id)
assert_equal(4, nil.object_id)
assert_equal(20, true.object_id)
assert_equal(8, nil.object_id)

assert_equal(false, ObjectSpace._id2ref(0))
assert_equal(true, ObjectSpace._id2ref(2))
assert_equal(nil, ObjectSpace._id2ref(4))
assert_equal(true, ObjectSpace._id2ref(20))
assert_equal(nil, ObjectSpace._id2ref(8))
end

def test_jruby_objectspace
@@ -65,27 +65,32 @@ def finalizer(results)
end
end

# JRUBY-4839
def test_finalization
[true, false].each do |objectspace|
JRuby.objectspace = objectspace
obj1 = "lemon"
obj2 = "apple"
results = []
def body(objectspace, results)
JRuby.objectspace = objectspace
obj1 = "lemon"
obj2 = "apple"

ObjectSpace.define_finalizer obj1, finalizer(results)
ObjectSpace.define_finalizer obj2, finalizer(results)
ObjectSpace.define_finalizer obj1, finalizer(results)
ObjectSpace.define_finalizer obj2, finalizer(results)

obj1_id = obj1.object_id
results << obj1.object_id

ObjectSpace.undefine_finalizer obj2
ObjectSpace.undefine_finalizer obj2

obj1 = nil
obj2 = nil
end

# JRUBY-4839 GH #3028
def test_finalization
[true, false].each do |objectspace|
results = []

obj1 = nil
obj2 = nil
body(objectspace, results)

t = Time.now
(JRuby.gc; sleep 0.1) until (Time.now - t > 5) || results.length > 0
assert_equal ["finalizing #{obj1_id}"], results
(JRuby.gc; sleep 0.1) until (Time.now - t > 5) || results.length > 1
assert_equal "finalizing #{results[0]}", results[1]
end
end
end
5 changes: 1 addition & 4 deletions test/mri/excludes/TestRipper/Filter.rb
Original file line number Diff line number Diff line change
@@ -1,6 +1,3 @@
exclude :test_filter_column, "needs investigation"
exclude :test_filter_filename, "needs investigation"
exclude :test_filter_filename_unset, "needs investigation"
exclude :test_filter_lineno, "needs investigation"
exclude :test_filter_lineno_set, "needs investigation"
exclude :test_filter_token, "needs investigation"

5 changes: 2 additions & 3 deletions test/mri/excludes/TestRipper/ScannerEvents.rb
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
exclude :test_embexpr_end, "needs investigation"
exclude :test_event_coverage, "embexpr_end due to parser/lexer bug"
exclude :test_tokenize, "needs investigation"
exclude :test_lex, "off-by-one one-liner no newline"
exclude :test_location, "off-by-one one-liner no newline"

4 changes: 2 additions & 2 deletions test/objectspace.index
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
test_objectspace
test_cache_map_leak
jruby/test_objectspace
jruby/test_cache_map_leak
4 changes: 2 additions & 2 deletions test/slow.index
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
test_command_line_switches
test_launching_by_shell_script
jruby/test_command_line_switches
jruby/test_launching_by_shell_script
1 change: 1 addition & 0 deletions truffle/README.md
Original file line number Diff line number Diff line change
@@ -15,6 +15,7 @@ University Linz](http://ssw.jku.at).
* Chris Seaton
* Benoit Daloze
* Kevin Menard
* Petr Chalupa
* Thomas Würthinger
* Matthias Grimmer
* Josef Haider
Original file line number Diff line number Diff line change
@@ -1275,7 +1275,7 @@ public NameNode(RubyContext context, SourceSection sourceSection) {
public Object name(RubyModule module) {
CompilerDirectives.transferToInterpreter();

if (!module.hasName()) {
if (!module.hasPartialName()) {
return nil();
}

Original file line number Diff line number Diff line change
@@ -417,7 +417,11 @@ public String getName() {
}

public boolean hasName() {
return name != null || givenBaseName != null;
return name != null;
}

public boolean hasPartialName() {
return hasName() || givenBaseName != null;
}

@Override