Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: jruby/jruby
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: c54357cd075a
Choose a base ref
...
head repository: jruby/jruby
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: 5d6ddf88631c
Choose a head ref
  • 12 commits
  • 61 files changed
  • 1 contributor

Commits on Jul 6, 2017

  1. Mega Mega Patch. Parser largely pushes most things as ByteList.

    Up to this point all things are still being accessed through String versions
    of methods from IRBuilder on.  So this should be zero sum...However...there
    is a problem I have not figured out yet where I think a RubyHash is getting
    messed up.  So there will be at least one more commit before this is fully
    zero sum.
    
    I have made all existing methods and deprecated.  They probably will only
    work so long as the strings represent 7bit ascii or are utf-8 but since they
    are all internal methods no longer used I do not fully care.  If some external
    projects are using them then I do not expect they will notice since we have
    had mbc issues for years.
    
    I had considered making all identifiers share the source line they exist on
    to prevent extra array construction and copying.  It could be changed later.
    
    Also I still make a token string in the lexer and as a side-effect we get
    and identifier ByteList.  Once this is fully working we can just delete
    token creation code and only use the identifier.
    enebo committed Jul 6, 2017
    Copy the full SHA
    b9d3782 View commit details
  2. Copy the full SHA
    c7beb68 View commit details
  3. Copy the full SHA
    9fad20c View commit details
  4. Copy the full SHA
    f342c29 View commit details
  5. Copy the full SHA
    3288cbe View commit details
  6. Copy the full SHA
    6a2d09e View commit details
  7. Copy the full SHA
    ffbf7bd View commit details
  8. We need to be like we were before in making a string from a bytelist …

    …in the helper. Note: Once we switch to bytelist this method will never be called again.
    enebo committed Jul 6, 2017
    Copy the full SHA
    fc02bf0 View commit details
  9. Some name capture not using bytelists yet (still runs over assumption…

    … in joni
    
    on bytes encoding).
    enebo committed Jul 6, 2017
    Copy the full SHA
    bf2a2b4 View commit details
  10. local variables should use StringSupport instead of toString() in dep…

    …recated getVariables() method.
    enebo committed Jul 6, 2017
    Copy the full SHA
    fb57a88 View commit details
  11. Update parser for some changes for 2.4. This is only partial and I am…

    … mixing
    
    in another activity at the same time....Making the source look much more like
    MRIs parse.y by using static imports.
    enebo committed Jul 6, 2017
    Copy the full SHA
    c38e1d9 View commit details
  12. Update for 9.2.0.0-SNAPSHOT

    enebo committed Jul 6, 2017
    Copy the full SHA
    5d6ddf8 View commit details
Showing with 6,486 additions and 5,094 deletions.
  1. +4 −4 core/pom.xml
  2. +0 −2 core/src/main/java/org/jruby/RubyModule.java
  3. +17 −0 core/src/main/java/org/jruby/RubyRegexp.java
  4. +3 −6 core/src/main/java/org/jruby/RubySymbol.java
  5. +20 −6 core/src/main/java/org/jruby/ast/ArgumentNode.java
  6. +14 −3 core/src/main/java/org/jruby/ast/AttrAssignNode.java
  7. +7 −4 core/src/main/java/org/jruby/ast/BackRefNode.java
  8. +17 −4 core/src/main/java/org/jruby/ast/BlockArgNode.java
  9. +14 −2 core/src/main/java/org/jruby/ast/CallNode.java
  10. +14 −3 core/src/main/java/org/jruby/ast/ClassVarAsgnNode.java
  11. +14 −2 core/src/main/java/org/jruby/ast/ClassVarDeclNode.java
  12. +16 −4 core/src/main/java/org/jruby/ast/ClassVarNode.java
  13. +8 −0 core/src/main/java/org/jruby/ast/Colon2ConstNode.java
  14. +6 −0 core/src/main/java/org/jruby/ast/Colon2ImplicitNode.java
  15. +8 −1 core/src/main/java/org/jruby/ast/Colon2Node.java
  16. +15 −4 core/src/main/java/org/jruby/ast/Colon3Node.java
  17. +15 −4 core/src/main/java/org/jruby/ast/ConstDeclNode.java
  18. +16 −4 core/src/main/java/org/jruby/ast/ConstNode.java
  19. +15 −3 core/src/main/java/org/jruby/ast/DAsgnNode.java
  20. +15 −3 core/src/main/java/org/jruby/ast/DVarNode.java
  21. +9 −1 core/src/main/java/org/jruby/ast/DefnNode.java
  22. +14 −1 core/src/main/java/org/jruby/ast/DefsNode.java
  23. +20 −1 core/src/main/java/org/jruby/ast/FCallNode.java
  24. +7 −0 core/src/main/java/org/jruby/ast/FalseNode.java
  25. +13 −2 core/src/main/java/org/jruby/ast/GlobalAsgnNode.java
  26. +13 −2 core/src/main/java/org/jruby/ast/GlobalVarNode.java
  27. +15 −3 core/src/main/java/org/jruby/ast/InstAsgnNode.java
  28. +16 −4 core/src/main/java/org/jruby/ast/InstVarNode.java
  29. +7 −1 core/src/main/java/org/jruby/ast/KeywordRestArgNode.java
  30. +12 −4 core/src/main/java/org/jruby/ast/LiteralNode.java
  31. +15 −3 core/src/main/java/org/jruby/ast/LocalAsgnNode.java
  32. +15 −3 core/src/main/java/org/jruby/ast/LocalVarNode.java
  33. +9 −3 core/src/main/java/org/jruby/ast/MethodDefNode.java
  34. +7 −0 core/src/main/java/org/jruby/ast/NilNode.java
  35. +11 −3 core/src/main/java/org/jruby/ast/OpAsgnConstDeclNode.java
  36. +17 −10 core/src/main/java/org/jruby/ast/OpAsgnNode.java
  37. +13 −5 core/src/main/java/org/jruby/ast/OpElementAsgnNode.java
  38. +5 −0 core/src/main/java/org/jruby/ast/OptArgNode.java
  39. +7 −0 core/src/main/java/org/jruby/ast/SelfNode.java
  40. +15 −11 core/src/main/java/org/jruby/ast/SymbolNode.java
  41. +7 −0 core/src/main/java/org/jruby/ast/TrueNode.java
  42. +6 −1 core/src/main/java/org/jruby/ast/UnnamedRestArgNode.java
  43. +13 −5 core/src/main/java/org/jruby/ast/VAliasNode.java
  44. +14 −2 core/src/main/java/org/jruby/ast/VCallNode.java
  45. +3 −0 core/src/main/java/org/jruby/ast/types/INameNode.java
  46. +1 −1 core/src/main/java/org/jruby/ext/ripper/RipperLexer.java
  47. +2 −2 core/src/main/java/org/jruby/ext/ripper/RipperParser.java
  48. +2 −2 core/src/main/java/org/jruby/ext/ripper/RipperParser.y
  49. +2 −14 core/src/main/java/org/jruby/ir/IRBuilder.java
  50. +97 −4 core/src/main/java/org/jruby/lexer/LexingCommon.java
  51. +203 −134 core/src/main/java/org/jruby/lexer/yacc/RubyLexer.java
  52. +241 −50 core/src/main/java/org/jruby/parser/ParserSupport.java
  53. +1,520 −1,192 core/src/main/java/org/jruby/parser/RubyParser.java
  54. +409 −251 core/src/main/java/org/jruby/parser/RubyParser.y
  55. +88 −36 core/src/main/java/org/jruby/parser/StaticScope.java
  56. +27 −5 core/src/main/java/org/jruby/parser/StaticScopeFactory.java
  57. +3,323 −3,275 core/src/main/java/org/jruby/parser/YyTables.java
  58. +47 −0 core/src/main/java/org/jruby/util/StringSupport.java
  59. +2 −2 lib/pom.xml
  60. +1 −1 pom.xml
  61. +0 −1 test/mri/excludes/TestSyntax.rb
8 changes: 4 additions & 4 deletions core/pom.xml
Original file line number Diff line number Diff line change
@@ -12,12 +12,12 @@ DO NOT MODIFIY - GENERATED CODE
<parent>
<groupId>org.jruby</groupId>
<artifactId>jruby-parent</artifactId>
<version>9.1.11.0-SNAPSHOT</version>
<version>9.2.0.0-SNAPSHOT</version>
</parent>
<artifactId>jruby-core</artifactId>
<name>JRuby Core</name>
<properties>
<version.ruby>2.4.0</version.ruby>
<version.ruby>2.4.1</version.ruby>
<prawn.dir>${test.dir}/prawn</prawn.dir>
<spec.tags.dir>${spec.dir}/tags</spec.tags.dir>
<polyglot.dump.pom>pom.xml</polyglot.dump.pom>
@@ -27,7 +27,7 @@ DO NOT MODIFIY - GENERATED CODE
<jruby.test.memory.permgen>2G</jruby.test.memory.permgen>
<installer.gems>${jruby.win32ole.gem}</installer.gems>
<prawn.git.repo>git://github.com/sandal/prawn.git</prawn.git.repo>
<version.ruby.minor>0</version.ruby.minor>
<version.ruby.minor>1</version.ruby.minor>
<tzdata.version>2013d</tzdata.version>
<install4j.executable>/Applications/install4j 4/bin/install4jc</install4j.executable>
<jay.bin>jay</jay.bin>
@@ -47,7 +47,7 @@ DO NOT MODIFIY - GENERATED CODE
<parser.dir>core/src/main/java/org/jruby/parser</parser.dir>
<jruby.basedir>${basedir}/..</jruby.basedir>
<rubyspec.dir>${spec.dir}/ruby</rubyspec.dir>
<version.ruby.revision>56780</version.ruby.revision>
<version.ruby.revision>58053</version.ruby.revision>
<jruby.test.memory>3G</jruby.test.memory>
<mspec.dir>${spec.dir}/mspec</mspec.dir>
<build.date>${maven.build.timestamp}</build.date>
2 changes: 0 additions & 2 deletions core/src/main/java/org/jruby/RubyModule.java
Original file line number Diff line number Diff line change
@@ -3753,8 +3753,6 @@ public IRubyObject getConstantFromNoConstMissing(String name) {
}

public IRubyObject getConstantFromNoConstMissing(String name, boolean includePrivate) {
assert name == name.intern() : name + " is not interned";

final Ruby runtime = getRuntime();
final RubyClass objectClass = runtime.getObject();

17 changes: 17 additions & 0 deletions core/src/main/java/org/jruby/RubyRegexp.java
Original file line number Diff line number Diff line change
@@ -72,6 +72,8 @@
import org.jruby.util.cli.Options;
import org.jruby.util.io.EncodingUtils;
import org.jruby.util.collections.WeakValuedMap;

import static org.jruby.util.StringSupport.EMPTY_BYTELIST_ARRAY;
import static org.jruby.util.StringSupport.EMPTY_STRING_ARRAY;

import java.util.Iterator;
@@ -1427,6 +1429,21 @@ public IRubyObject to_s() {
} while (true);
}

public ByteList[] getByteNames() {
int nameLength = pattern.numberOfNames();
if (nameLength == 0) return EMPTY_BYTELIST_ARRAY;

ByteList[] names = new ByteList[nameLength];
int j = 0;
for (Iterator<NameEntry> i = pattern.namedBackrefIterator(); i.hasNext();) {
NameEntry e = i.next();
names[j++] = new ByteList(e.name, e.nameP, e.nameEnd - e.nameP, pattern.getEncoding(), true);
}

return names;
}

@Deprecated
public String[] getNames() {
int nameLength = pattern.numberOfNames();
if (nameLength == 0) return EMPTY_STRING_ARRAY;
9 changes: 3 additions & 6 deletions core/src/main/java/org/jruby/RubySymbol.java
Original file line number Diff line number Diff line change
@@ -771,9 +771,10 @@ public RubySymbol fastGetSymbol(String internedName) {

public RubySymbol fastGetSymbol(String internedName, boolean hard) {
RubySymbol symbol = null;
int hash = javaStringHashCode(internedName);

for (SymbolEntry e = getEntryFromTable(symbolTable, internedName.hashCode()); e != null; e = e.next) {
if (isSymbolMatch(internedName, e)) {
for (SymbolEntry e = getEntryFromTable(symbolTable, hash); e != null; e = e.next) {
if (isSymbolMatch(internedName, hash, e)) {
if (hard) e.setHardReference();
symbol = e.symbol.get();
break;
@@ -799,10 +800,6 @@ private static boolean isSymbolMatch(ByteList bytes, int hash, SymbolEntry entry
return hash == entry.hash && bytes.equals(entry.bytes);
}

private static boolean isSymbolMatch(String internedName, SymbolEntry entry) {
return internedName == entry.name;
}

private RubySymbol createSymbol(final String name, final ByteList value, final int hash, boolean hard) {
ReentrantLock lock;
(lock = tableLock).lock();
26 changes: 20 additions & 6 deletions core/src/main/java/org/jruby/ast/ArgumentNode.java
Original file line number Diff line number Diff line change
@@ -33,25 +33,34 @@
import org.jruby.ast.types.INameNode;
import org.jruby.ast.visitor.NodeVisitor;
import org.jruby.lexer.yacc.ISourcePosition;
import org.jruby.util.ByteList;
import org.jruby.util.StringSupport;

/**
* Simple Node for named entities. Things like the name of a method will make a node
* for the name. Also local variables will make a ArgumentNode. In the case of a local
* variable we will also keep a list of it's location.
*/
public class ArgumentNode extends Node implements INameNode {
private String identifier;
private ByteList identifier;
private int location;

@Deprecated
public ArgumentNode(ISourcePosition position, String identifier) {
super(position, false);

this.identifier = identifier;
this.identifier = StringSupport.stringAsByteList(identifier);
}

@Deprecated
public ArgumentNode(ISourcePosition position, String identifier, int location) {
this(position, identifier);
this(position, StringSupport.stringAsByteList(identifier), location);
}

public ArgumentNode(ISourcePosition position, ByteList identifier, int location) {
super(position, false);

this.identifier = identifier;
this.location = location; // All variables should be depth 0 in this case
}

@@ -82,13 +91,18 @@ public int getDepth() {
public int getIndex() {
return location & 0xffff;
}

public String getName() {
return StringSupport.byteListAsString(identifier);
}

public ByteList getByteName() {
return identifier;
}


@Deprecated
public void setName(String name) {
this.identifier = name;
this.identifier = StringSupport.stringAsByteList(name);
}

public List<Node> childNodes() {
17 changes: 14 additions & 3 deletions core/src/main/java/org/jruby/ast/AttrAssignNode.java
Original file line number Diff line number Diff line change
@@ -33,17 +33,19 @@
import org.jruby.ast.types.INameNode;
import org.jruby.ast.visitor.NodeVisitor;
import org.jruby.lexer.yacc.ISourcePosition;
import org.jruby.util.ByteList;
import org.jruby.util.StringSupport;

/**
* Node that represents an assignment of either an array element or attribute.
*/
public class AttrAssignNode extends Node implements INameNode, IArgumentNode {
protected final Node receiverNode;
private String name;
private ByteList name;
private Node argsNode;
private final boolean isLazy;

public AttrAssignNode(ISourcePosition position, Node receiverNode, String name, Node argsNode, boolean isLazy) {
public AttrAssignNode(ISourcePosition position, Node receiverNode, ByteList name, Node argsNode, boolean isLazy) {
super(position, receiverNode != null && receiverNode.containsVariableAssignment() || argsNode != null && argsNode.containsVariableAssignment());

assert receiverNode != null : "receiverNode is not null";
@@ -58,6 +60,11 @@ public AttrAssignNode(ISourcePosition position, Node receiverNode, String name,
this.isLazy = isLazy;
}

@Deprecated
public AttrAssignNode(ISourcePosition position, Node receiverNode, String name, Node argsNode, boolean isLazy) {
this(position, receiverNode, StringSupport.stringAsByteList(name), argsNode, isLazy);
}

public NodeType getNodeType() {
return NodeType.ATTRASSIGNNODE;
}
@@ -76,9 +83,13 @@ public Object accept(NodeVisitor visitor) {
* @return name
*/
public String getName() {
return StringSupport.byteListAsString(name);
}

public ByteList getByteName() {
return name;
}

/**
* Gets the receiverNode.
* receiverNode is the object on which the method is being called
11 changes: 7 additions & 4 deletions core/src/main/java/org/jruby/ast/BackRefNode.java
Original file line number Diff line number Diff line change
@@ -33,8 +33,10 @@

import java.util.List;

import org.jcodings.specific.USASCIIEncoding;
import org.jruby.ast.visitor.NodeVisitor;
import org.jruby.lexer.yacc.ISourcePosition;
import org.jruby.util.ByteList;
import org.jruby.util.DefinedMessage;

/**
@@ -49,14 +51,11 @@ public class BackRefNode extends Node {
* the character which generated the back reference
**/
private final char type;

/** ByteList for the name of this backref global */
private final DefinedMessage definedMessage;

public BackRefNode(ISourcePosition position, int type) {
super(position, false);
this.type = (char) type;
this.definedMessage = DefinedMessage.byText("$" + (char)type);
DefinedMessage.byText("$" + (char)type);
}

public NodeType getNodeType() {
@@ -71,6 +70,10 @@ public <T> T accept(NodeVisitor<T> iVisitor) {
return iVisitor.visitBackRefNode(this);
}

public ByteList getByteName() {
return new ByteList(new byte[] {'$', (byte) type}, USASCIIEncoding.INSTANCE);
}

/**
* Gets the type
*
21 changes: 17 additions & 4 deletions core/src/main/java/org/jruby/ast/BlockArgNode.java
Original file line number Diff line number Diff line change
@@ -36,22 +36,30 @@
import org.jruby.ast.types.INameNode;
import org.jruby.ast.visitor.NodeVisitor;
import org.jruby.lexer.yacc.ISourcePosition;
import org.jruby.util.ByteList;
import org.jruby.util.StringSupport;

/**
* An explicit block argument (&amp;my_block) in parameter list.
*/
public class BlockArgNode extends Node implements INameNode {
private final int count;
private String name;
private ByteList name;

public BlockArgNode(ISourcePosition position, int count, String name) {
public BlockArgNode(ISourcePosition position, int count, ByteList name) {
super(position, false);
this.count = count;
this.name = name;
}

@Deprecated
public BlockArgNode(ISourcePosition position, int count, String name) {
this(position, count, StringSupport.stringAsByteList(name));
}


public BlockArgNode(ArgumentNode argNode) {
this(argNode.getPosition(), argNode.getIndex(), argNode.getName());
this(argNode.getPosition(), argNode.getIndex(), argNode.getByteName());
}

public NodeType getNodeType() {
@@ -80,11 +88,16 @@ public int getCount() {
* @return it's name
*/
public String getName() {
return StringSupport.byteListAsString(name);
}

public ByteList getByteName() {
return name;
}

@Deprecated
public void setName(String name) {
this.name = name;
this.name = StringSupport.stringAsByteList(name);
}

public List<Node> childNodes() {
16 changes: 14 additions & 2 deletions core/src/main/java/org/jruby/ast/CallNode.java
Original file line number Diff line number Diff line change
@@ -37,6 +37,8 @@
import org.jruby.ast.types.INameNode;
import org.jruby.ast.visitor.NodeVisitor;
import org.jruby.lexer.yacc.ISourcePosition;
import org.jruby.util.ByteList;
import org.jruby.util.StringSupport;

/**
* A method or operator call.
@@ -45,20 +47,26 @@ public class CallNode extends Node implements INameNode, IArgumentNode, BlockAcc
private final Node receiverNode;
private Node argsNode;
protected Node iterNode;
private String name;
private ByteList name;
private final boolean isLazy;

public CallNode(ISourcePosition position, Node receiverNode, String name, Node argsNode,
Node iterNode) {
this(position, receiverNode, name, argsNode, iterNode, false);
}

@Deprecated
public CallNode(ISourcePosition position, Node receiverNode, String name, Node argsNode,
Node iterNode, boolean isLazy) {
this(position, receiverNode, StringSupport.stringAsByteList(name), argsNode, iterNode, isLazy);
}

public CallNode(ISourcePosition position, Node receiverNode, ByteList name, Node argsNode,
Node iterNode, boolean isLazy) {
super(position, receiverNode.containsVariableAssignment() ||
argsNode != null && argsNode.containsVariableAssignment() ||
iterNode != null && iterNode.containsVariableAssignment());

assert receiverNode != null : "receiverNode is not null";

this.name = name;
@@ -116,6 +124,10 @@ public Node setArgsNode(Node argsNode) {
* @return name
*/
public String getName() {
return StringSupport.byteListAsString(name);
}

public ByteList getByteName() {
return name;
}

17 changes: 14 additions & 3 deletions core/src/main/java/org/jruby/ast/ClassVarAsgnNode.java
Original file line number Diff line number Diff line change
@@ -36,23 +36,30 @@
import org.jruby.ast.types.INameNode;
import org.jruby.ast.visitor.NodeVisitor;
import org.jruby.lexer.yacc.ISourcePosition;
import org.jruby.util.ByteList;
import org.jruby.util.StringSupport;

/**
* Class variable assignment node.
*/
public class ClassVarAsgnNode extends AssignableNode implements INameNode {
private String name;
private ByteList name;

/**
* @param name id of the class variable to assign to
* @param valueNode Node used to compute the new value when the assignment is evaled
*/
public ClassVarAsgnNode(ISourcePosition position, String name, Node valueNode) {
public ClassVarAsgnNode(ISourcePosition position, ByteList name, Node valueNode) {
super(position, valueNode, valueNode != null && valueNode.containsVariableAssignment());

this.name = name;
}

@Deprecated
public ClassVarAsgnNode(ISourcePosition position, String name, Node valueNode) {
this(position, StringSupport.stringAsByteList(name), valueNode);
}

public NodeType getNodeType() {
return NodeType.CLASSVARASGNNODE;
}
@@ -70,9 +77,13 @@ public <T> T accept(NodeVisitor<T> iVisitor) {
* @return Returns a String
*/
public String getName() {
return StringSupport.byteListAsString(name);
}

public ByteList getByteName() {
return name;
}

public List<Node> childNodes() {
return createList(getValueNode());
}
Loading