Skip to content

Commit

Permalink
Showing 41 changed files with 8,909 additions and 564 deletions.
1 change: 0 additions & 1 deletion spec/truffle/tags/core/encoding/aliases_tags.txt

This file was deleted.

Original file line number Diff line number Diff line change
@@ -1,6 +1 @@
fails:Encoding::Converter.asciicompat_encoding accepts an encoding name as a String argument
fails:Encoding::Converter.asciicompat_encoding coerces non-String/Encoding objects with #to_str
fails:Encoding::Converter.asciicompat_encoding accepts an Encoding object as an argument
fails:Encoding::Converter.asciicompat_encoding returns a corresponding ASCII compatible encoding for ASCII-incompatible encodings
fails:Encoding::Converter.asciicompat_encoding returns nil when the given encoding is ASCII compatible
fails:Encoding::Converter.asciicompat_encoding handles encoding names who resolve to nil encodings
26 changes: 0 additions & 26 deletions spec/truffle/tags/core/encoding/converter/constants_tags.txt

This file was deleted.

5 changes: 0 additions & 5 deletions spec/truffle/tags/core/encoding/converter/convert_tags.txt
Original file line number Diff line number Diff line change
@@ -1,7 +1,2 @@
fails:Encoding::Converter#convert returns a String
fails:Encoding::Converter#convert sets the encoding of the result to the target encoding
fails:Encoding::Converter#convert transcodes the given String to the target encoding
fails:Encoding::Converter#convert allows Strings of different encodings to the source encoding
fails:Encoding::Converter#convert reuses the given encoding pair if called multiple times
fails:Encoding::Converter#convert raises UndefinedConversionError if the String contains characters invalid for the target encoding
fails:Encoding::Converter#convert raises an ArgumentError if called on a finished stream
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
fails:Encoding::Converter#convpath indicates if crlf_newline conversion would occur
slow:Encoding::Converter#convpath returns an Array
slow:Encoding::Converter#convpath returns each encoding pair as a sub-Array
slow:Encoding::Converter#convpath returns each encoding as an Encoding object

This file was deleted.

1 change: 0 additions & 1 deletion spec/truffle/tags/core/encoding/converter/inspect_tags.txt
Original file line number Diff line number Diff line change
@@ -1,2 +1 @@
fails:Encoding::Converter#inspect needs to be reviewed for spec completeness
fails:Encoding::Converter#inspect includes the source and destination encodings in the return value
4 changes: 0 additions & 4 deletions spec/truffle/tags/core/encoding/converter/last_error_tags.txt
Original file line number Diff line number Diff line change
@@ -1,7 +1,3 @@
fails:Encoding::Converter#last_error returns nil when the no conversion has been attempted
fails:Encoding::Converter#last_error returns nil when the last conversion did not produce an error
fails:Encoding::Converter#last_error returns nil when #primitive_convert last returned :destination_buffer_full
fails:Encoding::Converter#last_error returns nil when #primitive_convert last returned :finished
fails:Encoding::Converter#last_error returns nil if the last conversion succeeded but the penultimate failed
fails:Encoding::Converter#last_error returns an Encoding::InvalidByteSequenceError when #primitive_convert last returned :invalid_byte_sequence
fails:Encoding::Converter#last_error returns an Encoding::UndefinedConversionError when #primitive_convert last returned :undefined_conversion
17 changes: 0 additions & 17 deletions spec/truffle/tags/core/encoding/converter/new_tags.txt
Original file line number Diff line number Diff line change
@@ -1,19 +1,2 @@
fails:Encoding::Converter.new accepts a String for the source encoding
fails:Encoding::Converter.new accepts a String for the destination encoding
fails:Encoding::Converter.new accepts an Encoding object for the source encoding
fails:Encoding::Converter.new accepts an Encoding object for the destination encoding
fails:Encoding::Converter.new raises an Encoding::ConverterNotFoundError if both encodings are the same
fails:Encoding::Converter.new calls #to_str to convert the source encoding argument to an encoding name
fails:Encoding::Converter.new calls #to_str to convert the destination encoding argument to an encoding name
fails:Encoding::Converter.new sets replacement from the options Hash
fails:Encoding::Converter.new calls #to_hash to convert the options argument to a Hash if not a Fixnum
fails:Encoding::Converter.new calls #to_str to convert the replacement object to a String
fails:Encoding::Converter.new raises a TypeError if #to_str does not return a String
fails:Encoding::Converter.new raises a TypeError if passed true for the replacement object
fails:Encoding::Converter.new raises a TypeError if passed false for the replacement object
fails:Encoding::Converter.new raises a TypeError if passed a Fixnum for the replacement object
fails:Encoding::Converter.new accepts an empty String for the replacement object
fails:Encoding::Converter.new when passed nil for the replacement object when the destination encoding is not UTF-8 sets the replacement String to '?'
fails:Encoding::Converter.new when passed nil for the replacement object when the destination encoding is not UTF-8 sets the replacement String encoding to US-ASCII
fails:Encoding::Converter.new when passed nil for the replacement object when the destination encoding is not UTF-8 sets the replacement String to '\uFFFD'
fails:Encoding::Converter.new when passed nil for the replacement object when the destination encoding is not UTF-8 sets the replacement String encoding to UTF-8
Original file line number Diff line number Diff line change
@@ -1,31 +1,4 @@
fails:Encoding::Converter#primitive_convert accepts a nil source buffer
fails:Encoding::Converter#primitive_convert accepts a String as the source buffer
fails:Encoding::Converter#primitive_convert accepts nil for the destination byte offset
fails:Encoding::Converter#primitive_convert accepts an integer for the destination byte offset
fails:Encoding::Converter#primitive_convert calls #to_int to convert the destination byte offset
fails:Encoding::Converter#primitive_convert raises an ArgumentError if the destination byte offset is greater than the bytesize of the destination buffer
fails:Encoding::Converter#primitive_convert uses the destination byte offset to determine where to write the result in the destination buffer
fails:Encoding::Converter#primitive_convert accepts nil for the destination bytesize
fails:Encoding::Converter#primitive_convert accepts an integer for the destination bytesize
fails:Encoding::Converter#primitive_convert allows a destination bytesize value greater than the bytesize of the source buffer
fails:Encoding::Converter#primitive_convert allows a destination bytesize value less than the bytesize of the source buffer
fails:Encoding::Converter#primitive_convert calls #to_int to convert the destination byte size
fails:Encoding::Converter#primitive_convert uses destination bytesize as the maximumn bytesize of the destination buffer
fails:Encoding::Converter#primitive_convert allows a destination buffer of unlimited size if destination bytesize is nil
fails:Encoding::Converter#primitive_convert accepts an options hash
fails:Encoding::Converter#primitive_convert sets the destination buffer's encoding to the destination encoding if the conversion suceeded
fails:Encoding::Converter#primitive_convert sets the destination buffer's encoding to the destination encoding if the conversion failed
fails:Encoding::Converter#primitive_convert removes the undefined part from the source buffer when returning :undefined_conversion
fails:Encoding::Converter#primitive_convert returns :incomplete_input when source buffer ends unexpectedly and :partial_input isn't specified
fails:Encoding::Converter#primitive_convert clears the source buffer when returning :incomplete_input
fails:Encoding::Converter#primitive_convert returns :source_buffer_empty when source buffer ends unexpectedly and :partial_input is true
fails:Encoding::Converter#primitive_convert clears the source buffer when returning :source_buffer_empty
fails:Encoding::Converter#primitive_convert returns :undefined_conversion when a character in the source buffer is not representable in the output encoding
fails:Encoding::Converter#primitive_convert returns :invalid_byte_sequence when an invalid byte sequence was found in the source buffer
fails:Encoding::Converter#primitive_convert removes consumed and erroneous bytes from the source buffer when returning :invalid_byte_sequence
fails:Encoding::Converter#primitive_convert returns :finished when the conversion succeeded
fails:Encoding::Converter#primitive_convert clears the source buffer when returning :finished
fails:Encoding::Converter#primitive_convert returns :destination_buffer_full when the destination buffer is too small
fails:Encoding::Converter#primitive_convert clears the source buffer when returning :destination_buffer_full
fails:Encoding::Converter#primitive_convert keeps removing invalid bytes from the source buffer
fails:Encoding::Converter#primitive_convert reuses read-again bytes after the first error
Original file line number Diff line number Diff line change
@@ -1,10 +1,3 @@
fails:Encoding::Converter#primitive_errinfo returns [:source_buffer_empty,nil,nil,nil,nil] when no conversion has been attempted
fails:Encoding::Converter#primitive_errinfo returns [:finished,nil,nil,nil,nil] when #primitive_convert last returned :finished
fails:Encoding::Converter#primitive_errinfo returns [:source_buffer_empty,nil,nil,nil, nil] when #convert last succeeded
fails:Encoding::Converter#primitive_errinfo returns [:destination_buffer_full,nil,nil,nil,nil] when #primitive_convert last returned :destination_buffer_full
fails:Encoding::Converter#primitive_errinfo returns the status of the last primitive conversion, even if it was successful and the previous one wasn't
fails:Encoding::Converter#primitive_errinfo returns the state, source encoding, target encoding, and the erroneous bytes when #primitive_convert last returned :undefined_conversion
fails:Encoding::Converter#primitive_errinfo returns the state, source encoding, target encoding, and erroneous bytes when #primitive_convert last returned :incomplete_input
fails:Encoding::Converter#primitive_errinfo returns the state, source encoding, target encoding, erroneous bytes, and the read-again bytes when #primitive_convert last returned :invalid_byte_sequence
fails:Encoding::Converter#primitive_errinfo returns the state, source encoding, target encoding, erroneous bytes, and the read-again bytes when #convert last raised InvalidByteSequenceError
fails:Encoding::Converter#primitive_errinfo returns the state, source encoding, target encoding, erroneous bytes, and the read-again bytes when #finish last raised InvalidByteSequenceError
Original file line number Diff line number Diff line change
@@ -1,8 +1,2 @@
fails:Encoding::Converter#replacement returns '?' in US-ASCII when the destination encoding is not UTF-8
fails:Encoding::Converter#replacement returns \uFFFD when the destination encoding is UTF-8
fails:Encoding::Converter#replacement= accepts a String argument
fails:Encoding::Converter#replacement= accepts a String argument of arbitrary length
fails:Encoding::Converter#replacement= raises a TypeError if assigned a non-String argument
fails:Encoding::Converter#replacement= sets #replacement
fails:Encoding::Converter#replacement= raises an UndefinedConversionError is the argument cannot be converted into the destination encoding
fails:Encoding::Converter#replacement= does not change the replacement character if the argument cannot be converted into the destination encoding
Original file line number Diff line number Diff line change
@@ -1,6 +1,3 @@
fails:Encoding::Converter.search_convpath returns multiple encoding pairs when direct conversion is impossible
fails:Encoding::Converter.search_convpath indicates if crlf_newline conversion would occur
fails:Encoding::Converter.search_convpath raises an Encoding::ConverterNotFoundError if no conversion path exists
slow:Encoding::Converter.search_convpath returns an Array
slow:Encoding::Converter.search_convpath returns each encoding pair as a sub-Array
slow:Encoding::Converter.search_convpath returns each encoding as an Encoding object

This file was deleted.

1 change: 0 additions & 1 deletion spec/truffle/tags/core/encoding/default_external_tags.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
fails:Encoding.default_external with command line options is not changed by the -U option
fails:Encoding.default_external= calls #to_s on arguments that are neither Strings nor Encodings
windows:Encoding.default_external with command line options returns the encoding specified by '-E external'
windows:Encoding.default_external with command line options returns the encoding specified by '-E external:'
9 changes: 0 additions & 9 deletions spec/truffle/tags/core/encoding/find_tags.txt

This file was deleted.

Original file line number Diff line number Diff line change
@@ -1,3 +1 @@
fails:Encoding::InvalidByteSequenceError#incomplete_input? returns nil by default
fails:Encoding::InvalidByteSequenceError#incomplete_input? returns true if #primitive_convert returned :incomplete_input for the same data
fails:Encoding::InvalidByteSequenceError#incomplete_input? returns false if #primitive_convert returned :invalid_byte_sequence for the same data
4 changes: 0 additions & 4 deletions spec/truffle/tags/core/encoding/names_tags.txt

This file was deleted.

1 change: 0 additions & 1 deletion spec/truffle/tags/core/string/append_tags.txt

This file was deleted.

1 change: 0 additions & 1 deletion spec/truffle/tags/core/string/clear_tags.txt

This file was deleted.

1 change: 0 additions & 1 deletion spec/truffle/tags/core/string/concat_tags.txt

This file was deleted.

65 changes: 1 addition & 64 deletions spec/truffle/tags/core/string/encode_tags.txt
Original file line number Diff line number Diff line change
@@ -1,79 +1,16 @@
fails:String#encode when passed no options transcodes a 7-bit String despite no generic converting being available
fails:String#encode when passed no options raises an Encoding::ConverterNotFoundError when no conversion is possible
fails:String#encode when passed to encoding transcodes a 7-bit String despite no generic converting being available
fails:String#encode when passed to encoding raises an Encoding::ConverterNotFoundError when no conversion is possible
fails:String#encode when passed to encoding raises an Encoding::ConverterNotFoundError for an invalid encoding
fails:String#encode when passed options does not process transcoding options if not transcoding
fails:String#encode when passed options calls #to_hash to convert the object
fails:String#encode when passed options transcodes to Encoding.default_internal when set
fails:String#encode when passed options raises an Encoding::ConverterNotFoundError when no conversion is possible despite ':invalid => :replace, :undef => :replace'
fails:String#encode when passed to, from transcodes between the encodings ignoring the String encoding
fails:String#encode when passed to, from calls #to_str to convert the from object to an Encoding
fails:String#encode when passed to, options replaces undefined characters in the destination encoding
fails:String#encode when passed to, options replaces invalid characters in the destination encoding
fails:String#encode when passed to, options calls #to_hash to convert the options object
fails:String#encode when passed to, from, options replaces undefined characters in the destination encoding
fails:String#encode when passed to, from, options replaces invalid characters in the destination encoding
fails:String#encode when passed to, from, options calls #to_str to convert the to object to an encoding
fails:String#encode when passed to, from, options calls #to_str to convert the from object to an encoding
fails:String#encode when passed to, from, options calls #to_hash to convert the options object
fails:String#encode given the :xml => :text option replaces all instances of '&' with '&'
fails:String#encode given the :xml => :text option replaces all instances of '<' with '&lt;'
fails:String#encode given the :xml => :text option replaces all instances of '>' with '&gt;'
fails:String#encode given the :xml => :text option replaces undefined characters with their upper-case hexadecimal numeric character references
fails:String#encode given the :xml => :attr option surrounds the encoded text with double-quotes
fails:String#encode given the :xml => :attr option replaces all instances of '&' with '&amp;'
fails:String#encode given the :xml => :attr option replaces all instances of '<' with '&lt;'
fails:String#encode given the :xml => :attr option replaces all instances of '>' with '&gt;'
fails:String#encode given the :xml => :attr option replaces all instances of '"' with '&quot;'
fails:String#encode given the :xml => :attr option replaces undefined characters with their upper-case hexadecimal numeric character references
fails:String#encode when passed options returns a copy when Encoding.default_internal is nil
fails:String#encode when passed options normalizes newlines
fails:String#encode when passed to, from returns a copy when both encodings are the same
fails:String#encode when passed to, from returns the transcoded string
fails:String#encode when passed to, options returns a copy when the destination encoding is the same as the String encoding
fails:String#encode when passed to, from, options returns a copy when both encodings are the same
fails:String#encode! raises ArgumentError if the value of the :xml option is not :text or :attr
fails:String#encode! raises a RuntimeError when called on a frozen String
fails:String#encode! raises a RuntimeError when called on a frozen String when it's a no-op
fails:String#encode! when passed no options transcodes to Encoding.default_internal when set
fails:String#encode! when passed no options transcodes a 7-bit String despite no generic converting being available
fails:String#encode! when passed no options raises an Encoding::ConverterNotFoundError when no conversion is possible
fails:String#encode! when passed to encoding accepts a String argument
fails:String#encode! when passed to encoding calls #to_str to convert the object to an Encoding
fails:String#encode! when passed to encoding transcodes to the passed encoding
fails:String#encode! when passed to encoding transcodes Japanese multibyte characters
fails:String#encode! when passed to encoding transcodes a 7-bit String despite no generic converting being available
fails:String#encode! when passed to encoding raises an Encoding::ConverterNotFoundError when no conversion is possible
fails:String#encode! when passed to encoding raises an Encoding::ConverterNotFoundError for an invalid encoding
fails:String#encode! when passed options does not process transcoding options if not transcoding
fails:String#encode! when passed options calls #to_hash to convert the object
fails:String#encode! when passed options transcodes to Encoding.default_internal when set
fails:String#encode! when passed options raises an Encoding::ConverterNotFoundError when no conversion is possible despite ':invalid => :replace, :undef => :replace'
fails:String#encode! when passed to, from transcodes between the encodings ignoring the String encoding
fails:String#encode! when passed to, from calls #to_str to convert the from object to an Encoding
fails:String#encode! when passed to, options replaces undefined characters in the destination encoding
fails:String#encode! when passed to, options replaces invalid characters in the destination encoding
fails:String#encode! when passed to, options calls #to_hash to convert the options object
fails:String#encode! when passed to, from, options replaces undefined characters in the destination encoding
fails:String#encode! when passed to, from, options replaces invalid characters in the destination encoding
fails:String#encode! when passed to, from, options calls #to_str to convert the to object to an encoding
fails:String#encode! when passed to, from, options calls #to_str to convert the from object to an encoding
fails:String#encode! when passed to, from, options calls #to_hash to convert the options object
fails:String#encode! given the :xml => :text option replaces all instances of '&' with '&amp;'
fails:String#encode! given the :xml => :text option replaces all instances of '<' with '&lt;'
fails:String#encode! given the :xml => :text option replaces all instances of '>' with '&gt;'
fails:String#encode! given the :xml => :text option does not replace '"'
fails:String#encode! given the :xml => :text option replaces undefined characters with their upper-case hexadecimal numeric character references
fails:String#encode! given the :xml => :attr option surrounds the encoded text with double-quotes
fails:String#encode! given the :xml => :attr option replaces all instances of '&' with '&amp;'
fails:String#encode! given the :xml => :attr option replaces all instances of '<' with '&lt;'
fails:String#encode! given the :xml => :attr option replaces all instances of '>' with '&gt;'
fails:String#encode! given the :xml => :attr option replaces all instances of '"' with '&quot;'
fails:String#encode! given the :xml => :attr option replaces undefined characters with their upper-case hexadecimal numeric character references
fails:String#encode! when passed no options returns self when Encoding.default_internal is nil
fails:String#encode! when passed no options returns self for a ASCII-only String when Encoding.default_internal is nil
fails:String#encode! when passed options returns self for ASCII-only String when Encoding.default_internal is nil
fails:String#encode! when passed to encoding returns self
fails:String#encode! when passed to, from returns self
fails:String#encode raises ArgumentError if the value of the :xml option is not :text or :attr
fails:String#encode when passed to encoding transcodes Japanese multibyte characters
3 changes: 0 additions & 3 deletions spec/truffle/tags/core/string/encoding_tags.txt

This file was deleted.

1 change: 0 additions & 1 deletion spec/truffle/tags/core/string/start_with_tags.txt

This file was deleted.

1 change: 0 additions & 1 deletion spec/truffle/tags/core/string/upto_tags.txt

This file was deleted.

Original file line number Diff line number Diff line change
@@ -11,72 +11,36 @@

import com.oracle.truffle.api.CompilerDirectives.TruffleBoundary;
import com.oracle.truffle.api.dsl.Specialization;
import com.oracle.truffle.api.frame.VirtualFrame;
import com.oracle.truffle.api.source.SourceSection;

import org.jcodings.Encoding;
import org.jcodings.EncodingDB;
import org.jcodings.transcode.EConv;
import org.jcodings.transcode.Transcoder;
import org.jcodings.transcode.TranscoderDB;
import org.jruby.Ruby;
import org.jruby.runtime.ThreadContext;
import org.jruby.runtime.Visibility;
import org.jruby.runtime.builtin.IRubyObject;
import org.jruby.truffle.nodes.dispatch.CallDispatchHeadNode;
import org.jruby.truffle.nodes.dispatch.DispatchHeadNodeFactory;
import org.jruby.truffle.runtime.RubyContext;
import org.jruby.truffle.runtime.UndefinedPlaceholder;
import org.jruby.truffle.runtime.core.*;
import org.jruby.truffle.runtime.hash.HashOperations;
import org.jruby.truffle.runtime.hash.KeyValue;
import org.jruby.util.ByteList;
import org.jruby.util.io.EncodingUtils;

import java.util.ArrayList;
import java.util.List;

@CoreClass(name = "Encoding::Converter")
public abstract class EncodingConverterNodes {

@CoreMethod(names = "convpath")
public abstract static class ConvPathNode extends CoreMethodNode {

public ConvPathNode(RubyContext context, SourceSection sourceSection) {
super(context, sourceSection);
}

public ConvPathNode(ConvPathNode prev) {
super(prev);
}

@TruffleBoundary
@Specialization
public RubyArray convpath(RubyEncodingConverter converter) {
notDesignedForCompilation();

// Adapated from RubyConverter - see attribution there

Ruby runtime = getContext().getRuntime();

EConv ec = converter.getEConv();

Object[] result = new Object[ec.numTranscoders];
int r = 0;

for (int i = 0; i < ec.numTranscoders; i++) {
Transcoder tr = ec.elements[i].transcoding.transcoder;
Object v;
if (EncodingUtils.DECORATOR_P(tr.getSource(), tr.getDestination())) {
v = new RubyString(getContext().getCoreLibrary().getStringClass(), new ByteList(tr.getDestination()));
} else {
Encoding source = runtime.getEncodingService().findEncodingOrAliasEntry(tr.getSource()).getEncoding();
Encoding destination = runtime.getEncodingService().findEncodingOrAliasEntry(tr.getDestination()).getEncoding();

v = new RubyArray(getContext().getCoreLibrary().getArrayClass(),
new Object[]{
RubyEncoding.getEncoding(source),
RubyEncoding.getEncoding(destination)
}, 2);
}
result[r++] = v;
}

return new RubyArray(getContext().getCoreLibrary().getArrayClass(), result, result.length);
}

}

@CoreMethod(names = "initialize", required = 2)
@RubiniusOnly
@CoreMethod(names = "initialize_jruby", required = 2, optional = 1, visibility = Visibility.PRIVATE)
public abstract static class InitializeNode extends CoreMethodNode {

public InitializeNode(RubyContext context, SourceSection sourceSection) {
@@ -89,7 +53,7 @@ public InitializeNode(InitializeNode prev) {

@TruffleBoundary
@Specialization
public RubyNilClass initialize(RubyEncodingConverter self, RubyString source, RubyString destination) {
public RubyNilClass initialize(RubyEncodingConverter self, Object source, Object destination, Object options) {
notDesignedForCompilation();

// Adapted from RubyConverter - see attribution there
@@ -100,86 +64,79 @@ public RubyNilClass initialize(RubyEncodingConverter self, RubyString source, Ru
int[] ecflags = {0};
IRubyObject[] ecopts = {runtime.getNil()};

EncodingUtils.econvArgs(runtime.getCurrentContext(), new IRubyObject[]{getContext().toJRuby(source), getContext().toJRuby(destination)}, encNames, encs, ecflags, ecopts);
final IRubyObject sourceAsJRubyObj = getContext().toJRuby(source);
final IRubyObject destinationAsJRubyObj = getContext().toJRuby(destination);

EncodingUtils.econvArgs(runtime.getCurrentContext(), new IRubyObject[]{sourceAsJRubyObj, destinationAsJRubyObj}, encNames, encs, ecflags, ecopts);
EConv econv = EncodingUtils.econvOpenOpts(runtime.getCurrentContext(), encNames[0], encNames[1], ecflags[0], ecopts[0]);

if (econv == null) {
throw new UnsupportedOperationException();
}

if (!EncodingUtils.DECORATOR_P(encNames[0], encNames[1])) {
if (encs[0] == null) {
encs[0] = EncodingDB.dummy(encNames[0]).getEncoding();
}
if (encs[1] == null) {
encs[1] = EncodingDB.dummy(encNames[1]).getEncoding();
}
}

econv.sourceEncoding = encs[0];
econv.destinationEncoding = encs[1];

self.setEConv(econv);

return nil();
}

}

@CoreMethod(names = "search_convpath", onSingleton = true, required = 2)
public abstract static class SearchConvPathNode extends CoreMethodNode {
@RubiniusOnly
@CoreMethod(names = "transcoding_map", onSingleton = true)
public abstract static class TranscodingMapNode extends CoreMethodNode {

@Child private CallDispatchHeadNode upcaseNode;
@Child private CallDispatchHeadNode toSymNode;
@Child private CallDispatchHeadNode newLookupTableNode;
@Child private CallDispatchHeadNode lookupTableWriteNode;

public SearchConvPathNode(RubyContext context, SourceSection sourceSection) {
public TranscodingMapNode(RubyContext context, SourceSection sourceSection) {
super(context, sourceSection);
upcaseNode = DispatchHeadNodeFactory.createMethodCall(context);
toSymNode = DispatchHeadNodeFactory.createMethodCall(context);
newLookupTableNode = DispatchHeadNodeFactory.createMethodCall(context);
lookupTableWriteNode = DispatchHeadNodeFactory.createMethodCall(context);
}

public SearchConvPathNode(SearchConvPathNode prev) {
public TranscodingMapNode(TranscodingMapNode prev) {
super(prev);
upcaseNode = prev.upcaseNode;
toSymNode = prev.toSymNode;
newLookupTableNode = prev.newLookupTableNode;
lookupTableWriteNode = prev.lookupTableWriteNode;
}

@TruffleBoundary
@Specialization
public RubyArray searchConvpath(RubyString source, RubyString destination) {
notDesignedForCompilation();
public RubyHash transcodingMap(VirtualFrame frame) {
List<KeyValue> entries = new ArrayList<>();

// Adapted from RubyConverter - see attribution there

final Ruby runtime = getContext().getRuntime();
final RubyNilClass nil = nil();
ThreadContext context = runtime.getCurrentContext();
final byte[][] encNames = {null, null};
final Encoding[] encs = {null, null};
final int[] ecflags_p = {0};
final IRubyObject[] ecopts_p = {context.nil};
final Object[] convpath = {nil()};

EncodingUtils.econvArgs(context, new IRubyObject[]{getContext().toJRuby(source), getContext().toJRuby(destination)}, encNames, encs, ecflags_p, ecopts_p);

TranscoderDB.searchPath(encNames[0], encNames[1], new TranscoderDB.SearchPathCallback() {

public void call(byte[] source, byte[] destination, int depth) {
Object v;
for (RubyEncoding e : RubyEncoding.cloneEncodingList()) {
final Object upcased = upcaseNode.call(frame, getContext().makeString(e.getName()), "upcase", null);
final Object key = toSymNode.call(frame, upcased, "to_sym", null);
final Object value = newLookupTableNode.call(frame, getContext().getCoreLibrary().getLookupTableClass(), "new", null);

if (convpath[0] == nil) {
convpath[0] = new RubyArray(getContext().getCoreLibrary().getArrayClass(), null, 0);
}
final Object tupleValues = new Object[2];

if (EncodingUtils.DECORATOR_P(encNames[0], encNames[1])) {
v = new RubyString(getContext().getCoreLibrary().getStringClass(), new ByteList(encNames[2]));
} else {
Encoding sourceEncoding = runtime.getEncodingService().findEncodingOrAliasEntry(source).getEncoding();
Encoding destinationEncoding = runtime.getEncodingService().findEncodingOrAliasEntry(destination).getEncoding();

v = new RubyArray(getContext().getCoreLibrary().getArrayClass(),
new Object[]{
RubyEncoding.getEncoding(destinationEncoding),
RubyEncoding.getEncoding(sourceEncoding)
}, 2);
}
lookupTableWriteNode.call(frame, value, "[]=", null, key, nil());

((RubyArray) convpath[0]).slowPush(v); // depth?
}
});

if (convpath[0] == nil) {
throw new UnsupportedOperationException();
entries.add(new KeyValue(key, value));
}

//if (EncodingUtils.decorateConvpath(context, convpath[0], ecflags_p[0]) == -1) {
// throw EncodingUtils.econvOpenExc(context, encNames[0], encNames[1], ecflags_p[0]);
//}

return (RubyArray) convpath[0];
return HashOperations.verySlowFromEntries(getContext().getCoreLibrary().getHashClass(), entries, true);
}

}

}
288 changes: 97 additions & 191 deletions truffle/src/main/java/org/jruby/truffle/nodes/core/EncodingNodes.java
Original file line number Diff line number Diff line change
@@ -30,9 +30,13 @@
import org.jruby.truffle.nodes.RubyNode;
import org.jruby.truffle.nodes.coerce.ToStrNode;
import org.jruby.truffle.nodes.coerce.ToStrNodeFactory;
import org.jruby.truffle.nodes.dispatch.CallDispatchHeadNode;
import org.jruby.truffle.nodes.dispatch.DispatchHeadNode;
import org.jruby.truffle.nodes.dispatch.DispatchHeadNodeFactory;
import org.jruby.truffle.runtime.RubyContext;
import org.jruby.truffle.runtime.control.RaiseException;
import org.jruby.truffle.runtime.core.RubyArray;
import org.jruby.truffle.runtime.core.RubyBasicObject;
import org.jruby.truffle.runtime.core.RubyEncoding;
import org.jruby.truffle.runtime.core.RubyHash;
import org.jruby.truffle.runtime.core.RubyNilClass;
@@ -49,48 +53,6 @@
@CoreClass(name = "Encoding")
public abstract class EncodingNodes {

@CoreMethod(names = "aliases", needsSelf = false, onSingleton = true, required = 0)
public abstract static class AliasesNode extends CoreMethodNode {

public AliasesNode(RubyContext context, SourceSection sourceSection) {
super(context, sourceSection);
}

public AliasesNode(AliasesNode prev) {
super(prev);
}

@TruffleBoundary
@Specialization
public RubyHash aliases() {
notDesignedForCompilation();

final List<KeyValue> aliases = new ArrayList<>();

final Hash.HashEntryIterator i = getContext().getRuntime().getEncodingService().getAliases().entryIterator();
while (i.hasNext()) {
final CaseInsensitiveBytesHash.CaseInsensitiveBytesHashEntry<EncodingDB.Entry> e =
((CaseInsensitiveBytesHash.CaseInsensitiveBytesHashEntry<EncodingDB.Entry>)i.next());

final RubyString alias = getContext().makeString(new ByteList(e.bytes, e.p, e.end - e.p));
alias.freeze();

final RubyString name = getContext().makeString(RubyEncoding.getEncoding(e.value.getIndex()).getName());
name.freeze();

aliases.add(new KeyValue(alias, name));
}

aliases.add(new KeyValue(getContext().makeString("external"),
getContext().makeString(new ByteList(getContext().getRuntime().getDefaultExternalEncoding().getName()))));

aliases.add(new KeyValue(getContext().makeString("locale"),
getContext().makeString(new ByteList(getContext().getRuntime().getEncodingService().getLocaleEncoding().getName()))));

return HashOperations.verySlowFromEntries(getContext(), aliases, false);
}
}

@CoreMethod(names = "ascii_compatible?")
public abstract static class AsciiCompatibleNode extends CoreMethodNode {

@@ -230,60 +192,8 @@ public Object isCompatible(RubySymbol first, RubySymbol second) {

}

@CoreMethod(names = "default_external", onSingleton = true)
public abstract static class DefaultExternalNode extends CoreMethodNode {

public DefaultExternalNode(RubyContext context, SourceSection sourceSection) {
super(context, sourceSection);
}

public DefaultExternalNode(DefaultExternalNode prev) {
super(prev);
}

@Specialization
public RubyEncoding defaultExternal() {
notDesignedForCompilation();

Encoding encoding = getContext().getRuntime().getDefaultExternalEncoding();

if (encoding == null) {
encoding = UTF8Encoding.INSTANCE;
}

return RubyEncoding.getEncoding(encoding);
}

}

@CoreMethod(names = "default_internal", onSingleton = true)
public abstract static class DefaultInternalNode extends CoreMethodNode {

public DefaultInternalNode(RubyContext context, SourceSection sourceSection) {
super(context, sourceSection);
}

public DefaultInternalNode(DefaultInternalNode prev) {
super(prev);
}

@Specialization
public Object defaultInternal() {
notDesignedForCompilation();

Encoding encoding = getContext().getRuntime().getDefaultInternalEncoding();

if (encoding == null) {
return nil();
//encoding = UTF8Encoding.INSTANCE;
}

return RubyEncoding.getEncoding(encoding);
}

}

@CoreMethod(names = "default_external=", onSingleton = true, required = 1)
@RubiniusOnly
@CoreMethod(names = "default_external_jruby=", onSingleton = true, required = 1)
public abstract static class SetDefaultExternalNode extends CoreMethodNode {

public SetDefaultExternalNode(RubyContext context, SourceSection sourceSection) {
@@ -320,7 +230,8 @@ public RubyEncoding defaultExternal(RubyNilClass nil) {

}

@CoreMethod(names = "default_internal=", onSingleton = true, required = 1)
@RubiniusOnly
@CoreMethod(names = "default_internal_jruby=", onSingleton = true, required = 1)
public abstract static class SetDefaultInternalNode extends CoreMethodNode {

@Child private ToStrNode toStrNode;
@@ -368,79 +279,6 @@ public RubyString defaultInternal(VirtualFrame frame, Object encoding) {

}

@CoreMethod(names = "find", onSingleton = true, required = 1)
@NodeChild(value = "name")
public abstract static class FindNode extends RubyNode {

public FindNode(RubyContext context, SourceSection sourceSection) {
super(context, sourceSection);
}

public FindNode(FindNode prev) {
super(prev);
}

@CreateCast("name") public RubyNode coerceNameToString(RubyNode name) {
return ToStrNodeFactory.create(getContext(), getSourceSection(), name);
}

@Specialization
public RubyEncoding find(RubyString name) {
notDesignedForCompilation();

return RubyEncoding.getEncoding(name.toString());
}

}

@CoreMethod(names = "name_list", onSingleton = true)
public abstract static class NameListNode extends CoreMethodNode {

public NameListNode(RubyContext context, SourceSection sourceSection) {
super(context, sourceSection);
}

public NameListNode(NameListNode prev) {
super(prev);
}

@TruffleBoundary
@Specialization
public RubyArray find() {
notDesignedForCompilation();

final EncodingService service = getContext().getRuntime().getEncodingService();

final Object[] array = new Object[service.getEncodings().size() + service.getAliases().size() + 2];
int n = 0;

Hash.HashEntryIterator i;

i = service.getEncodings().entryIterator();

while (i.hasNext()) {
CaseInsensitiveBytesHash.CaseInsensitiveBytesHashEntry<EncodingDB.Entry> e =
((CaseInsensitiveBytesHash.CaseInsensitiveBytesHashEntry<EncodingDB.Entry>)i.next());
array[n++] = new RubyString(getContext().getCoreLibrary().getStringClass(), new ByteList(e.bytes, e.p, e.end - e.p));
}

i = service.getAliases().entryIterator();

while (i.hasNext()) {
CaseInsensitiveBytesHash.CaseInsensitiveBytesHashEntry<EncodingDB.Entry> e =
((CaseInsensitiveBytesHash.CaseInsensitiveBytesHashEntry<EncodingDB.Entry>)i.next());
array[n++] = new RubyString(getContext().getCoreLibrary().getStringClass(), new ByteList(e.bytes, e.p, e.end - e.p));
}

array[n++] = new RubyString(getContext().getCoreLibrary().getStringClass(), org.jruby.RubyEncoding.EXTERNAL);
//array[n++] = new RubyString(getContext().getCoreLibrary().getStringClass(), org.jruby.RubyEncoding.INTERNAL);
array[n++] = new RubyString(getContext().getCoreLibrary().getStringClass(), org.jruby.RubyEncoding.LOCALE);
//array[n++] = new RubyString(getContext().getCoreLibrary().getStringClass(), org.jruby.RubyEncoding.FILESYSTEM);

return new RubyArray(getContext().getCoreLibrary().getArrayClass(), array, array.length);
}
}

@CoreMethod(names = "list", onSingleton = true)
public abstract static class ListNode extends CoreMethodNode {

@@ -501,48 +339,116 @@ public boolean isDummy(RubyEncoding encoding) {
}
}

@CoreMethod(names = { "name", "to_s" })
public abstract static class ToSNode extends CoreMethodNode {
@RubiniusOnly
@CoreMethod(names = "encoding_map", onSingleton = true)
public abstract static class EncodingMapNode extends CoreMethodNode {

public ToSNode(RubyContext context, SourceSection sourceSection) {
@Child private CallDispatchHeadNode upcaseNode;
@Child private CallDispatchHeadNode toSymNode;
@Child private CallDispatchHeadNode newLookupTableNode;
@Child private CallDispatchHeadNode lookupTableWriteNode;
@Child private CallDispatchHeadNode newTupleNode;

public EncodingMapNode(RubyContext context, SourceSection sourceSection) {
super(context, sourceSection);
upcaseNode = DispatchHeadNodeFactory.createMethodCall(context);
toSymNode = DispatchHeadNodeFactory.createMethodCall(context);
newLookupTableNode = DispatchHeadNodeFactory.createMethodCall(context);
lookupTableWriteNode = DispatchHeadNodeFactory.createMethodCall(context);
newTupleNode = DispatchHeadNodeFactory.createMethodCall(context);
}

public ToSNode(ToSNode prev) {
public EncodingMapNode(EncodingMapNode prev) {
super(prev);
upcaseNode = prev.upcaseNode;
toSymNode = prev.toSymNode;
newLookupTableNode = prev.newLookupTableNode;
lookupTableWriteNode = prev.lookupTableWriteNode;
newTupleNode = prev.newTupleNode;
}

@CompilerDirectives.TruffleBoundary
@Specialization
public RubyString toS(RubyEncoding encoding) {
final ByteList name = encoding.getName().dup();
name.setEncoding(ASCIIEncoding.INSTANCE);
return getContext().makeString(name);
public Object encodingMap(VirtualFrame frame) {
Object ret = newLookupTableNode.call(frame, getContext().getCoreLibrary().getLookupTableClass(), "new", null);

final RubyEncoding[] encodings = RubyEncoding.cloneEncodingList();
for (int i = 0; i < encodings.length; i++) {
final Object upcased = upcaseNode.call(frame, getContext().makeString(encodings[i].getName()), "upcase", null);
final Object key = toSymNode.call(frame, upcased, "to_sym", null);
final Object value = newTupleNode.call(frame, getContext().getCoreLibrary().getTupleClass(), "create", null, nil(), i);

lookupTableWriteNode.call(frame, ret, "[]=", null, key, value);
}

final Hash.HashEntryIterator i = getContext().getRuntime().getEncodingService().getAliases().entryIterator();
while (i.hasNext()) {
final CaseInsensitiveBytesHash.CaseInsensitiveBytesHashEntry<EncodingDB.Entry> e =
((CaseInsensitiveBytesHash.CaseInsensitiveBytesHashEntry<EncodingDB.Entry>)i.next());

final Object upcased = upcaseNode.call(frame, getContext().makeString(new ByteList(e.bytes, e.p, e.end - e.p)), "upcase", null);
final Object key = toSymNode.call(frame, upcased, "to_sym", null);
final RubyString alias = getContext().makeString(new ByteList(e.bytes, e.p, e.end - e.p));
final int index = e.value.getIndex();


final Object value = newTupleNode.call(frame, getContext().getCoreLibrary().getTupleClass(), "create", null, alias, index);
lookupTableWriteNode.call(frame, ret, "[]=", null, key, value);
}

final Encoding defaultInternalEncoding = getContext().getRuntime().getDefaultInternalEncoding();
final Object internalTuple = getContext().makeTuple(frame, newTupleNode, getContext().makeString("internal"), indexLookup(encodings, defaultInternalEncoding));
lookupTableWriteNode.call(frame, ret, "[]=", null, getContext().newSymbol("INTERNAL"), internalTuple);

final Encoding defaultExternalEncoding = getContext().getRuntime().getDefaultExternalEncoding();
final Object externalTuple = getContext().makeTuple(frame, newTupleNode, getContext().makeString("external"), indexLookup(encodings, defaultExternalEncoding));
lookupTableWriteNode.call(frame, ret, "[]=", null, getContext().newSymbol("EXTERNAL"), externalTuple);

final Encoding localeEncoding = getContext().getRuntime().getEncodingService().getLocaleEncoding();
final Object localeTuple = getContext().makeTuple(frame, newTupleNode, getContext().makeString("locale"), indexLookup(encodings, localeEncoding));
lookupTableWriteNode.call(frame, ret, "[]=", null, getContext().newSymbol("LOCALE"), localeTuple);

final Encoding filesystemEncoding = getContext().getRuntime().getEncodingService().getLocaleEncoding();
final Object filesystemTuple = getContext().makeTuple(frame, newTupleNode, getContext().makeString("filesystem"), indexLookup(encodings, filesystemEncoding));
lookupTableWriteNode.call(frame, ret, "[]=", null, getContext().newSymbol("FILESYSTEM"), filesystemTuple);

return ret;
}

@TruffleBoundary
public Object indexLookup(RubyEncoding[] encodings, Encoding encoding) {
// TODO (nirvdrum 25-Mar-15): Build up this lookup table in RubyEncoding as we register encodings.
if (encoding == null) {
return nil();
}

for (int i = 0; i < encodings.length; i++) {
if (encodings[i].getEncoding() == encoding) {
return i;
}
}

throw new UnsupportedOperationException(String.format("Could not find encoding %s in the registered encoding list", encoding.toString()));
}
}

@CoreMethod(names = "inspect")
public abstract static class InspectNode extends CoreMethodNode {
@CoreMethod(names = { "name", "to_s" })
public abstract static class ToSNode extends CoreMethodNode {

public InspectNode(RubyContext context, SourceSection sourceSection) {
public ToSNode(RubyContext context, SourceSection sourceSection) {
super(context, sourceSection);
}

public InspectNode(InspectNode prev) {
public ToSNode(ToSNode prev) {
super(prev);
}

@CompilerDirectives.TruffleBoundary
@Specialization
public RubyString toS(RubyEncoding encoding) {
final ByteList nameByteList = encoding.getName().dup();
nameByteList.setEncoding(ASCIIEncoding.INSTANCE);

if (encoding.isDummy()) {
return getContext().makeString(String.format("#<Encoding:%s (dummy)>", nameByteList.toString()));
} else {
return getContext().makeString(String.format("#<Encoding:%s>", nameByteList.toString()));
}
final ByteList name = encoding.getName().dup();
name.setEncoding(ASCIIEncoding.INSTANCE);
return getContext().makeString(name);
}
}

}
Original file line number Diff line number Diff line change
@@ -1124,77 +1124,6 @@ public boolean empty(RubyString string) {
}
}

@CoreMethod(names = "encode", optional = 2)
public abstract static class EncodeNode extends CoreMethodNode {

@Child private ToStrNode toStrNode;
@Child private EncodingNodes.DefaultInternalNode defaultInternalNode;

public EncodeNode(RubyContext context, SourceSection sourceSection) {
super(context, sourceSection);
}

public EncodeNode(EncodeNode prev) {
super(prev);
}

@TruffleBoundary
@Specialization
public RubyString encode(RubyString string, RubyString encoding, @SuppressWarnings("unused") UndefinedPlaceholder options) {
final org.jruby.RubyString jrubyString = getContext().toJRuby(string);
final org.jruby.RubyString jrubyEncodingString = getContext().toJRuby(encoding);
final org.jruby.RubyString jrubyTranscoded = (org.jruby.RubyString) jrubyString.encode(getContext().getRuntime().getCurrentContext(), jrubyEncodingString);

return getContext().toTruffle(jrubyTranscoded);
}

@Specialization
public RubyString encode(RubyString string, RubyString encoding, @SuppressWarnings("unused") RubyHash options) {

// TODO (nirvdrum 20-Feb-15) We need to do something with the options hash. I'm stubbing this out just to get the jUnit mspec formatter running.
return encode(string, encoding, UndefinedPlaceholder.INSTANCE);
}

@TruffleBoundary
@Specialization
public RubyString encode(RubyString string, RubyEncoding encoding, @SuppressWarnings("unused") UndefinedPlaceholder options) {

final org.jruby.RubyString jrubyString = getContext().toJRuby(string);
final org.jruby.RubyString jrubyEncodingString = getContext().toJRuby(getContext().makeString(encoding.getName()));
final org.jruby.RubyString jrubyTranscoded = (org.jruby.RubyString) jrubyString.encode(getContext().getRuntime().getCurrentContext(), jrubyEncodingString);

return getContext().toTruffle(jrubyTranscoded);
}

@Specialization(guards = { "!isRubyString(arguments[1])", "!isRubyEncoding(arguments[1])", "!isUndefinedPlaceholder(arguments[1])" })
public RubyString encode(VirtualFrame frame, RubyString string, Object encoding, UndefinedPlaceholder options) {

if (toStrNode == null) {
CompilerDirectives.transferToInterpreter();
toStrNode = insert(ToStrNodeFactory.create(getContext(), getSourceSection(), null));
}

return encode(string, toStrNode.executeRubyString(frame, encoding), options);
}

@Specialization
public RubyString encode(RubyString string, @SuppressWarnings("unused") UndefinedPlaceholder encoding, @SuppressWarnings("unused") UndefinedPlaceholder options) {

if (defaultInternalNode == null) {
CompilerDirectives.transferToInterpreter();
defaultInternalNode = insert(EncodingNodesFactory.DefaultInternalNodeFactory.create(getContext(), getSourceSection(), new RubyNode[]{}));
}

final Object defaultInternalEncoding = defaultInternalNode.defaultInternal();

if (defaultInternalEncoding == nil()) {
return encode(string, RubyEncoding.getEncoding("UTF-8"), UndefinedPlaceholder.INSTANCE);
}

return encode(string, (RubyEncoding) defaultInternalEncoding, UndefinedPlaceholder.INSTANCE);
}
}

@CoreMethod(names = "encoding")
public abstract static class EncodingNode extends CoreMethodNode {

Original file line number Diff line number Diff line change
@@ -0,0 +1,239 @@
/*
* Copyright (c) 2015 Oracle and/or its affiliates. All rights reserved. This
* code is released under a tri EPL/GPL/LGPL license. You can use it,
* redistribute it and/or modify it under the terms of the:
*
* Eclipse Public License version 1.0
* GNU General Public License version 2
* GNU Lesser General Public License version 2.1
*
* Contains code modified from JRuby's RubyConverter.java
*/
package org.jruby.truffle.nodes.rubinius;

import com.oracle.truffle.api.dsl.Specialization;
import com.oracle.truffle.api.source.SourceSection;
import org.jcodings.Encoding;
import org.jcodings.Ptr;
import org.jcodings.transcode.EConv;
import org.jcodings.transcode.EConvResult;
import org.jruby.Ruby;
import org.jruby.runtime.builtin.IRubyObject;
import org.jruby.truffle.runtime.RubyContext;
import org.jruby.truffle.runtime.control.RaiseException;
import org.jruby.truffle.runtime.core.RubyArray;
import org.jruby.truffle.runtime.core.RubyBasicObject;
import org.jruby.truffle.runtime.core.RubyEncoding;
import org.jruby.truffle.runtime.core.RubyEncodingConverter;
import org.jruby.truffle.runtime.core.RubyException;
import org.jruby.truffle.runtime.core.RubyHash;
import org.jruby.truffle.runtime.core.RubyString;
import org.jruby.util.ByteList;
import org.jruby.util.io.EncodingUtils;

/**
* Rubinius primitives associated with the Ruby {@code Encoding::Converter} class..
*/
public abstract class EncodingConverterPrimitiveNodes {

@RubiniusPrimitive(name = "encoding_converter_allocate")
public static abstract class EncodingConverterAllocateNode extends RubiniusPrimitiveNode {

public EncodingConverterAllocateNode(RubyContext context, SourceSection sourceSection) {
super(context, sourceSection);
}

public EncodingConverterAllocateNode(EncodingConverterAllocateNode prev) {
super(prev);
}

@Specialization
public Object encodingConverterAllocate(RubyEncoding fromEncoding, RubyEncoding toEncoding, RubyHash options) {
return new RubyEncodingConverter(getContext().getCoreLibrary().getEncodingConverterClass(), null);
}

}

@RubiniusPrimitive(name = "encoding_converter_primitive_convert")
public static abstract class EncodingConverterPrimitiveConvertNode extends RubiniusPrimitiveNode {

public EncodingConverterPrimitiveConvertNode(RubyContext context, SourceSection sourceSection) {
super(context, sourceSection);
}

public EncodingConverterPrimitiveConvertNode(EncodingConverterPrimitiveConvertNode prev) {
super(prev);
}

@Specialization
public Object encodingConverterPrimitiveConvert(RubyEncodingConverter encodingConverter, RubyString source,
RubyString target, int offset, int size, RubyHash options) {
throw new UnsupportedOperationException("not implemented");
}

@Specialization
public Object encodingConverterPrimitiveConvert(RubyEncodingConverter encodingConverter, RubyString source,
RubyString target, int offset, int size, int options) {

// Taken from org.jruby.RubyConverter#primitive_convert.

source.modify();
source.clearCodeRange();

target.modify();
target.clearCodeRange();

final ByteList inBytes = source.getByteList();
final ByteList outBytes = target.getByteList();

final Ptr inPtr = new Ptr();
final Ptr outPtr = new Ptr();

final EConv ec = encodingConverter.getEConv();

final boolean changeOffset = (offset == 0);
final boolean growOutputBuffer = (size == -1);

if (size == -1) {
size = 16; // in MRI, this is RSTRING_EMBED_LEN_MAX

if (size < source.getByteList().getRealSize()) {
size = source.getByteList().getRealSize();
}
}

while (true) {

if (changeOffset) {
offset = outBytes.getRealSize();
}

if (outBytes.getRealSize() < offset) {
throw new RaiseException(
getContext().getCoreLibrary().argumentError("output offset too big", this)
);
}

long outputByteEnd = offset + size;

if (outputByteEnd > Integer.MAX_VALUE) {
// overflow check
throw new RaiseException(
getContext().getCoreLibrary().argumentError("output offset + bytesize too big", this)
);
}

outBytes.ensure((int)outputByteEnd);

inPtr.p = inBytes.getBegin();
outPtr.p = outBytes.getBegin() + offset;
int os = outPtr.p + size;
EConvResult res = ec.convert(inBytes.getUnsafeBytes(), inPtr, inBytes.getRealSize() + inPtr.p, outBytes.getUnsafeBytes(), outPtr, os, options);

outBytes.setRealSize(outPtr.p - outBytes.begin());

source.getByteList().setRealSize(inBytes.getRealSize() - (inPtr.p - inBytes.getBegin()));
source.getByteList().setBegin(inPtr.p);

if (growOutputBuffer && res == EConvResult.DestinationBufferFull) {
if (Integer.MAX_VALUE / 2 < size) {
throw new RaiseException(
getContext().getCoreLibrary().argumentError("too long conversion result", this)
);
}
size *= 2;
continue;
}

if (ec.destinationEncoding != null) {
outBytes.setEncoding(ec.destinationEncoding);
}

return getContext().newSymbol(res.symbolicName());
}
}

}

@RubiniusPrimitive(name = "encoding_converter_putback")
public static abstract class EncodingConverterPutbackNode extends RubiniusPrimitiveNode {

public EncodingConverterPutbackNode(RubyContext context, SourceSection sourceSection) {
super(context, sourceSection);
}

public EncodingConverterPutbackNode(EncodingConverterPutbackNode prev) {
super(prev);
}

@Specialization
public Object encodingConverterPutback(RubyBasicObject encodingConverter, int maxBytes) {
throw new UnsupportedOperationException("not implemented");
}

}

@RubiniusPrimitive(name = "encoding_converter_last_error")
public static abstract class EncodingConverterLastErrorNode extends RubiniusPrimitiveNode {

public EncodingConverterLastErrorNode(RubyContext context, SourceSection sourceSection) {
super(context, sourceSection);
}

public EncodingConverterLastErrorNode(EncodingConverterLastErrorNode prev) {
super(prev);
}

@Specialization
public Object encodingConverterLastError(RubyEncodingConverter encodingConverter) {
notDesignedForCompilation();

final org.jruby.exceptions.RaiseException e = EncodingUtils.makeEconvException(getContext().getRuntime(), encodingConverter.getEConv());

if (e == null) {
return nil();
}

return getContext().toTruffle(e.getException());
}

}

@RubiniusPrimitive(name = "encoding_converter_primitive_errinfo")
public static abstract class EncodingConverterErrinfoNode extends RubiniusPrimitiveNode {

public EncodingConverterErrinfoNode(RubyContext context, SourceSection sourceSection) {
super(context, sourceSection);
}

public EncodingConverterErrinfoNode(EncodingConverterErrinfoNode prev) {
super(prev);
}

@Specialization
public Object encodingConverterLastError(RubyEncodingConverter encodingConverter) {
notDesignedForCompilation();

final EConv ec = encodingConverter.getEConv();

final Object[] ret = { getContext().newSymbol(ec.lastError.getResult().symbolicName()), nil(), nil(), nil(), nil() };

if (ec.lastError.getSource() != null) {
ret[1] = getContext().makeString(new ByteList(ec.lastError.getSource()));
}

if (ec.lastError.getDestination() != null) {
ret[2] = getContext().makeString(new ByteList(ec.lastError.getDestination()));
}

if (ec.lastError.getErrorBytes() != null) {
ret[3] = getContext().makeString(new ByteList(ec.lastError.getErrorBytes(), ec.lastError.getErrorBytesP(), ec.lastError.getErrorBytesLength()));
ret[4] = getContext().makeString(new ByteList(ec.lastError.getErrorBytes(), ec.lastError.getErrorBytesP() + ec.lastError.getErrorBytesLength(), ec.lastError.getReadAgainLength()));
}

return new RubyArray(getContext().getCoreLibrary().getArrayClass(), ret, ret.length);
}

}

}
Original file line number Diff line number Diff line change
@@ -51,6 +51,7 @@ public static RubiniusPrimitiveManager create() {
nodeFactories.addAll(BignumPrimitiveNodesFactory.getFactories());
nodeFactories.addAll(FloatPrimitiveNodesFactory.getFactories());
nodeFactories.addAll(EncodingPrimitiveNodesFactory.getFactories());
nodeFactories.addAll(EncodingConverterPrimitiveNodesFactory.getFactories());
nodeFactories.addAll(RegexpPrimitiveNodesFactory.getFactories());
nodeFactories.addAll(ModulePrimitiveNodesFactory.getFactories());
nodeFactories.addAll(RandomPrimitiveNodesFactory.getFactories());
14 changes: 13 additions & 1 deletion truffle/src/main/java/org/jruby/truffle/runtime/RubyContext.java
Original file line number Diff line number Diff line change
@@ -12,6 +12,7 @@
import com.oracle.truffle.api.*;
import com.oracle.truffle.api.CompilerDirectives.TruffleBoundary;
import com.oracle.truffle.api.frame.MaterializedFrame;
import com.oracle.truffle.api.frame.VirtualFrame;
import com.oracle.truffle.api.nodes.Node;
import com.oracle.truffle.api.object.Shape;
import com.oracle.truffle.api.source.BytesDecoder;
@@ -27,6 +28,7 @@
import org.jruby.truffle.TruffleHooks;
import org.jruby.truffle.nodes.RubyNode;
import org.jruby.truffle.nodes.RubyRootNode;
import org.jruby.truffle.nodes.dispatch.CallDispatchHeadNode;
import org.jruby.truffle.nodes.methods.SetMethodDeclarationContext;
import org.jruby.truffle.nodes.rubinius.RubiniusPrimitiveManager;
import org.jruby.truffle.runtime.control.RaiseException;
@@ -310,6 +312,10 @@ public RubyString makeString(RubyClass stringClass, ByteList bytes) {
return RubyString.fromByteList(stringClass, bytes);
}

public Object makeTuple(VirtualFrame frame, CallDispatchHeadNode newTupleNode, Object... values) {
return newTupleNode.call(frame, getCoreLibrary().getTupleClass(), "create", null, values);
}

public IRubyObject toJRuby(Object object) {
RubyNode.notDesignedForCompilation();

@@ -331,8 +337,10 @@ public IRubyObject toJRuby(Object object) {
return toJRuby((RubyString) object);
} else if (object instanceof RubyArray) {
return toJRuby((RubyArray) object);
} else if (object instanceof RubyEncoding) {
return toJRuby((RubyEncoding) object);
} else {
throw getRuntime().newRuntimeError("cannot pass " + object + " to JRuby");
throw getRuntime().newRuntimeError("cannot pass " + object + " (" + object.getClass().getName() + ") to JRuby");
}
}

@@ -349,6 +357,10 @@ public org.jruby.RubyArray toJRuby(RubyArray array) {
return runtime.newArray(store);
}

public IRubyObject toJRuby(RubyEncoding encoding) {
return runtime.getEncodingService().rubyEncodingFromObject(runtime.newString(encoding.getName()));
}

public org.jruby.RubyString toJRuby(RubyString string) {
final org.jruby.RubyString jrubyString = runtime.newString(string.getBytes().dup());

Original file line number Diff line number Diff line change
@@ -16,6 +16,7 @@

import org.jcodings.Encoding;
import org.jcodings.EncodingDB;
import org.jcodings.transcode.EConvFlags;
import org.jruby.runtime.Constants;
import org.jruby.runtime.encoding.EncodingService;
import org.jruby.runtime.load.LoadServiceResource;
@@ -76,6 +77,7 @@ public class CoreLibrary {
private final RubyClass keyErrorClass;
private final RubyClass loadErrorClass;
private final RubyClass localJumpErrorClass;
private final RubyClass lookupTableClass;
private final RubyClass matchDataClass;
private final RubyClass moduleClass;
private final RubyClass nameErrorClass;
@@ -106,6 +108,7 @@ public class CoreLibrary {
private final RubyClass threadClass;
private final RubyClass timeClass;
private final RubyClass trueClass;
private final RubyClass tupleClass;
private final RubyClass typeErrorClass;
private final RubyClass zeroDivisionErrorClass;
private final RubyModule configModule;
@@ -177,9 +180,6 @@ public CoreLibrary(RubyContext context) {
// Exception
exceptionClass = defineClass("Exception", new RubyException.ExceptionAllocator());

// EncodingError
encodingErrorClass = defineClass(exceptionClass, "EncodingError");

// FiberError
fiberErrorClass = defineClass(exceptionClass, "FiberError");

@@ -192,6 +192,7 @@ public CoreLibrary(RubyContext context) {
// StandardError
standardErrorClass = defineClass(exceptionClass, "StandardError");
argumentErrorClass = defineClass(standardErrorClass, "ArgumentError");
encodingErrorClass = defineClass(standardErrorClass, "EncodingError");
ioErrorClass = defineClass(standardErrorClass, "IOError");
localJumpErrorClass = defineClass(standardErrorClass, "LocalJumpError");
regexpErrorClass = defineClass(standardErrorClass, "RegexpError");
@@ -296,7 +297,7 @@ public CoreLibrary(RubyContext context) {

// The rest

encodingCompatibilityErrorClass = defineClass(encodingClass, standardErrorClass, "CompatibilityError");
encodingCompatibilityErrorClass = defineClass(encodingClass, encodingErrorClass, "CompatibilityError");

encodingConverterClass = defineClass(encodingClass, objectClass, "Converter", new RubyEncodingConverter.EncodingConverterAllocator());

@@ -306,7 +307,9 @@ public CoreLibrary(RubyContext context) {

rubiniusModule = defineModule("Rubinius");
byteArrayClass = defineClass(rubiniusModule, objectClass, "ByteArray");
lookupTableClass = defineClass(rubiniusModule, hashClass, "LookupTable");
stringDataClass = defineClass(rubiniusModule, objectClass, "StringData");
tupleClass = defineClass(rubiniusModule, arrayClass, "Tuple");

// Include the core modules

@@ -417,6 +420,20 @@ private void initializeConstants() {

processClass.setConstant(null, "CLOCK_MONOTONIC", ProcessNodes.CLOCK_MONOTONIC);
processClass.setConstant(null, "CLOCK_REALTIME", ProcessNodes.CLOCK_REALTIME);

encodingConverterClass.setConstant(null, "INVALID_MASK", EConvFlags.INVALID_MASK);
encodingConverterClass.setConstant(null, "INVALID_REPLACE", EConvFlags.INVALID_REPLACE);
encodingConverterClass.setConstant(null, "UNDEF_MASK", EConvFlags.UNDEF_MASK);
encodingConverterClass.setConstant(null, "UNDEF_REPLACE", EConvFlags.UNDEF_REPLACE);
encodingConverterClass.setConstant(null, "UNDEF_HEX_CHARREF", EConvFlags.UNDEF_HEX_CHARREF);
encodingConverterClass.setConstant(null, "PARTIAL_INPUT", EConvFlags.PARTIAL_INPUT);
encodingConverterClass.setConstant(null, "AFTER_OUTPUT", EConvFlags.AFTER_OUTPUT);
encodingConverterClass.setConstant(null, "UNIVERSAL_NEWLINE_DECORATOR", EConvFlags.UNIVERSAL_NEWLINE_DECORATOR);
encodingConverterClass.setConstant(null, "CRLF_NEWLINE_DECORATOR", EConvFlags.CRLF_NEWLINE_DECORATOR);
encodingConverterClass.setConstant(null, "CR_NEWLINE_DECORATOR", EConvFlags.CR_NEWLINE_DECORATOR);
encodingConverterClass.setConstant(null, "XML_TEXT_DECORATOR", EConvFlags.XML_TEXT_DECORATOR);
encodingConverterClass.setConstant(null, "XML_ATTR_CONTENT_DECORATOR", EConvFlags.XML_ATTR_CONTENT_DECORATOR);
encodingConverterClass.setConstant(null, "XML_ATTR_QUOTE_DECORATOR", EConvFlags.XML_ATTR_QUOTE_DECORATOR);
}

private void initializeSignalConstants() {
@@ -1152,10 +1169,18 @@ public RubyClass getByteArrayClass() {
return byteArrayClass;
}

public RubyClass getLookupTableClass() {
return lookupTableClass;
}

public RubyClass getStringDataClass() {
return stringDataClass;
}

public RubyClass getTupleClass() {
return tupleClass;
}

public RubyBasicObject getRubiniusUndefined() {
return rubiniusUndefined;
}
Original file line number Diff line number Diff line change
@@ -102,4 +102,9 @@ public RubyBasicObject allocate(RubyContext context, RubyClass rubyClass, Node c

}

}
@Override
public String toString() {
return getName().toString();
}

}
Original file line number Diff line number Diff line change
@@ -54,7 +54,9 @@ public void set(ByteList bytes) {
}

public void forceEncoding(Encoding encoding) {
this.bytes.setEncoding(encoding);
modify();
clearCodeRange();
StringSupport.associateEncoding(this, encoding);
clearCodeRange();
}

6 changes: 5 additions & 1 deletion truffle/src/main/ruby/core.rb
Original file line number Diff line number Diff line change
@@ -15,7 +15,6 @@
require_relative 'core/rubinius/api/shims/lookuptable'
require_relative 'core/rubinius/api/shims/array'
require_relative 'core/rubinius/api/shims/rubinius'
require_relative 'core/rubinius/api/shims/lookuptable'
require_relative 'core/rubinius/api/shims/thread'
require_relative 'core/rubinius/api/shims/tuple'
require_relative 'core/rubinius/api/shims/undefined'
@@ -57,6 +56,8 @@
require_relative 'core/rubinius/common/integer'
require_relative 'core/rubinius/common/bignum'
require_relative 'core/rubinius/common/fixnum'
require_relative 'core/rubinius/api/shims/encoding'
require_relative 'core/rubinius/common/encoding'
require_relative 'core/rubinius/common/false'
require_relative 'core/rubinius/common/float'
require_relative 'core/rubinius/common/immediate'
@@ -88,6 +89,9 @@
# Load delta (ordered according to Rubinius' load_order.txt)
require_relative 'core/rubinius/delta/struct'

# Eagerly-load files that are normally lazily loaded via Rubinius::CodeLoader.require_compiled.
require_relative 'core/rubinius/delta/converter_paths'

# Load JRuby+Truffle classes
require_relative 'core/array'
require_relative 'core/fixnum'
23 changes: 23 additions & 0 deletions truffle/src/main/ruby/core/rubinius/api/shims/encoding.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
class Encoding
TranscodingMap = Encoding::Converter.transcoding_map
EncodingMap = Encoding.encoding_map
EncodingList = Encoding.list

@default_external = undefined
@default_internal = undefined
end

Encoding::TranscodingMap[:'UTF-16BE'] = Rubinius::LookupTable.new
Encoding::TranscodingMap[:'UTF-16BE'][:'UTF-8'] = nil

Encoding::TranscodingMap[:'UTF-16LE'] = Rubinius::LookupTable.new
Encoding::TranscodingMap[:'UTF-16LE'][:'UTF-8'] = nil

Encoding::TranscodingMap[:'UTF-32BE'] = Rubinius::LookupTable.new
Encoding::TranscodingMap[:'UTF-32BE'][:'UTF-8'] = nil

Encoding::TranscodingMap[:'UTF-32LE'] = Rubinius::LookupTable.new
Encoding::TranscodingMap[:'UTF-32LE'][:'UTF-8'] = nil

Encoding::TranscodingMap[:'ISO-2022-JP'] = Rubinius::LookupTable.new
Encoding::TranscodingMap[:'ISO-2022-JP'][:'STATELESS-ISO-2022-JP'] = nil
10 changes: 10 additions & 0 deletions truffle/src/main/ruby/core/rubinius/api/shims/tuple.rb
Original file line number Diff line number Diff line change
@@ -14,6 +14,16 @@ def self.pattern(num, val)
Tuple.new(num, val)
end

def self.create(*args)
ret = Tuple.new(args.size)

args.each_with_index do |arg, index|
ret[index] = arg
end

ret
end


end

614 changes: 614 additions & 0 deletions truffle/src/main/ruby/core/rubinius/common/encoding.rb

Large diffs are not rendered by default.

92 changes: 92 additions & 0 deletions truffle/src/main/ruby/core/rubinius/common/string.rb
Original file line number Diff line number Diff line change
@@ -527,6 +527,98 @@ def codepoints
end
end

def encode!(to=undefined, from=undefined, options=undefined)
Rubinius.check_frozen

case to
when Encoding
to_enc = to
when Hash
options = to
to_enc = Encoding.default_internal
when undefined
to_enc = Encoding.default_internal
return self unless to_enc
else
opts = Rubinius::Type::check_convert_type to, Hash, :to_hash

if opts
options = opts
to_enc = Encoding.default_internal
else
to_enc = Rubinius::Type.try_convert_to_encoding to
end
end

case from
when undefined
from_enc = encoding
when Encoding
from_enc = from
when Hash
options = from
from_enc = encoding
else
opts = Rubinius::Type::check_convert_type from, Hash, :to_hash

if opts
options = opts
from_enc = encoding
else
from_enc = Rubinius::Type.coerce_to_encoding from
end
end

if undefined.equal? from_enc or undefined.equal? to_enc
raise Encoding::ConverterNotFoundError, "undefined code converter (#{from} to #{to})"
end

case options
when undefined
options = 0
when Hash
# do nothing
else
options = Rubinius::Type.coerce_to options, Hash, :to_hash
end

if ascii_only? and from_enc.ascii_compatible? and to_enc and to_enc.ascii_compatible?
force_encoding to_enc
elsif to_enc and from_enc != to_enc
ec = Encoding::Converter.new from_enc, to_enc, options
dest = ""
status = ec.primitive_convert self.dup, dest, nil, nil, ec.options
raise ec.last_error unless status == :finished
replace dest
end

# TODO: replace this hack with transcoders
if options.kind_of? Hash
case xml = options[:xml]
when :text
gsub!(/[&><]/, '&' => '&amp;', '>' => '&gt;', '<' => '&lt;')
when :attr
gsub!(/[&><"]/, '&' => '&amp;', '>' => '&gt;', '<' => '&lt;', '"' => '&quot;')
insert(0, '"')
insert(-1, '"')
when nil
# nothing
else
raise ArgumentError, "unexpected value for xml option: #{xml.inspect}"
end

if options[:universal_newline]
gsub!(/\r\n|\r/, "\r\n" => "\n", "\r" => "\n")
end
end

self
end

def encode(to=undefined, from=undefined, options=undefined)
dup.encode! to, from, options
end

def end_with?(*suffixes)
suffixes.each do |original_suffix|
suffix = Rubinius::Type.check_convert_type original_suffix, String, :to_str
7,686 changes: 7,686 additions & 0 deletions truffle/src/main/ruby/core/rubinius/delta/converter_paths.rb

Large diffs are not rendered by default.

32 changes: 32 additions & 0 deletions truffle/src/main/ruby/core/shims.rb
Original file line number Diff line number Diff line change
@@ -200,3 +200,35 @@ class IO
SEEK_SET = 0

end

# We use Rubinius's encoding subsystem for the most part, but we need to keep JRuby's up to date in case we
# delegate to any of their methods. Otherwise, they won't see the updated encoding and return incorrect results.
class Encoding
class << self
alias_method :default_external_rubinius=, :default_external=

def default_external=(enc)
self.default_external_rubinius = enc
self.default_external_jruby = enc
end

alias_method :default_internal_rubinius=, :default_internal=

def default_internal=(enc)
self.default_internal_rubinius = enc
self.default_internal_jruby = enc
end
end
end

# We use Rubinius's encoding class hierarchy, but do the encoding conversion in Java. In order to properly initialize
# the converter, we need to initialize in both Rubinius and JRuby.
class Encoding::Converter
alias_method :initialize_rubinius, :initialize

def initialize(*args)
initialize_rubinius(*args)
initialize_jruby(*args)
end
end

0 comments on commit da248f2

Please sign in to comment.