Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added Digest::SHA256 #2815

Closed
wants to merge 1 commit into from
Closed

Added Digest::SHA256 #2815

wants to merge 1 commit into from

Conversation

Nax
Copy link

@Nax Nax commented Jun 12, 2016

The interface is the same as Digest::SHA1.

A lot of code is duplicated between the two, maybe we should refactor into a Digest::Utils module?
Or Digest::Utils::SHA for SHA-specific utils?
The padding function, for example, is the same between all SHAs.

Also, the Digest::* interface is probably gonna be implemented in the same way for every hash function, only the digest method will change.
Maybe we should have an interface with hexdigest and base64digest implemented (as a base class or mixin, I don't know if it's a thing in crystal)?

Tests are also provided, based on the test suite for SHA1.

"0123456701234567012345670123456701234567012345670123456701234567",
"8182cadb21af0e37c06414ece08e19c65bdb22c396d48ba7341012eea9ffdfdd",
"gYLK2yGvDjfAZBTs4I4ZxlvbIsOW1IunNBAS7qn/390=",
},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are those official test vectors? Or are they made up?

Do you have tests for known vulnerabilities in implementations, if there are any?

@ysbaddaden
Copy link
Contributor

Thanks! I don't have means for a proper review (I'm on a phone).

I believe SHA256 and SHA512 are actually the same thing (SHA2) but using different lengths. I'll read more about that and about SHA1, and see whether it could make sense to have a single SHA2 implementation, possibly sharing some code with SHA1.

How fast is this implementation, compared to using OpenSSL::Digest for example?

@Nax
Copy link
Author

Nax commented Jun 12, 2016

For the test vectors, I took the tests from sha1_spec.cr and rehashed them using shasum -a 256.
SHA-256 and SHA-512 are very similar indeed, but SHA-512 use 64 bit words. I've seen that crystal supports something close to C++ templates, so it may be possible to share some code between the two.

As for the speed, I don't know. This implementation is based on the existing Digest::SHA1 implementation. I am very new to crystal, so I have no idea how benchmarks are done here (do you have something like the Benchmark module in ruby?), but I would be glad to make one, and optimize the implementation.

@jhass
Copy link
Member

jhass commented Jun 12, 2016

Yes, our benchmark stdlib has roughly the same API as Ruby's and that of the benchmark-ips gem.

@Nax
Copy link
Author

Nax commented Jun 12, 2016

Ok, I did this small benchmark :

require "benchmark"
require "openssl"
require "digest/sha256"

n = 10000
strings = [] of String
ssl_sha256 = OpenSSL::Digest.new("sha256")

n.times do
    len = (rand * 100).to_i * 1024 # (1k..100k range)
    str = "a" * len
    strings << str
end

Benchmark.bm do |x|
    x.report("OpenSSL::Digest") do
        strings.each do |str|
            ssl_sha256.reset
            ssl_sha256 << str
            ssl_sha256.digest
        end
    end

    x.report("Digest::SHA256") do
        strings.each {|str| Digest::SHA256.digest(str) }
    end
end

It generates strings full of 'a's, with a random length between 1k and 100k.
On my machine, OpenSSL is twice as fast. However, when dealing with smaller strings (1 ~ 1k), Digest::SHA256 is up to twice as fast as OpenSSL. I suppose this come from the block reading method, in Digest::SHA* modules, this is done one byte at a time, while it is possible (but nontrivial due to padding) to read up to 512 bits into the buffer.

The interface is the same as Digest::SHA1
A lot of code is duplicated between the two, maybe we should refactor
into a Digest::Utils module?
Or Digest::Utils::SHA for SHA-specific utils?
The padding function, for example, is the same between all SHAs.

Also, the Digest::* interface is probably gonna be implemented in the
same way for every hash function, only the digest method will change.
Maybe we should have an interface with hexdigest and base64digest
implemented (as a base class or mixin, I don't know if it's a thing in
crystal)?

Tests are also provided, based on the test suite for SHA1.
@ysbaddaden
Copy link
Contributor

Thanks for digging more. I'll definitely take a closer look.

@Nax
Copy link
Author

Nax commented Aug 5, 2016

Any news about this ?

@asterite
Copy link
Member

asterite commented Aug 5, 2016

I originally implemented Digest::SHA1 in pure Crystal so we don't depend on openssl for websockets, which are needed for crystal play. The one that comes with openssl is fine, I think, so I don't think we'll want to implement every algorithm in pure crystal, mostly because of security reasons. In fact I can imagine removing Digest::SHA1 eventually once we can workaround some issues with building the compiler with openssl.

@Nax Nax closed this Aug 5, 2016
@Nax Nax deleted the sha256 branch August 5, 2016 15:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants