I can implement these features myself and send a PR if API looks good.

The Data Card already supports base64 encoding and decoding.

Streaming should continuously use a considerable amount of energy while a stream is running.

For encryption, use the Computronics Cipher Blocks.

The Data Card already supports base64 encoding and decoding.

Sorry, didn't noticed this. Ok, let's exclude it from issue.

For encryption, use the Computronics Cipher Blocks.

Or software Lua program. Or another addon. Or something else. Subject of this issue is to include it as core feature.

It doesn't make sense to have encryption on a card when there is a block doing it already. If you want encryption, install Computronics. The Data card is for hashing and encoding, and only hashing and encoding; basic data processing. No complex encryption.

It doesn't make sense to have encryption on a card when there is a block doing it already.

It is bad argument. Encryption is an operation that can be easy implemented by Java API, so why not merge it to core? What is difference of basic data processing from complex encryption? Why AES is complex operation while SHA256 isnt?

The point is that there is already something adding encryption, so doing it twice "just because it's not an addon" is stupid.

Furthermore: SHA-2 is very simple compared to AES. Not only does hashing only depend on the input, its also mainly used directly. If you implement AES you will always also implement a mode of operation for AES, such as CBC or GCM. AES alone is not very useful, SHA-2 alone is.

Furthermore: SHA-2 is very simple compared to AES.

It can be true if you implement AES and SHA256 by hand (but still arguable - look at SHA256 round and AES round). But usually it is implemented by Java API calls. And with Java API does not matter what you are implementing - SHA256 hashing, Blowfish or AES encryption - it is implemented in a simple way with ten lines of code.

Such things are only added because they would be incredibly slow if implemented in Lua. You need to think about what would be more complex to do if you were to implement it in Lua.

So, according to your point, AES is more complex than SHA256, SHA256 is implemented in data card (but software implementations are still exists), and AES should NOT be implemented in data card. Wat?

And, I will repeat it once again, it is Java API calls. No one is going to implement AES when API already exists.

It seems that any nontrivial crypto algs implementation on Lua will be incredibly slow, especially when many of them have native implementation in OS.

Yes, and that's why there are two blocks in Computronics to do encryption. It has already been discussed.

^^^ THIS
Alternatively: why have you not installed Computronics yet? It's the mod that makes OpenComputers crazy awesome!

Suppose I install Computronics (but I didn't). They encrypt data with some unknown algorithm with a key based on a six items (sic!). Should I build BuildCraft pipe system around it? Or maybe attach inventory adapter to it?

P.S. I do not need any feature listed in computronics, with exception, probably, of color lamps. Just merge 10 lines of code into master branch and add capability to do it without installing computronics, is it so hard?

There is already toHex and fromHex done in data card lib.
Starting new stream shouldn't reset whole card.
I know that OpenOS is one app based but Plan9k is fully multithreaded OS and we already have multiple services running there (like SSH, Magikoin, Entity). Making data card itself state-full would make multithreaded programs conflict with each-other.

The first useful suggestion in a thread of messages like "ITS ALREADY IMPLEMENTED IN COMPUTRONICS!!!11111".

Maybe then init should return some handle object with update and final methods? Then card will be stateless, but I do not know how Lua and Java GC would handle those objects.

It is possible I was considering it while implementing a data card at first but thought that it would be hard to reach the hard limit. IMHO Sangar placed it a bit too low but I understand his concerns about performance.

So after a bit of discussion on IRC here's the current compromise:

• This could go into a tier two data card.
• May need a T3 slot.
• Throttled (non-direct calls, i.e. at least one tick between operations).
• Power values TBD, Vex will post what the block uses and we'll see what to make of that :P

So it'd basically a higher tier and cost variant of the block, avoiding invalidating it. And it'd make sense, since it's functionally more versatile.

OT: Oh, and as for the limits on the data card, those were just kind of gut numbers >_>
I'm open to discussion on raising those defaults a bit.

Yeah, I agree that those features should go into T2 data card both for increasing building costs and avoid breaking of existing setups. Since I wasn't listening on IRC - what is current compromise for streaming operation? Will it be implemented?

AFAIK, throttled calls can be made specifying limit=1 on callback annotation to avoid unnecessary synchronization with main thread.

Didn't really talk about streaming support. I'm honestly not sure about this, can this even be streamed? I.e. have a buffered window of a limited size one writes to and reads from while encrypting/decrypting? Otherwise it'd be kind of pointless, since it'd also have to be memory limited, and then it can just be a fixed size limit for a non-streaming call.

limit=1: Yeah, that'd be an option.

Java have full capability of streaming support and almost nothing is accumulated in memory - just need to use this API. update function process data chunk-by-chunk - you feed it data, it returns processed data to you. finish function returns final piece of data (archive footer on deflate, final block on cipher).

I do not think that this call will be extremely useful, because 1 MB of data to process is relatively rare case, but it will allow to process arbitrary data size.

Great, I see no reason to not have it streaming then. Can return an object as userdata like the sockets on inet cards so it won't interfere across different programs.

I would love to see all this in a T2 Data Card. As for Computronics already having encryption, yes it does but RSA isn't what some people need.

makkarpov, if all you need from Computronics is the Colorful Lamp, might I suggest using OpenLights instead? It's essentially the same block (And the colorful lamp was inspired by it)

We extensively talked about this on IRC. I think @fnuecke now has a rather good idea about all this.

Thanks for suggestion, will take a look. About Computronics: are there any servers running it? If there are none, I will not be surprised. Installed it for test, placed RSA cipher blocks, started 65536-bit key generation four times, got 400% CPU load of all my cores. Limits? What is it?

P.S. Can someone make me unsee this and this code? Now I understood why he complained about difficult implementation of AES.

Some thoughts: we can also make a T3 data card that will have asymmetrical crypto (if anyone needs it), but done right, because at asymmetrical crypto Computronics definitely sucks. The main API ideas:

• All higher-tier data cards should inherit functionality of lower-tier ones
• Use EC cryptography, because it
  • Have smaller key sizes --> computations done faster
  • Have very simple key generation procedure --> no separate worker threads for key generation
  • Have complicated math behind the scenes --> no temptation to implement it yourself as Computronics done with RSA.
• Introduce function generateKeyPair(size: integer): userdata, userdata - generate pair of keys, where size can be either 256, 384 or 512 bits. Returns public and private key as userdata objects with method serialize.
• Introduce function deserializeKey(type: string, x: binary): userdata that creates key from it's representation.
• Introduce stream mode operation ecdh - a special stream mode that accepts keys instead of bytes.
  • Initialized with private key
  • Updated with one or more public keys
  • Generates a byte array when finished.
  • Main property is that ecdh(privA, pubB, pubC, ...) == ecdh(privB, pubA, pubC, ...) == ecdh(privC, pubA, pubB) thus making it suitable for hybrid crypto
• Introduce stream mode operations ecdsa_s with private key as parameter and ecdsa_v with public key and signature as parameters.
  • Generates or verifies a signature
  • Updated with byte stream
  • ecdsa_s returns signature on finish, ecdsa_v returns boolean - whether check was successful.
• All root functions (non-streamed ones) and ecdh operation should have rate limit of 1 call per tick.
• ecdsa operation may have weaker update limits because it just hashing on update.
• All of these operations are already done in Java, I or @fnuecke just should wrap it in userdatas, and I can provide code snippets for all operations I described.

Hmm... Tried to implement streamed operations and encountered and issue about that I had forgotten - Java crypto primitives aren't intended to be persisted (maybe they can be serialized, but it will be quite hacky). And there isn't included bouncycastle anymore. Looks like it can ruin all streaming ideas (although I can port hashes and AES to serializable ones using BouncyCastle classes, but with ECDSA and ECDH and Deflate it would be troublesome at current API specifications).

Will submit PR with these features (except streaming) implemented

As I said, we talked with @fnuecke extensively on IRC. You should wait...

Too late :P But I'm glad to hear how do you see operation of this card.

By the way, if the encryption in Computronics is using all your cores that's pretty bad indeed, I'd like to know how you would limit the processing. (I would just limit the bit size, but that doesn't sound like a good idea to me)

The reason why the implementation is manual was mainly for fun and challenge. It's not really worse than the one in native Java.

If you have suggestions, open an issue on Computronics' github page as this is going quite off-topic.