Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Doesn't support Python. #92

Closed
monodemono opened this issue Jan 24, 2014 · 28 comments
Closed

Doesn't support Python. #92

monodemono opened this issue Jan 24, 2014 · 28 comments

Comments

@monodemono
Copy link

Seems like a pretty huge issue to me.

@SaphireLattice
Copy link

OC can use native libraries so you can use this http://labix.org/lunatic-python.

@fnuecke
Copy link
Member

fnuecke commented Jan 24, 2014

Seems like a pretty huge issue to me.

Brilliant phrasing. The biggest hurdle in adding additional languages is that the "os" has to be reimplemented in that language (meaning all the Lua scripts that are in the repo have to be rewritten for the other language). Which is something I probably won't do, at least not in the near future. A while back EcmaXp posted something that looked like a python interpreter written in Lua (see #33), not sure how far he got with that.

If anyone wants to tackle this, what needs to be done to get basic functionality (i.e. to get a new language running) is to write an implementation of the Architecture interface.

OC can use native libraries so you can use this http://labix.org/lunatic-python.

I'd like to avoid adding additional native libraries unless absolutely necessary, so Jython sounds more likely. If additional languages are added over time (which is somewhat of what I'm dreaming of in the long term) I might have to rethink that, but that's my current stance anyway.

@ghost
Copy link

ghost commented Jan 25, 2014

Simple: This OpenComputers are based on Lua Native Library, Can't support python. and supporting python are require more code.
Detail: Think about if python are really allowed. Python will eat huge RAM, CPU Time, Can't use in real world.

Well you will not interest in coding python compiler. (Python code to Lua code), but i will give a link for helper.

https://github.com/EcmaXp/PyCraft
(It still not runable, currently don't have feature lite python code to lua code.)

I don't have much time because i MUST go army. :(
If somebody can hosting python script (mean like cgi-bin), It will take less time as can i use that.

... What? the Architecture interface?????

@pingbird
Copy link
Contributor

EcmaXp: python is faster than java, i dont think performance would matter
though python does require a ton more ram

@ghost
Copy link

ghost commented Jan 28, 2014

No, CPython is slower than java (OracleJVM have JIT), but PyPy (it have JIT) is faster than java in most case.

@ghost
Copy link

ghost commented Jan 28, 2014

I think write python interpreter in lua (PyCraft) is failed because CPython's code are very huge and almost python user will hate tinypy (because they can't use 99.9...% library)

I checked the Architecture and now i decided use that for Support Python in OpenComputers.

New PyCraft Memos.

  • PyPy3 <--( Py4J; use the 127.0.0.1 port X )--> Java
  • Running bytecode in PyPy3 Process (Not PyPy3 own, use custom executor for run bytecode for support persist (function(closure). clsss, instance, etc), security (deny call any internal function that mark safety), restriction CPU Time (after some interval) / Memory (PyPy don't have sys.getsizeof)
  • Require inital memory with 15 MB ~ and BIG executable (One platform require more than 30 MB, PyPy is very big)
  • python package 'dill' are support class pickle, or i will write my own persist for some stuff.

@ghost
Copy link

ghost commented Jan 30, 2014

Estimate Time: 3 days if i can use computer.

  • Persist something is not really problem. (Pickle modules will help, closure are can restore by exec code directly (i will use string format and only number value name for security))
  • Execute bytecode is require pure operator execute function for executor can track them
  • I don't have any idea for use scala. and no idea if i really write (still no code for python-side) python interpreter with persist and limitable, how to use python? (different computer block? Python card? And recipe?? ???)

@ghost
Copy link

ghost commented Jan 30, 2014

And it's clearly python will use different code for internal api (like fs)

@fnuecke
Copy link
Member

fnuecke commented Jan 30, 2014

If you want to have a go at it (which would be amazing), it doesn't have to be in Scala, you can implement the Architecture trait/interface in a Java class. And yes, there'd have to be some kind of concept for what language is used. For testing you can just initialize the value in the Machine class differently, though.

As for the language specific implementation (meaning everything in the lua/ folder), yes, that will have to be redone, which may turn out to be the largest chunk of work. But it reminds me that I wanted to move the source of the read-only memory to the architecture interface (so you could have a folder python/rom for that).

@ghost
Copy link

ghost commented Feb 1, 2014

This comment will update when PyCraft (https://github.com/EcmaXp/PyCraft) get updated.

Current Status ( 36 % ) in Python Side

  • Now available prototype bytecode executor
  • Now available run pystone test. (If i have much time i will use that)
  • Now available import function. (Only system builtin module, i will rewrite them.)
  • Now available closure thing.
  • Some speed patch are apply. (2500 pystone/sec, original pypy3 interpreter are 11000 pystone/sec, i know that is pretty slow than CPython 3, time will slove that problem)
  • Now available raise Exception and catch that.
  • Now start with support getsizeof (CPython-based, can cause problem with if low memory)
  • Now start with support builtin modules.
  • Now available security features. (Slow, because just hook pushvalue)
  • Require write some modules.
  • Require write pure operator module.
  • Require more builtin module. (Just from something import * is work by security module.)
  • Require support yield. (With fake object)
  • Require support keyword arguments.
  • Require support memory compute algoritum.

Current Status ( 0 % ) in Java Side.

  • ?

@ghost
Copy link

ghost commented Feb 5, 2014

I don't have time for learn java, and setup scala with eclipse, make build script are eat times.
Can you provide API thing (but use py4j) for PyPy Process?

http://py4j.sourceforge.net/ (It use TCP Protocol (and TCP bind port=25334) for communicate)

Callback provided, custom class, instance also.

@fnuecke
Copy link
Member

fnuecke commented Feb 5, 2014

Thanks a lot for the time you've put into this! I missed the last update, sorry for not responding to that.

I don't quite understand the intended architecture here. Assuming we use py4j as the bridge, what would actually run the python code? Surely not a separate python interpreter that has to manually be started on the host OS? Because that's what py4j seems to be meant for, from a quick look at it.

Unless I'm overestimating the effort implementing the Java side of things would take immensely, I'm afraid I won't have the time to look into this at this time. I may in a couple of weeks/months, but it would probably make more sense for someone who knows both Java/Scala and python to get involved in this, since my python skills are very shallow and very rusty.

@ghost
Copy link

ghost commented Feb 5, 2014

Well even if use py4j, It still require manual launch from java process, exclude that. no problem with that. I will provide the python module for execute python code, and API also.

Don't panic.

https://github.com/MightyPirates/OpenComputers/blob/master/src/main/java/li/cil/oc/util/ExtendedLuaState.scala
https://github.com/MightyPirates/OpenComputers/blob/master/src/main/java/li/cil/oc/server/component/machine/NativeLuaArchitecture.scala

https://github.com/MightyPirates/OpenComputers/blob/master/src/main/java/li/cil/oc/server/component/machine/Machine.scala#L36

package li.cil.oc.server.component.machine

trait RemotePythonSpider {
  def give_me_RemotePythonArchitecture(machine, collection_of_callback): RemotePythonArchitecture
  // communicate with python process. by py4j with this interface.
}
package li.cil.oc.server.component.machine

trait RemotePythonArchitecture extends Architecture {}
package li.cil.oc.server.component.machine
import py4j.GatewayServer;

class NativePythonArchitecture(val machine: Machine) extends Architecture {
  private var python: RemotePythonArchitecture = (object's NativePythonArchitecture).getNewRemote()
  def isInitialized = python.isInitialized
  def recomputeMemory() = python.recomputeMemory()
  def runSynchronized() = ...
  def runThreaded(enterState: Machine.State.Value): ExecutionResult = ...
  // problem. runThreaded and runSynchronized are almost same. callback server is one, but still require thread for call it.
  // signal processing and etc thing are in here.
  def init(): Boolean = ...
  def load(nbt: NBTTagCompound) = ...
  def save(nbt: NBTTagCompound) = ...
}

object NativePythonArchitecture {
   private val Process process;
   private val RemotePythonSpider spider;
   private val GatewayServer server;

   def getNewRemote(machine: Machine) = {
       spider.give_me_RemotePythonArchitecture(machine);
   }

   def attachedProcess(attached_spider: RemotePythonSpider) = {
      spider = attached_spider;
   }

   def start() = {
     // in different thread
     (extract python library include my library called 'ocpy', download pypy or request to download that (more than 30 MB))
     process = (java way's to make child process by "pypy-execute -m ocpy" with working dir in /tmp ?)
     server = new GatewayServer(this);
     server.start();
   }

That is pusudo code, start function must call from different thread.

@ghost
Copy link

ghost commented Feb 6, 2014

Ok, i don't have time now. (I will go korea army) Somebody know Python (and CPython, PyPy) /Java (and Scala) will continue my work (https://github.com/EcmaXp/PyCraft // forkable) or make a new code for supporting python in OpenComptuters.

Even if OpenComptuers not suppport python, it still very good mineraft mod. like first meet AE in minecraft. (My Minecraft Mod Rank: 0. Forge, 1. AE, 2. OpenComputers, 3. ThermalExpansion (And Arsunal Tool also), 4. Chisel (Thanks too many blocks for deco!))

@fnuecke
Copy link
Member

fnuecke commented Apr 22, 2014

So, there's an API for adding custom "architectures" now, which could be used to add python support in an addon mod, and I don't have the time nor motivation to implement this myself, so I'm closing this. If anyone wants to take this on, please feel free to contact me (e.g. on IRC) to get some pointers on how to use that API, if the JDoc isn't sufficient.

@fnuecke fnuecke closed this as completed Apr 22, 2014
@gdude2002
Copy link
Contributor

I know this ticket is relatively old - I've compiled Lunatic Python (as @dangranos suggested) into a .so, what's the best way to get OC to load it? I've just stuck it in the server directory, and I can't seem to find any documentation on this.

That is to say, I used this fork with Lua 5.1 (I can recompile with 5.2 if that's a problem)

@Techokami
Copy link
Contributor

Come into the IRC channel and talk with gamax92, he's created some example mods for adding new architectures to OpenComputers (which is how you really need to do it)

@gdude2002
Copy link
Contributor

Sure, I'll join - but I'm not looking to write a new arch here, I'd really just like to be able to require("python") and work with it when it's suitable. I don't writing a small mod to make OC load the library, though.

@ghost
Copy link

ghost commented Apr 5, 2015

  • cleanup *

@gdude2002
Copy link
Contributor

How are you planning on sandboxing this, IE making it "safe"?

I'd love to see OC python but my main problem with writing this myself is that cpython and jython are both almost impossible to sandbox

Sorry, something went wrong.

@ghost ghost mentioned this issue May 5, 2015
@EcmaXp
Copy link

EcmaXp commented Mar 25, 2018

I have recently started developing a new project.

https://github.com/EcmaXp/OpenPie

Sorry, something went wrong.

@EcmaXp
Copy link

EcmaXp commented Aug 11, 2018

unicorn (+ java binding), micropython based development and some tests have been successful.
image

Sorry, something went wrong.

@gdude2002
Copy link
Contributor

That is quite impressive. Is it testable yet? How do we build it?

@EcmaXp
Copy link

EcmaXp commented Aug 11, 2018 via email

@gdude2002
Copy link
Contributor

In that case, do you have a English checklist somewhere so that people can submit pull requests?

@EcmaXp
Copy link

EcmaXp commented Aug 11, 2018 via email

@EcmaXp
Copy link

EcmaXp commented Aug 21, 2018

@gdude2002 I just created new Cortex-M0 register emulator with C#
If you are interested in C # and Java, you can help porting to Java.

@EcmaXp
Copy link

EcmaXp commented Sep 2, 2018

image

asiekierka pushed a commit to asiekierka/OpenComputers that referenced this issue May 7, 2023
Added string versions of getStoredEU/getEUCapacity for huge values
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants