Saturday, September 16, 2006

Got thinking about it and was probably a little enthusiastic last night. Thought someone must have done this before and remembered java's RMI, which is about what I'm looking for (if you could talk to it from lisp). Trying to get lisp to talk RMI is an option, but the protocol is heavily based on serialization, which'd be a major pain. It'd also require more communication between the languages which, over ports, is never a good thing.

The other option already coded into java is CORBA (I was rofl when I found out there was a organisation called omg). This looked really good until I got to the bit about IDL. Tis a contract for the client, saying what the server will do. In java it is automagically converted into local CORBA objects for the remote (server) class. The problem is that if you want ALL the classes (as you do when your programming) you have to convert all the IDL; you have to specify whick classes you want called in advance, generate IDL and copy it to the client, where it has to be converted into client classes and linked in. This is pants if you want to use all the java library from list (and dont want to generate all the IDL) or want to write classes at runtime in lisp and have them available in java. (OK we could get going on dynamic IDL-generated class loading, but probably best not to...)

So the proposal is a string-based RMI, jacol has implemented most of this, but I'm not too sure its as simple as it could be. Here's my current thoughts on this:

There will be a java server listening on a port. Once connected there are two commands generated by the lisp end and handeled by the java server.
(new-object, class name, parameters...) => object number
(invoke-method, object, method, parameters...) => object number | primative
We'll communicate objects using ints. At no time will the lisp end have any access (beyond the level of a number and maybe classtime) to the contents of a class, this is the programmers mess to sort out.

Java server will have a hashtable of int->object and will perform a lookup to find the object to operate on. Running this as a java server should mean that its really fast loading as there is no warm up time for the compiler, and it is expecting to run swing code...

Now...we also want the lisp side to be able to set callbacks on certain events, eg - "java button 34 pushed, so lisp function (take-over-world 53) is run". There will only be a few situations where these are required eg, GUI, timeouts etc... and it introduces a whole world of pain from concurrency to the need for an always-on-lisp-function-table.

"So half way through evaluating a list function, someone clicks the "make the monkey bigger" button" this is an issue surrounding lisp concurrency.

One option would be to have a wrapper for a lisp stratement call back. Eg if the function demanded an action listener (or one of a select few classes), we create a new action listener would makes a rmi call to lisp. Lisp then has a similar server, that just takes a string command and (evals) it.

This gives us two directional communication -
  • lisp can tell java to do something now
  • lisp can tell java to call some lisp later if one of a finite number of events happen
  • java can tell lisp to do something now
  • java can't tell lisp to run something later... (although this should be kinda trivial)
Looks do-able, just need to look up concurrency issues for clisp...[edit]D'oh - ok I'm not posting much (at all) at the moment, but when that passing comment comes up first on a google search for "clisp concurrency" you know they havn't implemented it yet. Yet another example of the Holyer effect.

There is something very broken (in a good way for me) about google, it gives far too much preference to subdomains! And it does all get indexed very, very fast compared to the rest of the web!


  1. Anonymous17:34

    now that was brainy! love your website <3

  2. Anonymous05:27

    Your e-mail is dead, so last I checked. We talked once before, and I was curious what your background in programming was before this project. I am definitely interested in creating a similar system of formulaic input for rendering, but with a completely different application.

    Matt -