#clojure log - May 02 2008

The Joy of Clojure
Main Clojure site
Google Group
List of all logged dates

2:43 * cgrand installing enclojure

2:52 * cgrand ... or not (I figure I have to wait)

9:31 Chouser: "Downloads: not available" ... how often should I be clicking reload on that page?

9:31 Is every 5 seconds too much?

9:31 rhickey: :)

9:31 jteo: emacs should be good enough for anyone

9:36 Chouser: because elisp is as good as Clojure?

9:38 jteo: let's not go there. :)

9:40 Chouser: :-)

9:41 seriously though, the ancientness of elisp (and other parts of emacs) are major contriubtors to my failure so far to switch away from vim.

9:42 Not that viml (vim's scripting "language") can even be mentioned with a straight face in this context...

10:01 rhickey: I'm working on:

10:01 (gen-class

10:01 package-qualified-name

10:01 ;all below are optional

10:01 :extends aclass

10:01 :implements [interface ...]

10:01 :constructors {[param-types] [super-param-types], ...}

10:01 :methods {name [return-type [param-types]], ...}

10:01 :main boolean

10:01 :factory name

10:01 :exposes {protected-field {:get name :set name}, ...})

10:01 will spit out bytecode you can load immediately or store as .class file

10:01 Fully statically accessible name

10:02 Fully dynamic implementation, code goes in clojure namespace with same name

10:05 thoughts?

10:05 cgrand: Very nice! What does :factory stands for? A static method name?

10:05 Chouser: wow.

10:05 rhickey: cgrand: yees, set of static methods matching ctor sigs

10:06 ctor calls will flow through clojure init fn, if provided, allows you to morph args to super ctor and supply a state for the class

10:07 Chouser: people are going to try to use this for more than just Java interop, as part of the design of the clojure apps.

10:07 rhickey: aah, but state is final

10:07 Chouser: hehe. ok, nice.

10:07 rhickey: but can be actor or ref, so resulting objects are Clojure-savvy

10:08 transactional or asynch state for POJOs

10:09 Chouser: class name is required?

10:10 rhickey: yes, that is how the binding to Clojure code works - classname-->namespace-name

10:11 The idea is you'll gen-class once, then proceed to implement incrementally/iteratively in Clojure

10:11 The only part that's locked-down is what is supplied to gen-class

10:12 changing that means restarting JVM

10:18 cgrand: this is why I've held up on servlets, I'm hoping gen-class can be used to generate servlets and any other stubs that used to require Java

10:19 cgrand: rhickey: that was my first thought while reading this: no more hand-coded java stubs :-)

10:19 Chouser: rhickey: ah, so once you're done (gen-class foo ...), you can're reuse the name "foo" without restarting the JVM.

10:19 rhickey: chouser: yes,that's the static nature of Java I can't change

10:19 I've tried in this design to maximize the dynamic parts

10:20 but classnames are load-once-per-classloader

10:23 cgrand: what about providing an (optional) file name which would be loaded in static init? Or should all the init stuff be done in user.clj? (won't asnwer, must go)

10:23 Chouser: the methods list has no method bodies, does it -- just the interface description?

10:24 rhickey: methods is just for additional methods, the generated class will have all methods of superclasses, and yes, no code in gen-class at all

10:25 Chouser: ok, so you're generally do (gen-class Foo ...) and later (proxy [Foo] ...).

10:26 Sorry if I'm a little slow on the uptake here.

10:27 rhickey: gen-class builds a class that redirects method calls to vars named classname/methodname

10:27 ericthorsen: rich: how close are u to putting up the work?

10:27 I'm ready!

10:27 rhickey: it's in svn, but not done

10:28 Chouser: ohhh... gen-class links a Clojure namespace to a new Java class

10:29 ericthorsen: ok

10:29 rhickey: right

10:29 Chouser: which I now see you said up at the top.

10:29 the method functions get an extra first param for the class instance?

10:30 rhickey: yes

10:30 (gen-class org.clojure.MyComparator :implements [Comparator])

10:30 (in-ns 'org.clojure.MyComparator)

10:30 (defn compare [this x y] ...)

11:59 cgrand: rhickey: you mentioned that instances can hold state (as a final field?) how would one declare it and access it?

12:01 rhickey: was hardwired (to __state) when we last spoke, now there is a :state key, and an :init key which name the state and constructor initializer respectively

12:02 access like a normal field, will be public like all members in these classes

12:02 if :state not supplied, no state

12:04 init takes ctor-args and returns [[super-ctor-args] state]

12:05 s/returns/must return

12:05 cgrand: ok I understand now

12:14 So, prior to instantiation the namespace and its functions must be defined. This means that when the user doesn't control object creation (because he's using some framework) everything must be initialized as soon as Clojure ends initializing (eg through user.clj). Am I right?

12:18 rhickey: Partially. The generated class will create the namespace as a side-effect of having static Var fields. If there is no init, then objects can be created right away. Code will need to be loaded prior to method calls in order to avoid UnsupportedOperation. But in general, yes, user.clj is the path to pre-loading.

12:25 cgrand: I'm not averse to your suggestion of an option for the class to load its own code, need to think about any circularities

12:27 * rhickey leaves for lunch

14:13 leafw: is there any way to set stdout in clojure?

14:13 (with-out-str ...) works inconsistently

14:14 rhickey: leafw: bind *out*

14:22 lisppaste8: Chouser pasted "exception trying to implement a Map" at http://paste.lisp.org/display/60112

14:24 Chouser: How could I do dissoc correctly, but mess up assoc? Anyone have any ideas?

14:25 leafw: 'bind' is not a keyword in boot.clj

14:25 'binding'?

14:25 rhickey: yes, bind *out* with 'binding'

14:26 like with-out-str does

14:28 leafw: I a using with-out-str

14:28 but half the times it ends up not printing anything ,i.e. "prn" calls don't print anywhere

14:28 rhickey: with-out-str prints to a string

14:29 what do you do with the string it returns?

14:29 leafw: Ijust print it

14:29 prints with quoed newline and quoted quotes chars

14:30 I know this may not sound ver yhelpful.

14:31 code is at http://pacific.mpi-cbg.de/cgi-bin/gitweb.cgi?p=fiji.git;a=blob;f=src-plugins/Clojure/Clojure_Interpreter.java;h=8f664da07b8424cccdb1d38d746cc3c0b632927c;hb=1435e05539abc332386b515f7a4aa2df1f1236ac#l144

14:32 I created an interpreter, consisting of two JTextArea

14:32 exists in superclass AbstractInterpreter.

14:32 the ListThread is called via 'eval(String text)' method

14:33 abrooks: rhickey: I'd suggest that your respectable mop of hair may make up for lack of facial hair but you may want to consider growing a beard for good measure: http://blogs.microsoft.co.il/blogs/tamir/archive/2008/04/28/computer-languages-and-facial-hair-take-two.aspx

14:33 ;-)

14:33 rhickey: leafw: could you post a small example of your problem to paste.lisp.org?

14:34 leafw: I think I saw part of the problem

14:34 certainly on my side

14:34 I am returning from the while loop ... not ok if the parsing is not done.

14:36 yes, fixed. Oh well. thanks anyway.

14:37 * rhickey hopes hair makes up for beard...

14:40 rhickey: enclojure got dzoned - better get a download up soon :)

14:41 leafw: as long as java exists, there'll be those who bypass the pain by using clojure.

14:41 rhickey: Chouser: I can reproduce, looking at it now

14:48 Chouser: I'm guessing it has to do with the covariant return type

14:49 Associative.assoc returns Associative and IPersistentMap.assoc returns IPersistentMap

14:52 abrooks: rhickey: Going back to the awaiting a set of agents which mutually send to eachother. No, I don't know the number of jobs a priori. CountDownLatch isn't going to help me. Since almost exclusively all the jobs set to an agent are not originating from a thread outside of the agent pool, I don't have any way (that I can think of) to know when all queued jobs are done.

14:53 Agents feel like a good fit for the problem but I don't know how to know when they're done.

14:54 rhickey: how do you know to stop creating jobs?

14:54 leafw: After much testing, I remain clueless: the fallowing fails. (binding *out* (. System out)) -- but with-out-str never fails, and uses a similar construct

14:54 oh Isee

14:54 binding is like a let.

14:55 Chouser: rhickey: thanks. I'll try to turn that nugget into a fix.

14:58 abrooks: rhickey: An agent only adds jobs when it sees work to do. Eventually, there's no more work todo. The problem is finding the lowest cost path across a the matrix (where nodes have cost and edges are free).

14:59 rhickey: abrooks: are the agents in a list somewhere?

15:00 abrooks: could you countdown agent idleness rather than jobs?

15:02 abrooks: rhickey: I have the agents in a map. How can I detect their idleness?

15:03 * abrooks quickly heads over to the Clojure docs to make sure that he's not missing something obvious...

15:04 rhickey: abrooks: nothing automatic here, just brainstorming, they would have to report their idleness somehow

15:04 abrooks: Ah. Gotcha.

15:06 What about some sort of barrier that you could send off to a set of agents wait for?

15:07 rhickey: there is CyclicBarrier

15:07 abrooks: Is that a Java object?

15:07 rhickey: yeah in java.util.concurrent

15:08 if you have a fixed number of agents and they each know when they are done, that could work

15:08 abrooks: I'll take a look at that. Thanks.

15:08 rhickey: shame you can't increment CountDownLatch...

15:09 abrooks: Ah, here's the catch. A particular job knows when it's done but the agent does not.

15:09 rhickey: that's what I mant about incrementing countdownlatch, then each job issued would increment, each completed would decrement

15:10 abrooks: I could track the number of jobs going to zero but that would create a lot of contention on a single object.

15:10 (i.e. count up on send, count down on job completion)

15:11 rhickey: right

15:11 the problem isn't contention - AtomicInteger is plenty fast, the problem is polling for done

15:11 abrooks: Right.

15:11 rhickey: vs blocking for done

15:14 got it - use atomicinteger and exchanger - job setting count to zero enters exchanger, master waits on exchanger

15:14 abrooks: Right, but jobs could check the value before they exit and perhaps trigger some thread wakeup.

15:14 rhickey: ?

15:16 abrooks: If the "master" thread were to sleep or enter some sort of Java blocking primitive that I know nothing about but imagine it exists, the jobs could do the count up and count down. A job, before exiting could check the counter and unblock the master thread if the count was zero.

15:16 rhickey: every job issuer increments and every completing job decrements

15:16 See AtomicInteger and Exchanger

15:17 abrooks: Right. And following that decrement the completing job could wake up the master so the master can block instead of be polling.

15:17 rhickey: no peeking, decrementAndGet

15:17 when 0, exchange

15:18 abrooks: Okay, that. :) I don't know what Java offers. I've largely avoided Java for other languages when possible...

15:18 rhickey: java.util.concurrent rocks

15:20 abrooks: rhickey: I'll familiarize myself with it (and try to stop asking ignorant questions). Thanks!

15:21 rhickey: abrooks: you're welcome

15:56 abrooks: rhickey: java.util.concurrent does have some nifty tools. FutureTask is something I've wanted on a number of occasions and needed to fall back on periodic polling in the work thread on a cancellation flag.

15:57 ericthorsen: 1st alpha of enclojure is up. Please read the Getting Started page before you begin.

16:11 rhickey: abrooks: and remember Clojure fns are Callable, so plug right into that framework

16:11 abrooks: abrooks: Duly noted. :)

16:13 rhickey: ericthorsen: congrats! and thanks

16:14 abrooks: rhickey: Hm. I really need to stop talking to myself in IRC. The above was, of course, meant for you. :-/

16:50 rhickey: Chouser: your proxy problem is fixed

16:51 Chouser: rhickey: oh! I thought the problem was on my end.

16:52 rhickey: no, proxy needed to gen 2 methods when returns types are covariant

16:52 now it does

16:53 * Chouser runs: git svn fetch

16:54 Chouser: beautiful.

16:59 * bgeron nods

16:59 * bgeron does it also

16:59 bgeron: can take some time, though ;)

17:01 Chouser: oh! Yes, git is also beautiful, but I meant rhickey's fix.

17:01 * bgeron loves how it compresses nearly every project very well

17:01 bgeron: oh okay ;)

17:01 Chouser: abrooks: you saw my paste earlier? That's (the beginning of) that default-hash you wanted.

17:02 abrooks: Chouser: I didn't. Thanks for pointing that out.

17:05 Just as an FYI. Sourceforge can host Mercurial projects. I'm not sure about git but suspect that it's doable. Public (free) git hosting is also available here: http://repo.or.cz/

17:05 ... if Clojure would ever want to be hosted in either DVCS. %coughs%

17:05 * rhickey would like to switch to Mercurial

17:06 abrooks: Is there anything blocking that?

17:06 rhickey: not sure about IntelliJ support

17:06 lack of spare time

17:07 improve Clojure vs fiddle with VCS

17:08 abrooks: rhickey: Good reason. We don't want to distract you from Clojure. :)

17:18 rhickey: for the intrepid, if you want to play with gen-class in progress, what I just put up supports :extends, :implements, :state, :init, and :constructors

Logging service provided by n01se.net