#clojure log - Apr 15 2008

The Joy of Clojure
Main Clojure site
Google Group
List of all logged dates

0:23 abrooks: Chouser, frodwith: -> turns (-> x c b a) into (a (b (c x))). x is placed into a list if it's not a list already.

3:56 vincenz: What does . do?

4:43 Dumb question, how do I clear the repl variables if I'm doing interactive programming with SLIME?

4:58 clojure is great )

4:58 :)

5:18 cgrand: . is a pecial form for java interop

5:19 java: myObject.myMethod(arg1, arg2)

5:19 clojure: (. myObject myMethod arg1 arg2)

5:27 * vincenz nods

5:27 vincenz: Oh I thought the syntax was

5:27 (. myObject (myMethod arg arg2))

5:27 that's what I've been using, since it complained n (. myObject method)

5:50 cgrand: It depends which version you are using

5:51 see http://groups.google.com/group/clojure/browse_thread/thread/dd64bd49191f8062/9ef373abfb6e5b57

5:52 vincenz: Thank you

5:52 Btw, does the designer of clojure ever come in here?

5:55 cgrand: He connects to this channel every day but right now it's a bit early.

5:56 vincenz: Alright thank you, I've watched the two movies, one one clojure and concurrency and one on clojure squences. Ihaven't dug in yet into actually using it but overall it gave me a good feel for the language, and I agreed with most of his design decisions.

5:57 cgrand: On which decisions do you disagree?

5:57 vincenz: ne

5:57 Wel more like the lack of one

5:58 Anyways, in regards to sequences, I very much agree with the interface he chose for, and in my own research I have come to conclusions about a similar interace

5:58 What I feel is missing...

5:58 Hmm

5:58 Do you mind if I delay this conversation? colleagues are asking me for lunch

5:59 cgrand: (no, pb we seems to be in the same timezone... lunch time!)

5:59 vincenz: Ttyl

6:58 re

7:26 rhickey: Hi

7:27 rhickey: You're the author of clojure?

7:27 rhickey: yes

7:27 vincenz: I saw your two presentations on the concepts behind it and must admit I'm rather interested, and wanted to throw an idea at you.

7:28 (The one on clojure and concurrency, and the one on sequences)

7:28 rhickey: ok

7:30 vincenz: If I were to analogize clojure with an existing lisp, I would look at scheme. However scheme has something which clojure does not have: continuations. There are several types of continuations: the old noes, then the direct ones (shift/reset). I doubt the first would be well-behaved in the presence of the STM-stuff you have. The secon might be, but only limitied to some contexts. However, there is a third type coof continuation, 'sub-con

7:30 * vincenz hopes he didn't get cut

7:31 * vincenz apologizes for the ssh-lag-induced typos.

7:32 rhickey: That did get cut off - seems like a good question for the google group

7:32 vincenz: Alright, I will write a post tonight then. I am not certain how feasible it would be on the JVM.

7:33 But it would be an interesting excercise to find out how this meshes with the concurrency model of clojure.

7:33 Is there some white paper on the core primitives?

7:33 rhickey: Just the web site

7:33 vincenz: Ok

7:40 Just so I get it correctly: functions passed to agents enter into the agent's thread, and threads block on sync-blocks until such sync-blocks succeced, and only then continue with any possible code that may fall after it?

7:43 rhickey: Not quite. Each agent doesn't have a thread, but actions sent to agents are asynchronous and will happen on 'another' thread eventually, i.e. send returns right away. sync is synchronous, but not necessarily blocking at all - but a sync block may be run more than once until it succeeds

7:46 * vincenz nods

7:46 vincenz: Just like STM in Haskell :)

7:46 rhickey: sort of, there is no or-else

7:47 vincenz: I presume side-effects are delayed just as agent-calls are?

7:47 (E.g. IO)

7:47 rhickey: the only side effects you can use are agent sends

7:47 vincenz: What about 'println'

7:47 rhickey: but that is by convention - I can't control Java

7:49 vincenz: Actually, it would be interesting to make an IO agent. That way IO happens similarly to how all other agent stuff works. Maybe you could bottle the java-stuff in that IO agent as well. Sort of like an IO monad, except in this case it's a separate thread.

7:49 rhickey: I don't like monads

7:49 vincenz: You wouldn't have monads

7:49 rhickey: And not all Java stuff has side-effects

7:49 vincenz: But that way you wouldn't have something happen twice in case a sync-block is rerun.

7:50 rhickey: People can use agents for that, but I'm not going to automate since I can't determine when the Java call would need that isolation

7:50 vincenz: E.g. (dosync (print "Hello") (readsomevariable) (writesomevariable))

7:50 rhickey: (dosync (send an-agent print "Hello") (readsomevariable) (writesomevariable))

7:51 DIY

7:51 vincenz: Fair enough :)

7:51 I wasn't here to critique that choice, just an interesting topic to talk about :)

7:52 rhickey: Java is a hole I'm not going to plug - instead I've focused on giving safe, correct, efficient alternatives

7:52 * vincenz nods

7:52 vincenz: Well I wasn't targetting Java specifically, more the IO stuff, since that's well defined as a library as is. But indeed, a DIY approach there would suffice.

7:53 Besides, the JAva-side would probably make coding some stuff really painful if you needed data to come back.

7:54 rhickey: yes, transactional IO is not straightforward in any case

7:54 especially the I

7:54 * vincenz nods

7:55 rhickey: input/transaction/output will be a common pattern

7:55 vincenz: Either way, I would block your thread and thus block your sync pattern.

7:55 So it's probably a poor choice to do I insidea transaction

7:55 rhickey: blocking in sync is a no-go

7:55 vincenz: Continuations and transcations are also an open problem

7:56 I'll try to reread your exact threading model tonight as well as that paper I have in mind

7:56 and try to figure out an early possible solution, open for discussion

7:56 Though I wonder if this would even be possible on the JVM :|

8:00 Finally, I must say that your abstraction for sequences is really elegant. I've -vaguely- envisioned something similar (I work on optimizations of data-types, concretely sequences), but this was a very nice crystalization :)

8:01 rhickey: It's just cons/car/cdr freed from cons cells :)

8:01 vincenz: Sure, but you

8:01 '

8:02 Whoops, but you need a persistent model of a vector :)

8:02 rhickey: yes, extending to vectprs and maps is probably Clojure's contribution, though I don't claim novelty for any of Clojure

8:04 vincenz: And in the end, this is the model you need

8:04 It's a zipper like model

8:04 Iterator models are fundamentally broken

8:05 (And are completely useless, as soon as you alter your sequence through one of the iterators, you invalidate the other ones..)

8:06 And working in clojure almost makes me want to work in java again, as now you acn use emacs + clojure to do online coding + introspection quite easily :)

8:06 (defn get-methods [x] (. (. x (getClass)) (getMethods))))

8:07 rhickey: (.. x getClass getMethods)

8:08 vincenz: Thanks

8:08 * vincenz is still a total noobie to clojure

8:08 vincenz: But the fundamentals as explained in your videos made a lot of sense.

8:08 rhickey: good

8:08 vincenz: Then again, as you said it "All the haskellers in the audience should be nodding"

8:09 rhickey: :)

8:09 vincenz: you mean (.. x (getClass) (getMethods)) ?

8:10 rhickey: They all work but what I showed you is possible with the latest syntax

8:10 vincenz: Ah, I guess the version I have doesn't have that yet.

8:10 I tend to get 'No matching field found' exceptions when I don't use the parens.

8:10 rhickey: ok, not the latest

8:11 jteo: the perks of being the creator.

8:11 rhickey: No, it's up on SVN

8:11 I share :)

8:11 * vincenz coughs *darcs*

8:11 jteo: technically, you always get first dibs rhickey.

8:11 ;)

8:11 rhickey: true

8:12 vincenz: How hard was it ot make the backend for clojjure? Say that continuations won't work on JVM, how hard would it be to to make an interpreter on some other language to test out the combination of transcations with continuations?

8:17 rhickey: I'm not sure I believe in continuations ;)

8:18 vincenz: They can be quite powerful when properly used.

8:18 But perhaps you're thinking of traditional continuations, not direct-style ones.

8:18 ?

8:20 rhickey: the future-of-the-computation notion is bound to a notion of programs as calculations that doesn't resonate with me, and the implementation difficulties don't speak well for their practicality.

8:20 vincenz: Right, you're thinking of traitional ones

8:21 There are other types of continuations, namely direct-style/functional ones, which are far more interesting imho

8:21 rhickey: paper?

8:21 vincenz: They do not only have a starting point, but an ending point as well, and you only reify the path between those two points

8:21 sec

8:24 * vincenz can't find the url

8:24 vincenz: http://www.cs.rutgers.edu/~ccshan/recur/recur.pdf

8:24 This is my favourite paper on the subject

8:25 and it seems I abused dterminolgy, the proper terminology is "delimited continuations"

8:26 rhickey: will read, thanks

8:33 * vincenz wonders if this is a bug

8:34 vincenz: I know that you can't run 'alter' outside of a sync body, but I made an alter statement escape by putting it inside a function. I get a very odd error when I run this

8:34 (let ([dum (dosync (fn [](alter foobar (fn [x] (+ x 2)))))]) (dum))

8:34 foobar is simply (def foobar (ref 1))

8:35 http://rafb.net/p/p8ehT552.html

8:36 nm

8:37 Wrong let syntax, disregard, please.

9:55 rhickey: Let me know when you care to discuss the paper :)

9:56 rhickey: I won't get to read it until later on

9:57 vincenz: rhickey: No rush :)

12:09 Modius: Just wondering - and I don't know if Clojure would have to "inherit" this from the JRM or from how it interacts with it - does Clojure tend to make a reference eligible for garbage collection the last time it is referenced, or when it leaves scope? I have seen GCed languages do it both ways (C# does, Lispworks does, SBCL doesn't).

12:51 I fiddled around enough to answer my question - but now have one - if (make-huge) makes a new data structure of size such that only one fits in RAM, (let [x (make-huge)] (let [y (make-huge)] (+ 1 2))) <-- This runs out of RAM - would static analysis in Closure allow this sort of code to work, or would it be some JVM limitation? What would be the pros and cons of getting something like this working?

12:54 Chouser: what was the answer to your first question?

12:54 I imagine it might depend on which JVM you use.

12:55 jteo: i presume that since closure compiles to bytecode, it would follow the rules of the GC in the jvm you use.

12:56 Modius: The answer was, the language is retaining the reference for the full scope.

12:57 The behavior may be JVM related, or may be the language - both would be required to release x and/or y inside-scope.

12:57 It would take code analysis to know that x and y won't be used again.

12:58 If a desirable behavior, this could be taken care of via static analysis. What would be harder would be dealing with the case of calls - i.e. having the caller keep the reference alive. If this is inherent to Java calling, this early release may not be possible.

13:00 Chouser: I'm no language designer, but it may be impossible. Any function name can be made to point to a new definition at any time.

13:01 well... I guess if you have a lexical variable (like x above), you can see at compile time that it's not used or passed to any other funcs, so it could be freed.

13:01 But if it's passed to function f, even if f currently doesn't use it a later re-definition of f might.

13:03 Modius: It's possible as I've used environments that do it. Has nothing to do with function definition but how parameters are evaluated stored and handled at point of call.

13:04 (some-function (make-huge)) <-- "huge" will be made at point of call, and some-functoin will take it. To have it be eligible for collection, nothing about the caller can keep a reference to the "huge" after the call is made (or during).

13:55 jgracin: rhickey: thanks for elaborate explanation on my post regarding thread starvation.

13:56 vincenz: jgracin: link?

13:57 jgracin: http://groups.google.com/group/clojure/browse_thread/thread/960b717c4d5b95ad

13:58 rhickey: jgracin: sure, it'll take a while before everyone gets comfortable with the Clojure way of looking at those issues

14:00 Modius: Clojure does do releasing of locals on tail-calls, but no mid-function analysis

14:03 Modius: rhickey: Thanks! I'm interested to know if the caller releases its reference to a function, and if the behavior is inherently dictated by JVM to preclude if not.

14:03 (Or by Java calling conventions)

14:04 rhickey: no help from Java/JVM, there is no TCO, so the references are still on the stack. I manually set them to null before the tail-call.

14:13 Modius: I meant on "normal" calls - function to function

14:13 (somefunction (make-huge)) <-- Does my call maintain a reference as well, or does somefunction have access to all of the references? (Which it could, theoretically, null out)?

14:15 rhickey: all references are live until you make a tail call, no mid-function releasing

14:16 Modius: I'm talking about non-recursive cases now.

14:16 The normal call to a normal function.

14:16 (Thanks for the answers btw)

14:17 rhickey: tail has to do with the position of the call, not its recursiveness. a tail call is a call whose return value is the return value of the function that called it

14:18 (defn foo [] (bar) (baz)) - baz is a tail call

14:24 Modius: Thanks for helping me out with terminology. (defn foo [] (bar (make-huge)) (baz)) <-- when is the "huge" eligible for collection here - and to what degree is THAT dictated by Java calling?

14:26 rhickey: the stack for the call to bar should be unwound on return from bar, and thus huge eligible for collection then, no help needed from me.

14:27 But in this case, I release bar before calling baz:

14:27 (defn foo [] (let [bar (make-huge)] (baz)))

15:27 MarkJP: rhickey: I swapped a netbeans static java framework class with a clojure generated java wrapper that forwards calls to a clojure proxy....

15:28 object gets created on startup as it did before and the methods are being forwarded to the clojure proxy at runtime...

15:29 I am getting an assertion in one of the proxied methods about not being in an AWT thread

15:29 Assertion failed. WindowsAPI is required to be called from AWT thread only

15:30 I should still be on the AWT thread as the wrapper is being triggered just the same as it was before

15:55 ericthorsen: rich: If a clojure proxy is created on 1 thread and later called from another thread there _should_ not be a problem there yes?

16:20 rhickey: eric: should be no problem

16:21 MarkJP: how do you ensure it gets called on the AWT thread?

16:33 MarkJP: rhickey: If I replace the forwarding call with the original java code, it works

17:31 ericthor: rich: The threading issue was related to a override of a protected method needed at the java class level...guess we will have to look at that as well.

17:32 rhickey: you can override protected methods in proxies

17:32 MarkJP: how do I represent the backslash character in clojure

17:32 vincenz: rhickey: thanks for reply on list (I'm Christophe Poucet)

17:33 Chouser: MarkJP: \\

17:33 ericthor: I does bring up a question: What happens if you have a protected method in your proxy call? Does the clojure compiler cough on that?

17:33 rhickey: you can't access protected members in a proxy

17:34 ericthor: i know...but you can add additional clojure functions not overridden yes?

17:34 Chouser: MarkJP: that's either in a string literal, or a literal directly in the code: (print (str "hi" \\ "there\\Mark"))

17:34 MarkJP: cool thx

17:34 rhickey: eric: no, you can only implement/override methods from superclass/interfaces

17:35 MarkJP: and forwardslash is \/

17:35 ?

17:35 ericthor: remember, I'm generating the java class source from the clojure code...I was wondering if I could take advantage of that to have the static java class override the protected member and forward the call to a clojure call.

17:38 rhickey: if you are generating a Java shim class you can define a public method that exposes the protected member, then you can use that in Clojure

17:39 ericthor: ok...just wondering what options were open to me there.

17:40 rhickey: protected member exposing could be a good feature for he stub generator

17:41 Chouser: MarkJP: \/ or just "/"

17:44 ericthor: agreed

17:45 MarkJP: ericthor: there where 2 problems, the protected method, and overriding a method I shouldn't have

17:51 ericthor: mark: ok

17:51 ...and oops

18:04 vincenz: rhickey: how hard was it to make STM?

18:22 rhickey: vincenz: hard

Logging service provided by n01se.net