#clojure log - Jun 30 2008

The Joy of Clojure
Main Clojure site
Google Group
List of all logged dates

8:19 StartsWithK: anyone knows did yrb maybe changed his nick or something? two week ago we had discussion about wrapper for jogl lib and he expressed interest in using it.

8:20 repo is still at http://freehg.org/u/kresimir/neman/ not finished, but there are now 6 examples and i uploaded couple of other thigs.

8:20 so.. if anyone is interested

10:20 cemerick: So, I'd like to use a namespace via some alias -- e.g. say there's a foobarbaz namespace that I want to use, but its name is too long, or it conflicts with some name I want to use, and I don't want to refer it (i.e. (refer 'foobarbaz)). What I'd like to do is something like (alias 'foobarbaz 'someprettyname), and then be able to use foobarbaz defs and such via someprettyname/*.

10:21 I put together a macro that sorta-kinda works, but:

10:22 (a) I'm not sure I'm touching everything I need to in order to "ball up" the referred-to namespace into one that has the more preferable name,

10:22 and (b) this route leads to serious namespace pollution (i.e. lots of top-level namespaces that exist only as aliases)

10:23 rhickey: there will be namespace aliases at some point

10:23 cemerick: OK. I won't plumb down that dead end anymore, then.

10:23 rhickey: I can't imagine a good user-level solution

10:23 cemerick: Yeah, it got pretty messy pretty quick for me.

10:24 FWIW, I'm putting this out there to resolve the long-namespace problem related to aligning namespaces with Java packages (brought up here: http://groups.google.com/group/clojure/browse_frm/thread/4b875918d3255119?hl=en)

10:26 rhickey: I think Clojure will move towards Java-style namespaces, which are long, so aliases will be needed. Also to putting clj files in directories mirroring what Java does. This will allow me to compile bytecode with the Clojure namespaces and have Java tooling be able to find it

10:28 cemerick: Nice. I wasn't sure where you were heading with that, as previous comments you made in the group sounded skeptical of having one-to-one relationships between directories and packages/namespaces. +1 to that, IMO.

10:32 rhickey: when in Rome...

10:39 Chouser: Coming from C++-land, I've never been sure I liked Java's mapping of namespaces to directories, so I appreciate that hasn't been assumed all along here.

10:40 Ending up that way for good reasons, though, is entirely fine by me. :-)

10:41 rhickey: There are lots of things that leverage classpath-relativity

10:42 Chouser: Did create-struct used to act like defstruct does now?

10:43 rhickey: it's used by defstruct

10:44 elg: n i

10:44 -chat then i

10:44 oh, wrong channel sorry. (stupid client)

10:44 rhickey: Chouser: did you see seque?

10:46 I was wondering why you used Deque? (not in Java 1.5)

10:48 Chouser: rhickey: I did, I'm still picking through it.

10:49 Handling of nil and false is very nice.

10:50 The reason I did *enqueue* the way I did is because my initial use-case will have a sax parser as the producer, so I won't have the opportunity to return my seq members. Hence *enqueue*.

10:51 But perhaps it would be better to decompose those two parts (seque and *enqueue*) somehow.

10:51 rhickey: You can't make a lazy-seq on the parser?

10:51 Chouser: I'm gratified you caught the purpose @agt :-)

10:52 (to propogate exceptions)

10:52 rhickey: thanks for the docs :)

10:52 I didn't at first

10:52 Chouser: this *is* how I'm making a lazy-seq of the parser!

10:52 rhickey: oh, sax parser is callback based...

10:53 Chouser: right.

10:53 rhickey: yuck

10:53 Chouser: there are pull-parsers out there, which is a much nicer model, but they're not bundled with Java.

10:54 rhickey: if we standardize the sentinels, we could have a queue-seq

10:54 then just have your callback fill a queue

10:55 seque could use queue-seq

10:55 Chouser: I think I'd still need a separate thread for the parser and callbacks.

10:55 rhickey: QNIL, QEOS

10:57 Chouser: I did have a question that pertains to seque and my original ... if nobody consumes to the end of the seq, we've got a blocked thread hanging out there.

10:57 Will the thread be GC'ed?

10:57 rhickey: nope

10:58 Chouser: yeah, that was my guess. so it'd be easy to use the seq in a lazy context and leak threads. :-/

10:58 rhickey: yeah, I thought about a weak ref to the queue in the agent fn

11:00 java.lang.ref.WeakReference

11:01 * abrooks_ saw a very interesting talk on Friday regarding memory and virtual adress reclamation for long-term stale objects (not quite weak-refs, not quite GC).

11:01 abrooks: http://www.usenix.org/events/usenix08/tech/full_papers/tang/tang_html/index.html

11:02 Chouser: Are threads ever GC'ed while they're still running (or blocked)? I kinda assumed they wouldn't be.

11:02 abrooks: Only tangentially related but I thought it was interesting and perhaps of general interest.

11:03 Er, this is a better link: http://www.usenix.org/events/usenix08/tech/tang.html

11:03 rhickey: abrooks: very neat. What's nice for Clojure is as things like this end up in the JVM, Clojure gets them for free

11:03 abrooks: rhickey: Right.

11:10 Chouser: rhickey: I used Deque because I'm a complete Java noob. I wanted "Linked" instead of "Array", and didn't notice there was a "Queue" version. I spent a couple minutes trying to figure out what the difference was, but since it worked for me I just gave up and used it.

11:10 I see now that Deque is a Double Ended Queue, and I don't need that at all of course.

11:11 rhickey: ah. I just use the 1.5 JavaDoc so I don't even see things I can't use

11:11 trying to keep Clojure 1.5 compatible

11:13 drewr: rhickey: Those of us still on OS X 10.4 thank you. :-)

11:13 rhickey: that would be me too :)

11:14 Chouser: Hm, and my wife. So I guess I should also be greatful.

11:15 drewr: What are the big improvements 1.5 -> 1.6?

11:16 Chouser: Well, mainly these nifty double-ended queues.

11:16 ;-)

11:17 drewr: Interesting. I've never seen native support for deques in any language before.

11:22 abrooks: drewr: Python has deques: http://docs.python.org/lib/deque-objects.html

11:22 * abrooks tries to remember when deques arrived in Python

11:22 abrooks: Python 2.4

11:23 Google reminded me.

11:23 drewr: abrooks: By the beard of Zeus! I've never noticed that.

11:23 abrooks: :)

11:24 * abrooks reads PEPs on a regular basis. That makes it hard, however, to remember what goes in when and what falls by the roadside.

11:24 lisppaste8: rhickey pasted "seque with reclamation" at http://paste.lisp.org/display/63042

11:25 drewr: I check out PEPs too, but generally after the fact to see the motivation for a particular feature.

11:26 lisppaste8: rhickey annotated #63042 with "seque" at http://paste.lisp.org/display/63042#1

11:26 abrooks: drewr: http://www.python.org/dev/peps/peps.rss

11:28 rhickey: Chouser: that one cleans up threads when no one is using the seq anymore - even works for unbounded seques on infinite seqs, if you don't run out of memory first

11:28 drewr: abrooks: Thanks.

11:28 Chouser: rhickey: very cool.

11:29 is it using weak-NIL to notice when it's no longer needed?

11:29 it == agent, I guess.

11:29 rhickey: right - the consuming seq has a strong reference to NIL, the agent doesn't

11:30 Chouser: if you constrain the size of the queue and cause the agent to block, might it get stuck there and no go around again to discover it's not needed?

11:31 cemerick: We still need to keep PDFTextStream JDK 1.4 compatible -- there's a *lot* of people still on 1.3/1.4, and not planning on moving any time soon.

11:31 rhickey: Chouser: ugh, yup

11:31 Chouser: rhickey: sorry to snipe. :-/

11:31 * cemerick is responding to comments from 1/2 hour ago again :-P

11:33 Chouser: WeakReference doesn't seem to have a way to get some code run when the reference is broken. :-(

11:34 rhickey: that wouldn't work - but it can put them on a queue

11:34 I could use timeouts...

11:34 Chouser: oh! a timeout wouldn't be too bad.

11:35 .poll

11:36 rhickey: no, on the offer side

11:37 Chouser: but for all your work to do my use case any good, I've got to figure out how get callbacks in there.

11:37 oh, right. .offer

11:38 I'd rather not have 2 extra threads when I really only need 1.

11:49 hm, ([s] ... [[... :as s] s]) ... you'd never know s is immutable. ;-)

12:05 kotarak: is creating a closure with fn cheap?

12:19 rhickey: kotarak: yes, the fn gets compiled once and the fn value is a simple ctor call

12:22 kotarak_: ok. the problem was (is?) a macro which is needed because not all arguments are to be evaluated immediately.

12:22 now I wondered whether to put everything in a macro or into a driver function with a small macro wrapper.

12:23 rhickey: the latter

12:23 kotarak_: The wrapper creates a fn which may (or may not) be called by the driver

12:23 ah. ok.

13:01 lisppaste8: rhickey annotated #63042 with "seque w/timeout" at http://paste.lisp.org/display/63042#2

13:23 Chouser: rhickey: cool. Now for my sax parser... ;-)

13:24 my general approach of providing *enqueue* as a binding that my callbacks can use is sound, right?

13:24 rhickey: can we not just Lego-ize the seque code?

13:25 Chouser: I hope so, but I want to make sure I'm headed in the right direction.

13:26 I don't see how to get sax+callbacks to populate a seq or queue without it having its own thread.

13:26 my callbacks have to push data somewhere, and the place that makes sense is an LBQ

13:26 rhickey: can you just send an agent off with the sax parse method?

13:27 yes, push onto LBQ

13:27 Chouser: yes, that's what I was using the original seque (or with-bg-queue) agent and LBQ for.

13:29 I guess what I think I have to do is have something almost exactly like your seque (hopefully built out of the same parts) with the only difference being the agent fn

13:29 rhickey: when I look at seque, I see <-seq<-queue<-seq

13:30 and your parse: <-seq<-queue<-parser

13:30 so, queue-seq: <-seq<-queue seems to be a common part

13:32 Chouser: yes, but seque is controls the loop for the right-hand <-seq part, and is calling rest on it. I can't make my parser behave like that without essentially doing my original with-bg-queue stuff again.

13:33 so I guess I'd like to be able to ask seque (or a slightly different version of it) to still provide *enqueque* or *lbq*

13:33 rhickey: (defn seque [n s]

13:33 (queue->seq (seq->queue n s)))

13:34 Chouser: in fact *enqueue* ought to now do the keep-alive detection stuff and throw an exception or something the unwind the sax parser if its not needed.

13:35 s/the unwind/to unwind/

13:37 rhickey: (defn sax-seq [n xml]

13:37 (let [q (LBQ n)]

13:37 (send-off agent sax->queue xml q)

13:37 (queue->seq q)))

13:38 pseudocode of course, keep-alive the trick

13:40 queue/seq protocol will have to define eos/nil/exception sentinels

13:41 i.e. put exceptions in queue rather than tie consumer to producer agent

13:47 Chouser: ok, maybe I'm beginning to understand how that might work.

14:00 of course somebody in that agent needs to know about the sentinels and how to use them. If LBQ is the stock Java class, then sax->queue itself would have to be dealing with keep-alive, NIL, .offer, etc. which seems a shame.

14:01 rhickey: a function can hide some of that

14:01 Chouser: yes

14:02 I think we're actually approaching the same solution. But I'm still trying to understand if my *enqueue* is Bad or not.

14:27 Lau_of_DK: Chouser: Are you working on lazy xml ?

14:31 Chouser: Lau_of_DK: yep

14:31 Lau_of_DK: Interesting - How close are you to finishing ?

14:31 Chouser: well "working" is a bit of a stretch, but yeah that's the goal.

14:31 quite a ways.

14:31 Lau_of_DK: Please let me know when you complete, I'd love to see it

14:32 Chouser: once we've got seque hammered out, it'll be easy to generate a lazy seq of sax events. That's very close to done.

14:32 Lau_of_DK: Its it buggy or just not running ?

14:33 Chouser: but then I've got to build a lazy tree from that seq. I have a plan, but it's another whole stage.

14:33 the pieces just aren't written yet. no bugs. ;-)

14:33 Lau_of_DK: k

14:51 lisppaste8: Chouser annotated #63042 with "seque and with-seque" at http://paste.lisp.org/display/63042#3

14:52 Chouser: rhickey: I'm sure you'll hate the ugliness of using *enqueue* internally in seque, but it gets the job done without extra threads or duplicating code.

16:45 I can't use :when and :while in the same for?

16:46 rhickey: no, do you need to?

16:47 bpattison: should (pr "\u0022") produce "\"" with an extra backslash? I was expecting just a double quote (")

16:49 rhickey: pr prints readably, try print

16:50 Chouser: yeah, I want (filter p1 (take-while p2 s))

16:50 bpattison: okay -- that works -- thanks

16:51 Chouser: I was hoping to write that (for [i s] :when p1 :while p2)

16:52 no big deal. the classic formation's nearly as good as the for-comprehension would be.

16:53 kotarak: clojure is so much fun: I'm trying to understand monads and I implemented a let-bind which is in no way more wordy than Haskell's >>= notation. :)

16:54 rhickey: what does let-bind do?

16:56 kotarak: it does basically simplify: (bind m1 (fn [x1] (bind m2 (fn [x2] (do-something-with x1 x2)))))) to (let-bind [x1 m1 x2 m2] (do-something-with x1 x2))

16:58 rhickey: what does bind do?

16:59 kotarak: bind does: (defn bind [m k] (fn [x] (let [y (m x)] ((k y))))

16:59 oops. just a second. confusion

17:01 ok. It depends on the monad, I suppose (as I as said: just trying to understand). For the identity monad it is: (defn bind [a k] (k a))

17:04 For the error monad (whatever this may be): (defn [[t a] k] (cond (= t 'success) (k a) (= t 'failure) ['failure a]))

17:04 defn bind, of course

18:52 * meredydd waves

18:52 meredydd: What's the current status on regexps?

18:52 I heard rhickey talking about the possibility of making #"patterns" no longer instances of java.util.Pattern

18:53 (for the purpose of making them invokable)

18:53 And what's the situation wrt escapes?

19:18 rhickey: meredydd: no changes near term on regex

19:19 meredydd: Okay. So, it's still good old-fashioned leaning toothpicks?

19:19 rhickey: yup

19:20 * meredydd swears, but only gently.

19:20 meredydd: Native reader syntax for regexps is such a win to start with, it feels churlish to complain :)

19:21 rhickey: there have been several threads but each has petered out before consensus

20:29 slava: rhickey: does clojure allow reader macros?

20:33 rhickey: slava: no

21:58 Chouser: is having both drop and nthrest intentional?

22:01 more importantly, is there a function that takes a seq and returns the same lazy seq except shorter by one?

22:03 slava: rest?

22:04 Chouser: ah. :-) right, but shorter on the other end.

22:05 Sorry I didn't make that clear.

22:05 blackdog: butlast?

22:05 Chouser: right!

22:05 oh, it's there already. perfect, thanks.

22:05 blackdog: but it's not lazy?

22:05 slava: it can't be lazy, it has to evaluate the sequence to get the length

22:08 Chouser: it could be lazy, just ahead by one. Which is what I need, so I guess I'll write it.

22:10 (defn but-last [[n & r]] (when r (lazy-cons n (but-last r))))

22:10 not too painful I guess

22:10 rhickey: any reason the builtin butlast should be eager?

22:15 hm, I guess mine chokes if you give it a zero-length seq. that's ok for my use case though.

22:30 rhickey: Chouser: no reason, cgrand pointed that out, will fix at some point

23:33 Chouser: well, I've got lazy xml parsing working. a drop-in replacement for src/xml.clj

23:51 It's still pretty rough, but here it is in case anyone wants to throw rotten tomatoes: http://n01se.net/paste/nRFa

Logging service provided by n01se.net