#clojure log - Jun 01 2009

The Joy of Clojure
Main Clojure site
Google Group
IRC
List of all logged dates

4:21 cads: hey, would a multilanguage VM running java do tailcall optimization?

4:30 cgrand: cads: http://wikis.sun.com/display/mlvm/TailCalls the experimental branch of the JVM does TCO

4:35 cads: cgrand I could smooch you but I won't

4:36 this is great!

8:04 hsuh: hi... whats the current recipe for installing slime+swank? svn versions aren't working for me... anyone?

8:07 jdz: hsuh: you need swank-clojure

8:07 hsuh: from the git i think

8:07 hsuh: yes, all that

8:08 so what i currently get is: the REPL starts and is functional, but i get some errors (java.lang.Exception: Unable to resolve symbol: lazy-seq in this context (core.clj:70), java.lang.Exception: No such var: swank.swank/ignore-protocol-version (NO_SOURCE_FILE:3), java.lang.Exception: No such var: swank.swank/start-server (NO_SOURCE_FILE:5)) and slime keeps polling or something

8:21 hm... here we are.. replacing lazy-seq (not found) with seq makes things work.. anyone knows if that was removed from clojure or i need to require a library before calling it ?

8:25 Chouser: hsuh: yeah, lazy-seq has been gone awhile.

8:25 cemerick: it sounds like you have an old version of clojure. lazy-seq is definitely in clojure, since before v1.0 was marked.

8:26 Chouser: oh

8:26 cemerick: wha? :-/

8:26 Chouser: uh

8:26 sorry. I blame monday.

8:26 cemerick is, of course, correct.

8:26 cemerick: whew, I thought I was going bananas!

8:26 Chouser: you were probably thinking of lazy-cons

8:27 Chouser: cemerick: yes, of course. Thanks for making my mistake sound reasonable. :-)

8:27 cemerick: Chouser: heh. How goes the new gig, anyways? I hope all's well in general.

8:27 Chouser: yep, thinks are well, can't complain, etc. :-)

8:28 checked in some clojure code last week, so we'll see how that goes.

8:29 cemerick: is clojure still in skunk-works mode there?

8:30 Chouser: a little higher profile than that, but only in "prototype" code for now -- slated to be replaced by something that performs better later. ...again, we'll see. :-)

8:32 cemerick: Good luck. We actually just made some huge strides in performance. We now have a use-case for pmap, which *really* helps -- our older data structures weren't parallelizable, but starting early last week, we switched to a new set of impls that are...and wow, the difference is fun :-)

8:37 Chouser: ah, great.

8:38 did you see Rich's comment about pmap and chunked seqs?

8:39 cemerick: uh, no. My list inbox is oh-so-neglected.

8:39 Chouser: nah, it was a passing irc comment.

8:39 cemerick: ah. log link?

8:40 Chouser: http://clojure-log.n01se.net/date/2009-05-29.html#07:40

8:41 cemerick: oh, so chunking (not that I'm clear about what chunks are at all, of course) makes seqs almost naturally parallelizable, in that the work partitioning is almost done for you....?

8:42 Chousuke: hmm

8:42 cemerick: Our two usages of pmap happen to be in cases where we always have to invoke a particular fn over a dataset with a couple of different parameters, making for an obviously-parallelizable process.

8:43 Chouser: not quite. more like the overhead of doing 'next' (calling down the stack, allocating a new seq, etc.) is enough that in a lot of cases pmap doesn't make sense right now.

8:43 cemerick: Oh, I see. Reaching for the stars, I am :-)

8:44 Chouser: but with a chunked seq (such as a seq over a vector, for example) that overhead is required much less often, so pmap may make sense of less heavy funcs.

8:44 :-)

8:45 hsuh: cemerick: i just compiled from svn...

8:45 cemerick: I was thinking of how seq producer might know how to partition a dataset better than its consumer, so the chunks could be sized appropriately for pmap-ing.

8:45 Chouser: seq producers do the chunking, yes.

8:45 cemerick: hsuh: hrm. I don't work off of trunk, but I'd be very surprised if rich pulled something like lazy-seq...

8:46 hsuh: maybe i need to require something (lazy) ?

8:46 Chouser: hsuh: can you check at a repl if you've got lazy-cons?

8:46 hsuh: ser=> lazy-seq -> java.lang.Exception: Unable to resolve symbol: lazy-seq in this context (NO_SOURCE_

8:46 Chouser: if you do, something's definitely amiss.

8:46 cemerick: hsuh: no, it's in the std lib, as it were. I'd grab the latest release jar, start with that to get a baseline, and then move on to build from sources if you're so inclined.

8:46 hsuh: user=> lazy-cons java.lang.Exception: Can't take value of a macro: #'clojure.core/lazy-cons (NO_SOURCE_FILE:0)

8:46

8:46 cemerick: yeah, that's very old

8:46 hsuh: cemerick: k thanks

8:47 i've done something wrong :)

8:47 cemerick: hsuh: what svn URL are you using?

8:47 Chousuke: hsuh: did you get it from sourceforge? :P

8:47 cemerick: man, it's a groupthink in here :-)

8:47 hsuh: hmm err :)

8:47 yes...

8:47 Chousuke: heh

8:47 hsuh: ops

8:47 cemerick: hsuh: http://code.google.com/p/clojure/

8:47 Chousuke: clojure is on google code nowadays

8:48 hsuh: and i thought i was on the edge

8:48 hehe

8:48 cemerick: wow, I didn't think the SF svn was still up and running. Seems like it should be switched off.

8:48 hsuh: i'll ping jochu to update github page for swank-clojure

8:48 because github feels updated... so i blindly followed it..

8:49 well, thanks a lot

8:50 cemerick: hsuh: hth :-)

10:58 hrm, I'm getting odd pauses at the end of compilation cycles -- I suspect dangling agent threads. Anyone hit this before?

11:02 ah, removing the pmap calls I just added makes the compilation process finish immediately as before....

11:25 hiredman: http://groups.google.com/group/clojure/browse_thread/thread/b6f192eca22f6820 <-- the kind of maddess that insues if you combine quote and syntax-quote

11:27 Chouser: cemerick: didn't see your posts, sorry. If you use futures or agents (which pmap does), you'll have to shut down the agent pools to exit cleanly.

11:27 drewr: ..and even then maybe not :-)

11:28 cemerick: Chouser: I'm about to post to the group with a patch. To be clear though, none of the pmap calls were *invoked* -- but their usage in the code probably caused the loading of c.l.Agent, which sets the threadpools off on their way.

11:28 Chouser: well, I've been doing a lot of that, and haven't had issues. I saw there was a thread, but didn't read into it deeply enough to spot the difference.

11:28 hm.

11:29 cemerick: Chouser: the patch I'm proposing adds an Agent.shutdown() call to finally blocks in the entry points of clojure.main and clojure.lang.Compile. Does that seem reasonable?

11:29 Chouser: I don't think your diagnosis is correct.

11:30 clj -e "(fn [] (agent nil))" ;exits promptly

11:30 clj -e "(agent nil)" ; exits promptly

11:30 clj -e "(send-off (agent nil) identity)" ; hangs

11:30 cemerick: ah, that code *is* run, so yes, the agents are being used. My bad.

11:31 Chouser: cemerick: and so you know, agent shutdown calls and/or System/exit calls have been added and removed from the various entry points a few times. People always complain either way.

11:31 cemerick: (creating a default EMPTY value)

11:31 hrm.

11:32 Chouser: the problem the other direction being if you have an agent or something that shuts down the pool when it's the last one out, and your main program just falls off the end, an automatic pool shutdown would break things.

11:32 cemerick: well then, certainly no one could complain about a shudown call in clojure.lang.Compile, which would fix my immediate problem.

11:32 Chouser: hm.

11:35 cemerick: In general, it doesn't seem like everyone should keep tripping over this problem in different directions. It's a frustratingly difficult thing to debug if you're not attuned to it already. :-[

11:39 Chouser: in my experience, for any particular app it's pretty hard to know when all other threads are done and it's safe to exit.

11:39 drewr: cemerick++ It's extremely difficult to debug. I think I have mine figured out and then I see different behavior.

11:39 Chouser: it's hard for me to imagine how it could be solved automatically.

11:39 rhickey: hmm... #clojure on the plane - awesome

11:41 Chouser: rhickey: nice.

11:41 heading to CA?

11:41 drewr: Chouser: Yep, I finally had to remove agent activity from my script. Couldn't determine even for my own program what the right behavior was.

11:41 replaca: rhickey: cool. best wishes for the script bowl tomorrow!

11:41 rhickey: yup

11:41 cemerick: Chouser: why not declare that agents are shutdown when the main thread exits, period, and if you don't want that to happen, join as necessary?

11:41 rhickey: replaca: thanks!

11:41 cemerick: rhickey: Indeed, good luck :-)

11:41 rhickey: the 4 minute limit is quite short

11:41 cemerick: I just don't see why everyone that works with agents should have to trip over this particular issue.

11:41 replaca: rhickey: do you want me to make a hand-out/poster for you for the clojure meetup, or do you want to just let that be?

11:41 cemerick: or actually, not even agents, just a simple pmap usage ropes you into the same area...

11:42 (as in my case)

11:42 rhickey: replaca: I don't know that I could hand them out, but you could

11:43 replaca: rhickey: I won't be there - I guess I could come and see if there are some public places to post

11:43 rhickey: replaca: ah, right

11:43 replaca: I think they'll only check at the entrance to the sessions, so that would probably work

11:43 rhickey: I imagine those who would come are on the ggroup or here

11:44 replaca: rhickey: I'll do that.

11:44 rhickey: Yeah, I think you're right + anyone you sufficiently inspire

11:44 The twittersphere has had a bit of buzz about Stuart's book!

11:46 Chouser: cemerick: I guess that may be reasonable. For the cases where my code is carefully collecting the threads before agent-shutdown, it would be no harder to do that work and then rely on automatic agent-shutdown. *shrug*

11:46 triddell: rhickey: I'm sure you have other things on your mind but I wondered if Issue #100 on the tracker had any plans to get attention. It's a bug that basically stops compiled classes from working on IBM JVMs.

11:47 djpowell: Is there any way to dispatch on array types in multi-methods - or to use hierarchies to derive from array types

11:47 cemerick: Chouser: yeah, I'm not at all clear on what the solution is, but I can't imagine how frustrating it would be as a complete noob to hit this issue. That's why I'm agitating :-)

11:47 rhickey: triddell: I'd welcome a patch

11:48 cemerick: do you have a proposal?

11:48 * rhickey flew in late

11:49 triddell: rhickey: A patch was attached to the post that describes the original issue (it wasn't my post/patch so I didn't attach it to the issue)

11:49 clojurebot: rhickey is a T-1000 sent from the future

11:49 cemerick: rhickey: well, an immediate fix to keep compilation runs from hanging for 60s (add Agent.shutdown() in the finally block of main). I don't think anyone could complain about that.

11:49 triddell: rhickey: http://groups.google.com/group/clojure/browse_thread/thread/e64719d716c29ce0

11:50 rhickey: the patch works for me but I didn't know if it might have any other ramifications

11:50 cemerick: rhickey: On the solution in general, I'm generally aiming for something that will result in sane behaviour for noobs, and something that experienced users can easily work with (my current idea being, shutdown on exit from clojure.main in every case, and if you're clever enough to know that that's not what you want, make your main thread join until you're ready to be done).

11:51 Chouser says that's gone back and forth a couple of times, but the current state of play doesn't seem reasonable for the common case, or for new users.

11:51 (i.e. difficult-to-explain "hanging")

11:52 rhickey: triddell: someone with a CA needs to attach it to the issue

11:53 djpowell: Is there any way of writing an array class literal? Something like Ljava.lang.Object;?

11:54 rhickey: cemerick: I think there needs to be a recipe for both scenarios, by join you mean what?

11:55 djpowell: you need the class object or just a type hint?

11:56 djpowell: well, I wanted to do multi-method dispatch, with 'class' as my dispatch function, and have it handle array types

11:56 or - alternatively, to use derive to handle array types the same as lists - with a seq wrapper or something

11:57 cemerick: rhickey: sorry, that was a poor term choice. I just mean that if you want to delay your program's exit until your agents are done doing their thing, you should be responsible for holding the main thread until you're ready to exit. Lots of ways to do that.

11:57 triddell: rhickey: ok, I'll send a CA off and ping you about the issue after you've received it

11:58 cemerick: djpowell: we're just using (defmethod foo (class (make-array Character/TYPE 0)), etc

11:59 rhickey: ,(Class/forName "[Ljava.lang.Object;")

11:59 clojurebot: [Ljava.lang.Object;

12:00 djpowell: ah - I had some strange idea that the dispatch values had to be literals for some reason

12:01 rhickey: cemerick: we need to automate that, lest we make a change and the other side complains again, round and orund

12:01 djpowell: that will work ok - or I could just change the dispatch function

12:01 replaca: rhickey: on second thought, I may not sell the meetup too hard - we've already got about 35 people signed up and the space will be crammed with 50. Feel free to mention it in your talk or to interested folks and just give them my email/number for info.

12:01 rhickey: djpowell: dispatch value do not need to be literals

12:01 replaca: fair enough - I'm looking forward to it

12:02 replaca: rhickey: us too!

12:02 djpowell: rhickey: yeah cool. although on the subject of arrays, I did have some trouble type hinting that something was a char-array

12:02 cemerick: rhickey: I can see people wanting to do it in a lot of different ways. I hesitate to think I'd have enough perspective to propose a general-enough solution.

12:03 rhickey: A sneaky option would be to trim back the keepalive of the threads in Agent's Executors' threadpools. It's 60s now, and could be a lot lower.

12:05 rhickey: hit 2000 members on group today!

12:08 cemerick: really? It says 1983 right now...

12:08 rhickey: cemerick: I don't know how often they update that, the admin screens say 2000

12:08 cemerick: ah

12:09 rhickey: cemerick: what about letting people set *auto-shutdown-agents*, default true? puts it in their hands without establishing a policy

12:12 stuhood: rhickey: would it be possible to model them like threads, where each agent could be a 'daemon'?

12:12 cemerick: rhickey: yeah, that's fine by me. :-) I think c.l.Compile should just have it hardcoded, though -- there's no user code in involved in that path that could customize the behaviour, anyway.

12:20 rhickey: biab

12:30 replaca: dysinger: are you in the channel?

12:35 dysinger: yes

12:38 replaca: dysinger: are you coming to SF for the clojure meetup? I saw you said something like that on Twitter.

12:38 dysinger: If so, I want to make sure you're on the rsvp list

12:38 dysinger: y

12:38 I wasn't able to get logged in and rsvp yet

12:39 yes thank you - put myself and phil hagelberg on there

12:39 Tim Dysinger

12:39 replaca: dysinger: Don't worry, consider it done. I already had Phil on my list.

12:40 Anyone else coming can just ping me here or email me and I'll put you on the list

12:40 dysinger: thanks replaca

12:43 replaca: dysinger: np

12:46 Roux: anyone here got the 'programming clojure' book?

12:46 dnolen_: yes

12:47 triddell: triddell: yes

12:47 Roux: does it do a good job of explaining functional programming to people not familiar with that paradigm?

12:47 triddell: Roux: oops, yes

12:47 danlarkin_: replaca: wish I was on the west coast :) consider me there in spirit!

12:47 Roux: triddell: does it do a good job of explaining functional programming to people not familiar with that paradigm?

12:48 triddell: Roux: I think it does a very good job... I was new to it before clojure and it was the book I used to "get" it

12:48 Roux: great

12:48 :)

12:49 but with only 200 pages i would think it only has time to explain clojure library itself and no time to help ppl grok functional programming

12:50 triddell: in fact, I'm writing a review on Amazon right now and I'd encourage other readers to do the same and help to promote the book

12:51 replaca: danlarkin_: we all wish you were here too. I envision a future where we'll be able to have our own national events like the Ruby guys do.

12:52 Roux: replaca: you a python or a ruby guy?

12:53 replaca: Roux: both, to a certain extent

12:53 Roux: which softens your heart most of all

12:54 ?

12:54 triddell: well, it's two hundred pages of functional programming! because clojure is a Lisp it can cover what would take another language 200 pages in chapter or two... the rest is practical examples and api specifics (the majority of which highlight its functional nature)

12:54 dnolen_: Roux: also compared to Python and Ruby, Clojure is fairly miniscule. Daunting at first but then you realize how small Clojure actually is.

12:54 triddell: Roux: arg, well, it's two hundred pages of functional programming! because clojure is a Lisp it can cover what would take another language 200 pages in chapter or two... the rest is practical examples and api specifics (the majority of which highlight its functional nature

12:54 Roux: triddell: yeah bro! i bought the scala book, and it's about 700 pages!!!

12:55 replaca: Roux: well, Clojure's my favorite right now cause I feel like I can really rock'n'roll in it

12:55 dnolen_: compared to Scala, Clojure is nanoscopic.

12:55 Roux: Scala strikes me as very complex

12:55 scarily so

12:55 but quite clever and nifty too :)

12:56 replaca: Roux: python and ruby are a little different, but overall I prefer Ruby. I end up using python more because at work we're based on the CLR and IronPython gives me a nice interface to that

12:56 triddell: Roux: bought that too, was interested and then found clojure, never read the Scala book after it arrived

12:56 replaca: Roux: I've just begun learning scala

12:57 I'm interested in some of the more advanced communication stuff that's being implemented in scala (esp. by the twitter folks) and would like to bring some of that tech into the Clojure fold

12:57 since I'm mostly a distributed systems guy

12:58 Roux: cool

12:58 there is alot to like in scala

12:58 esp. if you come from a c++/java background

12:59 replaca: Roux: yeah, I'm not of the "one language to rule them all" school. Clojure feels best to me, but I don't want to be chauvinistic about it.

12:59 Roux: scala feels quite rubyish too

12:59 i mean you iterate through a list like this in scala: myList.foreach(x => println(x))

13:00 dnolen_: scala doesn't work via messaging tho, so that seems like an odd comparison.

13:00 Roux: it's a pity they didn't credit ruby as an influence on the wikipedia page, cos as a rubyist myself i see alot of ruby in scala and i dont think it's a coincidence

13:01 triddell: Roux: the other day I wrote a simple Rest/Json service for use on Google App Engine in under a hundred lines using Clojure... it was the most fun I'd had programming in quite a while... I'm not a language snob either but fun is fun!

13:01 Roux: yeah i cant wait to get my head around a lisp, the most i've done so far is just muck around with e-lisp a teeny tiny bit and it was alot of fun thinking and programming recursively

13:03 dnolen_: well it seems to me that Ruby -> Clojure is natural, since Ruby was Matz's attempt to merge his two favorite languages, Lisp Smalltalk with the UNIX shell convenience of perl.

13:04 Roux: yeah that also probably explains the similarity of ruby to scala too, in so far as ruby is an amalgum of the oop/functional paradigmns in a similar way that scala purports to be (though it's true that scala values immutability higher than ruby)

13:06 dnolen_: well there's definitely a connection Gilad Bracha (of StrongTalk fame, and developer of NewSpeak) said he's had many conversations with Odersky about Scala.

13:08 Roux: odersky probably just doesn't give ruby as an influence so he doesn't scare off the python dudes, who love to spit on the ground of anything connected to ruby

13:11 hiredman: clojurebot: have you heard the word about scala?

13:11 clojurebot: "you can usually accomplish a lot by laboriously enumerating type arguments everywhere." -- seen in #scala

13:11 Roux: clojurebot: have you heard the word about ruby?

13:11 clojurebot: Excuse me?

13:12 Roux: :D

13:13 Chousuke: clojurebot: ruby is <reply>Chunky bacon!

13:13 clojurebot: Roger.

13:14 Roux: nice one :)

13:15 Chousuke: I hope someone will trigger clojurebot's random auto reply with "ruby" and be thoroughly confused as a result

13:16 hiredman: clojurebot: outback steak house?

13:16 clojurebot: I don't understand.

13:16 hiredman: bah

13:16 clojurebot: steak?

13:16 clojurebot: I want to go to there

13:16 Roux: hehe although i dont know a single rubyist who hasn't read _why's guide ;)

13:25 triddell: finished writing your review yet?

13:25 im curious to see it

13:53 cemerick: replaca: coming in late, but I've begun to ponder what a couchdb clone would look like implemented with clojure. That'd be a big (but interesting) task.

13:55 danlarkin_: cemerick: beware NIHS

13:56 hiredman: a clojureql type deal for couchdb might be nice

13:56 cemerick: danlarkin_: oh, I quiver in terror in the face of NIHS :-)

13:56 ataggart: ~google NIHS

13:56 cemerick: couch-in-clojure is fancy more than anything else

13:56 clojurebot: First, out of 166000 results is:

13:56 NIHS

13:56 http://www.nihs.go.jp/

13:57 cemerick: Our Pacific friends will be happy to know the terror they instill in clojure programmers. :-)

13:57 hiredman: :P

13:57 hmmm

13:57 ataggart: lol

13:59 dysinger: cemerick - why re-write couchdb in clojure ?

13:59 it's awesome as is

14:00 cemerick: dysinger: I have no good reason right now, which (among other reasons) is why I'm not.

14:00 dysinger: I think it would be more interesting to write a couchdb view server for clojure

14:00 so you can write your map/reduce in clojure :)

14:00 cemerick: dysinger: that's something we'll probably be tackling in the next month or two, unless someone else gets there first :-)

14:00 stuartsierra: I already do that with Hadoop.

14:00 dysinger: yucky

14:01 cemerick: we couldn't handle the admin complexity of hadoop, unfortunately.

14:01 dysinger: we are looking at that too - thanks for your example code.

14:01 stuartsierra: welcome

14:01 dysinger: I just don't like how rube goldberg hadoop is at the moment.

14:02 I am leaning towards straight clojure and amqp

14:02 stuartsierra: It's not so bad once you get into it - most of the complexity is for edge-case optimizations.

14:02 dysinger: probably

14:02 hiredman: :|

14:03 dysinger: other thing I want to look at is terracotta and clojure for map/reduce in native lang.

14:03 replaca: cemerick: well, even if we don't need to recreate the wheel, it is interesting to think about how you'd tackle a problem like couchdb in Clojure and whether a Clojure implementation would be a good thing

14:03 dysinger: using agents ore something

14:03 stuartsierra: Does terracotta provide data locality?

14:03 dysinger: no

14:03 all our data is on s3 anyway

14:03 so import to hdfs or disk - doesn't seem to make much diff

14:04 stuartsierra: Most of Hadoop's performance comes from data locality.

14:04 dysinger: y

14:04 true

14:04 in order to keep hadoop happy hadoop must also be your storage engine

14:05 tis the nature of the map/reduce paper....

14:05 stuartsierra: Well, not necessarily. You do have to copy things in and out of HDFS, but you don't have to store it there permanently. I store everything on S3.

14:05 dysinger: er "as in the paper"

14:05 replaca: more generally, it *seems* right that a good set of clojure primitives would allow clojure to enter the problem space currently dominated by erlang in a nicer package

14:05 dysinger: doesn't swap in and out of s3 take a long time with large data sets ?

14:05 that's been my experience

14:06 stuartsierra: Not really, b/c you can parallelize the copy.

14:06 dysinger: across the nodes

14:06 hiredman: replaca: the main problem is all the cool message buses use erlang

14:06 replaca: hiredman: and maybe we need a cooler message bus of our own :-)

14:06 dysinger: still yeah - copying 4TB to each node is going to take a long time.

14:07 hiredman: replaca: maybe just use terracotta?

14:07 stuartsierra: Well, yes. I get around 1GB/min from S3 on an 18-node cluster.

14:07 dysinger: teracotta would be much more fun if it was dynamic language - capable

14:07 replaca: hiredman: could be. I don't understand terracotta well enough to know if it's the answer

14:08 dysinger: you have to AOT compile, instrument things and then describe each generated closure class in their XML files

14:08 and last I checked it required a hack to clojure

14:08 hiredman: dysinger: oh, I did not realize

14:08 dysinger: but still it seems to work

14:09 hiredman: the repl works

14:09 I am not sure how that would work if "dynamic" stuff doesn't

14:09 dysinger: paul stadig did these tests http://paul.stadig.name/2009/02/clojure-terracotta-yeah-baby.html a while back and he has a project on github

14:09 hiredman: yeah

14:09 I've seen them

14:09 dysinger: that's all I've been playing with.

14:09 would be cool to have it working

14:10 that would eliminate the need for hadoop and rabbitmq for me.

14:10 In theory

14:10 and move the problem into straight clojure code

14:10 clojurebot: "There is no problem in computer programming which cannot be solved by an added level of indirection." -- Dr Maurice Wilkes

14:10 dysinger: heh

14:10 hiredman: ~botsnack

14:10 clojurebot: thanks; that was delicious. (nom nom nom)

14:12 cemerick: dysinger: what would you use for primary storage, though?

14:13 hiredman: openmq is java

14:14 cemerick: openmq is jms, jms is painful :-(

14:15 dysinger: we use s3 and xfer to local disk

14:15 cemerick: maybe rhickey's forthcoming queue abstraction will help there

14:15 hiredman: cemerick: :|

14:15 dysinger: nothing beats erlang for messaging

14:16 I don't have any stats but if I had money to bet - I'd bet rabbitmq could take a better beating than any jms server

14:16 hiredman: dysinger: but having everything on the jvm would be nice

14:16 dysinger: I used to think that but it's not such a comfortable thing for me anymore.

14:16 anyway - it's not a requirement for me anyomer.

14:16 anymore

14:17 Our plan is to use rabbitmq & json to make the messaging language neutral

14:18 hiredman: http://clojure-log.n01se.net/date/2008-09-08.html#14:26

14:18 dysinger: we are connecting to rabbitmq with ruby, python, java and soon erlang

14:20 and if we can tie in the messaging primitives in clojure to it -> that would be nice.

14:20 (the currently being thought about but not yet existing primitives)

14:20 hiredman: I just going to say...

14:23 dysinger: hey technomancy - ready for some SF meetup ?

14:25 danlarkin_: cemerick & tashafa: my response on twitter didn't come off as light hearted as it did in my head before I typed it :)

14:25 cemerick: danlarkin_: heh, you worry too much :-)

14:25 technomancy: dysinger: woo!

14:26 cemerick: tashafa: hey, another Bostonite :-)

14:26 (not that I'm in Boston, but it's close, anyway)

14:40 dysinger: technomancy took a bunch of twiddling but I got emacs 23 working with mac environment.plist

14:40 now am happy eshell user

14:41 technomancy: ugh; I'm sorry you had to do that

14:41 dysinger: It's fine

14:41 we should add it to the starter kit

14:41 technomancy: manually editing plist XML files is not a task fit for a human

14:41 Chousuke: there's the plist editor :p

14:42 dysinger: I have a rake task to push to the plist file

14:42 technomancy: yeah, if you can automate it, I'm sure it would save poor future users a lot of pain.

14:42 dysinger: anytime my env vars change

14:42 I have never even used the environment.plist file before -> and I have been OS X since 2002

14:42 technomancy: would prefer a standalone solution, but that's definitely better than the current situation.

14:43 dysinger: I always just used ~/.profile

14:43 but emacs carbon ignores that

14:43 retarded

14:43 !

14:44 technomancy: dysinger: it's because it's launched from the finder/dock

14:44 dysinger: y

14:44 technomancy: I submitted a bug report about that a few years ago

14:44 not exactly holding my breath

14:44 dysinger: eshell should just read .profile like other terminal apps.

14:44 triddell: Roux: my review is up on Amazon... it's mostly a "feel good" piece than a technical review though

14:44 dysinger: s/read/source

14:45 technomancy: it should be easier to source .profile than it is now (as a workaround for the broken OS X behaviour), but the correct behaviour is to inherit

14:46 ataggart: ah nice! the meetup is 2 blocks from my office

14:47 dysinger: I just eneded up including osx-plist.el and adding a (if.... (eq system-type 'darwin)

14:47 block

14:47 it works

14:48 technomancy: dysinger: great; would love a patch

15:25 cemerick: hrm, what's the incantation to get a type hint to emit properly from a macro? I've tried `(.foo #^Bar someBar), `(.foo #^{:type Bar} someBar) so far...

15:26 Chouser: try `(.foo ~(with-meta `someBar {:type Bar}) ...)

15:26 hiredman: interesting

15:26 cemerick: Actually, #^Bar works according to (meta (second (macroexpand ...))), but I still get reflection warnings.

15:27 I meant :tag above, sorry

15:27 rhickey: #^Bar should be fine unless Bar is a variable or something

15:27 Chouser: right, me too. :-)

15:28 rhickey: does a non-macro version of the #^ work?

15:31 cemerick: rhickey: I was actually just trying to type-hint defmacro so as to limit the noise when printing reflection warnings :-)

15:32 rhickey: ?

15:32 cemerick: sorry, defmethod

15:33 Roux: rhickey: you look a bit like wittgenstein

15:34 rhickey: cemerick: ah

15:37 presuming multifn is a symbol is likely but not required...

15:38 leon: Hello

15:38 rhickey: you'll need to add metadata to the form, as Chouser showed, since you are unquoting it, but given it could be an arbitrary expression yielding a

15:39 MultiFn, you might want to wrap it in a call to identiy and add metadata to that call

15:39 identity

15:39 leon: I have a short question: is there a 'length' function in clojure for lists or any seqs? I cannot find any in the api/wiki

15:39 Chouser_: leon: count

15:39 leon: thanks Chouser

15:39 rhickey: Chouser - fastest gun in the west

15:39 ataggart: ,(count [1 2 3 4])

15:39 clojurebot: 4

15:40 leon: could go into the lisp/clojure differences :-)

15:40 hiredman: ,(counted? [1 2 3 4])

15:40 clojurebot: true

15:40 ataggart: ,(doc counted?)

15:40 clojurebot: "([coll]); Returns true if coll implements count in constant time"

15:40 ataggart: nice

15:40 Chouser: it helps when I have #clojure up in mutliple windows on the same desktop. :-P

15:40 cemerick: rhickey: why the call to identity?

15:41 Chouser: aslo wehn I quit caring abuot my tpying

15:45 rhickey: leon: what's the length of a hash table in CL?

15:46 leon: no idea

15:46 rhickey: CL/CL differences

15:47 hash tables don't have length in CL, they have... counts

15:48 only count works across collection types IMO

15:49 cemerick: I guess I'm afraid that somehow the expr used for multifn won't support metadata, just paranoia, but defmethod doesn't dictate it be a symbol - maybe it should

15:52 Chouser: I had hoped I wouldn't have to think about races in Clojure. But you throw enough agents, blocking sockets, and atoms in there and *poof* complexity.

15:52 Roux: rhickey: http://agbellotti.files.wordpress.com/2006/11/wittgenstein.jpg

15:53 rhickey: Chouser: is it inherent in blocking sockets, or is there something missing?

15:54 Chouser: did you look at the executors framework?

15:54 Chouser: If I had a coherent suggestion, I would have described it by now.

15:55 agents are my executor framework api. :-)

15:55 danlarkin_: oh wow! He does really look like wittgenstein

15:55 Chouser: I think the kinds of complexity I'm seeing can be had as soon as you mix agents and atoms.

15:56 or I guess agents examining/responding to any external state

15:56 Roux: danlarkin_: hehehe :D

15:56 triddell: triddell: I hope Rich brought his tweed jacket for JavaOne ;-)

15:56 rhickey: Chouser: yeah, but maybe they aren't a good fit, or need to expose some more of the support, e.g. interruption

15:57 Roux: danlarkin_: quite similar features but he lacks the 1000-yard stare of our friend ludwig: http://peernet.lbpc.ca/Philosophy/images/Wittgenstein.jpeg

15:59 Chouser: Roux: also Rich isn't quite as grainy

15:59 Roux: ;)

16:02 clojure is in good hands: Described by Bertrand Russell as "the most perfect example I have ever known of genius as traditionally conceived, passionate, profound, intense, and dominating", Wittgenstein is considered by many to be the greatest philosopher of the 20th century.

16:04 stuhood: rhickey,Chouser: at the risk of trying to solve a problem that isn't fully defined, perhaps agents should be able to be shutdown individually? such that they will finish any (sends) in the queue, but throw Rejected for new (sends)

16:05 Chouser: stuhood: I very nearly said something like that earlier today. I was thinking more that once all agents were in the finished state, the pools could shut down automatically.

16:05 but I don't know what "all agents" means

16:09 stuhood: yea... it seems like agents really just need to be a thread+queue abstraction: so each agent could be marked as a daemon or not, so that shutting down the pool would wait for all non-daemon agents

16:12 hiredman: (actor)

16:14 stuhood: they execute Actions, so why not? =P

16:15 Chouser: I think rhickey nailed (as I'm suprised) ... replacing several of my agents with single-thread exectors may remove a lot of my complexity.

16:15 hiredman: java.util.concurrent is pretty great

16:18 I feel sort of cheated never having been introduced to it before

16:29 cemerick: aaah, :tag's value needs to be a symbol, not a class! :-(

16:29 Chouser: wow

16:30 sorry, I didn't even consider that.

16:30 cemerick: yup, I'm that slow :-/

16:30 that should really be changed to be more accepting, no?

16:35 ah, no, nevermind, that's just the way it has to be.

16:47 Chouser: gah. why does every different lib have to have it's own callback class (or two or three)

16:47 technomancy: what's up with libraries always requiring the JAVA_HOME env variable to be set?

16:49 Chouser: oh, because they each have different signatures. :-P

16:49 technomancy: anyone want to recommend an alternative to mvnrepository.com that isn't down most of the time?

16:57 cemerick: ugh, beating gensyms into getting type-hinted isn't proving much easier than the last challenge

17:05 ah, blast from the past from cgrand saves the day :-) http://groups.google.com/group/clojure/browse_thread/thread/776f65f8a2f134f7

17:05 speak of the devil :-)

19:45 gcv: when it comes to handling file paths, is there an idiomatic Clojure alternative to using java.io.File? I just typed (.. (java.io.File. "/my/path/here") getParentFile getParentFile) in a source file and it looks verbose.

19:46 technomancy: gcv: c.c.java-utils/file works for me

19:46 though... only a few chars less verbose

19:46 you could do (file "/my/path/here/" ".." "..")

19:49 gcv: technomancy: thanks, that's pretty good. although it doesn't quite return the same thing: #<File /my> for my original example, and #<File /my/path/here/../..> using the file function.

19:49 dysinger: clojure never promised magic/non-verbose java interop :P

19:49 it's still java

19:50 technomancy: gcv: yeah, that's lousy. it'd be cool if "file" would automatically get the canonical version instead of just allowing crazy crap like that

19:51 dysinger: http://java.sun.com/j2se/1.4.2/docs/api/java/io/File.html#getCanonicalFile()

19:51 gcv: yeah, I was just wondering why file doesn't call that before returning

19:52 hiredman: well, it ain't a final class...

19:53 dysinger: (str .getCanonicalPath (file "/my/path/here/" ".." ".."))

19:53 hiredman: uh

19:53 no?

19:54 gcv: hiredman: getCanonicalPath already returns a string, I don't think you need the str call.

19:54 hiredman: and str is not comp, and you cannot compose method calls

19:54 dysinger: trie

19:54 true

19:54 true

19:54 also

19:54 I was just goofing around

19:54 hiredman: I know

19:55 dysinger: I am rusty - it's been a week since clojure coding :P

19:55 need back on horse

19:55 (.getCanonicalPath (file "/my/path/here/" ".." ".."))

19:55 hiredman: dysinger: I saw some video where some guy from australia was presenting about openmq and he said in his testing it beat activemq

19:55 dysinger: cool

19:55 hiredman: of course this was just some random video on the internet

19:56 dysinger: activemq isn't hard to beat - I think it uses derby by default for presistent message queueu

19:56 queue

19:56 hiredman: huh

19:56 well that makes sense

19:56 dysinger: java db

19:56 but you can configure it faster

19:56 hiredman: for some reason I was thinking activemq was erlang

19:56 dysinger: no that's rabbitmq

20:01 achin: I'm trying to port someone's pre-1.0 Clojure code to 1.0. I'm having trouble figuring out why you would test for something being an instance of clojure.lang.LazyCons. Would I want to change this to test for LazySeq? just use seq?

20:03 Chouser: I've never felt the need to test for LazyCons

20:03 what do that do with it if it's a LazyCons?

20:03 what if it's not?

20:03 achin: I'm trying to figure that out. I don't know enough about the library I'm porting.

20:04 hiredman: Chouser: if you want, for example, lazy code to be forced on a sandboxed thread

20:04 achin: This is clj-html. It's the bit of code that dispatches bits the DSL to expand it.

20:04 hiredman: achin: (instance? clojure.lang.LazySeq s) or something like that

20:05 achin: erm, I think clj-html has been updated

20:05 just pull from github

20:05 achin: hiredman: that was my guess, but I don't know enough about Clojure to know if LazySeq is a drop-in replacement for LazyCons.

20:05 hiredman: I *just* pulled.

20:05 hiredman: h,,,

20:06 hmm

20:06 achin: hiredman: It's still using lazy-cons.

20:06 hiredman: And stuff like rrest that's gone now.

20:06 hiredman: http://github.com/mmcgrana/clj-html/tree/master from this?

20:06 Chouser: rrest --> nnext

20:06 achin: chouser: right.

20:06 hiredman: I pulled from git://github.com/mmcgrana/clj-html.git

20:06 hiredman: weird

20:07 achin: Last updated 19 Jan.

20:07 hiredman: oh

20:07 heh

20:07 I fixed it locally

20:07 achin: Heh.

20:07 hiredman: no wonder it works for me(TM(TM)

20:08 achin: Would you mind sharing? I'm just getting into Clojure and I was using this as an exercise to get oriented.

20:08 hiredman: http://github.com/hiredman/clj-html/tree/master

20:09 wow

20:09 achin: Thanks.

20:09 hiredman: this was just sort of a quick fix, so some of it is not the best

20:10 achin: It's not like mine's going to be any prettier.

20:18 Hrm. If I write, (.toUpperCase "text"), everything's okay. If I wrap it in a macro, I get an unrecognized form exception. http://pastebin.com/mcbf21fd

20:22 Heh. That never even worked to begin with.

23:31 Chouser: Using the Executors API helped, I think, to solve the complexity I had, making it easier to see the (different) complexity I need. :-/

23:39 rhickey: heh

23:46 Chouser: If I need to serialize small pieces of work from n different threads, is there a performance consequence for using a single-threaded executor vs. an atom vs. a plain lock?

23:47 I suppose I should read that concurrency book I've seen waved around...

23:48 well, that's not a problem for tonight.

23:49 rhickey: hope you have a great day tomorrow. Knock 'em dead, and all that. :-)

23:49 rhickey: thanks!

23:49 * rhickey crashes

23:57 twopoint718: I'm curious about the role of the "this" in the arguments to "impl-foo" toward the bottom of defn impl-foo [this]

23:57 (str (class this)))

23:57

23:57 (defn impl-bar [this]

23:57 Sorry about that, this was the page I meant to reference: http://clojure.org/compilation

23:58 In particular, it looks like these clojure-generated methods are getting the class as their first argument. Is that correct?

Logging service provided by n01se.net