# #clojure log - May 26 2009

The Joy of Clojure
Main Clojure site
IRC
List of all logged dates

0:38 replaca: +1 for better error messages from the Clojure compiler!

0:39 "java.lang.ClassNotFoundException: java.lang. (PrettyWriter.clj:17)" with like 200 lines of irrelevant exception garbage is just not helpful.

0:45 technomancy: replaca: I hacked the slime trace buffer to dim lines that came from clojure.lang etc.

0:46 ... wow, and it looks like maven has *no* tests for dependency management! woooo!

0:52 replaca: technomancy: yeah, that would help (I'm actually doing this one from shell) but we really need to get generally better errors

0:53 technomancy: replaca: seems like a filtering problem to me

0:53 replaca: technomancy: is it true you're coming to SF for JavaOne and the Clojure meetup?

0:53 technomancy: well, a filtering problem and the underlying error message is just not useful in this case

0:54 technomancy: but reall, a compiler error is not a thrown exception - it's a known problem

0:54 *really

0:54 technomancy: replaca: yeah, I'm coming down to SF

0:55 just for the meetup actually, not for JavaOne.

0:55 replaca: awesome!

0:55 technomancy: yeah, pretty excited

0:55 replaca: do you have something you want to present?

0:56 technomancy: actually I was hoping to catch a peek of Swarmiji

0:56 replaca: we're just doing a few "lighting" presentations

0:56 yeah, me too

0:56 technomancy: may end up contributing to that project as it seems to cover the same ground as what we're doing

0:56 infrastructure-wise

0:56 replaca: I couldn't go down the night Amit showed it in Mt. View

0:57 I did convince Amit to discuss it again at the SF meetup (which wasn't too hard) so we should see it.

0:58 technomancy: great

1:01 durka42: how do you test to make sure a function really is lazy?

1:05 hiredman: you pass it a seq that contains a println

1:05 :P

1:07 replaca: ack. That error message above meant that I hadn't provided a return value for a method in a gen-class :-(

1:12 slashus2: I always liked tesla coils http://www.youtube.com/watch?v=Zi4kXgDBFhw

1:12 woops wrong channel sorry.

1:13 lgas: I always liked the band Tesla

1:42 bradford: I want to create a function that, given a range and a value, returns which slot in teh range the value is between

1:45 hiredman: "slot"?

1:46 bradford: err...but slot i mean (1 5 10) 6 would be in slot 2 (between 1-5 ... slot 1 would be 0 to 1)

1:48 hiredman: :(

1:49 what is this for?

1:49 bradford: it is for a bayesean classifier...in order to classify the ranges that numeric values fall into

1:50 hiredman: I think you might want to use a vector of actual ranges

1:50 [(range 0 1) (range 1 5) (range 5 10)]

1:50 ,[(range 0 1) (range 1 5) (range 5 10)]

1:50 clojurebot: [(0) (1 2 3 4) (5 6 7 8 9)]

1:53 hiredman: ,(first (filter #(contains? (first %) 6) (map vector [(range 0 1) (range 1 5) (range 5 10)] (range 3))))

1:53 clojurebot: nil

1:53 hiredman: bah

1:53 (fn [item] (reduce #(+ %1 (binary (> item %2))) (concat [0] range))))

1:54 hiredman: ,(filter #(contains? (first %) 6) (map vector [(range 0 1) (range 1 5) (range 5 10)] (range 3)))

1:54 clojurebot: ()

1:54 bradford: this binary function just returns 1 for trie and 0 for false

1:55 hiredman: ,(second (first (filter #((set (first %)) 6) (map vector [(range 0 1) (range 1 5) (range 5 10)] (range 3))))))

1:55 clojurebot: 2

1:58 bradford: ,(((fn [range] (fn [item] (reduce #(+ %1 (binary (> item %2))) (concat [0 0] range)))) (range 1 10)) 5)

1:58 clojurebot: java.lang.Exception: Unable to resolve symbol: binary in this context

1:59 hiredman: ,(contains? (range 10) 5)

1:59 clojurebot: false

1:59 hiredman: :(

1:59 ,(doc contains?)

1:59 clojurebot: "([coll key]); Returns true if key is present in the given collection, otherwise returns false. Note that for numerically indexed collections like vectors and Java arrays, this tests if the numeric key is within the range of indexes. 'contains?' operates constant or logarithmic time; it will not perform a linear search for a value. See also 'some'."

2:00 hiredman: ,(some (partial = 6) (range 10))

2:00 clojurebot: true

2:00 hiredman: ,(some (partial = 11) (range 10))

2:00 clojurebot: nil

2:01 bradford: , (((fn [range] (fn [item] (reduce #(+ %1 (if (> item %2) 1 0)) (concat [0 0] range)))) (range 1 10)) 5)

2:01 clojurebot: 5

2:03 hiredman: if you say so

2:04 its kinda strange though

2:04 hiredman: I don't see the advantage of that

2:04 hiredman: what is it supposed to do?

2:04 that function

2:05 ,((set (range 10)) 5)

2:05 clojurebot: 5

2:05 bradford: it will find what interval of the range that a value lies in

2:05 hiredman: if you say so

2:24 eliantor: hi i read that for each lambda a new IFn interface is created and stored in the PermGen memory, and that this memory area is never garbage collected?

2:25 is it true, and is it an issue?

2:25 hiredman: no, no, and no

2:27 eliantor: :)

2:27 slashus2: eliantor: Where did you read that?

2:29 eliantor: hiredman: :)

2:29 hiredman: (on that old blog post)

2:30 ~latest

2:30 clojurebot: latest is 1375

2:30 hiredman: (- 1375 1232)

2:30 clojurebot: *suffusion of yellow*

2:30 hiredman: clojurebot: thanks!

2:30 clojurebot: Excuse me?

2:30 hiredman: ,(- 1375 1232)

2:30 clojurebot: 143

2:31 hiredman: r1232

2:35 eliantor: hiredman: so if i plan to use a huge set of lambdas i better wrap them in eval??

2:35 hiredman: no

2:35 like, uh, just no

2:35 *sigh*

2:35 slashus2: hehe

2:36 eliantor: hiredman: maybe my question are really stupid, but i really don't know much of this stuff...

2:36 :)

2:37 thanks

2:37 hiredman: everything essential moves through eval any way, it's how a lisp works

2:38 eliantor: oh ok

5:22 a small question

5:22 for this statement => (map #(+ % 3) [2 4 7]) ; -> (5 7 10)

5:22 what does % signify

5:23 hoeck: ,`#(+ % 3)

5:23 where's clojurebot?

5:24 slashus2: ,(+ 1 2)

5:24 :-(

5:25 hoeck: adityo: anyway, #(+ % 3) is shorthand for (fn [x] (+ x 3))

5:29 adityo: ohh cool, that makes much more sense..im coming from CL :)

5:32 hoeck: adityo: yeah, its just a simple reader macro, I have seen this in scheme too, using <n> instead of %n

5:47 Gertm: can anyone help me set up my slime? I get an error saying: Exception in thread "main" java.lang.NoClassDefFoundError: clojure/main when I start it. Something is probably not configured well, but I can't figure out what

5:50 hoeck: Gertm: have you set up the classpaths, clojure.jar etc using M-x customize-group ?

5:51 Gertm: I have this one: (setq swank-clojure-jar-path "/usr/share/clojure")

5:51 I'll check the rest now

5:52 hoeck: Gertm: swank-clojure-jar-path should point to the actual clojure.jar

5:54 not the path, that name is a little confusing

5:54 Gertm: hoeck: ok thanks, that got it to work

5:57 hoeck: fine, glad I could help

8:39 gnuvince: Good morning

8:46 rhickey: hi

8:46 gnuvince: The server rebooted properly

8:47 I guess I can put off opening cv.tex :_

8:47 Chouser_: gnuvince: got backups in the budget now?

8:47 gnuvince: Chouser_: dunno, but I'm definitely bringing it up again

8:47 (and documenting that I brought the issue up again)

10:07 Chouser_: I've been fighting a race condition trying to shutdown the agent pools cleanly.

10:08 One complication is that I can't catch (for debugging, reporting, instrumenting, etc.) RejectedExecuction exceptions, because they bubble out of Action.execute()

10:08 rhickey: Chouser_: let me look

10:08 Chouser_: I temporarily hacked in a printf so I could see the agent causing the problem.

10:09 but that doesn't seem like the Right solution.

10:11 lisppaste8: Chouser pasted "extra info on RejectedExecution" at http://paste.lisp.org/+1QDM

10:12 Chouser_: to be clear, the race isn't in clojure itself, it's in my code trying to halt self-sending agents before shutting down the pools.

10:14 rhickey: Chouser_: do you just want them stuck on the agent like all other exceptions?

10:15 Chouser_: once there's a error callback (or whatever) that might be perfect.

10:16 Right now I'm not using the agent-errors because I have to poll it, so instead I wrap try/catch around the action like this: (send-off my-agent rpt-err real-action arg1)

10:16 rhickey: Chouser_: patch welcome for on-error callback fn

10:17 Chouser_: ...where rpt-err call the real-action in a try. Also a useful place to log state transitions, etc.

10:18 rhickey: wow - now reduce of filter of map over vector with chunked seqs faster than the equivalent loop!

10:18 Chouser_: whoa

10:18 faster than a non-chunked loop?

10:19 lisppaste8: rhickey pasted "faster chunked seqs" at http://paste.lisp.org/+1QDO

10:20 rhickey: yup

10:20 Chouser_: very cool

10:20 rhickey: 67ms on my machine

10:21 and the loop isn't even looking at a vector

10:21 Chouser_: I just noticed that

10:22 Chousuke: wonder how it'd work with a chunked Range

10:22 duck1123: is anyone here making use of clj-record?

10:25 I'm getting a class not found exception, which is odd, because I would exect the normal can't find the ns exception from clojure

10:26 lisppaste8: rhickey annotated #80844 "apples to apples" at http://paste.lisp.org/+1QDO#1

10:26 rhickey: wow, it's an even bigger difference over loop w/vector - 3x!

10:27 Chouser_: you're using the vector's natural chunk size? 32 per chunk?

10:27 rhickey: yup, uses the nodes directly (actually a small shim is allocated right now)

10:27 asbjxrn: I'm a bit out of the loop. Does this mean that you're now starting to focus more on performance?

10:28 rhickey: asbjxrn: no, this is about maximizing the value of Clojure's key abstractions

10:29 performance optimization overall is a black hole

10:29 asbjxrn: Cool.

10:30 rhickey: all the *'s will be going away, this will be built into map/filter/reduce etc

10:30 duck1123: thinking about performance never hurts, but performance optimization is rarely something you should loose sleep over.

10:31 Chouser_: or lose it, either.

10:31 stuartsierra: What is a "chunked" sequence and why is it faster?

10:32 asbjxrn: I don't have any issues with performance fow what I'm doing so far, so I'm not losing any sleep.

10:32 Chouser_: asbjxrn: exactly. Clojure's already too fast.

10:33 rhickey: stuartsierra: chunked sequences extend the seq abstraction to seq nodes that have small chunks inside them, so all laziness and allocation overhead is amortized - they are also perfectly normal seqs

10:33 gnuvince: Chouser_: "too fast"?

10:34 Chouser_: gnuvince: breaking speed limits. making other languages look bad. allowing sloppy coding to still perform well. ...there are plenty of reasons too fast can be bad.

10:34 rhickey: stuartsierra: map/reduce et al will recognize they've been handed a chunked seq and will process the entire chunk in one go

10:35 stuartsierra: Sort of like a buffer for a stream?

10:35 rhickey: so laziness and boxing overhead become 1/32 of what it was

10:35 gnuvince: Chouser_: I can hardly think of any.

10:35 Chouser_: gnuvince: I'm being silly. Please ignore me.

10:35 rhickey: yet it is still a lazy model, i.e. not everything is pulled into memory and no realization of intermediate results

10:35 gnuvince: If you can have nice, modular, maintanble code that also happens to be really quick, I say go for it

10:35 Ah

10:35 stuartsierra: cool

10:36 gnuvince: My sarcasm detector is deffective it seems

10:36 Chouser_: gnuvince: or my humor isn't funny. Either way. :-)

10:36 rhickey: this has replaced the streams idea, much nicer, safer, integrated, functional

10:38 rhickey: Chouser_: streams had two parts, an unsafe bit you weren't supposed to share, and an ability to create a locked-down stream. Resource management is orthogonal - the scope system could work with either

10:39 gnuvince: rhickey: are these new developments coded in pure Clojure or is there Java code involved?

10:39 danlarkin: I like

10:40 rhickey: the thing about streams is they put mutation (when needed) in the source, drastically reducing the sharability and functional nature. Chunked seqs put it in the return value creation, following the model of all of the persistent vecs/hashmaps

10:42 duck1123: hmm... would chunked seqs be what you would want for lazy sql?

10:45 rhickey: gnuvince: there's still some Java until I get around to implementing 'instance'

10:45 gnuvince: rhickey: ok

10:47 rhickey: stuartsierra: yes, like a buffer in some ways

10:49 gnuvince: this kind of infrastructure stuff has to be as fast as possible, currently only possible in Java - 'instance' will not have the overheads of proxy/gen-class

10:51 gnuvince: rhickey: I was asking because I'm curious if/how custom data structures could be created in Clojure

10:52 rhickey: gnuvince: they can now, in several ways

10:53 moving forward the recipe will be definterface (static) + instance (dynamic)

10:54 you can't get the highest perf without some static aspect

10:55 instance will be what many people think proxy is

10:55 Chouser_: I've seen some people doing some fun dynamic stuff with proxy -- will it stick around for such purposes?

10:56 gnuvince: rhickey: okay

10:57 Looking forward to it

11:09 rhickey: Chouser_: actually leveraging swapping the fns out?

11:10 Chouser_: rhickey: Yes, I think so. Or at least building up the fn map using domain-specific means.

11:10 rhickey: there's no harm to it remaining other than confusion about which to use when

11:11 Chouser_: I guess I don't remember enough to know if they really need to muck about with it at runtime, but that's my vague impression.

11:11 rhickey: instance will be much faster, and not subject to the travails of proxy for interacting with superclass stuff

11:11 Chouser_: yeah, proxy-super's not threadsafe, is it?

11:12 rhickey: o

11:12 no

11:13 so, this chunked seq experiment seems a smashing success - I'm going to try to wrap it up a bit and get in in trunk so maybe we can get more hands involved in doing the complete job

11:14 a persistent T-tree replacement for the red-black trees would be welcome

11:14 then the sorted colls will have interior nodes amenable to chunking

11:15 Chouser_: Will 2-3 finger trees' chunks be too small to get much benefit?

11:15 gnuvince: I missed most of the conversation about chunked seqs, so I'll ask: how do they work?

11:15 rhickey: Chouser_: probably, they are more like linked lists, which also won't chunk

11:17 gnuvince: there is an IChunkedSeq derivee of ISeq which adds chunked first/rest/next. Data structures that can support them will return seqs that implement IChunkedSeq. In my prototype I've written a chunked seq for PersistentVector

11:18 rhickey: map/reduce/filter will check to see if what they get from seq is chunked, if so they do a little loop over the chunk, and use chunked-cons to return another chunked seq themselves

11:18 gnuvince: chunked first/rest/next meaning what? first fetches a few elements and caches them?

11:19 rhickey: chunkedFirst returns an indexed chunk, supporting nth and count, chunkedRest returns a logical collection of items after the first chunk, chunked next does the same with the seq-or-nil protocol

11:20 gnuvince: but no, it's not a ache and in that sense differs from buffering - the underlying data structure needs to be able to provide indexed access to some small (e.g. 32) number of elements, as can the persistent vectors and hashmaps

11:21 gnuvince: ok

11:22 rhickey: Chouser_: thanks for the patch - do you think it should still put errors on the agent when a callback is supplied?

11:23 Chousuke: sounds like they should make "IO" seqs a fair bit faster too.

11:25 Chouser_: rhickey: good question

11:25 if the callback doesn't know how to handle a particular error, it doesn't have any way to put that error on the agent's list

11:26 rhickey: I always saw the callbacks as an alternative to the per-agent list

11:26 Chouser_: but it seems like most error-fn's won't care about the particular error and will end up calling clear-agent-errors.

11:27 rhickey: if I had a callback I wouldn't want agent-errors etc, maybe just some way for the agent to pause until restarted

11:28 but there is also the possibility of a protocol in the callback, via a return value

11:29 there are two independent problems currently handled together - workflow and reporting

11:29 Chouser_: a protocol to allow clearing, queueing on the agent's erroer seq, pausing until ...something??

11:30 rhickey: a lot of callbacks might only be about reporting

11:30 centralized logging etc

11:31 others might route a problem to a controller that would know to restart/reset the agent, if that was possible

11:32 but that might not be a synchronous decision

11:32 i.e. put error on queue and return

11:32 Chouser_: do you want to allow for multiple error-callbacks, so different concerns can be handled separately?

11:33 rhickey: Chouser_: I don't think so - people can add whatever multiplexing they like, but unlike watching, error handling strategies don't compose ad hoc

11:35 Chouser_: if the error-callback got the action args, that should be all it could need to decide how to restart later.

11:35 rhickey: what gets routed to the handler - just the exception?

11:35 heh

11:36 I think it's important not to get locked into the agent-errors/clear-errors model with this callback system

11:36 Chouser_: currently the callback gets the agent and the exception list (which it doesn't really need since it could call agent-errors)

11:36 hm.

11:37 rhickey: so yes, communicating as much as possible to the handler is good, using the return value of the handler to either proceed (with a possible new state), or go into an error mode until further notice, and then a way to reset when in error mode in order to proceed

11:37 later

11:38 Chousuke: hmm

11:39 Chouser: any reason "error mode" should be other than non-nil error list as it is now?

11:39 rhickey: from a workflow perspective either the agent is running or stopped

11:40 Chouser: if the callback just returns [:error e] instead of [:new-state :foo]

11:40 rhickey: it's an open question as to whether there would ever be a list, once you can get notified of the first error synchronously

11:40 the only reason they accumulated was due to the async

11:41 Chouser: the code I'm working on now uses the state of the agent for workflow

11:47 Chousuke: how about making it possible to add a redirect to another agent while the original agent is in "error mode"? that way, it could log users of the agent for example. and if the redirected-to agent errors, the same mechanism could be used to redirect further, or whatever you want.

11:48 Chouser: the callback patch I have in is synchronous, which I think is fine -- until the error is handled the original agent can't do anything else anyway.

11:49 if the callback throws something, I fall back on the default behavior -- queing it in the agent's errors.

11:49 Chousuke: if the error is something that can't be handled synchronously, a redirect mechanism could be useful.

11:49 Chouser: sure, but the callback can do a send-off, or add something to a BlockingQueue or whatever.

11:50 well, I think I've done as much as I can at the moment. If the requirements get firmed up a bit, perhaps I can make another go-around on the patch later.

12:19 What's the best way to make sure a function is not called from within an agent action (because it might deadlock)?

12:19 (assert (not *agent*)) ?

12:25 rhickey: Chouser: sound scary, how did this happen?

12:26 (assert (not *agent*)) will work

12:26 Chouser: I've got several self-sending agents, most blocked on reading or accepting sockets.

12:27 But now I want to shut it down. So I need to close the sockets, clean up the agent queues, and shutdown the pools.

12:27 Over the last several days I found several stragies that don't work. I now have one that does.

12:28 rhickey: does the socket api support the interruption protocol?

12:28 Chouser: at shutdown, I count up my agents, put that in a CountDownLatch, (send agt quit) for each agent, and then close the sockets.

12:29 oh, then await on my countdownlatch.

12:30 the quit fns check the agent state. If it's not yet :closed, they send-self. If it's :closed, they countDown the latch.

12:31 so the socket agents do what they do until they notice their socket is closed, at which point the go to :closed and stop self-sending.

12:32 this means every agent has a change to clean itself up properly then cleanly signal its done. The shutdown code is self-contained and can be sure all the agents are done before it unblocks and returns.

12:32 I don't really need to use interruption as I can close the sockets.

12:33 so this is all working fine -- no deadlocks, no RejectedExecution

12:33 ...unless you're in one of the agents when you try to shutdown. In that case you've got a deadlock on the agent waiting for a queued action on itself to complete.

12:34 I can document this and I don't expect it to be a big deal for users, but they will forget and I'd like an error instead of deadlock.

12:36 stuhood: the self sending agents seem really dangerous in general

12:37 if you were using an epoll type architecture, a parent thread could (send) when there was something for an agent (per connection) to do

12:39 Chouser: the only problems I've had with the self-sending agents are when trying to synrchonize them all for shutdown. And even then, it's only been a problem of spurious RejectedExecution exceptions. Do you see other problems?

12:40 stuhood: no, just the fact that you needed to use a countDownLatch

12:40 Chouser: ah. I always seem to have to do something like that when trying to collect all my agents.

12:41 stuhood: if the agents never blocked, then they would always be short lived

12:41 Chouser: rhickey has frowned at that in the past, but I haven't understood any alternative that I like at all.

12:44 stuhood: agents really seem like they need to be treated like a threadpool

12:45 hmm, but i guess your use case would apply well to a threadpool

12:45 nevermind me.

12:45 Chouser: :-)

12:46 I dabbled with managing Threads directly, and just having them loop (still not the epoll you mentioned), but that didn't really solve any of my problems and meant I couldn't redefine functions livetime (usefully)

12:46 stuhood: but if the agents weren't self sending, it would be easy to shut them down

12:47 emacsen: Is there any way to ask Clojure for all the methods that an object can be called with?

12:48 technomancy: emacsen: try clojure.contrib.repl-utils/show

12:49 emacsen: technomancy: Okay, question 2: Is there any nice way to search the docs by full text, rather than by function name? :)

12:49 technomancy: (doc find-doc)

12:49 emacsen: Well isn't that clever :)

12:49 technomancy: oh no, where's clojurebot?

12:49 * technomancy sends out an APB

12:50 emacsen: does the lib need to be included?

12:50 duck1123: it's in core

12:50 emacsen: no... I mean can I search contribs, for example

12:50 things I've yet to require

12:51 Chousuke: no

12:51 emacsen: ala perldoc, pydoc, etc

12:51 technomancy: emacsen: for that you need grep. =)

12:51 duck1123: i thought it did the whole classpath

12:51 Chousuke: hmm

12:52 technomancy: I don't think so, that would cause everything on the classpath to get required, which would balloon memory usage.

12:52 Chousuke: but there's online documentation for contrib, and it has a search box :)

12:56 replaca: ,(doc finddoc)

12:56 duncanm: hola

12:57 replaca: ,(doc find-doc)

12:57 duncanm: i really miss having @"foo" literal strings for paths (ala C#) on Windows

12:57 it's much easier to write @"C:\foo\bar" than doing search-n-replace to get "c:\\foo\\bar" or "c:/foo/bar"

12:58 can i write a reader macro to get @-strings?

12:58 Chousuke: no user-defined reader macros

12:58 technomancy: duncanm: can't you use regular slashes and have the JVM perform the conversion for you?

12:58 Chousuke: but you can have a (path "foo") macro.

12:59 replaca: Chousuke: you literally took the words out of my mouth :-)

12:59 emacsen: replaca: I know it's a dumb pet peeve, but it's a pet peeve nonetheless. He LITTERALLY took the words out of your mouth?

12:59 technomancy: Chousuke: that macro would already have the string parsed by the reader, so backslashes would get lost

12:59 hehe

13:00 Chousuke: technomancy: hmm, right.

13:00 replaca: emacsen: yeah, you should have seen it. Sort of painful, actually

13:00 emacsen: :)

13:00 replaca: emacsen: but to be precise, I meant that was literally the exact sentence I was typing, not just close

13:01 technomancy: replaca: english needs more parens

13:01 duck1123: Lojban FTW

13:01 Chousuke: no it doesn't. I tried lisp-syntax english once. painful.

13:01 emacsen: hehe, lojban! :)

13:01 replaca: technomancy: *everything* needs more parens. but we're helping it along

13:02 technomancy: Chousuke: you just need to get better at diagramming sentences

13:02 Chousuke: (reduce eat (get-cakes))

13:03 technomancy: nom nom nom

13:03 stuhood: (eat!)

13:03 replaca: emacsen: I was a loglan afficianado as a kid, but I found spanish was better for meeting girls

13:04 duck1123: I've been trying to get my wife to learn it with me, but I'll have to settle for teaching my kids

13:08 duncanm: if i write a path function for this, do i have to convert all control characters back to their non-control equivalent?

13:09 Chousuke: lojban has s/z+consonant syllables ;(

13:09 evil

13:14 duck1123: Is there a way to specify in my namespace that all of my tests are located in a different namespace?

13:18 basically I want to be able to say that net.mycyclopedia.model.entry's tests are in net.mycyclopedia.model.entry-test so I can use the former as the call to run-test

13:19 Chousuke: clojure's tests are in clojure-contrib. see how it's done there :)

13:19 technomancy: duck1123: were you the one asking me about an emacs function to switch between test and impl?

13:20 duck1123: technomancy: yes i was. I see that you made something, but haven't gotten around to trying it yet

13:21 I'm just now getting started with TDD now that my pom.xml is able to compile and test for me

13:23 technomancy: duck1123: I find it's easier to do TDD if you don't have to switch away from the test buffer to see the results, which is what clojure-test-mode does

13:30 duck1123: technomancy: I think I'm going to have to hack it a little bit because I keep my tests in a difeerent location (src/test/clojure/) than you do.

13:31 other than that, it looks very useful.

13:31 Raynes: ?

13:32 Lau_of_DK: Good evening gents

13:33 duck1123: it's so hard going back to using elisp, I keep wanting to use [] in places. (ie. let)

13:34 Raynes: Good evening Lau_of_DK.

13:34 technomancy: duck1123: no kidding

13:35 every time I type setq it makes me cringe

13:46 durka42: ,(seq? '[a])

13:47 no clojurebot...

13:47 * Chouser guesses false

13:47 durka42: anyway

13:47 Clojure=> (seq? '[a])

13:47 false

13:47 why?

13:47 Chouser: a vector is not a sequence

13:47 Raynes: Boom.

13:47 Chouser: most collections are not themselves seqs

13:47 kotarak: (seq? (seq [a])) => true

13:47 Chouser: ...the exception being PersistentList

13:48 Raynes: That list is so darn persistent!

13:48 cmvkk: that reminds me, why don't sequences implement Associative?

13:48 * durka42 apparently does not fully understand the seq abstraction

13:48 durka42: coll? was the function i wanted

13:48 cmvkk: i want to be able to do (assoc '(1 2 3) 1 4)

13:48 durka42: (assoc [1 2 3] 1 4)

13:49 cmvkk: well yes, it works with a vector

13:49 i'm just wondering if there's a specific reason that it doesn't work with lists

13:54 Chousuke: cmvkk: lists aren't associative.

13:55 cmvkk: is there any particular reason for that? just because it isn't very efficient, or something?

13:55 i'm mostly just curious

13:55 Chouser: lists don't associate anything. lookups via nth are O(n)

13:56 Chousuke: cmvkk: lists just contain items. they're not associated with keys (like maps) or their index (like vectors)

13:57 pretending that list items have an index would make them vectors, conceptually, and it'd be rather stupid :/

13:57 cmvkk: of course you're right. i wonder where I got the idea that you could index lists at all.

13:57 i was thinking that (get '(1 2 3) 1) would work too but it doesn't

13:57 maybe you could do that in common lisp or something? hmm.

13:59 it seems like it would be a useful abstraction anyway though, and I guess you could argue that elements in a seq are conceptually associated with their positions ("first", "second" etc)

13:59 but i get the idea

13:59 duck1123: technomancy: ok, I got clojure-test-mode working with my directory structure. Very nice.

13:59 technomancy: cool

14:00 dnolen: cmvkk: (nth '(1 2 3 4 5) 3) works

14:00 cmvkk: yeah.

14:03 Chousuke: cmvkk: I guess the difference here is that vector are associative becuase they're functions of indexes (meaning the operation of indexing is natural for a vector) while a you can look up an item in a list by its position, but the data structure itself does not make such an association.

14:03 cmvkk: that is true.

14:03 Chousuke: I'm sure there is some fancy word for what I'm trying to say :P

14:04 cmvkk: no, I get it, and that makes sense.

14:04 duck1123: so now that more people are up, has anyone had any luck getting clj-record to work in their project?

14:04 Chousuke: maybe intensional and extensional

14:04 cmvkk: anyway i was only thinking about it because i wanted to do something like (assoc "hello" 2 \q) but I think there's a java function for that somewhere

14:06 Chousuke: well, strings are indexed so that conceptually that should actually work, but I think it doesn't because the String class doesn't implement the needed Clojure interface :/

14:07 cmvkk: yep. i think that has to do with the fact that the java String class is final so nobody could make a clojure wrapper class for it

14:07 oh well.

14:14 hiredman: cmvkk: CharSequence

14:20 durka42: hiredman: what happened to clojurebot today?

14:21 mozinator: anyone here done some stuff with markov chains and clojure ?

14:21 durka42: hmm, clojure.core/replace kind of screw up maps

14:21 screws*

14:21 hiredman: oh

14:22 Chouser: ,(map identity {:a 1, :b 2, :c 3})

14:22 technomancy: crap! groovy jumped ahead of Clojure on github.

14:22 come on guys, let's get some more projects up there

14:23 we can't get beat by Groovy, that's embarrassing.

14:23 Chousuke: hm

14:23 gnuvince: rhickey featured on infoq.com

14:23 hiredman: gnuvince!

14:23 gnuvince: hiredman: hello

14:23 technomancy: gnuvince: is that good? worth trying to track down a non-flash version of the interview?

14:24 Chousuke: ,(replace '[foo bar zonk] [2 0 1])

14:24 clojurebot: [zonk foo bar]

14:24 duck1123: technomancy: one issue I've noticed with clojure-test-mode. I thought libraries weren't supposed to bind C-c {single letter}

14:24 hiredman: gnuvince: been waiting for that stuff to show up

14:24 technomancy: duck1123: ah yeah... I chose that binding when it was still part of my personal dotfiles and forgot to change it when I moved it. =)

14:25 gnuvince: technomancy: watching now, but I guess you could ask Rich if you should watch it :)

14:26 gnuvince: So far, the interviewer seems... lost

14:26 technomancy: I really don't need to see any visual with this kind of interview. annoying that they can't just post an mp3

14:40 I can name a function like this: (defn | [a b c] (b c a))

14:41 and it works...but i can not use the "|" symbol in hte repl

14:41 hiredman: bradford: works for me

14:41 Chouser: me too

14:42 hiredman: yep

14:42 Chouser: (| 5 + 6)

14:42 hiredman: unless b is a an IFn you are going to get an exception anyway

14:42 kotarak: | is not a valid symbol, IIRC.

14:43 bradford: i always get "input not complete" in my emacs minibuffer...

14:43 | seems to be valid

14:43 technomancy: (symbol? (read-string "|")) ;; => true

14:44 hiredman: bradford: well, that's emacs for you

14:44 Chouser: all chars besides *, +, !, -, _, and ? and alphanumerics are reserved.

14:44 hiredman: ~emacs

14:44 Chouser: kotarak: so you're right.

14:44 technomancy: bradford: you sure you're not trying to eval it as elisp? =)

14:44 bradford: yes, im sure :-)

14:44 technomancy: Chouser: how come (symbol? (read-string "|")) works then?

14:45 is it just one of those "it works by accident, but don't rely on it" things?

14:45 kotarak: technomancy: because Clojure doesn't enforce the rules.

14:45 bradford: anyway, i want this to express conditional probabilities in their mathematical form

14:45 technomancy: gotcha

14:45 bradford: P (a | b)

14:45 technomancy: Chouser: those are my patches on that issue. =)

14:45 I thought the reader defined what was valid though.

14:46 Chouser: technomancy: well there you go then.

14:46 technomancy: must be wrong

14:46 Chouser: :-)

14:46 I guess maybe that's a separate issue

14:46 that bug says "readable" which may not be identical to "supported"

14:46 technomancy: right

14:47 kotarak: rhickey once suggested || to delimit (more or less?) arbitrary symbols.

14:47 Not sure what was the outcome of that.

14:48 technomancy: that's too bad about reserved symbols though; I'd love to use ? to name a var

14:48 (for predicate functions that are destructive, I guess)

14:49 kotarak: Thread.interrupted() comes to mind. :)

14:49 Chouser: you're probably free to use non-ascii stuff. I don't think that'll ever conflict with a Clojure builtin

14:49 hiredman:

14:49 � would be for things that look destructive, but aren't

14:50 technomancy: nice

14:50 hiredman: (i.e. assoc, update-in, etc)

14:51 technomancy: (def ? fn)

14:51 hiredman: doesn't work

14:51 fn is a special form

14:51 technomancy: oh of course

14:51 hiredman: so you have to defmacro ?

15:08 ddonnell: what's the conventional way to distribute and consume libraries of clojure code?

15:09 technomancy: ddonnell: it's pretty ad-hoc right now.

15:09 ddonnell: :(

15:10 technomancy: ddonnell: the easiest way to support automated dependencies is probably to add a pom.xml that inherits from clojure-pom

15:10 but it's a bit of a hassle, even clojure itself hasn't made it into any mainstream public repositories yet.

15:10 ddonnell: thanks

15:10 dysinger: It's less hassle than all the other options

15:11 kotarak: ddonnell: there is also some effort going on for Ivy support

15:11 technomancy: dysinger: it'd be less of a hassle if we got clojure-pom into a public repo, I mean. =)

15:11 dysinger: and clojure-lang & clojure-contrib

15:11 technomancy: that too

15:11 dysinger: although I don't understand why the "-lang" part of that got started.

15:12 kotarak: It is not removed by order de Rich.

15:12 s/not/now

15:12 dysinger: All we need to do is create a clojure-specific maven repo

15:12 it would work for the ivy fanboys too.

15:12 kotarak: :)

15:12 technomancy: dysinger: last night I poked around at getting mvn support in corkscrew; it doesn't look like it'd be hard at all.

15:13 dysinger: until you look at all the plugin infrastructure :)

15:13 bradford: technomancy: do it dude :-)

15:13 technomancy: spent some time looking at the test cases for guidance, which turned out to be a mistake; repl-utils/show was more helpful

15:13 dysinger: you mean "it wouldn't be that hard to support maven repos"

15:14 buildr for example, uses maven repos and can figure out trans deps

15:14 but it can't do any plugins.

15:14 technomancy: dysinger: yeah, it's pretty easy to construct a project Model, and it looks like you can hand that off to a ProjectBuilder

15:15 dysinger: since the plugins don't work, you are forced back into writing a higher-order build tool or worse copy/pasting the same build snippets to all your projects.

15:18 duck1123: dysinger: the maven-clojure-plugin works fine for me

15:18 dysinger: y it's just not needed.

15:18 at least IMO

15:19 my clojure-pom is like 2 dozen lines of XML and does AOT, tests, packaging.

15:19 duck1123: It needs some more work, and the defaults need to be fixed, but I think there could be future in it

15:19 I just looked at it a month ago and said "I can do all that with maven and a couple antrun tasks"

15:20 duck1123: I'm a complete maven noob, but I've been learning a lot lately

15:21 dysinger: It's complex - that's why people hate it.

15:21 duck1123: Clojure needs some good maven archetypes too to make starting a new clojure library easy

15:21 dysinger: but it works.

15:21 and if you model your POMs correctly, often times projects have only a dozen lines of xml

15:22 duck1123 yeah - I have a README that says "if you are brown belt in maven fu, you can create a maven archetype from an existing clojure project"

15:22 duck1123: right now, the plugin isn't modeled correctly, but as soon as I figure out how to change those defaults, I'll push out my fork

15:23 saw that, not there yet. tried and failed

15:23 dysinger: I did a couple of them.

15:23 but it's almost as easy to just create a dir and put the base xml in the pom.xml than to use archetypes.

15:23 since clojure is low on ceremony

15:23 duck1123: what ever happened to that clojure project hosting site idea that someone had

15:24 dysinger: It's called "github"

15:24 :)

15:24 duck1123: we came up with a whole bunch of names, and then i never heard of it

15:24 hiredman: ~Xiphojura

15:24 duck1123: there's no way to make github a maven repository though, or is there

15:24 clojurebot: 2009:Jan:29:16:03:17 <hiredman> I call Xiphojura

15:35 Chouser: uses of proxy is one of the very few places in clojure code where (foo ...) is not a call to a foo. Is there any sane way to fix that?

15:35 (proxy [] [] (:foo [arg1 arg2] ...)) ?

15:36 (proxy [] [] [foo [arg1 arg2] ...)] ?

15:36 cmvkk: you just want to avoid using that syntax? what about (method foo ...) even though it's more verbose

15:36 kotarak: (proxy [] [] {foo (fn [arg1 arg2] ...})

15:36 Chouser: "method" isn't too bad.

15:37 I assume the upcoming 'instance' form will need a similar syntax

15:45 technomancy: duck1123: that's something that corkscrew can do though; unpack git projects just like any other dependency

15:46 since even if folks move towards the mvn repo format there will still be a lot of code that's just a raw repo

15:47 dysinger: are you just using plugins to do AOT and run tests?

15:52 duck1123: technomancy: that's actually what I has hope that'll fill this space. I just wish we could get all of the libraries out there to settle on a format making it easy to "require" a git-hub repository

15:53 I'm using the plugin, dysinger uses the ant tasks

15:57 technomancy: duck1123: as long as using a lib is as simple as putting its src/ on the classpath, corkscrew should be able to handle it.

15:58 duck1123: what if it's actually src/main/clojure/

15:59 technomancy: then it won't work. =)

16:00 duck1123: there's still too much variation in clojure libraries at this point to make something work easily

16:00 I'm sure you kknow this, do gems mandate a directory structure?

16:01 technomancy: duck1123: more or less; you put "require"-able stuff in lib and use bin for executables

17:01 devinus: what java book should i use to familiarize myself with it enough to use clojure ?

17:05 stuartsierra: The Sun Java Tutorials.

17:31 StartsWithK: fyuryu: hi

17:48 kotarak: it works :) hehe

17:48 kotarak: StartsWithK: :)

17:49 Just syncing the repo on my server. :)

17:50 I'll check the repo order.

17:52 StartsWithK: that should seed up the build, maybe 40-60 sec max after that

17:52 kotarak: StartsWithK: Hmmm... I actually have a custom resolver chain, which checks local, then shared and only then the server...

17:52 Ivy should cached clojure...

17:53 I think, the fork for the compilation is eating the time...

17:53 StartsWithK: that happens im my builds too, fork takes to long

17:56 also, cloak can now execute simple ant task and generate repl script for standard clojure repl, rlwrap and jline based ones

18:21 devinus: god clojure looks so cool but i'm not sure i'm smart enough to learn it

18:21 going through the code to clojure-contrib and i just can't grok even the comments! :-/

18:22 danlarkin: devinus: start with something simpler, one of rich's videos perhaps

18:22 devinus: "Dissociates an entry from a nested associative structure returning a new nested structure. keys is a sequence of keys. Any empty maps that result will not be present in the new structure."

18:22 hiredman: devinus: core.clj is much nicer

18:22 devinus: Woah.

18:23 i don't even know what a nested associative structure means!

18:24 hiredman: devinus: a map containing another map

18:24 devinus: i see

18:24 hiredman: ,{:a {:b 1} :c 3}

18:24 clojurebot: {:a {:b 1}, :c 3}

18:25 devinus: but then i read programming.reddit.com and i hear of things like stm, agents, multimethods...concurrency primitive! an explosion of wtf in my brain

18:33 danlarkin: just take it slow

18:36 kotarak: ,:@

18:36 clojurebot: Invalid token: :

18:37 kotarak: ,(keyword "@")

18:37 clojurebot: :@

18:45 bradford: I'm having trouble doing a reduce on a seq of vectors (pairs in this case)

18:47 hiredman: ,(reduce concat '([1 2] [3 4] [5 6]))

18:47 clojurebot: (1 2 3 4 5 6)

18:47 bradford: I tried to pass reduce a function like this: (defn reduce-pair [[x1 y1][x2 y2]] [(+ x1 x2) (+ y1 y2)])

18:48 hiredman: ,(reduce (fn [[x1 y1] [x2 y2]] [(+ x1 x2) (+ y1 y2)]) '([1 2] [3 4] [5 6]))

18:48 clojurebot: [9 12]

18:48 hiredman: ?

18:49 bradford: you're right, my code was correct...the arg I passed to it in the repl was wrong. :-(

20:25 danlarkin: technomancy: merge Perry Trolard's clojure-http-client branch :)

20:27 technomancy: oh; that looks nice

20:30 didn't realize you could have key-only cookies

20:31 merge'd!

22:49 * technomancy is hacking on a maven wrapper to make it a little nicer to work with clojure

22:50 technomancy: could someone take a look at a some mvn internals to give me some hints? I'm running into some weird roadblocks.

22:50 duck1123: I'll take a look

22:50 I'm not the greatest at maven yet

22:51 technomancy: duck1123: I'm trying to create an UnpackDependenciesMojo. that class has a "project" member, but it's protected, and I can't figure out how to set it.

22:52 duck1123: hmm... then beyond me at this point

22:52 sorry

22:54 technomancy: I even tried to load it up in Eclipse, (shock, horror) but it couldn't make head or tail of it.

22:56 I just don't know enough Java idioms to know where to look for that kind of thing; it's probably some obscure factory method somewhere, but I'm blanking.

22:57 danlarkin: check the XML config file which points back to code!

22:58 technomancy: heh

22:58 danlarkin: what's your take on the dependencies dance?

22:59 danlarkin: that it takes most languages a few years to sort it out

23:00 dnolen_: well since Clojure code tends to be really short, maybe we can get there sooner :)

23:01 technomancy: dnolen_: not to mention being built on an existing platform

23:01 duck1123: we have tons of java solutions for everything

23:01 danlarkin: it's a blessing and a curse that we have maven/ivy/whatever to consider too

23:02 technomancy: danlarkin: sure, but having one system for pure-clojure deps and another for other JVM-language deps seems like The Wrong Thing.

23:03 dnolen_: I'm not sold basing a anything on maven/ivy/whatever. I think we need something lispy, and people can plugin support for the Java stuff. The only library that I use that really requires me to mess around with jars is Compojure, and stuff like OpenGL and serial.

23:03 danlarkin: oh yes, what do other jvm languages do?

23:03 technomancy: dnolen_: I think the compromise is to wrap existing infrastructure in a lispy interface

23:03 dnolen_: technomancy: totally.

23:03 duck1123: I guess a lot of scala users use maven

23:04 dnolen_: but not support maven/ivy/ant directly.

23:04 just call into them.

23:04 technomancy: groovy has a wrapper around maven

23:04 duck1123: I was asking where to find info about maven, and was told to ask the scala room

23:04 Raynes: Someone should write a build system in Clojure.

23:04 Haskell has Cabal, OCaml has... Well something... Clojure should have something. :)

23:04 duck1123: corkscrew? ties? lancet?

23:05 technomancy: stu hasn't been to keen on making lancet anything other than an example project

23:05 maybe now that the book is out he'll have more time to hack on it?

23:05 dnolen_: cloak is interesting as well. I gave up reinventing to wheel for that part. Something like rake for moving files around and running tasks is nice.

23:05 danlarkin: I have yet to look at corkscrew, I do like the idea though

23:05 Raynes: duck1123: Well, I mean a build system that Clojure people actually /use/. And insofar I've seen no one using any of them. Guess I haven't looked very hard though.

23:06 dnolen_: I also think libraries are too simple at this point.

23:06 * Raynes has been slow in participating in the Clojure community due to his Haskell adventures lately.

23:06 duck1123: Raynes: that's the problem, all the libraries are scattered right now

23:06 dnolen_: community maintained meta-index is the answer to that.

23:06 danlarkin: cheeseshop equivalent

23:07 * technomancy supposes eventually a repo.clojure.org will be created

23:07 technomancy: duck1123: me too; it's not too tricky

23:07 Raynes: Like Hackage.

23:10 danlarkin: another problem with build systems & package management systems is it seems everyone's got a different setup of clojure on their systems

23:11 technomancy: danlarkin: well the way I've been doing it is that each project has its own copy of clojure checked out in its target/dependency directory

23:11 so that's not really an issue

23:11 duck1123: it gets even worse when you consider testing

23:11 danlarkin: and no standard library location... I *really* like the idea of python's site-packages directory...

23:12 dnolen_: danlarkin: that's the part that should be standardized. I never loved ASDF, but at least everything is one place.

23:12 I'm going with .clojure in home

23:12 duck1123: maven has ~/.m2/repository/ which is similar

23:12 dnolen_: inside there's ext (all OS specific libs and jar goes here)

23:12 technomancy: danlarkin: yeah, everything gets stored in ~/.m2/repository

23:12 it's just per-user rather than system-wide

23:12 danlarkin: and I use ~/clojure/src/ ... and there's the problem :) everyone's got a different setup

23:13 dnolen_: danlarkin: only because a script didn't set it up for you.

23:13 there's no reason to do this stuff by hand.

23:13 danlarkin: I totally agree

23:13 technomancy: danlarkin: if a build system handled it, you wouldn't even need clojure checked out on your box

23:13 duck1123: corkscrew generate {project name}, perhhaps?

23:14 technomancy: duck1123: eventually. but that's the easy part; I want to prove mvn integration is possible first. =)

23:14 danlarkin: technomancy: oh, but I don't really favor that idea personally, it just seems crazy to checkout all these deps each time rather than just define a standard place for them to be

23:15 technomancy: danlarkin: you could just make your classpath include the jars straight out of ~/.m2/repository/$FOO/etc, I just haven't done that yet since it's easier to keep it all unpacked in one place. 23:15 if you don't unpack, you can't add libraries at runtime, which is unfortunate 23:16 danlarkin: I think clojure should come with a clj binary/shell script that puts a default path on the CLASSPATH when it runs the jvm, and all clojure tools can use that path 23:16 technomancy: yeah, the lack of a shell script that comes with clojure is really boggling to me. 23:17 it just enforces the "you should never use clojure without Slime or an IDE" mindset 23:17 dnolen_: technomancy: well a shell script isn't very Windows friendly right? 23:17 not that I care about that... 23:17 danlarkin: clj.bat :) 23:18 duck1123: when will we be able to 'sudo apt-get install clojure clojure-contrib'? 23:18 hiredman: clojure is in freebsd ports and it comes with a launcher script 23:19 danlarkin: it's in gentoo portage too 23:19 technomancy: there's one in contrib, it just belongs in clojure 23:20 danlarkin: it belongs in clojure proper *and* it needs to define a standard library path for each architecture IMO 23:21 technomancy: hrm; well if I can't make any headway on calling Maven from the Java API I will have to fall back to plan B; just writing a pom.xml and shelling out to mvn. =\ 23:21 duck1123: danlarkin: I think I would be afraid to put too many libraries in a system-wide classpath 23:22 dnolen_: duck1123: why is that? they don't get loaded until you load them. aren't OS libs set up this way? 23:22 danlarkin: duck1123: well it'd just be a clojure classpath, and I don't think it affects the JVM at all until you load them 23:22 technomancy: there's a limit on the size of environment vars 23:22 hiredman: the java guys seem to recommend against a system wide classpath, I'm not sure why 23:22 technomancy: you're going to have conflicting versions of the same jars on it too 23:23 danlarkin: this is one of the things I really disdain about the jvm :-/ 23:23 dnolen_: well listing a bunch of .jars can be avoided with 23:23 -Djava.ext.dirs 23:25 hiredman: dnolen_: but that does not always work 23:25 dnolen_: when doesn't it work? 23:25 hiredman: I don't recall :P 23:25 dnolen_: :) 23:25 technomancy: any good examples of clojure.xml usage? 23:25 hiredman: and I never figured out why 23:26 technomancy: the test suite is empty. =\ 23:26 dnolen_: well it's worked for me, I'm moving all my jars there, will report back when it explodes. 23:26 hiredman: Chouser ran into it too, for all he recomends using java.ext.dirs to people 23:35 * danlarkin notices how hard it is to reach a consensus among even a handful of people :) 23:35 dnolen_: soon as somebody has a tool and tutorial, I'll use it, first in line. 23:36 hiredman: setting java.ext.dirs also overwrites the default value which is the place where stuff like crypto extensions get installed 23:39 danlarkin: so really, the way java "solves" this problem is that each project I start gets a full checkout of all its deps, including clojure.jar and clojure-contrib.jar? How heavy weight does it become to play around in the repl, or use emacs & slime in that case? 23:40 hiredman: eh? 23:41 I just throw everything in ~/.jars/ 23:41 dnolen_: hiredman: interesting, tho it looks like there's a lot of confusion about the relationship between -cp and java.ext.dirs 23:41 lots of article saying that they can't be used together, when I'm doing that right now with Clojure. 23:41 hiredman: jre6 supports wildcards in the classpath 23:42 CLASSPATH=$HOME/.jars/*

Logging service provided by n01se.net