#clojure log - Oct 08 2010

The Joy of Clojure
Main Clojure site
Google Group
List of all logged dates

0:00 tomoj: no

0:01 ,(supers (class :foo))

0:01 clojurebot: #{java.util.concurrent.Callable java.lang.Runnable java.io.Serializable java.lang.Object clojure.lang.Named java.lang.Comparable clojure.lang.IFn}

0:01 tomoj: keywords implement IFn

0:01 the reader also optimizes out some keyword access I believe

0:01 the compiler I mean

0:01 nollidj: alright. thanks.

0:02 tomoj: ,(supers (class #{}))

0:02 clojurebot: #{clojure.lang.IPersistentSet java.util.concurrent.Callable clojure.lang.IMeta clojure.lang.Counted clojure.lang.Seqable java.lang.Runnable java.io.Serializable java.util.Collection java.lang.Object clojure.lang.IEditableCollection java.util.Set clojure.lang.IObj clojure.lang.APersistentSet clojure.lang.IPersistentCollection java.lang.Iterable clojure.lang.AFn clojure.lang.IFn}

0:23 nollidj: is there any notion of providing a function to pre-fill fields in a record when it is created with (record-thing. foo bar baz) ?

0:34 chouser: nollidj: nothing standard yet. Either roll your own or use one of the macros that are floating around.

0:52 duncanm: dum de dum

0:58 coldhead: yo ChanServ can i get some ops too?

0:59 nollidj: i've seen some mentions of pmap on the web. is there anything else to make running stuff in parallel easy?

1:02 e.g., a parallel reduce that partitions the set being reduced over, reduces the partitions, and then combines their values could be nice

1:05 chouser: nollidj: there's been some work on that, but there's nothing really application-ready as far as I know

1:06 nollidj: clojure's data structures (hash-maps, sorted-maps, vectors) are trees internally and thus really well suited to parallel processing, like a parallel-reduce

1:07 there's a 'par' branch on the main clojure repo on github that uses the forkjoin framework to do that sort of thing, but I think it's based on a rather older version of clojure.

1:08 nollidj: i know it's deprecated

1:08 chouser: no, that's something else I think

1:08 nollidj: i was hoping someone had whipped something up in contrib. oh well. a parallel reduce should be pretty easy to implement, except for the obstacle of not executing too many futures at once

1:08 maybe it's a separate par library

1:08 chouser: the clojure.par namespace or something is even older and deprecated

1:09 KirinDave: That's like pre-1.1, right?

1:09 chouser: hm, it has chunks, so that's circa 1.1

1:11 wow, July 2009.

1:12 I wonder how hard it would be to merge

1:12 go go gadget git

1:15 KirinDave: I'm reading Joy of Clojure for the first time in a few months.

1:15 Some of these macros fogus has in here are pretty epic. :)

1:15 chouser: :-)

1:16 "epic macro" sounds really impressive

1:16 KirinDave: I'm looking at as-futures

1:16 Which is a pretty gnarly way to go about a sublanguage.

1:16 chouser: oh, yeah. that one messes with your head, doesn't it?

1:16 KirinDave: It's straightforward

1:17 But

1:17 The decision to partition via => is pretty amusing.

1:18 I thought that was frowned upon in clojure's culture.

1:19 Maybe things are changing? :)

1:19 chouser: this is chapter 8, right?

1:20 oh, 11

1:20 KirinDave: 11.

1:20 Listing 11.8

1:21 I'd be a hypocrite of the worst order if I said I disapproved, but I've gotta confess it unsettles me a little walking through that de-structuring. :)

1:21 It's a clever and useful macro tho.

1:22 chouser: hmmm

1:22 (doc condp)

1:22 clojurebot: "([pred expr & clauses]); Takes a binary predicate, an expression, and a set of clauses. Each clause can take the form of either: test-expr result-expr test-expr :>> result-fn Note :>> is an ordinary keyword. For each clause, (pred test-expr expr) is evaluated. If it returns logical true, the clause is a match. If a binary clause matches, the result-expr is returned, if a ternary clause matches, its result-fn, which must b

1:24 chouser: I think the more narrow an embedded DSL's usage, the more warranted (and useful) syntaxy features are likely to be.

1:24 hiredman: http://gist.github.com/616395

1:24 chouser: That's not to say Rich has approved every line of code in the book. :-)

1:24 hiredman: that's yours? looks pretty tidy.

1:25 vacho: (apply f args* argseq)

1:25 hiredman: :)

1:25 vacho: why do I get error when I run that?

1:25 #<CompilerException java.lang.Exception: Unable to resolve symbol: f in this context

1:25 chouser: http://gist.github.com/560765#file_abomination_desolation.clj

1:26 hiredman: with-db transforms that into thr traditional nested functions and binds for a monad

1:27 jcromartie: chouser: how could you

1:27 KirinDave: hiredman: Is that purely functional then?

1:28 hiredman: it's a state monad under the covers

1:28 * chouser beams at jcromartie

1:28 jcromartie: I was demonstrating the power of macros to a friend. I wrote: (defmacro dont)

1:28 KirinDave: chouser: Haha @ your abomination macro

1:28 chouser: Next up, (loop ...)

1:29 chouser: heh. can't use ; in imperative 'for' loop

1:30 KirinDave: Okay.

1:30 as-promises is actually starting to freak me out.

1:31 err, with-promises

1:31 hiredman: mmm promises

1:40 chouser: I must really not want to work on these slides. I'm finding the most ridiculous other tasks so interesting...

1:48 vacho: can someone please explain this from the book: (get a-map key not-found-val?)

1:48 get is the function name, is "a-map" a parameter?

1:53 tomoj: yes

1:54 you're callin gthe 'get' function and passing it a-map, key, and not-found-val

1:55 ,(get {:foo 1 :bar 2} :foo :not-found)

1:55 clojurebot: 1

1:55 tomoj: ,(get {:foo 1 :bar 2} :baz :not-found)

1:55 clojurebot: :not-found

1:56 tomoj: ,({:foo 1} :foo 3)

1:56 clojurebot: 1

1:56 tomoj: ,(:bar {:foo 1} 3)

1:56 clojurebot: 3

2:15 lenw: morning all

2:46 nollidj: = is the same as java's .equals(). how can i do identity comparison between two data structures?

2:47 lenw: (identical? x x)

2:48 nollidj: thank you. that seems to do it.

2:48 lenw: np

2:48 hiredman: nollidj: = is not the same as .equals

2:49 nollidj: i'm using it on clojure hashmaps, so in this case it is

2:50 modulo working on nil

3:14 yayitswei: hello folks

3:15 anyone used clojure on app engine, and experienced very slow deployment times?

3:15 like 10 min. to upload a simple "hello world" WAR

3:18 kryft: yayitswei: That's a great nick. :)

3:19 yayitswei: thanks :) it helps some people pronounce my name

3:19 kryft: yayitswei: (I know almost nothing about clojure, so I'm just here to make irrelevant comments.)

3:19 Hehe

3:19 yayitswei: haha. there's also #clojure-casual

3:19 kryft: You'd think wei wasn't hard to pronounce. :P

3:20 yayitswei: Well, to be fair, I guess I do *mostly* ask relevant questions. ;)

3:21 yayitswei: nice. I'm relatively new too but i could try to tackle some of your questions

3:25 kryft: yayitswei: I don't have any specific questions right now, but thanks. :) I joined the channel a few days ago to find out what book would best suit my needs (guess what chouser recommended?)

3:26 yayitswei: I've since bugged people with some general questions such as "what kind of numeric performance can you expect".

3:26 tomoj: kryft: you're the data mining dude, right?

3:27 my memory is terrible...

3:27 kryft: tomoj: Machine learning, yes. :)

3:28 tomoj: please stick around

3:28 yayitswei: kryft: nice. i just finished stuarthalloway's book

3:28 kryft: tomoj: I don't really do what most people would call data mining, but of course all machine learning can be seen as mining data for useful knowledge.

3:28 yayitswei: kryft: hence the performance queries?

3:30 kryft: yayitswei: Yes. :) So far I've used C++ for things that need to be fast, and while I actually like C++ more than most people who are drawn to more elegant languages, it would be nice if I could get 'close enough' performance from clojure.

3:30 yayitswei: (Which I should be able to, based on what people here have told me and what I've read on the web. So yay!)

3:31 tomoj: Don't worry, I will. :)

3:33 yayitswei: kryft: if you ever end up using clojure's STM's for concurrency let me know, I'd be interested to hear your experience with them

3:35 tomoj: kryft: hadoop?

3:37 kryft: tomoj: Come again?

3:37 tomoj: do you use it?

3:38 I use it and clojure and am wondering about how other people do so

3:38 because my workflow sucks

3:39 lein jar and java -cp .......... clojure_hadoop.job -job foo.bar/baz-job -input baz/input -output baz/output

3:39 then .../bin/hadoop -text baz/output to see the output

3:40 and any exceptions have terrible uninformative stacktraces

3:40 and I _know_ there has got to be a better way

3:59 ordnungswidrig: anybody using clojure-mode from technomancy repository?

4:01 hiredman: everyone does

4:02 ordnungswidrig: navigation in clojures-1.2.0/test.clj with CTRL-. fails for me

4:03 "No known definition for: do-report (in ^{:author)" I suppose it tails parsing the ns declaration correctly

4:06 kryft: tomoj: Ah, no, I have no idea what hadoop is. :D

4:09 tomoj: orly

4:09 it's a mapreduce framework

4:09 and some other stuff too I guess

4:10 esj: morning folks

4:11 kryft: Morning esj!

4:11 Did you dream of clojure?

4:11 I only dreamt of not being able to sing. ;/

4:15 esj: i dreamt I could program clojure.... then I woke up and it was as before :(

4:16 kryft: :D

4:20 tomoj: any alephers around?

4:20 kryft: esj: Actually I'm pretty sure I know what my first clojure project is going to be. I'm going to reimplement a dimensionality reduction algorithm that I originally wrote in C++.

4:21 esj: kryft: Super, if you'd like some help, I'm happy to try.

4:22 kryft: esj: Thanks, I probably will need some pointers (or at least they will be helpful) on the general design. :)

4:22 esj: no pointers - that's C++ remember !

4:23 kryft: esj: As in how to go from an object-oriented/generic framework to clojure-optimal.

4:23 :D

4:24 Chousuke: Your biggest problem will probably be figuring out how to express the algorithm without (too much) mutation.

4:24 kryft: Hmm, I finally feel the desire to upgrade my core2duo E6600 from 2006.

4:24 Chousuke: (and whether it'll still be efficient if you do that.)

4:24 tomoj: kryft: what kind of dimensionality reduction?

4:24 clojurebot: reduction is http://groups.google.com/group/clojure/msg/4f7e3ca1af3b8b18

4:24 tomoj: I'm intensely interestion in that topic recently

4:24 kryft: Chousuke: Right.

4:24 tomoj: It's an algorithm specifically designed for visualization.

4:25 tomoj: PCA?

4:25 Chousuke: one possibility is inventing a completely new algorithm that fits the functional model better :P

4:25 kryft: tomoj: No, a new algorithm. :) http://www.jmlr.org/papers/volume11/venna10a/venna10a.pdf <- here's our paper

4:26 Chousuke: It's basically just minimizing a cost function using conjugate gradient descent, so it should fit the paradigm quite easily

4:27 tomoj: waay over my head

4:28 kryft: tomoj: What did you do again? My memory is as terrible as yours, I'm afraid. Were you the guy working for a startup?

4:28 tomoj: yeah

4:28 Chousuke: the more mathy terms you can use to describe it, the better fit it likely is for a functional implementation

4:28 kryft: tomoj: Ok, so I did remember.

4:28 esj: off the top of my head I would agree that conj grad is gonna be suitable for recursion

4:29 tomoj: I am wondering whether random projections could be used to provide feature vectors for training "contentful image" classifiers

4:29 kryft: tomoj: How is that paper way over your head, btw? I don't think it should be more complicated than the stuff you're doing. :)

4:29 tomoj: too much unfamiliar jargon

4:29 kryft: Ah

4:30 tomoj: I still haven't finished undergrad and I am a philosophy major

4:30 kryft: tomoj: No wonder you ended up programming clojure. ;)

4:30 esj: tomoj: I don't know about 'random projections' specifically, but any dimensionality reduction is probably a good idea when feeding a classifier

4:30 tomoj: I basically just read papers over and over and over until I get some little shred of intuition about the solution

4:30 then try to implement it

4:31 the question is, what is the high-dimensional space for contentful images?

4:31 esj: whether the nature of the random porjection preserves the information most salient for your classification is probably going to be a matter of 'suck it and see'.

4:31 tomoj: can it involve edge-detection, etc?

4:31 kryft: tomoj: Define contentful?

4:31 tomoj: exactly

4:31 I don't want to have to define it

4:31 I just want to provide a training set with some images marked as contentful and some not

4:32 I saw an old paper about this which used hand-devised features

4:32 kryft: tomoj: Fair enough, I meant "define" as in "what do you mean by this word". :) "This training set" is an acceptable answer.

4:33 tomoj: you seen the feature in facebook where when sharing a link, you can page through images in the page you're linking to?

4:33 contentful images are the ones you want to show up in that list

4:33 kryft: tomoj: One thing you could do is try various feature extraction methods (eg. dimensionality reduction methods) for your training data and then see which ones seem to give you good separation.

4:33 esj: OK, so they're the images labelled 1 as far as your classifier is concerned ?

4:33 tomoj: yeah

4:34 I am doing similar things with text

4:34 I hope there is a unified approach..

4:35 can't quite understand the Johnson–Lindenstrauss lemma yet

4:36 esj: tomoj: kryft is correct, you just have to suck it and see.

4:36 tomoj: woohoo

4:36 esj: as I said yesterday, the general field of ML is quite unstructured.

4:36 tomoj: it's unfortunate

4:37 supposedly compling is taking a turn for better understood theoretical models

4:37 I guess if it works, though..

4:37 esj: if you cannot define a model that is able to describe the presence or absence of your features, then you have nothing to do but shotgun try everything

4:37 kryft: tomoj: Have you tried kernel pca?

4:37 esj: if do have a model, then all sorts of nice things open up

4:38 tomoj: kryft: I don't know exactly how that goes, but I don't think its an option

4:38 I can't keep the original high-dimensional data around forever

4:38 I need to be able to incrementally add to the reduced space

4:38 kryft: tomoj: Do you need to learn the model incrementally as well?

4:39 tomoj: yes

4:39 kryft: tomoj: Ok, so you need some kind of online algorithm then.

4:39 tomoj: random indexing works for text

4:39 kryft: tomoj: I don't know much about that. (Well, actually I don't know much about anything. :)

4:39 esj: Tomoj: Thats hard. For linear type projections you will be able to project new data down. However , the optimal projection found for the initial data set is is not going to remain correct.

4:39 tomoj: instead of starting in a high dimensional space, you just use a random projection and generate random low-dimensional vectors

4:40 jcromartie: *whoosh*

4:40 esj: although if your data is sufficiently nice, the optimum wont move too much and the projection will remain useful. But its a prayer.

4:40 tomoj: e.g. for a wordbag LSA approach you just add the random low-d vectors for each word in the context window to the accumlated vector for that word

4:40 kryft: I remember reading somewhere that by the time you get your MSc degree, you realize that you don't really know anything about anything. If you go on to get a PhD, you'll come to realize that neither does anyone else.

4:41 esj: tomoj: I don't know anything about random projections. Can you shoot me a link to an intro reference please ?

4:42 tomoj: esj: http://citeseerx.ist.psu.edu/viewdoc/download?doi=

4:42 kryft: tomoj: I assume you know that your features are going to change over time, that is, the definition of relevant content will shift?

4:42 tomoj: yes

4:42 ideally, a feedback loop :)

4:42 kryft: Right.

4:42 esj: tomoj: merci.

4:43 tomoj: that's about "random indexing"

4:43 kryft: And here I was feeling guilty for sitting around on IRC instead of dragging my ass to work.

4:43 tomoj: which is as I take it a particular approach to using random projections for dimensionality reduction

4:43 (where you don't actually have to do the reduction)

4:43 kryft: tomoj: "where you don't actually have to do the reduction"?

4:44 tomoj: you just generate random low dimensional vectors

4:44 but their dimensionality is high enough that with high probability any two of them are nearly orthogonal

4:45 so the projection preserves pairwise distances

4:45 I may be off my rocker

4:46 piecing together scraps from many papers I only sort of understand, and maybe I got it pieced together all wrong :)

4:47 kryft: tomoj: Ah, "don't have to do the reduction" in that sense.

4:47 Hmm, now I really have to drag my ass to work, bbl. :)

4:47 Even though this is technically work, I have a lunch appointment in 40 minutes.

4:49 esj: aaaah..... So you put your text into the matrix F, but this thing is mondo HD. So you want to reduce the dimension. But the salient trait is the pairwise distance between rows, F_w, as that is the context you're after. So you want a pairwise distance preserving projection ?

4:55 so then. The random index is just a way of producing an orthogonal basis. Then you effectively scan the documents and incrementally project each word onto that basis. Very smart !

4:55 tomoj: great paper, thanks :)

5:08 tomoj: esj: yeah

5:08 I just wonder how widely this can be applied

5:08 esj: its nice when authors bother to explain the intuition behind what's going on

5:08 tomoj: it seems strange to use it for classification

5:09 esj: i think the idea of the random projection is pretty generally valid

5:09 tomoj: because you end up taking the cosine similarity between a sum of a bunch of context vectors compared to just one smaller sum

5:09 esj: the problem is that these vectors are not particularly low dimensional themselves

5:09 or they wouldn't be orthoganol ;(

5:09 tomoj: like comparing the sum of the entire "spam" corpus to a single questionable document

5:09 esj: right

5:10 I'm using 1800

5:10 esj: have mercy !

5:10 tomoj: it performs reasonably well with a totally naive slow implementation

5:10 so I expect to be OK when I switch to a fast implementation from mahout or something

5:11 esj: i have no intuition, but that seems reasonable.

5:12 tomoj: the similarity values end up skewed

5:12 ususally (-0.02, 0.3) or so

5:12 esj: in what sense ?

5:12 tomoj: for cosine similarity

5:12 which could be [-1,1]

5:13 esj: up to 0.3 is pretty noisy.... I think.

5:13 15% noise ?

5:13 but maybe not too bad for the application.

5:13 tomoj: what is this noise you speak of

5:14 esj: sorry, I misunderstood you. I thought you meant that the random projection introduced a skew in the reduction, when you meant that distribution was skewed. Sorry.

5:14 there is no noise. Doh.

5:15 tomoj: I build up one vector by summing the context vectors observed in any document in the positive set

5:15 then compare that vector to individual documents to classify

5:15 doesn't seem right

5:17 esj: indeed, it seems you are comparing different things

5:18 tomoj: http://www.kyb.mpg.de/bs/people/weston/papers/unified_nlp.pdf

5:18 that paper has got me thinking

5:18 esj: i thought the projection allowed you to compare words

5:18 not words to documents

5:18 tomoj: yes, that is the original application

5:18 esj: but then I only scanned it at rapid pace, so might easily have missed the point.

5:18 tomoj: there was some work on using that output for SVMs

5:18 to do classification

5:19 I get decent results this way, though

5:19 the similarity range is just skewed

5:19 esj: do you require centrality for the the SVM ?

5:19 i'm lost

5:21 tomoj: I didn't read the SVM stuff closely, I just know it exists

5:21 esj: so why is the skew upsetting you ?

5:22 tomoj: http://acl.ldc.upenn.edu/coling2004/MAIN/pdf/70-640.pdf

5:22 well, I just feel like this is not the right model

5:22 I never saw anything like this in the research

5:23 (that last link is the SVM stuff)

5:24 wish I could pay sahlgren to talk to me for a few hours

5:24 esj: speaking off the top of my head, if you want to compare documents you want to put them into this context space.

5:24 temp01: coldhead:

5:25 esj: you have a method for doing it stepwise on words

5:25 so if you have the collection of contexts for each word in a document if you can find a sensible way to combine you have a representation of the contextual content of the documetn

5:26 so you do this for each document in the corpus, and feed that representation to the classifier of your choice

5:26 the slippy bit is aggregating the context-by-word to context-by-document

5:27 its tempting to say just add the context matrix produced for each word

5:27 the space is nearly orthogonal, so why not ?

5:28 but I'm way beyond myself by now, so I'm likely to be talking pure bollocks

5:30 tomoj: well, that's what I did

5:31 esj: awesome !

5:31 LauJensen: Good morning all

5:31 tomoj: the weighting scheme becomes very important I think

5:31 esj: hello Lau.

5:31 tomoj: because both spam and non-spam contain many instances of common words

5:31 esj: that should be OK though

5:31 tomoj: filtering stopwords helps, but doesn't solve the problem

5:31 esj: because the classifier shouldn't care

5:32 tomoj: oh, hmm

5:32 I'm not actually really running a classifier

5:32 I just directly compare (cosine similarity) the summed contexts by document to a single context for a questionable document

5:32 LauJensen: Are you rebuilding SpamAssassin ?

5:32 esj: tomoj: You should really throw it at a classifier

5:32 tomoj: no, a menagerie of classifiers

5:32 yes

5:33 esj: tomoj: indeed, try a full menagarie.

5:33 tomoj: well, I meant I have a bunch of different classification problems

5:33 but, yeah, that too :(

5:33 maybe mahout will save my day

5:34 LauJensen: tomoj: What are you doing?

5:34 esj: there is a tried and tested method of PhD students in Machine Learning which is to get http://rii.ricoh.com/~stork/DHS.html the computer manual part of that

5:34 which is a uniform interface to all the classifiers in the book

5:34 so you can rapidly iterate through all your options

5:35 its intellecutally bankrupt, but I've seen it done.

5:36 tomoj: hehe

5:36 temp01: cais2002: You Head!

5:36 coldhead: sry

5:37 cais2002: temp01: did u start from letter c? how come some are skipped?

5:38 temp01: you're the first nick with letter c

5:40 tomoj: LauJensen: uhh, content classification

5:45 notsonerdysunny: Hello everybody, I was just exploring cake .. but haven't been successfull using it .. I got http://gist.github.com/616564 when I did "cake deps" can anybody help?

5:57 _ato`: notsonerdysunny: /var/lib/gems/1.9.1/bin/cake:9:in equire': no such file to load -- rubygems (LoadError)

5:57 sounds like you might need to install rubygems http://rubygems.org/

5:58 fliebel: morning

6:08 LauJensen: notsonerdysunny: how did you install it ?

6:14 notsonerdysunny: LauJensen: I installed it via gem install

6:15 and then just added the directory containing the cake to path

6:17 LauJensen: notsonerdysunny: On which OS?

6:18 notsonerdysunny: linux ubuntu 10.04

6:18 LauJensen: actually I think it might be your path thats messing things up. When you install via gems you shouldn't touch the path: http://github.com/ninjudd/cake <-- See installation

6:23 notsonerdysunny: LauJensen: If I don't add the gems directory to path then .. how would it know where the executable is?

6:23 raek: just saw this very neat summary of the Clojure web ecosystem: http://groups.google.com/group/clojure/msg/64e8ed91791d81d2

6:24 LauJensen: notsonerdysunny: As I recall, once you've installed Ruby and the gem system, the cake gem is installed somewhere local to that application

6:24 I havent done any permanent path fiddling on Arch IIRC

6:26 notsonerdysunny: LauJensen: I am new to the gem thing .. is it ok if reinstall using gem install?

6:26 LauJensen: Probably, but also clear cake from your PATH or reboot if you're not sure

6:29 Anybody got a link to that hilarious post about lisp syntax that fogus did?

6:30 got it

6:31 esj: share with the class ?

6:31 notsonerdysunny: LauJensen: http://gist.github.com/616606 is what I got when I reinstalled it via "gem install -V cake" now cake is not in path .. what do you think I should do?

6:32 LauJensen: esj: http://blog.fogus.me/2010/08/30/community-standards/

6:32 esj: merci

6:32 LauJensen: notsonerdysunny: What does lein deps get you now?

6:34 notsonerdysunny: "lein deps" seems to work fine

6:34 I just created "lein new cakeexp" and then ran "lein deps" .. it seems fine

6:35 but I want to try cake ...

6:36 bobo_: notsonerdysunny: link /bin/cake to /var/lib/gems/1.9.1/gems/cake

6:36 except use the correct path

6:40 notsonerdysunny: bobo_: I got http://gist.github.com/616608

6:40 same as before I redid the installation ..

6:41 mrBliss: LauJensen: do you use rcirc or erc?

6:41 LauJensen: erc

6:41 mrBliss: me too, but I just read rcirc is the standard irc client for emacs. Have you got any experience with it?

6:42 LauJensen: yea, I didnt like it. Couldnt get logging and notifications right with it

6:42 mrBliss: were there any advantages over erc?

6:43 LauJensen: Its more lightweight I guess

6:44 mrBliss: ok, I'll just stick with ERC.

7:07 AWizzArd: LauJensen: please give me the timing: (do (import 'java.io.ByteArrayOutputStream) (def baos (ByteArrayOutputStream. 100000000)) (time (dotimes [i 80000000] (.write ^ByteArrayOutputStream baos 0))) (.reset baos))

7:07 LauJensen: AWizzArd: 796.92 msecs

7:09 AWizzArd: fast, what cpu?

7:09 LauJensen: you mean, 'which cpu' ? :)

7:09 AWizzArd: yes

7:09 LauJensen: I actually dont know

7:09 AWizzArd: <--- not a native speaker

7:10 LauJensen: Intel(R) Core(TM)2 Duo CPU T6600@2.20 GHz x 2

7:10 AWizzArd: Though it is faster than my cpu or the one of lpetit: why is it so slow? The timing you gave boils down to about 100 mb/sec. This is 1% of what should be possible.

7:11 Yes, this is byte by byte, but still...

7:13 LauJensen: and what is shocking me even more: lpetit translated this into a very simple pura Java program. And on my machine the pure Java code runs much slower. How is that possible?

7:17 LauJensen: I'll think about it once I get back, I have an appointment I have to run to, sorry :)(

7:18 AWizzArd: kk

7:26 _ato: AWizzArd: I get the same times for Clojure and Java code, around 410 ms (after JIT warmup)

7:26 I guess maybe ByteArrayOutputStream is just slow, setting an array directly is much faster

7:27 ah.. one thing is it's thread-safe

7:28 AWizzArd: _ato: a BAOS is thread safe?

7:28 _ato: can you please post your clj and java which gave you 410 msecs?

7:32 _ato: AWizzArd: http://gist.github.com/616651

7:34 AWizzArd: _ato: ah okay, so you also notice that Java at first is way slower than Clojure in this case.

7:34 _ato: AWizzArd: and yep, BAOS locks with "synchronize" keywords on the methods: http://www.java2s.com/Open-Source/Java-Document/6.0-JDK-Core/io-nio/java/io/ByteArrayOutputStream.java.htm

7:35 AWizzArd: The interesting this is that I warmed up the cache, but not so deeply as you did, and I will now try this too.

7:35 _ato: AWizzArd: it's probably only slower cause my Clojure process had already warmed the JIT since I was doing other stuff in it earlier

7:35 AWizzArd: _ato: btw, what CPU do you have?

7:35 _ato: model name : Intel(R) Core(TM) i7 CPU 860 @ 2.80GHz

7:36 AWizzArd: AAh, Core i7 are real beasts.

7:36 Okay, so the synchronized thing has impacts on the performance.

7:37 Although only one thread is doing the work here.

7:40 _ato: http://gist.github.com/616655

7:40 AWizzArd: what the..

7:41 _ato: looks like Clojure's "locking" is much slower than synchronize even.

7:42 AWizzArd: amazing

7:42 _ato: they might be implemented differently, but it's the same sort of thing. Locks can be pretty expensive

7:43 AWizzArd: so, the synchronized is causing that speeddown, great, you solved this mystery

7:43 So a C program should also be slowed down drastically when locking each access

7:44 _ato: in theory at least

7:44 * _ato tries it

7:46 AWizzArd: _ato: what Clojure version do you run? Your 50 msecs translate into 2250 on my machine, with a recent clojure.jar

7:46 _ato: erm.. some old version 1.1

7:47 some git snapshot of 1.1

7:47 AWizzArd: I see.

7:47 _ato: might need type hinting or something, I dunno

7:50 AWizzArd: but not by a factor of 45 times, and I ran it also several times now

7:51 strange

7:51 _ato: 64 bit os?

7:53 _ato: yeah

7:54 very similar results with C: http://gist.github.com/616670

7:54 I just timed with the shell "time" command. Too lazy to figure out the C API for getting current milliseconds

7:56 AWizzArd: _ato: cool, thank you very much, I am interested in those results.

7:57 _ato: So, the C program is faster, but it also shows the same effects, that locks are very expensive.

8:00 _ato: yeah

8:02 fairly similar times to the JVM actually.. my crappy C being maybe 20% faster. (There's probably much faster ways to implement that in C, but in the interest of making the microbenchmarks similar kind of code...)

8:07 AWizzArd: _ato: this benchmark is important for me, because it gives me a base to compare my Clojure serializer lib with.

8:18 lpetit: hello

8:18 AWizzArd: Hi lpetit

8:18 I wasn't away yesterday.

8:19 And I got your Java program and tested it, just moments after you left.

8:19 lpetit: if some people meet problems with a recent release of ccw, please talk. I was not playing with it that much the last weeks, so don't count on me finding the bugs by myself :)

8:19 AWizzArd: _ato has just tried out also some interesting things

8:19 lpetit: AWizArd: hi

8:19 which ones ?

8:19 AWizzArd: lpetit: http://gist.github.com/616651 and http://gist.github.com/616655 and http://gist.github.com/616670

8:20 The slowness of the BAOS comes from it being synchronized.

8:21 And clojures locking is even much more inefficient that Javas 'sychronized'.

8:21 lpetit: as expected, I would say, no ?

8:22 and direct C on arrays is equivalent to direct clojure/java on arrays

8:22 did I read the figures correctly ?

8:22 _ato: yep

8:24 lpetit: great

8:24 _ato: and pthread's mutex seems to add a similarish amount of overhead as Java BAOS's synchronize

8:27 chouser: did you know you can put multiple files in a single gist?

8:28 _ato: chouser: yep, but I use an emacs command for gisting which doesn't support that

8:28 chouser: AWizzArd: why do you say clojure's locking is more inefficient than Java's 'synchronized'?

8:29 AWizzArd: chouser: _ato found that out

8:29 _ato: chouser: AWizzArd is going off this http://gist.github.com/616655 as compared to ByteArrayOutputStream's synchronize write() method. I may well be doing something dumb there ;-)

8:29 AWizzArd: See http://gist.github.com/616655

8:30 Here _ato simulated the synchronized that we find in the BAOS class.

8:30 Writing to baos takes 450 msecs

8:30 With _atos locking that became 9000 msecs

8:31 While writing directly into a byte arrat, without locking, got it down from 450 to 50

8:32 I am currently facing a totally different problem. When I compare my my serializer (time (with-store-writer baos (dotimes [i 80000000] (store-state (byte 0))))) with direct baos writing (time (dotimes [i 80000000] (.write ^ByteArrayOutputStream baos 0))) then I see that my serializer takes 8x more time.

8:33 The store-state fn is part of the PStorable protocoll. My hope/idea now is that I need to AOT the code and measure again.

8:34 Did anyone of you experience a speedup in calling protocol fns after AOTing them?

8:34 chouser: I see that locking compared to not locking is much slower. Was there also a test against Java's "synchronized"?

8:34 because I'm pretty sure the same bytecode is generated.

8:34 AWizzArd: chouser: yes, as I said: clojure locking: 9000 msecs. java synchronized: 450 msecs

8:34 http://gist.github.com/616651 <--- here _ato measured javas synchronized

8:34 chouser: And there are no other differences in the code?

8:35 ah, that's the gist I was missing, thanks.

8:35 AWizzArd: yes

8:35 _ato: here's a more direct translation: http://gist.github.com/616708

8:35 AWizzArd: (/ 9000 450.0)

8:35 ,(/ 9000 450.0)

8:35 clojurebot: 20.0

8:35 AWizzArd: So it seems that clojures 'locking' takes about 20x longer than Javas synchronized

8:36 chouser: this is Clojure 1.2?

8:37 _ato: heh nah, some old 1.1 snapshot I had lying around. ;-) I should retry on 1.2, just a sec

8:37 AWizzArd: rhickey: wb

8:37 rhickey: hi

8:37 AWizzArd: _ato: can you download a fresh clojure.jar from http://build.clojure.org/ and time it with that?

8:38 rhickey: _ato: we're not doing naive benchmarking again, are we? :)

8:39 _ato: woah... 1.2 is much worse "Elapsed time: 10472.37607 msecs"

8:39 chouser: rhickey: it's our favorite kind! :-P

8:39 _ato: rhickey: don't look at me! it's AWizzArd's benchmark :p

8:39 chouser: today, Java's synchronized vs. Clojure's locking. Yeah, I know.

8:40 rhickey: for instance, the Java compiler might know every entry in a new byte array is already 0

8:40 where's the Clojure code?

8:40 chouser: http://gist.github.com/616655

8:40 moving the cast to primitive outside the loop helps some

8:40 _ato: compilers are sneaky like that

8:41 chouser: rhickey: did you see I merged master into par? Would you like that committed onto your par branch? It seems to run.

8:41 rhickey: chouser: I've seen little - been in Denmark all week

8:42 LauJensen: rhickey: Hey, thanks for your talk in Aarhus, was very interesting, especially the final upshot about Pods :)

8:42 chouser: oh, I can't keep track of your crazy globe-trotting schedule. :-) I just sent email last night.

8:42 AWizzArd: rhickey: compare http://gist.github.com/616651 and http://gist.github.com/616655 and http://gist.github.com/616670

8:42 rhickey: chouser: I think David Liebke has already got the par bits working on master (other direction)

8:42 chouser: oh, ok

8:42 rhickey: AWizzArd: I'm not going to spend my time on that right now

8:43 chouser: so hopefully we will move forward with with par bits in master so par can move forward there

8:44 chouser: rhickey: sure. I'd think the direction doesn't really matter. I mean, the source tree I have here is all of par + all of master -- could be committed either place.

8:44 but I won't move til I hear what Liebke's got

8:44 AWizzArd: rhickey: not required, just if you are interested. One thing that is way more interesting for me: will AOTed Protocoll functions be called much faster than non-AOTed ones?

8:44 rhickey: chouser: yeah, I'm not sure if he tweaked a few bits - he's looking to move par forward

8:45 AWizzArd: shouldn't be faster at all

8:46 chouser: he's a better person for it.

8:46 rhickey: heh

8:46 chouser: hm, maybe that's not what I meant

8:51 The creator of http://clojure.org/cheatsheet doesn't want to update it for 1.2, recommends we link to http://clojuredocs.org/quickref/varsonly/Clojure%20Core instead

9:32 abedra: rhickey: was there ever a thought about enabling a second arg to promise to enable the await call to have a timeout?

9:32 chouser: (doc await-for)

9:32 clojurebot: "([timeout-ms & agents]); Blocks the current thread until all actions dispatched thus far (from this thread or agent) to the agents have occurred, or the timeout (in milliseconds) has elapsed. Returns nil if returning due to timeout, non-nil otherwise."

9:32 chouser: oh, on promise.

9:33 abedra: yeah

9:37 AWizzArd: Aah, I found my bottleneck! I just noticed that accessing a binding is much less efficient than accessing a "normal var".

9:46 rhickey: I have a Protocol PStorable that offers a fn store-state. And I did (extend Byte PStorable {:store-state foo}). Now I call store-state 80 mio times in a loop vs 80 mio calls of foo directly. For me store-state times 5400 msecs vs 3800 msecs when using foo directly. Do you have an idea why it is so?

9:49 rhickey: AWizzArd: why wouldn't it be?

9:49 AWizzArd: I would think that calling a protocol fn is at least as efficient as calling an ordinary function.

9:50 rhickey: AWizzArd: why?

9:51 AWizzArd: Probably I misunderstood the intention of Protocols. Now that I see those results it seems they are more a replacement for defmulti and are more efficient than that. But ordinary fns are still more efficient as they don't have to dispatch a type.

9:51 rhickey: AWizzArd: why spend time on this?

9:52 premature optimization ____

9:52 AWizzArd: rhickey: not really. I want my serializer to be fast. When storing gigabytes of data there is a difference between 12 minutes and 2 mins.

9:53 rhickey: um, 5400/3800 < 2, not 6

9:54 AWizzArd: Yes yes, that is not the main problem. I noticed that writing directly into a ByteArrayOutputStream is 8x faster than using my serialize function. And I want to understand why.

9:54 I noticed that the main cause of this is that (serialize obj) accesses the var *stream*, which is a binding.

9:54 rhickey: AWizzArd: ok, but I don't want to spend more time on optimization right now

9:54 stuartsierra: rhickey: abedra & I had a question: is there a reason promise/deliver doesn't offer a "read with timeout" or "check if ready" function?

9:55 jcromartie: I love Docco http://jashkenas.github.com/docco/

9:55 rhickey: stuartsierra: It's been requested a few times, as well as some generic is-ready. What's needed is a decent design that fits into the ref framework

9:55 stuartsierra: ah ok

9:56 jcromartie: Is there a Clojure code coloring lib yet? It seems like a pretty simple thing to do for Clojure, considering the hard work of parsing is already done.

9:56 stuartsierra: So what we really want is "deref with timeout" and "will deref block?" functions

9:56 jcromartie: Wow. Moment of zen here.

9:56 rhickey: stuartsierra: cemerick had some requirements in this area and we've discussed it on irc a few times

9:56 abedra: ah

9:56 i see

9:57 rhickey: stuartsierra: you need to think about to what abstractions those correspond, and have we got the hierarchy aligned, do we need a new interface etc. I don't want meaningless methods because we hang this on the wrong thing.

9:58 stuartsierra: sure

9:58 abedra: agreed

9:58 stuartsierra: We basically wanted to make sure it wasn't left out for technical reasons

9:58 rhickey: nope

9:59 I just haven't had time to work on it and haven't seen anything other than 'features' proposed

9:59 stuartsierra: ok

9:59 abedra: rhickey: stuartsierra and I are working on building out some stuff with zeromq and this is where it came up

10:00 rhickey: we can write up a proposal this afternoon

10:01 rhickey: a good start would be to make a little spreadsheet of the reference types and see where this fits, is it it's own abstraction (e.g. MaybeDeref?). Don't forget delays and futures

10:01 stuartsierra: right.

10:01 promises, delays, futures, agents, refs, atoms, vars, ... anything else?

10:02 rhickey: stuartsierra: that's probably it

10:02 stuartsierra: ok

10:02 rhickey: IMightDeref

10:03 dnolen: heh

10:03 abedra: :)

10:04 stuartsierra: IWishIMayIWishIMight

10:04 rhickey: dnolen: what? I like it!

10:08 AWizzArd: When I want to implement my own OutputStream, a combo of BufferedOutputStream and DataOutputStream, to get rid of this Java synchronized, then deftype is what I should use, yes?

10:09 stuartsierra: yes

10:09 AWizzArd: good

10:10 chouser: ,(:doc (meta #'loop))

10:10 clojurebot: "Evaluates the exprs in a lexical context in which the symbols in\n the binding-forms are bound to their respective init-exprs or parts\n therein. Acts as a recur target."

10:10 chouser: that's kinda nice, compared to (doc loop)

10:13 rhickey: chouser: I give in on that. If someone wants to submit a patch with doc support for the special ops, I'll take it, as long as they contain links to the full docs and aren't too long themselves

10:22 chouser: ok. I'll probably take a swing at it if nobody else wants to: http://www.assembla.com/spaces/clojure/tickets/454-docstrings-for-special-ops

10:23 rhickey: chouser: great, thanks! - just assign it to yourself then

10:30 shoover: ahhh, open source in action

10:52 sthiagar: I am using clojure.contrib.lazy-xml to parse-trim a XML string (returned from a java API) and filter to an element which contains another XML document. I assign this XML document into a map and then lazy-xml parse it to get to some text elements. When number of strings parsed in the first time is less, this works fine, when its more I hit a Caused by: java.util.concurrent.ExecutionException: org.xml.sax.SAXParseExceptio n: <Line

10:53 jcromartie: ouch LauJensen you're getting slammed on HN

10:53 sthiagar: I think its because of lazy sequences and tried putting doall around the lazy-xml parse-trim and around the map where the inner xml is extracted, to no avail

10:53 jcromartie: I appreciated the Uncle Bob post but wow

10:53 LauJensen: jcromartie: Yea, people love to cry out loud. We're really in the age of emotions

10:54 I think its amazing how sensitive 'hackers' are. I make a comment stating that its hard for me to think about correct robust programming and Ruby in the same sentence, and then the whole world breaks down, I mean come

10:54 on..

10:54 jcromartie: just yesterday

10:54 I was using Ruby

10:54 and this drove me nuts...

10:54 LauJensen: this?

10:54 clojurebot: This is a test.

10:54 jcromartie: "asdf"[0..3] returns "asdf"

10:54 that's OK

10:55 "asdf"[4..5], out of bounds, returns ""

10:55 "asdf"[5..6], also out of bounds, returns nil

10:56 LauJensen: O_o ? :)

10:56 jcromartie: I really shouldn't comment, or else I'll get The Internet (tm) on my back

10:56 fogus_: jcromartie: Is "asdf"[4..5] out of bounds? Maybe it grabs the \0 ?

10:56 jcromartie: :)

10:56 abedra: LauJensen: you're safe here :)

10:57 jcromartie: fogus_: I asked on #ruby and they said it had nothing to do with null-terminated strings :P

10:57 but I never got a good answer

10:57 LauJensen: abedra: thanks :)

10:58 fogus_: jcromartie: Hmm. Oh well, I'm out of ideas then

10:58 abedra: LauJensen: I say that as a recovering rubyist

10:58 even though I still maintain some ruby libraries like RCov

10:59 chouser: sthiagar: your first post got cut off. Can you paste the exception and/or code at a paste site?

10:59 LauJensen: abedra: Thats nice to know. Im not actually sure if its Ruby guys who are doing the whining. Apparantly people nowadays explode if you say anything other than "yea.. thats cool". The moment you disagree with something, you have to be very very careful unless the globe melts down. I think its pathetic. I understand it in teenage woman but not in adult developers

11:00 abedra: LauJensen: But.. But.. Somebody on the internet is WRONG!!!!

11:00 LauJensen: Anyway, Ive started 1 flamewar today, time to kick back and relax, nice weekend everybody

11:00 abedra: LauJensen: See you at the conj!

11:00 LauJensen: abedra: Yea I wish :| I'm sending my best man Christophe :)

11:01 abedra: LauJensen: I thought you were coming as well

11:01 fliebel: Hey, if I need to use a jar without docs or source, is there a way to tell which methods it exposes?

11:02 sthiagar: chouser: this is the exception http://pastebin.com/MqEpupUP

11:02 stuartsierra: fliebel: try Mycroft

11:03 abedra: ^^

11:03 it's a lifesaver

11:04 fliebel: stuartsierra: That's a search engine, right? I was hoping there was a way to inspect the class files or something in the fashion.

11:04 stuartsierra: No. Mycroft is a Clojure/Java runtime inspector

11:05 http://github.com/relevance/mycroft

11:05 fliebel: stuartsierra: Ah, that's something different from what google shows me :)

11:05 stuartsierra: It's a much-used name :(

11:06 chouser: I looked at mycroft, but I didn't understand it.

11:08 stuartsierra: It's not well-documented yet

11:08 Add it as a dependency, then do ...

11:08 (use 'mycroft.main) (start 8080)

11:08 Then (inspect foo) will pop up a browser showing everything about foo

11:09 fliebel: nice

11:09 stuartsierra: I think that should be (run 808)

11:09 (run 8080)

11:09 whatever

12:42 jkkramer: Question: in Lau's latest post (http://bestinclass.dk/index.clj/2010/10/taking-uncle-bob-to-school.html) he says towards the end, "some of the functions in vector are generating anonymous functions everytime they're called" -- re a function that has a letfn. is that really true? i thought fns in letfn only get compiled once and don't incur any particular performance hit

12:43 hiredman: lau is an ignorant jerk, so I recomend ignoring him

12:44 technomancy: jkkramer: it's not true; you are right. runtime cost is just an instantiation of a precompiled class.

12:45 jkkramer: technomancy: k thanks, i'm not going crazy then

12:53 dysinger: I'm leaving for Clojure Conj by way of Vancouver on Monday - it's coming quick!

12:53 :)

12:55 amalloy: wow, if lambdas took a compilation every time you ran them, idiomatic clojure code would be terrible

13:05 ohpauleez: Can someone explain static functions in clojure 1.3 quickly

13:05 or point me to a document

13:08 raek: ohpauleez: http://www.assembla.com/wiki/show/clojure/Enhanced_Primitive_Support

13:08 ohpauleez: raek: thanks!

13:09 raek: basically, calls to a static function is hard-comiled to that function instance

13:09 i.e. redefining the var will not alter the behaviour of functions using the static function

13:10 this enables the compiler to do some additional optimizations, and true primitive support is possible

13:33 wooby: the conj is coming!

13:39 chouser: I know exactly how you feel.

13:40 * seancorfield is so bummed he can't attend the conj :(

13:44 technomancy: I love conjferences.

13:47 amalloy: technomancy: conjurances?

13:48 Raynes: Yay, 14 days. :D

13:50 freakazoid: where is the conj?

13:50 wooby: freakazoid: durham, nc: http://clojure-conj.org/

13:51 freakazoid: Ah, too far.

13:51 It would have to be in the bay area for me to go

13:55 amalloy: freakazoid: where in the bay area are you?

13:56 KirinDave: So, if I said, "I am going to go to a place in SF in an hour or so, does anyone want to pair program clojure?"

13:56 Would I have any takers?

13:56 amalloy: KirinDave: if you said it on a weekend...probably

13:56 chouser: KirinDave: if by SF you mean Fort Wayne, sure!

13:57 moogatronic: chouser: you are in Indiana?

13:57 KirinDave: amalloy: This _is_ my job.

13:57 But the stuff is gonna be open source soon, so I don't mind other people seeing it.

13:57 moogatronic: <- Bloomington

13:57 amalloy: KirinDave: i'm aware. but it ain't mine

13:57 chouser: moogatronic: yep

13:57 KirinDave: amalloy: ;)

13:57 amalloy: Might be though if clothesline ends up working out.

13:57 moogatronic: I grew up in Ft.W though.

13:57 chouser: moogatronic: ah, cool!

13:57 KirinDave: Webmachien is pretty damn awesome.

13:58 moogatronic: my parents still live there, maybe I'll take you up on that Pair offer someday. =)

13:58 chouser: moogatronic: sure, it'd be fun

14:02 freakazoid: amalloy: mountain view

14:02 amalloy: freakazoid: i worked in mountain view for a couple years. in SF now, though

14:02 freakazoid: I've thought about moving there a few times, but it's unlikely to ever happen. The years that I should have lived in SF have passed.

14:03 KirinDave: There is always portland, freakazoid.

14:03 amalloy: heh. well, i didn't do it on purpose, but here's where the jobs were

14:03 moogatronic: everyone gives portland love

14:03 KirinDave: I dunno what kind of magic people expect from a city tho.

14:03 You gotta work it like any location.

14:03 freakazoid: I was just in portland for oscon. It was like a ghost town, but it was very easy to get around on public transportation and to get cabs.

14:04 Cabs are essentially a joke in most of the bay area.

14:04 bartj: I have seen my colleague do this in emacs - change some clojure code, save it and then automatically execute the changed function on the Repl

14:04 is this possible in vimclojure?

14:04 chouser: here too

14:04 KirinDave: They're trying to grow fast and not doing very well.

14:04 moogatronic: i've been in smallish college towns for most of my life now.

14:04 bike to everywhere

14:04 freakazoid: I walk to work, and bike to shoreline lake and back (by moffett field) for exercise

14:05 essentially, I bike the distance from the train station to google and then back

14:05 moogatronic: in indiana people still sometimes throw stuff at you if you're on a bike on the road... even college towns.

14:05 freakazoid: (because I don't work for google)

14:05 moogatronic: though that happened to me in Livermore, CA once too.. Maybe there's a bullseye on my back.

14:06 freakazoid: Livermore has a lot of rednecks

14:06 amalloy: moogatronic: tjat

14:06 er

14:06 moogatronic: is it like LLNL + rednecks?

14:06 amalloy: that's crazy. i've never been anywhere where people would throw stuff at bikers

14:06 freakazoid: Pretty much.

14:06 moogatronic: i pretty mcuh rode to LLNL, and to the bart station

14:06 freakazoid: Easy solution to that problem. Live somewhere with open carry and carry a pistol on your hip.

14:07 moogatronic: haha, IN is concealed carry

14:07 i have a permit, but never carry

14:07 freakazoid: People don't throw shit at you if they think you might return the favor with bullets.

14:07 Heh, big lump in your spandex shorts. "But it is concealed!"

14:07 mrBliss`: I'm glad I live in Europe

14:08 jcromartie: so it might seem kind of silly, but is there a reader that *keeps* comments?

14:08 moogatronic: mrBliss: SUVs running redlights have almost killed me too.

14:08 freakazoid: jcromartie: I've thought about that too.

14:08 amalloy: jcromartie: ; comments, or (comment s)?

14:08 jcromartie: both

14:08 oh

14:08 I have it

14:08 convert ; ... into (comment ...)

14:08 freakazoid: jcromartie: Genesis's bytecode format was completely decompilable to commented, formatted source.

14:09 ; is not a reader macro?

14:09 amalloy: (comment) is a macro; you could probably just override it with your own macro that does something else

14:09 chouser: jcromartie: insufficient. Would also like one that keeps all whitespace.

14:09 mrBliss: moogatronic: that's true, but here at least idiots don't get to carry guns

14:09 freakazoid: mrBliss: neither do non-idiots ;-)

14:09 jcromartie: I want to implement Docco or Pycco for Clojure

14:09 amalloy: chouser: try using cat instead of a repl

14:09 jcromartie: mrBliss: yeah, only police or psychopaths

14:09 chouser: amalloy: oh!

14:09 amalloy: of course!

14:09 jcromartie: and the difference is sometimes hard to tell

14:09 freakazoid: jcromartie: you say that lik ethere's a difference… yeah, that

14:10 people's fear of guns is amusing

14:10 mrBliss: jcromartie: never encountered any violence in my life yet, never seen a real fight

14:10 moogatronic: now if we can only tie this into a discussion on emacs and vim, which is better, and include religion somehow.

14:10 freakazoid: it's amazing how little people trust one another but how much they're willing to trust government at the same time

14:10 jcromartie: JESUS EDITS MODALLY

14:10 amalloy: moogatronic: only right-wing nuts would suggest that

14:11 freakazoid: not right wing-nuts?

14:11 moogatronic: i think the parse is valid either way

14:12 chouser: jcromartie: heh. so do emacs users. shhhhh

14:12 mrBliss: freakazoid: In Europe you can trust your government more than in the US (don't get me started on health care and republicans). But now I'll switch back to emacs and clojure :-)

14:12 jcromartie: yes please

14:12 freakazoid: heh, I'd say people DO trust their governments more than in the US.

14:13 CAN is a different matter.

14:13 jcromartie: let's talk about using the reader and preserving comments

14:13 :P

14:13 mrBliss: jcromartie: why do you want to preserve them?

14:13 jcromartie: I'm sure I'd figure it out if I actually *tried* it

14:13 mrBliss: I want to associate comments with forms

14:13 freakazoid: jcromartie: it seems to me you'd have to do something to the compiler as well

14:13 jcromartie: so that I can line them up in HTML output, a la Docco

14:13 http://jashkenas.github.com/docco/

14:14 or Pycco http://fitzgen.github.com/pycco/ or Rocco http://rtomayko.github.com/rocco/

14:14 freakazoid: or maybe stick the comments in metadata via a macro, when metadata is available to you?

14:15 jcromartie: since 'read' reads the next form from a stream, I could try to read comments from the stream first

14:15 and alternate, read-comments, read

14:15 building a list of pairs of comments and forms

14:15 nested

14:16 freakazoid: hmm

14:17 naah

14:17 just build a datastructure you can easily strip comments from

14:17 sandGorgon: is there any web framework in clojure with something like DB migrations - or any other way of handling schema changes ?

14:17 freakazoid: sort of the same thing

14:17 but the comments would just be tagged as such

14:18 jcromartie: either way

14:18 freakazoid: you can do it with macros, though

14:18 just have a macro that returns your pair

14:23 chouser: stuff the whitespace and comments into metadata of nearby forms

14:25 freakazoid: It would be cool to have a datastructure that could be unparsed into commented source as well

14:25 deparsed?

14:25 I guess that's "serialized"

14:25 or formatted

14:25 chouser: right

14:25 printed

15:03 djwonk: question about clojure-conj... when does it wrap up on Saturday. figuring out travel plans.

15:33 pjstadig: on the registration page it says 5:30pm Saturday http://clojureconj.eventbrite.com/

15:34 but i'm not sure if that's intentional or eventbrite's addition

15:36 fogus_: It could work! http://craplogo.me/logos/Clojure_logo.png

15:38 chouser: ehe

15:38 +h

15:51 rhickey: what does "Approval: vetted" mean? is this workflow documented somewhere I keep forgetting about?

16:01 shanmu: I am trying to setup emacs+slime+clojure. The latest version of swank-clojure and slime seem to have problems when going to source code from breakpoints

16:02 I am getting "error in process filter: cond: Elisp destructure-case failed: nil"

16:03 any help please?

16:04 I am getting "error in process filter: cond: Elisp destructure-case failed: nil" when pressing 'v' on any backtrace

16:04 raek: shanmu: do you use the snapshot version of slime? I have heard that it frequently breaks compability with swank-clojure...

16:04 shanmu: raek: yes its the snapshot version

16:04 hiredman: clojurebot: swank

16:04 clojurebot: swank is try the readme. seriously.

16:05 shanmu: raek: is there a stable repo for slime (for clojure) somewhere>

16:05 pjstadig: clojurebot: suddenly

16:05 clojurebot: CLABANGO!

16:05 raek: when I install the stuff I need on a new computer, I use the ELPA version of slime and slime-repl

16:05 shanmu: I think the ELPA one is the recommended

16:06 Dawgmatix: as someone who fought slime issues yesterday - i just want to say that the whole process is way more complicated than it needs to be

16:06 shanmu: raek: but I am using other lisps as well (CCL)

16:06 raek: whatever swank-clojure recommends is probably the best http://github.com/technomancy/swank-clojure

16:06 hiredman: Dawgmatix: complicated how? did you readme the readme and do what it says?

16:07 Dawgmatix: yes i did.

16:07 raek: where "best" means "what I got to work"

16:07 hiredman: the readme has 2 steps

16:07 well, maybe 3

16:08 Dawgmatix: the readme broke for me because a) i already had slime installed

16:08 hiredman: 13:12 < Dawgmatix> yes i did.

16:08 ordnungswidrig: why does slime reports version mismatch against swank-clojure?nil (slime) 20100404 (swank)?

16:08 AWizzArd: deftype is not for extending other classes right? Either genclass or proxy?

16:08 hiredman: well obviously you didn't do what the readme said then

16:09 Dawgmatix: b) even after nuking everything for some reason now i am in a state where if i use slime-connect, then clojure-mode doesnt launch

16:09 automatically for clj files

16:09 shanmu: Dawgmatix: I also have the same issue.. I have slime installed for other lisps

16:09 hiredman: ordnungswidrig: http://github.com/technomancy/durendal has an example of how to stop that warning

16:09 raek: AWizzArd: yes, I think so

16:10 hiredman: ordnungswidrig: you can also just use durendal

16:11 scottj: making a few small formatting/info display changes to clojars-web, anyone have any they really want?

16:11 ordnungswidrig: hiredman: reading the lisp…, autocompile sounds nice

16:12 Dawgmatix: i would suggest renaming slime / swank at this point. at least this will allow parallel installs of slime for lisp / clojure

16:12 ordnungswidrig: hiredman: but, "is it safe?"

16:12 shanmu: hiredman: I understand the a clean new install from ELPA would work, but is there someone here who can help with plugging in clojure/swank into an existing slime?

16:13 * raek looks forward to whatever nREPL becomes

16:13 zaphar_ps: shanmu: I created my own set of customizations for clojure swank: http://bitbucket.org/zaphar/dotfiles/src/tip/.emacs.d/conf.d/004-my-clojure.el

16:13 raek: shanmu: are you using slime-connect to connect to the swank server?

16:14 hiredman: ordnungswidrig: well, technomancy is also the maintainer (reluctantly) of swank-clojure

16:14 raek: the swank server is most often started by leiningen or cake nowadays

16:14 iirc, swank-clojure.el is deprecated

16:14 ordnungswidrig: hiredman: I know :-)

16:16 raek: shanmu: can you successfully connect to a swank-clojure server with your current setup?

16:19 shanmu: raek: I tried both ways...slime-connect and M-MxSlime

16:20 raek: yes I can connect fine to clojure swank

16:20 raek: when does it blow up?

16:20 shanmu: raek: when there is an exception and I try to go to source from the back trace (by pressing 'v') it blows up

16:24 * raek tries that on his own setup

16:24 raek: man. what a nice feature.

16:24 shanmu: zaphar_ps: your script looks promising

16:25 raek: exaclty... its a blessing!

16:25 if/when it works :)

16:28 raek: I know very little about slime except for some basic usage, so I'm afraid I'm out of advice...

16:33 zaphar_ps: shanmu: yeah It just launches the lein swank command and connects slime to it.

16:34 shanmu: zaphar_ps: but my current setup can connect to it... after an exception occurs, thats where the problem starts

16:35 zaphar_ps: exception?

16:35 clojurebot: http://www.mindview.net/Etc/Discussions/CheckedExceptions

16:36 shanmu: zaphar_ps: when there is an exception, I try to go to source code from backtrace

16:36 zaphar_ps, then I get an exception

16:37 zaphar_ps: error in process filter: cond: Elisp destructure-case failed: nil

16:39 zaphar_ps: shanmu: I'm afraid that one I can't help with :-(

16:40 shanmu: zaphar_ps, raek: thanks for the tips, anyways!

16:45 TeXnomancy: shanmu: if you have some ideas for improving compatibility with Clojure and CL at the same time that would be great. personally I don't use CL, so I am not in a position to know what's broken myself.

16:47 shanmu: TeXnomancy: I will try.. but I am not an expert in elisp/swank :(

16:47 nollidj: anyone here use jackson? i am using it now, and i want to deserialize into an abstract generic type. said type is from a third-party library, so i can't annotate it directly to get it deserialized properly, and i can't seem to specify a deserializer on a field of the type because it is generic

16:58 scottj: have an opinion on what clojars search results should look like? Tell me which of these three you prefer http://img84.imageshack.us/img84/2412/clojars.png

17:03 nollidj: oh, i typed into the wrong channel.

17:04 jcromartie: scottj: I like the leftmost

17:04 the bullet is distracting though

17:05 I'd even out the spacing between the lib name/version, description, and author/date lines

17:05 and then add a little more space between each lib's listing

17:05 and de-emphasize the description and author and date a touch (with shading)

17:06 scottj: shading?

17:07 raek: scottj: I like the first one

17:08 then the third

17:08 scottj: good those are both my designs :)

17:08 not that there's that much different about them

17:08 jcromartie: scottj: like a de-emphasized color

17:10 or actually a horizontal divider might be nice

17:10 when in doubt, copy RubyGems

17:13 amalloy: has anyone used SS's clojure-hadoop?

17:14 i'm picking a client to use atm, and i'd love to be able to justify a choice of clojure

17:14 ohpauleez: amalloy: I haven't, but I'm planning on it

17:15 with hadoop+lazy seqs+delays+ persistent ds, it makes a lot of sense

17:17 pjstadig: scottj: number 1

17:18 ohpauleez: scottj: #1 for sure

17:19 More often than not, I search clojars, then look on github for the project, check the commit dates and look at the project.clj file

17:20 If clojars could pull out :url from a project.clj (when present), that would be AWESOME

17:20 (totally icing on the cake though)

17:21 I also often when to sort the results based on date or version (no idea how you're generating these results), but that's about the only other thing I've wished for from clojars

17:25 scottj: ohpauleez: I've already implemented the url thing, but right now only showing it on the actual project page not search results

17:25 ohpauleez: ahh, awesome! (scottj)

17:25 "you're the man now dog"

17:26 scottj: if we added that and put brackets around the name/version there'd be no need for the project page and our clicks (and ad revenue) would go down :)

17:31 actually it was all pretty easy because ato had actually implemented it all he just didn't display it :)

17:31 rlb: I forget, does clojure have something like and-let?

17:32 scottj: rlb, what language is and-let from?

17:33 rlb: http://srfi.schemers.org/srfi-2/srfi-2.html

17:33 It's like an and, but it lets you name each intermediate value, for use in subsequent forms without recomputation.

17:34 scottj: not in clojure

17:41 KirinDave: I wonder if I could entice people in SF to coffee bar on sunday afternoon for a hackmoot

17:55 ohpauleez: KirinDave: Being in OR is rough sometimes, stuck between Bay Area Clojure people and Seajure

17:55 KirinDave: ohpauleez: I'm in SF

17:55 I thinkt here are many people here

17:55 I've got permission to start open sourcing this clothesline stuff

17:55 ohpauleez: yeah, a good size for sure

17:55 KirinDave: I wonder if people would be interesting in helping me get it ready for launch.

17:55 hiredman: technomancy gave a preview of his conj talk at seajure last night

17:55 ohpauleez: hiredman: How was it?

17:56 KirinDave: I'm more than happy to dig in

17:56 hiredman: it was good, I work with him so I get a lot of lein updates daily

17:56 ohpauleez: ahh, awesome

17:57 hiredman: he found some elisp that takes org-mode and turns into a slide show in a special purpose buffer

17:57 so the whole presentation is in emacs

17:58 ohpauleez: whoa, totally rad

17:59 KirinDave: Ugh

17:59 “Let's make this ugly, awkward, and nearly intractable.”

18:00 slyrus_: KirinDave: what's a hackmoot?

18:03 * technomancy suspects it's like an entmoot, but shorter

18:05 KirinDave: Yes. And it involves hacking ant talking about it.

18:05 slyrus_: I assume you mean "hacking and" not "hacking on ant?

18:06 KirinDave: correct. Sorry, typo

18:06 A little hectic getting drink 2 here.

18:06 slyrus_: i'm not around sunday, but i'd be up for a clojurian lunch in SF one of these days

18:07 lancepantz: technomancy: do you have a bar stool or anything you use for stand up programming/

18:08 technomancy: lancepantz: I have a recliner next to my standing desk I use a couple hours a day

18:08 lancepantz: that's a good idea

18:08 technomancy: gotta work your way up to it

18:09 lancepantz: ninjudd and i spent the morning raising our desks and stacking milk crates on top

18:09 Raynes: Why would you want to stand up while programming?

18:10 lancepantz: supposedly it helps combat add

18:10 KirinDave: Raynes: It can be better for your back.

18:10 lancepantz: Wait what?

18:10 Raynes: I don't think standing at a computer for hours on end would be very helpful to me.

18:11 ninjudd: ADD

18:11 technomancy: I think it helps combat lethargy

18:11 </anecdote>

18:12 lancepantz: supposedly, i don't have a source, i guess some conference talked about it

18:12 i like it though

18:12 although my feet do hurt

18:14 amalloy: until recently there was a guy at my office whose desk had automatic raising/lowering functionality. he did about half and half, said it was because sitting so much is bad for your back

18:14 lancepantz: yeah, i did find a bunch of links that it leads to heart disease somehow

18:14 KirinDave: lancepantz: Consider jellin'.

18:15 lancepantz: That NYT article?

18:15 ohpauleez: the artists and programmers here at PushButton and Playdom are 30% standing 70% sitting

18:15 KirinDave: They were sorta playing fast-and-loose with the study.

18:15 ohpauleez: with a lot of people converting to standing

18:15 lancepantz: KirinDave: that's the one

18:15 ninjudd: my mom is a school counselor, and she was to a conference yesterday on strategies for dealing with ADD. one of the main ones they talked about was using a standing desk. which is funny, because i just reread the technomancy's take interview before i talked to her...

18:16 ohpauleez: ninjudd: link?

18:16 found it

18:16 lancepantz: i think ideally i would like to raise and lower

18:16 ohpauleez: really? i want to see it

18:17 ohpauleez: http://blog.fogus.me/2010/06/28/take-8-phil-hagelberg/

18:17 lancepantz: oh, i thought you meant to ADD thing

18:17 KirinDave: I wonder if I will ever get on take 8 now that I am a full time clojure dev for a hot startup.

18:17 Probably not.

18:17 If they ask, the jig will be up regarding xkcde. :\

18:18 lancepantz: how many full time clojure developers do you think there are in the world?

18:18 100 would be my guess

18:18 KirinDave: Probably a few hundred.

18:19 ninjudd: they didn't mention keyboard pants at the conference though...

18:19 lpetit: Raynes: it probably also depends a lot on personal issues, and I come to think more and more while time passes on, the age you are. I certainly wouldn't have understood this at, say 20, while now, at 37, it does have some appeal to me (especially after the lunch :-) )

18:21 lancepantz: i can kinda jump and sway while coding, i dig that

18:22 Derander: keyboard pants!

18:23 that's brilliant!

18:34 lpetit: Derander: yeah. People in my office already start to try to escape from me when I start talking about clojure for too long. I'm pretty sure that by wearing keyboard pants I'll definitely be placed in my own office :-p

18:34 But really I like the concept ! :)

18:34 Derander: I bet you could actually make them

18:38 Raynes: lpetit: Fair enough. I guess I just don't understand enough about human anatomy and they're life styles to totally understand why standing rather than sitting would be beneficial in any way.

18:39 In good don't-knock-it-until-you've-tried-it style, I'll save my complaints until I've actually seen it in action.

18:43 carkh: huu lau are you there ?

18:44 lpetit: Raynes: while by no means being an expert in this field myself, I've read things here and there : staying in the same position for years is not good for the back. So having several hours in a standing position, and several hours in your seat is a good compromise. And also, there are some particular problems that can arise when sitting too long a day for too many years : hemorroids, to name them (too much pressure on your back, and some

18:46 of course, not a thing a young man thinks about every day he wakes up, and that's totally normal and understandable ! (Oh man, is it me saying this ?)

18:52 jarpiain: rlb: (domonad maybe-m ...) is like and-let except it doesn't short-circuit on false (just nil)

18:53 KirinDave: Oh god bless ring.

20:34 ssideris_: how do you guys handle the situation when you are hacking on a project and at the same time at it's dependencies

20:34 is there a source dependency feature in leiningen?

20:34 hiredman: yes

20:34 checkouts

20:35 ssideris: ah nice

20:36 thanks, I think this will help me organise things much more cleanly

20:37 hiredman: I am not sure where it's documented, but I believe if you create a directory named checkouts and put checkouts of the dependencies in there, the src dir of the dependency will end up on the classpath

20:38 ssideris: yeah it seems that this is was'ts implied here: http://github.com/technomancy/leiningen/blob/master/README.md

20:38 I'll give it a go

21:59 bhenry: clojurebot: where is everyone?

21:59 clojurebot: Excuse me?

21:59 bhenry: clojurebot: you heard me

21:59 clojurebot: bhenry: I'm just giving you a second chance

22:00 Raynes: sexpbot: whatis UGT

22:00 sexpbot: UGT = Universal Greeting Time: http://www.total-knowledge.com/~ilya/mips/ugt.html

22:09 dnolen: Objected Oriented Programming in JavaScript and Clojure, http://dosync.posterous.com/30036733

22:10 feedback appreciated

22:18 jeff__: good stuff. i'll look forward to the next post.

22:19 dnolen: jeff__: thx for the feedback

22:21 jeff__: i'm fairly new to the clojure community. i'll add your blog to my "watch list". btw, how do you syntax highlight clojure source in html?

22:22 Raynes: dnolen: I enjoyed it.

22:22 dnolen: Raynes: thx!

22:22 Raynes: dnolen: I'll be watching for the next installment.

22:22 jeff__: I don't know what you're looking at, but it doesn't look like he highlighted it at all.

22:23 jeff__: sorry, i was looking at other entries on his blog: http://dosync.posterous.com/

22:24 Raynes: jeff__: Looks like he is embedding gists.

22:24 $google embedding gists

22:24 sexpbot: First out of 128000 results is: Embedded Gists - GitHub

22:24 http://github.com/blog/122-embedded-gists

22:24 Raynes: Headshot!

22:25 dnolen: jeff__: yeah just embedded gists, you can actually just past gist links and posterous embeds them intelligently for you. pretty sweet.

22:25 jeff__: nice. that may come in handy.

22:25 dnolen: s/past/paste

22:26 Raynes: dnolen: Really? I should have considered posterous a bit more, I suppose.

22:26 dnolen: Raynes: it's not totally ideal but it's much quicker than me waiting for me to write my Clojure blog engine ;)

22:27 Raynes: I'm running wordpress at the moment.

22:32 jeff__: i have a question about multimethods. is it possible to have a multi-arg multimethod? seems like all examples i find are single arg.

22:34 for example, this doesn't work:

22:34 (defmulti testfun (fn [a b] (keyword (str a))))

22:34 (defmethod testfun :test [a b] (println a b))

22:35 hiredman: yes it does

22:35 jeff__: then i'm using it wrong.

22:35 (testfun 'a 'b)

22:35 yields

22:35 java.lang.IllegalArgumentException: No method in multimethod 'testfun' for dispatch value: :a (repl-1:11)

22:36 oops

22:36 hiredman:

22:36 jeff__: sorry

22:36 :)

22:37 i obviously need another coffee.

22:40 scottj: I think your multi and (defmulti testfun keyword) are equivalent

22:41 I mean (comp keyword str)

22:41 hiredman: no

22:42 scottj: how so? to me it looks like the dispatch doesn't have to have take the same number of args as the method

22:43 hiredman: it doesn't

22:43 but his dispatch function takes two args and returns the keyword of the first arg

22:44 (defmulti foo keyword) will pass the two args to keyword

22:44 Raynes: And (comp keyword str) would do the same thing.

22:44 Making for a spectacular explosion.

22:44 scottj: ,(do (defmulti testfun (comp keyword str)) (defmethod testfun :test [a b] (println a b)) (testfun "test" "bar"))

22:44 clojurebot: DENIED

22:45 Raynes: You'd think people would stop trying to def things after the 10 thousandth try.

22:45 :p

22:45 hiredman: str will take the two args and turn them into a single string and keyword will turn it into a keyword

22:46 which is not the same thing as taking two args, return the keyword of the str of the first arg

22:46 scottj: ok I see now, I would have thought that too but my repl was actually giving me wrong results there must be some basic practice with redefining multimethods I'm not aware of

22:47 what I pasted was actually working because I'd first defined the multi with the valid dispatch and redefining it apparently isn't picking up the change

22:47 Raynes: I was implying that it would do the same thing (as (defmulti foo keyword)) of which I was also wrong. Still, that could cause a less spectacular but more sinister explosion if you weren't expecting that.

22:47 _ato: ah yeah... that changed in 1.2

22:47 jeff__: i wonder if that's what tripped me up. i was doing this in the repl last night and kept getting errors like this: Wrong number of args (1) passed to: user$eval83$fn (repl-1:12)

22:48 scottj: ah man I need to stop looking at IRC I missed a touchdown!

22:48 jeff__: i just started a new repl, typed up that simple example and didn't even notice that it was working this. i think the evolution of my multimethod in the repl yesterday was the issue. it works in a fresh repl.

22:49 Wrong number of args (1) passed to: user$eval83$fn (repl-1:12)

22:49 sorry for the duplicate paste of the error

23:40 gerryxiao: hello

23:40 what's fn?

23:41 Fn is empty interface

23:41 ,(fn? :meta)

23:41 clojurebot: false

23:42 gerryxiao: ,(ifn? :key)

23:42 clojurebot: true

23:42 gerryxiao: ,(fn? +)

23:42 clojurebot: true

23:43 dnolen: (doc fn?)

23:43 clojurebot: "([x]); Returns true if x implements Fn, i.e. is an object created via fn."

23:43 dnolen: gerryxiao: ^

23:44 gerryxiao: dnolen: i know, but what is the meaning of Fn interface?

23:45 dnolen: gerryxiao: IFn, the interface require for something to act as a Clojure function

23:45 s/require/required

23:46 in any case you would use fn? normally to check if something is a function

23:46 gerryxiao: yes, iFn is for function,but Fn is for What?

23:47 amalloy: gerryxiao: Fn is functions defined with (fn). IFn is anything that is callable as a function, eg (:foo my-map)

23:47 gerryxiao: hmm

23:48 ,(fn? (fn []))

23:48 clojurebot: true

23:48 dnolen: ,(fn? #())

23:48 clojurebot: true

23:48 dnolen: ,(fn? {})

23:48 clojurebot: false

23:49 gerryxiao: ,(fn? :a)

23:49 clojurebot: false

23:49 dnolen: (:a {:a 'foo})

23:49 amalloy: ,(ifn? a)

23:49 clojurebot: java.lang.Exception: Unable to resolve symbol: a in this context

23:49 dnolen: ,(:a {:a 'foo})

23:49 clojurebot: foo

23:49 amalloy: ,(ifn? :a)

23:49 clojurebot: true

23:50 gerryxiao: i'm looking at clojure src, find that Fn.java just define one empty interface

23:51 a tag ?

23:58 scottj: looks like it

Logging service provided by n01se.net