#clojure log - Jul 20 2009

The Joy of Clojure
Main Clojure site
Google Group
List of all logged dates

3:17 yason: Vkthkstt!

3:28 dang

3:28 writing passwords blinded with a dead display is bad :)

6:09 Fossi: hi

6:10 i just stumbled over another piece of lisp-1 fun again: having a struct named foo and normally calling your variables foo as well

6:11 make a typo in the parameter list and it won't even fail on you, only retun nil for a value of the struct

6:11 so, note to self: name structs struct-foo

6:29 AWizzArd: Fossi: yes, I also stumpled over that some months ago. But I did not decide yet if this has more to do with Clojure being Lisp-1 or being dynamically typed.

6:29 Why would one name variables after their type in statically typed langs?

6:51 cky: Fossi: I can't comment on Clojure convention, because I'm new to it, but in Scheme, record type objects are usually bracketed with angle brackets.

6:51 e.g., <foo>

6:55 AWizzArd: cky: in Clojure it would not be typical to put names of structures into angle brackets.

6:57 cemerick: although I smile a little every time I'm reminded that clojure is perfectly happy with symbols like <foo>...forgive me for my java/python background :-)

6:58 AWizzArd: It could be the convention for a project. If it is used consistently then I wouldn't mind.

6:59 For example, we use fnparse (thanks for the tip hiredman) and all rules begin with a !.

7:05 Chousuke: cemerick: I think clojure is fine with any unicode character :P

7:06 cemerick: yeah, almost :-)

7:06 Chousuke: actually, it even accepts ' ' as a symbol character

7:06 cemerick: right, just not through the reader

7:06 Chousuke: yes it does.

7:07 that's a double-width space :)

7:07 cemerick: ah, couldn't tell that in irc :-)

7:07 Chousuke: user=> (def funky symbol 5)

7:07 #'user/funky symbol

7:07 user=> funky symbol

7:07 5

7:07 :P

7:07 I think that's a bug, though.

7:08 AWizzArd: The biggest problem with that will be actually *typing* these unicode chars.

7:08 Chousuke: it's not too difficult with an input method editor :P

7:10 Fossi: worse i've seen was sql which had right-tp-left markers in it

7:11 *t

7:11 init was like: insert into foo values (";("somestring

7:12 makes you go 'wtf?'

7:12 AWizzArd: (def √² 1.4142135)

7:16 cemerick: AWizzArd: if you're on a mac, then typing such things is incredibly simple (e.g. no alt+... codes to remember)

7:17 AWizzArd: i see

7:17 cemerick: √ is just option-v for example -- there are even some mac keyboards that have the 'special' characters printed on the front face of the key

7:17 Fossi: only that all the other letters are on a weird comb

7:17 key combo

7:20 cemerick: ah, this is the one I was thinking of: http://matias.ca/tactilepro

7:20 Chousuke: Fossi: job security? :p

7:21 Fossi: and the infallable apple-q

7:21 Chousuke: my mac layout has no quick key for lambda :(

7:21 opt-l gives fi

7:21 rys: heh, the unicode chars in my term while we discuss this made cemerick's 2nd to last comment appear to come from me

7:21 cemerick: Chousuke: that's easy to change

7:21 rys: (for me at least)

7:22 * Fossi has a us-intl/dvorak keyboard by typematrix

7:22 Fossi: so my keylayout is totally different anyway :)

7:22 AWizzArd: I am using the NEO layout (not qwerty), and with it I also have easy access to greek symbols and mathmatical symbols and programmers symbols.

7:23 Fossi: yeah, neo is nice as well

7:23 Chousuke: but does it support APL?

7:23 AWizzArd: Fossi: it is following the same ideas like dvorak, but more specialized on german, while dvorak is a bit better for english.

7:24 * cky uses Dvorak and loves it, but yay for German-specialised layouts too. :-)

7:24 * Fossi is german ;)

7:25 AWizzArd: Ah ok, that's why you know neo

7:25 cky: Hehehehe. :-)

7:25 Fossi: i decided to learn dvorak though because it's better supported by hardware manufacturers

7:25 and the difference is not as big with a nice layout like this keyboard has

7:26 AWizzArd: Well, my keyboard also has good support: http://www.daskeyboard.com/

7:26 Fossi: if you haven't you should check it out. best ever :)

7:26 Chousuke: there's a Finnish optimised layout called DAS. It has some characters that are officially used in Finnish for certain loanwords but which no-one really uses :P

7:26 AWizzArd: As it does not show any printings on the keys... it fits perfectly.

7:26 Fossi: AWizzArd: well, i would annoy my collegues to death with that

7:26 pair programming gets kinda hard :)

7:26 Chousuke: partly because they're impossible to type with Finnish qwerty.

7:26 AWizzArd: I ask my workmates to type blind *g*

7:26 cky: AWizzArd: That's the real way to type! :-)

7:27 cemerick: hrm, we should do a poll of clojure programmers' nationality

7:27 Fossi: actually i find it easier to type dvorak on daskeyboard then on a qwerty

7:27 cky: My keyboard still has qwerty keycaps printed on, so it doubly confuses my workmates, who cannot hunt-and-peck. :-P

7:27 AWizzArd: very good

7:28 rys: I can't remember the last time I looked at my keyboard to type anything

7:28 cky: rys: Exactly!!

7:28 rys: The keys could be blank

7:29 cky: Whereas, back when I was learning Dvorak, I shifted the keys on my laptop at the time, with the severe downside that the home keys aren't where they are supposed to be. :-(

7:29 * Fossi did the same at home

7:29 Fossi: now it's much better

7:29 * cky uses the home keys _way_ too much to work without them.

7:30 cky: By home keys, I mean the bumpy keys where U and H are in Dvorak. :-)

7:30 Fossi: huh? i though you meant physically

7:30 ah, ok

7:30 never had a problem with that really

7:30 cky: Well, without the bumpy keys, my hands tend to drift too much.

7:31 Fossi: then again a divider in the middle helps a whole lot

7:31 cky: ...that is true.

7:31 Fossi: extra points for it being the enter key :D

7:31 cky: :-)

7:32 * Fossi would never buy a 'normal' layout keyboard anymore

7:32 cky: Well, here in the US it's not easy to find a Dvorak keyboard, but that doesn't mean I will use any layout other than Dvorak (if I ever learn German I'll learn NEO too :-P).

7:33 Fossi: i ordered mine from the us actually

7:33 cky: *nods* Yes, I suppose I should find some suppliers.

7:33 Fossi: it's not like they have them in store

7:33 cky: *nods*

7:33 Fossi: as said: reallt check out typematrix

7:33 it's *awesome*

7:34 cky: Thanks! :-)

7:38 AWizzArd: Fossi: try this one please: http://schnell-schreiben.de/stv2/

7:38 fsm: Hi, I have made a raytracer as a project to learn cloure. Here is a pic. http://solardriftwood.com/aa.png

7:38 AWizzArd: Or the english version (measuring typing speed): http://speedtest.10-fast-fingers.com/

7:38 fsm: clojure*

7:39 AWizzArd: fsm: did you write the RT from scratch? Or used Harrops version?

7:39 fsm: I wrote it from scratch

7:39 I do not know what Harrops refers to

7:39 cemerick: fsm: very nice! :-D

7:40 AWizzArd: fsm: good good

7:40 fsm: Since then it does texture mapping and bump mapping, but I don't have a nice image of that yet

7:40 But, I have a performance issue relating to boxed primitives

7:41 I have profiled and most of the bottlenecks are in operations of vectors of doubles [x y z] and I am trying to find the fastest representation

7:41 When I converted to using make-array and aget, instead of [] and nth, I lost 30% performance

7:41 AWizzArd: you probably introduced reflection then

7:42 fsm: When I profile, I see millions of calls to Number.ops methods where I would hope for native arithmetic

7:42 I turned on reflection warnings, there's no reflection

7:42 The other weird thing, is I tried making some vector math into macros instead of functions, that caused a 20% slowdown

7:42 I guess the java hotspot optimizer is very clever if you call the same function a lot

7:43 cemerick: fsm: perhaps the unchecked math fns are appropriate?

7:43 ,(doc unchecked-add)

7:43 clojurebot: "([x y]); Returns the sum of x and y, both int or long. Note - uses a primitive operator subject to overflow."

7:43 cemerick: etc....

7:43 fsm: I saw that, but i am using floating point unfortunately

7:43 I tried adding type hints also, but that seemed to cause a performance hit, i guess more redundant boxing/unboxing was introduced

7:44 cark: do you coerce your floats ?

7:44 fsm: Say I have two vectors [1.0 2.0 3.0] and [4.0 5.0 6.0] and I want to add them together, is there a faster way to access the variables than nth

7:44 I tried coercing the floats, that caused a small performance drop

7:45 I am perplexed that using make-array to make a double array is so slow

7:45 Chousuke: it's slow because it does a full copy :/

7:46 cemerick: that, or the array is java.lang.Float[] instead of float[]

7:46 fsm: I was using make-array Double/TYPE which refers to double

7:46 Unless I am mistaken

7:46 cemerick: no, you're not :-)

7:46 cark: nope this is correct

7:47 fsm: So, even when I use macros and everything expands out to just basic arithmetic and aget, or to using nth, millions of calls to Numbers.obs.* show up

7:47 ops

7:47 I mean

7:47 Chousuke: hmm

7:47 cark: and you're using the -server flag when starting java ?

7:48 Chousuke: are you passing more than two arguments to +?

7:48 fsm: I have nested my operator +, that caused a big speed boost

7:48 I have not been using -server flag

7:48 cark: ah, this should help a lot

7:48 fsm: OK I will research that

7:48 cemerick: yeah, the client compiler is completely different

7:49 hrm, the only thing preventing support for unchecked ops in Numbers are more overloads :-)

7:49 fsm: Back in a minute, going to do a performance check

7:50 cark: you need to take the warm up into account, compilatin only occurs after quite a few runs of each function

7:50 fsm: Yep, I have written a timer macro that times only the actual executions

7:51 Anyway, I must say that writing this project is a very rewarding experience

7:52 The tools of functional programming are like having a CNC machine compared to hammer and chisel

7:52 cemerick: fsm: In general, you need to let whatever code paths you're benchmarking "warm up" at least a half-dozen times before most of the compiler optimizations have been baked in.

7:53 fsm: Yes, with raytracing the same routine is called for each pixel on the screen, causing a great many runs

7:54 Chousuke: It sounds to me that dealing with arrays in Clojure is somehow unnecessarily difficult :/

7:57 fsm: OK, using -server results in a consistent 3% speed boost

7:58 Anyway, I will clean up the code and post it somewhere in the next day or two

7:59 Meanwhile my algorithmically equivalent Objective-C version was on the order of 10x faster

8:00 But Clojure is an excellent project, I look forward to solving this problem

8:01 Chousuke: Is there any easy way of determining where boxing happens? :/

8:02 cemerick: essentially at any function boundary, aside from unchecked-* ops AFAIK

8:02 fsm: Does that include aget

8:02 AWizzArd: fsm: maybe you can inline here and there

8:03 ,(doc definline)

8:03 clojurebot: "([name & decl]); Experimental - like defmacro, except defines a named function whose body is the expansion, calls to which may be expanded inline as if it were a macro. Cannot be used with variadic (&) args."

8:03 Chousuke: cemerick: can you tag functions to tell clojure that they return primitive values? :/

8:03 fsm: I tried converting everything to macros already, unfortunately that reduced performance, I think by reducing the effectiveness of the hotspot optimizer

8:04 cemerick: not that I know of, no

8:04 fsm: But if aget is boxing the value it returns, that could be the crux of the problem i am having

8:05 Chousuke: fsm: are you doing (let [foo (double (aget ...))])? or something

8:05 cemerick: fsm: macros don't necessarily impact the optimizer -- that is determined by the code that your macros emit. So, unless the code they're emitting is more efficient than your normal fns, the macro usage will be slower.

8:05 Chousuke: cemerick: expanding macros bloats the code though.

8:06 fsm: I am not using that let syntax, because it would mean I have to unpack and repack my vectors to do operations on them.

8:06 cemerick: I think the bottom line issue here is that there's no unchecked math for floating points / doubles

8:06 fsm: Which is not very lispy

8:06 AWizzArd: fsm: do you have a short and simple example where a macro slowed down your code?

8:06 Chousuke: fsm: IIRC the optimisation tips page recommends that all primitive "casts" be done in a let, though :/

8:06 AWizzArd: The funny thing is that in the compiled version that you execute no macros exist anymore.

8:07 fsm: (defn dot [v1 v2] (+ (+ (* (vx v1) (vx v2)) (* (vy v1) (vy v2))) (* (vz v1) (vz v2)))))

8:07 ^^ rewriting that to be a macro slowed down my code

8:08 cemerick: Chousuke: the perf of the fns emitted by the macro is way more important than the 'size' of what it's emitting, though.

8:08 fsm: the vx/vy/vz are wrappers for either nth or aget - vx v1 = (aget v1 0)

8:08 and when I make vx/vy/vz into macros too, even more performance drop, but only 5% or so

8:08 Chousuke: cemerick: well, more code equals less effective use of cache.

8:08 cemerick: which may make a huge difference, if you're unlucky.

8:08 fsm: I tried representing my vectors as [1.0 2.0 3.0] and as make-array

8:09 cemerick: Chousuke: true, true.

8:09 fsm: using make-array/aget instead of [] and nth caused an overall 30% speed drop

8:09 * cemerick is still waiting for the sufficiently-smart compiler.

8:09 Chousuke: fsm: that dot function you pasted is not doing primitive math, though :/

8:10 cemerick: Chousuke: which isn't avail. for floats/doubles

8:10 fsm: I wrote a version that was (defn dot [#^doubles v1 #^doubles v2]

8:10 and that was slower

8:11 Chousuke: cemerick: hm :/

8:11 fsm: if i define vx/vy/vz to be macros that resolve to (aget v1 n), I get bad performance but it should be operating on primitives

8:11 but in the profiler i see Number.ops being called

8:12 Chousuke: but if what cemerick says is true and primitive ops are not available for doubles, then boxing is inevitable :/

8:12 but, raaah

8:12 fsm: oh, I misread that

8:12 Chousuke: GHC took over an HOUR to compile and then it dared fail at the end.

8:12 cemerick: well, there's no overloads for the unchecked math in Numbers *shrug*

8:13 there may be a good reason for that that I'm not thinking of, I suppose

8:14 fsm: The java_interop page shows an example working on a java array of primitive floats, supposedly

8:14 Anyway, thanks to everyone

8:15 I will post the code tomorrow and if anyone wants to experiment on it, they are welcome

8:24 Hmm in RT.java, aget takes an array of primitives and returns a primitive

8:24 and in core.clj, aget is defined as {:inline (fn [a i] `(. clojure.lang.RT (aget ~a ~i)))

8:24 does that call into clojure.lang.RT cause the return value to get boxed?

8:25 The docs suggest that values passed to a function are boxed, so perhaps the return value is too?

8:25 Chouser: generally, yes. but macros and :inline are an exception.

8:26 Chousuke: macros don't get the actual value anyway :)

8:26 Chouser: right. both macros and :inline expand at compile time and thus the call "disappears"

8:27 in this case what's left is a Java interop call which is not necessarily boxes.

8:27 boxed

8:28 fsm: Anyway it be super fast, it boils down to a static function that does only return xs[i];

8:28 should be*

8:29 It's getting too late for abstract thinking, time for sleep.

8:29 Thanks to everyone for your help.

8:38 Fossi: is it bad to (apply str (rest word))?

8:38 s/word/some-string/

8:38 AWizzArd: not if you need the result

8:39 Chousuke: ,(doc subs)

8:39 clojurebot: "([s start] [s start end]); Returns the substring of s beginning at start inclusive, and ending at end (defaults to length of string), exclusive."

8:39 Chousuke: probably a fair bit faster :P

8:39 Fossi: that kinda was the question, sorry

8:41 Chousuke: though sometimes you do need to work with char sequences. str uses a StringBuilder internally so at least it's faster than naive string concatenation in java :)

8:42 Fossi: so (str (Character/toUpperCase (first "foo")) (subs "foo" 1)) should be better then (apply str (Character/toUpperCase (first "foo")) (rest "foo"))

8:42 *than

8:42 my engllish is so borked today

8:43 Chouser: I'd recommend shooting for clarity and maintainability first -- worry about speed only if you have to.

8:44 Fossi: well, imho they are kinda both readable, i do not wonder about performance so much, but rather about which would be considered better style

8:47 Chousuke: hmh

8:48 I wonder which of my installed ports depends on x11 :P

8:49 Chouser: I think 'str' is less complex than 'apply str', and you don't have to think about whether the use of 'rest' vs. 'next' is important, etc. I'd vote for the first form.

8:50 Chousuke: kind of annoying, trying to upgrade ports and then something tries to install all of X11

8:51 * Chouser hugs ubuntu.

9:11 * Fossi hugs gentoo

9:11 Fossi: although they are kinda 'slacking off' lately

9:13 Jomyoot: do you guys prefer light on dark or dark on light?

9:13 jdz: dark on light is usually easier on the eyes

9:14 Jomyoot: why do the textmate geeks use light on dark then?

9:14 weissj: can someone tell me the easiest way to "un"-lazy a sequence? the laziness is causing this not to work: (map await (map #(send-off (agent %) do-one-file targetDir) mylist)))

9:14 jdz: they are geeks?

9:14 Jomyoot: I followed their convention for years

9:14 kiddies

9:14 what u call them

9:14 jdz: well, i meant that as an answer

9:15 cemerick: weissj: you probably want doall

9:15 weissj: ,(doc doall)

9:15 clojurebot: "([coll] [n coll]); When lazy sequences are produced via functions that have side effects, any effects other than those needed to produce the first element in the seq do not occur until the seq is consumed. doall can be used to force any effects. Walks through the successive nexts of the seq, retains the head and returns it, thus causing the entire seq to reside in memory at one time."

9:15 chessguy_work: the first two test functions on http://blog.objectmentor.com/articles/2009/07/19/uncle-bob-jsps-learning-clojure are basically reduce and map, respectively, right?

9:15 cemerick: or maybe dorun, if you don't care about the results of the map

9:16 weissj: cemerick: i care about getting the entire seq from the inner map

9:17 in other words i want the entire inner map calculated before any awaits are done

9:17 chessguy_work: ,(reduce + [1 2 3])

9:17 clojurebot: 6

9:17 * Fossi loves ligth on dark at night. at work it's dark on light though

9:18 jdz: good thing i can easily change the brightness of the screen

9:18 and my laptop actually does it automatically

9:19 Fossi: i had a script a while back that switched between light and dark mode :)

9:19 * Fossi wonders if xrandr invert would do the trick equally well

9:20 chessguy_work: i think it's something like (defn roll-list [game list] (reduce roll (conj game list))) but i don't have an interpreter in front of me

9:20 cemerick: weissj: then I think it'd be (map await (doall (map #(send-off ...))))

9:20 I've never used agents though, so that's just a guess, really

9:20 weissj: cemerick: i'll give it a try, thanks!

9:21 chessguy_work: (crickets chirping)

9:24 ,(let [x 3] x)

9:24 clojurebot: 3

9:25 angerman: how do i cast a class to a different interface?

9:25 chessguy_work: ,(let [roll (fn [g p] (conj g p))] (roll [2] 3)

9:25 clojurebot: EOF while reading

9:25 angerman: e.g. JRubyScriptEngine to Invocable

9:25 chessguy_work: ,(let [roll (fn [g p] (conj g p))] (roll [2] 3))

9:25 clojurebot: [2 3]

9:27 * angerman kind of fails to see how casting is done in clojure

9:27 Chouser: weissj: when creating/sending/awaiting an agent all at once like that, you might prefer 'future'

9:27 chessguy_work: ,(let [roll (fn [g p] (conj g p)), roll-list (fn [g l] (reduce roll (conj g l)))] (roll-list [] [1 2 3]))

9:27 clojurebot: [1 2 3]

9:27 angerman: while in java i"d just do (Invocable)variable ...

9:28 Chouser: angerman: you generally don't need to cast. what are you trying to do?

9:28 cemerick: angerman: why are you looking to perform a cast? Such a thing is essentially not present in clojure, as it's dynamically typed.

9:28 weissj: Chouser: i will look into 'future', thanks :)

9:28 angerman: Chouser, cemerick that's what I expected

9:29 I'm trying to call invokeFunction on a jruby Script Engine

9:29 cemerick: angerman: if you're looking to invoke a method on an object, and you want to avoid reflection, you can type-hint the object

9:30 (.invokeFunction script-engine-obj ...more-args...) will do it

9:30 (.invokeFunction #^ JRubyScriptEngine script-engine-obj ...more-args...) if you want to avoid reflection

9:30 bah: (.invokeFunction #^JRubyScriptEngine script-engine-obj ...more-args...)

9:32 chessguy_work: how would i apply a function to something n times?

9:33 Chouser: ,(map inc (repeat 5 2))

9:33 clojurebot: (3 3 3 3 3)

9:33 chessguy_work: ermm, sorry, i'd want 10 there

9:33 though i'd want to pass in the 0 too

9:34 Chouser: I'm afraid I don't understand the question at all.

9:34 Chousuke: iterate?

9:34 chessguy_work: sorry, i'm trying to get rid of a for-like loop

9:34 (defn roll-many [game n pins]

9:34 (loop [i n g game]

9:34 (if (zero? i)

9:34 g

9:34 (recur (dec i) (roll g pins)))))

9:35 Chousuke: iterate returns a seq though

9:35 ,(drop 9 (take 10 (iterate inc 0)))

9:35 clojurebot: (9)

9:35 chessguy_work: yeah this is more like a reduce over a range, where the index is ignored

9:35 angerman: cemerick: hmm it seems the arguments are my problem

9:36 cemerick: angerman: you're getting an error?

9:36 angerman: it expects a java.lang.Object and I was giving it a String

9:36 Caused by: java.lang.ClassCastException: java.lang.String cannot be cast to [Ljava.lang.Object

9:36 cemerick: it's expecting an Object[]

9:36 Chouser: maybe varargs?

9:36 cemerick: (probably the arguments to the jruby fn?)

9:36 Chouser: ooh

9:36 chessguy_work: ,(nth (iterate inc 0) 9)

9:36 clojurebot: 9

9:37 angerman: cemerick: yes

9:37 cemerick: angerman: in that case, (into-array ["some" "args"]) might be what you want

9:37 angerman: cemerick: using a vector doesn't resolve it :/

9:37 hmm into-array

9:37 cemerick: vectors aren't arrays

9:37 * angerman is still learning

9:37 cemerick: angerman: :-)

9:37 Chouser: actually, if it wants Object[], try (to-array ["some" "args"])

9:38 cemerick: yeah, I can never remember in what contexts a String[] isn't useful as an Object[]

9:38 rhickey_: chessguy_work: (reduce roll game (range 1 n))

9:39 jdz: rhickey_: nice one!

9:39 rhickey_: oh, you're not using n

9:39 cemerick: the fact that uncle bob is interested in clojure is interesting

9:39 jdz: a bit confusing, though

9:39 chessguy_work: rhickey_, yeah, i don't want the index

9:41 ,(let [roll (fn [g p] (conj g p)), roll-many (fn [g n p] (nth (iterate (roll g p)) n)] (roll-many [] 20 0))

9:41 clojurebot: Unmatched delimiter: ]

9:41 chessguy_work: ,(let [roll (fn [g p] (conj g p)), roll-many (fn [g n p] (nth (iterate (roll g p)) n))] (roll-many [] 20 0))

9:41 clojurebot: java.lang.IllegalArgumentException: Wrong number of args passed to: core$iterate

9:43 rhickey_: (nth (iterate #(roll % pins) game) n)

9:44 the best thing to do with that bowling game is to step far away from that original logic

9:45 chessguy_work: ,(let [roll (fn [g p] (conj g p)), roll-many (fn [g n p] (nth (iterate #(roll % p) g) n))] (roll-many [] 20 0))

9:45 clojurebot: [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]

9:45 chessguy_work: shazam :)

9:46 rhickey_: the 'kata' is almost a poster child for what I think is wrong with TDD and OO today

9:46 http://butunclebob.com/ArticleS.UncleBob.TheBowlingGameKata

9:48 Write a class named “Game” that has two methods

9:48 roll(pins : int) is called each time the player rolls a ball. The argument is the number of pins knocked down.

9:48 score() int is called only at the very end of the game. It returns the total score for that game.

9:48 drewr: ugh, he distributes it in a ppt

9:49 rhickey_: the roll method returns void!!, so right away we have a stateful object. The score method is only valid at certain times.

9:49 chessguy_work: rhickey_, not in the clojure version

9:49 :)

9:49 rhickey_: So with this awful premise, we start TDD, our system for robust and responsible programming - yikes

9:50 chessguy_work: yes, the CLojure version should be much different. It's just the point I;ve tried to make about OO and TDD, people are starting with faulty premises and never questioning them

9:51 3rd slie - "Clearly we need the Game class"

9:51 slide

9:51 angerman: cemerick: thanks.

9:51 chessguy_work: i think TDD and functional programming are approaching the problem from different directions and can meet in the middle

9:51 angerman: cemerick: whee, I can now call maruku (ruby) from clojure ... still need to work on the return type though :/

9:51 chessguy_work: (the problem of code quality in a stateful reality)

9:52 * Chouser closes the .ppt

9:53 * jdz never opened the .ppt

9:53 rhickey_: chessguy_work: well, there's reality and our representation of it in our programs - the latter need not be stateful to the degree it normally is in OO

9:54 chessguy_work: rhickey_, agreed, but there need not be the lack of testing of functional programs that there is either

9:55 AWizzArd: typically it is easier to test functional programs

9:55 chessguy_work: but it's done a lot less, i think

9:56 rhickey_: chessguy_work: sure, but my key point is the unquestioned presumption of stateful objects by TDD advocates using traditional OO langs - building on sand

9:56 but feeling good about it due to the tests

9:57 chessguy_work: well, i guess for a non-parallel project it's ok

9:57 rhickey_: chessguy_work: seriously not ok

9:58 I think there a lot of Clojure users now who'll testify to the increased robustness of their non-concurrent programs

9:58 chessguy_work: anyway, i think TDD leads to a lot less stateful design in general than non-TDD

9:59 rhickey_: I'd hate for people to think that FP is only for concurrency

9:59 AWizzArd: I absolutely test nearly everything.

10:00 It is not responsible to not write tests (with some few exceptions).

10:00 chessguy_work: not _only_ but that's certainly a place where it thrives

10:00 Chouser: I still have only dabbled in Clojure's concurrency support.

10:00 AWizzArd: When writing professional code for applications one wants to sell means one should test-is it.

10:01 drewr: my programs are much better by not even thinking about state until forced to

10:01 ...which is usually much later

10:01 chessguy_work: rhickey_, part of the premise of TDD is that it leads to code that's easier to test (because it's less stateful, though that's not usually stated) and, thus, more flexible

10:02 rhickey_: AWizzArd: I'm not trying to be anti-testing, this just seemed a good example of a conundrum I have with people advocating TDD and not questioning mutable OO - discipline on top of spaghetti

10:03 chessguy_work: i agree that it could be made more explicit, but it's there

10:03 rhickey_: chessguy_work: I haven't seen any connection whatsoever between the two - lots of examples just like this one - pokeable things with tests that poke them

10:04 and mock pokeable things of course

10:04 Chouser: /topic discipline on top of spaghetti ... seriously not ok

10:05 rhickey_: the only thing I want on spaghetti is a good red sauce :)

10:05 jackdempsey: hehe

10:05 drewr: Chouser: +1

10:05 clojurebot: Why are you asking *him*

10:05 * jackdempsey loves him some carbonara too

10:05 chessguy_work: aww, man, now you guys are making me hungry

10:05 jackdempsey: haha

10:05 cark: you're missing on spaghetti carbonara !

10:05 not red at all

10:06 on topic, i mostly unit test data structures, and then lots of repl testing and end product testing

10:08 most of the things we (i anyways) do aren't that hard to do, how wrong can you be summin an invoice from a database table ?

10:09 jdz: doing read-only stuff has some benefits

10:10 chessguy_work: oh well, back to the real world

10:10 kylesmith: quick question for someone: I'm blowing the stack on the defn on large data (> 1000), why? (defn get-bounds [objects coordinate-func]

10:10 "Returns a seq of the min and max coordinates"

10:10 (let [coords (map coordinate-func objects)]

10:10 (reduce #(list (map min (first %1) %2)

10:10 (map max (second %1) %2))

10:10 (list (first coords) (first coords)) coords)))

10:10 drewr: lisppaste8: url

10:10 lisppaste8: To use the lisppaste bot, visit http://paste.lisp.org/new/clojure and enter your paste.

10:11 kylesmith: sorry, I'm relatively new to irc.

10:11 drewr: kylesmith: np

10:11 Chousuke: hm

10:13 you could probably make that clearer if you used destructuring and vectors instead of the list function.

10:13 kylesmith: yes, of course. I'll do that.

10:16 AWizzArd: rhickey_: yes, I understood you. Automatic tests are also helpful in functional programming to protect from changes. A new coder in the company changes an old side-effect free function which still works for most cases, but for one important part it doesn't. Via Contribs test-is this can sometimes be caught.

10:17 Chousuke: kylesmith: the coordinate-function returns a seq of pairs?

10:17 lisppaste8: kylesmith pasted "get-bounds" at http://paste.lisp.org/display/83852

10:18 kylesmith: that docstring isn't the best; just a list of [[x y z] [x y z]] representing min and max

10:18 rhickey_: has everyone been ok with chunked map et al in master?

10:18 Chousuke: kylesmith: hmm

10:19 drewr: rhickey_: no issues so far

10:19 Chousuke: what kind of an object is a coordinate?

10:21 * rhickey_ wishes chunks branch were named scratchpad - and git fu for that other than copy+remove?

10:21 kylesmith: well, I pass in a seq containing sets of atoms, and get the center of mass. but that part works just fine. you can just pass in the coordinates directly and use identity or whatever.

10:21 rhickey_: any git fu

10:22 Chouser: git branch -m chunks scratchpad ...I think

10:23 kylesmith: Oh, and technically, the initial value to reduce is arbitrary, but it should get overwritten by the end.

10:23 jackdempsey: yea that should do it

10:23 git branch (-m | -M) [<oldbranch>] <newbranch>

10:23 -M forces even if newbranch exists

10:26 Chousuke: kylesmith: something like http://gist.github.com/150361 ?

10:27 * Chousuke used M-x gist-buffer for that :P

10:27 Chousuke: didn't work so well, though. no syntax colouring and I had to dig up the URL from the *messages* buffer

10:28 kylesmith: Chosuke: Yes, that is identical to my version, except for variable names. And it still works for 1500 elements and crashes for 2000.

10:29 drewr: kylesmith: Can you annotate with an example of call? What's a coordinate-func do?

10:29 s/of call/call/

10:29 Chousuke: kylesmith: maybe it's coordinate-func that's crashing?

10:30 kylesmith: okay, give me a second

10:31 Chousuke: coords holds on to most of the seq, too.

10:32 rhickey_: Chouser: afaik, -m is only for local branch renaming

10:33 Chousuke: I don't think it's possible to rename public branches without confusing the repositories of people who have pulled it previously

10:33 Chouser: rhickey_: hm. http://kerneltrap.org/mailarchive/git/2008/6/9/2070944

10:33 Chousuke: it's fairly simple to fix though

10:34 rhickey_: ParallelArray (used by clojure.parallel) is not making it into JDK7, so I've been playing with using forkjoin directly and supporting parallel ops for pvector et al

10:34 drewr: git push origin :chunks && git branch -m scratchpad chunks && git push origin scratchpad

10:34 that's destructive though, as Chousuke said

10:35 also assumes origin is the remote

10:36 lisppaste8: kylesmith pasted "get-bounds example" at http://paste.lisp.org/display/83853

10:36 rhickey_: so far so good, pvmap (maps a fn on a vector returning a new vector) is only 20 lines of Clojure, shows linear speedup vs (into [] (map f v)), this being the latest new improved 'into' here

10:36 drewr: you could leave the remote chunks and push a duplicate scratchpad (safest) but in this case it wouldn't matter

10:37 jackdempsey: nice rich

10:39 rhickey_: which mean on this machine (quad core) now 3.5x as fast as j.u.ArrayList

10:39 and in the latter case you only end up with a dangerous ArrayList, not a persistent vector

10:40 I'll try http://github.com/guides/rename-a-remote-branch if no one has a better idea

10:40 kylesmith: nicely done

10:40 Chousuke: rhickey_: maybe you should have multiple scratchpad branches instead of one though. it would be easier to merge the bits that are deemed good enough to go into master.

10:41 rhickey_: overall I think adding parallel ops to the Clojure data structures is going to be a huge win

10:42 Chousuke: if you had all your experiments separated as scratchpad/chunks, scratchpad/pvector, scratchpad/other-cool-thing etc, you could easily combine them by just creating a test branch and doing git merge sp/foo sp/bar (providing they don't conflict)

10:44 rhickey_: can do vectors and maps: map, reduce, possibly parallel map merge, parallel remove-keys, select-keys

10:46 Chousuke: does it parallelise a chunk at a time or something?

10:46 rhickey_: one nice thing, the new forkjoin (jsr166y) jar is only 53k

10:46 Chousuke: it splits up the tree recursively

10:50 lisppaste8: kylesmith pasted "get-bounds loop-recur" at http://paste.lisp.org/display/83854

10:50 kylesmith: still no luck

10:52 sh10151: xml emit functions seem to strip whitespace

10:52 is there a way to avoid this?

10:52 Chouser: xml parse strips whitespace, and xml emit adds new different whitespace

10:52 sh10151: oh ok

10:53 so xml parse strips whitespace

10:53 is there a way to avoid this?

10:53 :)

10:53 Chouser: clojure.contrib.lazy-xml has an emit that doesn't insert new whitespace

10:53 I feel like a politician.

10:53 sh10151: that doesn't bother me as much as the missing whitespace

10:54 Chouser: answering different questions than are being asked.

10:54 kylesmith: I need to leave soon, so if anyone figures out the solution to my problem, please email me.

10:54 sh10151: all righty then

10:54 xslt it is

10:54 :-/

10:55 groovy had this same asinine problem

10:55 Chouser: sh10151: afaict, both clojure high-level parsers trim whitespace

10:55 sh10151: clojure should be better than groovy, right? :-D

10:55 Chouser: sh10151: you can use java xml parsers directly (I'd recommend http://www.extreme.indiana.edu/xgws/xsoap/xpp/)

10:56 sh10151: i don't really want to parse much, just substitute in some values

10:56 can use Transformer and set some input parameters

10:56 so much boilerplate though

10:56 Chouser: or you can look at fixing the clojure parsers. I'd welcome a patch to lazy_xml.clj and lazy_xml/with_pull.clj that added a paramter to turn off whitespace trimming

10:56 Chousuke: kylesmith: I think I get the problem

10:57 kylesmith: try wrapping the map in the reduced function into doall

10:57 kylesmith: I just did that 1 second before you said, and it works!

10:57 sh10151: the default should be whitespace preservation

10:57 otherwise you are changing the data

10:57 Chousuke: kylesmith: It just dawned on me that with a thousand coordinate, you're basically generating a lazy seq that has a thousand thunks wrapping it

10:58 Chouser: sh10151: I agree

10:58 kylesmith: yep, came to the same conclusion

10:58 rhickey_: Chouser: I think cgrand might have done that at one point...

10:59 Chouser: http://www.mail-archive.com/clojure@googlegroups.com/msg13009.html

10:59 rhickey_: apparently so

10:59 kylesmith: thanks, Chousuke and goodnight all

11:05 lisppaste8: rottcodd pasted "object-seq" at http://paste.lisp.org/display/83855

11:05 rottcodd: is there something like object-seq built-in? I couldn't find anything

11:07 Chouser: that's a beautiful paste. Thanks for the complete working example!

11:07 I don't know of any such function.

11:08 Did you notice yours halts on a nil form?

11:08 (object-seq (java.io.PushbackReader. (java.io.StringReader. "1 nil #{bar foo}"))) => (1)

11:08 rottcodd: didn't notice that

11:08 drewr: rottcodd: interesting, I have a use for that right now

11:08 hadn't thought about it that way

11:12 Chousuke: hmm

11:12 lisppaste8: rhickey annotated #83855 "object-seq w/nils" at http://paste.lisp.org/display/83855#1

11:17 Jomyoot: What's a nice EMACS theme for working with Clojure?

11:18 drewr: i find the default colors quite attractive

11:18 Chousuke: hmm... (let [eos (Object.)] (take-while #(not (identical? % eos)) (repeatedly #(read reader nil eos))))

11:18 not much gain from using higher-order functions in this case I suppose

11:20 Chouser: Chousuke: your version would be indentical pre-lazier

11:20 I guess that still doesn't count as much gain. :-)

11:21 Chousuke: it does read a bit better though.

11:22 "take while not end of stream from <seq-of-stuff-when-read-repeately> "

11:22 +d

11:27 achim: you could save ~10 chars by replacing "(not (identical?" with "(not="

11:28 if you're playing golf ;)

11:28 Chousuke: that's not the same.

11:28 achim: but for (Object.), it should be

11:29 Chousuke: I wonder if Object falls back to an identity comparison ./

11:31 achim: Chousuke: i believe, = falls back to .equals, which (again, i believe) should be identity comparison for Objects

11:32 Chouser: couldn't Foo.equals(x) do whatever Foo wants, even if x is just an Object?

11:33 achim: http://java.sun.com/j2se/1.5.0/docs/api/java/lang/Object.html#equals(java.lang.Object)

11:33 Chousuke: Chouser: no

11:33 Chouser: if x.equals(y), then it must be that y.equals(x)

11:34 achim: IIRC, (= x y) maps to (.equals x y) in the end

11:34 for non-clojure things, that is

11:35 Chousuke: the order matters, .equals isn't necessarily symmetric in java-land.

11:35 but if you ask "eos", you won't get any false positives, because you can be sure it's nothing but an Object

11:37 rhickey_: I think identical? makes it clearer that this is a sentinel - i.e. we don't care about value at all

11:38 we are not testing for equality but identity - is this the eos sentinel?

11:40 but Java does guarantee (not= (Object.) anything-else)

12:05 Fossi: as much as i like "Programming Clojure", it's horrible for looking up things

12:09 ieure: It’s not really a reference book, is it?

12:14 Fossi: not so much

12:16 angerman: I'm trying to compile a script using ant

12:16 now my script uses javax.script.ScriptEngineManager, which I think should be in the JDK

12:16 but I fail to build the correct target :/

12:17 Fossi: maybe your jdk isn't set to the correct path?

12:17 angerman: how do I tell ant to instruct clojure to compile using the JDK?

12:19 Fossi: how can i access static public inner enums?

12:19 Chouser: OuterClass$EnumClass/ValueName

12:19 * Fossi already tried slashing and pointing like a madman

12:19 Fossi: ah. $

12:19 * Fossi goes to read up on that

12:20 Chouser: It's just the JVM's internal representation for inner classes

12:20 Fossi: eh. weird.

12:20 i mean to access it that way then

12:21 angerman: Fossi:maybe I'm too stupid to get ant to do it :/

12:21 Fossi: angerman: worked for me out of the box, so i don't really know

12:21 Chouser: java thinks it's good for . to mean lots of different things (sub-package, inner class, instance member, class member, etc.) Clojure not so much.

12:21 Fossi: the jdk path thing was just a wild guess

12:27 Jomyoot: Does Clojure have special theme support in emacs?

12:27 for specific elements to clojure

12:27 ieure: Jomyoot, There’s clojure-mode.

12:28 Fossi: there's an emacs mode if that's what you mean

12:28 with syntax highlighting

12:28 Jomyoot: is there some intersting light theme?

12:28 other than than those in emacs-theme?

12:28 Fossi: more like keyword highlighting really :D

12:29 dgfitch: clojure/java newb alert: I'm trying to compile the HEAD of clojure-contrib on github, and getting [java] java.io.FileNotFoundException: Could not locate clojure/walk__init.class or clojure/walk.clj on classpath: (dataflow.clj:17)

12:29 angerman: hmm should not (compile 'foo) compile foo.clj

12:29 I get java.io.IOException: No such file or directory (maruku.clj:34)

12:30 Jomyoot: well clojure is more syntax lisp than emacs lisp

12:30 clojure is more sytnax rich

12:30 i meant

12:30 so i hope for more highlighting capability

12:31 angerman: now that like 34 does not make any sense to me: http://gist.github.com/150429

12:34 Chouser: angerman: you got compile to work and now you're asking about line 34 of that gist?

12:34 angerman: Chouser: no, compile breaks stops with java.io.IOException: No such file or directory (maruku.clj:34)

12:35 Chouser: hm. you've got a 'classes' directory and its in your classpath?

12:36 angerman: no.

12:36 so I need that ... hmm

12:36 Chouser: ,(doc compile)

12:36 clojurebot: "([lib]); Compiles the namespace named by the symbol lib into a set of classfiles. The source for the lib must be in a proper classpath-relative directory. The output files will go into the directory specified by *compile-path*, and that directory too must be in the classpath."

12:37 Chouser: ,*compile-path*

12:37 clojurebot: nil

12:37 Chouser: ,(doc *compile-path*)

12:37 clojurebot: "; Specifies the directory where 'compile' will write out .class files. This directory must be in the classpath for 'compile' to work. Defaults to \"classes\""

12:37 angerman: hmm ok

12:38 Chouser: when you try to create a file in a directory that doesn't exist, unix kicks back a "No such file or directory"

12:39 Fossi: that one got me as well :)

12:40 angerman: Chouser: thanks I'm getting closer

12:40 now I have the following issue: java.lang.RuntimeException: java.lang.ClassNotFoundException: maruku$render__34 (NO_SOURCE_FILE:0)

12:41 Chouser: you're sure that 'classes' dir is in your classpath? you restarted your jvm?

12:42 angerman: yes

12:42 i simply create a classes folder

12:42 and there is a maruku$... in that folder now

12:43 Chouser: yes, but in order to find that .class file after clojure creates it, 'classes' must be in your classpath

12:44 a normal directory layout is for you to have src/survey/maruku.clj and classes/, and to have src and classes in your classpath

12:45 then when you compile you should end up with a bunch of files like classes/survey/maruku*.class

12:45 angerman: magic

12:45 it did work

12:45 Chouser: great.

12:46 angerman: now I need to find out why I can't compile it with ant ...

12:58 how do i figure out the default classpath?

13:19 Lau_of_DK: default?

13:23 Anyone using a Macbook Pro here tonight?

13:25 sh10151: I use one at home

13:26 but I am at work right now and don't have one on me

13:26 Lau_of_DK: Ok - It doesnt matter if you have it now - I just need someone who can answer a few questions in private about it

13:26 sh10151: knock yourself out

13:27 * sh10151 wonders if that idiom translates. :)

13:27 Lau_of_DK: It does

13:55 angerman: what does this cause and how do i fix it? java.lang.Exception: namespace 'survey.maruku' not found after loading '/survey/maruku'

13:56 erohtar: angerman: it means that the namespace being defined in the file is not what it is supposed to be

13:57 angerman: i suspect it should be survey.maraku

13:57 angerman: erohtar: hmm. ok, how am I supposed to name it?

13:57 erohtar: angerman: (ns survey.maraku)

13:57 angerman: assuming that the survey directory is on ur classpath

13:58 Chouser: the directory containing survey is on your classpath

14:00 * angerman is trying to see how that relates to the ant setup

14:02 * angerman notes: having two files of the same name in two different locations, having the nearly same content is a bad idea :(

14:03 cemerick: yet again, I'll meaninglessly rededicate myself to posting our ant build process

14:04 (which now only compiles clojure files that have changed since the last build -- dropped our full build time from ~60s to 10s)

14:09 weissj: I am using lancet ant stuff, but the ant functions are created by 'define-all-ant-tasks' function which repeatedly calls defmacro. but my own code isn't read properly because it calls ant functions and they aren't defined at read time. how do i fix this? i need to run 'define-all-ant-tasks' at read time, somehow.

14:10 Chouser: cemerick: not using lancet?

14:11 cemerick: no. We'll check it out at some point, but innovating in build processes is pretty low on my priority list. I've been hacking ant files for 10 years, so it's a pretty productive environment for me.

14:11 * Chouser nods

14:11 * cemerick sobs a little when he realizes he's been hacking ant files for 10 years :-/

14:12 weissj: cemerick: i'm sorry :) xml as code is terrible

14:12 that's why i like lancet. groovy has a similar thing called AntBuilder.

14:12 cemerick: eh, it's horrible, but better than all the rest

14:12 (or, all the rest that I've ever bothered to look at)

14:12 weissj: cemerick: i disagree, lancet and AntBuilder are much better.

14:13 it's real code

14:13 cemerick: right, that's where the "...that I've bothered to look at..." part comes in.

14:14 weissj: cemerick: ah. well i am trying out lancet. it's not 100% complete, but even a beginner like me can see how to get the last little bits working

14:15 anyway, can someone point me to how to get a function to run at read-time?

14:15 i know lisps are supposed to be able to do that 'whole language all the time' thing.

14:16 ieure: weissj, (defn foo [] …) (foo) ?

14:16 cemerick: weissj: you mean reader macros, I presume?

14:16 weissj: ieure: uh, how does that run the function at read time?

14:16 cemerick: i guess

14:16 cemerick: user-defined reader macros are not supported by clojure at the moment

14:16 it's a topic of some debate, yet

14:16 Chouser: weissj: macros run at a time slightly after read, but before compile

14:18 weissj: ok, so my code "requires" lancet. lancet has a function 'define-all-ant-tasks' that defmacro's in a loop.

14:18 Chousuke: hm, wasn't there some reader form that gets evaluated at read time?

14:18 weissj: but my code doesnt' compile because it calls the functions defined by those macros that haven't been created yet.

14:19 Chouser: weissj: ah.. can you call those macros before your functions are defined?

14:19 weissj: now i can manually call 'define-all-ant-tasks' in a repl before i load my file, and that works, but that seems hackish

14:19 Chouser: but my file won't even be read in properly.

14:19 Chousuke: can't you call the macro at the top of the file?

14:19 weissj: it refers to functions that don't exist

14:20 (at read time, anyway)

14:20 or compile time, for that matter

14:20 Chouser: compile time isn't instantaneous

14:20 earlier top-level forms are evaluated before later top-level forms are compiled.

14:20 try it

14:20 weissj: Chouser: oh, ok, i didn't know that's how it worked. let me try.

14:24 Chouser: putting '(define-all-ant-tasks)' at the bottom of lancet.clj seems to fix it. i wonder why it wasn't already there?

14:25 Chouser: weissj: good question

14:25 I guess you might want to define your own ant tasks first or something?

14:25 * Chouser doesn't know anything about lancet

14:26 weissj: Chouser: it has a '-main' defined so i guess it was meant to run standalone instead of as a lib... not sure. seemsa lot more useful to me as a lib.

14:28 so, i've added some fixes to lancet, some ant stuff doesn't work without it. who do i submit patches to?

14:29 Chouser: lancet's not in contrib, right? So it's just Halloway and however he wants to handle it.

14:30 you could try writing to the clojure google group, or to Halloway directly. Or fork the github repo. Not sure what he'd want.

14:30 weissj: Chouser: ok thanks

15:34 rhickey_: name game again, need a naming pattern for vector fns, and parallel vector fns, taking and possibly returning vectors, e.g. (pvmap f v) -> v and (pvreduce f v) -> x

15:34 weissj: can someone explain to me what 'partial' does? the doc doesn't make sense to me. it seems like both the function you pass in and the function it creates end up being called the same way?

15:34 hiredman: couldn't they just go in clojure.vector?

15:35 ,(partial + 1)

15:35 clojurebot: #<core$partial__4401$fn__4403 clojure.core$partial__4401$fn__4403@8ff944>

15:35 hiredman: ,((partial + 1) 2)

15:35 clojurebot: 3

15:35 cemerick: rhickey_: pvmap and pvreduce are no good?

15:35 weissj: oh! yeah. that's what i want. i want to use map but my function takes 2 args. i want one to come from the list i pass to map, and the other to always be the same

15:36 kotarak: dv pv ...

15:36 rhickey_: cemerick: I like them, just checking

15:36 cemerick: +1 for me

15:36 hiredman: ,(map (partial + 5) (range 5))

15:36 clojurebot: (5 6 7 8 9)

15:36 kotarak: weissj: there is also #(): (map #(+ % 5) (range 5)) partial only works for the #(+ 5 %) case.

15:37 weissj: kotarak: i was about to ask, what if my partial arg isn't the first one :)

15:37 thanks

15:37 rhickey_: with a 10-line helper, pvmap and pvreduce are 10 lines each, and have no forkjoin code in them

15:37 cemerick: I'm intrigued

15:38 lisppaste8: rhickey pasted "pvmap pvreduce work in progress" at http://paste.lisp.org/display/83874

15:39 kotarak: Does .. have some advantage over ->?

15:42 rhickey_: I'm trying to get rid of most of the details, but you'll still have to know a little bit about the representation of the vectors in order to define new parallel ops

15:43 kotarak: not really

15:43 weissj: how come (map #(future (do-one-file % targetDir)) mylist) blocks until all the 'do-one-file' (which download and unzips a zip file) have completed? i would think by the doc of future, it would return right away bc it spawns threads.

15:44 kotarak: rhickey_: it's pretty much obsolete then, because -> is more general...

15:44 hiredman: weissj: because you are printing the result

15:44 drewr: weissj: you're not passing the fn, you're passing the value

15:44 hiredman: and keep in mind that map is lazy

15:44 kotarak: weissj: (def x (map ...))

15:44 rhickey_: kotarak: it's classic

15:44 kotarak: archaic? ;)

15:44 weissj: er, so which is it?

15:45 drewr: er, sorry, I was wrong

15:45 forgot future is a macro

15:45 weissj: ah ok, i hoped so because it really did spawn threads

15:46 drewr: :-)

15:46 weissj: so then it only blocked because i used the repl?

15:46 hiredman: weissj: and map is lazy, so unless you force the map somehow, the futures will not be generated

15:46 kotarak: weissj: you probably want something like (def x (reduce #(conj %1 (future (do-one-file %2 target-dir))) [] mylist)

15:47 drewr: weissj: what happens when you wrap it with a doall?

15:47 hiredman: or use doall, or vec, or etc etc

15:48 weissj: hiredman wins. it's because the repl printed the return value. it doesn't block if i put a 'nil' at the end of the function that calls that stuff

15:48 actually that is not quite true - it doesn't do anything either because the list is not consumed

15:49 so yeah, doall is prob the answer

15:50 now, that answers why it didn't behave like i expected in the repl. but i DO want it to block. so i just need to (doall (map (deref xx))) on it?

15:53 hiredman: oh

15:53 weissj: you want pmap

15:53 ,(doc pmap)

15:53 clojurebot: "([f coll] [f coll & colls]); Like map, except f is applied in parallel. Semi-lazy in that the parallel computation stays ahead of the consumption, but doesn't realize the entire result unless required. Only useful for computationally intensive functions where the time of f dominates the coordination overhead."

15:54 weissj: hiredman: ooh, sweet. this function is now getting trivially short. love it :)

15:58 one thing I'm having difficulty with is 1) knowing there's already a function that does what I want and 2) even if I suppose there must be one, finding it. find-doc only gets you so far.

16:04 Chousuke: hmm

16:05 Clojurebot need a more sophisticated AI so you could ask it about the API :P

16:15 weissj: i suppose that is a problem with any language. all you can do is search the api and hope for a hit. half the time you don't even know the right term to search for.

16:16 i would never have known about pmap. after reading about agents, i wouldn't have thought to search for a parallel map function.

16:17 i guess we need to just read the doc on everything :)

16:17 Chouser: I recommend: reading other people's code (such as core.clj, stuff in contrib), hanging out here (watch other people's answers, ask your own questions), and finally start answering questions here.

16:17 that last step is when you really start to learn... :-)

16:18 weissj: Chouser: yeah, i've only been at it about 2 weeks, and i already see people coming in here behind me asking even more newb questions that i already know the answer to :)

16:18 TheArthur: can I create a sparse vector in clojure?

16:18 weissj: this channel is a great resource, plus the clojure.org website and stuart halloway's book

16:19 TheArthur: and is there an easy way to create a vector full of 0 s?

16:19 Chouser: for somewhat faster bootstrapping, you can look at the couple of places that have categorized functions, to help you find fns "near" other ones in conseptual space.

16:20 weissj: for example, 'map' and 'pmap' appear next to each other here: http://clojure.org/sequences

16:20 TheArthur: vectors are not sparse. For that, use a sorted-map or hash-map.

16:20 ,(vec (repeat 20 0))

16:20 clojurebot: [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]

16:21 Chouser: but there's a vector of zeros, length 20.

17:53 ieure: I’m trying to do some shebang scripting with Clojure, but I keep getting this error.

17:53 http://gist.github.com/150935

17:53 The code: http://gist.github.com/150933

17:53 I don’t get it. The code is runs, then I get an exception.

17:55 Chousuke: try removing the ns declaration

17:55 ieure: Chousuke, Same error.

17:56 Chousuke: hmh

17:57 technomancy: ieure: are you using the bash script from contrib?

17:57 ieure: technomancy, No, it’s whatever wrapper MacPorts has.

17:58 * technomancy wishes we could standardize on one and include it in Clojure itself

17:58 ieure: Indeed.

17:58 It would also be nice if it wasn’t insane to package up CLI tools in JARs.

17:58 Unless there’s a better way of doing it than what’s listed on WikiBooks.

17:59 technomancy: yeah, the JDK is positively useless when it comes to CLI integration

17:59 Chousuke: hm, I can't even use my clj script as an interpreter :/

17:59 ieure: You could have left out from "when" on.

17:59 technomancy: ieure: a friend of mine was working on a project to make that a little less painless: http://github.com/dkellum/hashdot

18:00 kinda rough around the edges in terms of installation, but it is worlds better than the "java" launcher

18:00 ieure: I think the best thing about Clojure is that it leverages everything in the JDK. As opposed to it’s downside, which is that it leverages the JDK.

18:00 Chousuke: heh

18:00 ieure: Seriously, I hate Java.

18:00 technomancy, 404.

18:01 technomancy: ieure: right; that should be https://github.com/dekellum/hashdot/tree

18:01 ieure: Thanks.

18:01 technomancy: it's a command-line tool that's written by someone who's actually used a terminal before! amazing.

18:02 <insert rant about single-dash-full-word arguments />

18:03 the cool thing about hashdot is that you install it once and symlink it to clj, jruby, rhino, etc; and it loads different property profiles based on what symlink it was invoked with.

18:10 ieure: hashdot looks nice, thought it doesn’t solve the .jar issue.

18:10 technomancy: nope. =\

18:10 it's focused more on server-side JVM than distribution

19:00 krumholt_: i have this list '(a a a b b c c c c d a a) and i want => ((a 3) (b 2) (c 4) (d 1) (a 2)) anyone knows a clever way to do this?

19:01 technomancy: are you sure you want a list and not a map as the return value?

19:01 krumholt_: they have to be ordered

19:02 i dont want to know how many a's where in the list. i want to know how many where there in a row

19:03 like (a a a b a a a) => ((a 3) (b 1) (a 3)) the order of the elements is still there it is just mor compact

19:08 replaca: krumholt_: I would write a little func that used (take-while #(= (first foo) %) foo), count, drop and loop/recur to build the list

19:08 krumholt_: more clojure-y to build the result as a vec (and it lets you use cnj and get the right order)

19:09 but there might be a better way

19:10 krumholt_: a take-while looks usefull thanks

20:07 technomancy: ieure: I'm not using the http client for authenticated requests, but if you're seeing arguments ignored I will try to fix that.

20:08 ieure: technomancy, Your HTTP client worked fine. The contrib one ignored the :headers I gave it.

20:08 technomancy: oh gotcha

20:09 yeah, I've found it's a bad idea to implement features I don't use, but I'm happy to answer questions if you've got ideas for a patch.

20:09 ieure: I just had to generate the authtoken myself. And I guess there isn’t anything standard that does Base64 the way HTTP wants it.

20:09 technomancy, Maybe, I’m really not comfortable enough with Clojure to go hacking on it just yet.

20:10 From what I saw, I’d need some way to frobnicate the connection before sending the request.

20:11 Maybe a (with-connection) macro that I could call (request) inside of. Not rally sure.

20:12 technomancy: ieure: something like that is probably needed for connection pooling anyway

20:12 but I haven't been doing much HTTP work these days

20:33 ieure: Hmm, the whole contrib.http.agent stuff seems to be pretty broken. This should work, right?

20:33 (use 'clojure.contrib.http.agent)

20:33 (let [a (http-agent "http://google.com")]

20:33 (response-body-str a))

20:33 It returns nil. Same for response-body-bytes.

20:33 But I can get the headers okay.

21:06 duncanm: because of Clojure, i started looking into Swing this weekend

21:07 it's really not that bad, right? in fact, it's a pretty powerful toolkit

21:10 drewr: duncanm: it's tedious, but much more fun with clojure

21:10 duncanm: yeah, with Clojure, i think it's not a bad toolkit at all

21:37 uninverted: Does Clojure have any way to get the length of a sequence? I can't see anything in the docs.

21:42 mudphone: univerted: try count

22:07 weissj: is there a way to give a function param a default value, other than just adding a new section without that param, and just calling the first section with the default value?

22:08 ie like (defn x ([y z] (str y z)) ([y] (x "yo")))

22:09 that would give z a default of "yo"

22:17 drewr: weissj: defnk in clojure.contrib.def gives you keyword args that have default values

22:30 duncanm: have any of you looked into JavaFX? i'm confused as to what it is

22:30 i thought it's a new language with a set of Java libraries (built on top of Swing)

22:31 but i can't find any resource on using the JavaFX libraries using something other than JavaFX script

22:31 cemerick: weissj: using map destructuring with a default value is more idiomatic, and doesn't require an external lib

22:32 duncanm: sun hasn't followed through on their prior statements about supporting using javafx libs from other languages

22:33 they're supposedly going to open things up eventually, but it's in the air at this point

22:34 duncanm: ahh

22:34 but it's still possible, right? it's just yet another jar

22:48 cemerick: yeah, I've read of people figuring some stuff out (using jruby, I think?). A lot of the API is reflection-driven though, so it's not as straightforward as running javap and backing into things.

23:13 Jomyoot: Do you guys prefer light on dark or dark on light background?

23:14 skalnik: Depends on what

23:14 Code I like light on dark, everything else, dark on light

23:19 scottj: Jomyoot: dark on light because I have a bright work environment and I like all my windows to be consistent, and its much easier ot have everything light than everything dark (webpages don't convert well)

23:24 Chouser: agriffis (who's apparently not here at the moment) has a script which switches all his xterms and gvim windows between light-on-dark and vice-versa based on the time of day.

23:30 duncanm: Chouser: that's cute

23:44 fsm: Hello everyone, I am back with my double primitive performance issue

23:45 I have a test case if anyone wants to look http://solardriftwood.com/testdouble.clj

23:45 This test tries various methods of storing and accessing arrays of doubles

23:45 But, looking in the profiler, all the math is done using Numbers.add instead of native ops, no matter what

23:46 Am I doing something wrong?

23:47 hiredman: fsm: have you set warn-on-reflection?

23:47 clojurebot: performance

23:47 clojurebot: Gabh mo leithscéal?

23:47 fsm: Yes, I have set warn-on-reflection, there is no reflection

23:47 hiredman: bah

23:47 clojurebot: you worthless bag of bones

23:47 clojurebot: I don't understand.

23:48 fsm: I am quite perplexed about why nth and [] is as fast as using native double arrays

23:48 and I am a bit perplexed why none of my math is done with primitives

23:48 I have been through the whole range of tips on the java_interop page

23:49 Clojure is great but having virtual method calls inside my arithmetic is slowing down my raytracer alot

23:49 hiredman: fsm: I am fairly certain (+ (vxfn vectPrim) 1.0) should result in a reflecton warning

23:49 fsm: Sample image here btw: http://solardriftwood.com/aa.png

23:50 I just ran again with warn-on-reflection, no reflection occurs

23:50 rhickey: fsm: you've read this carefully? http://clojure.org/java_interop#primitives

23:51 hiredman: fsm: reflection warnings do not happen when you run code

23:51 rhickey: and: http://clojure.org/java_interop#optimization

23:51 fsm: Yes, I believe I have tried every trick in the book

23:51 hiredman: the happen at compile time, when you paste code into the repl for example

23:51 fsm: My test case is here: http://solardriftwood.com/testdouble.clj

23:52 I have seen refl warnings before in my other code, no refl warnings in this code

23:52 My issue is that Number.ops is being called for what should be primitive arithmetic by my understanding

23:52 rhickey: nothing in the optimization section says use aset-double, it's slow

23:52 fsm: aset-double is only used three times, outside the performance loops

23:53 I have also tried into-array, no difference

23:53 rhickey: "aget/aset are overloaded for arrays of primitives"

23:53 fsm: I am enjoying Clojure greatly, I am just a bit perplexed on this issue

23:53 That is, if I have a macro that uses aget, why is Number.ops used on the resulting double, instead of native operator +

23:54 In the profiler, I can see that Number.ops is called for every single arithmetic in that test case

23:55 Basically, for my raytracer I am seeking the most efficient way to represent vectors of 3 doubles (x, y, z)

23:55 rhickey: you'v e built all of this macro stuff - you shouldn't macro-ize until you have a hand-written thing you want to generate

23:55 fsm: I did that to test the performance of macro vs function

23:55 I found that macro-izing actually caused about 30% speed drop in my raytracer

23:56 Nonetheless, I wish to get rid of all these Number.ops

23:56 The test case compares macro and function versions, but either way, Number.ops is used

23:57 I have looked at the implementation and found that aget turns into a static method that has the body return xs[i]

23:57 Is the return value of aget boxed?

23:58 hiredman: fsm: first line of http://clojure.org/java_interop#optimization

23:58 fsm: Yes, does that apply to return values as well? That is the thing I couldn't work out myself.

23:59 rhickey: you need to use primitive locals, as described in those sections, and primitive arrays, and the primitive array constructors double-array etc

Logging service provided by n01se.net