#clojure log - Jul 21 2009

The Joy of Clojure
Main Clojure site
Google Group
List of all logged dates

0:00 hiredman: I guess maybe be a mention of return values would be good there

0:00 fsm: yes

0:00 function invocation uses the invoke method on IFns, which returns an Object

0:01 fsm: Is double-array different from make-array Double/TYPE ?

0:01 rhickey: if you cross a fn boundary things are passed and returned as objects, your inner loop should be inside one fn

0:01 passing whole arrays in and out is no problem

0:01 as arrays are objects

0:02 fsm: Ok I got no speed difference using double-array vs make-array.

0:02 I understand the let [] binding method to get a primitive double, let me explain why it doesnt quite work for what I want to do

0:03 Basically, all my 3D code revolves around (x, y, z) tuples. Arithmetic operations are usually done to all three at once, in a series (you might add two vectors together and scale the results).

0:04 There will be an explosion of code if you use let [] bindings to pull out the individual values every time

0:04 Unless I am mistaken?

0:04 rhickey: you're calling a function to get a single thing out of a vector, that will never be fast until you put your vector, and the aget and its return into the body of a fn - all you are seeing here is boxing overhead and learning nothing about the perf of the primitive ops

0:05 er, array, not vector

0:06 fsm: Are you saying, e.g. to assign the result of aget to a variable and then do several operations on that primitive variable?

0:07 rhickey: do you have an example of some (small) real thing you are trying to make faster? I really can't make heads or tails of that code

0:07 fsm: OK sure let me post my actual project, 1 sec

0:08 rhickey: but in general, if the array is hinted, the compiler will keep the result of aget primitive as long as it can, so will do with math ops involving other primitives, assignment to other primitive arrays etc. As soon as you pass it across a fn boundary it will box it and primitive game is over

0:08 fsm: Please excuse my pidgen-functional programming, this is my first time since haskell in the 90s.

0:08 http://solardriftwood.com/tracer.clj

0:09 You can see near the top, my little vector math library, that is used extensively

0:09 I have tried macros and double arrays with aget but this method had the fastest performance

0:10 hiredman: thosemacors

0:10 those macros

0:10 icky

0:11 (sorry to change the subject)

0:11 fsm: They are the fastest accessors for the contents of the tuple that I have found

0:11 I appreciate their ickyness

0:11 hiredman: you are evaluating the args to the macros multiple times within the macros

0:11 rhickey: If you want the fastest math you will need primitives and primitive arrays right now

0:12 roughly 20x faster than boxed arithmetic

0:12 fsm: Yes, I switched my code to primitive arrays

0:12 I changed makeVect to use make-array Double/TYPE

0:12 and I changed my vx vy vz macros to (aget array index)

0:12 and I got 30% slower

0:13 rhickey: array and index were primitives?

0:14 hiredman: for example, (makeVect [1 2 3]) turns into [(nth [1 2 3] 0) (nth [1 2 3] 1) (nth [1 2 3] 2)] instead of something like (let [foo [1 2 3]] [(nth foo 0) (nth foo 1) (nth foo 2)])

0:14 rhickey: you need to make sure every participant in the computation is primitive, otherwise you will actually do more boxing

0:14 fsm: I changed to this macro

0:14 (defmacro makeVect [x y z] `(let [v# (double-array 3)] (aset-double v# 0 ~x) (aset-double v# 1 ~y) (aset-double v# 2 ~z) v#))

0:15 arohner: rhickey: would it be possible to make a "warn-on-slow-math" flag, similar to warn on reflection?

0:15 fsm: And this macro for accessor (defmacro vx [vect] `(aget ~vect 0))

0:15 And yet, these changes caused a performance drop

0:16 I also tried this for my accessor macro (defn vxwithhint [#^doubles vect] (aget vect 0))

0:16 And that gave the same speed as using [] and nth

0:18 rhickey: I think your problem might be in constructing the array tuples

0:19 you definitely do not want to create a vector in order to create an array

0:20 fsm: Yes

0:20 rhickey: you can keep an array of 3 zero's around and clone it in make-vect

0:20 fsm: That second form with all the nth horror, that is only used rarely in some non performance critical places

0:20 rhickey: then aset the members

0:20 you cannot use nth, only aget

0:21 fsm: The one that hiredman is referring to

0:21 The problem is how to cleanly express (vecAdd v1 (vecSub v2 v3)) and get primitives used

0:22 Where v1, v2, v3 are tuples of three double values

0:22 rhickey: v1/2/3 have to be double-arrays

0:22 fsm: I found using double-arrays to have same performance as [] and nth, presumably because of boxing

0:23 rhickey: then vec-add can hint (defn vec-add [#^doubles v1 #^doubles v2] ...)

0:23 fsm: Yes, when I did that, I got the same performance as using [] and nth

0:23 I swear

0:23 duncanm: heh

0:24 fsm: That is why I wrote the testdouble.clj to evaluate these alternatives

0:24 rhickey: that doesn't indicate anything about arrays, just that you didn't have everything primitive

0:24 fsm: Indeed

0:24 rhickey: so, first you need a fast make-vect

0:25 fsm: what about makeVect and vx macros in my testdouble.clj?

0:26 duncanm: no need to write any macros, right?

0:26 (defn make-vect [x y z]

0:26 (let [v (make-array Double/TYPE 3)]

0:26 (aset v 0 x) (aset v 1 y) (aset v 2 z)

0:26 v)) ;; is this fast?

0:27 fsm: That is what I have, except i call aset-double

0:27 And consider my vxfn for a non-macro version

0:27 of vx *

0:27 duncanm: 23:53 <rhickey> "aget/aset are overloaded for arrays of primitives"

0:28 fsm: aset is not overloaded on my clojure install

0:28 I get a comple failure

0:28 compile*

0:29 Perhaps my clojure is out of date, but the net result should be the same

0:30 notostraca: Anyone here interested in Clojure for... games?

0:31 duncanm: fsm: which version are you running?

0:33 fsm: I have the macports version, 20081217_0

0:34 hiredman: wow

0:34 that is old

0:34 fsm: Hm, that is the trouble with macports

0:34 I will install the latest version and re-test

0:35 rhickey: lisppaste8: url

0:35 lisppaste8: To use the lisppaste bot, visit http://paste.lisp.org/new/clojure and enter your paste.

0:35 rhickey pasted "fast make-vect" at http://paste.lisp.org/display/83896

0:42 duncanm: rhickey: if make-vect were a function, it'd be slower?

0:43 rhickey: duncanm: x y z would have to be boxed

0:45 fsm: OK, same results on clojure-1.0.0

0:45 I saw your fast make-vect

0:45 The next problem is how to access the contents of that vector without boxing

0:46 I know I can use let binding to assign the result of aget to a double

0:47 But, then the question is how to cleanly write a vecAdd function to add the elements of two arrays together

0:47 hiredman: all the operations would have to be macros that emit calls to things that handle primitives

0:48 fsm: Yes

0:48 hiredman: or just use aget

0:48 fsm: But how to pull the primitives out of the array without them being boxed/unboxed?

0:48 I did that, result was boxed

0:49 (guessing from the calls to Number.ops)

0:49 so if I have two java double arrays (1.0 2.0 3.0) (4.0 5.0 6.0) how can I end up with a java double array (5.0 7.0 9.0) (i.e. sum) without boxing/unboxing?

0:50 duncanm: rhickey: is the only way to pass primitives around is to avoid Fns and use macros?

0:50 fsm: The only way I can see is to use amap perhaps?

0:50 But that doesnt work for functions like dot and cross product, that require access to the elements of the arrays in all kinds of funky orders

0:51 And then even the result of amap/areduce would be boxed I guess, meaning another box/unbox for any operation on the resulting value

0:53 hiredman: fsm: so you want something like (makevec (+ (aget v1 0) (aget v2 0)) (+ … so you just write a macro that emits that code

0:53 fsm: Yes, I wrote my macros to do that

0:53 But that caused boxed arithmetic functions to get called

0:53 I checked with macroexpand

0:53 And the profiler

0:54 Sorry to cause the tech support call from hell to you guys

0:54 I really enjoyed learning clojure so far

0:55 If you see my vx macro in testdouble.clj it does what you suggest

0:55 but the result of (+ (aget v1 0) 1.0) is a call to Number.ops

0:57 duncanm: fsm: how can you tell that?

0:58 lisppaste8: rhickey annotated #83896 "vec-add" at http://paste.lisp.org/display/83896#1

0:58 rhickey: 20x faster than vecAdd

1:00 fsm: a 1.0 literal is not a primitive double, use (double 1.0)

1:02 fsm: OK let me do some testing

1:02 brb

1:02 rhickey: I think your time will be dominated by array allocation

1:10 fsm: OK here is a tricky finding

1:10 That vec-add function up there results in calls to unchecked-add

1:11 But when I change the numbers in the vectors to floating point by putting a .0 on the end

1:11 My old friend Number.ops reappears

1:11 and performance drops 50%

1:12 And I have to change aset for aset-double

1:12 hiredman: uh

1:12 22:01 rhickey : fsm: a 1.0 literal is not a primitive double, use (double 1.0)

1:13 fsm: oops

1:14 hiredman: my understanding is when done properly, aset will always be faster than aset-double

1:21 fsm: OK, I did some poking around, it looks like adding two doubles results in a call to Numbers.add() which should hopefully resolve to the version that takes two doubles

1:21 I am going to plug this stuff into my raytracer and compare the performance

1:25 Jomyoot: how would i define line-height for .emacs.el

1:28 thearthur: how can i clear all the deftest'ed tests in all namespaces?

1:28 Jomyoot: which emacs project brow3ser do you guys use for clojure?

1:39 thearthur: is there a clojure-contrib chat room also?

1:39 fsm: OK, I retrofitted the fast vector code to my raytracer

1:40 A 10% performance drop over [] and nth

1:40 :/

1:40 Maybe the use of macros is causing me to blow out the cache

1:48 The same result as my own attempt to use double-arrays, so there it is

1:48 Thanks anyway everyone

2:45 kylesmith: Does anyone know how to print structures with circular loops? I have a structure that has an atom for the parent and children. When I try to print it at the repl, I blow the stack. I'm guessing it's trying to print the children, then the parent, etc etc.

2:47 (I wrote this code when I first learned clojure, so it sucks.) Zippers are able to access the parent implicitly, so should I just give up and rewrite it?

2:50 fsm: Any of my ray tracing helper friends still around?

3:13 yangsx: (my-fn (first my-coll)) gives me expected results, but (map my-fn my-coll) gives NullPointerException at LazySeq.java:47 because my-fn has to do IO processing. I know that's a problem related to laziness, but don't know how to avoid.

3:13 jdz: yangsx: try doall or dorun

3:14 yangsx: jdz: (doall (map my-fn my-coll)) does not help.

3:16 jdz: is it in the same place in the code as the simple my-fn invocation that works?

3:24 yangsx: jdz: yes. I'm using slime interactively. I think laziness is off with REPL and that's why doall/dorun do not solve my problem.

3:24 jdz: what do you mean "is off"?

3:25 yangsx: jdz: I thought it's eager.

3:25 jdz: the repl is besides the point here, really.

3:25 if my-fn does i/o, then it must have the stream open

3:26 that's all there is to it

3:26 if you have to process my-coll with my-fn, then do it with doall where the stream is open

3:30 yangsx: jdz: my-fn calls another function, which has (with-open [..] (do-all ..)). It's strange it should fail with my-fn, but other functions calling it works as expected.

3:31 work*

3:31 jdz: well then you'll have to provide the code (using lisppaste)

3:36 yangsx: jdz:thanks for the help; have to sort unrelated code first

4:42 Fossi: hi

4:47 kylesmith: Would anyone like to help me out with some performance optimizations?

4:52 rottcodd: is it possible to go up directories when loading files? eg (load "../foo")

4:56 jdz: rottcodd: that will be relative to the working directory of your JVM

4:57 rottcodd: jdz: so it doesn't search the classpath?

4:58 jdz: load does not

4:58 classpath is for namespace-related functions

4:58 like require, use, import

5:17 rottcodd: actually, i have misinformed you.

5:17 ,(doc load)

5:17 clojurebot: "([& paths]); Loads Clojure code from resources in classpath. A path is interpreted as classpath-relative if it begins with a slash or relative to the root directory for the current namespace otherwise."

5:22 gko: In Classpath means in a .jar ?

5:23 jdz: not necessarily

5:23 one can put either .jars or directories in classpath

5:28 gko: Looks like file must be named like: a_b_c.clj

5:28 and not: a-b-c.clj

5:28 jdz: well, you should know that: http://clojure.org/libs

5:29 gko: oh... my bad..

5:31 jdz: learn something new every day :)

5:31 gko: Is the general rule: "use" for libraries, "load" for own code?

5:31 :)

5:31 jdz: i don't use load at all, yet

5:31 load is helpful if you want to split a nampespace into many files

5:34 gko: ok

6:09 Jomyoot: Dear Emacs users in this channel. Do you guys use any project management or project browser plugin for emacs?

6:18 rottcodd: Jomyoot: I don't bother, the tools I use most for browsing code are M-. (slime-edit-definition) and M-, (slime-pop-definition-stack)

6:19 Jomyoot: What if you have codes in different files different folders?

6:19 very deeply located folders

6:20 how do you quickly manage all these definitions

6:20 jdz: same

6:20 rottcodd: M-. will find any definition that is loaded

6:20 Jomyoot: hmm

6:20 i will hopefully learn soon

6:21 but for a newbie. transitioning in from being used to IDE. Do you guys think ECB is good?

6:23 I might go as far to ask: does Slime provide code completion both for Clojure and Java Interop?

6:30 hd_: hey, i got errors while building clojure from git repository. These are the errors http://pastebin.com/d18261d17. Any idea?

6:32 rottcodd: Jomyoot: have you looked at enclojure: http://www.enclojure.org/

7:29 Fossi: is there a library yet that generates documentation for a directory path (using gen-html-docs or such)?

8:32 fsm: Hi. Floating point performance solved in my raytracer.

8:32 I misread this line "All arguments are passed to Clojure fns as objects, so there's no point to putting non-array primitive type hints on fn args."

8:33 Missed the 'non-array' part.

8:33 cark: oh nice

8:33 fsm: Thanks everyone, that improved my vector routines speed 5x and my raytracer improved by 25%

8:33 I did some profiling and made an overall speed boost of 100%

8:33 Chousuke: nice.

8:33 cemerick: excellent :-)

8:34 Chousuke: are you doing this on clojure 1.0?

8:34 fsm: Yes, I am now.

8:34 cemerick: looks like the "math as fast as java" claim is being borne out

8:34 Chousuke: trunk has some neat new stuff that might make things even faster.

8:35 also, did you try the new escape analysis features in JDK 1.6b14?

8:35 I'm still interested in seeing how it affects "real" code

8:35 fsm: Here is a sample image of testing depth-of-field effects: http://solardriftwood.com/dof.png

8:35 Just for fun.

8:35 I think I am restricted on java versions, being on osx

8:36 Chousuke: oh. yeah, only 1.6b13 for os x for now :/

8:36 cemerick: 1.6_13 is the latest on OS X, AFAIK

8:37 fsm: I just checked, I am on 1.5, I am going to switch to 1.6 and compare

8:37 jdz: what's left is to publish the source and let somebody else do the benchmarking :)

8:37 Chousuke: fsm: there's no escape analysis on the OS X version of 1.6 yet though :/

8:38 cemerick: 1.6 *is* faster than 1.5, though. Not orders-of-magnitude, but still...

8:38 Chousuke: oh wait

8:38 there's a developer preview on ADC

8:38 apparently has 1.6.0_15

8:39 fsm: I will publish the source into public domain once I have improved the quality and added some features

8:39 Chousuke: hm, public domain is problematic

8:39 use MIT or something instead :P

8:40 PD is a problem because it's ineffective in some jurisdictions AFAIK :/

8:41 * Chousuke downloads the java DP to see if it has escape analysis

8:43 Chouser: was rhickey really in here until 1am EST?

8:44 cemerick: clojurebot: ~rhickey

8:44 clojurebot: "The guy is totally awesome." -- Daniel Weinreb

8:44 cemerick: clojurebot: ~rhickey

8:45 clojurebot: he works hard so you don't have to

8:45 cemerick: :-)

8:45 Chouser: ~seen rhickey

8:46 clojurebot: rhickey was last seen joining #clojure, 31 minutes ago

8:46 jdz: clojurebot: I totally agree (with you and Dan)

8:46 clojurebot: Titim gan éirí ort.

8:46 jdz: ye, whatever

8:47 talking to bots is fun

8:50 fsm: Interesting, Java 1.5 vs 1.6 and Clojure 1.0 vs 1.1 alpha, performance difference is under the noise floor for my short tests

8:50 I will try ADC preview later

8:51 jdz: clojurebot: ~rhickey

8:51 clojurebot: rhickey is totally awesome!

8:51 jdz: i win

8:52 damn, i'm repeating Dan

8:52 clojurebot: ~rhickey

8:52 clojurebot: rhickey is totally awesome!

8:52 jdz: hmm

8:59 Chouser: reading "iterators must go" http://lambda-the-ultimate.org/node/3520

9:00 beautiful and condemning analysis of C++ STL iterators

9:01 3 slides into the proposed solution and state sneaks right back in.

9:01 cemerick: whew, just the thought of hacking C++ is scarier to me than ever (now that I'm completely steeped in clojurey-goodness)

9:05 fsm: Is this an acceptable forum for C++ bashing?

9:05 If so, I would like to bash C++.

9:05 Chouser: it is interesting how much these "ranges" seem to be targetted at solving many of the same problems as the seq library

9:06 I suppose if you operate them over immutable collections they'd be fine...

9:06 cemerick: yeah, 'cause those are all over the place in C++

9:06 * cemerick snorts

9:08 cemerick: I guess I shouldn't be so harsh. (Almost) all my C++ work has been in connection with various FFIs, which certainly made things a lot more painful than they probably would be in a real C++ app.

9:09 Chouser: oh, immutable collections? Yeah, never seen one in C++.

9:09 rhickey: Chouser: I argued against STL iterators to no avail when they first came out. At the time I especially hated that they came in two disconnected parts. The pressure to retain the pointer model was intense

9:10 Chouser: rhickey: did you see this paper? That's exactly what he's doing differently -- start and end tied together

9:10 rhickey: Chouser: yes, saw it and sighed

9:11 now what, 15 years later?

9:11 Chouser: but I guess I was observing that though ranges fix some of the syntax pain of iterators, the truly deep and panicky pain that iterators have caused me are more due to the mutable collections than the iterators themselves

9:12 rhickey: I keep forgetting you were deep into C++

9:13 rhickey: that, and the fact that they really can't serve as elements of interfaces. I.e., how many Java APIs do you see that take/return Iterators? They are for inside-a-function use only

9:17 Guy Steele argues against Lisp lists and sequential processing in general here: http://groups.csail.mit.edu/mac/users/gjs/6.945/readings/MITApril2009Steele.pdf

9:19 a bit hard to follow from the slides, but Clojure is well positioned here as few people keep their important data in lists in Clojure. The vectors/maps/sets are set up for parallelization

9:20 fsm: Hello Richard, I got my head around my floating point issues, many thanks.

9:20 Jomyoot: how to always turn paredit-mode on ?

9:21 fsm: I had misread the rule about avoiding type-hinting function arguments.

9:21 rhickey: fsm: sure. I'm not sure you will get the most blazing perf using small vectors/arrays as points in Java/Clojure due to the allocation vs say in C++ where they could be value types

9:22 but you should be able to match Java in any case

9:23 fsm: Yes, performance is constrained mainly because in C/C++ you get intel's libraries which render such 3d math operations into SSE

9:23 However, I did get a 30% speed boost once I figured it out, and then with some profiling, an overall 100% boost, still more work to do

9:23 rhickey: fsm: great!

9:24 cemerick: that's roughly our biggest single hotspot as well -- spatial indexing of 2D shapes isn't as demanding as ray-tracing, but I suspect the allocation patterns are similar

9:24 * cemerick looks forward to cheap mutable/immutable conversions, and new-new :-)

9:24 rhickey: what about keeping each set of Ds in it's own array?

9:25 its

9:26 cemerick: eh, we're trying to not get overly clever yet

9:26 fsm: OK, I am going to leave a big render going and get some sleep. Thanks again.

9:27 cemerick: we've done enough perf work to make everything else tolerable -- we need to build about 4 more levels on top of what we have now before we stop to take a look at perf in a serious way

9:28 rhickey: in my audio work I'd always split and stripe arrays of aggregates - instead of [{x y z} {x y z} ...] [x x ...] [y y ...] [z z ...]

9:28 cemerick: worst case scenario is that we buy a couple more U's of CPU until we do an incremental rev on the final product.

9:28 rhickey: but it is a pain to manage

9:28 AWizzArd: One thing I will do is profiling our AI engine in a few weeks and finding out where the time is spent. I would like to hear your opinions, if there is potential room for improvements.

9:29 cemerick: yeah, I'd like to preserve a proper DOM without doing backflips with the bookkeeping -- though we'll do whatever we can when we get to the optimization part

9:29 rhickey: well, I'm convinced parallel ops on vectors/sets/maps will be a big win for Clojure, and then it's just a matter of more multi in your multicore

9:30 today I hope to put a granularity knob on the pvmap/pvreduce stuff I showed you yesterday

9:30 cemerick: well, I've experimented with using a base class for all of the coordinate fields of each shape, and it provides a huge boost -- I suspect new-new will compile down to the same thing

9:31 base Java class, that is

9:31 Chouser: That Steele pdf is quite interesting.

9:31 rhickey: also will expose fork and join, so you can build multi-call nested aggregate parallel operations and feed the same pool

9:32 Chouser: is it fair to say the length stored for each node a vector essentially represents his new "base case" for interior tree termination?

9:32 cemerick: I saw a link somewhere with the latest (bad?) news about fork/join in jdk7, but can't find it now...

9:32 rhickey: Chouser: yeah, but I don't think using binary trees as lists is that compelling. The bigger story is trees in general, and Clojure is in good shape there

9:32 cemerick: I'd like to see that link

9:33 Chouser: yes -- log32 is fantastic for trees, vs. log2

9:33 cemerick: rhickey: heh, I thought you had pasted it :-P

9:33 rhickey: Chouser: and using trees for vectors and hash tables

9:33 cemerick: something about a particular JSR not going to be ready in time?

9:33 Chouser: both theoretically ("essentially constant time") and practically for how cache lines work.

9:34 rhickey: cemerick: huh, I've been working with FJ all weekend in response to a request to provide some input

9:35 cemerick: ach. Perhaps I'm more confused than usual.

9:35 (there's a lot of JSRs flying about lately)

9:35 or, news thereof, that is

9:36 rhickey: yeah, I do know ParallelArray is out, and that's a great thing as it forced me to look at applying FJ directly to Clojure's data structures.

9:39 * rhickey needs to figure out how to continue working on Clojure full time...

9:40 Chouser: pvreduce eliminates the "SUM=0" from reduce that Steele describes as bad.

9:40 as of course it must

9:41 working on Clojure and working in Clojure have very different profit structures. :-/

9:42 rhickey: Chouser: exactly. I was wondering if those doing the latter might be willing to, e.g. pay for 'support' contracts so I can continue to do the former

9:42 some more writeoff-able than donations

9:43 something

9:43 Chouser: seems plausible, but dunno if we're there yet.

9:43 rhickey: right, the tricky part is surviving from here to 'there' without getting sucked into something else

9:44 Chouser: I'm at maybe 20% of my time writing Clojure instead of C++, but nobody here is committed to it -- extra cost would simply be a reason to remove that 20%

9:44 yeah, I understnad.

9:56 AWizzArd: rhickey: btw, is there still no one working on Fingertrees?

9:56 rhickey: AWizzArd: I'm not

9:57 AWizzArd: I will see if I can talk a Haskell friend into it. He is interested in Clojure and tree structures :-)

9:57 rhickey: I'd love to have cinc and newnew in order to be able to write it in Clojure rather than Java, but you could use proxy and just swap in newnew to make it faster later...

9:57 newnew will not be called newnew

10:01 re: newnew and mutable locals, one concept I had early on was to make it so that mutable locals could not be closed-over. The problem is, that creates an asymmetry, getting around which requires a box. That is the story with Java's inner classes

10:01 AWizzArd: hmm

10:02 rhickey: other langs supporting closures and mutable locals secretly create a box for you and all closing-over fns share the box. This is the source of toy OO examples in Scheme's etc

10:02 the third option, dunno if used anywhere, if for each closure to get its own mutable 'hole', not shared

10:04 AWizzArd: But mutable locals will come?

10:04 rhickey: the logic is, closing over a value gives you a value with that name, closing over a hole gives you a hole with that name, *not* a reference to a hole

10:06 since creating member fields in newnew is done by closing over let-bound locals, let would need to be able to create a mutable binding, let's say like (let [x (volatile 42)] ...)

10:07 what I am considering is that any closure over x would get its own volatile x with no connection to the others

10:07 Chouser: rhickey: so ... multiple closure instances in the same let would get separate mutables?

10:07 rhickey: right

10:08 in the 'normal' case of newnew there would only be one closure

10:08 but there would be no shared holes in the environment, a huge plus for me

10:09 and no allocation for a box, so a true replacement for mutable fields

10:11 Chouser: From my experience teaching old time C++ coders to use JavaScript, that's actually the behavior for closures that they expect.

10:13 shoover: Jomyoot: (add-hook 'clojure-mode-hook (lambda () (paredit-mode 1)))

10:13 Chouser: would local mutables be mutable outside of a newnew?

10:14 I mean... (let [x (volatile 42)] (set! x 96)) would work?

10:16 rhickey: Chouser: yes, I think set! would become aware of volatiles

10:17 they would be as dangerous as Java volatiles, and have the same threading semantics

10:17 vs unadorned mutable fields in Java

10:17 Chouser: hm... extending let to do mutables feels almost worse to me than extending newnew to do interfaces.

10:18 it the only purpose is for mutable newnew fields, can they just be added to newnew?

10:19 rhickey: Chouser: but newnew works like this: (let [x 42 y 13] (newnew [Foo] [] (amethod [] ... x ... y))

10:19 Chouser: I was thinking the syntax to the newnew special form doesn't have to be pretty... between fn, defn, destructuring, etc. nobody would have to use newnew's own syntax directly.

10:20 rhickey: I know that's the plan, but it makes a hole: (let [x (volatile 42)] ..do-evil-things.. (newnew ...) ..more-evil-things..)

10:20 rhickey: Chouser: leveraging the concept of closure for fields is a nice unifying principle. Having a separate way to name fields makes for a duality similar to Java's and a lot of muck

10:21 Chouser: (let [x (volatile 0)] (while (not= x 20) (println x) (set! x (inc x)))

10:21 rhickey: yeah, ugly

10:21 Chouser: there! now I don't need to learn all this crazy high-order stuff.

10:22 but I guess if you're committed to that, you can do it with refs or atoms now.

10:22 rhickey: right

10:23 and higher-order has to win on its own merits, i.e. dotimes vs the above

10:23 Chouser: and it does, so I guess that's ok.

10:23 AWizzArd: What is the advantage of mutable locals (vs the use of atoms)?

10:24 rhickey: AWizzArd: they can't be shared

10:24 Chouser: surely somebody has played with building closures like your "unshared hole" idea. I wonder if there are pitfalls I'm not seeing.

10:25 rhickey: Chouser: people coming from Scheme/CL would definitely be confused by the lack of sharing

10:26 of course, and closure allows for sharing, and multiple calls to the same closure from multiple threads is a definite problem at the Java Concurrency in Practice level

10:26 and/any

10:26 Chouser: huh. I was thinking of (let [x (volatile)] (take 10 (repeatedly #(newnew ...)))) and how nice that they each get their own x

10:27 * danlarkin wishes I didn't have work to do and could instead follow this conversation

10:27 rhickey: Chouser: I agree totally, but that's not how it works today in Lisps, nor ever

10:27 Chouser: but I can see that it would be surprising for (let [x (volatile)] (newnew A ...) (newnew B ...)) to not share

10:27 rhickey: which is fine by me, that's why I left out mutable locals the old way

10:27 Chouser: esp. since the multiplt methods within A *would* share.

10:28 multiple

10:28 rhickey: Chouser: yes, but the whole (newnew) thing is kind of new, so expectations should be weak

10:28 Chouser: yes

10:29 rhickey: and volatile is a great disclaimer

10:29 watch out!!

10:29 dangerous explosive variable

10:29 Chouser: interesting to put it on the value rather than the local name. looks like any of the reference type constructors, but of course isn't at all.

10:30 rhickey: if you don't have JCIP-level skills in Java you should never use volatile in Clojure

10:30 jdz: ☣volatile☣

10:30 rhickey: but I do, and have to in order to write Clojure, and currently need to leave Clojure to do so

10:30 Chouser: I'm completely unsure about the syntax

10:30 Chouser: ok

10:31 rhickey: may even be (let-volatile [x 42]...

10:33 in any case, unshared volatiles are the only way I would consider them for Clojure, so looking for feedback about the confusion/expectation issues

10:33 It does fly in the face of the history of 'environments', but IMO they always had a faulty story re: values vs places

10:34 Chouser: Since the scope of the shared part is a single newnew, I do think it might be less confusing to have them declared syntactically inside the newnew but outside the method defs

10:34 AWizzArd: what is jcip?

10:34 danlarkin: AWizzArd: java concurrency in practice

10:35 AWizzArd: ah oki, thx

10:35 rhickey: Chouser: still trying to get them out of let? : )

10:35 Chouser: I understand the unifying argument, but unsure if I find it compelling.

10:36 yessir!

10:36 rhickey: I'd have to audit the code to see if there were any places in Clojure's implementation where I *needed* a mutable local and couldn't afford to allocate something like an AtomicReference. Likely...

10:36 Chouser: (let [x (volatile 1)] (set! x 2) (newnew ...))

10:37 that newnew object has a mutable field it can call x -- it starts with a value of 2?

10:37 rhickey: newnew's x starts at 2 and has an independent lifetime thereafter

10:37 Chouser: ok

10:38 rhickey: Expectations aside, I don't think the behavior is tricky

10:39 I just hate to have another concept for 'fields', but maybe we'll be forced into it...

10:40 basically, volatile is for experts only

10:40 Chouser: (newnew :extends AFn :mutable [x 42] (invoke [] (println "My x is " x))) ; just a pretend syntax for discussion

10:41 ((newnew :extends AFn :mutable [x 1] (invoke [] (set! x 2) (newnew ...)))) ; hey look, I did it anyway!

10:41 rhickey: exactly

10:42 Chouser: at the expense of uglier syntax in all cases, plus an extra allocation when AtomicReference was "unaffordable" :-P

10:42 AWizzArd: Btw, is all this mutable locals stuff, and newnew and so on, for rewriting Clojure in Clojure?

10:42 rhickey: and people aren't writing (newnew [AFn]...), they are writing (defn foo [x]), putting pressure to lift the :mutable to arglists

10:43 Chouser: what!? when would defn get a mutable?

10:44 rhickey: once you have a volatile in scope, how to deal lwith it in nested scopes becomes an issue. There really isn't a difference between a let body and a fn body and a class implementation in terms of privileges, necessity, safety or desirability of a mutable

10:45 Chouser: it wouldn't, but if the only was was in newnew, and people don't want to write (newnew :extends AFn etc ...

10:46 the only mutable was in newnew

10:46 Chouser: AWizzArd: I find it uncomfortable when a language provides features that I can't write myself in the language. This starts with types, so you get C++'s ability to define classes, overload operators, etc. Ruby takes you further with some ability to create flow control mechanisms. Lisps's macros of course are another whole category of power.

10:47 AWizzArd: Clojure today goes a long way because of macros, proxy, etc. but I still can't write, for example, finger trees that are competitive with Clojure's builtin collections without dropping to Java. I'd love to see that fixed. newnew (and mutable locals) would fix that.

10:49 rhickey: I assume an inner newnew would again get its own mutable -- you expect that to be undesirable?

10:49 rhickey: there are 11 volatiles in Clojure - all in ref types and caching seqs

10:50 Chouser: no, desirable, just not really different than volatile in let

10:50 there are some other mutable locals under the scope of synchronization, harder to search for

10:51 making them volatile would be harmless

10:52 Chouser: oh, right. yeah, the only argument I've got left is making the syntactic scope match the sharing scope.

10:52 that's probably sufficiently countered with the "expert-only" nature of the thing.

10:53 kylesmith: Would anyone care to help me with some performance optimization?

10:54 rhickey: Chouser: but if scopes matched like that we wouldn't have closures, we'd have to redeclare everything in each nested scope

10:55 I admit that inheriting a 'hole' is weird though

10:56 Chouser: hm, yes.

11:01 lisppaste8: Anniepoo pasted "untitled" at http://paste.lisp.org/display/83921

11:01 Anniepoo: I'm having noobie problems compiling to disk

11:03 Chouser: Anniepoo: your file must be named foo/edu/hhp/sl/automation/lessonPrimEditor/ui/LessonPrimPane.clj and foo must be mentioned in your classpath

11:04 Anniepoo: I believe both of those are true

11:04 AWizzArd: Chouser: I see. So, mutable locals are mutable objects with Java speed.

11:04 Anniepoo: I'm wondering where the repl in La Clojure is picking up classpath

11:05 Chouser: AWizzArd: a Clojure mutable local would be like a regular Java instance field

11:06 rhickey: if volatile was part of newnew, I'd move back to the school of 'closing over it captures its value' (immutably), or, perhaps, is disallowed (like in Java), and similarly you could copy it into an immutable binding and close over that

11:07 hamza: is there a way to create an array an set its elements initial value i am creating an 2d array thats 1000x1000 elements and loopint through it and settings all its elements to zero takes more than 30 secs?

11:09 Chouser: rhickey: ohh.. I wasn't thinking about inner newnews that were created via, say, #()

11:10 rhickey: so with let-volatile you might use #(set! x y) inside somewhere, while with newnew-volatile you'd have to do something more complicated inside.

11:10 ...most likely just avoid the higher-level function and do it with loop/recur

11:12 don't java arrays promise to be zeroed out when created?

11:13 hamza: thats what i thought but after creating checking some random indexes returns nil as their value

11:13 thats why i am trying to zero them.

11:14 AWizzArd: hamza: did you try to-array-2d?

11:15 Chouser: hamza: oh, you must have an array of Objects. Did you want an array of ints instead?

11:16 hamza: i use (def atest (make-array Integer/TYPE 1000 1000))

11:16 how can i create an array of ints?

11:16 Chouser: you've got it.

11:17 ,(aget (make-array Integer/TYPE 1000 1000) 5 5)

11:17 clojurebot: 0

11:18 hamza: thx when i tested this i got nil no idea why now i get zero

11:19 kk my prev. decleration was (make-array Integer 1000 1000) thats why i got nil.

11:20 Chouser: yep, that'd do it.

11:22 hamza: one more question at some point i have to iterate over this array for now i am using doseq to iterate is there a faster way to do this like using higher level functions?

11:23 Knekk: while we are on the subject, how would I crate an initialize an array of structures then?

11:24 Chouser: hamza: you might look at amap

11:24 Chousuke: amap clones the array though

11:24 Chouser: yes!

11:25 Chousuke: you probably want to avoid that if you're initializing it :P

11:25 Chouser: oh

11:25 well, I was answering hamza, not Knekk

11:25 Chousuke: oh, heh.

11:26 I didn't even realise there were two people talking about arrays :P

11:26 Knekk: I just piped in with a question

11:26 Chousuke: Knekk: an array of structs is just an array of Objects

11:27 Knekk: make-array takes an Object type, right, so I'd use (type structname) ?

11:28 Chousuke: hmm

11:28 I don't think that would work.

11:28 Knekk: well, it doesn't complain

11:28 Chousuke: you'd need (type (struct structname "some" "dummy" "vals"))

11:28 Chouser: anyway, I guess you're outside my realm of experience now. But the things to look at are amap, areduce, loop/recur, primitive locals, while keeping an eye on the mutable vector and pvmap, pvreduce stuff

11:28 Knekk: but I can't aset the array elements

11:29 Chousuke: Knekk: the structname has type clojure.lang.PersistentStructMap$Def

11:29 the actual struct type is probably different.

11:29 ... a PersistentStructMap, I would guess :P

11:30 Knekk: yeah

11:30 Chousuke: repl agrees: user=> (defstruct b :foo) user=> (type b) clojure.lang.PersistentStructMap$Def user=> (type (struct b 'foo)) clojure.lang.PersistentStructMap

11:32 Knekk: ah, better

11:33 thanks

11:38 mebaran151: I'm still not used macros: is there a good guide online how to write them

11:39 astoddard: I have an absolute beginner question. Using clojure.contrib.repl-utils/source, I get a copyright message and (ns clojure.core) nil. e.g. user => (source identity). Does this mean no source is available?

11:40 Chousuke: hm

11:40 Chouser: astoddard: wierd. Usually you'd see "Source not found" if that's the case.

11:41 ,^#'identity

11:41 clojurebot: {:ns #<Namespace clojure.core>, :name identity, :file "clojure/core.clj", :line 841, :arglists ([x]), :doc "Returns its argument."}

11:41 Chouser: astoddard: what does yours say?

11:42 astoddard: I get {:ns #<Namespace clojure.core>, :name identity, :file "clojure/core.clj", :line 0, :arglists ([x]), :doc "Returns its argument."}

11:42 Chouser: line 0 is the problem there. what version of clojure do you have?

11:43 astoddard: I am using the most recent clojure downloaded with the "Programming Clojure" book. Apparently Clojure 1.1.0-alpha-SNAPSHOT

11:44 Chouser: astoddard: link?

11:44 astoddard: http://www.pragprog.com/titles/shcloj/source_code

11:46 The repl is also started with a batch file included with the download. Is that likely a problem?

11:46 Chouser: a batch files shouldn't hurt anything

11:47 lisppaste8: Anniepoo annotated #83921 "untitled" at http://paste.lisp.org/display/83921#1

11:48 Anniepoo: I made a little GUI as a first project. I'm moving that to a new IntelliJ project to start something real

11:49 the ns throws MethodNotFoundError for this unless I remove the :use clause

11:51 should I not be :use ing clojure.core and clojure.contrib.seq-utils?

11:51 Chousuke: you don't need to :use clojure.core

11:51 Chouser: no need to :use clojure.core -- that happens automatically

11:52 Anniepoo: maybe paste the whole stack trace? Can't think why you'd get MethodNotFound

11:52 Anniepoo: ok, hang on

11:53 Chouser: astoddard: hm, I see the line 0 issue for that download of clojure as well...

11:53 lisppaste8: Anniepoo annotated #83921 "untitled" at http://paste.lisp.org/display/83921#2

11:54 Anniepoo: that's the trace after removing clojure.core

11:54 Chouser: I remember we were getting line 0's for a while, but I don't remember for which versions of Clojure

11:55 Anniepoo: I'm running off the 'clojure-1.0' that downloads when you download La Clojure

11:55 before I was running on a direct d/l of Clojure off the website

11:56 Chouser: Anniepoo: you've got clojure.jar in your classpath twice

11:56 not sure that's the cause of your problem, but it's not recommended.

11:56 Anniepoo: hmm... wonder why

11:56 ok, I'll figure out who'se giving it to me

11:57 Chouser: two different paths

11:58 Chousuke: It looks like you don't have the classes/ dir in the classpath either.

11:58 astoddard: Chouser: I just tried using the clojure-contrib.jar downloaded with the book together with the clojure-1.0.0.jar. source works properly.

11:59 Chouser: astoddard: ok

11:59 astoddard: you might want to email Halloway, or the google group

12:00 astoddard: Will do, should let them know that the book and the supplied source don't work together.

12:00 Thanks for the help.

12:02 Anniepoo: thanks for spotting that. It was indeed the problem. the IntelliJ UI is confusing - there's a checkbox I'd assumed was 'I want this', it's actually 'add this to the jars'

12:04 AWizzArd: astoddard: how many pages does the pdf have?

12:11 Anniepoo: whoo hoo, small victories, extended a java class with clojure

12:12 is there an equivilent for the java import foo.mep.*; idiom?

12:13 Chouser: Anniepoo: no. it's exclusion is a feature. :-)

12:14 Anniepoo: I'm aware it leads to stylistic horrors in java. But there's a style argument on the other side as well

12:15 I'm cut and pasting code to make something new, and currently there's a mass of imports

12:16 the old code used a bunch of swing borders and weird stuff, the new probably won't

12:16 Chouser: not sure of the controlling argument, but I think it's been pretty well decided. One point is that there are some planned performance features that will benefit if you have a minimal set of imports

12:16 Anniepoo: ah

12:16 that sure trumps me

12:18 my arg would have been that, in the absence of IDE support for automatic cleanup, you tend to accumulate a bunch of 'lies' over time

12:19 Chouser: yeah. Maybe IDEs need to support cleanup. :-)

12:19 Anniepoo: if I see code that imports javax.swing.* i don't assume I'm actually using EtchedBorder somewhere

12:21 Chouser: Anniepoo: Personally I really dislike the * -- if I don't have intimate knowledge of all the packages being * imported, I have no idea where any given classname in the code is coming from.

12:21 Anniepoo: ok, it's Sergey's fault ;c)

12:21 yes, I understand the argument

12:22 Chouser: I understand your argument about lies, but I guess I find it easier to do some searches when cleaning up the code to remove extra imports than to resolve non-specific imports to get my code working in the first place.

12:22 Anniepoo: well, having the IDE do the cleanup is the right solution

12:23 with modern IDE's we should all be thinking more about literate programming

12:32 Fossi: is anybody using lancet seriously?

12:33 Raynes: Fossi: Apparently, it's not ready for extensive use yet. So I don't think so.

12:34 Someone needs to pull Stuarts lever and get him working on it. :D

12:34 Fossi: either that or work on it themselves as they go ;)

12:35 technomancy: Raynes: we might have to wait for the 2nd edition of Programming Clojure for that

12:35 Raynes: If I was more experienced with build tools, I'd extend it myself. :\

12:38 technomancy: well corkscrew is only 200 LOC; very easy to understand. =)

12:39 Fossi: lancet is 139 so far

12:40 technomancy: oh snap!

12:40 well 200 includes comments and newlines

12:40 Fossi: :)

12:40 technomancy: and it handles dependencies

12:41 Fossi: why did you start corkscrew? didn't like lancet?

12:41 technomancy: which happens to be the only non-trivial part of building clojure projects

12:41 Fossi: stuart told me lancet was mostly for learning rather than actual use

12:41 Fossi: ah, ok

12:42 is corkscrew a wrapper over maven or such?

12:42 technomancy: it uses maven to resolve dependencies

12:42 it can also check out dependencies from svn or git

12:43 Fossi: well, if stuart has no intent to put lancet forward, maybe i should use corkscrew then

12:43 technomancy: the ant approach of "XML is just another programming language; wheeee!" is very disturbing

12:43 Fossi: although i like ant better, but maven's dependency stuff is nice

12:43 yeah, that's why i like lancet so much

12:43 technomancy: Fossi: if you did, you would be like the fifth corkscrew user ever; so be aware it's not very polished. =)

12:44 Fossi: kinda makes the worst part of ant better

12:44 technomancy: well isn't that all there is to ant though?

12:44 isn't it just an engine for converting XML into action?

12:44 Fossi: more or less ;)

12:45 technomancy: to me the only interesting thing about that whole mess is the dependency system

12:45 *that whole mess being Java build systems in general

12:46 Fossi: yeah. it really is a big mess

12:47 cemerick: rhickey: FWIW, the mutable locals idea you floated ~10:30 EDT sounds nifty to me.

12:55 hserus: i've heard ivy is a better dependency management system than maven

12:57 technomancy: why's that?

12:58 they use the same repository format underneath; that's all I care about.

12:58 better for building Java projects is irrelevant to me. =)

13:00 hserus: am trying to find a mail from a colleague comparing these two.. actually a lot of my colleagues were very vary about it

13:01 i've generally stuck with ant though

13:16 mebaran151: how would I write a macro that could pass its an argument to its expressions? I'm trying to write something that works like (with-cursor [cur] ;;exprs that use cur as though it were part of a function definition.

13:19 Chouser: mebaran151: the normal way to do that now is with dynamic binding

13:19 (binding [*cursor* my-value] ...)

13:19 functions called from in there can use *cursor* and will get the value my-value

13:19 mebaran151: but binding seems like an ugly globals hack

13:20 I'd like it if they could define their own name for it

13:20 Chouser: it's only a global name, not a global value

13:20 oh

13:20 mebaran151: so it would look like I just took out the fn [cur]

13:20 Chouser: I must not understand what you want

13:20 mebaran151: I had a simple defn that simply took an argument of one function and passed it the cursor

13:21 (with-cursor (fn [c] ... do stuff with the cursor)

13:21 I think it would be nice if I could do (with-cursor [c] (exprs that reference c, which will receive the cursor)

13:22 Chouser: oh, sure... so you just want a macro that you call like (with-cursor [c] body) which emits (with-cursor* (fn [c] body)) ?

13:23 mebaran151: yeah

13:23 something like that

13:23 actually

13:23 what I really need

13:24 is something that will take ([arg] a b c) and turn it into (fn [arg] a b c)

13:25 Chouser: you can use #(a b c %)

13:25 mebaran151: how would that work with my ~exprs clause

13:26 Chouser: if you have a with-cursor fn that takes a single fn as an argument, you can call it like (with-cursor #(a b c %))

13:26 mebaran151: but would that work inside the macro that is constructing the function?

13:27 Chousuke: though with-cursor is really not a good name if it doesn't establish some kind of a context.

13:27 Chouser: if you have a macro, I don't see why you'd need #()

13:27 gulagong: hi there

13:27 i have a little problem, and i think you can help ;)

13:27 i want to create an array (create-array default-value dim1 dim2 dim3 dim4 ....)

13:27 and i dont know how to make a recursive call with the rest of the dimension parameter (rest dimensions).

13:27 maybe i'm a fool but you help me anyway ;)

13:28 mebaran151: I guess I should call it cursor-action

13:28 Chousuke: mebaran151: how is your cursor-action any different from a normal function that takes a single parameter, though? :/

13:28 Chouser: gulagong: you've got a list or vector of the dimension sizes? (let [dims [10 10 10 5]] ...) ?

13:29 mebaran151: Chouser, I'm trying to get rid of the fn thunk

13:29 gulagong: i did that ...

13:29 but i want (defn create-array [default & dims] ....)

13:29 Chouser: gulagong: ok, then maybe (apply make-array type dims)

13:29 mebaran151: right now as a normal defn it looks like (cursor-action [f] where f is a function of one argument that will be passed the cursor

13:30 gulagong: thx :)

13:30 now i see ....

13:30 so simple ;)

13:30 mebaran151: so when I call it, it looks like (cursor-action (fn [c]))

13:30 Chouser: mebaran151: sure, or (cursor-action #(.... % ))

13:31 mebaran151: I'd like to get rid of that call to fn so it would look like (cursor-action [c] (exp1) (exp2) (exp3))

13:31 I know it's not essentially

13:31 but it would be a nice bit of syntax sugar

13:31 Chousuke: mebaran151: ah, so you want just (defmacro cursor-action-macro [c-name & body] `(cursor-action (fn [~c-name] ~@body)))

13:31 mebaran151: yeah

13:31 what does the @ do?

13:31 Chousuke: that way you can do (cursor-action-macro cursorname (do-stuff-with cursorname))

13:32 mebaran151: exactly

13:32 Chousuke: mebaran151: it's the splicing unquote

13:32 Chouser: or (defmacro cursor-action-macro [& fn-stuff] (cons `cursor-action fn-stuff))

13:32 mebaran151: I see

13:32 so I'd still need my basic version

13:32 which takes a raw-fn

13:32 cursor-action-raw or something

13:32 Chouser: mebaran151: that's the best way to do it, yes

13:32 Chousuke: mebaran151: you don't need it, but it's easier that way

13:33 mebaran151: call it cursor-action*

13:33 mebaran151: okay

13:33 Chousuke: with Chouser's version, the syntax will be (cursor-action-macro [c] stuff) which is a bit more clojurey I guess.

13:34 hm.

13:34 wait. it's actually missing the fn :p

13:34 Chouser: heh. oops.

13:35 sorry

13:36 (defmacro cursor-action [& fn-stuff] `(cursor-action* (fn ~@fn-stuff)))

13:41 slime continues to make clojure harder for people to learn

13:42 "deong: Pressing “1″ is no longer necessary in recent versions of swank-clojure" ... ah, that's fantastic.

13:42 (re: http://nflath.com/2009/07/initial-thoughts-on-clojure/ )

13:43 technomancy: way to go! :-)

13:43 mebaran151: thanks Chouser Chousuke!

13:43 that works perfectly

13:44 Chouser: mebaran151: power to the programmer! :-)

13:48 technomancy: Chouser: verbose stack traces are there whether you use slime or not. =(

13:48 mebaran151: is there a step debugger for clojure?

13:48 Chousuke: I have heard that java debuggers work.

13:49 I haven't tried myself :P

13:49 Chouser: technomancy: yes, but slime for a long time has hidden all but the most recent "cause", which as noted in that blog sometimes leaves out important info

13:49 I'd rather have too much info than not enough (though of course exactly the right amount would be better than either extreme)

13:49 technomancy: Chouser: ah right, sure.

13:49 yes, swank-clojure very much has the "my first clojure project" feel to it at times.

13:49 Chouser: heh

13:49 technomancy: working on that one step at a time. =)

13:50 drewr: I wish jochu hung around here so we could chat about it

13:50 we've emailed a little but that's about it

13:50 technomancy: dimming the irrelevant lines in the traces helps a lot

13:51 drewr: he's basically passed on maintenance to me

13:51 anything you want to talk about? =)

13:51 Chouser: what's the logic? dim everything in the clojure namespace?

13:51 drewr: technomancy: good to know :-)

13:51 sounds like I need to use your fork then

13:51 technomancy: Chouser: clojure.(lang|core), swank.*, and java.*

13:51 that last one may be overzealous. =)

13:52 drewr: he gave me commit rights to his, so I keep them in sync

13:52 drewr: great

13:54 technomancy: I guess I should look at the github network; lots of people have been hacking

13:54 drewr: the only issue I'm having at the moment is the swank-clojure-find-package regexp, which I need to work on

13:55 technomancy: drewr: check out swank-clojure-import I just committed

13:56 it prompts you for a class, then searches the classpath and adds the proper :import declaration

13:58 mebaran151: I would be nice if clojure had import ns.* decls

13:58 any technical reason it doesn't?

14:01 Chouser: mebaran151: it's exclusion is a feature. :-)

14:01 not sure of the controlling argument, but I think it's been pretty well decided. One point is that there are some planned performance features that will benefit if you have a minimal set of imports

14:01 mebaran151: ah

14:02 Chouser: Personally I really dislike the * -- if I don't have intimate knowledge of all the packages being * imported, I have no idea where any given classname in the code is coming from.

14:02 mebaran151: the other downside though, is in an overengineered library like BerkeleyDB

14:03 import every factory, factory factory, and wrapper factory factory can get very tedious, and usually, in this case, it's very clear where the class comes from

14:03 cemerick: mebaran151: totally off-topic, but what do you think of BDB?

14:03 mebaran151: cemerick, it's pretty fast, well constructed

14:04 I'm gonna put some nice bindings up on github soon that wrap it's transactional stuff right

14:04 it's safter than TokyoCabinet (it always gets the locking right...)

14:04 I think it's a pretty good fit for serializing clojure maps, plus you could index by an arbitrary key

14:04 cemerick: I pinged oracle for licensing info, and got a pretty laughable response.

14:04 Chouser: safer or faster?

14:05 mebaran151: safer

14:05 not faster

14:05 Chouser: hm.

14:05 ok, thanks.

14:05 mebaran151: being pure java is nice

14:05 though with the way I'm designing this library, plugging in TokyoCabinet would be trivial

14:06 drewr: technomancy: fwict, it gets the ns sexp but doesn't isolate the namespace name

14:06 mebaran151: you just have to some how give me a cursor and database handle, and a way to open a transaction, and the rest of the code just falls into place

14:06 cemerick, BerkeleyDB is GPL no?

14:07 cemerick: mebaran151: not GPL, a custom copyleft license. Doesn't work for a commercial project, tho.

14:07 mebaran151: when I read the license, they said if you keep it on your own server, it's your own business

14:07 otherwise you have to give them the deed to your house

14:07 cemerick: yeah, we're redistributing/embedding

14:08 mebaran151: ah, it would actually be pretty good for that

14:08 were you using the C edition or the JE edition?

14:08 cemerick: that's what I figured, but they're not making it easy

14:08 JE

14:08 Chouser: cemerick: maybe they'd rather you embed Oracle

14:09 mebaran151: tokyocabinet is actually pretty light

14:09 it might work for embedded situations

14:09 cemerick: Chouser: it's a desktop app, of all things :-)

14:09 mebaran151: it's good if you don't highly multithread your app

14:09 Chouser: cemerick: really? does "desktop app" mean a browser based app that looks like a desktop?

14:10 mebaran151: but under lots and threads, the TokyoCabinet really expects you to put it behind the tokyo tyrant server

14:10 cemerick: TokyoCabinet is *only* GPL, so that's right out. (the travails of being commercial! :-P)

14:10 mebaran151: only, I thought it was LGPL...

14:10 cemerick: Chouser: nope. Desktop, netbeans platform, etc.

14:11 mebaran151: it's LGPL

14:11 cemerick: mebaran151: ah-ha, you're right

14:11 mebaran151: just checked the webpage

14:11 cemerick: hrm, I'll have to take a look

14:11 mebaran151: that means you can commercial use it as long you don't hide the patches?

14:11 for a server side project thought

14:11 TC likes to wedge itself

14:11 and deadlock itself

14:11 cemerick: hrm, that's unfortunate

14:11 mebaran151: and corrupt itself under high thread load

14:11 BDB hasn't had any of those problems, though it's slower

14:12 cemerick: LGPL is fine for redistribution. Changes have to also be LGPL.

14:12 mebaran151: yeah

14:12 TC is a pretty friendly license and it comes with fulltext search built-in

14:12 cemerick: mebaran151: doesn't sound like much of an endorsement from you, though ;-)

14:13 mebaran151: if you don't need to really pound it with a lot of concurrent access, it's great

14:13 see, he wrote a nifty server component for those situations

14:13 technomancy: drewr: huh; well I guess I don't have funky enough ns declarations to mess it up

14:13 let me know what you find though

14:14 cemerick: hrm. I think we'd be just fine serializing all writes, actually.

14:14 mebaran151: Tokyo Tyrant is a class act, but I didn't want the extra deployment headache, at least righ tnow

14:14 cemerick: we use couch in-house, mostly because it's so damn easy to use.

14:15 technomancy: it's easy to get spoiled by couch. =)

14:15 cemerick: one of these days, we'll get around to writing a clojure view server

14:15 mebaran151: the Tokyo Tyrant server api though really is a joy

14:15 clojurebot: the website api refers to last release

14:15 technomancy: cemerick: you've seen the one on github?

14:16 drewr: technomancy: meta screws it up; e.g. http://github.com/richhickey/clojure-contrib/blob/6fea921c91d453c750586c960ec92caacf4fa5e6/src/clojure/contrib/def.clj

14:16 cemerick: technomancy: no -- is it danlarkin's?

14:16 technomancy: cemerick: no, it's called clutch iirc

14:17 drewr: the other way to solve that would be to make ns support docstrings. =)

14:17 though I guess we'd still need to update swank for that

14:17 drewr: tru dat

14:18 cemerick: technomancy: thank you for the tip!

14:19 technomancy: cemerick: well, I haven't actually _used_ it. =)

14:19 worth knowing about though.

14:19 Chouser: mebaran151: which version of TC are you talking about?

14:19 cemerick: technomancy: definitely worth a look. We're deep into swing of all things right now, but that should change next month.

14:23 Lau_of_DK: Very big problem: We have a clojure project at work, which we have been building for quite some time. In essense it traffics data between ftp sites and parses/interprets the data. This project was moved into the testing fase today. In doing that we changed the adresses from local ftp sites to external ones. When we started a REPL and loaded the code = No problems, then we ran it = Exception thrown. Then I manually evaluted the function that was t

14:25 Chouser: Lau_of_DK: that cut off at "evaluted the function that was t"...

14:25 Lau_of_DK: Then I manually evaluted the function that was throwing the error, without changing any code, then whole program executed flawlessly. What could be the cause of this?

14:25 hiredman: Lau_of_DK: my quick fix for networking stuff is always to pass -Djava.net.preferIPv4Stack=true to java, dunno if that will help you

14:26 Lau_of_DK: is this all on the exact same jdk?

14:26 Lau_of_DK: Yep - Same system, same jdk, same everything

14:26 Chouser: what exception?

14:27 krumholt: are you using namespaces? did evaluating the function in a different namespace change something?

14:27 cemerick: oh, TC doesn't build on windows. There's a fork, but, ugh.

14:27 Lau_of_DK: It was the sftp function which runs "ls" on the remote server which returned "No such file" - This should not (and have never) occur when the directory is non existant or empty

14:27 krumholt: Yes - Evaluating it in the same namespace fixes the problem, so we didnt try in others

14:28 krumholt: Lau_of_DK, are you using comparison of symbols? not exactly sure if that could cause the problem

14:29 Lau_of_DK: Not to my knowledge

14:30 krumholt: is it possible have a look at the function that caused the problem?

14:31 Lau_of_DK: sec

14:31 I have to switch to vpn

14:31 hiredman: Lau_of_DK: is this always running against the same ftp daemon?

14:33 I've never rolled an ftp client myself, but I hear the protocol has a lot of leeway so maybe you are running into a slight difference in behaviour somewhere

14:33 Lau_of_DK: hiredman: Yes

14:34 Still there?

14:34 hiredman: ~ping

14:34 clojurebot: PONG!

14:34 Lau_of_DK: hiredman: How would that account for code not working on first evaluation, and then working again on the second?

14:34 (defn download-messages

14:34 []

14:34 (with-sftp-connection c

14:34 (let [files (debug "Retrieving list of messages" "OK" (get-messages c))

14:34 edifacts (map #(debug (str "Downloading: " %) "OK"

14:34 (download-message c %)) files)]

14:34 files)))

14:34 This is the bit that breaks down

14:35 Chousuke: hm

14:35 Lau_of_DK: And the part that fails is (debug "Retrieving...") which is a macro that works exactly like Time.

14:35 hiredman: erm

14:36 Chousuke: laziness issue? :P

14:36 Lau_of_DK: If I then manually execute the body (get-messages c) then it works

14:36 hiredman: yeah, nothing is forcing the map

14:36 Lau_of_DK: I tried it in a doseq, that didnt make a difference, its not the map thats dying

14:39 Chouser: unless you specifically want edifacts to be lazy, I'd stick a (doall ...) around it just to be sure

14:40 Lau_of_DK: I agree, and thats done in other parts, but thats not the problem right now, its the (get-messages c) thats failing, so any input why that might be? And why a second evaluation then fixes it ?

14:41 Chousuke: It would be useful to see the debug macro, and the get-messages function :P

14:41 krumholt: Lau_of_DK, i am assuming debug is just loging something and then evaluating (get-messages c) ?

14:41 Lau_of_DK: krumholt: yep, debug is exactly like the time macro in core

14:41 Chouser: did you paste the exception somehwere?

14:41 Lau_of_DK: (defn get-messages

14:41 [channel]

14:41 (doall

14:41 (for [line (.ls channel (str edi/*remote-read-path* '* edi/*ext*))]

14:41 (.getFilename line))))

14:41 No, I just told you what it said - I dont have it here

14:42 cschreiner: Hello all

14:43 I have a question regarding debugging clojure inside emacs using slime. How do you do it?

14:44 hiredman: Lau_of_DK: so get-messages is throwing the exception?

14:44 Lau_of_DK: As I recall

14:45 krumholt_: Lau_of_DK, is edi your namespace?

14:45 Chouser: Lau_of_DK: if at all possible, I'd recommend getting the actual stack trace -- can be quite valuable as a debugging aid.

14:45 Lau_of_DK: Yep

14:45 Chouser, I'll dig it up

14:45 Chousuke: any chance get-messages is getting evaluated twice?

14:46 Lau_of_DK: Chousuke: What are you thinking?

14:47 Chousuke: well, a macro is involved

14:47 * Chousuke is just exercising his divination skills

14:47 Lau_of_DK: No, there's not chance

14:49 Chousuke: then my last remaining guess is that you're passing in an invalid connection or something

14:52 Lau_of_DK: No it connects just fine

14:52 the log says "Connecting" "Connected" "Retrieving list of files", and then it borks

14:52 or rather, hangs there

14:53 krumholt_: Lau_of_DK, for me it sounds namespace related why not (str edi/*remote-read-path* "*" edi/*ext*) instead of (str edi/*remote-read-path* '* edi/*ext*) ? i don't really know if that can be wrong

14:55 mebaran151: cemerick, one thing I just noticed glancing at the TokyoCabinet docs again

14:55 the iterator is not thread safe global, which is probably why I experienced so many issues

14:55 if you lock on that, everything work out

14:56 hiredman: Lau_of_DK: NPE?

14:58 I would add a (prn line) before the (.getFilename line)

14:58 Lau_of_DK: Chouser: I cant get that stack-trace, because the systems are hardcoded to accept certain IPs, mines not one of them, so I'm only getting "Auth fail"

14:58 But I just dont understand the logic - How can you evaluate a function and have it fail, then evaluate the exact same code, and have it run fine ?

14:58 hiredman: or even (prn (class line))

14:58 well obviously something is not the same

14:59 Chouser: Lau_of_DK: lots of ways: different dynamic bindings, different threads, different lazy realizations

15:02 cemerick: mebaran151: after looking at it for a bit, my biggest initial issue is the lack of windows support in the official codebase

15:03 having full-text indexing builtin is pretty compelling, though. I may give the windows-compatible fork a roll.

15:05 angerman: is there a default approach to create a drawable area with clojure?

15:05 hiredman: uh

15:05 just use swing?

15:05 angerman: I'd like some kind of a panel on which I can draw and use the repl to interactibly play with it.

15:06 hiredman: clojurebot: swing?

15:06 clojurebot: Pardon?

15:06 hiredman: clojurebot: swing is <reply>http://java.sun.com/docs/books/tutorial/uiswing/

15:06 clojurebot: Ack. Ack.

15:06 angerman: hiredman: so like creating a JFrame adding a JPanel and owerwriting it's draw-something method with a global Graphics2d element?

15:06 hiredman: clojurebot: swing?

15:06 clojurebot: http://java.sun.com/docs/books/tutorial/uiswing/

15:07 hiredman: angerman: sure

15:07 cemerick: angerman: there's also processing (with the clj-processing wrapper), which is very pleasant

15:09 angerman: cemerick: well I basically just need stuff like drawing lines. And I have no experience with processing

15:11 mebaran151: cemerick, that fork is so broken

15:11 that was one of the reasons we switched: I have a dev on windows and it made him cry

15:11 cemerick: oh, bummer

15:11 mebaran151: that fork is unspeakably broken

15:12 cemerick: I guess I'll stop reading about all the greatness of TC

15:12 mebaran151: and compiling it was insane

15:12 angerman: TC?

15:12 cemerick: I've long wanted to have FTI baked into an embedded DB, but I've never seen a good impl.

15:12 mebaran151: if you need something like a key value store

15:12 cemerick: angerman: Tokyo Cabinet

15:12 mebaran151: qdbm might do it for you

15:12 angerman: though that

15:12 just got it working yesterday

15:13 and played a bit with the tabledb

15:13 cemerick: well, I presume that qdbm is essentially legacy, given TC

15:13 angerman: qdbm?

15:13 * angerman is confused

15:14 angerman: but yes. Compiliing TC on OS X was insanely stupid. Especially as I wanted a different prefix and OS X Java 1.6 is 64bit

15:14 cemerick: angerman: http://www.google.com/search?q=QDBM

15:15 angerman: cemerick: yes probably

15:19 mebaran151: qdbm is windows okay

15:20 angerman: http://gist.github.com/151535

15:20 if anyone's interested on installing TC on os x

15:23 mebaran151: ports are your friend

15:25 angerman: mebaran151: sometimes :)

15:26 mebaran151: oh you mean to get java to work

15:26 angerman: mebaran151: well. ports are fine as long as you follow the ports setup

15:27 mebaran151: java on Mac OS X is mostly a mess anyway

15:28 angerman: well ...

15:28 mebaran151: cemerick, I've actually had qdbm work with ruby bindings before on windows

15:28 so if you don't need super performance, it's probably sufficient

15:42 rapido: anyone interested in a immutable btree implementation? (to replace tokyo cabinet and to be in line with the clojure spirit)

15:48 mebaran151: rapido, that persists to disk?

15:48 disk persistence is tricky

15:49 rapido: mebaran151: yes, to disk, like berkeleydb

15:50 mebaran151: but much simpler, because of immutability and append-only properties.

15:50 hiredman: I think using an agent+watcher would work pretty well

15:51 rapido: hiredman: my btree implementation has another interesting property: it is canonical

15:52 mebaran151: rapido, really?

15:52 rapido: the order of insertions doesn't matter: it will always create the same tree

15:52 mebaran151: but often in the life of an app, you may have to delete something, else your data becomes explosive

15:53 is there a way to prune older versions (it would be great to have a btree that at any given time you could access its state from the last n places)

15:53 rapido: mebaran151: you can always 'garbage collect' - you just have to explicitly state which trees are 'root' (reachable) and garbage collect

15:54 mebaran151: ah, that would actually be pretty neat

15:54 but wouldn't garbage collection involve rebuilding the whole tree, or would it be a simpler op than that?

15:55 rhickey: rapido: sure, where is it?

15:56 mebaran151: also berkeleydb has pretty good memory caching builtin

15:57 rapido: rhickey: i just finished version 0.1 (in java) and it is a little bit rough around the edges

15:58 rhickey: i'm still experimenting with different page-sizes.

15:59 currently, what works is that i make the bottom nodes the fattest (4k) and the top nodes the leanest (binary node)

16:00 this scheme works because when you 'mutate' a immutable tree, a lot of top nodes are created, while the bottom nodes pretty much stay as they are

16:02 but the property that they are canonical brings additional savings.

16:02 you can do O(1) equality checks (probabilistically)

16:04 i guess i have to write a small paper to explain the workings - and then release the code.

16:05 rhickey: rapido: sounds neat - let us know when we can try it

16:07 rapido: sure, i'll let you know - i think this stuff fits clojure like a glove.

16:32 ericthorsen: Anyone know what clojure.core/import* is? I see this in the import macro in the latest core.clj ... wondering if this is a typo?

16:34 Chouser: ericthorsen: import used to be a function, I believe, and is now a macro that calls import*

16:34 hm

16:34 hiredman: ericthorsen: a lot of things that are "special forms" are really macros that use something even more primitive, the more primitive form has the same name but suffixed with a *

16:35 ericthorsen: Chouser: ok...I may just have some build problems...

16:35 Chouser: yeah, scratch that. import was a special form and is now a macro

16:35 Anniepoo: I've got a fragment of a paintComponent method (.fillRect g (.getBounds this))

16:35 ataggart: anyone know if we get some kind of notification once our CA has been received?

16:36 Anniepoo: this is a bug because, oops, fillRect takes 4 args, x,y,w,h

16:36 Chouser: ataggart: your name should eventually show up on http://clojure.org/contributing

16:36 ataggart: looks like you're there

16:36 ericthorsen: Chouser: I found it in Compiler.java...my jars must be out of sync???

16:36 ataggart: aha! excellent, thanks

16:37 Chouser: ericthorsen: could be -- clean and rebuild?

16:37 hiredman: Anniepoo: ?

16:37 ericthorsen: Chouser: yup...I'm playing with your clojure-compiler

16:37 Chouser: ericthorsen: oh! my.

16:37 hiredman: Chouser: look what you did!

16:37 Anniepoo: is there some idiomatic way to fix this? I see a few ways to do it, but wonder if there's an idiom since

16:37 Chouser: ericthorsen: it's not really ready for that...

16:38 Anniepoo: the fix feels clunky

16:38 hiredman: Anniepoo: which fix?

16:38 ericthorsen: Chouser: Well, I was doing some code analysis for the enclojure plugin and saw that you did so much more already...I was hoping I could use some of that work

16:39 Chouser: ah. well, perhaps then. Just don't expect it to, you know, compile anything. :-)

16:39 ericthorsen: did you consider using Clojure's own Compiler.analyze?

16:39 Anniepoo: well, in this case the x,y,w,h have individual accessors, I could call them, or I could do the .getBounds once into a let variable

16:39 ericthorsen: Chouser: I just need to analyze part for now

16:40 Anniepoo: just wondering what's more idiomatic

16:40 Chouser: Anniepoo: if a method returns an object the other method simply can't accept, you've run into an ugly little corner of the Java API

16:41 hiredman: Anniepoo: I imagine most idiomatic would be to wrap the call to .fillRect in a multifn

16:41 Anniepoo: and wanting some (bar (foo .getBlah .getMep .getGort))

16:41 that is the same as (bar (.getBlah foo) (.getMep foo) (.getGort foo))

16:41 hiredman: that takes either X or x y z w

16:42 Chouser: (bean obj) ?

16:43 hiredman: (defn fill-rect ([g this] (let [foo (.getBounds this)] (fill-rect g (.getX foo) (.getY foo) (.getWidth foo) (.getHeight foo)))) ([g x y w h] (.fillRect g x y w h)))

16:43 something like that

16:44 Anniepoo: yah

16:44 Chouser: (map (partial get (bean obj)) [:x :y :w :h])

16:45 Anniepoo: what's bean?

16:45 Chouser: ,(doc bean)

16:45 clojurebot: "([x]); Takes a Java object and returns a read-only implementation of the map abstraction based upon its JavaBean properties."

16:45 Anniepoo: chouser, you're on the right track here

16:45 kotarak: There are functions I really never use.... That's scary. I probably miss a lot...

16:46 Chouser: hm... but if you're then passing to a method you can't use apply anyway, so ...

16:47 Anniepoo: ah, ok

16:47 Chouser: ericthorsen: clojure.contrib.repl-utils/expression-info uses Compiler.java to analyze a Clojure expression. That's more likely to be correct than anything in clojure-compiler

16:47 technomancy: that definition of bean is not very helpful for people who have never heard of a JavaBean

16:48 * technomancy had no idea what it meant until he saw it used in code

16:48 Anniepoo: much happier with all the suckiness moved off onto Java

16:48 Chouser: ,(bean 5)

16:48 clojurebot: {:class java.lang.Integer}

16:48 ericthorsen: Chouser: I'll take a look at that.

16:48 Anniepoo: me wishes I'd never heard of a javabean

16:48 * technomancy considers himself lucky, I suppose

16:49 Chouser: ,(.invoke (first (filter #(= (.getName %) (str "intern")) (.getMethods clojure.lang.Var))) clojure.lang.Var (to-array [*ns* 'expression-info (fn [expr] (let [fn-ast (clojure.lang.Compiler/analyze clojure.lang.Compiler$C/EXPRESSION `(fn [] ~expr)) expr-ast (.body (first (.methods fn-ast)))] (when (.hasJavaClass expr-ast) {:class (.getJavaClass expr-ast) :primitive? (.isPrimitive (.getJavaClass expr-ast))})))]))

16:49 clojurebot: #'sandbox/expression-info

16:49 Chouser: oops sorry

16:49 ,(expression-info '(let [x 5] x))

16:49 clojurebot: {:class java.lang.Integer, :primitive? false}

16:50 Chouser: hiredman: that was honestly an accident -- meant to paste to another window. sorry!

16:51 hiredman: *shrug*

16:51 I wouldn't have noticed if you didn't saying anything

16:51 Chousuke: wait, what

16:51 hiredman: that is gnarly looking

16:51 Chouser: shhhh

16:51 clojurebot: Who??

16:51 hiredman: what??

16:52 I meant Chousuke there

16:53 Chousuke: That just demonstrates that a blacklist is never enough :P

16:54 mebaran151: is it possible to define a multimethod that has a possible if it is fired without any arguments

16:54 Chouser: who needs special forms?

16:54 mebaran151: yep

16:54 mebaran151: oh how?

16:54 hiredman: what?

16:54 mebaran151: sorry for the typo'ing

16:55 hiredman: I did not actually understand that

16:55 mebaran151: a multimethod that has an action when called without arguments

16:55 hiredman: but I guess Chouser did

16:55 oh, sure

16:55 mebaran151: I gotta pay attention to my edits more

16:55 Chousuke: mebaran151: just have the dispatch function accept any number of arguments

16:55 then return some value for the no-args case

16:55 Chouser: (defmulti foo (fn [& args] (if args (type (first args)) :no-args)))

16:56 (defmethod foo :no-args [] "none")

16:56 or something

16:56 kotarak: (defmulti foo (fn [& args] (if (zero? (count args)) :zero :bla))) (defmethod foo :zero [] ...)

16:58 mebaran151: thanks all

16:59 hiredman: ((comp type first list) 1 2 3)

16:59 ,((comp type first list) 1 2 3)

16:59 clojurebot: java.lang.Integer

16:59 hiredman: ,((comp type first list))

16:59 clojurebot: nil

16:59 technomancy: did they ever fix the tarbomb for the release?

17:03 angerman: how would i compute a number in [0,1] into a base4 or base9 number? best would be to be able to access the digets seperatly

17:05 converting 1/8 should in base 4 should be something like '(0 2)

17:05 ataggart: ,(Integer/parseInt "110" 2)

17:05 clojurebot: 6

17:06 ataggart: ,(Integer/toString *1 4)

17:06 clojurebot: java.lang.IllegalStateException: Var clojure.core/*1 is unbound.

17:06 ataggart: ,(Integer/toString 6 4)

17:06 clojurebot: "12"

17:07 hiredman: http://blogs.azulsystems.com/cliff/2009/07/javaone-slides.html <-- slides from Cliff's javaone presentations

17:07 angerman: ,(Double/toString 0.5 4)

17:07 clojurebot: java.lang.IllegalArgumentException: No matching method: toString

17:08 ataggart: only integral types support a radix

17:08 angerman: hm.

17:08 well I know how the alogrithm works. it's basically like take number x multiply by base, take int part and continue with fraction part

17:08 it can go to infinity though :/

17:09 like 1/3. -> 0.11111...

17:12 Chouser: sounds like a nice place to use a lazy seq

17:12 angerman: yes that's probably the best approach. I'm just too new to clojure to get that implemented :/

17:14 Chouser: the input is a fraction in base 10 and the output should be a seq of digits, each less than base?

17:14 angerman: Chouser: yes.

17:15 so basically (def to-base [base frac] ...)

17:16 (let [t (* base frac] (int-part t) (to-base base (frac-part frac)))

17:16 something like that.

17:17 hiredman: "Each conversion passes all data thru cache"

17:17 :(

17:18 Chouser: (defn radix-seq [base frac] (lazy-seq (cons (rem (int frac) base) (radix-seq base (* base frac)))))

17:18 not at all sure that's right

17:18 note it always generates a infinite seq, so be careful printing it

17:20 angerman: Chouser: how would I take the first n elemts of it?

17:20 Chouser: (take n (radix-seq 4 1/3))

17:21 angerman: ok, now comes the interesting part

17:21 I have four functions H0, H1, H2 and H3

17:22 so if I have (1 1 1 1 1) I'd like them to get (H1 (H1 (H1 (H1 (H1 '(0 0))))))

17:23 so I guess I'd have to have a map of {0: H0, 1: H1, 2:H2, 3: H3} and then have the somewho applied to the sequence.

17:24 Chouser: (apply compose (map [H0 H1 H2 H3] [1 1 1 1 1]))

17:24 ((apply compose (map [H0 H1 H2 H3] [1 1 1 1 1])) '(0 0)) ; I guess

17:25 angerman: Chouser: hmm. does that translate 0 to H0 and 1 to H1?

17:25 Chouser: ,(map [:h0 :h1 :h2] [0 1 0 2])

17:25 clojurebot: (:h0 :h1 :h0 :h2)

17:25 angerman: nice

17:26 * angerman goes to look for compose

17:26 Chouser: sorry

17:26 comp

17:26 Chousuke: comp

17:27 angerman: wow, this is awesome

17:27 I seriously need to learn more clojure

17:27 ok, so what was this all about.

17:27 I'm trying to do the mapping into a room-filling-curve (? engl)

17:28 mapping [0,1] -> [0,1]^2

17:28 * technomancy is guilty of sometimes bending problems so he can use comp and partial

17:28 technomancy: because they're so awesome

17:28 Chouser: I never use them.

17:28 except to solve others' problems.

17:29 my problems all seem to be about java interop

17:30 technomancy: =(

17:32 jbondeson: every problem i have with clojure-java interop i remember how painful .net-com interop is.

17:32 and then it just doesn't seem so bad

17:33 technomancy: oh totally. also sometimes it also helps to remember a time when you were kicked in the crotch.

17:33 (depending on the Java library)

17:33 Chouser: oh, it's not bad, it's just not (apply compose (map ...)) :-)

17:34 and beating on Java apis at a REPL with some custom macros at hand is pretty amazing

17:36 angerman: hmm ok, so I do have a canvas to draw on. and it seems to draw at least a little bit corretly

17:37 how do I generate samples in [0,1] e.g. 100 evenly spaced?

17:37 technomancy: angerman: are you using processing for this?

17:38 Chouser: (range 0 1 0.01) ?

17:38 ooh, or (range 0 1 1/100)

17:38 angerman: technomancy: no. A JPanel hooked into a JFrame redrawing using a Buffered Image

17:38 jbondeson: yeah, the second will give you the exact, first one will have rounding fun

17:40 angerman: gna...

17:40 rhickey: aargh - how/where do I alter Clojure's build.xml to get a build-time .jar dependency?

17:41 angerman: for drawPolyline I need my [(x1 y1) (x2 y2) ....] to be in the form [x1 x2 ...] [y1 y2 ...] length

17:42 * rhickey doesn't grok ant

17:44 jbondeson: rhickey: are you trying to have it pick up a jar path from the command line?

17:44 Chouser: contrib depends on clojure -- could check it's ant xml

17:45 rhickey: jbondeson: whatever it takes

17:45 angerman: how do I round a float to the next int?

17:45 well round a fraction

17:45 notostraca: ceil?

17:46 angerman: well ceil and floor do no rounding .

17:46 jbondeson: rhickey: is this going into the compile-clojure or compile-java nodes?

17:47 i know how to get a new classpath into the java node, but haven't done anything with the javac nodes

17:47 rhickey: compile-java

17:47 kotarak: angerman: wild guess: Math/round?

17:47 angerman: kotarak: not found ;/

17:47 hmm...

17:47 jbondeson: you can promote the classpath property to a node and add path nodes

17:47 let me throw one up on a paste

17:49 lisppaste8: jbondeson pasted "ant classpath" at http://paste.lisp.org/display/83954

17:49 jbondeson: that's from one of my build.xml files

17:49 i'm by no means an ant expert

17:49 but that's how i've been throwing in new jars to the classpath at build

17:49 ataggart: there are about a half-dozen ways to do it. More detail might help.

17:50 rhickey: ok, got the jar found, now: class file has wrong version 50.0, should be 49.0

17:51 working now, thanks

17:53 notostraca: angerman: http://code.google.com/p/clojure-contrib/source/browse/trunk/src/clojure/contrib/math.clj

18:04 ataggart: alternately: http://github.com/richhickey/clojure-contrib/blob/6fea921c91d453c750586c960ec92caacf4fa5e6/src/clojure/contrib/math.clj

18:07 hiredman: ~def expt

18:07 ugh

19:02 mebaran151: I'd like to model my bdb bindings as a giant lazy hash

19:02 however, I haven't figured out a good way to handle managing cursor resources

19:03 hiredman: mebaran151: put a {} in an agent and add a watcher

19:04 * hiredman has not thought this through completely

19:05 Anniepoo: Yikes! I'm writing this little desktop GUI editor for work. I'm halfway done with it and just noticed that the basic problem it's supposed to solve is the knapsack problem

19:09 mebaran151: hiredman, oh you mean to put the key back right when somebody asks for it

19:09 *the cursor back

19:09 but then the disk has to seek all over again

19:10 hiredman: I am not actually thinking about the cursor

19:31 mebaran151: I had a simple implementation based on that exact idea, with a watch coming in behind the scenes to put the cursor in

19:31 but I slowly ran out of cursors because they weren't being closer

19:31 *closed

19:32 technomancy: is there anything like agent-local bindings?

19:34 I want to wrap everything that an agent executes in a single binding call

19:34 dreish: So what's this thing I keep seeing on the twitters about Sunn O))) and rhickey?

19:35 technomancy: dreish: I can't figure that out.

19:37 dreish: I think it's spam-bots re-tweeting things they suspect may be popular

19:37 dreish: technomancy: That was my best guess as well.

19:37 technomancy: but it has a very poor algorithm for determining how to get clicks

19:38 dreish: Not sure I'm reading this correctly, but it looks to me like Agent.java has to go out of its way a bit to recreate the bindings of the thread that sent the action. There's probably room to do something like per-agent bindings, but I don't see it implemented already.

19:38 technomancy: is there any way to get something like the current thread ID?

19:39 I could just look up a value in a map

19:39 I guess Thread/currentThread ?

19:40 dreish: Yeah, (.getId (Thread/currentThread))

19:41 technomancy: or just the thread itself if it's a map ke

19:41 y

19:41 dreish: Looks like the ID is unique among all threads that ever were.

19:42 I have a repl that evals everything in a thread so it can be backgrounded if it takes too long, and each time I call the above, I get a different ID.

20:10 Raynes: Honey, I'm home.

21:52 ericthorsen: Chouser: (analyze (read-string "(ns ns.test (import (java.lang String)))"))

21:53 Chouser: is giving me...java.lang.RuntimeException: java.lang.Exception: Unable to resolve symbol 'clojure.core/import*'

21:53 Chouser: I'm using the latest of all

21:56 durka42: ericthorsen: shouldn't that be :import

21:56 ericthorsen: durka42: Yes...a typo on my paste..

21:56 (analyze (read-string "(ns ns.test (:import (java.lang String)))"))

21:58 durka42: (doc analyze)

21:58 clojurebot: "/;nil; "

21:58 durka42: i get cannot find symbol analyze :)

21:58 where is it

21:59 hiredman: ~def c.l.Compiler

21:59 ericthorsen: durka42: sorry...it's in Chouser's net.n01se.clojure-compiler project

22:00 durka42: (doc reductions)

22:00 clojurebot: "clojure.contrib.seq-utils/reductions;[[f coll] [f init coll]]; Returns a lazy seq of the intermediate values of the reduction (as per reduce) of coll by f, starting with init."

22:01 durka42: ,(reductions (fn [sum nextpair] (prn 'schedule sum) [(+ (first sum) (first nextpair)) (second sum)]) '[1 a] '[[2 b] [3 c] [4 d]])

22:01 clojurebot: java.lang.Exception: Unable to resolve symbol: reductions in this context

22:01 durka42: (use 'clojure.contrib.seq-utils)

22:01 ,(use 'clojure.contrib.seq-utils)

22:01 clojurebot: nil

22:01 durka42: ,(reductions (fn [sum nextpair] (prn 'schedule sum) [(+ (first sum) (first nextpair)) (second sum)]) '[1 a] '[[2 b] [3 c] [4 d]])

22:01 clojurebot: ([1 a] [3 a] [6 a] [10 a])

22:01 durka42: ^ am i reinventing the wheel here?

22:05 hiredman: ,(take-while identity (iterate rest '([1 a] [2 b] [3 c] [4 d])))

22:05 clojurebot: Execution Timed Out

22:05 hiredman: whoops

22:06 no, I don't think you are reiventing the wheel

22:11 * durka42 notices that that wasn't exactly what i wanted

22:11 durka42: ,(reductions (fn [sum nextpair] [(+ (first sum) (first nextpair)) (second nextpair)]) '[1 a] '[[2 b] [3 c] [4 d]])

22:11 clojurebot: ([1 a] [3 b] [6 c] [10 d])

22:11 durka42: that's better

22:11 but i implemented it with recursion instead

22:13 hiredman: I think you could do it with reduce

22:13 durka42: reduce gets me [10 d]

22:15 hiredman: ,(reduce #(conj %1 (first (pop %1))) [] [[1 :a] [2 :b] [3 :c]])

22:15 clojurebot: java.lang.IllegalArgumentException: Wrong number of args passed to: sandbox$eval--3134$fn

22:15 hiredman: anyway

22:15 something along those lines

22:15 peek

22:15 I think, not pop

22:16 ,(reduce #(conj %1 (+ (last %2) (first (pop %1)))) [] [[1 :a] [2 :b] [3 :c]])

22:16 clojurebot: java.lang.IllegalStateException: Can't pop empty vector

22:16 durka42: ,(reduce (fn [as b] (conj as [(+ (ffirst as) (first b)) (second b)])) '[[1 a] [2 b] [3 c] [4 d]])

22:16 clojurebot: java.lang.IllegalArgumentException: Don't know how to create ISeq from: Integer

22:16 durka42: ,(reduce (fn [as b] (conj as [(+ (ffirst as) (first b)) (second b)])) [] '[[1 a] [2 b] [3 c] [4 d]])

22:16 clojurebot: java.lang.NullPointerException

22:16 durka42: fail

22:17 hiredman: ,(reduce #(conj %1 (+ (last %2) (first (pop %1)))) [[nil 0]] [[1 :a] [2 :b] [3 :c]])

22:17 clojurebot: java.lang.NullPointerException

22:17 hiredman: bah

22:19 durka42: ((fn [a b & cs] (prn a b) (if cs (recur (+ a (first cs)) (second cs) (nnext cs)))) 1 :a, 2 :b, 3 :c, 4 :d)

22:19 ,((fn [a b & cs] (prn a b) (if cs (recur (+ a (first cs)) (second cs) (nnext cs)))) 1 :a, 2 :b, 3 :c, 4 :d)

22:19 clojurebot: 1 :a 3 :b 6 :c 10 :d

22:58 Chouser: ericthorsen: clojure-compiler is probably just out of date. Haven't touched it in a while.

23:30 yangsx: My program prints this message: Exception in thread "Call-on-write Stream" java.lang.RuntimeException: java.lang.StringIndexOutOfBoundsException: String index out of range: 56

23:31 I guess it's caused by the Java library I'm using

23:32 XOM java XML library

23:34 Very strange: run the function again and it does not print such error message any more.

23:35 and gives me the same result

23:35 hiredman: where is the full stracktrace + code?

23:35 yangsx: No stacktrace at all.

23:35 hiredman: (.printStracktrace *e)

23:35 Stack

23:36 yangsx: I mean not the usual stack trace with slime

23:36 hiredman: *shrug*

23:36 that is not a lot of information to debug from

23:39 lisppaste8: yangsx pasted "strange Exception" at http://paste.lisp.org/display/83979

23:39 yangsx: hiredman: yeah, I'm puzzled, too. The printed stacktrace seems not so helpful.

23:42 hiredman: I imagine it is slime stepping in and being "helpful"

23:43 well, decompose, re-think, add calls to prn

Logging service provided by n01se.net