#clojure log - Aug 27 2008

The Joy of Clojure
Main Clojure site
Google Group
List of all logged dates

7:19 achim_p: hi!

7:20 is there a way of concatenating vectors which is more efficient than (vec (concat v1 v2)) ?

7:29 (vec (concat vec1 vec2)) takes linear time, as i understand, because concat only uses the sequential views of the vectors, right?

7:29 i thought there might be a faster way, depending on the implementation of the vector type ...

8:33 rhickey: achim_p: yes, sequential, there's no deque or virtual vectors yet

9:30 achim_p: rhickey: okay, thanks!

9:31 what are clojure vectors based on? are they bagwell vlists? i believe you mentioned it in one of the videos, but i can't find it now

9:31 rhickey: achim_p: they are not vlists, but bitmapped hash tries like the maps, the difference being the keys are implied and the range is full

9:38 Chouser: I wouldn't think it'd be too hard to use proxy to whip up a virtually concatonated pair of vectors

9:42 achim_p: rhickey: i'll have to dig deeper into this, i'm new to this functional stuff ;)

9:42 thanks again - also for clojure, it's really exciting!

9:42 rhickey: you're welcome!

9:44 blackdog: rhickey, did you see this from jrose@sun on the jvm list? Folks, invokedynamic, continuations and tailcalls are brewing (in

9:44 three different locations) and interface injection will (I hope)

9:44 become real after method handles go mainstream.

9:45 tail calls? is that the tail call optimiation stuff?

9:46 rhickey: blackdog: yes, that will be the real tailcalls. I'll be speaking at the JVM Languages Summit and hope to get the scoop on where things are at.

9:47 blackdog: yea, glad to see you going and getting the recognition

9:47 java luminary i think they say :)

9:47 Chouser: It's pretty amazing how little tailcalls are needed in Clojure. I guess it'll be nice for mutual recursion.

9:48 rhickey: Chouser: well, I designed based on what was possible

9:49 Chouser: sure, but even if there were real tailcalls, I would still prefer loop/recur when it would work -- I think it's actually clearer on a couple levels.

9:49 rhickey: even with all the talk now, so far that work is relegated to the playground JVM, and who knows when it will be in a JDK, and then how long before companies are willing to move to that JDK

9:51 drewr: What would be the advantage of real TCO over the tricks loop/recur does now? What it incur less overhead because you're having to mimic the behavior?

9:52 rhickey: drewr: real TCO lets you call another (non-self) function without growing the stack, loop/recur is limited to internal calls

9:54 drewr: Ah, I guess I've only ever needed internal calls.

9:55 achim_p: Chouser: like a list/vector of vectors? that wouldn't be much better than a sequence. you'd have to visit each subvector and determine its count to figure out which element of which subvector a given (global) index maps to

9:56 rhickey: you need TCO when you are trying to create a concrete result using recursion, and at each step you will conditionally call a different function to process the rest. There are some elegant algorithms that require TCO.

9:58 Chouser: achim_p: Well it's probably not a good general solution, but if you had a particular case with a small number of very large vectors, you might be able to write something that performs better than just appending them all into a single vector.

10:00 so is there any interest or desire to use something like setf in Clojure?

10:00 (def m {:a1 {:b1 [:c1 :c2 {:d3 5}]}})

10:00 (setf ((nth ((m :a1) :b1) 2) :d3) 9) => {:a1 {:b1 [:c1 :c2 {:d3 9}]}}

10:02 rhickey: Chouser: setf is based upon there being a 'place' that is changed, there's no place there as it is functional, so setf is not the model. We've talked about <- (analogous to ->) here before

10:02 Chouser: hm.

10:03 rhickey: and yes, I'm waiting for <-, but no one has gotten desperate enough. I;'m surprised, as once you get nested it gets pretty arduous)

10:04 (<- m :a1 :b1 2 :d3 9)

10:04 Chouser: http://clojure-log.n01se.net/date/2008-07-15.html#09:22

10:05 Parth Malwankar has posted related ideas to the forum recently.

10:11 achim_p: Chouser: i'm looking for a data structure that suppports a fairly efficient (sub-linear) "drop-nth". my initial idea was concatenating two subvecs, but this is really slow on large vectors. a vector-of-vectors kind of thing will fragment very fast in my case, because i'll be dropping lots of nths ;) (all of them, eventually)

10:11 rhickey: (assoc-> m :a1 :b1 2 :d3 9) ?

10:11 Chouser: Currently: (-> m :a1 :b1 (nth 2) :d3) => 5

10:11 cemerick: it'd be nice to be able to invoke Callables as fns...

10:12 Chouser: So perhaps: (<- m :a1 :b1 (nth 2) :d3 9) ?

10:12 rhickey: cemerick: that would add a conditional to all invocations where there is none now

10:13 cemerick: yeah, I know it's unworkable -- just musing away

10:13 Chouser: achim_p: ah, I see. What about a sorted-map with integer keys?

10:15 (<- m :a1 :b1 (nth 2) :d3 (do 9))

10:15 rhickey: achim_p: you want drop nth and the result to still have fast indexed lookup?

10:17 achim_p: yeah, not sure whether that's possible at all ...

10:17 rhickey: Chouser: that's why I renamed to assoc->, the macro needs to know if can rebuild with assoc, so arbitrary accessors can't be supported (without getting into defsetf and places)

10:17 achim_p: not as far as I know

10:18 Chouser: rhickey: ok, so it could assume "get" on the way in and "assoc" on the way out.

10:19 rhickey: Chouser: maybe best not to get stuck on -> analogy, but idea is to just take keys followed by val and do nested assoc

10:19 right

10:19 Chouser: what of the final item -- a value or a function?

10:20 It looks like Parth's pretty close to this already.

10:22 rhickey: in the original discussion there was the possibility of the last expression being either a value or a parenthesized function-expression of the (presumed existing) value, missing first arg, like: (+ 5) or (assoc k v)

10:24 or could have separate alter-> assoc->

11:01 parth_m_: rhickey: Just as Chousers mail on updating nested structures on the group.

11:01 I was wondering if you had a chance to see the field-write function here http://groups.google.com/group/clojure/msg/c312ba23efcf4c17

11:02 I needed it quite badly as I have started using 3-4 levels of nesting for a program I am writing.

11:03 The improved field-read posted by Graham Fawcett also seemed nice to me.

11:05 Chouser: parth_m_: I didn't understand your explanation for why field-write can't be a macro.

11:05 oh, nm, I got it now. :-)

11:05 rhickey: there's no need for field-read: (-> nx :a :b :c)

11:06 Chouser: you want to be able to pass in a computed access-spec.

11:08 oh cool -- with -> you can even provide default values: (-> nx :a (:b {}) :c)

11:08 parth_m__: sorry I got dropped off

11:08 rhickey: -> rocks

11:09 Chouser: although if you have vectors mixed in you need to name "nth" explicitly.

11:10 parth_m__: I would be happy to see field-write (or something like it) in clojure or contrib.

11:11 The use case I had for field-read was that as structures become nested, I don't really hand code the access-specification.

11:12 Chouser: did you see you can use the -> macro?

11:12 parth_m__: I have a separate function get-access-path that takes a keyword and returns a vector that is the access-spec.

11:13 I am sorry, my client died and lost the messages.

11:13 Chouser: 11:05 < rhickey> there's no need for field-read: (-> nx :a :b :c)

11:14 parth_m__: Is there a way to pass a "apply" a vector to ->? Maybe there is but I couldn't figure it out.

11:15 Chouser: I think you'd have to use (eval), which isn't too happy.

11:15 parth_m__: Oh.

11:16 rhickey: parth_m_: (reduce get nx [:a :b :c])

11:17 parth_m__: Oh. Thats nice ... thanks :)

11:22 Chouser: "reduce get" works on vectors, but you lose the default-value feature of ->

11:24 parth_m__: I am not really thinking of (reduce get ..) as a replacing ->. Just a complement for field-write which has a similar use and feel.

11:25 Chouser: oh, of course. -> is much more general than (reduce get ...)

11:26 It just seems like it'd be nice to have a symmetrical get and set, but for that to work you need a comprehensive get.

11:27 if the different ways to do nested get have different feature sets, it's harder to have a symmetrical set to go with them.

11:28 rhickey: Chouser: CL's places are the general solution, but for Clojure without the in-place aspect

11:28 parth_m__: Chouser: Are you thinking of something like nested sets etc?

11:30 Specifically for field-write/read struct, accepting vectors is an important use case in my view as access-path [:a :b :c] etc. may be generated at runtime based on the final key. At least thats how I am using it.

11:31 It allows me to build layers on top of the field-write ... e.g. (def reg-write [core reg value ] (field-write core :content value (reg-path reg)))

11:42 Chouser: being a function (instead of a macro) would be helpful for integrating with commute and alter as well.

11:44 mac__: Hello, can someone help me out with gen-and-load-class? I'm trying to generate a class in order to specify a finalizer (yes I know that's considered bad java but humor me) but I can't get it right. This is what I got: (in-ns 'test)

11:44 (clojure/refer 'clojure)

11:44 (gen-and-load-class 'test.Hello)

11:44 (defn Hello-finalize []

11:44 (println "oh no!"))

11:44 And it works if I make up some own method and put it in with a :method parameter to gen-class but this does not seem to work. Is it because finalize is protected or something?

11:44 oops, sorry for the linebreaks, just pasted in some stuff :)

11:45 Chouser: gen-class doesn't generate finalize methods

11:45 mac__: I see, so I would have to do that in java then?

11:46 Chouser: It used to (for all classes), but it made all gen-class classes much slower at runtime, so that feature was backed out.

11:46 mac__: Well that's sound I guess since .. .yeah ^^ the gc would go crazy

11:46 Could be an optional thing though

11:47 Chouser: I guess for now you would have to use Java, though I don't think it'd be too hard to give gen-class -- right, an option to generate finalize.

11:47 mac__: like :finalize or something in the arguments

11:47 Chouser: right

11:47 mac__: ok thanks

11:48 Chouser: rhickey: what do you think, would you accept a ":gen-finalize true" patch for gen-class?

11:50 rhickey: Chouser: can you get one by using :methods currently?

11:50 mac__: I tried but just got exceptions.. don't know if I got the syntax right though

11:50 Chouser: Hm, I may have jumped to a conclusion there. Hang on.

11:50 rhickey: if not, yes, patch ok

11:51 mac__: Would it look like this? :methods [['finalize []]]

11:51 Cause that won't work for me :)

11:53 Chouser: mac__: try: (gen-and-load-class 'foo.Hello :methods [['finalize [] Void/TYPE]])

11:54 mac__: ah Void/TYPE, cool didn't know about that. Trying...

11:56 Chouser: I seem to be able to define the class and instantiate a bunch, but I'm not sure if finalize is getting called or not.

11:56 I don't really know Java well enough to be sure.

11:56 cemerick: finalize is never guaranteed to be called

11:57 mac__: Yeah it's hard to verify if that worked :) but at least there is a finalize method on that class now.

11:57 Chouser: you'd think after making 10000 foo.Hellos, that some would go away...

11:58 mac__: Yeah if you don't hang on to them

11:58 Try doing (System/gc), not guaranteed to do anything but it may help

11:58 cemerick: my understanding is that, even if a foo.Hello is gc'd, the finalize method isn't guaranteed to be called

11:58 mac__: I got nothing when I did it

11:59 Chouser: I'm also not sure of the context finalize would be called in -- will it still have access to stdout?

11:59 cemerick: as long as the jvm isn't in the process of exiting, then yes

11:59 mac__: Yeah you can do anything from a finalize pretty much.

12:01 Chouser: The finalize method generated this way is "public" -- does that signature differ enough from the "protected" one provided by Object that it won't be called by the gc?

12:01 mac__: And they are not guaranteed to be called before exit of the jvm but in a lenghty process they should get called before the gc reclaims the objects memory.

12:02 Chouser: I don't know but it doesn't seem to work anyway. I did a java test earlier today and got printouts from finalize when doing a System.gc

12:02 So I smell a difference

12:02 Chouser: hm

12:03 parth_m: rhickey: Do you see value in field-write (and possibly its complement field-read - for usability) being added to clojure? or contrib?

12:05 rhickey: parth_m: yes, but the interface matters a lot to me, field and write are bad names

12:06 parth_m: Agreed. This was just something I started doing because I needed it.

12:06 Any suggestion? On names. Possibly any implementation improvements?

12:13 Chouser: mac__: I'm not sure what's going on. A quick hack to generate protected finalize methods still doesn't generate any output.

12:16 oh, got it.

12:16 mac__: Solved it :) Did you put any argument in your clojure finalize method? I didn't at first and it will get sent the java object... so it worked for me now that I added an argument to it

12:16 Chouser: right :-)

12:16 (defn Hello-finalize [this] (println "finalize"))

12:16 mac__: yep

12:17 Chouser: without the arg it was throwing an exception which is of course ignored in finalize.

12:17 mac__: Exactly. So in conclusion, it works with :methods if you just do it right ;)

12:17 Chouser: :-)

12:20 mac__: Well thanks for the help anyway, might have just given up and done it in java if I didn't have someone to discuss it with.

12:22 Chouser: never give up and do it in java. ;-)

12:24 mac__: That's a good quote

12:25 hoeck: rhickey: hi, i made the multimethod version of print, dispatching on class

12:26 sorry, i mean hi all :)

12:30 parth_m: Hello :)

12:35 rhickey: hoeck: cool - have you sent in a CA?

12:38 hoeck: rhickey: yes, about 2 weeks ago

12:38 rhickey: hoeck: ok - I've been on vacation and haven't checked the box since returning

12:40 hoeck: rhickey: ah, should i email you the file or add it to clojure-contrib or the like?

12:41 rhickey: you can put it as an attachment in a message to the group

12:43 hoeck: rhickey: okay

12:45 lisppaste8: parth_m_ pasted "field-read/field-write" at http://paste.lisp.org/display/65959

12:46 parth_m: rhickey: any suggestions on the names ... I am quite bad with names. I have pasted the latest version.

13:08 rhickey: Got to go. Please mail any suggestions you might have on the group and I can roll them in. Good day!

13:29 mac__: I need help with gen-class again.. How do I define a constructor that has a different signature from the superclass (Object in my case). Is this not correct if I want a constructor that takes an Object as it's argument? :constructors {[Object] []} I'm getting this exception when trying to instantiate it: java.lang.UnsupportedOperationException: nth not supported on this type: PersistentHashMap

13:32 hmm might be something else

13:37 yep, I made a mistake with the return value from init.

14:00 notyouravgjoel: java -cp jline-0.9.91.jar:clojure.jar jline.ConsoleRunner clojure.lang.Repl << will this still work with the newest clojure?

14:01 Chouser: I haven't used jline, but I can't think of any recent changes that would break that if it used to work.

14:02 notyouravgjoel: k; getting started has it listed, but it isnt working

14:04 Chouser: hm. you're getting errors?

14:05 notyouravgjoel: 'Exception in thread "main" java.lang.NoClassDefFoundError: jline/ConsoleRunner

14:05 java.lang.NoClassDefFoundError: jline/ConsoleRunner

14:06 Chouser: Does it work without the jline stuff?

14:07 notyouravgjoel: yep

14:08 ah, just figured it out

14:08 sorry, was rather stupid on my part

14:11 Chouser: np.

14:23 rhickey: lisppaste8: url

14:23 lisppaste8: To use the lisppaste bot, visit http://paste.lisp.org/new/clojure and enter your paste.

14:24 rhickey pasted "Multi-key get/assoc" at http://paste.lisp.org/display/65964

14:25 cemerick: rhickey: ooh, that looks nice

14:27 rhickey: cemerick: the other option is to force the 'multi-key' into a vector

14:28 cemerick: it seems quite pleasant the way it is

14:29 rhickey: I agree, Parth's not here to explain his use-case, but this method is much more efficient than his whatever the interface

14:31 Chouser: I wonder if it would be useful to be able to provide defaults to allow building out the initial structure.

14:31 Parth may be content with (apply mk-assoc nx v)

14:35 lisppaste8: rhickey annotated #65964 with "mk-assoc that builds" at http://paste.lisp.org/display/65964#1

14:39 Chouser: that's fine for maps

14:39 foo=> (mk-assoc {:a [:b0 :b1 {:c 9}]} :a 2 :c 7)

14:39 {:a [:b0 :b1 {:c 7}]}

14:39 foo=> (mk-assoc {} :a 2 :c 7)

14:39 {:a {2 {:c 7}}}

14:41 rhickey: Chouser: what do you want the second one to do?

14:42 Chouser: I'd have to specify that I want to build a vector at that level instead of a map

14:42 I haven't thought of a good API

14:42 (mk-assoc {} :a '(2 []) :c 7)

14:42 ick

14:42 rhickey: I don't think it's a real problem

14:42 Chouser: ok

Logging service provided by n01se.net