#clojure log - Jul 18 2008

The Joy of Clojure
Main Clojure site
Google Group
List of all logged dates

1:14 mebaran151: in the Netbeans plugin, can you access the current buffer to test it?

1:31 also, how do I convert a Java Hashmap to a Clojure hash?

1:31 any convenience method?

1:43 hoeck``: mebaran151: no, only `seq'

1:43 mebaran151: if I turn it into a seq, do I just get a list of tuples?

1:44 hoeck``: yes, a list of Hashmapentrys

1:44 (apply hash-map (mapcat #(list (.getKey %) (.getValue %)) (seq h)))

1:44 would then convert it to a clojure hash-map

1:47 slava: sounds like something that should be built-in

1:48 hoeck``: maybe it is already??

10:34 meredydd: Okay, this is a new one:

10:34 java.lang.ClassCastException: clojure.lang.PersistentHashMap cannot be cast to java.util.Map

10:35 ("...and why the hell not?")

10:38 user=> (instance? java.util.Map {})

10:38 false

10:42 mebaran151: I'm getting an incomprehensible class cast exception

10:42 I'm trying to create an httpservlet, following the guide online and using a proxy object, but it doesn't seem to want to play along...

10:44 meredydd: rhickey: aha, the very man I'm looking for.

10:44 rhickey: hey

10:45 meredydd: (instance? java.util.Map {}) --> false.

10:45 I'm fairly sure that shouldn't be happening.

10:45 You implement Collection in APersistentMap, but although you import Map, you never implement it

10:45 rhickey: right now the Clojure collections only implement java.util.Collection

10:46 meredydd: Um...why?

10:47 Any chance you could add basic Map support to APersistentMap, pretty please?

10:48 rhickey: That's all I've gotten to so far. Also, Map is a pain, in that several key functions require implementations of Set (entrySet/keySet)- I haven't determined what it takes to provide them efficiently

10:48 meredydd: you could submit a patch

10:49 meredydd: I am very willing to do so.

10:49 (This one's something of a deal-breaker for this app :) )

10:49 ISTR there's some sort of licensing hoop I should jump through

10:49 Does it involve a paper transaction, or is "I hereby assign the rights to copy, reproduction and my firstborn child to rhickey" sufficient?

10:50 rhickey: http://clojure.org/contributing

10:51 meredydd: Aha - that's new. Thanks.

10:51 rhickey: One issue is that Map doesn't implement Collection

10:51 meredydd: Hmm...so it doesn't. Do you have a particular objection to my leaving the existing Collection implementation in place, and just adding a Map implementation on top of it?

10:52 rhickey: that would be the plan, but I don't know if there are issues

10:52 meredydd: Okay. I'll have a go.

10:52 hoeck: is it possible to get the class object of a class generated with gen-and-load-class by using Class.forName?

10:53 rhickey: another gotcha is that the easiest way to implement Map is by deriving from AbstractMap, but APersistentMap already derived from AFn, you you'll have to choose

10:53 meredydd: hoeck: Why don't you just use Object.getClass()?

10:54 (Bah. Transatlantic snail mail - that's going to hurt. Better hope Subversion's merging is decent...)

10:54 hoeck: meredydd: pivot (a java gui framework) requires you to give a classname as a string

10:54 in order to start the desktop-app

10:55 rhickey: hoeck: if you define a class ns.Foo with gen-and-load-class then just saying ns.Foo will give you the class

10:55 hoeck: ah, you may be into gen-and-save-class territory if you want Java clients to see it by name

10:56 meredydd: rhickey: All right to send you a patch for visual review before the agreement hits you? Best not build further code on top of something that's barking up the wrong tree.#

10:56 rhickey: meredydd: yes, and you can email me a scan while the paper is en route

10:57 hoeck: rhickey: yeah right, that works pretty well, but a pivot desktop application is normally startet from the commandline and expects a classname as the app-object to start

10:58 rhickey: hoeck: gen-and-save-class

10:58 was made for those scenarios

10:59 * hoeck is trying gen-and-save-class

11:02 meredydd: rhickey: Ah, thanks for that. Read that just before I sealed the envelope :P

11:05 hoeck: rhickey: thanks!

11:48 meredydd: rhickey: Are you keeping source-level compatibility with a particular JDK version?

11:57 rhickey: meredydd: yes, jdk 1.5 please

11:57 meredydd: rhickey: 1.5 has generics, right?

11:58 rhickey: please avoid generics

11:58 meredydd: (not that I'm actually using them, but there's a piece of API that wants a Set<Map.Entry>, where the 1.4-and-earlier ones just have a Set)

11:58 rhickey: your parameters will all be Object anyway

11:59 ah, well then you'll be ok as Clojure's entries are Map.Entry

12:00 meredydd: So I see. Makes life easier.

12:01 Of course, they'll all be delivered to me as Objects, so there will be one unpleasantly-ugly cast at the return statement for that method

12:01 (which, thanks to the magic of type erasure, will be completely removed for compilation. *sigh*)

12:23 rhickey: anyone with Ant knowledge here? I'm looking at putting boot.clj etc in correct classpath-relative dirs in the jar, plus thinking about including contrib eventually, so need the ant script to find .cljs anywhere in the src tree and put them in the corresponding spot in the jar

12:23 right now it pulls .cljs from /src and pus them in the root, hard-wired

12:23 puts

12:25 meredydd: rhickey: Looking at your map implementations, I see what's obvious in hindsight, namely that map lookups are performed with Object.equals() semantics, rather than Util.equal()

12:26 grahamf: rhickey: hi Rich

12:26 rhickey: grahamf: hi

12:26 meredydd: rhickey: Do you have a method for dealing with what happens if, say, an Integer and a BigInteger with the same numerical value are (assoc)ed onto the same map?

12:27 grahamf: the 'copy' task in Ant is pretty flexible, you can provide an input filter (for cljs only) and specify a target dir, choosing whether to flatten or maintain the original structure

12:27 http://ant.apache.org/manual/CoreTasks/copy.html

12:27 rhickey: meredydd: this came up in - it's not something I can fix, as I don't own those classes and their definitions of hashCode

12:28 meredydd: yep, that's what I guessed

12:29 and I'm presuming you're not willing to take the necessary performance penalty to put java.lang.Number checks into Util.hash() like you have with Util.equal()?

12:29 rhickey: people using mixed numeric keys will need to normalize with casts if they want to look up with different types, there's an aoutstanding todo to make sure reductions are always done in Clojure math

12:30 meredydd: not sure, the maps preceded the move to Java's boxed types

12:31 meredydd: Hmm. Well, I'm going to vote for correctness over performance here, but I'm aware it's adding instructions to the lookup path

12:31 (also, that it involves implementing a non-standard hash function on Numbers, which is icky, but at least it's an internal ickiness rather than an external ickiness)

12:32 If I'm feeling energetic, I'll lob you a patch on that one too - but as I can't branch'n'merge with someone else's repo, I'll probably stick to doing one at a time.

12:32 grahamf: rhickey: re: the ant job, what would relate a clj file to its destination dir? Do you need to analyze each clj file to determine its target dir, or it is just a matter of filtering and maintaing the right dir structure?

12:34 meredydd: Regardless, in the meantime I shall implement keySet() and values() with Object.equals() semantics, as that's what native Clojure sets do.

12:34 rhickey: grahamf: the idea is for it to end up the same, x/y/z.clj shoud end up in /x/y/ in the jat

12:34 jar

12:35 meredydd: don't bother with the hash for now, too big a change, I'll need to think about it

12:36 meredydd: rhickey: Roger.

12:37 rhickey: grahamf: if you look at Clojure's build.xml, I'm looking to replave the fileset entry for cljs, but don't know how copy job would fit in

12:37 * rhickey knows nothing about ant and has lived happily so far

12:38 grahamf: rhickey: heh. I'll take a quick look.

12:42 rhickey: hm, that looks like it should already do what you want. I just added a src/graham/test.clj to my local copy, ran 'ant' and it added a graham/boot.clj to the new jarfile

12:44 right, I see now that you don't need copy. I had assumed you were generating a temporary 'build' dir from 'src', and then jarring the 'build'.

12:47 d'oh, and so you are, 'classes' is your 'build'. :-)

12:51 * meredydd smacks head against wall.

12:51 meredydd: The 'remove()' function exists, incompatibly, in the two interfaces (Map and Collection).

12:52 rhickey: Looks like the PersistentMap classes can be Maps or Collections, but they can't be both.

12:53 rhickey: meredydd: all of the mutation methods are optional, can you not derive from both because of remove?

12:53 meredydd: Unfortunately not. They both take one Object arg, and Collection requires it to return boolean whereas Map requires it to return Object

12:53 rhickey: aargh

12:54 meredydd: And "optional" just means "is allowed to throw UnsupportedOperationException"

12:54 So...how much internal mangling depends on *PersistentMap classes being Collections? :)

12:54 rhickey: it's user code I'm concerned about

12:55 meredydd: Hmm. Not to trivialise the concern, but I don't think that many people would expect a map to implement Collection

12:56 they'd expect it to implement, well, Map.

12:56 Unless...do you document explicitly that a persistent map is a Collection rather than a Map?

12:56 rhickey: but the only useful methods would have been iterator and size

12:56 meredydd: True...but we lose iterator() in Map.

12:57 rhickey: I expect Java Maps to implement Collection and am pissed they don't

12:57 meredydd: (On the other hand, I'm guessing people haven't done too much hardcore Java interop work, otherwise someone would have been complaining about maps not being Maps before now...)

12:57 rhickey: Agreed, entirely.

12:58 Unfortunately, that, like so many things, is a poor decision long since fossilised by the weight of the Java platform.

12:58 rhickey: maps are documented to implement Collection, but I think the key is that they be Iterable<Map.Entry>, which we could implement independent of Collection

12:58 meredydd: This is what I'm thinking.

12:59 (hmm...turns out IPersistentMaps are already explicitly Iterable)

13:00 rhickey: ah, there's more to it, contains and toArray

13:01 we seem to be getting to the point I've been at before with this...

13:02 off to lunch, back soon

13:03 meredydd: k

13:17 rhickey: meredydd: so I think for now you should just implement a class that wraps a Clojure map and implements Map

13:17 meredydd: Hmm, define "for now"

13:17 Because I believe that to be an untenable long-term solution.

13:18 rhickey: in what respect?

13:19 (.someJavaMethod x (jmap m))

13:19 meredydd: It's a huge mismatch between the languages if the data type you use and recommend as replacement for arbitrary data structures doesn't conform to the Collections API

13:20 rhickey: that's a theoretical argument, being immutable they are hardly replacements

13:20 i.e. couldn't swap one for the other in existing code

13:20 meredydd: Fair enough, but it's what you explicitly recommend everyone else uses instead.

13:21 It's not like immutable Maps aren't something Java's designed to cope with - there's a documented Right Way to do it

13:21 rhickey: and so you might have to say (jmap m), doesn't seem prohibitive

13:22 meredydd: Okay, this will take a moment, because I'm trying to get to the little bit inside me screaming "It's ugly as sin!"

13:22 How about the other question: Why is it more important that a map be a Collection than that a map be a Map?

13:23 (* trying to get to the rationale behind the little bit inside me...)

13:25 rhickey: allows for most general code, non-reflective calls to size for instance

13:25 toArray

13:26 A map is a collection of entries

13:26 Java got that part wrong

13:26 Java got read-only wrong by not making it a base interface

13:26 meredydd: Are you talking about calls from Clojure code? Because it's very easy to keep those properties while still making it a map.

13:27 I agree with you that Java got that part wrong.

13:27 However, if you're talking about Java interop, which is what implementing the Collections API is all about

13:27 then making a map a Map is *vastly* more important for smooth working than making a map a Collection

13:27 *smooth interworking

13:28 (and if you do end up needing to something expecting a Collection of Map.Entrys - unlikely in java code, because of precedent - you could wrap that (uncommon) case in a fncall

13:29 rather than the common one ("I have Clojure's native representation of a map, I want to pass it to Java code that's expecting its native representation of a map)

13:30 You seem to be concentrating on correcting an (admittedly bone-headed) decision in the design of an API...at the expense of actually conforming to it.

13:31 rhickey: there are far more Java APIs that expect Collections

13:32 meredydd: True, but if you can find me one that's expecting a Collection of Map.Entries, I'll be very surprised

13:32 rhickey: Many don't care what the element is

13:33 they need the size and to iterate

13:36 meredydd: Hmm. I can't actually contest that argument (that you'll want use a map more often as a collection than as a map) - my impression and experience disagree with it, but I don't have hard figures to show it either way.

13:37 Do you believe that you need to use it that way *so* much more that it's worth violating all these expectations, so you have to make the funcall in the Map case rather than the Collection case?

13:37 rhickey: I'm not arguing against the utility of supporting the Map interface, but I also didn't create the incompatibility between Map and Collection that thwarts us here. Whether Clojure maps are more Maps than Collections is a philosophical debate, they are no more inherently one than the other

13:39 meredydd: Hmm. There's a bit of practicality as well as philosophy here. By its nature, the only time the distinction is relevant is when doing Java interop.

13:39 rhickey: Given that they are already Collections, and it would be simply to make (jmap m) work, I think that's the way to go, and by 'for now' I mean until I can determine if switching is worth it

13:39 meredydd: Well, I see I'm not going to convince you, so I'll archive this patch and revert my tree.

13:40 rhickey: what tends to happen in these cases is switching from one to another exposes the costs of change in unanticipated ways

13:41 meredydd: I'll agree with that; the Argument from History is firmly on the side of Collections.

13:41 It's on the Argument from What's Nicer where we disagree.

13:41 rhickey: well, toArray is used by Clojure, for instance

13:42 meredydd: That's irrelevant; you can have a toArray without implementing Collection.

13:42 rhickey: and access it via what type?

13:43 meredydd: I'd go for inventing a common superinterface for IPersistentMap and IPersistentCollection, myself

13:43 rhickey: but it works on Collections too

13:44 * meredydd thwocks himself - you're right, you'd have to special-case Maps.

13:45 meredydd: I guess I'm just upset by the inversion - in Java, it's a Map, and you call a function (entrySet()) to get a Collection of MapEvents

13:45 rhickey: Clojure supports treating all collections except Java Maps, pretty uniformly, making the change would add Clojure maps to the except column

13:45 meredydd: in Clojure, it's the other way around

13:45 Worse - it puts Clojure maps *and* Java maps in the "except" column

13:45 rhickey: ?

13:46 meredydd: uhh...which comment was that ? to?

13:46 the "In Clojure it's the other way around", or "clojure maps *and* Java maps in the 'except' column"?

13:47 rhickey: "it puts Clojure maps *and* Java maps in the "except" column", to which it do you refer?

13:47 meredydd: Oh - what I meant was that, currently, Clojure puts Java maps in the "except" column

13:48 rhickey: depends on your perspective, not from the Clojure perspective

13:48 cemerick: Is there a concrete use-case for making clojure maps Java maps?

13:48 rhickey: from there, only Java Maps are exceptions

13:48 meredydd: whereas if you made Clojure maps into Java maps too, you'd end up with *both* in the except column

13:48 which is pretty much what you'd said two lines earlier.

13:48 rhickey: right

13:49 meredydd: cemerick: Well, I started with the Principle of Least Surprise - I'm using Clojure to drive Java code, and throwing Maps into Java methods, and got a nasty shock when I found out an IPersistentMap wouldn't go into a Map parameter

13:49 rhickey: I tried to mazimize the genericity of code, for instance Clojure uses toArray, but others could use isEmpty or contains

13:50 meredydd: cemerick: I'm afraid I don't have the language-usage survey experience to say beyond "Passing them to things that expect maps", just as rhickey is saying "Passing them to things that expect collections"

13:51 (and "using Collection methods on them locally")

13:51 Hmm. Here's a thought...

13:51 rhickey: right now in Clojure if you say #^Collection x, x can be anything except a Java Map

13:52 meredydd: rhickey: Do you have a general mechanism for automatic coercion when you're looking up parameters for Java interop?

13:52 cemerick: I don't think having that survey would help, anyway. IMO, ensuring that clojure maps are collections is incredibly important. There has to be an overriding concern.

13:52 meredydd: (eg when you're converting a Number to an int to pass to an int-typed parameter)

13:53 ...actually, no, scrap that idea. It's an ugly hack, and an abstraction that can be made to leak like hell in a number of different ways.

13:53 rhickey: meredydd: just special handling of primitives

13:54 meredydd: Okay. I've just spent a relatively enjoyable hour arguing with rhickey about language design, but it's time to get some actual work done.

13:54 cemerick: rhickey: do you still need ant assistance?

13:55 meredydd: (jmap) doesn't completely satisfy me, but I understand why you don't want to lose Collection compatibility from IPersistentMap

13:56 so, when are we likely to see (jmap) in svn? (Not to rush you; just so I know whether to bother implementing one myself in the meantime)

13:56 rhickey: cemerick: I'll try grahamf's suggestion and call for help again when stuck, thanks

13:56 meredydd: if you implement jmap I'll add it

13:56 meredydd: okies.

14:10 Hmm...is there a sensible way of getting a lazy seq-backed Collection? So far I'm doing (apply list (vals map)), which is unpleasant and requires a complete eager traversal of the seq

14:11 Apart from that, here's a basic first draft: http://pastebin.com/m68e7610d

14:12 cemerick: meredydd: (vals map) is already a lazy collection

14:12 rhickey: meredydd: sets don't implement Set yet, but could

14:12 meredydd: cemerick: ha, so they do.

14:13 Now why did I assume ISeqs weren't collections?

14:13 rhickey: Do vectors implement List?

14:13 * cemerick shrugs

14:13 cemerick: no, they don't

14:14 meredydd: http://pastebin.com/m4c10e1af

14:15 Oh, argh.

14:15 *That*'s why I wanted native Map support.

14:15 Because that's how Clojure does stuff, I'm passing a map of maps into my Java function

14:15 rhickey: look, you need to go easy on me here, there are only so many hours in a day...

14:15 meredydd: now I'm going to have to explicitly unpack them all from Clojure IPersistenMaps

14:16 heh, sorry. I'll go away and write unpacking fns.

14:16 rhickey: I'm not against support List or Set and there should be no problems doing so, but haven't had time to do that

14:16 patches welcome

14:17 meredydd: rhickey: Agh. Mind if I wing you a patch for a "getJavaMap()" in IPersistentMap? Cause I can't use (jmap) from inside my Java code

14:18 rhickey: I had presumed you'd be doing this in Java and just calling from jmap

14:18 meredydd: (which of course now means that each time I unpack a Map in my Java code, I have to check whether it's a Map or an IPersistentMap, then either unpack or not, depending on whether it's Java or Clojure calling my method...

14:19 rhickey: what do you mean unpack?

14:19 meredydd: Well, I'm passing a complex pseudo-data-structure from Clojure to Java

14:19 the Java method expects a Map - that's fine, I can do that with (jmap)

14:20 But there are other maps stored within this map

14:22 rhickey: JavaMapOnPersistentMap class, implements Map and IPersistentMap

14:22 ?

14:22 meredydd: (for example, {"meredydd" {"A-level" {"mark" 55} "GCSE" {"mark" 60}})

14:23 rhickey: wouldn't need to be unwrapped

14:23 meredydd: How so?

14:23 rhickey: because (jmap m) would still be a Clojure map

14:23 meredydd: Yes...but it's a pain going in the other direction.

14:23 Imagine I pass that blob above into my Java code.

14:24 The Java code wants to find out what mark I got in my A-level.

14:24 rhickey: Look there ar eJava interfaces for all of the Clojure collections, use them in Java

14:24 meredydd: Yep, I know.

14:25 I didn't say it's impossible, just that it requires unpacking and forces bits of my Java code to know that the calling fn was written in Clojure rather than Java.

14:25 Anyway, dinnertime.

14:25 cemerick: meredydd: since the current impl of clojure maps doesn't line up with what your Java code is expecting, why not build Java maps in clojure?

16:32 JamesIry: Gang, some nerd humor for your enjoyment: http://james-iry.blogspot.com/2008/07/java-is-too-academic.html

17:00 cemerick: JamesIry: Nice, is the 'academic' bit a slight riff on the dysinger episode? :-P

17:01 JamesIry: cemerick: inspired by it and similar discussions about many languages that I like

17:03 cemerick: Yeah. The "academic" comments are rational coming from "trade" programmers, but it's almost always silly coming from anyone who's serious about it.

19:48 slava: JamesIry: i never understood why 'academic' is used as an insult in some programming communities

20:13 lisppaste8: fyuryu pasted "Is it lazy?" at http://paste.lisp.org/display/63932

20:13 fyuryu: calling the defined function in list comprehension is lazy, right?

20:24 JamesIry: slava, me neither.

20:27 Chouser: fyuryu: yes, seek-left should only be called until it produces a non-nil

20:27 you can put a prn in there to see what it does

20:31 fyuryu: ok, thanks. Doh!, I didn't think about prn, I just evaluated that in the REPL

20:32 Chouser: so I always got what I wanted, but wasn't sure abt the rest

Logging service provided by n01se.net