#clojure log - Jun 04 2008

The Joy of Clojure
Main Clojure site
Google Group
List of all logged dates

1:54 albino: this? http://vim.sourceforge.net/scripts/script.php?script_id=221

2:03 Ycros: could be. I should have bookmarked it

2:05 actually no, I think it was Limp (which is linked to from the one you found)

10:49 Chouse1: after I picking that little fight yesterday, I lost track of one of my initial questions...

10:49 Is it better to use (. Class forName "org.postgresql.Driver") or (. java.sql.DriverManager registerDriver (org.postgresql.Driver.))

10:50 ...or is there a better way altogether?

10:53 rhickey: I didn't get time to look at why simply naming org.postgresql.Driver was insufficient. But I kind of dislike Class forName - I'd go with the latter

10:53 (even in Java I've encountered this with JDBC drivers)

10:54 there is also more of a declarative, preferences-based registry I think

10:54 Chouse1: Ok. I hadn't seen any examples doing the later. I didn't know if there might be bad side effects or consequences.

10:54 rhickey: the whole thing is built on the driver self-registering as a side effect of loading - ugh

10:55 Chouse1: yeah. I'm sure whoever made it up thought it was rather clever.

10:56 rhickey: from the DriverManager docs:

10:56 As part of its initialization, the DriverManager class will attempt to load the driver classes referenced in the "jdbc.drivers" system property. This allows a user to customize the JDBC Drivers used by their applications. For example in your ~/.hotjava/properties file you might specify:

10:56 jdbc.drivers=foo.bah.Driver:wombat.sql.Driver:bad.taste.ourDriver


10:56 might be a better approach

10:57 Chouse1: ok

13:02 I just wrote a proxy that maps a vector of vectors to an AbstractTableModel.

13:03 About half of the Clojure method bodies are fewer characters than the Java method names.

13:03 rhickey: heh

13:03 Chouse1: (getColumnName [i] (nth cols i))

13:04 rhickey: doing some Swing?

13:04 Chouse1: I am. First time, really.

13:04 also first time using java.sql, as yesterday's outburst indicated.

13:04 rhickey: :)

13:06 Chouse1: Using doto, I can build up a swing widget tree using code that mimics the hierarchy pretty well, and without naming most of the intermediate panels.

13:06 ...which is pretty nice. The Java examples I'm looking at quickly become a tangle of awkwardly named local vars.

13:07 rhickey: using (JThis. ...) makes that pretty declarative too

13:08 Chouse1: Yeah I converted all my (new...)s to (JFoo. ...)s this morning. I'm still pretty ambivalent.

13:08 I think if I had my syntax highlighter pick out the JFoo.'s in a different color, that might make the difference.

13:08 rhickey: Groovy has 'builders' for Swing just for that, Clojure comes close with no special support

13:08 highlighting would be excellent, yes

13:10 la_mer: Has anyone really pushed around enclosure yet?

13:10 Chouse1: btw, I wrote my own result-seq because I needed a vector for each row instead of a struct map. With (.method obj) and #() syntax, is significantly shorter.

13:11 sorry, resultset-seq

13:11 la_mer: I'm meaning to, but haven't yet. I get the impression the creators are using it pretty heavily.

13:11 lunchtime.

16:23 for clojure to work in javascript, you might not need a lispreader in js, but surely you'd at least need the compiler, wouldn't you?

16:23 otherwise how would (eval) work?

16:27 rhickey: It depends, no eval is distinctly possible

16:29 Chouse1: as in, clojure might work even if it didn't have eval?

16:29 la_mer: wow, gen-class is a whopper

16:29 incredibly cool, though :-)

16:30 rhickey: there are 2 distinct ways to think about ClojureScript, one is a Clojure interpreter in JavaScript, the other is a compiler from Clojure to JavaScript (written in Clojure, and doing macroexpansion there)

16:32 Chouse1: right. They would have distinct feature sets or usage profiles, right?

16:32 rhickey: I think a Clojure (subset) to js compiler is more interesting then sending Clojure code to the browser for interpretation, YMMV

16:33 Chouse1: that would mean some kind of (to-js) function or macro that consumes a file or clojure code/data structures and produces a string, right?

16:34 rhickey: yes

16:34 Chouse1: someone mentioned starting that already.

16:34 cgrand

16:35 http://clojure-log.n01se.net/date/2008-05-29.html#17:56

16:35 rhickey: there are js correlates for many of the special ops, then just do macroexpansion on the clojure side to get more language, plus something like the Java syntax for talking to the js stuff

16:36 Chouse1: I'm trying to think if there are any features I think are really useful that would require a compiler/interpreter on the js side.

16:36 rhickey: a very early Clojure prototype was written in CL and emitted js

16:37 the thinking then was that js could be compiled in both java/.net, but .net's jscript had too many small differences

16:38 but a small lisp->js is pretty easy.

16:40 interesting quote here: http://bc.tech.coop/blog/030920.html

16:41 "Those of you who are familiar with more traditional functional languages, such as Lisp or Scheme, will recognize that functions in JScript are fundamentally the Lambda Calculus in fancy dress. (The august Waldemar Horwat -- who was at one time the lead Javascript developer at AOL-Time-Warner-Netscape -- once told me that he considered Javascript to be just another syntax for Common Lisp. I'm pretty sure he was being serious; Waldemar's a

16:41 Chouse1: that got cut off, but I've heard it elsewhere.

16:42 In fact, I'm pretty sure I've seen posts from the guy who started JavaScript that he was hired to create a "scheme with Java-like syntax"

16:44 http://weblogs.mozillazine.org/roadmap/archives/2008/04/popularity.html

16:44 rhickey: cross-compiling Clojure is very doable. I compiled to both Java and C# for a long time. The only thing that doesn't map well is blocks (do) as expressions - if only the had kept C's comma operator!

16:45 and loops as expressions

16:45 Chouse1: "Whether that language should be Scheme was an open question, but Scheme was the bait I went for in joining Netscape"

16:46 developing code for use in the browser would either require a compilation step, or clojure running in a server.

16:47 rhickey: I imagine most Clojure web apps would have Clojure running in a server

16:47 Chouse1: I suppose if you had the latter, you might with some effort be able to load updated functions without reloading the whole app in the browser.

16:47 rhickey: as cgrand is doing, js and html gen could happen together

16:48 most of the servlet containers have comet support now

16:50 funny, this just came by - ecmascript 4 to js compiler written in server-side Python:

16:50 http://ecmascript4.com/

16:51 Chouse1: huh.

16:54 rhickey: I think the important thing that makes the Clojure->js job small is that the macros get expanded by and in Clojure, so you never have js versions of cond, when etc, just if

16:54 Chouse1: I assume cgrand is just writing functions to do the conversion. Any reason to think that's not the best way?

16:55 la_mer: rhickey: Am I right in thinking that, assuming each namespace has appropriate metadata lined up, that gen-and-save-class could be used to dump a pile of loaded clojure to disk?

16:56 rhickey: la_mer: no, the Clojure code behind a gen-ed class is not in the .class file, just the stub

16:57 Compiled Clojure code can have data literals in it, more complex to pre-compile.

16:58 at some point I might look at an ahead-of-time compiler

16:58 la_mer: Ah, that's why I'm getting "UnsupportedOperationException: XXXX not defined" when I try to load up the generated class -- it's not finding a matching .clj file to load the impl's from.

16:59 rhickey: the impl can be loaded pretty late, but must be before it's called

16:59 can also be changed after objects exist

17:00 Chouser: I haven't looked too closely at cgrand's js stuff

17:01 but bringing up a Lisp is always the same - a handful of special ops to bootstrap

17:02 Chouse1: right, I'm wondering if you think implementing the conversion in macros and clojure funcs would be better or worse than some other option, like an alternate Compiler class.

17:04 la_mer: Yeah. I just caught wind of the gen-and-save-class utility, and thought that perhaps it would enable source-less distribution.

17:04 rhickey: Chouser: ah, well, certainly not a Compiler class in Java

17:05 Chouse1: :-) You would have written yours in clojure if you could have, would you.

17:05 rhickey: definitely, it was always a hope to bootstrap it, but practically, once you have a working compiler, you're pretty tired :)

17:06 Chouse1: heh

17:07 rhickey: Take the Clojure time machine: http://clojure.svn.sourceforge.net/viewvc/clojure/trunk/src/lisp/clojure.lisp?view=markup&pathrev=85

17:08 Chouse1: hm, looks like CL ;-)

17:08 rhickey: CL to Java/C# compiler for an early Clojure

17:11 I don't think you'd need a full Clojure in js, but the ability to gen js from sexprs with macros would be really nice

17:13 so you use macros to get a tree of special ops, then walk the tree and emit js

17:13 Chouse1: I've written quite a bit of js (we've got over 38K lines for our project at work), and I'd be pretty frustrated if the environment were crippled or any less dynamic than js is by itself.

17:14 yeah, that doesn't sound too hard, does it? I wonder how far cgrand has gotten.

17:14 rhickey: wow! 38k

17:14 Chouse1: yeah, scary, right?

17:15 that doesn't include the 3rd-party libs we use (zimbra, google maps, openlayers, etc.)

17:15 rhickey: keeping all of the js goodness would have to be a focus

17:15 but you'd take the same approach as with the Java - define minimal syntax support for direct calls

17:16 Chouse1: I guess one question would be -- how much would the data consumed by js* have to look like clojure source?

17:16 rhickey: do you eval your json?

17:16 Chouse1: yeah, but that could stay json.

17:17 rhickey: then either way, I think a js reader for Clojure data format wouldn't be too much

17:17 or translate to json at the gateway

17:17 Chouse1: you'd want js* to understand literal hash and vector syntax of course, but it clearly wouldn't need to do sync or agents.

17:18 ...so would it need persistent data or not?

17:18 rhickey: right, there isn;t a need to bring semantics to js it doesn;t already have

17:18 I don;t think persistent

17:18 * rhickey can't type '

17:18 Chouse1: I guess the clojure idioms might start to break down if you have js arrays instead of vectors. ...not sure.

17:19 rhickey: depending on your data size you can just copy on write

17:20 but it is an issue for Clojure idioms

17:21 Chouse1: I don't think having conj modify an existing array would be acceptable, for example. If you wanted to do non-persistent you'd have to use a different verb.

17:21 rhickey: yes, definitely

17:21 but Clojure has aset for Java arrays

17:22 Chouse1: ah, good point.

17:22 but being forced to use aset all the time would be painful.

17:22 rhickey: the main problem is data size - copy on write preserves the semantics but doesn't scale

17:22 but you may not need to scale

17:23 Chouse1: ...so I think that answers it (in my mind anyway) -- we'd need *some* kind of immutable list and hash on the JS side.

17:23 rhickey: list is easy

17:23 Chouse1: right, it might not need to scale, but it'd have to be there in order for it to feel anything like clojure.

17:24 rhickey: js has everything you need to implement the hash map

17:24 just arrays

17:25 Chouse1: hm. It'd be interesting to port hashmap to clojure so that it would work in js.

17:25 rhickey: Clojure leans heavily on Java's good ephemeral memory handling, not a known attribute of js

17:25 Chouse1: yeah, but again that mainly has to do with scaling. ..and the JS VMs will keep getting better.

17:26 rhickey: I think you'd write it by hand in js and preload is as runtime support

17:26 Chouse1: why?

17:27 rhickey: because you don't have a Clojure->js compiler yet

17:27 Chouse1: but you could have a COW vector to get you by... ;-)

17:28 or even just JS array support.

17:28 rhickey: object models

17:29 I'm not arguing against it

17:29 Just that, I would have had to have had Clojure very far along in order to have it generate PersistentHaspMap

17:30 ymmv

17:30 Chouse1: hm. Yeah, I guess the JS situation isn't any further along than you would have been with the Java version.

17:31 rhickey: It depends a lot on what the js would look like - functional or an object?

17:32 Chouse1: which in my experience hinges mainly on whether you want to use JS inheritence. If you don't need inheritance, closures are great for creating objects.

17:33 * rhickey just looked at PersistenHashMap - Clojure still couldn't generate it :)

17:33 Chouse1: gah!

17:33 rhickey: It leverages Java, as I think it should given the layer it is at.

17:33 when in Rome

17:34 Chouse1: what's missing?

17:34 I guess the flip side is having created hashmap in the host language, you can use it outside of clojure if you want.

17:35 rhickey: classes with typed fields, primitive bit math, static methods

17:36 yes, the Clojure Java lib has utility outside of Clojure, and can leverage all kinds of things I don't intend to add to Clojure

17:36 lest Clojure inherit the complexity of Java

17:37 Chouse1: but that's because it's a Java class. surely that kind of functionality could be designed for and implemented in clojure.

17:37 if not, then isn't that a problem?

17:39 rhickey: no, because in order to make a language go all the way down and match the efficiency it has to have primitive types

17:39 that was no to - isn't that a problem

17:40 yes you can implement abstractions like that, but they will have some cost due to their higher-level nature

17:41 Chouse1: oh, ok.

17:42 As long as the barrier is performance and not expressiveness, I'm ok. ;-)

17:43 rhickey: I think Clojure holds it own on expressiveness

17:44 Chouse1: yes

17:44 rhickey: OTOH, I'm pretty stunned that I was able to get the primitive math as fast as Java without emitting math primitive bytecodes

17:45 Chouse1: It feels a little odd to have to use java interop (proxy, gen-*-class) to create data structures that have compatible APIs with the buildins, but I guess that makes sense in light of your performance statement.

17:47 rhickey: a big benefit of that is that the resulting thing is native. virtual method dispatch is also very fast. Some other JVM dynamic langs interpose a wrapper/bridge class between everything

17:47 to map to their object models

19:24 Chouser: what's the time for this on your machine:

19:24 (loop [acc (double 0) ind (int 0)]

19:24 (if (== ind (int 100000000))

19:24 acc

19:24 (recur (+ acc

19:24 (. java.lang.Math (sqrt ind)))

19:24 (unchecked-inc ind))))

19:24 vs what you posted?

19:28 Lau_of_DK: Anybody know what the reason for implementing primitive support in Clojure was?

19:28 rhickey: speed

19:29 some of us do audio/image processing

19:29 nice to stay in Clojure

19:30 arbscht: avoids the "glue language" trap neatly

19:30 Lau_of_DK: oh - I just heard a talk you gave, rhickey, to a group of Lispers from NYC and just near the end I heard the sentences "there will never ever be primitives in Clojure" ?

19:31 rhickey: and there aren't, really. It doesn't generate the bytecodes for primitive arithmetic itself

19:32 It isn't something I expected to work this well. Never say never I guess :)

19:32 Lau_of_DK: Alright, thanks for clearing it up :)

19:32 Good talk though - You got quite the applause

20:01 Ycros: rhickey: hey, I saw an IRC log of you saying that Clojure doesn't need conditions/restarts because it can be implemented using dynamic function binding?

20:02 rhickey: is there something I can read, or can you shed some light on how this might be done?

20:03 rhickey: http://groups.google.com/group/clojure/msg/842e8bb058015282

20:03 Ycros: thanks :)

20:04 rhickey: sure. note I said similar, not exactly CL conditions

20:05 Ycros: oh I see, that's fairly simple

20:05 rhickey: simple meaning easy or simplistic?

20:10 Ycros: easy I guess

20:11 rhickey: you can build richer abstractions on that if needed

21:26 Chouser: ah, I was trying to use a primitve ind, but was missing the comparison literal.

21:28 but still, my posted loop ranges from 3132 to 3293 msecs, while rhickey's with unchecked-inc ranges from 3121 to 3293

21:37 rhickey: you missed my answer, and nothing else.

21:37 I had tried to use a primitve ind, but missed the comparison literal.

21:37 but still, my posted loop ranges from 3132 to 3293 msecs, while yours with unchecked-inc ranges from 3121 to 3293

21:38 rhickey: unchecked-inc doesn't matter, but I think declaring ind int does - yours went from 5secs to 3.5 here

21:39 Chouser: oh! hm. I wonder why the difference between the improvement here and there.

21:40 Lau_of_DK: this might be a really dumb question, but is the swank-clojure any different from a regular swank ?

21:43 rhickey: Chouser: I don't see how they could be the same speed - hmmm....

21:44 since both the compare and the inc have to be generic without the hints

21:44 are you java 6?

21:46 Chouser: right. When I tried (int 0) for ind but forgot (int 1000...), the performance was worse, I assume because of extra boxing and unboxing.

21:46 rhickey: you are on java 6?

21:48 Chouser: I think so. I'm trying to verify that.

21:49 rhickey: java -version

21:49 Chouser: ah. yeah, 1.6.0-b09

21:50 rhickey: interesting. I'm stuck on 1.5, I've heard the optimizations are much better on 6, looking forward to that. I guess the main point from the benchmark was that Clojure is the fastest ;)

21:51 Chouser: :-)

21:51 rhickey: To be fair I think Kawa supports some hints that would make it faster

21:52 And I'm not sure that is the maximum opt from SBCL

21:52 Chouser: yeah, it's hard to be fair in a comparison like that without knowing each language pretty well.

21:52 rhickey: but Clojure should always be == Java in this area now, which means plenty fast

21:52 Chouser: I thought a C version might be interesting.

21:53 for some value of interesting.

21:53 rhickey: sqrt is bad, some arithmetic would be better

21:53 rms or something

21:54 Chouser: Java's sqrt is bad, or what?

21:54 rhickey: sqrt is not primitive arithmetic

21:54 Chouser: ah, true.

21:56 rhickey: and dominates his benchmark

21:56 fyi, from REPL:

21:56 (-> System .getProperties (get "java.runtime.version"))

22:01 Chouser: thanks

22:04 (reduce + (map #(. java.lang.Math sqrt %) (range 10000000)))

22:06 That's just hopeless, isn't it.

22:06 rhickey: 2.1081849486439312E10

22:07 Chouser: If I replaced #() with something defined using definline, range would still be returning boxed numbers, right?

22:07 and + would be generic?

22:12 Lau_of_DK: rhickey, what are the terms for building commercial software off clojure?

22:14 rhickey: Chouser: takes 4 secs here

22:15 Chouser: yeah, it's 1/10 the benchmark. sorry for the confusion

22:16 10000000 instead of 100000000. how could you have missed that? ;-)

22:16 Lau_of_DK: http://clojure.org/license.html

22:16 rhickey: sorry, doing 2 things at once here

22:18 Lau_of_DK: Thanks Chouser :)

22:20 rhickey: Chouser: laziness in map will dominate:

22:20 (time (reduce #(+ %1 (. java.lang.Math sqrt %2)) 1 (range 10000000)))

22:24 Chouser: that's still 898 msecs instead of 327 msecs for the loop version.

22:29 rhickey: I consider that pretty good, considering range is a seq abstraction

22:31 Chouser: oh, sure! I'm not saying it's bad, I'm just kicking around with more idomatic forms.

22:38 rhickey: the cool thing about the primitive math is that it is amenable to being the target of higher-level macros, like amap and areduce

22:39 I can imagine whole DSLs for different formula-building patterns

22:39 but it will be macrology, the higher-level constructs have some fixed costs

22:44 and 10 million steps of reduce and range in a second is fine perf for most apps. Only serious crunchers need get into this micro-optimization

22:44 Chouser: yep, good point.

22:45 JamesIry: rhickey: This is somewhat off topic of Clojure the language and more about its implementation. I've been reading papers on various STM impls lately. I remember you writing that Clojure uses MVCC. Do you have any links about the kinds of workload that MVCC based STM is good or not good for? Or has there even been any research into MVCC STM?

22:47 rhickey: JamesIry: I read a bunch of papers on STM while I was designing Clojure's. They all left me wondering a bit about the context of use...

22:48 i.e. they seemed to focus on dropping STM under otherwise mutation-oriented apps...

22:48 fine grained field-level refs...

22:48 JamesIry: Agreed. Many of the benchmarks they user are hashtables or arrays where everything is a mutable ref.

22:48 s/user/us

22:48 rhickey: while I always was thinking about refs to immutable persistent data structures, more of a record orientation

22:48 JamesIry: s/user/use!

22:49 rhickey: so turned to databases for inspiration

22:49 as I consider STM to be the same problem in many ways

22:49 JamesIry: Very much so!

22:50 rhickey: spent lots of time with Gray and Reuter's "Transaction Processing"

22:50 and whatever info I could get on Oracle and Postgresql

22:53 I wasn't aware of another MVCC STM when I did my work, but this came up recently:

22:53 http://www.esw.inesc-id.pt/~jcachopo/jvstm/

22:53 JamesIry: I think it would be interesting to investigate the kinds of workload where the overheads of MVCC are small compared to the overheads of redoing the work, vs where the overhead is large. One interesting possibility is to have a pluggable STM impl so that application writers could do their own perf testing to determine which fits best.

22:54 Thanks for the link

22:55 rhickey: after reading Gray and Reuter I would never want anything other than MVCC, in an STM or a DBMS

22:55 JamesIry: :)

22:55 rhickey: not that they advocate it, my own conclusion

22:55 Lau_of_DK: rhickey, "never" ?

22:55 rhickey: for some value of never

22:55 JamesIry: Yeah, how 'bout a very low contention scenario

22:56 rhickey: I just don't see where MVCC loses

22:56 low contention everything works well

22:57 Gray and Reuter do show some serious disdain for optimistic concurrency

22:57 That's why Clojure has real blocking

22:58 JamesIry: One paper I've been studying (http://www.cs.rochester.edu/~vmarathe/research/papers/2008_PPoPP.pdf) has an algorithm specifically optimized so that low contention is cheap. It's C based, though, and may be too expensive to translate into JVM terms.

23:01 rhickey: will read, thanks

23:01 JamesIry: The central part is based on 64 bit CAS - and that's not something that you can rely on being reasonable on every JVM

23:03 I'll have to pick up the Gray and Reuter book

23:03 rhickey: I have a philosophical problem with allowing more work to proceed than can succeed

23:04 JamesIry: rhickey: I hear you. It's okay if the work is cheap compared to the costs of blocking...but at the datase level that's almost certainly not true. When dealing with in memory refs to large immutable structures it probably still isn't true.

23:05 That's one problem with the benchmarks that many STM papers use. They pound the hell out of the refs, but few actually do much work in between reads and writes.

23:06 rhickey: I just see no correlation to the reality I;ve experienced writing MT programs in those papers

23:06 JamesIry: It's a classic microbenchmark miopia.

23:06 rhickey: For me, STM is a way to automate and make correct the same kind of locking that would be done manually in a real program

23:07 for super-contended caches the stuff Cliff Click is doing will work fine, I wouldn't try to do that with STM

23:07 JamesIry: Sure...but in some cases doing CAS and retries makes sense when you do things manually.

23:08 Yeah...his stuff is definitely the kind of thing where special purpose constructs like concurrent maps make more sense.

23:08 rhickey: he's in the CAS world

23:09 there's likely no one-size-fits all

23:10 JamesIry: That's why it would be interesting to see a suite of benchmarks againt various STMs. Some benchmarks could do real work so that there's a cost to any retries. Others could just pound crap out of the refs so that blocking itself becomes a major overhead.

23:11 Users and designers of STMs could then evaluate which work load is more nearly like their own - or, as I was thinking earlier, empirically try several impls.

23:12 rhickey: It's an area (realistic benchmarks) that needs attention

23:13 JamesIry: One thing I'm excited about is languages like Clojure taking concurrency seriously as a first class concept.

23:14 rhickey: It's got to be more than an add-on or it won't work. I don't think people can just drop STM into the same old approach. Immutability has to be taken seriously, for instance

23:15 JamesIry: Agreed.

23:16 rhickey: I would hope Clojure offers a model (persistent data structures in refs) that is compatible with it's STM approach

23:16 JamesIry: It's going to be a learning curve, though. It will take a while for people to wake up from the illusion that state can be encapsulated.

23:17 The only way to encapsulate state is to limit the scope of what it can possibly affect - OO doesn't do that at all well. The state of one object affects the transitive closure of all objects with references to it.

23:18 rhickey: A recent thing I added to Clojure allows you to make Java objects whose state was a ref - transactional objects if you will. Still thinking through the implications...

23:19 JamesIry: Not sure I follow

23:19 rhickey: the problem with OO is the direct references to mutable things. Making them indirect and transactional, to immutable things, can let it feel quite similar while ensuring that people can only ever see consistent objects, without needing locks

23:19 JamesIry: I got ya

23:24 rhickey: one of the nice things about using a ref to an immutable map, say, for an object's state, vs a bunch of fields each of which was an STM ref, is that you can, outside of a transaction, get a consistent object

23:27 JamesIry: Yup. The context in which I've been exploring STM is to create one for Scala. Scala's story on immutability isn't as good as Clojure's, though. It's got immutable persistent structures in the standard libary, but it doesn't really discourage mutable objects either. So I forsee much trouble with people trying to play with ordinary mutable objects in transactions and getting strange results.

23:28 Chouser: Scala doesn't have stong immutable vectors or hashes, does it?

23:29 JamesIry: Vector, no. Set, hash map, etc yes

23:29 rhickey: biting the bullet on immutability for all of the core data structures is one of the best things I did for Clojure. It's worked out better than I thought and I've gotten very little pushback from users, surprisingly.

23:29 Chouser: hm. I must have missed that.

23:30 JamesIry: http://www.scala-lang.org/docu/files/api/scala/collection/immutable$package.html

23:32 rhickey: what are the perf characteristics for the hash map O(log N) ?

23:32 immutable != persistent

23:32 JamesIry: Most of their structures are persistent

23:33 Chouser: are those copy-on-write?

23:33 JamesIry: I think there's some ugliness in the way hashmap is done, though. I vaguely recall that that it was high on the list to get reimplemented.

23:33 Chouser: I don't think they used copy on write

23:34 rhickey: persistent just meaning old versions still available, or persistent meaning with perf characteristics intact? Amortized or not?

23:34 if you happen to know...

23:34 JamesIry: I don't know all the perf characteristics. Some are amortized using laziness and whatnot.

23:35 It would be interesting to get you and them in a room. You guys are solving similar problems on the same platform.

23:35 Room could be virtual, of course :-)

23:35 rhickey: Yes, I'd love to. I have to admit to having spent almost no time with Scala :(

23:36 JamesIry: Well...it's a bit like Java and ML had a love child.

23:36 Very different design philosophy from Clojure!

23:37 rhickey: Clojure owes some bits to ML

23:37 no type system though

23:37 JamesIry: Yeah...I gathered the explicit ref types was a bit inspired by ML

23:37 rhickey: exactly

23:38 JamesIry: One of the things on my list is to investigate the Clojure/Scala boundary. Both will encode closures differently, for instance, but it seems like it might be easy enough to translate.

23:39 rhickey: Well, Clojure is pretty transparent to Java - there are interfaces for everything, it uses the same stack and calling conventions, and there are no magic arguments, trampolines, CPS etc

23:40 no wrappers

23:40 JamesIry: Same with Scala - that's one area where both languages took the same philosophy: embrace the platform don't try to beat it into submission.

23:40 rhickey: right

23:41 JamesIry: In fact, with Scala if you stick to the Java-like subset you get pretty much the same bitecode as you'd get from Java. Things like closures just compile to instances of interfaces called FunctionN (where N is the number of arguments).

23:42 s/bitecode/bytecode

23:42 Sigh...fat fingers tonight

23:42 rhickey: Clojure has a single interface, IFn, with overloaded invoke method for different arities

23:42 JamesIry: Ah

23:43 rhickey: allows you to pass a variadic function object to map et al

23:44 JamesIry: Right...good point. Scala's variadic functions are more like Java - the variadic argument becomes a single argument. The difference is Java uses and array, Scala uses an immutable list.

23:44 I suspect there are a fair number of differences like that. Many of them largely driven by the dynamic vs static nature of the languages.

23:45 rhickey: yes

23:46 I expect between the two we cover a lot of what people with this, er, mindset might want

23:46 JamesIry: Yup...the languages aren't going to attract the same crowd.

23:46 Chouser: I was attracted to both.

23:47 ...though I guess I'm pretty firmly here now. ;-)

23:47 JamesIry: Well, me too :)

23:47 rhickey: I do think they both attract people interested in programming at a higher level than Java.

23:48 People will divide on static vs dynamic as always

23:48 JamesIry: Of course. Plus the standard Lispy macro stuff to which Scala has no answer at all right now.

23:49 rhickey: I see the next divide being about mutability, where Clojure and Scala may end up on the same side, the FP guys having gotten there first, but Java and the scripting langs having no good story

23:49 JamesIry: Well, I shouldn't say no answer, things like optional laziness and a reasonbly flexible syntax allow Scala to do SOME of the things I might want to whip up a macro for in a lisp. But only some.

23:50 rhickey: that is THE divide. the static vs dynamic flame wars are silly when compared to something as fundamental as that.

23:51 rhickey: and yet, most people still don't even see that coming

23:52 multicore is going to smash into imperative programming like a train wreck

23:54 Chouser: every method of scala's hashmap is synchronized -- that means locked, right?

23:54 JamesIry: The hashmap impl is the one that sucks right now. I know its on their list

23:54 Chouser: even the get method, apparently because a get can trigger a mutation.

23:54 they should just use rhickey's.

23:54 ;-)

23:54 JamesIry: :)

23:55 Damn straight!

23:55 rhickey: Chouser: that's not inherently wrong, if they yield effective immutability

23:55 Chouser: I can't think why they shouldn't. Same license, I believe. rhickey's is Java, so they could just start using it.

23:55 rhickey: I don't think it's incorrect, just not as efficient.

23:55 JamesIry: Chouser: The api would be wrong, but the core algo would be fine

23:56 rhickey: it's not wrong, it's just silly. Plus, if I expect I'm using a normal immutable and persistent structure I'm not going to think about things like deadlock

23:57 Chouser: it looks like they keep a singly-linked list of changes to the structure, and occasionally re-collect into a new hash table. ...but I could be reading it wrong.

23:57 rhickey: those are all possible implementations, delta chains etc

23:58 would need to be synchronized

23:58 Chouser: JamesIry: sure, but tidy little scala wrapper around PersistentHashMap would be easy.

23:58 rhickey: not necessarily a deadlock risk, if there are no nested locks

23:58 I tried a bunch of things like that

23:59 JamesIry: rhickey: that's a good point. I suppose if some equals or hashmap method on a key was synchronized there could be risk of deadlock, but that's a pretty pathological case.

Logging service provided by n01se.net