#clojure log - Jun 12 2008

The Joy of Clojure
Main Clojure site
Google Group
List of all logged dates

0:32 jcrites: hey... I just saw a neat presentation about Clojure

0:33 I was wondering -- if I were going to convert some Java code to Clojure, how would I implement the visitor pattern? is there a better Clojure idiom?

8:12 cgrand: rhickey: wow, thanks for working on reduce

8:13 rhickey: cgrand - wait, let me check it in...

8:14 ok, now up

8:14 reduce problem was a bug from my perspective

8:14 cgrand: cool, thanks

8:15 rhickey: but I would caution that you beware the REPL itself can hold onto arg exprs, so when you want to test this, wrap in a let/fn/times or something

8:18 cgrand: ok

8:18 btw I ran into the reduce problem while playing with Wide Finder 2

8:19 rhickey: ah, I've been meaning to look at that, glad you are

8:22 cgrand: I translated Bray's ruby script to Clojure nearly line by line -- it surprised me to not have to fully rewrite

8:23 It's certainly not the most idiomatic clojure code...

8:23 rhickey: I fiddled with WF1 a bit, wrote about it in the group

8:24 I think Clojure's strength will be in the ease-of-parallelization area

8:24 http://groups.google.com/group/clojure/browse_frm/thread/7f2180bba2bb67c3/ca161d86b3d8045f?lnk=gst&q=wide+finder#ca161d86b3d8045f

8:24 many new features make that even easier now

8:25 pre regex support

8:25 pre destructuring

8:25 cgrand: yup, right now I have a parallel version which is the serial one with a pdoseq macro instead of a doseq

8:25 rhickey: that is very cool

8:26 cgrand: but it not performs pretty well on the T2000

8:26 -> it doesn't perform

8:26 rhickey: wf1 wasn't very parallelizable

8:27 if you want input let me know

8:27 cgrand: you still recommend to use agents to dispatch lines ?

8:28 rhickey: not sure about the granularity, lines may be too small

8:29 there are 2 aspects to performance here - one is absolute perf, the other is speedup due to parallelization

8:29 if you get the latter, it's still a benefit

8:31 the former can be a sinkhole as people do block i/o etc

8:31 really a waste of time

8:31 but should try to beat Ruby :)

8:31 lisppaste8: cgrand pasted "Wide Finder 2 (serial code)" at http://paste.lisp.org/display/62110

8:49 cgrand: this code is already faster (but not that much) than Ruby: on my box for 10M lines I get 3'45" (Ruby) and 2'45" (Clojure)

8:51 blackdog: cgrand, is the regex compilation cached?

8:52 cgrand: afaik the compiler turns regex literals into constants

8:54 (the constants being the compield java.util.regex.Pattern

8:54 )

8:56 blackdog: ok, that's interesting, so one doesn't need to re-pattern in advance then

8:59 cgrand, is that jruby or cruby your testing on?

8:59 cgrand: cruby

9:09 rhickey: blackdog: yes, regex literals #"..." are compiled on read

11:40 Lau_of_DK: Any body here got some experience with the 5 Guess Mastermind algorithm ?

11:48 rhickey: Lau_of_DK: please try to keep the dialog here focused on Clojure, thanks

11:50 Lau_of_DK: rhickey, I'm entering (or trying to) a 5 guess algorithm written in clojure in a contest - does that make it relevant ?

11:51 rhickey: only if related to the Clojure part

11:55 Lau_of_DK: rhickey, I understand that you want to keep it relevant - but this is one of the only places I get to hang out with Clojure coders, and everybody here code in Clojure so I imagine any general discussion about an algorithm would end up in actual code talk

11:55 Personally I would really enjoy it, if there were freedom for that kind of discussions, without keeping a tight censorship

12:18 yrb: is the (Class.) syntax to create a new instance only in the svn branch? and is a good idea to follow the svn or better to stick with releases?

12:19 rhickey: (Classname. ...) is svn only

12:20 I think many here track svn

12:25 Lau_of_DK: Questions about the emacs stuff: When I start up swank clojure, I have 2 seperate REPLs running, one is *inferior-lisp* and one is *slime-repl clojure* - is this the same for you guyS?

12:38 If I do (recur (inc cnt) (take cnt fibs)) and cnt start on 1 - will this then take 2 ? Or is cnt not incremented until the loop restarts?

12:47 nvm

14:12 albino: rhickey: watching the "clojure for java programmers", thanks lots of good info. One thing if I may make a suggestion, next time if possible can you restate the audience questions? Or maybe talk the hosts into giving audience member mics.

14:13 rhickey: Yes, I'm terrible with that... sorry

14:13 albino: rhickey: np, still way worth the time to watch

14:19 rsynnott: Lau_of_DK: that's normal; you want to use the slime-repl one

14:19 Lau_of_DK: rsynnott, okay thanks - and yea thats what Im doing, just wondering why I had a second

14:22 rsynnott: I think the other is the lisp's stdin/out)

14:22 or the clojure's, in this case :)

14:23 Lau_of_DK: oh ok. One of the gabs in my "Lisp education" is that I never really found any good explanations of all the workings of emacs, slime, swank and all that. Got any good links? :)

14:24 rsynnott: nope

14:24 slime is more or less a link to the remote repl, with convenience stuff and a fancy inspector and so on

14:24 it uses data from the remote lisp to do indentation and completion and similar (which is very nice)

14:25 I wonder will someone do CUSP for Clojure? :)

14:25 (CUSP is a Java replacement for the Emacs bit of slime)

14:26 Lau_of_DK: rsynnott, thanks for the explanation, youre a link in your own right :)

14:27 I'm not sure how difficult it is to build a editor/repl interaction, but not taking that into account, it wouldnt be a huge task writing a good IDE for clojure

14:30 rhickey: have you tried enclojure?

14:31 Lau_of_DK: Yes, I've tried it

14:32 rsynnott: I've tried one netbeans UI for non-java-language before; it isn't an experience I'd care to repeat :)

14:32 I'm happy with SLIME

14:32 (it'd be nice to have an inspector and so on, of course)

14:33 Lau_of_DK: I never thought I'd say this, but I'm actually pretty happy with the emacs integration. If Enclojure gets a better interaction with the REPL and a few other details sorted out, I might switch to it though

14:33 ericthorsen: rsynott: what problems did you encounter? We are building the enclojure plugin for clojure. Have a REPL, code browser, debugging support, etc.etc.

14:34 There is still much work to do but Netbeans has taken care of quite a bit of heavy lifting.

14:34 http://www.enclojure.org

14:36 rsynnott: I just didn't like netbeans very much, really :)

14:40 Lau_of_DK: ericthor, can you evaluate single defn's directly to the REPL in Enclojure yet?

14:56 In C#, if I wanted "Bits" to become "Bitz" I could just do str[3] = "z" - Is there anyway I can do that operation from Clojure on a java string?

14:57 Chouser_: you want to mutate the string!?! ;-)

14:59 (str (.substring "Bits" 0 3) "z")

14:59 Lau_of_DK: Yes Chouser, I have this one stupid assignment that I've spent an hour on now, its driving me crazy

15:00 Chouser_: (.replace "Bits" "s" "z")

15:05 rsynnott: assignment?

15:05 are you using this for college stuff or something?

15:06 Lau_of_DK: No its just a project

15:07 I'm trying to do a Mastermind puzzle solver

15:27 rsynnott: ah

15:27 cool :)

15:27 I did one of those years ago... in C

15:27 yuk

15:27 no, actually, it was in turbo pascal :S

15:34 blackdog: jumping in, i have a start of a clojure plugin for jedit,

15:34 it has a console and can run jedit macros, but i'm having difficulty evaling a buffer

15:35 i'll post what i have if anyone is interested

15:38 well not evaling, but it doesn't seem to use the same classloader as the console, cos i can't find the vars in there

15:38 Chouser_: blackdog: clojure uses its own classloader

15:39 rhickey: but that classloader is a child of the one used to load Clojure

15:40 blackdog: i'm not sure, i was going well, with clojure macros being able to populate the clojure console,

15:40 with stuff, but doing the same kind of thing by evaling the buffer didn't work, maybe slava will help :)

15:40 jedit is his baby

16:36 Lau_of_DK: I have a long function that is throwing a Typecast error - Is there anyway I could get a linecount that fires this exception, instead of just 20 lines of java dump?

16:51 rhickey: new release, new site up

17:00 la_mer: oooh, shiny :-)

17:04 rhickey: regarding the license -- have you gotten any more feedback on it w.r.t. commercial *use* of clojure? (I'm remembering now the slight dust-up over the license some months ago on google groups)

17:05 (FWIW, I agree with your interpretation of the CPL, I'm just wondering if anyone else has expressed concerns / disagreement with your interpretation)

17:10 Chouser_: Lau_of_DK: aren't there line numbers on each line of the stack trace?

17:11 Lau_of_DK: Yes, but to files like boot.clj, swank-clojure.clj and so on

17:12 I just want something like Cast exception: Line 5, Col 2, in my own file

17:14 Chouser_: Lau_of_DK: perhaps you can nopaste the function and stack trace?

17:14 Lau_of_DK: nopaste?

17:14 Chouser_: http://paste.lisp.org/new/clojure

17:17 Lau_of_DK: thanks, i'll get to a little later, I have to finish something before bed

17:26 rhickey: la_mer: no complaints for commercial use. May be some GPL incompatibilities

17:28 slava: rhickey: nice site

17:28 what CMS are you using?

17:31 rhickey: that's wikispaces

17:31 slava: hopefully one day it will be something written in clojure :)

17:31 rhickey: I guess, if someone's interested in that domain

17:37 la_mer: rhickey: that's a plus, IMO ;-)

17:37 slava: why did you pick CPL and not BSD?

17:40 rhickey: because I want derived work to give back

18:01 slava: i rewrote your persistent-vector implementation in factor; is this code bound by the CPL?

18:01 does a rewrite count as a derived work

19:20 rhickey: slava: I think that depends on how much of a copy it is and how much original - it might

19:27 slougi: hi. is there an example somewhere on how to use external java libraries from clojure? I'd like to play with using it in conjunction with Qt Jambi.

19:28 rhickey: if they are in your classpath, you can import them just like built-in Java libs

19:29 slougi: how would that be? (I am not entirely fluent in Java)

19:29 rhickey: how to put in classpath or how to import?

19:30 slougi: how to import :)

19:30 rhickey: parallel.clj, which comes with Clojure. imports the forkjoin library - you could look at that

19:31 slougi: thanks for the pointer. and sorry for the newbie questions, i honestly didn't find a good answer before.

19:32 slava: rhickey: have you considered adding compile time type checks of any kind to clojure?

19:32 rhickey: a little bit - since I track types for reflection elimination

19:33 I like the idea of optional checks that generate warnings

19:36 slougi: doc for import is in: http://clojure.org/namespaces

19:45 slougi: rhickey: cool, i missed that before. I was looking in java interop. thanks =)

19:50 wow, this is very cool

19:50 the first lisp that actually interacts with the toolkit i want to use in a decent way

19:52 slava: clojure has a lot of potential, it is well-designed and it will hopefully get more people interested in lisp in general

19:52 Ycros: oh wow, when did the site get a redesign

19:52 slava: my interest in clojure stems from its use of persistent data structures and iterators. i don't care about java interop so much :)

19:52 slougi: it's the first lisp that i like the feel of to be certain

19:53 njbartlett: I want to call the Compiler.load method from Java, but I need to specify a classloader. Can I somehow put it into the LOADER static field?

19:53 rhickey: Ycros: redesign is still underway, but had to cut over with the release since that's where the new docs are

19:56 njbartlett: could you explain a bit why?

19:57 njbartlett: rhickey: OSGi integration. I want to compile some Clojure code contributed by a bundle, using that bundle's classloader

19:57 rhickey: for use only by that bundle?

19:58 njbartlett: rhickey: Not necessarily, the result could be shared

20:00 rhickey: Hmm I guess the clojure-generated classes are never going to form exportable API from that bundle, but instances could find themselves in other bundles as services

20:01 rhickey: I know nothing about OSGi, but wonder how cross-sibling sharing works, normally couldn't be seen

20:02 njbartlett: OSGi doesn't use hierarchical classloaders. Instead they're arranged in a graph, with explicit imports and exports of Java packages

20:04 rhickey: The classes Clojure generates on the fly should never be seen directly by name, only through interfaces in the Clojure libs, e.g. IFn

20:04 njbartlett: But just thinking aloud here... to be able to write exportable classes in Clojure, I would have to get the compiler to generate a class from a clj in response to a class load event, which means I need a special classloader

20:04 rhickey: Hmm but you can generate interface instances as dynamic proxies right?

20:05 rhickey: exactly - I can;t depend on the supplied classloader's ability to load bytecode from memory - I must create my own - but could parent from supplied loader

20:05 njbartlett: Right that's what i need to work out, how to supply you a parent classloader

20:05 rhickey: interface instances yields instances of the interfaces, at least with proxy, so there too name is not exposed

20:05 njbartlett: I assume your classloader does normal parent-first delegation

20:05 rhickey: it does

20:06 I sued to use the context classloader, which would have given you a way...

20:06 used

20:06 njbartlett: Right. But now?

20:07 rhickey: now it's hardwired to RT.ROOT_CLASSLOADER

20:07 you could, as an experiment, make that non-final and set it temporarily to try your idea

20:08 njbartlett: Uhuh. I was already thinking this wasn't going to work without submitting back patches

20:08 rhickey: let's see if it works at all first. I'm willing to work on making this hookable

20:09 njbartlett: BTW my codename for this little project is Closgiure :-)

20:09 rhickey: yikes

20:09 :)

20:10 I think there are some fundamental conflicts between these graph-like classloaders (and need to re-parent all the time) and gen-and-load-class

20:10 njbartlett: So ROOT_CLASSLOADER is static, like everything else... that means if I set it temporarily I can only compile in a single thread

20:11 rhickey: you can try it out and I can make it thread-local with vars

20:11 njbartlett: Okay

20:12 rhickey: if you see compiler.load does a lot of that already

20:13 njbartlett: Yeah I'm just trying to get my head around your, um, unorthodox Java coding style!

20:14 rhickey: I still question why you need to use that other classloader

20:14 what happens if you don't?

20:17 njbartlett: Say the Clojure code imports package org.foo. That package will be visible to the containing bundle because it imports org.foo at the bundle level, but it won't be visible to the Clojure bundle itself. Essentially I want to make a Clojure bundle that performs a "compilation service" for other bundles.

20:17 rhickey: I don't think that is sound

20:18 njbartlett: Okay, I don't know enough about Clojure yet. It's an experiment.

20:18 rhickey: Clojure is a compiled language, whose compiled named entities are static and global (per classloader), just like Java's

20:19 i.e. there's no difference between clojure/rest and Math.max

20:19 njbartlett: Right, but your compiler is executed at runtime

20:19 rhickey: doesn't matter, it's atill trying to obtain compiled performance, so names must be bound, not looked up every time

20:20 again, like Java, code that uses Math.max gets linked/bound once to a global entity

20:20 njbartlett: What do you mean, names must not be looked up every time?

20:21 rhickey: The alternative to my unorthodox static style is to put names in instance environment objects in which things are looked up

20:21 njbartlett: Well no it doesn't. Somebody asks their classloader to load the Math class, and that gets delegated up to the boot classloader, and then the static method if invoked.

20:22 In theory there could be an alternative implementation of Math. Actually not because of a special rule (only the boot classloader is permitted to define classes in the java.* namespace)

20:22 rhickey: once only, it's not looked up every time, and it's looked up in a global place

20:22 per classlader

20:22 classloader

20:23 njbartlett: Right, I don't see what this has got to do with my approach to integration with OSGi

20:24 rhickey: Having one Clojure and many unrelated classloaders puts the onus on Clojure to work like OSGi

20:24 which provides some non-hierarchical sharing

20:24 njbartlett: I don't believe it does

20:25 rhickey: let's set the class part aside. If you load some fn ns/foo, that is globally available to all modules using the same Clojure

20:26 Which wouldn't conflict with the Java hierarchical model, since they would all be beneath whatever loaded Clojure

20:26 njbartlett: Ah I see what you mean about the sharing your do

20:27 How do you make these things visible? It's clearly not just via classloaders

20:27 rhickey: it kind of is ,since I use Java's static semantics

20:28 njbartlett: Right but you assume you always have the same parent classloader

20:28 rhickey: so the module that loads Clojure is like Clojure's version of the System classloader

20:29 it's a root for everything running under that Clojure

20:29 njbartlett: I see.

20:30 rhickey: using statics ensures that if you loaded a second Clojure under another clasloader, it would be independent

20:30 completely

20:30 njbartlett: Okay, I have tried separate instances of Clojure, one per bundle, but it's just too damn slow for that.

20:30 Like 5 seconds startup time

20:31 Fine if you're only doing it once for the whole JVM...

20:32 rhickey: then I would say that you need to take a different approach to modularity for using a shared Clojure than classloaders - using Clojure's namespaces for instance, because the OSGi separate classloaders aren't really giving you any isolation in this case

20:32 njbartlett: Right. In other words, Clojure doesn't add any value to OSGi and vice versa. Or so it seems.

20:34 rhickey: possibly not, but with understanding of how it works we might be able to address your objectives in using OSGi

20:35 because you certainly can share an instance, and Clojure has very good thread independence

20:36 and namespaces

20:36 njbartlett: I'm committed to OSGi so the question for me is whether Clojure can be useful to me in that environment.

20:37 rhickey: I've talked with people using NetKernel about using UUID named tear-off namespaces

20:37 njbartlett: Right I followed some of that thread

20:38 rhickey: your loader service could load code in a guaranteed unique namespace, and deliver IFns to consumers, dropping the ns after load

20:40 njbartlett: I'm going to have to learn more about Clojure and come back to this. I think I've come at it with some invalid preconceptions from doing Scala etc

20:41 rhickey: feel free to follow up on the group

20:41 njbartlett: Will do, thanks

20:49 * Chouser pays the bills in ruby while dreaming of clojure on rails.

20:49 Chouser: or something.

20:52 rhickey: ruby?

20:54 Chouser: yep. on rails. It was my choice as or a year or two ago. Now I have to live with it.

20:55 "as of"

20:55 rhickey: would you want a full-stack rails-alike for Clojure?

20:57 Chouser: not sure. The main benefit I was hoping for from Rails was an architecture with pluggable features.

20:57 need a user authentication system with email-reset of password? plug it in. that sort of thing.

20:58 If properly done, I would think a modular (non-full-stack) system could allow for that.

20:58 ...and in practice, rails doesn't even do a particularly good job. "plug it in" includes generating code in your app source tree, which rather complicates upgrades.

20:58 * rhickey hates ORM

20:58 dudleyf: Chouser: Have you looked at Merb?

20:59 I'd like to see something more like that for Clojure

20:59 Working with ActiveRecord has made me hate ORM, too

20:59 Chouser: I haven't done enough of this kind of app to know how to design an API that cuts across at the "right" places. I was hoping Rails would be an easy way to import that kind of expertice, but I'm not overly impressed.

21:00 I've always been a bit cold to ORM. If I knew of a good replacement for the relational database, I'd definitely look into that.

21:00 dudleyf: peeked at it, but it's way too late now.

21:01 rhickey: heh, I think the problem is the O part

21:01 dudleyf: I like the database part, It's the objects that bother me

21:01 rhickey: not the R

21:01 dudleyf: Heh, yeah

21:01 rhickey: I've done OODBMS, no mapping, it was not a good experience

21:02 Chouser: they both seem a poor fit. My data doesn't really fit neatly into tables, and after forcing it there both I have to work pretty hard to express what I mean, and the DB has to work pretty hard to make it performant.

21:02 dudleyf: I don't have much experience with JDBC, is it possible (easy) to get back something that looks like a seq of maps?

21:02 rhickey: that can happen too

21:02 JamesIry: dudleyf: that would be very easy

21:02 Chouser: hm, not having ever used an OODBMS I thought maybe they were the Solution.

21:02 dudleyf: Chouser: CouchDB?

21:02 rhickey: (doc resultset-seq)

21:03 JamesIry: dudleyf: JDBC result sets are more or less the Java equivalent of seq of maps

21:03 Chouser: peeked at that too.

21:03 Ycros: activerecord is the worst part of rails (I'm also a rails developer)

21:03 dudleyf: Thanks

21:03 rhickey: The problem with OODBMS is that information is not objects

21:03 JamesIry: For a completely alternative way to interface to a SQL DB, see also HaskellDB. Probably not appropriate for a dynamically typed language, though.

21:04 rhickey: it can be sliced an indefinite number of ways

21:04 Chouser: tuplespace

21:04 rhickey: but an OODBMS bakes an applications view of the data in to the data

21:04 Chouser: I've been flirting with RDF too.

21:05 see, this is part of why I picked rails -- I wasn't confident enough in any particular alternative. Oh well. For now I just have to live with it.

21:05 ...learn as much as I can about what I would really want.

21:07 Ycros: merb is actually a lot better

21:08 it has cleaner code, less magic, runs faster, uses less memory

21:08 and it doesn't tie you down to activerecord

21:08 of course, you do lose the ability to simply drop in a lot of rails plugins (some of the more generic ones still work)

21:09 dudleyf: rhickey: Do you have large projects in Clojure?

21:09 rhickey: Clojure is a large project

21:09 Not yet, starting to use it in my work

21:09 dudleyf: Well, right :-)

21:14 Are namespaces enough to keep the various modules of code separated from each other in a large project?

21:14 rhickey: works for JAva

21:15 dudleyf: But Java has classes as well as packages

21:16 rhickey: and lots of the functions in those classes are getName/setName etc, not even needed with maps

21:17 namespace names can be multi-component as well, so you could go one level further to mimic class scope

21:19 blackdog: rhickey, how does that work? multi-component?

21:20 rhickey: foo-bar-baz/fred, or foo.bar.baz/fred

21:20 just convention

21:20 blackdog: ok

Logging service provided by n01se.net