#clojure log - Jul 16 2008

The Joy of Clojure
Main Clojure site
Google Group
List of all logged dates

9:06 rhickey: Anyone XMPP savvy?

9:10 * drewr has familiarity

9:39 Chouser: you might need an xml parser that doesn't strip whitespace

9:39 rhickey: Chouser: ?

9:42 Chouser: xml.clj trims whitespace around text regardless of whether it's meaningful or not.

9:45 rhickey: Chouser: what's an example?

9:50 Chouser: (xml/parse (org.xml.sax.InputSource. (java.io.StringReader. "<a><b>1</b> <c>2</c></a>")))

9:50 That loses the space between 1 and 2

9:51 emit then inserts newlines where they don't belong.

9:52 There may be many XML formats where that doesn't matter, but there certainly are some where that'll break things, xhtml being one of them.

9:52 and xhtml can be used in xmpp

9:53 rhickey: would you want a flag toggling the ws dumping?

9:53 because otherwise in those other format you get a lot of useless child elements

9:53 Chouser: that's what I've got in my (still unreleased) lazy-xml lib

9:54 rhickey: emit is a toss off, admittedly

9:54 Chouser: but I think a "better" solution is to use a mechanism for interacting with the parsed tree that makes it easy to ignore whitespace you don't care about

9:55 same for xml comments. It's nice to leave those things in there if your going to modify something a little and write it back out.

9:55 yeah, I added a :pad option to my emit in case you want the newlines.

9:56 rhickey: in any case, I don't expect to be using xml.clj for XMPP, as it's streamed

9:57 Chouser: hm, good point.

9:57 rhickey: might try the smack lib first: http://www.igniterealtime.org/projects/smack/index.jsp

9:57 I'm thinking about xmpp for distributed Clojure

9:58 Chouser: oh, of course you don't want to deal with the xml directly. what was I thinking. :-)

9:59 well, for what it's worth we actually used xmpp for inter-app communication at work.

9:59 we got it working, but ended up ditching it in favor of our own custom XML protocol over TCP.

10:00 xmpp had a lot of features we didn't care about, but that the servers tried hard to support. We ended up with hard-to-debug events where the server appeared to be at fault, but it was hard to be sure.

10:01 We were happy to remove our depenence on a Jabber server.

10:05 But who knows what we may have been doing wrong. YMMV

10:10 meredydd: Is there an Official Answer to what (alter) returns?

10:11 Right now, it returns the new value

10:12 as does (ref-set) and (commute) - but the docs don't say anything official, and I'm leery of building on undocumented functionality :)

10:15 Chouser: yeah...that was my (admittedly arms-length) impression of using XMPP for IPC.

10:15 rhickey: Chouser: thanks for the info on your experience, that's why I asked, but I don't want to build from nothing. Other option is JXTA, but far fewer usingthat

10:15 meredydd: That said, I've just spent a happy week rolling my own IPC system for this product in Clojure, supporting exactly what I want and nothing more.

10:16 Chouser: it's a pity because it seems like it'd be nice: xml, highly available servers, etc.

10:17 we also considered UPnP, but that never got past the design stage

10:18 rhickey: I really like extending the chat metaphor to applications, including group messages, pub/sub, presence etc

10:19 Chouser: yep. I think we used direct messaging and pub/sub. Not sure if we used presence or not, and I don't think we used group messaging.

10:20 rhickey: as you said, xmpp has more than is needed, but rebuilding 1/3 of xmpp is still a big job

10:20 Chouser: I didn't implement any of this, but I seem to remember pub/sub being poorly supported or somehow causing us trouble.

10:20 I wonder if you built your own server (or at least had more complete control or command of it) if that would make the difference.

10:21 rhickey: I like the availability of openfire and smack from the same source, both Java. I would hope for good client/server compatibility there

10:21 Chouser: We're C++ shop using a Java-based server -- it a mysterious black box to us.

10:22 rhickey: meredydd: yes, they return the new value

10:22 meredydd: rhickey: cool, thanks. Could I request a doc-patch to make that explicit?

10:29 rhickey: meredydd: docs fixed but svn won't let me check in - aargh

10:29 meredydd: rhickey: Eh-oh. Server problem, or SVN internal error?

10:30 Chouser: hm, when SVN at sourceforge was failing the other day, I couldn't even do a fetch, but it worked for me just now.

10:31 meredydd: Chouser: Where's work, by the way?

10:31 Chouser: http://www.fourddev.com/

10:32 website's pretty sparse and out of date, but there ya go. ;-)

10:34 rhickey: Chouser: 403 Forbidden on checkin

10:35 Chouser: rhickey: :-( sorry.

10:39 rhickey: Chouser: what jabber server(s) were you using?

10:45 Chouser: I belive we used wildfire

10:48 oh, didn't notice you were gone. I believe we used wildfire

10:49 rhickey: Chouser: and for a client lib?

10:50 Chouser: http://delta.affinix.com/iris/

10:52 rhickey: Chouser: in spite of your experience I think I'll continue to explore xmpp - how long ago did you try it?

10:52 * Chouser pokes through the revision history.

10:56 Chouser: looks like we gave up on jabber in January of 2007

10:56 I hope it works better for you. :-)

10:56 rhickey: me too!

11:00 Chouser: it looks like we used "rosters" and "presence" earlier, and switched to "pubsub" and "status" later. I don't really know what those mean.

11:01 I seem to recall the main issue being flakey presence. In our application, we really couldn't have users "disappearing" if they weren't actually disconnected, but we chased just such problems a lot.

11:14 cemerick: open Q: since clojure has its own very expressive structured data format (sexprs/vectors/maps), why use an XML wire protocol? Or would XMPP only be an envelope for readable clojure?

11:15 rhickey: cemerick: the latter, message bodies would be readable Clojure

11:17 Chouser: rhickey: looked at lazy-seq closing at all yet?

11:27 rhickey: Chouser: I planted it in my head before ECOOP, but it really poses profound philosophical problems...

11:27 close being inherently not functional

11:27 Chouser: hm, yep, good point.

11:28 I don't have a great need for it, but you mentioned you might look at it after ECOOP, so I thought I'd ask.

11:28 rhickey: I have on my agenda making lazy-cons a special op, because it is so heavily used

11:28 doing so will allow me to make it one alloc instead of three

11:29 Chouser: ok

11:29 rhickey: and at that time I might extend the interface to include close, but haven't worked out sound semantics

11:30 e.g. what if you've aliased a closeable seq and one consumer closes it before the other is done?

11:30 Chouser: yeah, it really messes with the value semantics of seqs, doesn't it.

11:30 rhickey: right

11:34 potentially at least, but I'm still thinking about it. In some ways, someone calling close simply makes as much of the seq as anyone has read its effective length. As long as everyone sees the same elements it might not matter. But the threading aspects are tricky too

11:38 Chouser: hm. And the use case is so that func x can return a line-seq to func y which can then close any underlying file that might or might not be there?

11:39 rhickey: Chouser: that's one, but the general problem is one of non-memory resources associated with a seq source - files, threads etc

11:39 need to be released when no one cares about the seq anymore

11:41 Chouser: right, but specifically where the resource is created by the seq-creator, not passed in?

11:41 or do I still have that wrong?

11:42 rhickey: doesn't really matter, as soon as you pass off the seq

11:42 to some code that doesn't know about the resource

11:42 Chouser: yeah, ok.

14:33 jgrant: does clojure support something like common lisps keyword args ?

14:34 rsynnott: you could probably use a hash and destructure in the function declaration

14:36 Chouser_: jgrant: no direct support, but it's not uncommon to say something like:

14:37 Chouser: (defn foo [a b & opts] (let [opthash (apply hash-map opts)] ...))

14:38 or with destructuring like rsynnott suggested:

14:38 jgrant: thx

14:39 Chouser: (defn foo [a b & opts] (let [{:keys [c d e]} (apply hash-map opts)] ...))

14:39 either of those could be called like: (foo 1 2 :d 5)

15:26 rapido: anyone seen http://will.thimbleby.net/misc/ ?

15:27 maps instead of lists as the primitive structure

15:27 (for code and data)

15:27 albino: I looked at it yesterday, but didn't glean much

15:27 rsynnott: "simplicity of the syntax and design of LISP... and a syntax similar to that of Smalltalk"

15:27 weird

15:28 rapido: it's interesting as an idea

15:28 and it is immutable to the core

15:29 it seems that more and more (research) languages and frameworks are leaning towards immutability

15:31 and we also won't need no stinking loops (www.nsl.com)

15:31 albino: what's the perceived benefit of a map instead of a list as the default structure?

15:32 rapido: unification of concepts?

15:32 although i think a multi-map is even more general

15:32 albino: huh, he's listing xml as one of the benefits

15:34 rapido: xml is not fundamental. but key/value pairs are

15:35 albino: "or example maps can easily represent lists but not vice versa." <- is that really true?

15:35 It isn't easy to go from (1 "a" 2 "b") to a map of 1->"a", 2->"b" ?

15:36 rapido: albino: not so. a list can encode a map. but a map *implementation* is more efficient for key/value lookup

15:36 we agree

15:36 in my language enchilada, everything is a list ... but ....

15:37 albino: rapido: I was gonna ask you what you were implementing your language in these days, but for some reason I couldn't remember the name

15:37 rapido: i flag every (sub) list if its sorted (ascending)

15:37 if its sorted, its considered a multi-map

15:38 albino: why call it multi-map instead of sorted-map ?

15:38 rapido: certain operations perform faster (sorting, lookup, etc) when a list is sorted

15:38 multi-map allows multiple keys

15:39 so (1 "a" 1 "b" 2 "c" 3 "d" ) is allowed

15:39 albino: so non-unique keys then

15:39 rapido: yep

15:40 albino: so when you do a get on 1 do you get ("a" "b") ?

15:40 rapido: yep

15:41 it's very regular

15:41 in enchilada, every 'key' and 'value' is an expression

15:41 albino: there needs to be a #exotic-langs or something where all these cool language creators hang out and piss each other off

15:41 rapido: lol

16:13 ericthorsen: rhickey: question on the debug extensions namespace and pathing... given a clojure namespace 'org.me.package in source file stuff.clj, would the resulting file path be org/me/package/stuff.clj ... assuming you moved to the java package/path style for namepaces?

16:16 rhickey_: ericthorsen: I was just thinking about that. If the ns is org.me.package then the file should be package.clj in org/me

16:17 ericthorsen: rhickey: so the compiler will ignore the physical file name (if it does not match)?

16:17 rhickey_: but then a fn foo defined therein will be of class org.me.package.foo, so there's a mismatch with Java's class->file relationship

16:19 ericthorsen: rhickey: but there is no file package in that case. It would be looking for org/me/package/foo.class yes?

16:19 rhickey_: ericthorsen: there are 2 aspects - the source and source path in the debug info and the classnames of the generated fns

16:19 ericthorsen: understood

16:20 rhickey_: so the source and source path would always be based on the file/dirs

16:22 so maybe it would be as you said org/my/namespace/whatever.clj

16:23 so saying org.my.namespace.foo was in whatever.clj would match up classpath-wise

16:23 hmm, but not good for the folks doing the require stuff...

16:26 although leaving the namespace as a dir would allow for multiple files defining a ns, maybe some load order issues

16:33 any lib/require authors/users want to chime in?

18:07 meredydd: Argh. Another compiler bug.

18:10 Ooh, wow. I can only assume this to be a reader bug.

18:10 (It really doesn't help that the reader exceptions give you no hints as to what file it's reading when it encounters its errors)

18:12 Chouser: meredydd: but maybe you can give us a hint? :-)

18:13 meredydd: Ha. Nope, not a reader bug, just a spectacularly unhelpful error message.

18:13 Which reminds me - the next time rhickey can bear to see my face without running away, I want to ask him about the general state of backtraces in Clojure

18:17 Chouser: I have found the stack trace dumps to be rather opaque, but with a bit of practice I'm getting better at picking out the useful info from among the noise.

18:18 Lau_of_DK: It takes a little practice - but I dont see why Clojure it self could provide that useful information ?

18:18 meredydd: Chouser: I agree, and the same thing's happening to me.

18:18 Lau_of_DK: Find a bug in Clojure takes 50x the time it would do in Vs2005

18:18 meredydd: But so much of the time, the info Just Isn't There

18:18 Chouser: clojure's not generating the stack trace, the JVM is.

18:18 meredydd: or, worse, it's in about four levels of "Caused By:"

18:18 which is monstrously unhelpful, because you have to do the picking-through-the-noise bit four times

18:18 Lau_of_DK: Chouser: ,then forget the stack trace, there's probably a better way to go about it

18:19 Chouser: meredydd: well that's not really worse is it. having the information buried is better than not having it.

18:19 meredydd: True. Invert that "so much of the time" and "worse" :P

18:19 Chouser: ;-) sorry to pedantic

18:20 meredydd: ...well, given the context, that missing verb is too tempting to pass up :^D

18:20 Chouser: I wonder if one could write an out try/catch block that then picked through the stack information for you to highlight the most useful bits

18:20 s/out/outer/

18:20 meredydd: Not really helpful if you do run-time introspection of exceptions

18:20 Chouser: it's not? why not?

18:21 meredydd: Oh, just that you'd need to do your try/catch magic for *every* try/catch expression

18:22 Chouser: you can't pick through the whole trace from an outer catch?

18:22 meredydd: I suppose my stack-trace-munging code could have a good go at it...

18:22 Chouser: Most of the time, it doesn't make it to an outer catch

18:22 Chouser: oh, in your code base that may be. :-)

18:23 meredydd: But, as I say, I funnel most of it through a single piece of code that pretties it up to send somewhere else, so I guess I could do it there

18:23 Chouser: well the rest of us want to use it too, such as at a repl

18:23 meredydd: uh-huh. Probably put it together as a library function of some sort

18:24 give it an exception, and it'll give you a "clojure native" backtrace.

18:24 Chouser: yeah, something like that

18:24 meredydd: To a first approximation, I guess you could just "grep -v clojure.lang"

18:24 Chouser: yep. i tend to look first for .clj files

18:25 you'd also want to try to detect "compile time" exceptions an display them differently

18:25 meredydd: Ah...again, my own biases - this is an app with large hunks of Java code, so it's likely the call stack will be weaving in and out between the two languages

18:26 Chouser: hm. I suspect this would be the problem in general -- what each of us considers "important" varies by just enough.

18:26 meredydd: Chouser: Hmm...aren't they all subclasses of CompilerException?

18:26 Chouser: for example, I would imagine Rich generally wants to soo the clojure.lang items. :-)

18:27 meredydd: nope, not when macros are involved

18:27 meredydd: Oh, joy.

18:27 Chouser: (defmacro foo [] `(nil))

18:27 (foo)

18:27 meredydd: I'd imagine Rich (or anyone else hacking core) to be a very marginal case, though.

18:28 Chouser: java.lang.NullPointerException, but it's at compile time so the stack trace doesn't really go through any .clj code

18:28 meredydd: For comparison, it's not as though ordinary Java backtraces go through interesting bits of the JVM just in case there's a core dev watching :P

18:28 Chouser: hmm

18:29 meredydd: Chouser: Hmm...you could probably detect that case.

18:29 "if there's no Clojure code in here, treat it like an ordinary Java backtrace"

18:32 Chouser: actually, you may be right. it says NullPointerException at the top, but it may actually be a CompilerException. I'm not quite sure.

18:33 meredydd: ...and when even you're still not quite sure, that's a strong indication that the error message system is, ahem, insufficiently clear :)

18:40 In any case, if you're interested, the compiler bug has just vanished into the ether.

18:40 Maybe might have been a downstream product of the reader bug, which at a certain point was claiming to successfully read an ill-formed file, and perhaps there was some data-structure brokenness that propagated to the compiler

18:41 Unfortunately, I didn't think to save it, so I can't reproduce :S

18:42 Chouser: ah, well. you didn't happen to be doing any regex stuff, were you?

18:43 meredydd: no, why?

18:44 Chouser: something like #"\w+" will kill your reader quick

18:45 ...although my test of it just now had more robust results than when I've tried in the past.

21:31 koneill: morning all ... it's there a log for this c

21:31 hannel ? ... sorry about the line break

21:32 arbscht: http://clojure-log.n01se.net/date/

21:32 koneill: thanks

Logging service provided by n01se.net