9:26 cgrand: Did you know that the function passed to pmap must not return nil?
9:36 Chouser: I did not.
9:37 cgrand: It seems to be an unintended consequence of using a LinkedBlockingQueue.
9:38 rhickey: any reason not to try parallel.clj?
9:39 cgrand: apart from being too lazy to install one jar? No :-)
9:40 rhickey: pmap may go away
9:43 cgrand: rhickey: oh, ok, I'll look at parallel.clj then.
9:44 blackdog: coming from the non lisp world i find the docs sometimes difficult to understand, e.g. -> macro, i just worked out is a kind of "member access" for a struct
9:44 it could be a good idea to note things like that in the docs
9:45 rhickey: except it's not, it's a more general "through" or "thread"
9:45 blackdog: yes, that's why i said "kind of"
9:45 but for the beginner
9:45 that's maybe the predominant use case?
9:46 i'm not sure, that's what I'm about to use it for :)
9:46 rhickey: I'm not saying there couldn't be more explanation, right now the docs are a reference, so a bit terse
9:46 but misaligned analogies can confuse more than enlighten
9:47 Chouser: You never *need* the -> macro, it just allows you to arrange your code a bit differently.
9:47 rhickey: examples would help, something I hoped would happen on the Wiki
9:47 blackdog: well, i'd like to do a simple member access type thing to a map and I don't want to use (get) all the time
9:47 is there another way?
9:48 rhickey: you hardly ever have to use get given maps are functions of their keys
9:48 blackdog: oh, this is something I've not understood then
9:48 rhickey: and keywords are functions of maps, if you're using them as keys
9:49 blackdog: ah ok, there are the missing links, thanks!
9:49 rhickey: user=> ({:a 1 :b 2} :b)
9:49 2
9:49 user=> (:b {:a 1 :b 2})
9:49 2
9:50 blackdog: or maybe I'm the missing link ;) cheers
14:52 rapido: erlang has proven that copy semantics make concurrency easy
14:53 immutable data structures have efficient copy semantics
14:53 but how do immutable data structures scale in a distributed setup?
14:54 rhickey: erlang unifies the local and distributed model
14:54 clojure doesn't
14:55 some reasons here: http://
14:55 rapido: but how do you efficiently share huge data structures along different distributed nodes?
14:55 rhickey: erlang doesn't efficiently share huge data structures over the wire
14:55 rapido: erlang doesn't solve that
14:55 i believe there is a way to do that
14:56 using distributed hash tables
14:56 rhickey: I think the distributed case should be handled separately. I don't believe in transparent distribution
14:56 sure, tuplespaces linda etc. Mnesia too
14:57 rapido: i believe DHT is build to handle churn better than tuplespaces or mnesie
14:57 mnesie <- mnesia
14:58 rhickey: DHT is sort of an implementation detail, no?
14:58 model is still associative
14:58 rapido: no, i think it is an ideal substrate. you can leverage any datastructure on top of a DHT
15:00 rhickey: but is hashing important in some non-implementation-detail way?
15:01 rapido: there are dht implementations that leverage near-keyness as opposed to (scattered) hashes.
15:02 you can build a persistent vector on top of a dht
15:03 just store the sha-1 hash at every tree node
15:03 the value is the serialization of the tree node. the key is its sha-1 hash
15:04 replace pointers to sub-trees with their sha-1 hashes
15:05 does that make sense?
15:06 rhickey: it could, but I am unsure about whether you really want distributed versions of the same data structures you use in-proc
15:06 other options are to put Clojure under Terracotta
15:07 rapido: but such data structures could scale to tera bytes, going beyond jvm's
15:07 rhickey: so many other issues dominate - scalability, fault tolerance etc
15:08 not uninteresting, but a large problem space
15:09 rapido: i've had my fare share of faulty disks, nodes etc.
15:10 that's why i think immutability is so important
15:10 rhickey: no argument here :)
15:11 rapido: i generally don't like the 'encapsulation' argument
15:11 i don't want an object to hide that it is connecting to sybase, opening files, etc
15:12 because when such object throws an exception
15:12 it better close the sybase connection, file handle, etc
15:13 rhickey: I make that point in most of my talks, encapsulation just means "I'm in charge of this mess", not that there isn't a mess
15:13 not a substitute for immutability by any stretch
15:14 rapido: everyone is re-implementing the same hard stuff <- resource management.
15:15 try , catch .... oh, i forgot finally :)
15:17 you'd better don't catch c++ exception. it is almost certain that you've caught a memory leak at the same time
15:17 rhickey: programming is not a solved problem yet
15:19 rapido: not everybody agrees, but i think it was/is a good choice that clojure is build on top of the jvm
15:20 rhickey: I have no regrets
15:20 as a new language, Clojure has a lot of utility
15:20 excellent performance
15:20 great portability
15:21 rapido: its the magic of hotspot - not everybody appreciates the 1000 man-years that went into it
15:21 rhickey: exactly
15:22 rapido: hotspot goes back to self technology
15:22 the language self i mean
15:24 may be shez scheme is the exception
15:24 rhickey: I like the separation of concerns, I can focus on my language and not on code generation or socket libraries
15:25 rapido: schez scheme achieves amazing compiler feats
15:25 but it didn't took 1000 man-years to achieve those feats
15:26 rhickey: yes, but hotspot's dynamism sets it apart
15:26 rapido: agreed
15:27 i've implemented my own language (enchilada)
15:27 first try: java
15:27 second try: java
15:27 third: factor
15:27 fourth: factor :)
15:27 five to seven: haskell
15:28 and finally: scala
15:28 i got addicted to a typed language after the haskell experience :)
15:29 but i finally went for the jvm, exactly because i don't like to bother with native socket implementations
15:29 myself
15:31 i didn't consider clojure yet - sorry about that ;)
15:33 rhickey: that's ok, I'll likely not get around to reimplementing Clojure in Clojure either :)
15:34 rapido: aaahhhh, the art of bootstrapping
15:34 factor has done it
15:35 factor started on the jvm
15:36 eventually, the factor compiler was written in factor
15:36 generating native code
15:36 for various platforms
15:37 (still using the jvm implementation of factor i believe)
15:37 i think that's pretty cool
15:37 (but i could be wrong there)
15:38 hmm, according to the definition, that's not strict bootstrapping
15:39 ... bootstrapping is confusing ...
15:40 rhickey: what programming languages do you like? (apart from clojure of course)
15:41 rhickey: Common Lisp
15:42 I like the ideas in Haskell
15:42 but am concerned about type system expressivity
15:43 rapido: yeah, types can hinder
15:44 i like apl/j/k because it so different and still effective
15:44 it so <- it is so
15:44 Chouser: that's really key. It wasn't 'til I was trying to do an XML filter DSL in Scala that I realized how much static types can *really* get in your way.
15:44 rhickey: I disagree with types in many ways. They represent an extreme narrowing of our ability to categorize
15:44 Lau_of_DK: Syntax question: This simple decoder should be able to take anything from 1 integer, to an infinate number of integers, it works with either 1 or 2... whats wrong?
15:45 (defn decode
15:45 ([x] (char (+ 97 (- (/ x 2) 1))))
15:45 ([x &more] (apply str (list (decode x) (decode &more)))))
15:45 rhickey: I think the type theorists wis
15:45 I think the type theorists wish programming ws
15:45 was math, but it isn't
15:46 rapido: did you know that there are infinitely many types of numbers?
15:46 they are surreal numbers
15:46 (just teasing there)
15:46 la_mer: Speaking of distributed computing (look at me, creating a tangent of a conversation from a half hour ago!), has anyone come across any reasonable overview of all of the different distributed computing frameworks available for Java apps (hadoop, terracotta, gigaspaces, coherence, etc., etc., etc.)?
15:47 Lau_of_DK: rhickey, you wrote the language, can you see why my overload isnt working?
15:47 rhickey: & more
15:48 not &more
15:48 rapido: la_mer: check out: freepastry, overlayweaver, bamboo, etc
15:50 Lau_of_DK: rhickey, okay, if I change the param to "& more" and in the function I change it to "more" then it compiles, but it throws an error when I run it with more than 1 arg - Is this because I need to add some splicing operator?
15:51 rhickey: you don't need the list call
15:51 Chouser: (defn decode ([x & more] (str (char (+ 97 (- (/ x 2) 1))) (when more (apply decode more)))))
15:52 la_mer: rapido: Thanks for the names of more frameworks to look at. Do you know of any writeups comparing/contrasting them, their approaches, where they fit in the stack?
15:53 Lau_of_DK: Chouser, rhickey, thanks guys :) You really fill the gap between here and descent tutorials :)
15:53 rapido: la_mer: this page has many links: http://
15:54 i found it very valuable
15:54 la_mer: rapido: Thanks for that. p2p is a whole area unto itself, I suppose.
15:55 rapido: la_mer: my gut feeling is that p2p *is* distributed computing
15:55 la_mer: I saw on google groups that someone has put together a clojure wrapper for hadoop, so I suppose we'll start there
15:56 rhickey: people have played with NetKernel and Clojure too
15:56 I'm interested in trying Shoal (from Glassfish)
15:57 la_mer: rapido: p2p means no central control (to me, anyway) -- we'll be looking to deploy to a grid architecture.
15:58 rapido: hey! i didn't know about shoal! thanks for the link rhickey!
15:58 rhickey: the Shoal API is so simple, it's irresistible
15:58 rapido: la_mer: the world is made that way: there is no central control ;)
15:59 ahh, jtca
15:59 jtxa
16:00 rhickey: how did you find shoal?
16:01 rhickey: interested in trying, haven't tried
16:01 good enough for Glassfish clustering, apparently
16:01 rapido: la_mer: what's your stake on clojure and grid architectures? (who's *we* ?)
16:03 la_mer: we is http://
16:03 we do a lot that isn't publicly available / marketed, FWIW
16:04 rapido: to bad i don't live near western massachusetts
16:04 too bad
16:05 la_mer: We are developing a new service / product this summer, and I'm considering using clojure in part. The real meat of the functionality will be the processing of documents in varied and interesting ways, all of which are computationally (and storage) intensive. Not targeting a grid architecture would be foolish at this point.
16:05 rapido: Where are you now?
16:05 rapido: la_mer: the netherlands .. amersfoort (google that!)
16:07 la_mer: rapido: Looks very pleasant :-)
16:08 rapido: la_mer: you would consider using clojure in a production setting?
16:08 la_mer: rapido: Very close to it, yes.
16:08 rapido: that means you have power
16:09 la_mer: rapido: lol :-) In what way?
16:09 rapido: or you are good in convincing 'management' to try something new to beat the 'competition'
16:10 la_mer: I *am* management. :-D
16:10 rapido: la_mer: i rest my case...
16:10 la_mer: We're also a small company trying to solve tough problems, and we always need to be using the best raw materials we can lay our hands on.
16:11 rapido: la_mer: well, for me it is certain that you don't represent classical management
16:12 la_mer: No, I suppose not. Of course, it remains to be seen whether that's a good thing or not. ;-)
16:15 rapido: la_mer: managing people (=developers). that's difficult
16:15 herding cats
16:16 of course, managing non-developers is the real task
16:17 and managing your own company is the most interesting 'challange'
16:20 la_mer: It's definitely a challenge. And, FWIW, I actually think that sales and "business development" is *way* harder than software development. People are hard, machines are easy.
16:25 rapido: got to go..... i'm mutating to a different location ....
16:25 la_mer: rapido: good talking with you
17:11 Lau_of_DK: For string comparison, does Clj have something like if ("x" isin "extreme") ...true ?
17:12 rhickey: Clojure strings are Java Strings
17:13 Lau_of_DK: So I gotta go with (. String compare) or something like that?
17:14 rhickey: one of us will have to look it up, and it's not going to be me :)
17:15 dudleyf: (. "foobar" contains "foo")
17:15 Lau_of_DK: haha
17:15 rhickey, no I wasnt asking for the specific function, I would just prefer not to call anything (. java) if there already was something in lisp that did the same :)
17:15 Thanks dudleyf
17:16 Chouser: or (.contains "foobar" "foo") -- how much simpler could a lisp version be?
17:16 rhickey: I'll probably not be adding (contains x y) in order to avoid (.contains x y)
17:17 Lau_of_DK: Come one guys :)
17:17 rhickey: All of the Java stuff is nicely documented, were I to wrap it re-documenting it would fall on me - no point
17:17 Lau_of_DK: I dont know - Im just prone to picking stuff that arent Java, as far down the road as I can
17:17 rhickey: that's not going to let you leverage Clojure fully
17:17 Lau_of_DK: But I get the point, so forget the stupid question :)
17:18 Chouser: not a stupid question! I definitely went through an "ew, I have to touch Java?" phase.
17:19 But it's pointless. You're on the JVM, you have the Java libs -- use them!
17:19 Lau_of_DK: Youre right
17:19 And for IO and GUI Im all over them :)
17:19 Chouser: and string. ;-)
17:19 rhickey: everyone goes through that phase, I think
17:20 Lau_of_DK: Thanks guys, thats good to know
17:20 Chouser: regex is a bit of an exception, I think, because the Java API is so... wordy.
17:20 Lau_of_DK: Last night I redid some of the Eulers that I had already handled in SBCL or Python, I was pretty amazed at the speed of Clojure
17:20 rhickey: ew, Java libs --> ooh, Java libs
17:20 Lau_of_DK: hehe
17:20 The libs are nice, and huge
17:20 which is something I missed very badly in SBCL
17:21 dudleyf: I think a lot of Java API's could use some Clojure lipstick
17:21 rhickey: Chouser: regex still too wordy?
17:22 dudleyf: fair enough, but driven by need and added value
17:22 dudleyf: rhickey: Oh, absolutely.
17:22 rhickey: Clojure's wrapping of fork/Join is pretty sweet vs. the Java
17:23 IMO
17:24 .., doto etc a ripe area as well
17:25 dudleyf: I've always hated the Java IO API's
17:25 rhickey: not sure how to fix them...
17:26 dudleyf: No, me neither. It's kind of baked into their design.
17:26 Chouser: duckstream helps the IO libs a bit
17:26 Lau_of_DK: Whats to hate about them ?
17:26 rhickey: layers
17:26 Lau_of_DK: They seem pretty straight forward
17:26 rhickey: layers
17:26 Chouser: rhickey: no, re-seq and re-find are good. I wrote a re-split that I like too.
17:26 rhickey: more layers
17:26 dudleyf: I really don't like the decorator approach
17:26 Lau_of_DK: If the performance is descent, why worry about layers?
17:27 rhickey: wordy
17:27 Chouser: re-split does what?
17:27 Lau_of_DK: btw rhickey, wasnt it tonight you were giving that talk to the java group ?
17:28 rhickey: last Thursday, still haven't looked at the 3 hours of material
17:28 Lau_of_DK: Send them to me, I'll run through them in a jiffy
17:31 rhickey: 2gb
17:31 probably needs editing :)
17:32 Chouser: user=> (re-split #"\\W" "different/separators between:these")
17:32 ("different" "/" "separators" " " "between" ":" "these")
17:32 Lau_of_DK: Ok. How did they take it? Were you pitching Lisp in itself?
17:37 baggico: man. i blab so much about lisp to my colleagues at work...
17:37 well... when something sets me off anyway :)
17:38 people are generally either: "huh?" or receptive.
17:41 rhickey: Chouser: re-split wraps String.split?
17:42 or lazy?
17:42 Chouser: lazy
17:42 The implementation looks a lot like re-seq. ;-)
17:43 I generally use it in combination with (partition) to do search/replace in strings.
17:44 (for [[text url] (partition 2 (re-split #"..." text))] ...)
17:45 that's what converts plaintext urls in the irc log into <a> links
17:47 Lau_of_DK: this I dont get
17:47 (import '(java.lang.Math))
17:47 (. abs -5)
17:47 Error: Abs not found in this context
17:47 but it works fine in Java
17:47 Chouser: (import '(java.lang.Math abs))
17:48 Lau_of_DK: gives the same error
17:48 rhickey: all of java.lang is already imported
17:48 Chouser: oh, sorry.
17:48 (. Math abs 5)
17:48 rhickey: there you go
17:51 Chouser: Lau_of_DK: get it?
17:57 Lau_of_DK: Yes sir, thanks
17:58 Chouser: very good. :-)