#clojure log - Nov 10 2009

The Joy of Clojure
Main Clojure site
Google Group
List of all logged dates

0:00 _ato: hmm

0:00 ,(split-with #{" "} "foo bar")

0:00 clojurebot: [() (\f \o \o \space \b \a \r)]

0:00 _ato: ,(map str (split-with #{" "} "foo bar"))

0:00 clojurebot: ("clojure.lang.LazySeq@0" "clojure.lang.LazySeq@3f32ee98")

0:01 _ato: ,(map #(apply str) (split-with #{" "} "foo bar"))

0:01 clojurebot: java.lang.IllegalArgumentException: Wrong number of args passed to: sandbox$eval--5184$fn

0:01 hiredman:

0:01 _ato: ,(map #(apply str %) (split-with #{" "} "foo bar"))

0:01 clojurebot: ("" "foo bar")

0:01 _ato: nevermind

0:01 qed: there are also the str-utils clojure-contrib libs

0:01 which have a re-split in them

0:01 hiredman: ,(doc split-with)

0:01 clojurebot: "([pred coll]); Returns a vector of [(take-while pred coll) (drop-while pred coll)]"

0:01 _ato: h right

0:02 I was thinking of partition-by

0:02 which wouldn't have worked anyway

0:04 sudoer: what does the . mean here? (. parsedLogFlow start)

0:05 hiredman: . is the java interop special form

0:05 _ato: same as: (.start parsedLogFlow)

0:05 hiredman: or (parsedLogFlow/start)

0:05 _ato: call the .start() java emthod on paresedLogFlow object

0:05 hiredman: or possibly parsedLogFlow/start

0:06 _ato: could be field access

0:06 _ato: ah yeah true

0:07 sudoer: I see, what is the point of it then?

0:07 oh, so I can do oop with java?

0:07 hiredman: no

0:07 it is for java interop

0:08 (don't do oop)

0:08 wavis: does assoc return a sorted map when given a sorted map?

0:08 * somnium wonders what java oop in clojure would look like

0:09 _ato: wavis: yes

0:09 wavis: cool. if it uses a custom comparator, that's also preserved, i presume?

0:09 _ato: yep

0:09 wavis: gracias

0:10 sudoer: what does it mean when you say java interop?

0:11 _ato: sudoer: interoperating with code written in java. for example using a java library in a clojure program

0:11 hiredman: ,(Math/sqrt 25)

0:11 clojurebot: 5.0

0:12 hiredman: ,(macroexpand '(Math/sqrt 25))

0:12 clojurebot: (. Math sqrt 25)

0:12 wavis: ,(.exists (File. "/etc"))

0:12 clojurebot: java.lang.IllegalArgumentException: Unable to resolve classname: File

0:12 wavis: ,(.exists (java.io.File. "/etc"))

0:12 clojurebot: java.security.AccessControlException: access denied (java.io.FilePermission /etc read)

0:13 sudoer: ok, I understand

0:14 am I supposed wrap those calls with dorun? I see java interop cored wrapped in dorun, I read the dorun definition, but I am not sure why I would use it

0:14 hiredman: uh

0:14 sudoer: sorry, im a clojure newb, I assume my question made no sense?

0:14 hiredman: right

0:15 have you read the doc for dorun?

0:15 ,(doc dorun)

0:15 clojurebot: "([coll] [n coll]); When lazy sequences are produced via functions that have side effects, any effects other than those needed to produce the first element in the seq do not occur until the seq is consumed. dorun can be used to force any effects. Walks through the successive nexts of the seq, does not retain the head and returns nil."

0:15 sudoer: I am tryint to understand what this does: (dorun (. parsedLogFlow start) (. parsedLogFlow complete))

0:15 trying to understand why dorun is used there

0:15 hiredman: sudoer: maybe a typo

0:15 I would guess they meant to use do

0:16 sudoer: the code works

0:16 hiredman: sure

0:16 but that could just be an acident

0:17 it also looks like either old code, or code by someone not so plugged into the clojue community

0:17 sudoer: its a clojure newbie

0:17 hiredman: because of the direct use of the . form

0:17 somnium: the source for do run is shorter than its doc string

0:17 hiredman: :P

0:18 qed: how do i keep track of the number of times ive called a function, inside of a function

0:18 hiredman:

0:18 seriously?

0:18 qed: is that a stupid question?

0:19 hiredman: I guess it depends, but it kind of seems like it is

0:19 qed: i mean, here's the deal

0:19 i have this function: (defn step [n] (if (even? n) (quot n 2) (+ (* 3 n) 1)))

0:20 now i have another function that continues to call step until we can't go any lower

0:20 err until we hit 0

0:20 hiredman: I know this problem

0:21 qed: i want to find out which number i input <1000000, that has the most number of steps

0:21 hiredman: iterate+filter+sort+count

0:21 qed: so im trying to determine how many steps something takes

0:21 hiredman: not filter. take-whil

0:21 qed: like I said, I know this problem

0:21 qed: hey i believe you

0:21 im just explaining my reasonining

0:21 reasoning

0:22 i dont really understand how to write what you're describing

0:23 how does sort+count work?

0:23 in this context

0:24 somnium: let me check my euler dir

0:24 qed: somnium: thanks, not trying to cheat if i can avoid it

0:24 that's boring

0:24 hiredman: you sort the results of interate+take-while

0:25 which will give you longest chain

0:25 and the first thing in the chain will be the number

0:25 odd, I can't find this in my euler.clj

0:26 somnium: its the collatz one

0:26 qed: oogatz

0:27 hiredman: clojurebot: hiredman's euler.clj?

0:27 clojurebot: euler is http://clojure-euler.wikispaces.com/

0:27 hiredman: bah

0:27 clojurebot: hiredman's euler.clj

0:27 clojurebot: hiredman's euler.clj is so embarassing

0:28 qed: i wont be going there heh

0:28 somnium: all my solutions were painfully slow :(

0:28 but there's an easy bi... nm

0:28 technomancy: _ato: yeah, I'm going to make leiningen use build.clojure.org as a default repo before I hit up the release.

0:28 sudoer: hiredman: so is the dorun useless in that situation?

0:28 technomancy: _ato: all the functionality is there, it just needs a bit more polish, particularly in the bash script

0:29 hiredman: sudoer: it's hard to say, if those java methods return lazy-seqs that you want to realize for side effects it would make sense, but not otherwise

0:29 and I doubt they do

0:29 qed: im not understanding tak-while

0:29 hiredman: I would guess they meant to use do

0:29 qed: (take-while (> 3) [1 2 3 4 5 6 7 8])

0:30 how would i do that?

0:30 hiredman: qed: (> 3) is not a function

0:30 so it would not

0:30 qed: (take-while (> [1 2 3 4 5 6 7 8] 3)) ?

0:30 hiredman: you need something like (partial > 3) or #(> 3 %) or (fn [x] (> 3 x))

0:30 qed: no

0:31 qed: ah yes this partial business

0:31 somnium: qed: (take-while #(> % 1) (iterate (fn [n] (if (even? n) (/ n 2) (inc (* n 3)))) 9))

0:32 oops, meant to send that one to the bot

0:32 that gets you a function that returns an expansion, try to take it from there

0:33 qed: thanks somnium

0:33 im off for the evening

0:33 thanks hiredman and all for the help

0:46 twbray: Wondering if it's evil to use send-off with a placeholder first arg just because I want a long-running operation in its own thread. More idiomatic to just create a new thread?

0:50 _mst: twbray: `future' provides an easy way of spawning a new thread

0:50 ,(doc future)

0:50 clojurebot: "([& body]); Takes a body of expressions and yields a future object that will invoke the body in another thread, and will cache the result and return it on all subsequent calls to deref/@. If the computation has not yet finished, calls to deref/@ will block."

0:50 twbray: _mst: Thanks, will check that out.

0:50 I guess I can just do (.start .... as well

0:51 wavis: ,(send-off (agent nil) #(Thread/sleep 9999))

0:51 clojurebot: #<Agent@1aacdc9: nil>

0:51 wavis: i was expecting to be locked out of that one

0:51 _mst: yep, (.start (Thread. #(...)))

0:52 _ato: ah neat, scp uses a really trivial protocol so I can just make it talk to my own code that verifies the uploaded files and if they're okay puts them into the right place in the repo :)

0:52 twbray: wavis: (send-off (agent nil) makes me uncomfortable....

0:52 _ato: http://blogs.sun.com/janp/entry/how_the_scp_protocol_works

0:53 and that dispatch on ssh key thing that github does is easy too, you can just add a command=whatever to the end of each line in authorized_keys and it'll run it instead of the user's shell or scp or whatever :)

0:53 wavis: twbray: i suppose if you're starting with nil and want to immediately start initializing it, it could be practical. it does return the agent after all

0:54 _mst: _ato: yep, probably a good idea not to let them run whatever command they want ;)

0:54 shoover: twbray: I've seen several examples where an agent calls send-off on itself as a loop with a quit check. See the bottom section of http://clojure.googlegroups.com/web/ants.clj

0:55 twbray: shoover: thanks

1:00 hiredman: apache-mina has some kind of sshd in java

2:07 piccolino: Lazy sequences don't have to be infinite do they?

2:10 hiredman: nope

2:10 ,(map inc (range 10))

2:10 clojurebot: (1 2 3 4 5 6 7 8 9 10)

2:10 piccolino: I didn't think so.

2:10 But then I'm having a hard time understanding what exactly lazy-seq wants. How do you terminate the sequence?

2:11 hiredman: nil

2:11 piccolino: Oh, OK.

2:11 hiredman: ,(lazy-seq (cons 1 nil))

2:11 clojurebot: (1)

2:11 piccolino: I thought nil could be a part of a list, so I thought that would be alright.

2:11 hiredman: it's not really about what lazy-seq wants, but what cons wants

2:11 ,(lazy-seq (cons 1 '()))

2:11 clojurebot: (1)

2:12 hiredman: piccolino: nil can be part of a list

2:12 but the second form in a cons is not an item in a list

2:12 it is a list (or nil)

2:12 piccolino: Ah, OK. So you can't use lazy-seq with conj?

2:13 hiredman: never seen it done

2:16 piccolino: Man, this doc string still makes no sense to me. "yields a Seqable object that will invoke the body only the first time seq is called, and will cache the result and return it on all subsequence seq calls."

2:17 How do I make an indefinitely long sequence that will definitely end at some point?

2:17 I don't want the same expression called over and over, I need it work through some data.

2:29 hiredman: ,((fn a [n] (when (not (zero? n)) (lazy-seq (cons n (a (dec n)))))) 10)

2:29 clojurebot: (10 9 8 7 6 5 4 3 2 1)

2:30 hiredman: usually you can get by with the various sequence functions with out resorting to lazy-seq

2:30 piccolino: Yeah, I was looking at iterate, but it says the function must be side-effect free.

2:31 hiredman: *shurg*

2:31 piccolino: I apprecite it, though, that's helpful.

2:31 I find it confusing just what exactly lazy-seq returns and how it does its thing.

2:32 hiredman: lazy-seq returns a lazy seq

2:32 piccolino: Also, it requires recursion on a function?

2:32 hiredman: it's generally how it's done

2:33 piccolino: Yes, a lazy seq, but how exactly that contains my code, is what I mean.

2:34 hiredman: essentially you construct a list

2:34 piccolino: Like, in the cons in your lazy seq, if you call first on the lazy seq, clearly the cons has to be executed, but the second argument is deferred?

2:34 hiredman: the second are evals to a lazy-seq

2:34 arg

2:34 piccolino: Oh, duh.

2:35 I think I got it.

2:35 Less magical than I thought.

2:35 hiredman: :)

2:45 piccolino: Urg. cons forces me to build it in backwards order.

2:47 Oh nevermind.

2:48 Chousuke: lazy-seq is kind of cheating :)

2:48 "here, I'm consing to this sequence that doesn't exist yet"

2:55 piccolino: I have no idea how to explain it better, but I sure wasn't gonna understand that just from the docs.

3:01 wavis: If I create a function inside of a dosync, and the function references a ref, will the snapshot state of the ref be captured, or will, the function deref again later each time it is called?

3:01 tomoj: creating a function inside of a dosync sounds like a really bad idea

3:02 wavis: why?

3:02 tomoj: well, ok, not a _really_ bad idea

3:02 compiling the function should have state side effects, it shouldn't need to be in a dosync

3:03 try this

3:03 (let [foo (atom 5)] (defn bar [n] (swap! foo + n)))

3:03 then do (bar 3), (bar 4), whatever

3:04 you don't need a dosync you just need to close over the state

3:05 uh, also, s/should/shouldn't/

3:05 wavis: i hear you. the function doesn't have side effects though. i have to create *a* function inside, but I could create one that references the ref outside, and have the inside function call the outside function

3:06 tomoj: I dunno what you mean

3:07 wavis: so (let [a (ref nil) fa (fn [] @a)] (dosync (stuff) (fn [] (fa))))

3:09 fa gets the state of a

3:09 the function created in the dosync just calls fa

3:09 clojurebot: function is <Chouser> there is one class per fn, one instance of it per closure

3:09 tomoj: why do you need fa

3:09 I mean, why can't you just put (fn [] @a) inside the dosync

3:10 wavis: that's the question. will calling that anonymous function return to the state of the ref inside of the original dosync, or will it deref the new state of the ref

3:11 tomoj: I still think wrapping a dosync around the fn is... strange

3:12 wavis: i'm altering a ref and setting it to (proxy [stuff] (stuff [])) ... i guess i could create the proxy outside the dosync

3:14 tomoj: I think the answer to your question is that it will be deref'd again and again

3:14 creating the function doesn't store the value anywhere

3:14 ,(let [a (ref 3) fa (fn [] @a)] (dosync (alter a + 4) (fa)))

3:14 clojurebot: 7

3:15 wavis: ok i'm going with that. It's what I intuitively expect, but I wasn't sure

3:17 ,(let [a (ref 3) fa (fn [] @a) fb (dosync (alter a + 4) fa)] (dosync (alter a + 1)) (fb))

3:17 clojurebot: 8

3:18 wavis: er. that actually doesn't do anything different. anyway...

3:19 ,(let [a (ref 3) fa (fn [] @a) fb (dosync (alter a + 4) (fn [] @a))] (dosync (alter a + 1)) [(fb) (fa)])

3:19 clojurebot: [8 8]

3:19 wavis: there.

3:23 tomoj: yeah I don't understand the point of that at all

3:50 osaunders: Can you get a reference to the current function?

3:52 Chousuke: you can give it a local name as in (fn foo [] ... (foo ...))

3:52 osaunders: You can't not give it a name?

3:53 I mean, you can't do it without giving it a name?

3:53 Chousuke: that's correct

3:53 there's no "this"

3:53 but the name is local, so if you need it there's no harm in giving it one.

3:54 it'll also make stacktraces more readable

3:54 osaunders: OK.

3:55 You're only supposed to be able to def once right? If I def foo to 1 I shouldn't be able to def it to 3 again later. I should need set! to do that. Is the REPL an exception to that rule?

3:56 Chousuke: you can use def as many times as you want

3:56 you only shouldn't :)

3:56 osaunders: Right, OK.

3:56 Because I was thinking that would be funny with recursion.

3:56 Because if you have defs inside your function you'd re-def them each recursion.

3:57 Chousuke: yeah. and that would be horrible. :P

3:57 osaunders: If that wasn't allowed it might be a problem.

3:57 Yeah.

3:57 Chousuke: defs anywhere other than top level are a problem.

3:57 sudoer: can I have multiple "classes" in a clojure file?

3:57 Chousuke: sudoer: gen-classed ones? not as far as I know.

3:58 sudoer: can I define java classes in a clojure file?

3:58 Chousuke: well, every function is a java class, so yes :P

3:58 but gen-class is for constructing custom classes.

3:58 sudoer: Chousuke: the code I have uses gen class

3:58 osaunders: Chousuke: Thanks for the help.

3:59 Chousuke: but often it's not needed. You can just use proxy.

3:59 yason: osaunders: you can use (binding) to "change" defs within a certain dynamic scope.

3:59 sudoer: I am totally new to clojure, I am trying to port this to clojure http://code.google.com/p/cascading/wiki/CrawlDataWordCountCascade

3:59 osaunders: yason: binding?

3:59 Chousuke: and there are a couple new things upcoming that you will be able to use to create "classes" :)

3:59 osaunders: You mean the [foo 1, bar 2] stuff?

4:00 Wait, no.

4:00 I really don't know :-P

4:00 sudoer: so can I just write it as regular java and have that included in my program?

4:00 yason: osaunders: it's one of Clojure's special forms

4:00 Chousuke: sudoer: no, not like that :)

4:00 sudoer: hmm

4:01 Chousuke: sudoer: if you have java code, keep it in a java file :P

4:01 sudoer: i dont even know how to program java

4:01 Chousuke: but you don't need to program java for that example.

4:01 sudoer: this is going to be tough!

4:01 Chousuke: you can implement the PipeAssembly subclasses through proxy

4:02 sudoer: Chousuke: what do you mean proxy?

4:02 Chousuke: (doc proxy)

4:02 clojurebot: "([class-and-interfaces args & fs]); class-and-interfaces - a vector of class names args - a (possibly empty) vector of arguments to the superclass constructor. f => (name [params*] body) or (name ([params*] body) ([params+] body) ...) Expands to code which creates a instance of a proxy class that implements the named class/interface(s) by calling the supplied fns. A single class, if provided, must be first. If not provid

4:03 sudoer: Chousuke: ok, I am reading about prozy now, my goal is to get this working today!!

4:04 yason: sudoer: check out some of the Swing demo applications for Clojure, they generally use proxy to implement event listeners

4:04 sudoer: Chousuke: have you programmed cascading before?

4:04 so can I instanciate objects with proxy?

4:05 yason: sudoer: they look like real objects to Java

4:06 sudoer: can I declare the fake class though?

4:06 or must they be classes that already exist?

4:07 yason: sudoer: you don't have to: just list the interfaces you want your proxy to emulate and implement whatever methods you know you'll need

4:08 sudoer: yason: how would I implement something like this where there are no methods, just the public interface? http://pastie.textmate.org/691616

4:09 can I use proxy still for this situation?

4:10 yason: sudoer: You're eventually going to need an instance of PipeAssembly at some point?

4:10 sudoer: yason: of ImportLogAssembly, not directly of PipeAssembly

4:11 yason: sudoer: that's what you probably should proxy. In Java, you have to subclass something to implement your own stuff; in CLojure you proxy something to do that

4:12 sudoer: but ImportLogAssembly is just a subclass of PipeAssembly; I would assume that some Java code actually needs to do something with the PipeAssembly since ImportLogAssembly is clearly an implementation of it

4:13 sudoer: so can I still proxy in this situation?

4:13 yason: sudoer: you generally don't want to replicate how the subclass hierarchy is generally built in the corresponding Java code. You want to narrow down to the essential interfaces/classes that you need in order to plugin to this framework

4:14 sudoer: I don't know, I don't know enough about this PipeAssembly thing to know that.

4:14 sudoer: yason: yes, I will call this later: Pipe importPipe = new ImportLogAssembly( "import pipe" );

4:14 yason: that means I must be really familiar with the underlying java code I guess

4:14 yason: sudoer: okay, so where is importPipe used then?

4:15 sudoer: yason: it gets called later like so: Flow importPagesFlow = new FlowConnector().connect( localPagesSource, importedPages, importPipe );

4:15 yason: I am trying to port this example to clojure since I dont program in java: http://code.google.com/p/cascading/wiki/CrawlDataWordCountCascade

4:15 yason: sudoer: connect() method takes a Pipe right?

4:16 sudoer: not PipeAssembly?

4:16 sudoer: yason: it appears so, yes

4:17 yason: sudoer: so you probably want to proxy a Pipe so that you can do your own stuff with the pipe object

4:17 sudoer: but it seems to me that your example is a really bad one as for porting to clojure

4:18 sudoer: yason: its not a good fit? why is it bad?

4:18 yason: sudoer: it seems to be mostly about how java does things; you would want to know what the hell that mess is _supposed_ to do and implement that in Clojure, in a clean way

4:19 sudoer: Java defines lots of classes and instantiates lots of objects because it has to. I almost never do that in Clojure because I write Clojure, not Java. I've only used proxy for Java interoperations and I try to minimize the interop parts of my code

4:19 sudoer: so the question is how to solve the same problem that Java code solves, instead of how to write that Java code in Clojure

4:20 sudoer: hmm, unfortunately there is only one example of cascading and clojure used together and it was written by a clojure newb

4:21 and I'm a newb to both clojure and java

4:21 somnium: porting java to clojure without being familiar with either sounds like a painful endeavor

4:22 yason: sudoer: you could learn the basics of Java first, say I/O and file operations. Then write that in Clojure.

4:22 sudoer: yeah, haha

4:22 i mean I know basic java, but I really do not enjoy programming in java at all

4:23 i wrote some apps for android a couple of years ago

4:24 yason: sudoer: I don't like Java either so I like to keep it to the minimum.

4:24 sudoer: well take some very basic java program that only relies on the standard libraries and doesn't come with its own framework or anything, and try to do the same in Clojure

4:24 sudoer: i dont know any of the idiosyncrasies of java because I dont use it every day, I try to avoid it as much as possible

4:25 yason: yeah, that is probably a good idea

4:25 Chousuke: most of that code seems to be news anyway

4:25 sudoer: or I could bite the bullet and write java for now, that would probably be the fastest route

4:27 somnium: FWIW, thinking of java as garbage-collected C helps me find it more tolerable ;)

4:27 Chousuke: hm.

4:27 sudoer: somnium: objective c!

4:28 somnium: hmm, I think I'd rather have objective c if given the choice

4:28 Chousuke: those extended assembly classes are silly

4:28 they only have a constructor that calls a super method at the end :/

4:29 yason: sudoer: how about just writing clojure and looking up Java as you go, on a need-to-know basis? That's what I practically did.

4:29 sudoer: yason: I want to write it in clojure

4:30 yason: sudoer: so pick a problem that's generic and not in the Java problem space

4:32 Chousuke: looks like you can replace them with a function like (defn make-import-log-assembly [the-name] (let [p (PipeAssembly.) regex (RegexSplitter. (Fields. "url" "raw")) import-pipe1 (Each. name (Fields. "line"), regex) import-pipe2 (Each. import-pipe1 ...)] (.setTails p result-pipe) p))

4:33 hopefully.

4:33 if PipeAssembly is abstract... :-(

4:33 Is proxy capable of defining the constructor?

4:34 sudoer: Chousuke: that is a clojure question, not question about cascading?

4:35 somnium: Chousuke: have you been using the new features?

4:35 sudoer: Chousuke: I think it is abstract : http://www.cascading.org/javadoc/cascading/pipe/SubAssembly.html

4:36 somnium: I tried gen class, but it was 10x times slower than 100 lines of java

4:36 yason: Chousuke: I don't know but constructor in a proxy doesn't really make any sense because there's no object for you to construct.

4:38 somnium: ~proxy

4:38 clojurebot: proxy is <Chouser> proxy teases with its ease of use, then suddenly betrays.

4:38 Chousuke: somnium: gen-class itself is not the problem.

4:38 somnium: yes, the function lookups and state I guess? at any rate, it will be wonderful if we can get java speed without ever touching java

4:38 Chousuke: somnium: it creates a plain java class. it's the clojure code that's slower, because it's more dynamic and using primitives is more difficult :)

4:39 it can be optimised, but it's not very easy to match java speed. not yet, at least.

4:39 defclass etc. are supposed to help with that

4:46 somnium: does recur compile down to a for-loop? the code I needed just looped along a hashmap subclass, if (x typeof y) PersistentFoo.create() ... but I didn't know how to optimize the clojure version

4:47 (using clojure.walk probably didn't help)

4:47 Chousuke: did you eliminate reflection?

4:49 somnium: it didn't touch java much, I just used #^hashmap.keyset which had strings and the rest in clojure

4:51 Chousuke: that would be #^HashMap$KeySet :)

4:51 somnium: it seemed the java was faster at creating clojure objects than clojure, though its entirely likely my clojure code was poor

4:51 yes, I don't actually recall :) just only one direct java call and then condp instance?

4:51 I probably used #^Set or something

4:52 Chousuke: Oh well, I need to hurry away now. later.

4:52 somnium: bye

4:52 sudoer: Chousuke: one question about regular java: importPipe = new Each( importPipe, new Fields( "url" ), new RegexFilter( ".*\\.pdf$", true ) ); <--- isnt that a bad writing style to rewrite teh variable with the same variable?

4:53 Chousuke: sudoer: well, I think it's icky

4:53 certainly is in clojure, because it doesn't actually overwrite the old reference

4:53 it just shadows it.

5:28 ordnungswidrig: hi

6:16 AWizzArd: ~seen cemerik

6:16 clojurebot: no, I have not seen cemerik

6:17 AWizzArd: ~seen AWizzArd

6:17 clojurebot: AWizzArd was last seen in #clojure, 0 minutes ago saying: ~seen AWizzArd

6:26 Kjellski: I´ve got a question about the clojure bit-mapped hash tries: When we add one new thing, the path to that thing need

6:26 s to be created.

6:27 So, the pathes to every other thing begfore is just copied from old nodes?

6:27 I´m talking about these slides: http://clojure.googlegroups.com/web/ClojureConcurrencyTalk.pdf?gda=59CHlkwAAAB9TgGpbzbMg0BCq3i8IqXnCz2xnAVnahGaLFuwC71R5C1I1OtZCnFmn_w8aWW3nzHB8Gzz9WpxuUbp_zNxpCdh_Vpvmo5s1aABVJRO3P3wLQ

6:28 Page 21

6:28 _ato: Kjellski: what do you mean by "every other thing" just the path to the node you add needs to be copied

6:29 Kjellski: I know, but the pathes to the old nodes, where do they come from? Do we just copy old path and add one entry to the bottom node?

6:30 In other words: Is the copied path to the new node, a "java cloned" path that has all its entries and we´re adding one entry to the last one...?

6:32 _ato: Kjellski: oh right. No the whole path has to be changed, not just cloned... as each copied node in the path will have a different address

6:33 Kjellski: But Why? Wouldn´t it be just fine to copy every "not leaf" nodes until we get to the one that needs the new one?

6:35 chouser: Are you arround? ^^

6:35 _ato: suppose the path is nodes a b c. suppose our nodes currently look like a = [x b w], b = [c e r], c = [d e f]

6:36 we want to add a new value to node c

6:36 we create a new node C' with [d e f g]

6:36 but Clojure's data structures are immutable

6:36 so we can't just change b and a to opint at it, so we need to copy them too

6:37 so we make b' = [c' e r] and a' = [x b' w]

6:37 a' becomes our new referene to the data structure

6:39 Kjellski: Huh? a and b never changed. What I mean is just, looking at the picture of page 21 from the slides, our references in your example, to the xw and er and rf nodes, come from the old ones right? Our dashed lines to the shared structure, can be copied from old nodes...

6:40 _ato: yes, so x w don't change, but a and b need to because there's a new version of c, then there needs to be a new version of b that points at the new c, and thus there needs to be a new a to point the new b

6:41 Kjellski: Okay... that makes sense... sorry... I think I´ve got it. Thanks!

6:42 Do you know how "big" the node Objects are?

6:43 _ato: at most they have 32 entries, but they're only as large as they need to be

6:43 (plus wahtever java's normal object overhead is, probably 8 bytes or so)

6:44 also the tree is really shallow, because each node has up to 32 branches

6:44 so the paths that get copied are really short

6:47 Kjellski: I think that is pretty neat. ^^

6:50 ol3: hello, i am searching for a pathname-type method

6:50 _ato: yeah, they're pretty amazing.

6:51 ol3: is there something like that in java?

6:51 Kjellski: What is a pathname-type method?

6:51 _ato: ol3: is that a CL thing? what does it do, return the file extensions?

6:52 ol3: it returns the type of the file, normally the file extension

6:55 _ato: as far as I know there's nothing built-in to do that

6:55 Kjellski: And in the not normal case?

6:55 Why not just split at the last dot of path?

6:55 _ato: ,(last (.split "foo.txt" "\\."))

6:55 clojurebot: "txt"

6:56 _ato: ,(last (.split "foo.tar.gz" "\\."))

6:56 clojurebot: "gz"

6:56 _ato: ,(next (.split "foo.tar.gz" "\\."))

6:56 clojurebot: ("tar" "gz")

6:56 _ato: hmm

6:56 splitting extensions is hard because of cases like that :(

6:57 ol3: yes, that will work

6:57 ,(.substring "foo.bar" (+ (.lastIndexOf "foo.bar" ".") 1))

6:57 clojurebot: "bar"

6:57 ol3: but split looks better

6:57 ,(last (.split "foo.tar.gz" "\\."))

6:57 clojurebot: "gz"

7:06 ol3: ,(doc file-seq)

7:06 clojurebot: "([dir]); A tree seq on java.io.Files"

7:06 ol3: ,(file-seq "/")

7:06 clojurebot: java.lang.ClassCastException: java.lang.String cannot be cast to java.io.File

7:06 ol3: ,(file-seq (java.io.File. "/"))

7:06 clojurebot: java.security.AccessControlException: access denied (java.io.FilePermission / read)

7:06 ol3: ,(file-seq (java.io.File. "./"))

7:07 clojurebot: java.security.AccessControlException: access denied (java.io.FilePermission . read)

7:07 ol3: (take 1 (file-seq (java.io.File. "./")))

7:07 ,(take 1 (file-seq (java.io.File. "./")))

7:07 clojurebot: java.lang.RuntimeException: java.security.AccessControlException: access denied (java.io.FilePermission . read)

7:33 stephenj: ~def let

7:54 ~def pmap

7:59 ~def split-at

8:07 ~def future

8:26 Guest35930: I'm struggling to get with-out-writer in duck-streams to give me new lines when I (println ). How do I do this?

8:28 tomoj: Guest35930: it should already do that

8:30 Guest35930: Oh yes, it does! Notepad just isn't showing it. :)

8:30 tomoj: maybe it's a windows \r\n problem?

8:31 e.g. maybe if you did (println "\r") instead it would work? I dunno

8:31 Guest35930: Quite probable, I'm on an XP machine at the moment. I'll give that a try.

8:37 stephenj: Guest35930: Does the answer look different if you open it in Wordpad (rather than Notepad)?

8:38 Wordpad handles \n and presents it well - whilst Notepad insists on \r\n data

8:39 Guest35930: stephenj: Wordpad shows the intended output.

8:40 stephenj: Guest35930: So depending on which tool / environment, the output needs to adapt (or not)

8:41 Depend on what the desired results is...

8:41 Guest35930: Well I'm planning on feeding this file to a DOS program, so I probably need \r\n?

8:42 stephenj: probably

8:42 Sorry, must leave now - hope that helped

8:43 Guest35930: It did indeed, cheers.

8:50 It did require the \r. Thanks for the help tomoj.

8:53 tomoj: I wonder if there is a good way to automatically add \r

9:04 * cemerick feels dirty for using locking :-P

9:49 Kjellski: Is there any AI - Clojure approaches out there? Some libs?

9:49 -s

10:10 hamza`: hey guys, i have two maps, i would like to get a list of common keys right now i am using reduce to iterate map1 check map2 and build a vector? is there a better way to do this?

10:12 chouser: any nil or false values in either map?

10:12 hamza`: no they are a list of movies and scores

10:13 * a map of

10:13 chouser: ,(let [m1 '{a 1 b 2 c 3} m2 '{b 2 d 4 c 3}] (filter m1 (keys m2)))

10:13 clojurebot: (b c)

10:14 chouser: probably doing essentially the same work as you described -- walking through one map, looking up each key in the other

10:14 hamza`: wow, thx thats a lot shorter then reduce..

10:15 chouser: yw

10:36 tomoj: I feel bad wrapping up and hiding clojure.test/is

10:36 but I don't like it very much either

10:36 :(

10:37 chouser: what do you dislike?

10:37 tomoj: it just feels awkward to me

10:37 trying to translate things like "response.should have_content"

10:38 (is (has-content? response "foo")) works but seems... yeah, awkward

10:38 maybe I'm just not used to it yet

10:39 chouser: I've got very shallow experience with testing frameworks, so I don't have much opinion.

10:39 is "response.should have_content" from an existing framework/language?

10:43 cemerick: I'm guessing that's ruby

10:43 has a distinct AppleScript taste to it.

10:43 or, rails, I should say

10:44 chouser: Assuming 'response' is a application object, having a 'should' method seems a bit odd

10:45 rys: smells like rspec to me

10:45 cemerick: right, I was keying on the 'response' bit -- doesn't rails use rspec?

10:45 rys: And if it is, it's response.should have_content("string") I think

10:46 Although I haven't coded any ruby in an age

10:47 Ah, yes, that's what chouser said upstream

10:47 * rys returns to lurking

10:48 cemerick: I'm sure an rspec-esque test framework could be put together. Not sure I'd use it, tho. :-)

10:55 tomoj: yep, it's rspec

10:55 rspec puts a should method on everything :/

10:58 kefka`: Question about clojure.contrib.pprint: is there an easy way to set a left margin?

10:59 ,(+ 1 2)

10:59 clojurebot: 3

10:59 kefka`: (just making sure I'm connected; ERC is acting weird)

10:59 Anyway, I'm looking to set a left-margin in clojure.contrib.pprint

11:07 johny: hi

11:07 what's wrong with my installation : http://paste.lisp.org/display/90105 ??

11:07 the agent example does not return properly

11:10 tomoj: johny: nor does mine

11:10 chouser: johny: once the agent thread pools are used, clojure.main isn't sure all the work is done yet, so won't quit automatically

11:11 (doc shutdown-agents)

11:11 clojurebot: "([]); Initiates a shutdown of the thread pools that back the agent system. Running actions will complete, but no new actions will be accepted"

11:11 AWizzArd: cemerick: good that you suggested to continue the fixing in Contribs logging module.

11:13 cemerick: well, what's right is right. Some may worry about the fact that the solution would result in a deref call on every log, but I have to think it's worthwhile.

11:14 I suppose we could add an init fn to the ns, but that's just kicking the can down the road.

11:16 johny: when 'i'm using await it also waiting forever if not message was sent

11:16 is it a normal behavior ?

11:16 rhickey: I am considering protocols on interfaces being specified on a per-method basis, rather than (defprotocol AProtocol :on AnInterface ...)

11:17 this because: there will often be a name remapping

11:17 and, if there are extant interfaces, a single protocol might map to more than one

11:17 the tradeoff is it is more verbose in the 1:1 case

11:19 (defprotocol AProtocol

11:19 "A doc string for AProtocol abstraction"

11:19 (bar [a b] "bar docs" :on AnInterface/barMethod) ...)

11:19 chouser: johny: (let [a (agent nil)] (await a)) returns instantly for me

11:21 rhickey: renaming will be common because 'bar' will be a namespace-global name of a function while barMethod is an instance method on a specific class?

11:21 johny: (def node0 (agent 0)) (await node0) does not for me

11:22 chouser: johny: at a repl that returns instantly for me as well. What version of Clojure are you using?

11:22 AWizzArd: cemerick: yup, i absolutely agree and hope this will be fixed.

11:22 rhickey: chouser: could be for many reasons - you want to say my-fn here and myMethod there, or are mapping to a pre-existing interface with names you don't like

11:23 chouser: rhickey: will there be protocols on protocols, or is there even any value in that?

11:23 johny: 1.0.0

11:23 rhickey: chouser: no protocols on protocols, share impls by mixing in method maps

11:24 johny: calling it with "java -cp /opt/clojure/clojure.jar clojure.main /tmp/simpleagent.clj"

11:24 chouser: rhickey: defprotocol doesn't provide any impls anyway, right?

11:25 johny: and with java 1.5.0_13 , maybe too old ?

11:25 rhickey: chouser: I'm still not sure about :on, the multiple interface case might be fabricated, and the name mapping could go in a single map

11:25 chouser: right, no impls in defprotocol

11:25 sharvie: Hi, anyone know how to do the equivalent of ruby's respond_to? in clojure?

11:25 chouser: rhickey: anyway, so :on would only be needed for interop cases, either consuming or providing a Java interface.

11:26 rhickey: (defprotocol AProtocol :on AnInterface :mapping {this-fn thatMethod ...} ...)

11:26 johny: same pb with 1.6 :s

11:27 rhickey: chouser: :on is for extension via interfaces - could be a bout mapping to existing, allowing for extension via Java, or just fast-path (since deftypes could implement)

11:27 the :on interface will be a fastpath

11:27 tested first

11:28 chouser: rhickey: what about using the top-level :on as default, but allowing per-method :on to for renaming and disambiguation.

11:28 cgrand: and regular interop will use "implements", right?

11:29 chouser: I like having each defproto's method named once, rather than having to repeat it for the renaming cases.

11:29 rhickey: cgrand: maybe (extend AType AArotocol a-method-map BProtocol b-method-map ...)

11:31 chouser: if top-level :on then I don't think per-method could be anything other than name mapping (i.e. not to introduce another interface)

11:32 (a-method [args] "docs" :maps-to aMethod) or something

11:32 sharvie: exit

11:35 cgrand: rhickey: most of the time when you write (defprotocol AProtocol :on AnInterface ...) AnInterface will be an interface also written by the protocol author, no? Or are you thinking of retrofitting existing clojure interfaces (eg mapping IPersistentCollection/cons to conj)?

11:37 rhickey: cgrand: I don't want to decide this one way or the other - I am presuming both will occur with equal frequency (protocol author defines interface, protocol author maps to existing interface to allow for extension via protocols)

11:37 cgrand: ok

11:37 rhickey: obviously in the first case you have more control, but still might need mapping just due to Java name limitations

11:39 so if top-level :on and name inside, need better name for :maps-to (a-method [args] "docs" :maps-to aMethod

11:42 chouser: I don't understand why top-level :on preclude per-method interfaces if per-method interfaces could ever be an option.

11:44 rhickey: chouser: because it would be too confusing

11:44 chouser: oh

11:45 rhickey: seeing A Protocol :on AnInterface leads one to believe implementing that interface is sufficient

11:45 whereas the name mapping is more of an implementation detail, given good names and docs on the interface side

11:47 I guess :maps-to could just be :on again

11:48 chouser: allow for a list of interfaces at the top level, and for any conflicting method names require a per-method :on ?

11:49 rhickey: chouser: I've pretty much talked myself out of mapping to more than one interface, barring a real use case

11:49 chouser: okey-doke.

11:49 rhickey: so, :on in method for name-mapping?

11:50 lisppaste8: cgrand pasted "grouped by interface" at http://paste.lisp.org/display/90109

11:50 chouser: sure

11:50 cgrand: forget that then :-)

11:52 chouser: I could use a way to refer to a clojure ticket in a contrib commit.

11:59 qed: sigh -- i just re-read the bipolar lisp programmer essay

11:59 that is me to a T

12:03 tomoj: qed: sounds like me a bit too

12:06 at least the "abject failure" part :)

12:18 qed: tomoj: haha -- no i can just relate to the boredom, the bad grades feeling surrounded by phonies, etc.

12:26 Knekk: Intensity: yuck

12:31 yeah, that was #wrong

13:34 tomoj: wow, I got enlive's selectors working with the DOM tree from HtmlUnit

13:34 I didn't think it would be this easy :D

13:36 hiredman: htmlunit++

13:36 cgrand: tomoj: cool! Did you use a custom zipper?

13:37 tomoj: uh, should I have? :/

13:37 I just took the dom tree from htmlunit and made xml/element structs out of it

13:38 gonna put the dom elements from htmlunit into metadata so that I can use your slick selectors and then take the results and do htmlunit-ish things to them like clicking/form-filling/etc

13:38 cgrand: no, I don't think that a custom zipper would be enough

13:38 tomoj: well I don't even have that :O

13:39 I admit enlive is still mostly a mystery to me

13:46 cemerick: rhickey: is the only JVM lang summit video coming out the short stuff on youtube now, with bad audio?

13:54 rhickey: cemerick: It was recorded by InfoQ, I expect their usual v. good treatment eventually, but it is usually several months between event and release

13:55 cemerick: ah, ok

13:55 I was excited to see *some* video, but figured there had to be something better coming.

14:01 chouser: So far this month the most frequent poster to the google group has over four times as many posts as the person in second place.

14:02 cemerick: chouser: you should post the top ten every month or something. :-

14:02 :-P

14:03 rhickey: chouser: yes, how do you tell someone nicely that they need not participate in every discussion?

14:03 technomancy: along with Clojure's position on the Github languages high-score board. =)

14:03 (which, btw, just surpassed Scheme to take position 20.)

14:05 rhickey: technomancy: do they produce a sorted list anywhere?

14:05 technomancy: rhickey: no, but I had a script somewhere to build one.

14:06 if you're curious: CL is 19, Scala is 18, and Erlang is 17

14:06 danlarkin: The top10 is here: http://github.com/languages

14:07 djork: wow that's a very different picture than SourceForge

14:07 cemerick: I wouldn't have thought github would be attractive to CL folk.

14:08 chouser: It seems likely that the impression people have of a community could be influenced proportionally to the volume posted that portray various personalities and attitudes.

14:09 rhickey: hrm, richhickey/clojure is the most watched and most forked *Java* project this week/month/overall !?

14:09 so isn't contributing to Clojure stats I guess

14:10 chouser: rhickey: I assume that's correct. contrib drives the clojure stats

14:10 danlarkin: rhickey: http://github.com/richhickey/clojure/graphs/languages

14:10 chouser: haha! http://github.com/richhickey/clojure-contrib/graphs/languages

14:11 shoover: You could always make core.clj more verbose

14:11 rhickey: I guess cinc will fix that

14:11 chouser: apparently contrib is written mostly in javascript

14:11 rhickey: heh

14:11 chouser: er, largely

14:23 replaca_: kefka`: are you there still?

15:26 Maddas: Looks like Github doesn't know about Fortran, Ada, or Cobol :-/

15:27 Oh, it does know Fortran.

15:31 Chousuke: No Ada? :/

15:31 technomancy: Maddas: http://p.hagelb.org/github-langs.clj.html

15:32 hiredman: (ha ha VimL)

15:33 Maddas: technomancy: Thanks, I found a list on the site itself too.

15:33 * Maddas had forgotten about Arc :-)

15:33 technomancy: Maddas: you're not the only one! =)

15:36 Chousuke: Is Arc actually still being developed?

15:36 technomancy: no

15:40 Maddas: Paul Graham also forgot about it? =)

15:41 hiredman: http://groups.google.com/group/unladen-swallow/browse_thread/thread/4edbc406f544643e <-- I wonder if this means google is looking for another dynamic language

15:41 I imagine they'd end up with rhino though

15:43 cemerick: rhino is really far behind on AOT compilation, last I knew

15:44 rhickey: hmm, I missed this last week: http://fwierzbicki.blogspot.com/2009/11/leaving-sun.html - so JRuby and Jython guys all gone from Sun now? That's really too bad

15:44 cemerick: more likely js than rhino in particular

15:44 given V8

15:45 cemerick: right, I forgot they had their own internal vm

15:46 headius: rhickey: I feel for the folks still at Sun, actually

15:46 rhickey: headius: I can imagine it's simply awful, waiting like this

15:46 headius: when we left in july we didn't like the idea of the axe hanging for a couple more months

15:46 chouser: I'm under the impression Paul Graham is still using and possibly making improvements to arc.

15:46 fogus_: technomancy: Paul Graham *does* still work on Arc... he blasted me about implying otherwise. :O

15:46 headius: now it could be year(s) what with the EU objection

15:47 rhickey: headius: yes, amazing

15:47 headius: the oracle/sun deal has probably done more to set back java platform dev than anything in the past decade

15:47 hiredman: :(

15:47 headius: rhickey: btw, I want to try writing a plugin for Duby that adds first-class lang support for clojure refs, collections, and stm

15:48 rhickey: cool

15:48 headius: I'll ping you when I get some time to work on it

15:48 cemerick: has anyone played follow-the-money on the EU thing? I doubt there's principled objections involved.

15:48 headius: that's quite a quote

15:49 headius: cemerick: and you can feel free to quote me

15:49 before this deal I might have said "JavaFX has probably done..." :)

15:49 chouser: heh

15:56 hiredman: VimL's not just there, it's above Emacs Lisp and all the other lisps.

15:57 hm. I've got a literal UTF-8 string in a .clj file that when written out to a data file is having it's multibyte characters replaced by question marks.

15:58 oh, I bet it's FileWriter's fault.

16:00 ah, and it worked when run from the command-line because my default encoding there is UTF-8, but when run from cron it must be something else...

16:02 hiredman: chouser: :(

16:02 yegge should do something about vim's scripting language while he is doing something about emacs

16:03 chouser: he doesn't believe in vim because the smart people don't use it

16:03 hiredman: chasing down all the places where you have to set the checkbox to "UTF-8" is a pain

16:03 chouser: :(

16:03 could be true, I guess

16:04 fogus_: pg uses vim, that must count for something right?

16:04 jabley: fogus_: beat me to it

16:04 chouser: oh, does he? I didn't know that.

16:04 he uses etherpad.

16:04 fogus_: etherwhat?!?

16:05 That's soooo last week

16:05 chouser: heh

16:06 http://etherpad.com/ep/pad/slider/foundervisa -- PG using etherpad

16:16 hiredman: clojurebot: git fbacc4a5751fa5c15baa599b5a058cd81b05a247

16:16 clojurebot: Added bound-fn to define thread-local binding aware functions bound-fn captures the thread-local bindings in effect where the function is defined. Installs these bindings before executing the body. This is useful for helper functions running in a different thread. Excluded with-bindings from clojure/main.clj to prevent name clash. Fixes #170 Signed-off-by: Chouser <chouser@n01se.net>

16:17 hiredman: whee

16:21 djpowell: do you think clojure's println and stuff should be respecting platform newlines?

16:22 hiredman: what do mean?

16:22 jbendig: \n versus \r\n?

16:23 hiredman: but, how exactly?

16:23 djpowell: yeah

16:23 hiredman: what if I want a literal \n on windows?

16:23 djpowell: (def *nl* (System/getProperty "line.separator"))

16:23 chouser: (prn-str) should return "\r\n", "\n" or "\r" depending on platform

16:23 djpowell: well it shouldn't rewrite \n's

16:23 chouser: ?

16:23 djpowell: chouseer: yes

16:23 er sry typo

16:24 hiredman: djpowell: so, again, "what do you mean?"

16:24 djpowell: hiredman: println and prn should output a platform specific newline at the end of the line

16:24 hiredman: oh

16:24 djpowell: any literal \n's will remain #x0a

16:24 hiredman: simple enough

16:24 I though it did that

16:25 djpowell: not sure - i think it doesn't ?

16:25 hiredman: ~def prn-str

16:25 failure to render correctly in firefox

16:26 ah

16:26 ~def newline

16:27 djpowell: actually, make a *line-ending* dynamic var, so you can rebind it if you want unix outputting on windows or whatever

16:28 hiredman: you can dynamically rebind newline to a fn that returns the line.separator property

16:33 djpowell: how are you supposed to use *err* btw? rebinding *out* to it?

16:36 chouser: djpowell: sure, or just write to it (.write *err* "error text\n")

16:38 djpowell: yeah, but then you miss out on some stuff like auto-flushing, and er newline handling

16:38 i noticed that clojure.main printlns to *err*, which is making some assumptions about what class *err* is

16:39 chouser: indeed. won't work if *err* is an integer for example.

16:39 djpowell: only PrintWriters support .println, and if you're rebinding, you probably don't want to be using a printwriter, cause they eat exceptions

16:39 ho ho

16:39 chouser: :-) sorry

16:39 hiredman: ,(doc *err*)

16:39 clojurebot: "; A java.io.Writer object representing standard error for print operations. Defaults to System/err, wrapped in a PrintWriter"

16:50 spuz: I'm trying to write a paint function that draws on a BufferedImage, however as the paint method is called from Java, it cannot have the image passed into it as a parameter. Also as I don't know the width and height of the image until runtime, I can't define it as a var. How could I use something like dynamic binding to first create the buffered image and then access it in a paint function?

16:51 I've tried something like:

16:51 (def *img*) (defn draw [] (set! *img* (create image))) (defn paint [graphics] (drawImage graphics *img))

16:52 but apparently, you are not allowed to re-bind a var with set! until it has been initially bound

16:54 chouser: you'll implement the paint method using proxy?

16:57 _ato: spuz: use a closure when you make your proxy, like? (let [image whatever] (proxy [whatever] [] (paint [g] (draw g image))))

16:57 spuz: chouser: yes

16:58 _ato: if image isn't created at that point, or you need to change it, then make it an atom/ref

17:00 hmm is it just me or is everyone receiving 7 or 8 copies of each mailing list post? either google groups or my smtp server seems to have gone crazy

17:01 _mst: hm, no weirdness for me

17:02 _ato: hmmm.. it looks like it's google groups, there's different timestamps for the duped messages by their SMTP servers, oh well

17:04 shoover: spuz: using binding somewhere on the call stack, then you can set! if you want: (def *img*) (defn draw [] (binding [*img* (create-image)] (let [g (create-graphics)] (paint g)))) (defn paint [g] (drawImage g *img*))

17:05 spuz: shoover: hmm thanks

17:11 the-kenny: Is there a library like contrib.server-socket, just for client-sockets?

17:14 patrkris: Could Clojure get some of the benefits of Erlang if the JVM still supported green threads? I'm thinking for massively concurrent scenarios, where the overhead of context switching would be large.

17:17 hiredman: patrkris: I think it's telling that green threads are not used alot these days

17:18 mostly because smp with native threads is less complex

17:19 patrkris: hiredman: yeah, well ... but shouldn't you be able to gain something without the context switching? I'm not suggesting to use only green threads instead of OS threads, but for highly concurrent scenarios, it may be nice to use green threads

17:19 The same idea that Stackless Python uses

17:20 As far as I know, Stackless Python uses green threads, and it is one of the reasons it is used for Eve Online, which exhibits concurrency on a massive scale

17:20 hiredman: well, python is horrible at concurrency anyway

17:20 so, uh, any example with python is a deviant case

17:21 patrkris: well, yeah there are problems with the global interpreter lock and so on

17:22 hiredman: actually, the wikipedia article on green threads says that native threads outperform green threads for context switching on linux

17:22 patrkris: really... i'll check that out

17:22 hiredman: patrkris: as I said, I am aware of python's limitations in that regard, so I don't need you to repeat them to me :P

17:22 patrkris: sorry

17:24 hiredman: and explaining why python is bad at concurrency does not make it less bad

17:24 spuz: patrkris: Scala has erlang type concurrency, I'm not sure whether it uses green threads though

17:25 patrkris: hiredman: i was just mentioning stackless python as an example of a system that uses green threads - not trying to advocate python in particular

17:25 hiredman: spuz: it doesn't

17:54 the invokedynamic pdf on hackernews uses the word "reified"

17:54 first time I've seen it outisde of #clojure

17:54 :P

17:58 djpowell: you hear it a lot with RDF

17:59 spuz: djpowell: RDF?

18:00 djpowell: spuz: http://www.w3.org/RDF/

18:00 hiredman: In the Core Reflection API, the simulation signature 〈as(*),rs(*)〉 is

18:00 always 〈Object[], Object〉. JRuby has about 8 simulator signatures,

18:00 and Clojure has about 21.

18:00 interesting

18:01 djpowell: spuz: a data model for representing information (on the web) as graphs with an xml seralisation and stuff

18:01 hiredman: I am subscribed to something called planet rdf

18:02 "In a nutshell,

18:02 function pointers have arrived in the Java virtual machine"

18:03 spuz: is invokedynamic actually going to appear in 7?

18:04 hiredman: yeah

18:05 it's in, whenever 7 chooses to make an appearance

18:06 spuz: well this paper is way too heavy going for me, but maybe someone will write a blog post as to what it actually means :p

18:06 hiredman: I suddenly have a feeling that rich's protocol work and invokedynamic are connected

18:06 spuz: the paper or invokedynamic?

18:06 there are a lot of blog posts on invokedynamic

18:06 spuz: hiredman: both

18:07 yeah I've seen a few but I think haven't properly understood the context

18:14 replaca_: Rich makes no secret of the fact that he is voraciously following other developments in academia, the JVM, etc.

18:15 hiredman: sure

18:16 I just hadn't caught onto to protocols+invokedynamic yet

18:17 Chousuke: that paper is quite interesting.

18:17 haven't even finished reading page three yet though. But so far it has been :P

18:19 hiredman: the paper is over my head too

18:21 I like the fact that the paper is java java java λ calculas

18:28 _ato: hehe the java syntax for it is pretty horrible (like the curried greeter example)

18:30 could be worse though I guess

18:53 hiredman: http://scienceblogs.com/goodmath/2009/11/philosophizing_about_programmi.php

19:25 lusory: hi, does anybody know whether there are sample codes showing how to pack/unpack data from the network?

19:27 duncanm: hmm

19:28 i just watched this techtalk on Google's new Go language, it's a bit underwhelming

19:30 froog: link?

19:31 duncanm: froog: it's on youtube

19:34 aha, it's twbray

19:35 twbray: duncanm: G'day

19:36 duncanm: twbray: i was gonna leave a comment on your blog (or something); i saw your tweet about having functions with too many arguments from a few weeks ago, i think a judicious use of maps and structs will help with that

19:37 twbray: duncanm: Yeah, I eventually poured a bunch of somewhat-related things into a map and passed that around instead. Hey, if some of them were functions would that be O-O?

19:37 duncanm: twbray: yeah, it's a well-known technique used by Schemers

19:39 twbray: the trade off is that dispatch is explicit (which is tedious to implement again and again), the plus side is that there's no distinction between functions and methods (usually just a symbol, ie in Smalltalk)

19:40 twbray: duncanm: Don't understand "dispatch is explicit"

19:41 duncanm: twbray: you have to remember to pull out the right method from the map when you call, and use it on the right data, on more traditional OO systems, you can't make that mistake

19:43 twbray: Right, the O-O flavor is not very strong.

19:51 duncanm: twbray: do you also write a lot of JRuby code?

19:52 twbray: i really like that you can pass blocks in Ruby to make instances of interfaces (like java.io.FilenameFilter)

19:52 twbray: duncanm: Not really. I'm usually in MRI

19:52 Although the last piece of Ruby code I wrote ended up getting deployed in JRuby/GlassFish, which is not terribly rare these days

20:02 Damn... seem to be stubbing my toe on a nasty little bug. Is Clojure 1.0 very far out of date? Worth updating?

20:08 hiredman: twbray: have you seen map destructuring?

20:09 twbray: hiredman: nope

20:09 clojurebot: hiredman <3 XeLaTeX

20:09 hiredman: hmmmm

20:10 hard to say, I usually use clojure from git, but the times I have used 1.0 I haven't hit any differences

20:10 but my use of 1.0 is limited

20:11 twbray: I have a ref to a map that's refusing to deref when accessed in a sub-thread. Blecch. Works perfectly called from REPL.

20:11 hiredman: what do you mean refusing to deref?

20:12 twbray: I do (sort-by last > @unsorted) where @unsorted is a ref to a map and it works fine at top-level, when executed in a thread I get "java.lang.RuntimeException: java.lang.ClassCastException: clojure.lang.Ref cannot be cast to java.lang.Number"

20:13 hiredman: are you using agents at all?

20:13 twbray: Oh............. hold on. I'm a moron.

20:13 hiredman: I would suggest dropping a call to prn in there

20:13 (the subthread)

20:14 twbray: D'oh, the map values are refs to numbers not numbers. The error message is perfectly correct and I'm a dork.

20:14 hiredman: if you are using agents at all it is very easy to accidently add layers of refs

20:14 :P

20:14 prn is your friend

20:33 burny: I'm giving a presentation on clojures to my university Programming Language Concepts class tomorrow. We've studied scheme, prolog, and javascript, most of the people are familiar with java and c++. Any suggestions on area's in particular to cover for a general overview of the language in about 15/20 minutes?

20:38 arbscht: clojurebot: rationale

20:38 clojurebot: rationale is http://clojure.org/rationale

20:38 hiredman: you might take a look at rich's "clojure for java programmers" and see what he covers

20:39 clojurebot: blip.tv

20:39 clojurebot: blip.tv is http://clojure.blip.tv/

20:39 burny: yea, I've gone over it a couple times

20:39 thanks

20:40 hiredman: there are a lot of little neat corners of the language

20:40 callability as an interface anthing can implement

20:56 chouser: rhickey: will the next release be called 2.0, since we're now past Clojure's second year?

20:58 rhickey: chouser: good question, I don't think so

20:59 duncanm: hiredman: ah, that's a good point

20:59 twbray: HAH! Am now keeping 25+ on average of the 32 cores in a Niagara maxed processing the Wide Finder dataset.

21:00 rhickey: twbray: cool

21:01 twbray: Gonna have to get a JVM/Solaris guru to look at this puppy, swings wildly back & forth between high concurrency & occasional single-threadedness, I'm sure there are JVM options & so on to help.

21:02 durka42: garbage collection?

21:02 hiredman: ~google headius jvm flags

21:02 clojurebot: First, out of 279 results is:

21:02 Headius: My Favorite Hotspot JVM Flags

21:02 http://blog.headius.com/2009/01/my-favorite-hotspot-jvm-flags.html

21:03 twbray: Yeah... currently java -d64 -Xmx18000m -XX:+UseParallelGC

21:06 rhickey: now with dates - http://pragmaticstudio.com/clojure

21:08 twbray: I suppose that a big clojure program is going to generate disposable java objects at a ferocious rate...

21:08 rhickey: twbray: yes, lots of ephemeral garbage

21:10 twbray: Interestingly, my program had the computer maxed until it hit its max heap (18G) and now the concurrency is varying between 1x and 8x, presumably because it's competing with gc all the time.

21:10 headius: twbray: you might want to try +UseConcMarkSweepGC

21:10 it should help a bit

21:10 parallelGC doesn't actually run concurrently

21:11 it just stops the world for less time by using multiple threads to GC

21:11 you must be generating a ton of garbage

21:11 I don't know if that's typical for clojure ornot

21:11 twbray: Well, it's Wide Finder... it has a ref to an int for each URL (a few thousand of 'em) and it's updating them at many tens of thousands per second.

21:11 hiredman: I generate tons of garbage :P

21:12 twbray: Gosh.... does this mean that immutability isn't free?!? I'm shocked... shocked.

21:12 hiredman: that doesn't sound like it should generate garbage

21:12 twbray: Well, every time I do (ref-set some-int (inc @some-int)) the old value is now garbage. Right?

21:13 rhickey: hiredman: boxed numbers can yield a lot of garbage

21:13 hiredman: oh

21:13 right

21:13 * headius shakes his fist at fixnums

21:13 hiredman: twbray: (alter ref inc)

21:13 rhickey: yes, please

21:14 * twbray shuffles off to read up on "alter"

21:14 hiredman: ,(doc alter)

21:14 clojurebot: "([ref fun & args]); Must be called in a transaction. Sets the in-transaction-value of ref to: (apply fun in-transaction-value-of-ref args) and returns the in-transaction-value of ref."

21:14 headius: rhickey: we need to get a pool together for fixnums in mlvm

21:14 rhickey: headius: I think it's the #1 problem for dynamic langs on the JVM

21:15 perf-wise

21:15 headius: probably...and #2 would be lack of closure-conversion during inlining

21:15 if we had both of those I'd have far fewer headaches

21:15 twbray: Why would alter generate less garbage?

21:15 rhickey: twbray: it doesn't, just more idiomatic

21:16 hiredman: twbray: I dunno that it would, but alter is prefered to ref-set

21:16 twbray: Crap... 30 minutes into the run, it's now single-threaded :(

21:16 OK, gc research after dinner.

21:16 headius: twbray: log some GC stuff to see if that's the problem

21:16 hiredman: you might be ok with commute instead of alter, but I always get confused so I stick to alter

21:17 headius: it shouldn't impact you *that* much

21:17 clojurebot: much is permittted

21:17 headius: -XX:+PrintGCDetails

21:17 if you see incremental GC runs streaming by, or lots of full GCs, you'll know it's an issue

21:17 twbray: headius: will do

21:18 duncanm: oh, this is fun

21:18 twbray: you're gonna write a blogpost on this debugging session, right? ;-)

21:19 twbray: Oh yeah. Lotsa fodder here. But I want some good numbers.

21:20 As in "dramatically faster than the equivalent perl code on a multi-core machine"

21:20 duncanm: twbray: the perl code isn't using the multi-core at all, is it?

21:20 chouser: headius: what's closure-conversion during inlingin?

21:20 twbray: "equivalent single-threaded perl code" I mean

21:20 duncanm: ah

21:20 * _ato wants to make some changes to technomancy's project specification syntax: http://gist.github.com/231501

21:22 headius: chouser: current JVMs will not inline closures all the way back to where they're instantiated, since the method they're passed to ends up being megamorphic

21:22 a ruby example: [1,2,3].each {|x| ...}

21:22 chouser: megamorphic means types with perhaps the same method name but no parent class in common with that method name?

21:23 headius: you'd like "each" and the closure to inline into the caller, but they can't because each gets used for lots of different call paths

21:23 chouser: ah

21:23 headius: megamorphic means they're all of some "closure" type, but there's many different ones

21:23 unless each only ever receives one closure, it won't inline it

21:24 that means you can't get most inlining optimizations whenever there's an intermediate call like "each"

21:26 jrockit guys said it wouldn't be hard to add, but it's not in any current JVMs

21:29 rhickey: more frustrating to me than lack of fixnum support is that the majority of comparison benchmarks on the interwebs are numeric-heavy...so we end up looking sucky

21:31 rhickey_: headius: exactly, the most trivial things look the worst, too

21:32 headius: yes

21:32 stupid fi

21:32 fib

21:32 duncanm: heh

21:32 headius: if I had time for more projects I might work with maxine guys to hack fixnums in

21:33 duncanm: oh, i read about maxine, it looks pretty cool

21:33 headius: do you know what the status is for interface injection?

21:33 headius: duncanm: fading quickly...no resources to work on it

21:33 at this point I think I might like fixnums more

21:33 but I want it all

21:33 duncanm: oh, that's too bad, that's one feature that I find very exciting

21:34 headius: I find then all exciting

21:34 I wish I knew hotspot better to help out :)

21:34 bbl

22:45 qed: oh man im excited for RubyConf

22:45 S. Halloway has a talk about Clojure

22:45 wooby: will the vids be up somewhere?

22:46 qed: yeah eventually

22:47 actually they will likely be up the same day

22:47 wooby: awesome

22:48 qed: i hope clojure sticks around for awhile

22:48 JAS415: oh jeeze

22:48 me too

22:48 qed: ive really grown to love it, except for the whole JVM thing

22:48 ;)

22:48 i mean, if i was a java programmer in the first instance, i would be loving clojure

22:49 but for me this is more like my first foray into java

22:49 ive avoided it for such a long time

22:49 i wrote some long long ago in a far away place

22:49 wooby: qed: what did you work in primarily?

22:50 qed: wooby: C++ at first, then I got turned on to perl, then to ruby, ruby has lasted me awhile, then i found clojure

22:50 well actually i found a lot of languages in between ruby and clojure

22:50 but clojure is the only one i thought to be worth learning

22:51 i played with haskell for awhile, got bored

22:51 clojure has kept me interested

22:51 wooby: yeah i tinkered with haskell also

22:51 qed: i think most of us did

22:52 clojure is where the haskell rejects go

22:52 :X lol

22:52 wooby: followed the java -> ruby -> scala trajectory and settled on clojure finally, love it... feels like i won't want or need to learn anything else for a long time

22:52 chouser: where would clojure go if it doesn't stick around for awhile?

22:52 qed: chouser: i didnt mean anything by that

22:52 chouser: oh, ok.

22:53 qed: i guess what im saying is that im praying for a long and healthy prosperous language

22:53 chouser: :-)

22:53 qed: i really enjoy clojure and cant wait to see the things people will inevitably build for it

22:53 wooby: i cant wait to see more clojure jobs ;)

22:53 qed: oh man

22:53 if i could get a job writing clojure...

22:54 wooby: i know

22:54 qed: the world would likely explode

22:54 wooby: that's the ultimate

22:54 qed: that's like LAMBDA

22:54 wooby: second only to being paid to work on my own toy languages lol

22:54 qed: haha

22:54 hey now -- clojure aint just a toy anymore

22:54 or at least i dont see it that way

22:54 it's maturing pretty rapidly

22:54 wooby: i know but my stuff is ;)

22:55 qed: chouser: what do you see the 5 year outlook for clojure being?

22:55 chouser: nothing too serious, just sort of your wishes and hopes

22:57 chouser: a variety of hosts besides JVM -- iphone, jvm, clr, javascript, chickenscheme, maybe C++

22:57 qed: i didnt even consider that

22:58 what is clr chouser ? ive heard that term passed around a lot.

22:58 wooby: it's like microsoft's jvm

22:58 chouser: qed: that's the C# runtime

22:58 qed: ahhh

22:58 what about ruby?

22:58 :X

22:58 _ato: parrot

22:58 qed: is that heresy?

22:58 wooby: chouser: have you looked at nu?

22:58 chouser: yes, parrot

22:58 wooby: I don't think so

22:59 qed: the iphone would be really interesting

22:59 chouser: someone will have solved the dependency problem well, so you'll declare your deps, and someone can install with a single command that also fetches and installs the dpes.

22:59 wooby: chouser: http://programming.nu/

22:59 qed: chouser: fwiw i appreciate so much the work youve done on clojure

23:00 its not said enough

23:00 thank you

23:00 chouser: heh. well, thanks. I do hardly anything.

23:00 rhickey is the workhorse

23:00 qed: you're part of the machine, and you being in here right now answering my questions as you've done for the past month or so

23:00 has not gone unnoticed

23:01 so thanks :)

23:01 chouser: is there any active documentation project going on?

23:02 one thing i miss is a really complete set of documentation that includes code samples

23:03 chouser: wooby: No, I haven't looked at nu before.

23:03 st3fan: nu is interesting

23:03 devlinsf: @qed It's a work in progress

23:03 wooby: chouser: it's probably only worth a look if you're into obj-c/cocoa at all, but i've found it a really nice way to build things for macs

23:03 chouser: and the runtime definitely pays the way for clojure-on-cocoa :)

23:03 *paves

23:04 chouser: qed: there are a few efforts related to docs going on. That'll be solved in the next 5 years too. :-)

23:04 devlinsf: @qed There is some effort going on trying to improve the documentation for contrib. Tom F. is doing a lot to create refernce documentation

23:06 danlarkin: s/someone/technomancy :-D

23:09 technomancy: danlarkin: ssshhh! I'm not _quite_ ready yet. =)

23:09 burny: Any suggestions for my presentation? http://docs.google.com/present/edit?id=0AZJoMk7c8j5uYXd6NWNoY3hjbm1fNjJrazV0N2Jodg&hl=en

23:10 wooby: burny: looks awesome, sp on pg 3: "Functinonal Languaguage"

23:10 reading though

23:10 chouser: wooby: yeah, any lisp-like syntax with macros is a win, even if it's a thin layer over a host language

23:11 _ato: burny: slide 5 s/Full/Fully/

23:12 burny: _ato: wooby: haha oops! thanks

23:13 wooby: burny: is it for school?

23:14 burny: wooby: yea, Programming Languages course

23:14 durka42: if we're into spelling, Breif on slide 2

23:15 wooby: burny: i like the compojure/incanter examples, cool way to show the power

23:16 _ato: burny: I'd say something about how the mind-blowing persistent hash maps and vectors are immutable yet still fast and memory efficent (due to state sharing), but it depends on the audience :)

23:17 burny: _ato: I'm gonna do a run through, if I have some time I'll be sure to add it in, thanks.

23:17 chouser: burny: should use (doseq ...) instead of (dorun (for ...)) unless you're specifically trying to show 'for'

23:19 burny: chouser: Thanks, I didn't know that

23:19 devlinsf: chouser: got a minute to talk about c.c.map-utils?

23:20 wooby: burny: do you go to RIT by any chance?

23:20 qed: technomancy: now i know why you got turned down for your rubyconf proposal

23:20 what with stuart H. getting his clojure presentation accepted and all

23:22 chouser: devlinsf: hm, my name is in that file, isn't it.

23:22 I wonder why

23:22 timothypratley: stupid question, I'm formatting a patch and have gone blank on how to run the tests

23:22 chouser: ah, deep-merge-with.

23:22 technomancy: qed: yeah, but my talk was actually related to ruby. =\

23:22 devlinsf: chouser: deep-merge-with

23:22 chouser: devlinsf: what's up?

23:23 timothypratley: for clojure itself? ant jar test

23:23 timothypratley: thanks!

23:23 devlinsf: chouser: I've had a post about two libs on the dev list, c.ctable-utils & c.c.map-utils. I think the table-utils lib needs a little more time, but the map-utils is ready to go

23:24 chouser: do I need Rich's approval for a patch?

23:24 chouser: devlinsf: not for contrib, no.

23:25 devlinsf: chouser: okay, so an assembla ticket w/ docs, test & code could get applied. Who would do the actual application?

23:25 chouser: devlinsf: I see your posts. I had forgotten I was responsible for map-utils. I'll take a look at some point.

23:26 devlinsf: chosuer: thanks

23:27 timothypratley: BUILD FAILED Target "test" does not exist in the project "clojure". <--- should I be worried?

23:28 arrrrghhhh!!!! ignore me, I was in a really old snapshot directory for some reason

23:29 *slinks away*

23:35 * technomancy contemplates pulling the trigger on the swank-clojure 1.0 release tonight

23:35 JAS415: does anyone use clojure from a VPS?

23:37 nvm found something about it through google :-)

23:47 burny: wooby: yes I do

23:49 wooby: me too

23:50 burny: very cool

23:50 se/ce/cs/one of the other hundreds of computing majors?

Logging service provided by n01se.net