#clojure log - Sep 08 2010

The Joy of Clojure
Main Clojure site
Google Group
List of all logged dates

0:14 thunk: I'm a little unclear on the distinction between toplevel variable bindings and function argument bindings.

0:14 At the toplevel, a symbol refers to a variable which is bound to a value.

0:14 But are function argument symbols bound directly to values, or do they also refer to variables (which may shadow variables referred to by toplevel symbols) that are bound to values?

0:14 It's probably a mostly meaningless distinction, but I'd like to have it straight.

0:16 I guess what I'm asking is, what exactly *is* a variable object, and does it come into play in function argument bindings?

0:18 wwmorgan: thunk: I think if you understand distinction between Symbol and Var then that will go a long way to clearing up your confusion

0:19 to partially answer your question, a Symbol in clojure is just a (possibly namespace-qualified) string. In an OO sense, the state of a Symbol contains no binding information

0:20 thunk: Ok, so far so good.

0:21 Then each namespace probably has a table that maps symbols to variables?

0:22 wwmorgan: thunk: yes. The Namespace object contains the bindings

0:22 thunk: And those variables may or may not be bound to actual values.

0:24 wwmorgan: a Namespace maps Symbols to Vars (or java.lang.Class objects). See ns-resolve

0:25 thunk: And function arguments are also just symbols. But do argument values get lexically bound directly to those symbols, or is there also a variable intermediary?

0:26 arohner: thunk: locals created by let, and fn arguments are bound directly to symbols. They're immutable

0:27 thunk: arohner: Ahh, I see.

0:28 I'm coming from a CL background, so this is new to me.

0:28 wwmorgan: thunk: you can't get to the "underlying Var" of an argument to a function from within that function. If you needed something like that, you would pass the Var object into the function using #'

0:33 thunk: wwmorgan: Ok. That makes sense. So there's one there, but it's inaccessible to preserve immutability.

0:34 Tutorials tend to omit these things, presumably to make things easier to understand. But they tend to confuse me instead.

0:34 arohner: thunk: no, there's not a variable object there

0:34 the locals map is just a clojure map of symbols to values

0:35 wwmorgan: thunk: I don't think that was the motivation for it.

0:36 thunk: Ok. I must have misunderstood you. I've got it now, though.

0:36 Thanks much.

1:14 So then what is it that variables actually do?

1:14 If symbols are mapped direcly to values in local let and defn forms, couldn't the same also be done in the namespace map?

1:15 wwmorgan: thunk: I found this. There may be other reasons for it. http://clojure.org/vars

1:23 thunk: Thanks wwmorgan.

1:24 There's something I'm still not getting, but I should probably just let it go, and use clojure until it makes sense.

1:25 wwmorgan: thunk: if you speak java then it's quite illuminating to read the source, or at least the interface definitions, of clojure.lang.Symbol, clojure.lang.Namespace, clojure.lang.Var, etc.

1:27 thunk: Ok, will do.

2:01 defn: hello all

2:06 replaca: good evening defn

3:00 LauJensen: Good morning all

3:17 jave: id like a pointer to the src of a real world compojure app

3:17 id like to find some hints on how to handle global state and such

3:18 LauJensen: jave: Would Moustache work, or does it have to be Compojure?

3:18 jave: moustache would be ok I think.

3:19 havent done web dev in a while so I'm rusty on how to aproach where to store things like a global cache and so on

3:19 LauJensen: In a couple of hours Im blogging about bestinclass.dk and how it's built using Enlive/Moustache. I said something about it in the past when I opensourced it: http://bestinclass.dk/index.clj/2010/05/refresh-your-cache--best-in-class-has-been-baked.html, http://bestinclass.dk/index.clj/2010/06/best-in-class--now-open-sourced.html

3:20 I haven't been able to convince any of my clients to OpenSource any actual web applications though

3:20 jave: coolness

3:20 I just need some hints

3:20 in this particular instance I'm writing some glue code for sonar

3:21 LauJensen: There are many hints in those posts. If you check out admin.clj and templates.clj in the Github repo you should be set for greatness

3:21 jave: splendiferous

3:21 LauJensen: precisely

3:22 jave: clojure makes it actually fun to code again

3:23 LauJensen: Yes it does. And after 3 years I had almost gotten used to it, but a few days of Java fixed me right up :)

3:24 jave: heh

4:35 LauJensen: jave: blogpost/screencast is up now, with a 7 minute explanation of the setup

4:37 notsonerdysunny: is there something that would automatically generate a xml file which would detail the struct/class and its member variables or functions along with the argument types and return types...

4:39 LauJensen: Not that I know of

4:41 notsonerdysunny: http://groups.google.com/group/clojure/browse_thread/thread/9f69e649c98a9617 .. is the reason I am asking

4:41 but is it straightforward to write one ?

4:42 LauJensen: Sort of. But I think you'd want to look at clojure autodoc

4:42 notsonerdysunny: LauJensen: if so .. would have a suggestion where I could start...

4:45 what I am trying is much simpler .. what I would like is some way to get a list of all the defined classes/structs and interfaces (probably there is a function for this..) and run through its elements .. (I can do that right?) and create the html file ..

4:45 *xml file

4:45 if these functions exist .. what would they be?

4:47 hoeck: notsonerdysunny: maybe look at http://en.wikibooks.org/wiki/Clojure_Programming/Tutorials_and_Tips#Invoking_Clojure_from_Java and then create an XML for the classes mentioned there, e.g. clojure.lang.RT

4:48 notsonerdysunny: with reflection, you get everything but the parameter names of methods

4:48 notsonerdysunny: actually that should be fine ..

4:49 can I access their types ?

4:49 hoeck: sth. like (map (.getParameterTypes %) (.getMethods clojure.lang.RT))

4:51 notsonerdysunny: oh that is neat ..

4:52 hoeck: did you visit the thread I had mentioned above?

4:53 hoeck: notsonerdysunny: yes

4:54 notsonerdysunny: hoeck: do you think we can redefine defstruct to do the job .. would you have a suggestion?

4:55 hoeck: from what I can see, it should not be necessary to wrap defn, because you would use RT.var to look up and invoke fns

4:56 either you wrap the defstruct macro and create an xml, or you use java reflection to create it

4:56 I'd rather use reflection

5:03 notsonerdysunny: just out of curiosity .. I heard clojure can actually infer types .. is it possible for us access this infered type?

5:05 hoeck: notsonerdysunny: i.e. http://gist.github.com/569861

5:07 notsonerdysunny: ... hoek: thanks .. I guess this is waht I was looking for ..

5:07 *what

5:27 leiningen always creates the same project file .. can I make it create a different default project.clj? like for instance always add swank-clojure to its dev-dependencies ...

5:32 _na_ka_na_: hey guys how do I extract the keys of defstruct?

5:34 AWizzArd: _na_ka_na_: when you have an instance, try (key your-struct)

5:34 keys (not key)

5:34 _na_ka_na_: suppose i have (defstruct e :a :b) . .want something like (struct-keys e) => (:a :b)

5:34 from the definition, not an instance

5:36 AWizzArd: _na_ka_na_: with a nil-instance you can do: (keys (struct e))

5:38 _na_ka_na_: AWizzArd: nice idea, but doesn't this deserve a built in function?

5:38 I'll be calling this a lot of times so I'll have to form a closure over the nil-instance

5:39 AWizzArd: I won't object. However, in my opinion this needs no built-in, as this is not very often called and runs very efficient.

5:39 _na_ka_na_: I can memoize it .. hmm

5:39 AWizzArd: Yes. But then again: why would one want to call it multiple times?

5:40 And (time (dotimes [i 1000000] (keys (struct e)))) was done for me within 110 msecs ==> 110 nanoseconds per call.

5:41 _na_ka_na_: I'm using a struct instance and persisting it to my table in database ... I blindly assume that the struct instance has exactly the same keys as columns of table ..

5:41 that's why i need the keys

5:42 AWizzArd: (def get-struct-keys (comp keys struct))

5:43 _na_ka_na_: cant do that, what if the struct has more keys than the defstruct ?

5:43 I want to restrict the struct to exactly the keys of defstruct

5:43 struct is coming from an outside source

5:45 AWizzArd: get-struct-keys will create a fresh struct, with all struct values set to nil.

5:46 So you will get a seq of exactly the keys of defstruct, not of a specific instance.

5:46 fliebel: morning

5:47 _na_ka_na_: oh ok got it now

5:48 fliebel: Why do functions in Clojure/core often have multiple bodies, up to 3 arguments before they resort to using & args?

5:49 LauJensen: fliebel: Thinking about any function specifically?

5:49 fliebel: http://github.com/clojure/clojure/blob/b578c69d7480f621841ebcafdfa98e33fcb765f6/src/clj/clojure/core.clj#L1936

5:49 comp

5:50 LauJensen: I think for comp as well as a few others, its a performance consideration

5:50 fliebel: It does this in a nested fashion, where it deals with one function with one arg, two funtcion siwht on arg, two functions with 2 rgs, etc...

5:51 Chousuke: it's a performance optimisation

5:51 fliebel: So using a b c is faster than using & args?

5:52 Chousuke: yeah

5:53 fliebel: I feel guilty now, for planning to use comp in a non-speedy way :)

5:54 LauJensen: ,(time (dotimes [_ 1e6] (+ (+ 1 2) (+ 3 4))))

5:54 clojurebot: "Elapsed time: 371.225 msecs"

5:54 LauJensen: ,(time (dotimes [_ 1e6] (apply + [1 2 3 4])))

5:54 clojurebot: "Elapsed time: 1730.082 msecs"

5:54 fliebel: (what is 1e6?)

5:55 Chousuke: 1 and 0 zeroes :)

5:55 er

5:55 6 zeroes

5:55 fliebel: I see

5:55 But why is the apply option so slow?

5:56 Chousuke: fliebel: well in this case it's cheating a bit because + for two args gets inlined

5:56 so it's not that apply is slow, it's that + when inlined is fast :P

5:56 it basically compiles down to a simple method call.

5:56 LauJensen: + online inlines when it has 2 args

5:57 fliebel: Chousuke: So how is that related to comp?

5:57 LauJensen: ,(time (dotimes [_ 1e6] (+ 1 2 3 4)))

5:57 clojurebot: "Elapsed time: 1160.466 msecs"

5:57 Chousuke: but apply still needs to break down the list of args so that it can call the rest arg version etc.

5:57 so it needs to do more work compared to a plain function call

5:58 fliebel: okay, but isn't that compensated by all the bodies to choose from?

5:59 Chousuke: and compared to fixed arity rest args are slower because a list of the args must be built at runtime.

6:00 fliebel: okay

6:00 Chousuke: so in terms of speed inline > fixed arity > rest args > apply

6:03 LauJensen: > ruby

6:05 zmila: ,(type 1e6)

6:05 clojurebot: java.lang.Double

6:05 LauJensen: ,1e6

6:05 clojurebot: 1000000.0

6:10 fliebel: LauJensen: So that's 7 zeroes then ;)

6:11 LauJensen: :)

6:13 fliebel: Okay… there is assoc, assoc-in, update-in, but where is update?

6:14 LauJensen: just use update-in

6:15 fliebel: could do that yea…

6:15 LauJensen: (defn update [m k f] (update-in m [k] f))

6:15 there you go :)

6:16 fliebel: I expected to see the oposite in the source, update-in utilizing update.

6:24 LauJensen: man you're a hard guy to satisfy, gimme a sec I'll rewrite core...

6:24 fliebel: haha, don't bother I'll use update-in :)

6:25 LauJensen: :)

6:25 fliebel: But yea, there are times Clojure makes me want to rewrite core ;)

6:26 LauJensen: Yea me too, Update it a little, rename the project to SLEDGE HAMMER! and release it as my own :)

6:28 fliebel: Add OO back in… :P I mean, I don't miss the objects and inheritance to much, but in Python I can do the equivalent of (doc str) and see all the methods I can use to modify it.

6:29 LauJensen: OO? You mean that broken model for modeling real world entities, which only old farts still use?

6:29 Guess I could try to dig up some old irellevant papers on it :)

6:30 fliebel: LauJensen: Nah, I only mean the helpful propertie that I can lookup methods I can use to fumble with the object in question, rather than dig through the whole api to look for something that might be of use.

6:31 LauJensen: http://clojure.org/data_structures

6:31 did you read that fliebel ?

6:31 fliebel: LauJensen: Sure, a dozen times or so :)

6:31 LauJensen: There are not many datastructures and the interfaces are very similar, it doesn't take too long to get familiar with

6:32 fliebel: I know… everything in Clojure makes sense in some way, but sometimes I just want to see all things I can de to a string, or a seq, or whatever.

6:33 ouch… forgot about that one… "Can't take value of a macro"

6:37 Any better way to do this? (apply (apply comp list-of-fns) args))

6:38 That is… take a seq of fns and thread the seq of args through all of them.

6:39 LauJensen: its a little like

6:39 ,(map (juxt inc dec) [1 2 3])

6:39 clojurebot: ([2 0] [3 1] [4 2])

6:39 LauJensen: Or did I misunderstand your intention ?

6:42 fliebel: LauJensen: I think so… If you entered [+ inc] for the fns and [1 1] for the args, it should return 3

6:43 LauJensen: oh

6:43 zmila: like apply-in :)

6:43 fliebel: zmila: Another new function for me :) I'll look it up

6:44 zmila: just invented the name, no such function, i think

6:44 fliebel: nope :(

6:44 LauJensen: If anybody liked this, feel free to upvote :) http://news.ycombinator.com/item?id=1671490

6:45 fliebel: zmila: It's more like apply->

6:47 #clojure, the new vote-ring for everything Clojure :) *reads blogpost*

6:51 LauJensen: Nice stuff :)

6:51 LauJensen: thanks

6:52 fliebel: LauJensen: I'm currently doing a lot of thinking and planning with defn to write our own :)

6:52 LauJensen: If you could code at the speed of thought, you'd be done by now :)

6:52 I just learned that Planet Clojure is written in Python :(((((((( Awww the agony of defeat :(

6:53 fliebel: LauJensen: Yea, just like Apple retail stores running their administration on Windows(true story)

6:54 LauJensen: really? That seems weird considering how Arch > Osx > A cardboardbox > Windows

6:55 fliebel: LauJensen: Replace the box with Wine for gaming :P

6:55 LauJensen: gaming... hrmf :)

7:00 fliebel: My new kick-ass plugin system: http://gist.github.com/569958

7:02 LauJensen: ,(reduce (comp inc +) [1 1])

7:02 clojurebot: 3

7:03 fliebel: LauJensen: Let me think about that one...

7:05 LauJensen: Why do you use reduce instead of apply?

7:05 LauJensen: (defmacro comp-> [fns args] `(reduce (comp ~@fns) ~args))

7:05 its a little dirty, but it saves you a nested apply

7:05 (comp-> [inc +] [1 1]) => 3

7:07 fliebel: There was a discussion on reduce/apply a while back, where Christophe pursuaded me that you should always prefer reduce to apply, except when calling str or concat.

7:08 fliebel: LauJensen: Why?

7:10 LauJensen: Also, they don't work the same way, for my solution (comp-> [+ inc] [1 1 1]) => 4 for yours it's 5

7:11 This is what mine comes down to (defmacro comp-> [fns args] `((comp ~@fns) ~@args))

7:11 LauJensen: ah right

7:18 fliebel: LauJensen: Is the macro any better than using apply?

7:19 LauJensen: No

7:20 fliebel: LauJensen: Would you mind explaining why reduce is better than apply in most cases?

7:21 Chousuke: is it? hm

7:21 I thought apply was better.

7:21 LauJensen: I'll try to relay what Christophe said. reduce makes more sense to the reader because the contract is so clear. (reduce f a) means that some kind of linear accumulation is going on, even if you don't know f.

7:22 Chousuke: at least, apply str ... is better than reduce str ... :)

7:22 LauJensen: concat and str are the exceptions, because of their memory/stack use

7:23 fliebel: LauJensen: So I would do (reduce + [1 2 3 4 5]) rather than (apply + [1 2 3 4 5])?

7:23 LauJensen: yea

7:24 fliebel: … still don't get it… apply is one operation, while reduce will do 4.

7:24 Chousuke: fliebel: that apply does reduce internally anyway

7:24 or + does

7:24 LauJensen: ~source apply

7:25 Chousuke: I suppose reduce can be faster in this case because it can use chunks to work with the vector

7:25 which might not be possible with apply. I don't know.

7:26 LauJensen: ,(time (dotimes [_ 1e6] (apply + [1 2 3 4 5])))

7:26 clojurebot: "Elapsed time: 1891.233 msecs"

7:26 LauJensen: ,(time (dotimes [_ 1e6] (reduce + [1 2 3 4 5])))

7:26 clojurebot: "Elapsed time: 1443.119 msecs"

7:26 Chousuke: and there is a new reduce protocol that allows reduce to have type-optimised implementations making it even faster.

7:26 AWizzArd: Apply can not be written in the same level as Clojure code runs

7:26 It requires a higher level, for some "magic"

7:26 Chousuke: but that might've been disabled for 1.2?

7:33 LauJensen: Chousuke: Doesn't look like it made it in

7:43 mrBliss: LauJensen: why were you using Windows in your screencast? I thought you were a serious Windows hater.

7:43 LauJensen: mrBliss: I wanted to try Camtasia

7:44 mrBliss: LauJensen: okay, nice screencast (audio quality dropped a bit towards the end)

7:45 LauJensen: yea it did - I couldn't figure out why so I decided to launch anyway. Should probably get a professional mic

7:45 and thanks

7:46 AWizzArd: Did Rich mention when the number branch will flow into Master?

7:46 LauJensen: didn't that happen already ?

7:46 AWizzArd: "number branch" as in: unboxed ints and doubles as default.

7:46 I don't know :)

7:46 LauJensen: he merged equiv a while ago, I think that included everything

7:49 hmmm

7:49 Audience snapshot: Based on internet averages, bestinclass.dk is visited more frequently by males who are in the age range 25-34, have no children, have no college education and browse this site from home.

7:52 AWizzArd: btw, where can I find clojure-contrib.jar at http://build.clojure.org/ ?

7:53 octe: can i call the superclass from a proxy-class' method?

7:53 mrBliss: http://build.clojure.org/job/clojure-contrib/lastBuild/org.clojure.contrib$complete/

7:53 AWizzArd: octe: (doc proxy)

7:54 octe: Note that while method fns can be provided to override protected methods, they have no other access to protected members, nor to super, as these capabilities cannot be proxied.

7:54 that means no?

7:54 or only in the context of an overriden protected method?

7:54 AWizzArd: mrBliss: oh thanks. How did you get there? I mean, what is the click-path to that page?

7:54 octe: "[...] args - a (possibly empty) vector of arguments to the superclass constructor [...]"

7:55 This is the only way I think.

7:55 octe: to the constructor ,yes

7:55 mrBliss: AWizzArd: clojure-contrib -> last build -> search for ...contrib:complete... -> voila

7:55 AWizzArd: mrBliss: oh okay, though I don't find this specifically erognomic to use.

7:56 It was very easy when I just had to click on clojure-contrib and found a direct link to the .jar

7:56 mrBliss: this is because of the contrib overhaul (split into modules)

7:57 AWizzArd: So that people can decide to include only more granular pieces of Contrib into their own .jar files?

7:58 mrBliss: I think so

8:00 raek: AWizzArd: http://build.clojure.org/releases/ or http://build.clojure.org/snapshots/

8:03 AWizzArd: raek: is there also a click path to go there, or is a direct link required?

8:03 raek: I haven't found any click path for those... :(

8:04 AWizzArd: k

8:12 hoeck: octe: proxy-super

8:15 chouser: but beware, proxy-super is not thread-safe

8:16 AWizzArd: How can one make a fn static?

8:18 chouser: rub it in your hair?

8:19 LauJensen: haha

8:23 AWizzArd: Rich mentioned something like :static to do this.

8:23 (defn ^{:static} foo [arg] ...)

8:24 (doc defn) doesn't tell

8:24 clojurebot: "([name doc-string? attr-map? [params*] body] [name doc-string? attr-map? ([params*] body) + attr-map?]); Same as (def name (fn [params* ] exprs*)) or (def name (fn ([params* ] exprs*)+)) with any doc-string or attrs added to the var metadata"

8:24 raek: I think the syntax is ^:static or ^{:static true}

8:24 it's still experimental, I think

8:24 AWizzArd: static, in what sense?

8:25 LauJensen: static in the case means primitive args and returns, cannot be redefined

8:25 AWizzArd: I want to type-hint args as primitives

8:27 chouser: AWizzArd: Either ^:static or ^{:static true}

8:30 LauJensen: As I recall (defn ^:static myfn [^int x] (+ (int 5) x)) should be a static fn, taking an int and returning one as well

8:39 chouser: "Only long and double primitives are supported"

8:40 but (defn ^:static myfn [^long x] (+ (int 5) x)) does seem to work.

8:40 gives you a class with an invokeStatic method that takes a primitive.

8:48 LauJensen: ok, well, long and double is enough

8:48 dnolen_: (member:defn ^:static myfn ^long [^long x] (+ 5) x)), takes long and returns long, no need to hint 5 in master branch of 1.3

8:48 LauJensen: I just vaguely recall seeing Rich do an example with ints

8:48 I might be wrong

8:48 dnolen_: erg, (defn not (member:defn

8:50 LauJensen: cool

9:05 AWizzArd: LauJensen: long and double is enough: yes, as long one can still do (int my-long) to make it an int again.

9:26 octe: i've written a clojure function to parse a line.. but it's very ugly: http://paste.lisp.org/display/114347 how would i write this in an idiomatic way?

9:31 chouser: an example of the kinds of lines it should parse would help me read the code you've got.

9:32 octe: sure.

9:32 http://paste.lisp.org/display/114348

9:33 rodgertq: on the topic of parsing, and lazy sequences...

9:33 fbru02: hey guys , i was experimenting with this http://pastie.org/1145780 and i want the output that it is mentioned there, can anybody help??

9:34 rodgertq: I am running into some out of mem exceptions when parsing through a large file (~95MB)

9:34 the file format is basically Clojure forms, 1 per line

9:34 chouser: octe: I think I'd recommend regex for something like this.

9:35 rodgertq: I use line-seq to read, read-string to parse each line, and then reduce by a map of specific IDs I'm looking for

9:35 octe: chouser, ah, a list of regexps would work

9:35 chouser: octe: even just one or two should do it, I'd think

9:36 rodgertq: there are 35 IDs in this case is, the whole thing should be lazy, and I would expect the vast majority of the large file's data to just be discarded and GC'd but this doesn't seem to be happening

9:41 sample of the parse function and the input are here http://gist.github.com/570138

9:44 raek: octe: are you making an IRC client/bot too?

9:45 chouser: octe: http://paste.lisp.org/display/114348#1

9:45 octe: not sure that's exactly right

9:45 LauJensen: rodgertq: If you slurp 95 MB of text into strings, you're at least putting 190Mb into memory, because of the way Java Strings are implemented.

9:46 rodgertq: but shouldn't line-seq read that lazily?

9:46 (as opposed to slurp)

9:46 raek: octe: http://gist.github.com/570143

9:47 I use the following regexen:

9:48 (def message-regex #"^(?::([^ ]+) +)?([^ ]+)(?: +(.+))?$")

9:48 (def param-regex #"(?:(?<!:)[^ :][^ ]*|(?<=:).*)")

9:49 chouser: fbru02: your input is confusing.

9:49 '("Clojure "ds " "Clojurism"")

9:49 ,'("Clojure "ds " "Clojurism"")

9:49 clojurebot: ("Clojure " ds " " Clojurism "")

9:49 chouser: ,(count '("Clojure "ds " "Clojurism""))

9:49 clojurebot: 5

9:49 chouser: fbru02: do you really mean to have a list of 5 things, including two symbols and an empty string?

9:50 fbru02: chouser: sorry i didn't mean that , I only meant "Clojure" "randomword" "Clojurism"

9:51 LauJensen: rodgertq: yea its lazy, but that does change much if you're holding the head

9:51 raek: octe: feel free to use whatever you like from it

9:51 chouser: fbru02: so a list of three strings?

9:51 fbru02: chouser: yes

9:52 chouser: fbru02: what you had seems to work.

9:52 ,(map (comp first #(re-seq #"Cloju" %)) ["Clojure" "randomword" "Clojurism"])

9:52 clojurebot: ("Cloju" nil "Cloju")

9:53 fbru02: ah let me check your slight modifications

9:53 chouser: nice ! it is idiomatic to do it this way?

9:53 clojurebot: amespaces are (more or less, Chouser) java packages. they look like foo.bar; and corresponde to a directory foo/ containg a file bar.clj in your classpath. the namespace declaration in bar.clj would like like (ns foo.bar). Do not try to use single segment namespaces. a single segment namespace is a namespace without a period in it

9:53 chouser: fbru02: sure, nothing wrong with that.

9:53 fbru02: chouser: thanks ! :)

9:54 rodgertq: LauJensen: ok, could be an impedance mismatch in my understanding of how lazy sequences are implemented in clojure, vs. F# (which I'm more familiar with)

9:54 chouser: could also say (map #(first (re-seq #"Cloju" %)) ...)

9:54 LauJensen: rodgertq: I don't know much about F# except that nobody were able to implement my transient news reader successfully in F#

9:54 chouser: fbru02: or: (map #(re-find #"Cloju" %) ...)

9:55 fbru02: chouser: re-find sounds good (checking on clojuredocs.org) :)

9:56 chouser: rodgertq: what version of Clojure are you using?

9:57 rodgertq: chouser: 1.2

9:58 chouser: rodgertq: nothing in read-log jumps out at me as holding the seq head, but since it gets passed to to-forms, grok-items, and add-meta, any of those could accidentally retain the head as well

9:58 rodgertq: let me update the gist to include those

10:00 actually I think it's to-forms that's doing it

10:01 http://gist.github.com/570138

10:01 to-forms is expecting a collection of lines, returning a collection of forms

10:02 lenw: LauJensen: I had a look through the bestinclass code and it has helped me a lot - just wondering if there is an easier way to decode incoming params from a web post ?

10:02 LauJensen: lenw: you mean than read-response?

10:03 lenw: LauJensen: was looking in the comments.clj at the receive-comment func

10:03 LauJensen: sec

10:03 lenw: ta

10:04 LauJensen: lenw: Well, easier or not I dont know, but thats a fairly naive way to do it and there's a lot of stuff that should be added to make it general purpose. What I would do if I were you was add a helper to read the body, and have that account for zero params, missing params, duplicate params etc and just re-use that

10:05 I don't use that version on any of my proprietary applications. And I think that there's actually already a helper in Ring which does the job, if you can find it, use it :)

10:05 lenw: LauJensen: thanks for the advice as always - off to look around

10:05 LauJensen: np :)

10:05 rodgertq: hmmm

10:06 nope

10:07 chouser: rodgertq: to-forms doesn't call eval, does it?

10:07 rodgertq: chouser: changed to-forms to take the reader directly and reduce it's result, get the same OOM exception, updated gist http://gist.github.com/570138

10:07 read-string

10:07 which likely does call eval

10:08 chouser: no, I think it doesn't.

10:08 rodgertq: ok

10:08 chouser: which is good. eval is more than you need. creates classloader instances I think.

10:08 rodgertq: k

10:09 chouser: read-string should be safe as far as memory consumption is concerned, as should your various map and reduce calls.

10:10 rodgertq: so in the repl, calling all of this I have:

10:10 (def lc (indexed (read-log "latecancels.20100826.log")))

10:11 this is the smaller list of things I want to pluck out related data in the larger logs from

10:11 then for the larger file

10:11 (def o (read-log "orderstat.20100826.log" (filter-by-id lc)))

10:12 filter-by-id just checks to see if the supplied collection contains the id from larger file and if so, conj's it to the result

10:13 chouser: rodgertq: you might try skipping some of the intermediate steps

10:14 rodgertq: try a simple filter (maybe with a regex) on the line-seq and conj that into your set -- make sure the set of strings at least fits in memory.

10:15 rodgertq: could try that, though in a couple of cases I have more than one UUID in the input, so context matters

10:16 chouser: yeah, it'd just be a rough test

10:16 rodgertq: ah, right, if it has it, parse it, then procee

10:16 d

10:17 chouser: well, I was thinking just try to reduce the amount of code involved as much as possible and see if you're still running out of memory.

10:18 if that fixes it, then you know something in the code you removed is what's breaking things.

10:18 lenw: LauJensen: ring middleware wrap-params looks good

10:21 octe: raek, yes

10:22 chouser, thanks for your input :)

10:24 LauJensen: lenw: good to know

10:35 anonymouse89: any vimclojure users know how to easily put the last line entered on the repl onto the current line?

10:37 LauJensen: anonymouse89: IIRC its ':q! emacs M-x slime M-p'

10:43 sleepynate: LauJensen: huh.. i get -- zsh: command not found: emacs

10:44 LauJensen: sleepynate: thats too bad man, thats cuz you a looser! :)

10:44 anonymouse89: LauJensen: right..

10:45 sleepynate: perhaps you meant to join ##flamewar instead of #clojure ? :)

10:47 anonymouse89: but to answer your question.. no, i can never get gorilla working right anyways :)

10:47 anonymouse89: i just tslime

10:47 anonymouse89: sleepynate: atm I'm starting a repl using ConqueTerm

10:48 sleepynate: anonymouse89: ahh, that works too

10:48 anonymouse89: and then just gorilla for stuff in files to eval

10:48 sleepynate: i keep a vim open in tmux and use ^B:split-window

10:49 this way also i can have many things pipe to the same repl if i wish

10:50 anonymouse89: so if you're editing a file and want to eval a line how does that get piped to that repl?

10:50 sleepynate: visual select, C-c C-c

10:51 anonymouse89: oh, then just switch over and paste

10:51 sleepynate: nah

10:51 LauJensen: hehe

10:51 ok I wont say a word :)

10:51 sleepynate: anonymouse89: uno momento.. i show

10:58 anonymouse89: http://imgur.com/a/JQn8q/workflow_for_vimclj

10:58 first frame, i have repl open next to vim simple euler problem

10:59 repl is running in tmux (and you see, there are other repls for python/php etc.. i use this technique in many languages

10:59 anyways, then V13j to select the 13 lines

11:00 then hit C-c C-c to pipe out to repl

11:01 it's not as "nice" as nailgun et al, but it is simple and consistent :)

11:01 Bronsa: sleepynate: what's that wm?

11:01 sleepynate: Bronsa: xmonad

11:01 Bronsa: blah

11:01 haskell

11:01 sleepynate: Bronsa: ;)

11:01 Bronsa: never tried stumpwm?

11:01 100% pure common lisp tiled wm

11:02 sleepynate: yea, i have

11:02 i like xmonad :)

11:02 Bronsa: i use stumpwm by 1 year

11:02 never tried xmonad

11:02 chouser: Bronsa: does it have tabs in each tile frame?

11:03 sleepynate: all in all, they're not that vastly different.. but i came to clojure from haskell after a long-time-ago far-far-away scheme past :)

11:03 Bronsa: what do you mean with "tabs"

11:03 chouser: my wm is going away in my distro, so I'm shopping for a replacement.

11:03 anonymouse89: chouser: going away?

11:03 sleepynate: Bronsa: so i've been using haskell much longer than any CL variant :D

11:03 anonymouse89: sleepynate: thanks for letting me into you workflow btw

11:03 chouser: Bronsa: multiple application windows in the same frame, with little tab buttons along the top top switch between them.

11:04 Bronsa: chouser: nope

11:04 alpheus: How can I add a doc string to a defrecord form?

11:04 sleepynate: anonymouse89: np. the most important part to me is not getting lost no matter which machine i'm on

11:04 Bronsa: but you can show/hide application in the same frame

11:04 chouser: anonymouse89: ion3. written by a mentally unstable person which has caused all manner of difficulties for distros. ubuntu's dropping it. can't blame 'em.

11:05 anonymouse89: chouser: I'm using wmii which has stacking. this is somewhat like what you're talking about with tabs

11:05 Bronsa: http://img842.imageshack.us/img842/1092/screenshotnt.png

11:05 anonymouse89: chouser: you can divide into columns and then have stacks for each column

11:07 ordnungswidrig: Bronsa: that are some colours!

11:07 sleepynate: Bronsa: i suppose to big thing is... if you preen the same .conf for years... it's hard to switch over :D

11:07 Chousuke: chouser: which is too bad, since ion3 is excellent

11:08 Bronsa: ordnungswidrig: yeah

11:08 Chousuke: chouser: but have you tried awesome? it's good too.

11:08 sleepynate: Bronsa: this is actually new to me, having xmonad *inside* gnome. but i am required to use ubuntu if i'm going to use linux, and ubuntu doesn't like you to use a vanilla WM :/

11:09 Bronsa: so

11:09 chouser: Chousuke: some friends tried awesome and came back to ion. ...though I forget why, now.

11:09 Bronsa: you actualy use xmonad as the gnome's default wm?

11:10 LauJensen: chouser: Why do you call Tuomo mentally unstable?

11:10 opqdonut: ion3 rocks

11:10 and tuomov has disappeared

11:11 there are some people trying to pick up maintainership

11:11 raek: alpheus: since a defrecord from only implement methods of already existent protocols/interfaces, the docstrings usually go on the protocols/interfaces

11:11 chouser: LauJensen: seems self evident to me, based on his writings. I suppose others could reach different conclusions.

11:11 LauJensen: chouser: In my oppinion, I think you should be careful labeling people as such

11:12 opqdonut: I think he's merely opinionated

11:12 chouser: LauJensen: fair point. I'd like to amend my statement: ion3 was written by someone that's very difficult to work with.

11:13 opqdonut: and doesn't want to or can't communicate civilly

11:15 sleepynate: and paranoid

11:15 don't forget that part :)

11:15 raek: alpheus: (defprotocol Source "A thing that produces items" (take! [source] "Removes one item from the source and returns it.")) (defrecord LineSource [buffered-reader] Source (take! [_] (.readLine buffered-reader)))

11:17 alpheus: raek: I'm still trying to get defprotocol. I think it's more object-oriented than I need (for my present purpose). I'm essentially using defrecord for something a map could do. The records are something to be returned from functions, or collected in sequences.

11:17 sleepynate: chouser: are you familiar with notion?

11:18 chouser: sleepynate: aware of it.

11:18 sleepynate: it looks like it's both new and not very active

11:20 raek: alpheus: simple is good, so if ordinary maps would be sufficient, I'd recommend that

11:21 if you need polymorphism based on the type of the first argument, use records

11:21 or multimethods

11:22 alpheus: That makes sense.

11:23 raek: multimethods are great for doing dispatch based on one (or multiple) of the keys of a map

11:25 alpheus: Looking again at my copy of Practical Clojure, I see that I mis-read the guideline about preferring defrecord to defstruct. It didn't say anything about maps.

11:25 raek: if you find yourself adding a type key just to be able to do polymorhpism, maybe records are a better fit. if the thing to dispatch on is a part of the existing value, multimethods might be better

11:25 structs are one kind of maps

11:26 they just store some of the keys differently

11:26 but both structs and hash maps implement IPersistentMap (which records also do, btw)

11:27 chouser: there's very little reason to use structs anymore

11:27 raek: ...or if performance is an issue, go with records

11:29 Bahman: Hi all!

11:33 hugod: Is there a way to establish a thread local value for a var? or do I need something like http://gist.github.com/570249

11:34 opqdonut: binding

11:34 is thread-local

11:35 hugod: I need a thread local value that is an implementation detail - I don't want the caller to have to bind a value

11:40 and I don't want to create the binding on each invocation

11:40 arohner: hugod: can't you bind inside the function definition?

11:42 hugod: arohner: I would rather not - potentially expensive initialisation

11:42 arohner: hugod: I don't understand enough about the problem to help you

11:44 hugod: arohner: I'm probably worrying for nothing and should just bind inside the function.

11:44 thanks

11:46 s/bind/construct/

12:15 joshua-choi: Hey, everyone, can a macro capture at all what file and at what line it's called at, whenever it's called?

12:16 danlarkin: I think that is part of &env?

12:16 slyrus: hey joshua-choi, anything new with fnparse?

12:16 joshua-choi: I've just started working on it for the first time in a long time now.

12:16 slyrus: did you get my email re: contexts and/or check out my SMILES parsing code that makes heavy use of them?

12:16 cool

12:17 A common-lisp port would be nice too :)

12:17 joshua-choi: Yesterday I carefully rewrote the core so that rules are not functions with metadata anymore, since that's apparently subject to removal.

12:17 slyrus: Hmm, no I didn't—well, when did you send it?

12:17 slyrus: the fnparse/clojure SMILES parser is MUCH nicer than my hand-did CL parser

12:18 oh, a while back -- a github message, IIRC

12:18 joshua-choi: Oh, I think I did reply to that

12:18 slyrus: there wasn't too much there, I was just saying that I make heavy use of contexts after you suggested avoiding them

12:18 joshua-choi: danlarkin: Thanks. I'll check it out

12:18 slyrus: Ah, yes, I remember

12:18 I still think you should probably use defmaker instead

12:18 Oh well

12:19 slyrus: (The thing is that I think that contexts can make problems harder to determine. defmaker forces you to isolate whatever contextual information you're passing, which is better I think.)

12:20 slyrus: ok, I'll take another look

12:20 the code in questions is here: http://github.com/slyrus/chemiclj/blob/master/src/chemiclj/smiles/read.clj

12:21 if you free up any time to take a look at what someone with too little knowledge of fnparse whips up at first glance :)

12:21 joshua-choi: I need to improve the examples

12:21 It's quite impressive, I think

12:21 slyrus: I see the only defmaker I use is something I ripped off from your examples :)

12:22 fnparse? I agree!

12:22 joshua-choi: Oh, no, your code

12:22 slyrus: heh. thanks!

12:22 joshua-choi: Yeah, I do need to improve the examples, especially clojure.clj, to make use of defmaker

12:22 Ugh, I don't wanna write that YAML parser

12:22 chouser: macros can the the file and line number. they aren't in &env

12:23 joshua-choi: chouser: Can't?

12:23 chouser: they can. I've done it.

12:23 gimme a sec

12:23 joshua-choi: Oh good

12:23 This will make rules able to say what line they're at when they make an error.

12:24 hiredman: you can pull a line number off &form, yes?

12:24 chouser: (:line (meta &form)) should give you the line number

12:24 and *file* should be the filename

12:25 slyrus: joshua-choi: I'd like blog about my fnparse experiences writing chemiclj. might contribute to the global tutorial on things to do (and to not do) to effectively use fnparse

12:26 joshua-choi: hiredman and chouser: Wonderful

12:26 chouser: I've got a potential use case for fnparse coming up. Haven't looked at it before.

12:26 joshua-choi: slyrus: That'd be great; I think the API is effectively frozen now

12:26 slyrus: chouser: I highly recommend it!

12:27 joshua-choi: chouser: Currently, tutorials are nonexistent, examples are poor, but doc strings are okay

12:27 chouser: I'm afraid I have to learn lein first.

12:27 joshua-choi: What may be confusing is that it is actually split in two, with differences

12:27 You don't need lein.

12:27 Actually, doc-strings are poor too. I'm about to revise them.

12:27 * slyrus resists the urge to fork a lein that removes all of the maven crap and just uses the checkouts directory for classpath dependencies

12:28 joshua-choi: First, though, I'm going to try to make rules able to say what line they're at.

12:28 chouser: I've a week or two out from digging into fnparse

12:28 joshua-choi: Before I revise documentation

12:29 chouser: I'm about to push a new tagged version today; use that when you do

12:29 chouser: ok, great.

12:29 joshua-choi: Contact me if you run into problems or questions

12:29 chouser: thanks, I will!

12:32 I'm having trouble finding where in Compiler.java code is emitted to call static methods

12:32 ought to be a generated call to the fn object's invokeStatic, right? But I'm not seeing it.

12:32 joshua-choi: slyrus: I'm going to try to finish FnParse before I look at your example. :(

12:32 But contexts are fine

12:32 It's just that I prefer defmaker

12:32 If it weren't for you, though, I would have removed them

12:33 rhickey: chouser: not sure I understand, not StaticMethodExpr?

12:33 lpetit: Hello, does this look like a corrected bug or still a bug, for you : "The class name the JRE appears to be complaining about is "compile__stub/net/cgrand/parsley/lr-plus/TableState".  The "lr-plus" part seems wrong.  I don't think it's legal to have a hyphen in a class name."

12:33 It happens with IBM V6.0 JRE for a ccw user. Other users using sun jvms (including me) have no problem

12:35 TableState is a protocol

12:36 rhickey, chouser: ^^^ maybe you know, 'cause I don't have an IBM JRE for testing, and the test done by the user was with a late 1.2 RC, but not the final release

12:37 slyrus: joshua-choi: like I said, if there's a better way, I'm all ears, but for the moment that was the only way I could make sense of tracking the global-ish state without doing it by hand w/ clojure's mutable data types

12:37 I'm looking into defmaker right now

12:37 joshua-choi: slyrus: That's precisely why it's not so good in my opinion: contexts make it global-ish.

12:39 slyrus: well, if you've got global-ish state in the thing you're trying to parse, what other options are there?

12:39 chouser: rhickey: sorry, I meant :static functions

12:39 joshua-choi: slyrus: Passing them as arguments into user-defined rule-makers.

12:40 Maybe there isn't much of a difference...? I'm not so sure now.

12:40 Hmm

12:40 slyrus: yeah, that's just what I was thinking :)

12:41 perhaps rather than a global context a special variable that goes partway up the stack might be useful

12:41 s/goes/is bound/

12:41 sexpbot: <slyrus> perhaps rather than a global context a special variable that is bound partway up the stack might be useful

12:41 lpetit: must leave now, will try solve my issue by other means :'-(

12:42 rhickey: chouser: see InvokeExpr.parse, will generate a StaticInvokeExpr

12:42 joshua-choi: slyrus: I'm not sure that I understand, but I'm suspicious of the idea anyway. :) In any case, contexts aren't *that* bad...at least it's part of the state argument, rather than being a mutable global variable.

12:43 Ah yes. defmaker forces you, however, to more explicitly express *where* you're using that contextual data.

12:43 slyrus: yes, and if it's part of the state argument, how is that so different from defmaker?

12:43 oh, I see

12:43 joshua-choi: It forces you to state everywhere it's needed

12:44 The bad thing about that is just that—it may be tedious, especially if you're adding something to an already written parser

12:44 But I think that it is better, net-wise

12:44 That way, where it's used is not so hidden, a disadvantage associated with big data structure arguments and imperative assignment.

12:45 It is a trade-off.

12:45 slyrus: the idea of the special variables (to use the CL term, are the called the same thing here? it's whatever happens when you (bind [...]) things would be similar but you would avoid the tedium of passing the stuff as args

12:46 I notice I have a bunch of defns that take context as the first arg. probably good candidates for defmaker

12:46 joshua-choi: Precisely. I have bind, by the way.

12:46 I mean, I hate bind.

12:47 It has given me bad trouble before. :(

12:47 I have to go for a bit!

12:47 slyrus: ok, nice talking to you!

12:49 and in CL bind is just let, but with extra magic for special variables. it can certainly easily be overused.

13:00 chouser: rhickey: ah, perfect. thanks.

13:17 joshua-choi: slyrus: Are you there? I want to ask a question.

13:27 slyrus: yes

13:27 I sent you a github message re a simple defmaker example

13:28 joshua-choi: I got your message by the way. I don't think that's not a good place for a rule-maker.

13:28 Either keep it as a regular function or change it to a rule. There is never any point to having a rule-maker with no arguments.

13:28 slyrus: well, yes, that's the next question :)

13:29 joshua-choi: But I think I've already decided the answer to my own question. It's if I should convert every standard rule maker into a macro...so that they can determine at what line they were called in.

13:29 I think the answer is no.

13:29 In order for a rule to be able to give an error that says at what line the rule was defined at, the rule maker that made that rule must be a macro...

13:30 slyrus: oh, right. ok, I'll keep plugging away at my defn-'s then

13:30 joshua-choi: But there's that adage saying that macros are a pain and should be used if you really really need them

13:31 I would like to say, though, that(h/defrule bond-symbol-order

13:31 (h/hook #(get {\- 1 \= 2 \# 3 \$ 4} %)))

13:31 is okay.

13:32 slyrus: I think I started with a bad example. That was deep down inside some defn-'s. I need to go back up to a function that I call directly from a rule and figure out how to replace that with a defmaker

13:33 process-ring might be a good example if you're looking at my code

13:33 right now I do (h/alter-context process-ring ring-num bond-symbol) ... and all sorts of stateful magic happens based on what's in the context

13:34 at a minimum, I could remove the explicit passing/setting of the context by using a defmaker for that

13:34 but I guess I'm not quite sure when in a defmaker what the difference between the args and the subsequent args to, say, an enclosed h/hook are

13:37 and I suppose my defmaker forms should look like (defmaker <foo> ....) to remind me that it's just a (special (well, not in _that_ sense)) rule?

13:42 chouser: joshua-choi: could you examine the fn metadata to get a line number in cases where it wasn't created by a macro?

13:44 joshua-choi: chouser: If I understand you correctly, then no, since what I want are the line numbers where the function was *called*, rather than where it itself was defined.

13:44 I think only macros can do that, with (meta &form).

13:45 slyrus: Don't use angle brackets to delineate rule-maker symbols. I don't know if we should adopt some other sigil/etc...but I don't think angle brackets themselves are a good idea.

13:45 Use 'em for rules, rather than rule-makers.

13:49 * slyrus still needs to figure out what exactly a rule-maker is doing that's different than a rule

13:49 joshua-choi: slyrus: A rule-maker is a function (or macro?) that returns a rule.

13:49 lit is a rule-maker. So is hook.

13:50 slyrus: how is a rule-maker different than a rule with a hook?

13:50 I think I need to work through some examples to figure this stuff out

13:50 joshua-choi: I need to go! But a rule-maker is any function/macro that returns a rule, nothing more.

13:50 slyrus: ok, ttyl

14:18 AWizzArd: rhickey: https://visualvm.dev.java.net/relnotes.html#changes

14:18 “Recognizing Clojure, Groovy, JRuby, Jython and Scala runtimes”

14:19 rainerschuster: uh, nice

14:19 headius: what on earth does that mean?

14:20 rainerschuster: profiling?

14:20 detecting performance bottlenecks by running variant code path

14:20 headius: seems unlikely; unless they've done a lot of work, profiling jruby with java tools would miss interpreter

14:21 pretty sure jruby's the only one in that list with an interpreter, though

14:21 rainerschuster: huh? maybe they're into the ASTs ..

14:21 headius: yeah, beats me

14:21 nobody contacted me about it though

14:21 rainerschuster: don't know. is it possible to hook into the interpreters

14:22 headius: there's no standard API for hooking language-specific calls, and only a JRuby-specific API for capturing "call" and "return" events across interpreter and compiler

14:22 it's certainly possible they're using that, I guess

14:23 rainerschuster: i'm a .net guy, don't know how the JVM works

14:23 AWizzArd: I don’t know what exactly this means, but it sounds like they have improved this profiler for Clojure, at least to some extent.

14:23 headius: perhaps they're unmangling some compiled names

14:24 all supposition!

14:24 AWizzArd: We will see it soon. I guess in the next jdk they will include version 1.3+

14:24 rainerschuster: monitor of performance and memory consuptions is quite low level, nothing related to the interpreters

14:24 headius: could also just be support for writing visualvm plugins in those languages

14:24 chouser: "VisualVM now recognizes the Clojure, Groovy, JRuby, Jython and Scala runtimes, enabling these developers to easily find the appropriate processes."

14:25 rainerschuster: plugin architecture

14:25 chouser: http://profiler.netbeans.org/blog/images/visualvm_13_altjvmlang.png

14:25 rainerschuster: maybe one prof per int

14:25 headius: chouser: ahh, so it's recognizing from command line that it's one of those lanuages

14:25 or via some other mechanism

14:25 chouser: I think it just can tell when a particular vm pid has clojure inside it.

14:26 headius: seems simple enough

14:26 chouser: oh, or perhaps from the command-line, though that sounds a bit fragile

14:26 of course a vm can have jruby *and* clojure libs in it -- which icon to use then? :-)

14:26 headius: jruby does register some mbeans, so they could use that...but who knows how they're doing it right now

14:26 yeah, exactly

14:27 some sort of lambdaruby icon

14:27 chouser: ooh, lambdaruby!

14:27 hugod: doesn't jvm have support for different "stratum"

14:28 AWizzArd: Interesting that they noticed Clojure.

14:28 hugod: jdwp that is

14:28 chouser: interesting that they noticed anything at all besides java.

14:28 AWizzArd: but not Scheme implementations or ABCL

14:29 chouser: hm

14:30 cemerick: chouser: hey, just toss a badge on the icon per-"runtime"! :-P

14:30 headius: cemerick: with an alpha mask

14:30 cemerick: headius: and little animated gif wiggles

14:30 headius: ooo

14:31 rainerschuster: some folks here from the clojure-clr branch?

14:31 chouser: we know jruby is bigger and heavier, so should have a little clojure badge in the corner.

14:31 * chouser hides from headius behind cemerick

14:32 headius: :)

14:32 cemerick: sha-ZAM! :-D

14:32 headius: if it takes bigger and heavier to run on android unmodified...so be it

14:32 chouser: nice

14:32 headius: though...that's kind of self-defeating too

14:33 I'm glad we've kept the interpreter, in any case

14:33 chouser: I just learned of team that ported a bunch of ruby code to C++ for performance.

14:33 I heard about it too late. jruby would have been perfect

14:33 or at least better than C++

14:38 headius: that's for sure

14:38 I'm working on some improvements in jruby 1.6 that should put it closer to java for general execution perf

14:39 boxed numbers are a hard problem, unfortunately (for clojure as well)

14:39 danlarkin: hiredman: ^

14:41 dnolen: headius: have you look at the new static fns feature in 1.3, a reasonable work around given the situation.

14:42 ninjudd: cemerick: just read your hosted REPL proposal. good stuff.

14:43 cemerick: ninjudd: Thanks; let me know if you have any ideas/feedback/whatever.

14:43 ninjudd: cemerick: i've been having thoughts of writing something similar to clean up the protocol used by cake to communicate with the persistent jvm

14:43 cemerick: I'm hoping to knock it out this week, at least in prototype fashion.

14:44 ninjudd: cemerick: it seems to me like what you're proposing is more than just a REPL protocol, right? you talk about it being for non-interactive use too

14:44 cemerick: ninjudd: Nifty. By all means, make sure I'm making it so that you can use it.

14:44 Yeah, I'm going past the typical interactive usage model.

14:45 Necessarily so, really. The requirements of tooling demand it IMO.

14:45 headius: dnolen: static fns, no

14:45 sounds like a dodge

14:45 how does it work?

14:46 ninjudd: cemerick: right, i need that too to be able to send cake commands to the jvm

14:47 joshua-choi: Has anyone looked into aliasing protocols? E.g. (clojure.contrib.def/defalias P another.ns/P)? Does it completely work?

14:50 ninjudd: cemerick: one thing i would need to be able to use it for cake would be a way for threads to report back incremental output as they run. for this reason, i was planning for each socket to represent a single thread and open multiple sockets or use clojure's concurrency constructs if i needed to do things asynchronously. do you think if would be too hard to do incremental output with a message based protocol?

14:51 cemerick: this is also something people expect with an interactive REPL.

14:51 headius: dnolen: found a post on static fns

14:52 my hands are somewhat tied as to what I can add to ruby, but I've considered extensions to allow optional static typing

14:52 I'm just happy I've gotten fully-boxed math (like in fib) to exceed clojure perf

14:54 dnolen: headius: well with static fns, Clojure perf on fib is identical to Java.

14:55 headius: because it's basically Java then :)

14:55 it's a little surprising how many "long" declarations are needed, looking at this example: http://clj-me.cgrand.net/2010/06/10/primitive-types-support-for-fns-coming-to-a-clojure-branch-near-you/

14:56 cemerick: ninjudd: that's interesting. It's not quite what you want/get in an interactive setting, insofar as you still want the information coming back to be packaged as discrete messages (so you can track and associate them with client-side processes, etc).

14:56 dnolen: headius: it's a bit more clever then that, static fns can still used higher order. things still need to be shaken out, but will cool when usable with vectors of primitives (map, reduce, filter)

14:56 headius: that level of type hinting is unnecessary now in the 1.3 branch

14:57 headius: well on literals anyway.

14:57 headius: yeah, I'd assume literals wouldn't need hints

14:57 most anything else I'd expect to need it though

14:58 clojure has a much simpler dispatch path as well, so even with static type hints in jruby we still need to dynamically bind most things

14:58 ninjudd: cemerick: would it simplify the protocol to just make it synchronous and use multiple connections if you need to send multiple commands in parallel?

14:58 headius: with no type hints at all I can get fib(38) down to around 3.5s, which I think is pretty good for fully-boxed

14:59 cemerick: ninjudd: well, that would be the only way to do it given what I've sketched out so far. I'll have to think about what you've suggested though.

15:00 headius: dnolen: I assume the collections support for primitives will mean shipping specialized versions of them

15:00 int vector, long vector, etc

15:01 ninjudd: cemerick: you could open one connection for the interactive repl, and one for the tooling (using future and such to run commands asynchronously)

15:01 cemerick: right

15:02 but it's possible that asynchronous, multiple reply messages will end up being a simple (optional) extension

15:02 chouser: headius: the different primitive types for collections are generated via macros

15:02 headius: but the ultimate result is specialized collections

15:02 chouser: yes

15:03 no

15:04 they're all clojure.core.Vec, but it uses different "ArrayManager" classes internally

15:04 valure are still boxed going in and out

15:04 ninjudd: cemerick: sure. this is kind of along the same lines of what Laurent was talking about with telnet...

15:04 chouser: values

15:05 headius: chouser: that's what I figured

15:06 cemerick: ninjudd: not quite -- he was hoping to avoid any concrete protocol entirely in that scenario, which makes things pretty difficult. Multiple asynchronous returns is far, far simpler on first blush.

15:11 ninjudd: cemerick: right, there would still need to be some sort of protocol for the synchronous mode, so you know what is coming back over the socket (stdout, stderr, return value, exception) and so you can send interrupts and stdin out from the client

15:11 cemerick: I'm not certain that there *will* be a synchronous mode.

15:12 A given client can make an async connection appear synchronous to the user at the keyboard.

15:13 ninjudd: that's why i used the conditional "would"

15:13 cemerick: ah, sure

15:14 ninjudd: now that i think about it, once you implement the synchronous protocol, it isn't much of a jump to make it work asynchronously

15:15 cemerick: the protocol itself has little to do with sync vs. async.

15:15 Insofar as you can base the former on the latter from a client perspective, I'll be focusing on the async.

15:15 chouser: wise

15:16 cemerick: Assuming I can get the backend working nicely, the biggest challenge will be to get it to talk swank (or to get SLIME to talk whatever-this-REPL-thingy-will-be-called).

15:17 chouser: emacs is your first client target?

15:17 cemerick: oh, no

15:17 ccw, enclojure, hopefully vimclojure, and perhaps cake

15:17 * cemerick nudges ninjudd ;-)

15:17 chouser: ok

15:18 cemerick: emacs is tied to swank for the foreseeable future, but I've no desire to be in the same position.

15:31 ninjudd: what is the accepted way to throw exceptions in clojure code? should I use gen-class to create a custom exception class, or just throw java.lang.Exception?

15:31 hiredman: clojurebot: exceptions?

15:31 clojurebot: http://paste.lisp.org/display/74305

15:31 cemerick: IllegalArgument, IllegalState, and IOException will bring you a long ways.

15:32 hiredman: ^-- list of exceptions that already exist

15:32 cemerick: eh, most of those shouldn't exist to begin with

15:32 chouser: ninjudd: also consider clojure.contrib.condition

15:33 hiredman: sun.jvm.hotspot.debugger.win32.coff.COFFException

15:33 no one ever expects that one

15:33 cemerick: (throw (COFFException. "blah")) ;; this actually means we can't connect to the server ;-)

15:34 chouser: heh

15:34 technomancy: es.com.sun.inquisitions.SpanishInquisition

15:34 ninjudd: hehe. never heard of c.c.condition. looks interesting

15:36 raek: it uses gen-class under the hood to make a special exception holding a map

15:37 and exception handlers are selected by the result of a function applied to the map

15:37 much like multimethods...

15:39 ninjudd: so does c.c.condition supersede c.c.except?

15:43 cemerick: chouser: do you have any interest in #clojure logs from before 2/1/08? I have just a few.

15:44 heh, almost all joins and parteds though :-)

15:52 raek: ninjudd: have you read this? (link from docs): http://groups.google.com/group/clojure/browse_frm/thread/da1285c538f22bb5

15:55 arohner: what was the name of the clojure webapp channel?

15:56 kencausey: arohner: #clojure-web?

15:56 arohner: kencausey: thanks

15:57 kencausey: arohner: You might want to read /msg alis help for future reference ;)

15:59 ninjudd: raek: just did.

16:00 raek: don't think there is much more info on that manner than that... :)

16:19 rhickey: I'd like shorter names for unchecked-*, so optimized code isn't so ugly

16:20 a first cut would be inc! and dec! etc, where ! indicates danger

16:21 something similarly easy to append would be fine too

16:21 sleepynate: what about "uck"

16:21 you know you wanna uck-dec

16:22 lpetit: Hello all

16:22 mrBliss: Add * to the end? Also replace multiply, add etc with * + /.

16:23 lpetit: I confirm there's a problem with the compilation (either live or AOT) of e.g. types, if the namespace containing the type definition cointains dashes.

16:24 It's quite easy to reproduce: create src/net/yournick/lr_plus.clj , and in this file, have just (ns net.yournick.lr-plus) (deftype Foo)

16:24 AOT compile this , and shebang! in your classes/ directory you see the following folder structure: classes/net/yournick/lr-plus/

16:25 lr-plus/ contains the Foo.class file

16:26 the problem is that while apparently with Oracle JVMs everything works fine while you don't try to AOT compile it, it does not work (even if not AOT'ed) with IBM JRE 6:

16:26 Caused by: java.lang.ClassFormatError: JVMCFRE068 class name is invalid; class=compile__stub/net/cgrand/parsley/lr-plus/TableState, offset=0 (lr_plus.clj:8)

16:27 The net result is that all users of ccw which use an IBM JVM simply cannot use it :-/

16:28 Could someone try to reproduce this on his machine ?

16:28 Or do you think I can file an issue directly ?

16:30 rhickey: lpetit: please file an issue, thanks

16:30 lpetit: ok

16:34 rhickey: do I leave all the defaults values ("Assigned to" blank, Milestone Backlog, Priority Normal, Component empty, Approval empty) ?

16:35 rhickey: yes

16:37 lpetit: ok done, ticket #432

16:37 rhickey: lpetit: thanks

16:37 lpetit: np, going to bed now, cu

16:37 rhickey: how about a global toggle for overflow checking?

16:38 arohner: rhickey: I'd prefer something more local, like a function or binding

16:39 cemerick: rhickey: I know you've not wanted to go down this road in the past, but separate namespaces seem to make a lot of sense here.

16:41 It'd even be a perfect example of wholesome use usage ;-) (:use 'clojure.unchecked)

16:43 chouser: cemerick: sure, might be fun.

16:44 cemerick: chouser: They're colloquy logs, which are XML :-P

16:44 chouser: cemerick: re: old logs, that is. take the spotlight off my newbie questions a bit. just email whatever format you've got.

16:46 clojure-log site is already based on two different formats.

16:48 cemerick: ninjudd: FYI, I've added your REPL question to the design notes, with a slight response.

16:49 Would you like write privs so you can add more ideas/issues as you think of them?

16:49 ninjudd: cemerick: sure. thanks

16:50 cemerick: ninjudd: just need your google docs email addy; /msg it if you like

17:59 mvid: is there a simple way to turn a sequence of strings into one large comma delimited string?

18:04 slyrus: ,(apply str (interpose "," ["this" "is" "bogus"]))

18:04 clojurebot: "this,is,bogus"

18:05 arohner: ,(clojure.string/join "," ["this" "is" "bogus"])

18:05 clojurebot: "this,is,bogus"

18:12 rainerschuster: someone interessted in helping translate the contrib libs to .NET?

18:14 slyrus: I'm sure there's a java library that does just that accessible by maven. just add it to your project.clj, run lein deps, restart your jvm, call the StringConcatenatorGenerator, loop through the sequence calling addStringToBeConcatenated, download the InterposerGenerator framework, connect that to the concatenator and then ask the StringConcatenatorRenderer to do the hard work for you. These...

18:14 ...java frameworks are awesome!

18:15 rainerschuster: huh?

18:15 is that related to my question?

18:15 slyrus: no

18:15 rainerschuster: puh

18:16 :)

18:16 slyrus: string concatenation (poor joke, I guess)

18:16 rainerschuster: translation to .NET could be seen as string concatenation

18:16 slyrus: heh

18:29 technomancy: dang; I was hoping they'd drop -master- from the version string in 1.3 =\

18:34 alexyk: ninjudd: ping

18:34 drewr: technomancy: inc

18:34 alexyk: lancepantz: ping

18:35 lancepantz: sup alexyk?

18:35 alexyk: lancepantz: so, finally with cake, how do I do a new proto, given I have a myclass.proto file in a proto/ subdirectory?

18:36 lancepantz: alexyk: few steps

18:36 add protobuf as a dev dep

18:37 you probably want it as a regular dep as well

18:37 alexyk: why both?

18:38 lancepantz: it has to be a dev dep to be able to use the task in the cake jvm, but it won't be put in an uberjar then

18:38 you may not need it if you're just using it to compile the java class

18:39 an then you'll also want to add :tasks [protobuf.tasks] to your project.clj

18:39 then cake proto should work

18:43 alpheus: As a lisp1, clojure doesn't need flet or labels? Do I just use let for a local function?

18:43 tomoj: also letfn

18:44 alexyk: lancepantz: where will it take protobuf.tasks, and how will it know which subdir has .proto files?

18:44 tomoj: I think there is some fancy stuff you can do with flet/labels that you can't with let/letfn though

18:45 lancepantz: alexyk: it's hard coded to look for protos in project_root/proto

18:45 i'm not sure what you meant by the first part of your q though

18:46 alexyk: lancepantz: are the protobuf.tasks in a file in the distro?

18:46 lancepantz: it's a namespace in clojure-protobuf

18:47 alexyk: wow

18:47 nice

18:47 so you cook jars with their own cake tasks!

18:48 lancepantz: ninjudd's effort at plugins, i think its pretty damn clean

18:50 AWizzArd: tomoj: could you please explain what fancy stuff you were thinking about, in respect to flet/labels?

18:55 alpheus: you can use let, or as mentioned by tomoj letfn.

18:58 tomoj: AWizzArd: I don't remember exactly, I think maybe it had to do with mutual recursion?

18:59 there was something about clobbering too

18:59 AWizzArd: tomoj: mutual recursion is the difference between CLs labels vs flet

19:00 tomoj: I dunno CL so the whole discussion was somewhat mysterious to me

19:01 AWizzArd: http://www.lispworks.com/documentation/HyperSpec/Body/s_flet_.htm

19:16 alexyk: AWizzArd: how's your in memory database project?

19:22 zoldar: hello, I'm having problems with utf-8 characters in slime, when I send string from emacs to repl (for example "jaźń") it hangs - after reconnect prompt is available again. however in clean repl everything is working fine. I'm using leinigen (lein repl, lein swank etc.). Here's my .emacs contents: http://paste.lisp.org/display/114366 .

19:22 AWizzArd: I did not have time in the past 16 days. But hopefully better times are ahead. Anyway, it is pretty close for going public.

19:30 hiredman: zoldar: slime has an encoding setting and so does the jvm, you need to get everything to agree on utf-8

19:31 zoldar: i ran "export JAVA_TOOL_OPTIONS="-Dfile.encoding=UTF8"" before running swank

19:34 hiredman: how are you starting swank?

19:34 zoldar: just lein swank

19:34 hiredman: you may need to set -Dswank.encoding to match the swank encoding from emacs

19:34 or slime encoding

19:35 http://github.com/technomancy/swank-clojure/blob/master/src/swank/swank.clj#L66

19:36 zoldar: swank.ecoding worked, thanks

19:49 tomoj: does sexpbot work in PM?

19:49 for stuff other than clojure eval I mean

19:56 wtetzner: is there a reader macro for 'verbatim' string literals?

19:57 for example, you could write "\n" instead of "\\n"?

19:58 hiredman: no

19:59 wtetzner: are there any plans to add it, or was it discussed already and deemed a bad idea?

19:59 phobbs: Hello everyone

20:00 is there a macro for automatically memoizing recursive functions?

20:00 chouser: wtetzner: I'm supposed to be coming up with a proposal for that.

20:00 wtetzner: oh, cool

20:00 phobbs: if not, I wrote one

20:03 hiredman: phobbs: why do you need a macro?

20:03 phobbs: hiredman: how else would you do it?

20:03 wtetzner: defn-memo

20:03 phobbs: That didn't work for me

20:03 it didn't memoize the recursive calls

20:04 is it supposed to?

20:04 hiredman: phobbs: it depends on how the function is recursively referenced

20:05 wtetzner: ah

20:05 defn-memo is implemented in an odd way

20:05 it just calls defn, and then calls alter-var-root

20:06 hiredman: if the function is created via defn, and calls itsself by name, then (defn foo [...] ...) (def foo (memoize foo)) would be enough

20:06 wtetzner: still seems odd that it wouldn't work

20:06 hiredman: if you recurse using recur then that won't work

20:06 phobbs: hiredman: that doesn't work

20:06 hiredman: phobbs: prove it

20:07 phobbs: I just did

20:07 my laptop can barely do (fib 30) that way

20:07 and chokes on (fib 50)

20:07 pathetic

20:07 it's obviously not working

20:07 with my macro I can do (fib 2000) in 37ms

20:08 hiredman: phobbs: prove it to me

20:09 phobbs: do you want me to give you code that calculates the Fibonacci function?

20:09 really?

20:10 tomoj: of course it chokes on (fib 50)

20:10 phobbs: if it's memoizing correctly, it shouldn't

20:10 wtetzner: (def fib (lazy-cat [0 1] (map + fib (rest fib))))

20:10 tomoj: "calls itsself by name"

20:11 phobbs: wtetzner: I know how to do that

20:11 sometimes that isn't convenient for larger functions

20:11 tomoj: does the one that can do (fib 2000) with your macro call itself by name?

20:11 hiredman: phobbs: how else will I know what kind of code we are talking about?

20:11 phobbs: (defn fib [n]

20:11 (if (<= n 1) 1

20:11 (+ (fib (dec n))

20:11 (fib (- n 2)))) )

20:12 sorry, should have pastebinned

20:12 but you get the idea

20:12 it would work if memoized

20:12 tomoj: I guess so

20:12 phobbs: if not, it would choke

20:13 with that code, (def fib (memoize fib)) doesn't help at all

20:13 and neither does replacing defn with defn-memo

20:13 octe: what's the idiomatic way of doing something like "while ((line = reader.readLine()) != null) { .... }" in clojure?

20:14 phobbs: octe: you could use a (loop [line (readline)] ...)

20:15 octe: with a test that would quit once line is nil

20:15 and recurring with (read-line)

20:15 hiredman: its because (defn fib []) expands into something like (def fib (fn fib []))

20:15 _ato: phobbs: fib 50 looks fine with for me, fib 2000 I'd need to increase the stack size: http://gist.github.com/571111

20:15 hiredman: so you lose the indirection through the var

20:17 phobbs: _ato: oh, I was doing the memoizing after defining the function

20:18 _ato: hmm... I get a stackoverflow error too, even with 1000. So maybe writing this macro wasn't a total waste of time

20:18 wtetzner: phobbs: how does the macro work?

20:19 phobbs: wtetzner: I'll post the code

20:19 wtetzner: k

20:20 phobbs: wtetzner: it basically searches for recursive calls (not recur, unfortunately...), and replaces those with memo-table checks

20:20 traversing the tree with zip

20:24 http://gist.github.com/571117

20:24 I couldn't figure out how to allow docstrings

20:24 that's not a huge deal right now though... I just wanted to use it for some exercises for school

20:25 instead of writing repetitive code

20:25 btw my clojure style is really horrible, I'm pretty new to the language

20:25 hiredman: what about (defn foo [] (let [foo inc] (foo 1))) ?

20:26 or (defn foo [] [(foo) (foo)])

20:26 phobbs: it would break

20:26 it's definitely not a finished product

20:26 hiredman: symbol manipulation is tricky

20:32 phobbs: wtetzner: it appears to use less than 1/10th the stack size

20:32 so that's something

20:36 wtetzner: phobbs: cool

20:37 phobbs: one thing I've learned with doing clojure is that a lot of self-recursive functions can be rewritten as self-recursive lazy sequences

20:37 phobbs: for example, http://pastebin.com/QaEibGCw

20:37 phobbs: just something to keep in mind when writing clojure code

20:39 phobbs: whoa, condp is awesome

20:40 wtetzner: haha, yeah

20:40 it's like a really powerful switch

20:40 phobbs: ^_^

20:40 thanks

20:41 btw, what does cons produce?

20:41 a list?

20:41 wtetzner: although, now that i think about it, (or (#{0 1} n) (+ val1 val2)) might be better

20:41 cons returns a sequence

20:42 phobbs: I know... but under the hood

20:42 wtetzner: try it

20:42 ,(type (cons 'x '(1 2 3)))

20:42 clojurebot: clojure.lang.Cons

20:42 phobbs: ,(class (cons 'a '(1 2 3 4 5 6 7 8)))

20:42 clojurebot: clojure.lang.Cons

20:42 phobbs: yeah

20:42 what is that?

20:42 wtetzner: i think it's just a class that implements seq

20:43 hiredman: ,(ancestors (class (cons 'a '(1 2))))

20:43 clojurebot: #{java.util.List clojure.lang.Sequential clojure.lang.IMeta clojure.lang.Seqable java.io.Serializable java.util.Collection java.lang.Object clojure.lang.Obj clojure.lang.ISeq clojure.lang.IObj :clojure.contrib.generic/any clojure.lang.IPersistentCollection clojure.lang.ASeq java.lang.Iterable}

20:43 phobbs: hiredman: thanks

20:44 _ato: phobbs: http://en.wikipedia.org/wiki/Cons

20:45 phobbs: heh, yeah I know

20:45 thought clojure might use something fancy like a vector

20:45 anyway, gtg

20:45 thanks everyone

20:49 _ato: phobbs: nah, Cons is just a simple class with two fields, pretty similar to every other lisp except perhaps that the cdr can be any sequence not just more cons cells: http://github.com/richhickey/clojure/blob/master/src/jvm/clojure/lang/Cons.java

20:50 hiredman: _ato: in most lisps cons cells can contain any two things, not something and a sequence

20:51 _ato: hiredman: ah yes, you

20:51 're right, my mistake

20:51 dotted pairs etc

21:16 alexyk: lancepantz: how do I tell cake proto to use existing google protobuf, installed?

21:16 ninjudd: ^^

21:17 ninjudd: it should use it if protoc is in your path

21:17 but it still needs to download the code to create the jar

21:22 alexyk: ninjudd: ah, I migrated the mac and protobuf was lost. It downloaded something, were building, asked for password, then died

21:22 ninjudd: should pick up where it left off if you run again

21:23 alexyk: ninjudd: java.lang.Exception: unable to find proto/clojure/protobuf/collections.proto on classpath

21:23 that's second time

21:23 ninjudd: renamed to extensions

21:23 alexyk: but it comes from cake proto!

21:23 ninjudd: s/collections/extensions/

21:23 alexyk: ah, I have it in my proto

21:24 ninjudd: your proto file likely has collections in it

21:24 alexyk: yep

21:24 ninjudd: did you get to sorted map yet? :)

21:24 yay, proto works

21:25 ninjudd: no, but i'll accept a patch ;-)

21:25 alexyk: a-mazin'

21:25 I need to finish that PhD first... cake proto rocks!

21:25 ninjudd: i did update sets though so you can do delete-on-append

21:26 alexyk: delete-on-append?

21:26 deleting from a buffer?

21:27 ninjudd: normally, there is no way to delete an element from a repeated field by appending to the protobuf

21:27 but i changed the set implementation so that you can

21:28 you can pass in a map of items to booleans and it will convert to a set, deleted the items that have false values

22:18 tomoj: cemerick: excellent work

22:21 quizme: (map #(apply % '(3 5) ) '(+ -) ) <--- why doesn't this return (8 -2) ?

22:24 oh...

22:24 ^^; hehe

22:25 (map #(apply % '(3 5)) [+ -])

22:32 tomoj: haha

22:32 that is confusing

22:33 ,((juxt + -) 3 5) ; relevant to your interests?

22:33 clojurebot: [8 -2]

22:40 duncan_bayne: Hi all, quick newbie question: following the instructions on http://weavejester.github.com/compojure/docs/getting-started.html but I'm getting "Wrong number of arguments to repl task" when I try to run the app. Am I using outdated documentation?

22:44 Ooh, that might do it: running Clojure 1.0.0, but project.clj asserts Clojure 1.2.0. I'll go fix that as a first step ...

22:49 scottj: oh wow I can't believe I didn't know v in slime debugger would highlight the relevant form

Logging service provided by n01se.net