#clojure log - Jul 09 2008

The Joy of Clojure
Main Clojure site
Google Group
IRC
List of all logged dates

0:50 Lau_of_DK: Top' of the morning Gents

0:55 hoeck: Lau_of_DK: good morning

1:32 Lau_of_DK: Gents, lets say I have an entire xml file loaded in zip/xml-zip format (trees with nested vectors I think) - How do I output this to an XML file in the easiet most fool-proof way ?

2:17 Gents, lets say I have an entire xml file loaded in zip/xml-zip format (trees with nested vectors I think) - How do I output this to an XML file in the easiet most fool-proof way ?

2:47 hoeck: Lau_of_DK: (binding [*out* (new java.io.OutputStreamWriter (new java.io.FileOutputStream "my.xml"))] (xml/emit x))

2:51 Lau_of_DK: hoeck? Have you tried that approach? xml/emit here generates garbage output with data distortion and awful formatting

2:55 hoeck: yes, the formatting is bad, i have tested it on a very simple xml file

3:15 Lau_of_DK: i forgot the (.close *out*)

3:17 lisppaste8: perspectivet pasted "loop-i-ness" at http://paste.lisp.org/display/63443

3:17 perspectivet: so I've been playing around with swt and clojure

3:19 I've been having some problems with the some of the basic functionality and I wondered if anyone had any ideas why my translation (pasted above) isn't working

3:21 (defn show-file ....

3:22 is supposed to be roughly a translation of the java snippet commented out above it

3:37 * hoeck is downloading swt

3:41 perspectivet: ah, boom

3:41 so if I do it simplistically it works

3:42 get rid of the doto's

3:42 just go up to a (. shell (open)) I get the expected result

3:43 oops

3:55 ktne: hello

3:55 does clojure have pattern matching?

3:57 perspectivet: in the haskell "case" sense of pattern matching?

3:57 ktne: i'm not familiar with haskell just with ocaml

3:57 match based on type and parameters

3:58 can you define for example multiple function bodies, one for each case?

3:58 *define

3:59 hoeck: ktne: no, it hasn't, it has only overloading on arity

3:59 ktne: ah

3:59 what about simple pattern matching, like a switch statement

3:59 does it have that?

4:01 hoeck: ktne: clojure has a `cond' clause, where one can check for several expressions

4:02 ktne: can you do something like this pseudocode: switch(shape) { case Rectangle(x1,y1,x2,y2): ; case Circle(x,y,radius): ; }?

4:04 hoeck: of course: (cond (rectangle? shape) (let [..] ) (circle? shape) ( ... ))

4:04 but clojure has multimethods too

4:06 with multimethods you are free to define you own dispatch functions

4:06 ktne: i'm reading about multimethods right now

4:07 hoeck: good :)

4:15 perspectivet: hoeck: found the problem I think

4:16 I'm a little unclear on let scope affects deallocation but I'm thinking that when the display goes out of the let scope it probably gets deallocated

4:17 hoeck: perspectivet: i tried your code, but nothing appears on my screen

4:17 ktne: how do i exit clojure repl? :)

4:17 perspectivet: Ctrl-d

4:17 ktne: thanks

4:17 perspectivet: since if I move display outside of defn I can at least get a window

4:17 hoeck: should i paste the new version?

4:18 hoeck: perspectivet: please

4:19 perspectivet: i always read that one has to it own deallocation in swt

4:20 perspectivet: one has to do one's own deallocation?

4:21 lisppaste8: perspectivet pasted "less-loop-i-ness" at http://paste.lisp.org/display/63446

4:22 hoeck: oh, sorry, i mean the deallocation of the swt gui objects like buttons and so on

4:22 perspectivet: I'll have to read up on the top level containers like Display to see if that holds for them too

4:23 child objects of display are fine as long as display is not in a let

4:24 Lau_of_DK: hoeck: Even with (. close *out*) it still mangles the XML, like "Title" becomes " Title " with excessive whitespace and linebreaks

4:24 hoeck: perspectivet: i see, do you have prior experience with swt or are you trying it the first time?

4:25 perspectivet: 1st time for both clojure and swt

4:25 how hard can it be? ;)

4:26 hoeck: ^ ^

4:26 perspectivet: as in see above?

4:32 hoeck: Lau_of_DK: xml/emit looks very simple, e.g. it always inserts the `encoding=UTF-8' header as a plain string.

4:36 meredydd: Lau_of_DK: Are you using (println) and friends? They insert spaces.

4:36 Eg: (println "I have" 1 "banana") --> "I have 1 banana"

4:36 despite the fact that I didn't put spaces in there.

4:37 whereas (str "I have" 1 "banana") --> "I have1banana"

4:39 hoeck: meredydd: emit uses (println (str ... ))

4:40 meredydd: Weird...

4:52 lisppaste8: perspectivet pasted "working-swt-snippet" at http://paste.lisp.org/display/63448

4:53 hoeck: perspectivet: your (second) sample works here!

4:53 perspectivet: it works somewhat, you still need the loop to be able to edit the text box

4:53 ktne: anyone here uses clojure with emacs?

4:53 hoeck: perspectivet: *joy*

4:53 ktne: i'm trying to use the clojure mode

4:53 perspectivet: ktne: I do

4:53 ktne: but it doesn't work

4:53 i've loaded clojure-mode.el

4:53 hoeck: ktne: me too

4:54 ktne: but the menu items are not bound to anything

4:54 perspectivet: menu items, what are those? ;)

4:54 no menus on my emacs...

4:55 ktne: hmm

4:55 it looks like i had to rename my clojure script to lisp

4:55 clojure-mode calls lisp command

4:55 it works now :)

5:07 lisppaste8: perspectivet pasted "more useful text swt snippet" at http://paste.lisp.org/display/63449

5:07 perspectivet: this one does multi-line

5:26 Lau_of_DK: meredydd Im using (str xxx) for passing new children when compiling the modified tree, and then I emit that tree in a (with-out-str) nest

5:32 hoeck: perspectivet: no window on my machine, something weird with scope is going on there

5:40 meredydd: Hey, this is a good one

5:40 java.lang.VerifyError: (class: clojure/fns/verdi_router/handle_connection__27590, method: invoke signature: (Ljava/lang/Object;)Ljava/lang/Object;) Inconsistent stack height 0 != 1

5:40 java.lang.VerifyError: (class: clojure/fns/verdi_router/handle_connection__27590, method: invoke signature: (Ljava/lang/Object;)Ljava/lang/Object;) Inconsistent stack height 0 != 1

5:40 I think I may have stumbled upon a compiler bug.

8:35 Chouser: meredydd: example?

11:37 ktne: hello

11:37 Chouser: ktne: hi

11:37 ktne: anyone using the enclojure plugin for netbeans here?

11:38 hi Chouser

11:38 Chouser: there's an #enclojure channel

11:38 ktne: ah

11:38 Chouser: I've dabbled with enclojure, but probably not enough to answer any questions.

11:38 ktne: thanks

11:54 la_mer: ktne: I'm using it daily at this point

12:23 ktne: anyone here who can tell me how to create an enclojure project?

12:23 i already have a source file

15:19 rhickey: hi all

15:19 Chouser: rhickey: how's it going?

15:19 rhickey: did you speak today?

15:19 rhickey: Good, except the connectivity at ECOOP is not that great, so I've been offline a lot

15:19 I spoke Monday and yesterday - both went well

15:20 wanted to thank every for carrying the ball here and on the group in my absence

15:20 nice to see this community growing :)

15:21 Lau_of_DK: Hey rhickey, have you been out prepping new screencasts?

15:21 rhickey: I got one of the two, on the other the screencasting sw didn't capture

15:22 but I'll probably hold off on that as I'm going to do a similar talk in Sept to the Boston Lisp Group

15:22 cemerick: There's a variety of new people here and there -- I personally know two developers that are using Clojure on small projects.

15:23 rhickey: cemerick: I'm definitely going to follow up on the gen-class load time issue when I get back

15:23 abrooks: rhickey: Oh, cool. When and where for the Boston talk?

15:23 Lau_of_DK: rhickey: ok sounds great

15:23 rhickey: Boston Sept 29th at MIT

15:23 abrooks: rhickey: I assume that's open to the public?

15:23 Lau_of_DK: Uuuuh MIT

15:23 The pressure is on

15:23 cemerick: rhickey: you mean the "recursive" classname reference bit? That'd be great.

15:23 rhickey: abrooks: yes, probably pre-reg so you get dinner

15:24 cemerick: That would allow me to cut out a bunch of little workarounds in our build/test process.

15:24 Chouser: seems like we have a pretty deep bench, even if none of us have particularly intimate knowledge of things. A few people around to answer some questions, just about any time apparently.

15:24 abrooks: rhickey: Good to know, thanks!

15:24 rhickey: cemerick: that and the perf issue

15:24 Chouser: yes, that's been fun to see

15:25 Having seen some presentations here I can tell you all that in using Clojure you are on the forefront of the multi-core oriented future

15:25 cemerick: rhickey: Excellent. I'd love to hear what you think is going on with the initialization performance -- I'd worried that I was on a wild goose chase, there.

15:25 * rhickey battery running low, I'm likely to fade out soon...

15:28 Lau_of_DK: Chouser: Did you see my posts about that xml/emit bug earlier today?

15:29 Chouser: Lau_of_DK: I did. I think you're right -- xml/parse and xml/emit both play a bit fast and loose with XML standards regarding whitespace

15:29 Lau_of_DK: oh okay, i'll just write a custom emitter then

15:30 Chouser: I don't how much code out there relies on current behavior, but I'll look at adding some options for my lazy-xml stuff to be a bit more strict.

15:30 Lau_of_DK: Just wanted to make sure, that it wasn't because I was using all 10 thumbs when writing the code

15:30 Chouser: I think if you just take a copy of emit but without any \n, you'll be ok.

15:31 You'll have lost any whitespace on the parse side, but at least you won't include incorrect whitespace during emit.

15:31 Lau_of_DK: That would one step better than what I've got now. Internet Explorer reads the xml file just fine, but xml/parse doesnt even recognize it as xml

15:32 Chouser: huh.

15:33 Lau_of_DK: It doubles on \n in the output file, and "Title" becomes " Title "

15:54 perspectivet: so I've translated an swt snippet to clojure, but the loop section is kind of gross

15:54 any ideas on how to clean it up?

15:54 http://paste.lisp.org/display/63449

15:56 erochester: you could use (do ...) instead of (let [] ...)

15:56 perspectivet: ah, right

15:56 just read about do actually

15:57 Chouser: (when-not ...) instead of (if (not ...))

15:58 you can drop a set of parens on each method call, either (.sleep d) or (. d sleep) for example

15:58 perspectivet: here let me paste the newest working version

15:58 Chouser: same for doto as well, I think: (doto s pack open)

15:59 perspectivet: yeah, you're right. don't know why I did it the way i did

16:03 lisppaste8: perspectivet pasted "cleaned-up-swt-translation" at http://paste.lisp.org/display/63468

16:04 perspectivet: much cleaner

16:06 Chouser: I'm sure there would be some torturous way to replace loop with take-while, map, etc... but I really don't think it would be clearer or better for what is an inherently non-functional API.

16:07 perspectivet: yeah, my thinking also

16:11 slava: when writing guis in a functional style you want to use something like the builder pattern...

16:12 instead of passing the parent widget to the child's constructor, have the parent constructor take &rest args

16:13 and replace setters with a map that you pass in

16:13 cemerick: We're biting the bullet and just using Netbeans/Java for our UI. It's decidedly "non-core", so we're going to do it wrong from the start. ;-)

16:14 Nafai: slava: Sounds nice.

16:15 slava: something shoes would work well in clojure i think: http://code.whytheluckystiff.net/shoes/

16:15 especially since most of the hard work is done for you by swing

16:15 Nafai: Swing is painful at times :)

16:16 slava: or swt or whatever the flvaor of the month java gui is :)

16:16 * slava doesn't keep track

16:16 Nafai: That's my day job right now, developing a Swing component

16:16 slava: msut be a big component if that's your entire job :)

16:17 Nafai: It's a fancy grid / table component

16:18 perspectivet: isn't swing performance still kind of lacking these days? at least compared to swt?

16:19 Nafai: I haven't found such; I used to use SWT at my last job.

16:20 perspectivet: how's the native look-n-feel these days? It was pretty bad last time I looked?

16:20 slava: from what i've seen lately, swing is starting to look more native than swt :)

16:20 swt needs a big overhaul on os x to convert it from carbon to cocoa

16:21 perspectivet: that's underway though from what I've seen

16:21 Nafai: so you think swing is the way to go for a graphics heavy java app?

16:22 slava: what is swt's story when it comes to opengl?

16:22 is it supported as well as swing?

16:22 perspectivet: with jogl panels and such

16:22 slava: good

16:22 perspectivet: http://www.eclipse.org/swt/opengl/

16:24 Nafai: perspectivet: I'm not sure how I feel. Both of the choices for SWT/Eclipse and Swing were handed down from up-above.

16:24 An advantage of Swing is there are a lot more 3rd party things

16:27 perspectivet: You don't need to ship swt jars either

16:27 which is nice

16:46 jgrant: in the context of thread local variables...

16:46 I've tried 2 approaches

16:47 the first being (def x) (binding [x ...]) (set! x ...)

16:47 this works but i know that this apparently not idiomatic of clojure

16:47 *winks about the exclamation in set! *

16:48 the other is (with-local-vars [x ...] (var-set x ...) (var-get x ...) )

16:48 this second approach is very tedious

16:49 and exceptions are thrown a local var is access but not yet set meaning even more exception handling code

16:49 is there another less verbose way to handle thread local variables ?

16:50 Chouser: you want to mutate the variable, not just stack bindings I guess?

16:50 jgrant: i do not need to mutate the root var just looking for thread isolation

16:51 (def x ...) is not thread-safe obviously from what i have seen

16:53 Chouser: I think your first form is actually correct.

16:54 (set!) is documented for thread-local bindings: http://clojure.org/vars

16:54 I think the exclamation is to signal mutation, which indeed is happening within the thread.

16:55 jgrant: yea it's much cleaner but i recall rich saying the set! should never be necessary in one of his talks but i'm not sure how else to do this ?

16:56 doing var-get on everything is ugly and error prone

16:56 cemerick: Am I wrong in thinking that binding does exactly this?

16:56 ...exactly what you want, I mean. :-/

16:56 jgrant: does what ?

16:57 cemerick: "Bindings created with binding cannot be seen by any other thread." http://clojure.org/vars

16:57 jgrant: yes i set! works and it's cleaner but why would rich mark it with the exclamation ?

16:57 yes bindings are thread local

16:58 what is dangerous about set! if it's only modifying a thread local variable ?

16:58 cemerick: jgrant: Oh, because mutation like that is Bad (tm).

16:58 jgrant: why is it bad ?

16:58 erochester: jgrant: Chouser's right. The exclamation just signals that the function mutates something. It's a convention from scheme and other lisps.

16:59 i think bad = makes concurrency hard, and isn't a functional way of approaching a problem

16:59 cemerick: jgrant: It's a stylistic indication that if you're using set!, you're probably not taking advantage of other, cleaner primitives.

16:59 jgrant: cemerick : have you tried using (with-local-vars (var-set ...) (var-get)) in thread safe code ?

16:59 what are those 'cleaner primitives' ? (I've asked a few times now)

17:00 cemerick: jgrant: I've not personally used with-local-vars, no.

17:00 jgrant: cemerick : (withlocal-vars) is hardly clean

17:00 it turns my code into an eyesore

17:00 cemerick: an example of a cleaner primitive: set! is often used to manage state within a loop, etc. It's far better to use loop/recur in that scenario.

17:01 Chouser: set! will throw an exception when used without a thread-local binding. I think it's safe and valid.

17:01 jgrant: loop/recur has nothing to do with thread-local vars

17:01 Chouser: You shouldn't be doing any mutation at all if you can help it, but if you must, I think using set! within a thread isn't particularly bad.

17:01 jgrant: Chouser : i agree

17:02 cemerick: jgrant: strictly speaking, no, it has nothing to do with thread-locals, but in the case of iterative processing, loop/recur is way better than doseq/dotimes + set!.

17:02 Chouser: jgrant: you're sure you need to mutate, and can't use recursion or stacks of binding calls or anything?

17:02 jgrant: cemerick : how specifically is loop/recur better than doseq/dotimes + set! ?

17:02 anyone care to define better ?

17:03 Chouser: no mutation.

17:03 with loop/recur, a local gets it's value at the top of a block, and then doesn't change until the block is done

17:03 jgrant: if this was common lisp then i wouldn't be asking the question ;-)

17:03 erochester: common lisp isn't really a functional language :)

17:04 cemerick: jgrant: the same way using foreach over an enumerable is better than iterating through it manually with counters in C#/Java -- more idiomatic, and there's the small bonus of having no side-effects

17:04 jgrant: in real life no 100% pure functional language works

17:04 Chouser: with any kind of mutation (including set!) you have to scan through all depths of the code indentation, and possibly even within called functions to see who might be calling set!

17:04 * cemerick smells some flames starting up

17:04 jgrant: cemerick : your're assuming that what is being modifed needs to be looped over but in many cases it does not

17:05 cemerick: jgrant: Yes, I was citing one example.

17:05 Chouser: I don't think there's a real argument to be had here. set! is provided for thread-local bindings because it is needed sometimes.

17:05 jgrant: Chouser : thanks that's what i needed to know ;)

17:05 cemerick: Chouser: agreed, it's a handy backdoor for those very, very rare cases

17:05 Chouser: Often mutation (using set! or def) is done when it would be better not to.

17:06 erochester: it's handy for global parameters, like common lisp uses dynamic variables.

17:06 Chouser: Clojure provides one or two "backdoors" compared to a purely functional language, but it provides a whole pile of features to make it easy to avoid mutation entirely.

17:07 jgrant: if you're willing, it'd be interesting to see your use case. Maybe somebody can come up with solution that is syntactically cleaner than set! or var-set, and also mutation-free.

17:51 ericthorsen: Is there a shortcut for (apply merge (;sequence of maps))? Similar to vec and vector?

17:53 nevermind...(reduce conj maps)

18:29 jgrant: Chouser : the use case is really a thread that generates a string based on some variables (i.e. it looks up values in a hash map) so there really is no looping

18:31 most important is that the variable that stores the string is thread local and that it can be modified over and over as the string is being generated and then finallly return

18:33 using with-local-vars and var-set var-get turns this into very verbose code

18:37 slava: tomhickey_: are you related to rich? :)

18:38 tomhickey_: slava: yes, he's my brother

18:38 slava: cool

18:38 jgrant: tom : are you also involved in the clojure implementation ?

18:39 slava: will you guys start a company Hickey & Hickey LLP to commercialize Clojure? :)

18:40 tomhickey_: jgrant: no. but i do develop with it

18:41 slava: =) that's all in Rich's hands

18:42 jgrant: tomhickey : cool, tom do you have any advice for me on my use case above ?

18:44 tomhickey_: jgrant: i wish. i think Chouser and cemerick provided much more help then i would be capable of

18:45 * tomhickey_ is a novice clojure developer

18:45 jgrant: tomhickey : thanks yea they did

18:47 StartsWithK: Why i can't get function medadata this way? (defn x [a] (println a)) (def get-meta [f] (meta (var f))) (get-meta x) But (meta (var x)) outside of a function works.

18:48 jgrant: slava : it's my understanding that rich will keep clojure open it's currently under the CPL v1.0

18:49 http://www.opensource.org/licenses/cpl1.0.php

18:53 looks like this has gone up recently --> http://clojure.org/state

18:53 (i dont remember it being there before)

18:57 the last sentence on that page says this : "In the local case, since Clojure does not have mutable local variables, instead of building up values in a mutating loop, you can instead do it functionally with recur or reduce."

18:57 So the question is what if the mutation is not in a loop ?

19:04 Chouser: jgrant: you might consider using the StringBuilder class.

19:05 But that's really just another form of the same "cheat", shoving the mutation down into a Java.

19:05 jgrant: yes that is something i'm considering

19:06 Chouser: So you're doing a lot of (set! foo (str foo "bar")) ?

19:07 jgrant: yes

19:08 Chouser: so you append "bar" and "alice" and "cindy". You might consider (str "bar" "alice" "cindy") instead.

19:08 (yes, I realize this is a drastice over-simplification that might be hard to implement)

19:08 jgrant: and if there is logic between "bar" "alice" and "cindy" ;)

19:09 Chouser : but thanks i see where you're going with this

19:10 maybe i can convince rich to take the exclamation out of set! ;-)

19:10 Chouser: right. (str (let [...] ... (apply str (map ... "bar"))) "alice" (reduce ....))

19:10 jgrant: after all set! is for thread local variable modification it does not modify the root context

19:11 Chouser: I really think the ! just indicates mutation. The ruby standard library is full of methods differentiated by ! for whether they mutate their object or not.

19:11 jgrant: Chouser : you are right by those are methods on classes and set! is a function/macro

19:11 so slightly different meaning

19:12 Chouser: well, if you say so. It seems to me they both indicate that the thing their operating on will return a different value afterward than they did before.

19:12 jgrant: Chouser : yes you're right

19:12 but why dont we have def! or var-set!

19:12 ?

19:12 (in clojure)

19:13 ;)

19:13 even more of a reason for def! because it modifies variables that are not thread local

19:15 Chouser: indeed. but you're not supposed to use def anywhere much.

19:15 bathtime for kiddos. bbl

19:16 jgrant: again it is necessary in apps that aren't threaded but have the same requirement as we've covered above with string building.

19:16 Anyone raise your hands if you're ok with def or set! being removed from clojure ;-)

19:18 yes we all agree imperative programming is 'evil' but a 99% functional 1% imperative language is still necessary for building real-world apps

19:26 especially since clojure is built on java and takes advantage of existing libraries.

19:29 meredydd: jgrant is right, I think, about the exclamation point - it's intended as a "whoa...are you *sure* this is necessary?"

19:29 I think you're dead wrong about state, though :)

19:30 jgrant: meredydd : how would you solve the problem described above with conditional string building ?

19:30 meredydd: *looks*

19:30 jgrant: (in a purely functional manner)

19:30 meredydd: What's the problem?

19:31 (sorry, didn't grok what you were trying to say)

19:31 could you state it again?

19:31 jgrant: sure...

19:31 ...a thread is generating a string stored in a thread local variable based on some conditions

19:32 meredydd: ah, and this is assumed?

19:32 jgrant: there's lots of concatenation going on

19:32 meredydd: Because if you're taking, as your starting assumption, that there is:

19:32 a) a thread, operating sequentially

19:32 b) storing its results in a thread-local variable

19:32 jgrant: ok lets use a real world example

19:32 a web server

19:32 meredydd: ...then yes, you're going to end up requiring imperative programming, because you've already assumed it :)

19:32 jgrant: we dont want to send all client the same payload right ?

19:33 ok then do you have another solution ?

19:33 meredydd: Okay...what parts of a web server?

19:33 Well, depends what the string generation is trying to *do*.

19:33 jgrant: servicing a request

19:33 and returning a payload

19:33 it's a dynamic payload based on params passed in to the server

19:33 meredydd: Uhh...so, to put it another way, the payload is a function of the request and its parameters? :)

19:33 jgrant: very simple

19:34 yes

19:34 ever used a function (in a mathematical sense) that modified a paramter ?

19:34 it's very common

19:35 meredydd: So, generally speaking, you'll want to write a function which takes the request as a parameter, and returns the response.

19:35 Really, jgrant? Give me an example.

19:35 jgrant: so everyone states i'm wrong (which i have no problem with) but no one has a solution yet ?

19:36 meredydd: It's difficult to have a discussion, when what you're essentially saying is "I need to do <procedural programming implementation technique>, so I need to use procedural programming!"

19:36 jgrant: is that what i'm saying ?

19:36 meredydd: My hunch is that whatever underlying problem you're solving can *also* be decomposed in a functional way.

19:37 jgrant: i'm open to any solution presented in fact i would love a purely functional solution instead

19:37 meredydd: For example, by the time you're thinking "servicing a request" = "generating a thread-local variable in another thread", then you've already committed yourself to a procedural approach.

19:37 So, present the problem :)

19:37 jgrant: ok forget the thread-local variable

19:37 meredydd: What does this web-server need to do?

19:39 jgrant: commonly what you already described, i.e. serve a payload based on incoming parameters

19:39 what most web servers do

19:39 meredydd: okay...in that case, I'd do something like:

19:39 jgrant: mereydydd : have you looked at web server implementations in functional langauges ?

19:39 meredydd: main loop:

19:40 (loop [] (let [s (. my-server-socket (accept)] (new Thread #(handle-connection s))) (recur))

19:41 jgrant: yes that's not the problem though

19:41 is generating the payload in the thread that's a problem

19:41 meredydd: (defn handle-connection [s] (.write s (generate-response (read-request s)))

19:42 [note that I'm hideously mangling the ServerSocket API here - because of how Java works, there's a bunch of (.getOutputStream) and friends needed, but they don't affect the main point)

19:42 jgrant: now we're going somewhere, and what would generate-response implementation look like ;)

19:42 meredydd: okay, fudging the generation of a filesystem path from a request (which I think you'll agree is pretty easy to do as pure-functional):

19:43 jgrant: not concerened with any of that stuff

19:43 it can all be handled purely functional

19:43 it's generating the payload i.e. string concatenation

19:43 meredydd: (defn generate-response [file-path] (let [finp (new FileInputStream file-path my-response (read-all-from finp)] my-response))

19:43 jgrant: how would you do it without def or set! ?

19:44 meredydd: What kind of string concatenation are you after?

19:44 jgrant: aha !

19:44 that would just load a static file form the filesystem

19:44 i'm talking about generating a dynamic response

19:44 meredydd: yeah...you asked for "what a web-server normally does"

19:44 All right...what do you want it to do?

19:45 jgrant: let's say you have a template

19:45 and values that will be used to populate it ?

19:45 meredydd: (defn generate-response [request] (str "<html><body>Hello, " (:name request) "</body></html>")

19:45 the "template" there is in S-expressions

19:45 jgrant: how would you make parts of the template conditional ?

19:46 meredydd: (defn generate-response [request] (str "<html><body>" (if (:name request) (str "Hello, " (:name request)) (str "Enter your name: <input name='name'...")) "</body></html>")

19:47 That part's conditional on whether you specified a :name parameter (assumed to be grabbed from the HTTP params)

19:47 If you wanted to load the template from a file, you could do something like:

19:48 jgrant: now what if the request needs to be 'cleaned' before being used ?

19:48 where is that stored ?

19:48 meredydd: 'cleaned' how? Shall we just say a functional cleaning?

19:49 jgrant: validation of incoming params

19:49 meredydd: (defn generate-response [dirty-request] (let [request (clean dirty-request)] ...code here...))

19:49 jgrant: decoding of values passed in

19:49 meredydd: Ah...and now I understand what you mean by "changing the arguments".

19:49 jgrant: and is let thread-safe ?

19:50 from what i understand let is not

19:50 meredydd: "mu."

19:50 (let)-bound variables are lexical, and never change value.

19:50 *lexically-scoped.

19:50 ahh...here's our confusion.

19:51 My assumption here is that (clean) doesnt' modify the "request" object.

19:51 because the request object is immutable.

19:51 Instead, what it does is return a new "request" object, whose value is a cleaned-up version of its argument.

19:51 This is how the Clojure collection functions work, as well. For example:

19:52 (let [a {:name "meredydd} b (assoc a :name "jgrant")] (println "a:" a "b:" b))

19:52 jgrant: i see

19:52 meredydd: prints "a: {:name "meredydd"} b: {:name "jgrant"}"

19:53 the value of "a" never changes - (assoc) just returns an immutable map which differs from the immutable map it was passed only in the value bound to the key :name.

19:53 jgrant: so we trade memory for going purely functional (which is what rich described on the state page of clojure.org)

19:54 now in a large web server we end up using LOTS of memory the more clients we have connecting

19:54 meredydd: As a rule of thumb, when your procedural experience tells you "I need to change the value of this variable", The Functional Way is "I need to declare a new variable which is some function of the value I currently have"

19:54 Aaah...yes. And this is where persistent collections come in.

19:54 For example, if I did:

19:54 (let [a {:name "meredydd" :data <lots-of-big-ugly-data>} b (assoc a :name "jgrant")] (println "a:" a "b:" b))

19:55 the value of "b" *shares* its memory storage of <lots-of-big-ugly-data> with "a".

19:55 jgrant: 'persistent' being a euphemism for imperative then ;)

19:55 meredydd: nope.

19:55 so you've only allocated enough new memory for the new "name" value

19:55 jgrant: persistent objects are mutable correct ?

19:56 meredydd: Nope. Immutable.

19:56 b is immutable, as is a.

19:56 jgrant: then they cannot be changed

19:56 meredydd: correct.

19:56 But when you create b, using a as your argument to (assoc), you re-use as many bits of "a" as you can get away with.

19:57 jgrant: so we use more memory

19:57 meredydd: (and you can get away with a lot - because the value of "a" won't change, you can re-use all its data structures!)

19:57 joubert: well, if you want to store more information you need to use more memory, no?

19:57 jgrant: why when it's just a modified copy

19:58 ?

19:58 meredydd: jgrant: That's the point - you don't. You only allocate enough new memory to hold the new entry you just created (in this case, ":name --> 'jgrant'")

19:58 It's not a "copy", like you're thinking of.

19:58 jgrant: if i have 32GB of data and use functional approach instead of the 'evil' imperative approach i could theoretically have to use 64GB instead

19:58 meredydd: Hold on one moment

19:58 jgrant: maybe not to the programmer but to the machine it is a copy

19:59 meredydd: Other way round, actually - the programmer can treat it as a copy

19:59 jgrant: or at least a modified copy

19:59 meredydd: (which is why I explained it to you that way)

19:59 but to the machine it's much more efficient than that.

19:59 jgrant: the point is the functional approach requires significantly more memory

20:00 dudleyf: The machine only has to keep track of the "diffs" between the original structure and the new one

20:00 meredydd: Yes, and I'm arguing that that point is incorrect.

20:00 jgrant: dudleyf : thanks

20:00 hadn't heard that yet

20:00 joubert: jgrant: that is exactly what meredydd was saying :-)

20:01 meredydd: http://www.skrbl.com/79556388

20:01 jgrant: I'm going to draw you a diagram.

20:01 jgrant: meredydd : thanks

20:02 joubert: (btw, there's also a great presentation at http://clojure.blip.tv about persistent, immutable data structures)

20:02 dudleyf: Rich has some interesting diagrams on how the data structures actually work in one of his presentations

20:02 :-)

20:02 meredydd: oh, that'd be marvellous, dudleyf.

20:02 Cause this interactive-whiteboard app is really killing me.

20:03 jgrant: Have you opened that link?

20:06 jgrant: meredydd : yes looking at it now

20:06 meredydd: okay. You see the map value "A" on the left?

20:07 jgrant: yes

20:07 meredydd: It has two entries, ":name --> meredydd" and ":data --> big-chunk-of-data"

20:07 it contains pointers to each entry

20:07 jgrant: right

20:07 meredydd: (and yes, I know how rich actually does this...this is a simplification, because the implementation details aren't important at this stage)

20:07 jgrant: sure

20:08 meredydd: Now, what that (assoc) function does is to make a new entry, for ":name --> jgrant"

20:09 jgrant: is see

20:09 meredydd: Now, it creates a map value "B", and puts in the pointers I'm drawing in red.

20:09 So, A is still immutable - I haven't *changed* the value of any existing variable.

20:10 But at the same time, what I have is a map that uses the same chunk of RAM to store our BIG_CHUNK_OF_DATA

20:10 jgrant: so A is updated with the existing values in A and the new ones in B ?

20:10 meredydd: but has a different value for the :name key.

20:10 A isn't touched at all.

20:10 There's no "updating">

20:10 jgrant: so how then is :name resolved correctly ?

20:10 meredydd: All I did was create a new map, with the pointers in red.

20:10 Well...

20:11 Chouser: in the webserver example, A would be the uncleaned parameters, B might be the cleaned ones.

20:11 jgrant: ah i see

20:11 meredydd: If I look up :name in A, it'll look down the blue pointers and find the blue entry

20:11 Chouser: If you want the cleaned ones, you look them up in B instead of in A.

20:11 meredydd: thanks, Chouser.

20:11 Chouser: meredydd: using skrbl looks like a lot of work. ;-)

20:11 meredydd: Chouser: You don't say.

20:12 jgrant: but i have 2 refs now

20:12 meredydd: uhh...

20:12 better use a different word - "ref" has a specific technical meaning in clojure.

20:12 you have two bindings, yes.

20:13 Chouser: one other little point -- if you stop using the uncleaned params (A), the JVM reclaims that memory (the little bit where :name->"meredydd" was stored) and it gets used for something else.

20:13 meredydd: Chouser: I was just about to explain that :)

20:13 jgrant: so each time a modify something i have to deal with a new ref

20:13 binding

20:13 ;)

20:13 meredydd: jgrant: Yes. As I say - when imperative stuff thinks "I need to change this value"

20:13 Chouser: jgrant: only if you want to. Do you want the cleaned ones? use those. The new object has new meaning.

20:14 meredydd: functional stuff thinks "I need to make a new variable, with a slightly-changed value"

20:14 jgrant: yea just thinking in my specific case

20:14 if i modify something x number of times then i have x number of bindings

20:14 meredydd: yep.

20:14 jgrant: x1 x2 x3 x4 etc etc etc

20:15 simply because i cannot rebind the new value to the existing symbol

20:15 meredydd: That's just a style thing. You actually don't need to do that many changes in sequence, under most circumstances.

20:15 Chouser: yes, but there are library of functions to make such multi-step "modifications" less necessary.

20:15 meredydd: If you find yourself doing that, you generally ought to look and see if you can do it more simply.

20:15 (or, as Chouser says, use a library function)

20:15 Chouser: another detail is that if you really really want to, you can use the same name for the new object:

20:16 (let [a 1, a (+ a 2), a (* a 5)] a) would return 15

20:16 meredydd: jgrant: Also note that if you're doing: (let [x1 (do-something x) x2 (something-else x1) x3 (and-again x2)] (...do something with x3))

20:17 jgrant: ...in that case, the virtual machine will notice that you're not using x1 or x2 any more, and deallocate them

20:17 (I'm drawing on the diagram, in green, what that would mean in our example)

20:18 jgrant: yea i get that

20:18 meredydd: jgrant: So, you don't use the extra memory.

20:19 jgrant: so how would you do conditional assignment of vars ?

20:19 i.e. some vars are only set if others have certain values

20:20 meredydd: (let [x (if (sky-is-blue) "blue" "not blue")] ...do something with x...)

20:21 If you're used to working in a language with a ?: operator, like Java or C, you'll be used to this kind of thing

20:21 jgrant: yes that makes sense

20:21 meredydd: (the Java equivalent is: { String x = sky_blue()?"blue":"not-blue"; })

20:22 except that, this being a functional language, we have *much* more powerful tools than ?:

20:23 We can use (cond), for example, which is like a turbo "if-elseif-elseif-elseif" version of ?:

20:24 jgrant: yea my main source of confusion has been using existing Java libraries (3rd party) and trying to convert the example code to idiomatic clojure code

20:26 meredydd: ah, yeah. That's definitely a problem.

20:26 My personal advice would be to practice with pure clojure code

20:26 Get your functional kung-fu up to scratch - get inside the mindset.

20:26 Then, once you understand *both* sides of the fence, you can bridge it efficiently.

20:27 jgrant: i'd love to but i'm set on using clojure right now while i'm coming up to speed , i'm blown away at how productive it is even while learning

20:27 meredydd: Otherwise, if you only know one side, you'll end up trying to write Java in S-expressions

20:27 jgrant: very true

20:28 meredydd: (which is all right as far as it goes - as you say, it's still a massive leap over plain Java - but you'll be missing out on a lot of the fun stuff too if you do that)

20:28 jgrant: so when does the O'Reilly clojure in a nutshell come out ;) ?

20:28 meredydd: In that case, if you can, try to keep your Java interface code neatly segregated into a few functions.

20:29 jgrant: well what i find is that even the kludgey code i have to come back and re-write usually doesn't take very long once i discover the right way

20:29 meredydd: :) yep

20:29 I recommend you make a religious point of never, ever using (set!), until you're really comfortable with it.

20:30 (set!) is a way of bending the rules, and once you get a feeling for the functional mindset (it won't take long), you'll have a good idea when to break the rules, and when there's a perfectly good functional way of doing it.

20:30 jgrant: i know set works with (binding [...]) does it also work with let or does the binding have to be mutable ?

20:31 meredydd: It has to be a (binding).

20:31 jgrant: ok good

20:31 meredydd: As I can't think of anything I've done in my last ten thousand lines of code that would *need* a mutable binding, I'm inclined to recommend you forgo them altogether until you're used to working with (let)

20:31 jgrant: meredydd : thanks alot for explaining all this

20:31 meredydd: np.

20:32 Chouser helped a lot, too - that "diff" comment was a one-line version of something I'd been trying to explain for ten minutes :)

20:32 jgrant: Chouser : yea thanks too !

20:37 meredydd: Right.

20:37 Anything else I can help you with?

20:37 cause I've just come back from London, and am in the mood for a good night's sleep :)

20:38 jgrant: meredydd : thanks, i'm sure i will have some more questions but get some sleep.

20:38 meredydd: Okies.

20:39 * meredydd hits the sack.

20:47 Chouser: "diff" was from dudleyf, but thanks anyway!

20:47 jgrant: Chouser : but yea i owe you anyway ;)

20:49 Chouser: it's a hard transistion to make, so it's great you're willing to put in the effort.

20:52 jgrant: Chouser : i've spent too much time with CL

20:52 it's bridging the java libs with clojure idioms that's getting me

20:52 Chouser: yeah, Java and its libs definitely push you toward mutable state

20:52 jgrant: but i must say there is a great community around clojure

20:58 Chouser: I think so too. I wonder how that happens. Some languages and projects just don't -- you ask the wrong question and you're treated like some kind of idiot.

21:03 cemerick: Chouser: is the archive you maintain of #clojure kept by a bot, or is that just your personal transcript of the channel?

21:04 Chouser: It's collected from the connection I speaking over right now -- irssi, not a bot.

21:04 It's converted to HTML by a clojure program.

21:05 wastrel: what sort of background is typical for someone picking up clojure?

21:06 cemerick: Lau got me thinking about throwing together a proper clojurebot.

21:06 wastrel: i know a lisp bot

21:07 cemerick: yeah, there's a lot of them out there. I think using a bot programmed in a different language is a cop-out. :-)

21:07 Chouser: cemerick: sure -- I'd be happy to hook the existing HTML generation into a bot instead.

21:07 cemerick: There's sure to be a java irc lib floating around, so it shouldn't be too hard.

21:08 Chouser: We'll see what happens. I'm on deadline for next Friday, so it won't happen until after then.

21:08 Chouser: yep

21:08 cemerick: too many side projects, absolutely no time :-)

21:09 Chouser: yep :-)

21:09 I started peeking at the Java security sandboxing stuff so we could allow the bot to eval expressions for us ... but then I got distracted by something else.

22:40 jgrant: is there an easy way to convert an IPersistentList to a java Array ?

22:48 Chouser: yes

22:50 to-array to get an array of Object

22:50 into-array to get a more specific type of array

22:54 jgrant: thanks !

Logging service provided by n01se.net