#clojure log - Jun 15 2008

The Joy of Clojure
Main Clojure site
Google Group
List of all logged dates

8:48 rhickey: Chouser: a var root is an atomic thing (a single Object). For last-one-in wins, write-over behavior nothing special is needed, other than that it be volatile, which it is.

8:49 read-modify-write is trickier, and that is why there are synchronized bindroot and commuteRoot, which you can use for safe ongoing multithreaded mutation of a var root (not recommended generally)

9:05 meredydd: What's the idiomatic way of dealing with Maps and Lists passed to you by Java code?

9:07 A lot of the syntactic pretty (using assoc to add elements, invoking a map to look up elements) is understandably dependent on the object being a Cojure persistent map

9:08 But is there a sensible way to do a conversion?

9:10 (sorry, I mean "idiomatic" rather than "sensible". I'm aware it's possible to (reduce) across (keys x), using (assoc)...)

10:02 rhickey: meredydd: (into {} your-java-map)

10:03 meredydd: aha. Thankee, rhickey.

10:05 rhickey: but note that seq/get/contains?/count/destructuring etc work for Java maps without conversion

10:55 meredydd: rhickey: Is the website your patch? If so, the redirects from sourceforge.net are all broken after the redesign (which is a pain, as they tend to score high on google)

13:03 jcrites_: how does one partially apply a function in closure?

13:03 say I want a function that adds one to a number

13:03 I want something like ....(+ 1)

13:03 do I need to use the fn form?

13:03 or is it possible to partially apply functions to other functions?

13:03 (or values)

13:06 ah, I see..... (partial + 1) is the same as (fn [x] (+ 1 x))

13:09 hoeck: jcrites: you can use #(+ 1 %) too

13:09 jcrites_: oh? cool, thanks!

13:09 what form is that exactly?

13:10 hoeck: jcrites: its an abbrevation for fn

13:12 #(...) => (fn [args] (...))

13:15 jcrites: is % special then?

13:16 Chouser: jcrites: inside #(), yes it is. Also: %, %1, %2, etc. and %&

13:16 oh, % is apparently special all the time -- only allowed inside #()

13:18 jcrites: % vs %&?

13:19 is %& "rest"?

13:19 Chouser: yup

13:19 jcrites: like %1 %&

13:19 ah, k

13:19 Chouser: yup

13:19 jcrites: Clojure is pretty neat

13:19 Chouser: yup

13:19 jcrites: :)

13:19 Chouser: that little feature is documented at http://clojure.org/reader?responseToken=e9a25f9ed12c8e758646d63847044670 -- search for Anonymous

13:29 jcrites: hmmm

13:29 how come closure lists print with ( ) ?

13:30 Chouser: classic lisp. what should it use?

13:30 jcrites: this confuses me

13:30 > [1 2 3]

13:30 [1 2 3]

13:30 > (rest [1 2 3 4])

13:30 (2 3 4)

13:31 is it something to do with sequences?

13:31 I would expect (rest [1 2 3 4]) to print [2 3 4] not (2 3 4)

13:31 maybe there is some subtle property I am missing

13:31 Chouser: that's a great question.

13:32 I think sequences also print with ()

13:32 jcrites: > (list 1 2 3)

13:32 (1 2 3)

13:32 so I guess [] is just for vectors

13:32 Chouser: (seq [1 2 3]) ==> (1 2 3)

13:32 yes, [] is just for vectors

13:33 "rest" and "first" work on sequences, and "rest" always returns a sequence.

13:33 jcrites: can a sequence also be a vector?

13:34 i.e., let's say I do (rest [1 2 3]) and I get a sequence. can I get back to the vector 'type' and index on it?

13:34 or is it hidden by the rest?

13:35 well let me find out :)

13:35 but even if it works it doesn't mean it's guaranteed to work right?

13:35 Chouser: sorry, watching kids too here...

13:35 seqs are generally views of the underlying object

13:35 I don't think you can get back to the underlying object from a seq

13:36 but you can choose to use more specific functions.

13:36 for example, cons works on seqs and returns a seq: (cons 4 [1 2 3]) => (4 1 2 3)

13:37 jcrites: how can cons take [ ]? is it treated like (cons 4 (seq [1 2 3])) ?

13:38 Chouser: exactly

13:38 all the seq-based functions wrap a (seq ...) call around their collection parameter

13:38 on the other hand, conj is implemented for each of the collections seperately

13:39 (conj [1 2 3] 4) ==> [1 2 3 4]

13:39 (conj {1 "one", 2 "two"} [3 "three"]) ==> {1 "one", 2 "two", 3 "three"}

13:48 meredydd: Anyone care to enlighten me as to why macros can be passed as function parameters without the compiler squawking?

13:49 I just attempted to use (and) in a (reduce), and got a nasty surprise

13:49 It executed just fine, but using the macro's code-transformation function

13:51 so, if I try: (def a true) (def b false) (def c true)

13:52 and then (reduce and [a b c])

13:52 I get: (clojure/let [and__152 (clojure/let [and__152 true] (if and__152 (clojure/and false) and__152))] (if and__152 (clojure/and true) and__152))

13:52 (where I'd normally be expecting 'false')

13:53 I appreciate that it might be a good idea to *allow* people to access a macro as a function...but does it have to happen silently like that?

13:54 hoeck: meredydd: macros are functions that are handled specially by the compiler

13:55 and its bad the compiler doesn't warn if you pass them around as functions

13:55 meredydd: I agree with you on both counts.

13:56 (although there should be a way to pass a macro as a funarg if you assure the compiler you know what you're doing)

13:58 hoeck: i guess macros being functions is an implementation detail for clojure

14:02 jcrites: what's the idiomatic way to represent factorial recursively in Clojure?

14:05 (defn fact [n] (* n (if (> n 1) (fact (- n 1)) 1))) is that good?

14:06 I'm new to Lisp

14:12 Chouser: meredydd: you can't pass macros where something (like reduce) is expecting a function.

14:12 hoeck: jcrites: totally valid

14:12 Chouser: But frequently trivial to wrap a macro call in a function: (reduce #(and %1 %2) [a b c])

14:12 meredydd: Chouser: On the contrary; I have just spent some time learning that you *can*...

14:12 jcrites: so I'm writing this function and I need a way for it to call itself

14:12 Chouser: meredydd: sorry. "shouldn't"

14:12 jcrites: does that mean I need to put it inside a let expression?

14:13 meredydd: Chouser: Yeah. I'm taking that as a given ;)

14:13 jcrites: and can functions in let expressions call themselves through their name in the let expression?

14:15 Chouser: functions defined using defn can call themselves by name (although depending on how you use it you might overflow Java's call stack)

14:15 functions defined using fn can be named for the purpose of calling themselves: (fn foo [x] (...can call foo here...))

14:16 jcrites: oh

14:16 I didn't know fn worked that way

14:16 I was trying to do some awkward (let [f (fn ...)]) thing :)

14:16 Chouser: yeah, there's an optional name parameter there. But that name is not valid outside the (fn) expression

14:16 jcrites: that's awesome!

14:16 it makes sense

14:17 hoeck: this name us also useful for debugging

14:17 Chouser: you still might need to do that if you want a function defined locally: (let [f (fn ...)] (f ...))

14:17 hoeck: it pops up in the stack if the named fn throws an exception

14:17 Chouser: or worse if f is recursive: (let [f (fn f [] ...)] (f ...))

14:18 ...because in that case it's really two different f's: (let [f1 (fn f2 [] ...use f2 here...)] (f1 ...))

14:19 jcrites: right. I was just trying to use let to give it a name for itself :) but if it has that built-into fn already I'm set.

14:19 so does Clojure have something like "letrec"?

14:19 or is that just "def"

14:19 Chouser: but you should be a bit careful doing recursive calls like that, since there's no tail-call optimization. You may have to use recur or lazy calls.

14:19 jcrites: oic

14:20 let me show you what I'm actually trying to do.... 1 sec

14:20 Chouser: jcrites: I don't think there's a letrec. I suppose you could write a macro that does (let [f (fn f [] ...)] ...) for you.

14:21 def and defn are always for globals

14:36 jcrites: let's say I have a list.... I have a value... if the value is non-nil, I want to append it; if it's nil I want to do nothing

14:36 should I write that logic myself or is there a built-in with that kind of meaning?

14:36 Chouser: vector, right? you don't append to lists generally.

14:37 jcrites: I was planning on building up a list with recursive calls and cons ....

14:37 Chouser: jcrites: out of curiosity, what's your language background?

14:37 jcrites: I'm trying to translate my little Scheme knowledge to Clojure :)

14:37 Chouser: cons "appends" at the beginning. Which is fine if that's what you want.

14:37 jcrites: most of my experience is with curly-bracket languages. I have some university experience in Scheme

14:38 Chouser: ok, very good. So you already know what I just said about lists. :-)

14:38 ok, so cons-ing onto a list returns a new list, right? It never modifies anything. So by "do nothing" you mean return the original list unchanged?

14:38 jcrites: right

14:39 yrb: jcrites: Just out of interest what is your opinion on all the parens... ;)

14:39 jcrites: I'm trying to write some parser combinators in Clojure as an exercise, if you know what those are :)

14:39 I don't like them :)

14:39 Chouser: I've heard them mentioned, but the words me nothing to me. :-)

14:39 jcrites: so a parser is a function that takes in a sequence and returns a sequence (possibly empty)

14:40 it returns what it has "matched"

14:40 so if your parser matches everything up to space " " it might take in the sequence "hello, world" and return the sequence "hello,"

14:40 a parser combinator is a function that builds parsers

14:40 Chouser: afaik, the only builtin thing that does a bit like you want is concat (or mapcat): (concat nil '(1 2 3)) ==> (1 2 3)

14:41 jcrites: you could imagine a parser combinator const that takes in a string. what it does is build a parser that matches exactly that string or fails (matches nothing, returns nil)

14:41 Chouser: but of course that only works if the first thing is a seq of some sort, not a single value.

14:41 jcrites: so I was trying to write some of these combinators in Clojure...

14:42 (defn success [] (fn [q] (list q)))

14:42 Chouser: yeah, ok, sounds like a good exercise.

14:42 jcrites: (defn failure [] (fn [q] nil))

14:42 Chouser: Hm, I may have read about those in On Lisp.

14:43 I don't think clojure has a builtin that does: (if x (cons x lst) lst)

14:43 jcrites: yeah I am just trying to make sure I'm doing things the right way :-)

14:43 Chouser: yup. nobel. :-)

14:43 jcrites: sometimes the urge to make something yourself is actually misdirected and only exists because you're doing something wrong, hehe

14:44 I've been bit by that before ;-)

14:44 Chouser: yep, especially in schemes and lisps.

14:45 jcrites: looks like cons may already do what I want kinda...

14:46 Chouser: really? maybe I misunderstood your question.

14:46 jcrites: here is my attempt at const:

14:46 (defn const [cq] (fn p [q] (if (= (first cq) (first q)) (cons (first cq) (p (rest q))) nil))

14:46 on entering it the repl is frozen so I must be doing something wrong :(

14:47 Chouser: missing a )

14:47 jcrites: basically, if the first of cq and q are equal then return a new list with that element, recur by calling myself

14:47 otherwise return nil (empty)

14:47 it looks like cons will, ... well, (cons 1 nil) => (1)

14:48 if I remember Scheme that's the same way Scheme does it

14:48 Chouser: ah! yes, but (cons nil '(1)) => (nil 1)

14:48 I had your question backwards.

14:48 nil acts like an empty list for cons

14:48 jcrites: I actually think I was thinking the wrong way ;-) or not explaining that the "rest" would be returned by a recursion

14:49 so you mentioned that this will be real Java recursion and that that could bite me

14:49 Chouser: yes

14:49 jcrites: if I were going to use a bunch of these all nested together, on files with thousands of characters, I'm guessing I would be screwed?

14:49 Chouser: but just replace (p (rest q)) with (recur (rest q)) and you're all set

14:49 ...and then you don't need to name p anymore

14:50 jcrites: oh, so recur is always a call to the self-fn ?

14:50 that's neat

14:50 Chouser: yes

14:50 or the surrounding (loop ...), whichever is closer.

14:50 It's a pretty elegant hack to work around the JVM's lack of tail call optimization.

14:51 jcrites: interesting

14:51 how does it work? I mean Clojure is compiled to jvm bytecode right?

14:51 Chouser: yes

14:52 I don't really understand it that deeply, but I think regular clojure functions calls (including recursive ones) are compiled into real Java function calls.

14:52 hense the stack overflow issue.

14:52 jcrites: yeah, it's complex... I wonder how the compiler avoids that with recur

14:53 Chouser: But recur is a special form that the Clojure compiler must turn into some kind of goto (with parameter munging).

14:53 jcrites: maybe it actually compiles to a JVM loop.... yeah, that makes sense

14:54 * Chouser stares at RecurExpr in Compiler.java and tries to make sense of it.

14:55 jcrites: link?

14:55 I'm no JVM expert but I've worked on compilers before

14:55 Chouser: um, sure, just a sec...

14:55 jcrites: oh no

14:55 I got an error

14:55 java.lang.UnsupportedOperationException: Can only recur from tail position

14:55 (defn const [cq] (fn [q] (if (= (first cq) (first q)) (cons (first cq) (recur (rest q))) nil)))

14:55 it isn't in the tail position I guess :(

14:55 Chouser: oh! I'm sorry, I misread your code.

14:56 no, of course the compiler's right -- you're passing the result of recur to cons

14:56 so even real tail call optimization wouldn't help you.

14:58 jcrites: so I can't implement my algorithm recursively with ease

14:58 perhaps I can get the cons inside the recursive call....

14:58 Chouser: well, you can as long as it's not too deep. :-)

14:59 right, that's one route -- rewrite your function to be tail-call-optimizable (something else I read about in On Lisp)

14:59 you actually might be able to make it lazy instead.

14:59 replace cons with lazy-cons

15:00 (...and put p back in)

15:00 meredydd: More questions: Why does (contains?) only work with maps, and what should I be using instead on (seq)s?

15:01 Chouser: contains? is constant time (or log(n)) on maps. for a seq you have to search linearly

15:01 (some #{3} [1 2 3 4])

15:01 meredydd: And there are two separate functions just to keep me from accidentally doing so?)

15:02 Aha. Ta.

15:02 jcrites: it sort of makes sense

15:02 Chouser: some is much more general than just finding a specific item in a seq

15:02 jcrites: contains? reads like a quick predicate, not a "search"

15:03 meredydd: Chouser: Yep, that's what I was planning on doing; just reckoned it ugly. However, the neat trick with the invokable set renders it quite passable :)

15:04 Chouser: meredydd: yeah. That's straight from rhickey -- I didn't think of it.

15:05 meredydd: Oh, plus, looks like (contains?) works exactly as advertised on vectors - if you feed it an index, it will tell you whether (get) will give you anything

15:05 Chouser: huh!

15:05 meredydd: So, it fulfils exactly the same contract. Clever.

15:06 I do love this language. Whenever I poke into "this looks like a pain", the answer regularly comes back as "oh, that *is* clever"

15:06 jcrites: :)

15:06 it has a lot of elegance and aesthetic appeal

15:07 and consistency to the design of the APIs

15:07 Chouser: I would give up some performance for those characteristics, but apparently I don't have to.

15:08 yrb: :P the cooler bit is that you can invent the language you need as you need it

15:08 jcrites: so Clojure manages not to work with boxes objects often?

15:09 meredydd: That's the wonder of Lisp, yrb. The wonder of Clojure is that so far, I'm spending my time doing that for my own purposes, rather than trying to work around corner cases in the language design (*cough* Common Lisp *cough*)

15:10 jcrites: Common Lisp has some ugly parts of the language that I sort of don't understand

15:10 like the fact that functions aren't values, and you need to do weird #' stuff

15:10 meredydd: jcrites: I believe rhickey's been adding some explicit native arithmetic. But normally, I think most stuff is boxed right now.

15:10 jcrites: That's to do with being a Lisp[2], rather than a Lisp[1] which is what Clojure is.

15:10 jcrites: how does it perform well then? :/ I would expect boxing and unboxing between every operation would be fairly expensive

15:11 meredydd: what does that mean? [1] and [2] ?

15:11 meredydd: jcrites: For numerical applications, sure. However, most people don't write numberical apps

15:11 fyuryu: and you can put type hints for the compiler

15:12 jcrites: but any application works with integers quite a lot.... iterators in a loop for instance

15:12 meredydd: If you're using iterators in Clojure, it's almost always implicitly, as a (for) or a (seq) - and the iterators themselves are written in Java using native arithmetic

15:13 jcrites: ahh

15:13 meredydd: As for the Lisp-1/Lisp-2 distinction, check out http://www.nhplace.com/kent/Papers/Technical-Issues.html

15:14 Basically, Lisp-1s (like Clojure, and I believe Scheme) have functions in the same namespace, whereas Lisp-2s (like Common Lisp) have separate namespaces for functions and everything else

15:14 yrb: I think that would be a micro optomisation also, any who knows what the jvm is doing...

15:14 anyway*

15:15 there is some really awesome tech in the jit

15:15 jcrites: true

15:15 I suppose it would detect if the boxed numbers are not being used as objects

15:16 yrb: I think it will even generate fall through code for "hot" code paths

15:17 jcrites: fall through code?

15:17 (I don't know a ton about optimization)

15:18 meredydd: Oh, yuck. Chouser, any idea how to do backquote-style quoting explicitly (ie like (quote))

15:20 yrb: like say you call a function with 4 params and it ends up at run time 9 times out of 10 3 params are the same it can gen native code specifcally for that case

15:21 but it isn't iimited to functions it is entire code paths i think

15:26 Does anyone know of any lispy gui toolkits that have a good design, I am tring to write a Swing wrapper (called Cling ;) ) just looking for a good design to get inspired

15:27 jcrites: well, you might take lessons from the declarative interface kits

15:27 in Yegge's recent post he talks about ...

15:27 in Rhinos and Tigers: http://steve-yegge.blogspot.com/2008/06/rhinos-and-tigers.html

15:28 yrb: That is the idea, I have been having a look at jelly swing

15:28 jcrites: ah, ok :)

15:35 Chouser: I believe (almost?) all clojure functions take boxed args and return boxed, so no boxing/unboxing.

15:36 jcrites: well, I mean if you have (+ n 1) there has to be some boxing

15:36 Chouser: ...except when calling Java functions, or the new native number stuff, in which case it tries to use native... somehow...

15:40 jcrites: numeric literals are boxed, probably at compile time

15:44 meredydd: I don't quite understand. you want ' is to quote as ` is to _x_ ?

15:45 meredydd: Yep.

15:45 Chouser: hm, I don't think there is such a beast.

15:45 meredydd: Looking at what macroexpand-1 says, it's all done with a very hairy hack inside the reader

15:45 Chouser: yeah, sound about right

15:46 meredydd: so I just rewrote the macro to require the user to backquote the args that go in.

15:46 Less pretty, but livable.

15:46 Chouser: http://n01se.net/paste/Yiq -- Clojure Compiler.java RecrExpr.emit

15:47 ...that appears to emit a bunch of local variable manipulation followed by a goto.

15:53 jcrites: hmmm

15:53 (first nil) => nil ?

15:54 yrb: meredydd: Just out of interest why did you need it `ed

15:54 meredydd: yrb: I'm writing a unit test framework.

15:55 jcrites: bah

15:55 meredydd: There's a utility macro for asserting that a macro expands as it should

15:55 jcrites: I found the bug

15:55 can you spot what's wrong with this function:

15:55 meredydd: (defmacro assert-macro [macro-expr expanded-expr]

15:55 `(assert (= (macroexpand-1 ~macro-expr) ~expanded-expr)))

15:55 jcrites: http://pastebin.com/d45395948

15:56 tricky bug I wrote there :-)

15:56 meredydd: yrb: Ideally, I'd like to do something like ~(quote expanded-expr) and ~(quote macro-expr)

15:56 jcrites: causes infinite recursion

15:57 what's the problem? the problem is that if both sequences are nil, (first nil) returns nil, meaning my function recurses infinitely :(

16:00 now I'm getting wrong number of arguments passed to cons... sheesh... I suck >_<

16:00 is 2 not enough?!! IS IT NOT ENOUGH?!

16:09 hmmmm

16:09 what have I done wrong with this function?

16:09 (defn const-p [cq q] (if (and cq (first-eq cq q)) (cons (first cq) (const-p (rest cq) (rest q)) nil)))

16:09 I keep getting the error: wrong number of args passed to cons

16:10 Chouser: (cons (first...) (const-p...) nil)

16:11 jcrites: oh I thought I put the nil on the if

16:11 did I really put it in the cons >_<

16:11 oh shit you're right. thanks!

16:11 Chouser: this is why you need an editor to help

16:11 jcrites: I agree :(

16:11 Chouser: what are you using?

16:13 jcrites: nothing

16:13 working on the REPL

16:15 yrb: emacs works really well with lisps

16:35 meredydd: does this not work? (defmacro macro-equal [me ee] `(assert (= (macroexpand-1 (quote ~me)) (quote ~ee))))

16:36 meredydd: yrb: Nope. The macroexpand will add namespace qualifications to symboles, whereas (quote) won't.

16:36 *symbols

16:38 yrb: ah I was assuming the user must also add namespace quals

16:41 meredydd: ...which is what I've fallen back to. But it's a PITA.

16:41 yrb: how about doing this? (defmacro macro-equal [me ee] `(assert (= (macroexpand (quote ~me)) (macroexpand (quote ~ee)))))

16:42 meredydd: Ooh, goodness. That's going to look ugly as sin on the debugging, but should cure the other stuff...

16:42 Good idea.

16:43 (The wheels on the bus go "hack hack hack"...)

16:44 yrb: that so isn't a hack... rofl... well not when you look back at java/c++ land

16:46 but your right will look ugly as sin debugging

17:14 * meredydd blink

17:14 meredydd: *blinks

17:25 jcrites: hmmmm

17:25 I love how easy it is to represent algorithms in Lisp

17:26 none of the... what he called "ceremony" :)

17:26 but it's hard to figure out the right way to do it some of the time

17:26 what I'm trying to do (recursively, which is perhaps why it's hard) is check if one sequence is a sub-sequence of another sequence. if so, return it, otherwise return nil

17:27 without writing too many conditionals...

17:28 I feel like there should be some elegant way to do this without the "if this - if that "

17:28 meredydd: yrb: If you're interested, it turns out that even that wasn't quite right.

17:28 jcrites: my algorithm now is basically..... if (both sequences are nil) true (else .....

17:29 else if the first element of each sequence is equal, then call self recursively, otherwise return nil

17:29 yrb: meredydd: Interesting, where does it fail?

17:29 meredydd: Because the args still had non-namespaced stuff, which (macroexpand) wouldn't touch

17:30 and apparently, nested `s don't work, so you can't invoke that quote on it inside the macro

17:32 It could probably be done with a macroexpand, followed by a code-walk to force the namespace qualifiers onto any unqualified symbols

17:32 (that is, "a full macroexpand, like you were suggesting, followed by a code-walk...")

17:33 But I think it's just passed my "can-be-arsed" threshold, and I'm just mandating backquotes on args passed in :P

17:34 yrb: I agree :)

18:03 pjb3: When I am in the Clojure REPL

18:03 If I define a function that will run a function in a thread

18:03 like this:

18:03 (defn in-thread [f] (.start (new Thread f)))

18:04 When I call it like this:

18:04 (in-thread #(prn "hi"))

18:04 it doens't print

18:04 but as soon as I evaluate something else

18:04 like:

18:04 ()

18:04 Then I see the output

18:05 Any idea as to what causes that?

18:07 JamesIry: pjb3: just a guess, but could it be that the repl is buffering stdio so that output in other threads doesn't appear randomly as you are typing.

18:08 pjb3: yeah, I suppose that makes sense

18:08 JamesIry: Should be easy to test...make in-thread do something with file IO instead of stdio

18:09 Then go look at the file in a separate window

18:13 Chouser: always use (flush) after any debugging (prn)s

19:54 jcrites: hey, anyone know how I can convert a sequence to a vector?

19:54 rhickey: (vec your-seq)

19:55 jcrites: (vector my-seq) ?

19:55 I tried:

19:55 rhickey: no, vec

19:55 jcrites: my version doesn't have that. is that new?

19:55 rhickey: In the latest release 20080612

19:56 jcrites: let me get that one. thanks :)

19:56 rhickey: if you are on an old release, use (apply vector your-seq)

19:56 jcrites: hmmm

19:56 how can (f x) be different from (apply f x) ?

19:57 * jcrites is confused :(

19:57 jcrites: or is it just with values of f which are special forms?

19:57 rhickey: the last arg to apply must be a seq, which gets 'unrolled' into args

19:57 (apply + [1 2 3])

19:57 jcrites: ahhhh, that makes sense then

19:58 thank you

19:58 rhickey: vs (+ [1 2 3])

19:58 np

20:06 jcrites: what would be the right way to append an element to the end of a sequence? (returning a new sequence)

20:07 I guess I am too dumb to do it recursively with recur :)

20:08 hmmm maybe I can do it!

20:08 oh, nope, I can't...

20:09 i'm not sure it's possible to do tail recursively because the cons can't be called until after the recursion returns

20:09 maybe you could do it somehow by reversing the sequence?

20:09 slava: jcrites: (reverse (cons elt (reverse seq)))

20:10 jcrites: would you consider that better or worse than converting it to a vector, vector-appending, and converting to seq? (the last step which may not be much of a "conversion")

20:10 slava: well, the underlying issue here is that you're peforming an operation which the data structure is unsuited for

20:10 if you need to do this once or twice, no big deal, if you're doing it in a loop consider just using a vector

20:11 jcrites: I guess you're right. I'm thinking in the non-functional mindset

20:11 I'm trying to implement Sparsec/Parsec combinators in Clojure as an exercise actually :-) the DS I'm discussing will actually be my "parse string"

20:11 I was hoping to get around Sparsec's ParseResult and have parsers return nil on no-match

20:12 then that means I need an end-of-string symbol

20:12 so this question came up from how can I append :eof to (seq "some string to parse")

20:12 and it looks like I should just do something like (defn append [v e] (assoc v (count v) e))

20:12 * meredydd prods rhickey again re: website

20:12 meredydd: rhickey: The sourceforge redirects are broken from what I can see.

20:13 jcrites: they are totally broken :/

20:13 AFAICT a "/" got lost somewhere

20:13 slava: i don't think parsec needs to be implemented with an 'end of string' symbol

20:14 jcrites: no, it doesn't need to be, I just think it would make it more elegant

20:14 slava: the way its usually done is with a list monad

20:14 jcrites: I mean in Scala for example, oh, I guess you aren't necessarily familiar with that, I forgot

20:14 slava: i am, somewhat

20:14 jcrites: Scala has an implementation called Sparsec, with a structure called ParseResult which is a boolean and some stuff

20:14 I was hoping for my Lisp (Clojure) version to return nil/non-nil

20:15 slava: in haskell's parsec, parsing yields a lazy list of results

20:15 jcrites: the problem with that is, what if your parser matches the whole sequence?

20:15 slava: where a parse result is some object (otuput by action parsers) together with the rest of the sequence

20:15 jcrites: the "rest of the sequence" is nil, then, which is the same as the error case

20:15 since they need to be distinguishable, I was considering adding a magic character on the end, designated :eof

20:15 in that way, all parsers will always terminate with "next" pointing to the eof character

20:16 slava: the error case outputs an empty list of parse results

20:16 not a list containing nil

20:16 jcrites: so parsers cannot match nothing?

20:16 how do you match * ?

20:16 slava: they can, in which case you get a list contianing nil as the parse result

20:16 jcrites: ah, interesting

20:17 I hadn't thought of returning a list of parse results at all :) although I was pondering how to implement a "fork" combinator

20:17 slava: that's how parser combinators work

20:17 each parser returns a list of parse results

20:17 jcrites: it makes sense why the list monad then

20:17 slava: parser concatenation is 'append' lifted to hte list monad

20:17 jcrites: aren't most of the parsers deterministic though? why do they need to return a list of results?

20:18 the only reason I had considered a list of results was for a nondeterministic "fork" operator

20:18 oh, is the list of results items matched in series?

20:18 slava: well, consider a parser like (a*)(a*)

20:18 jcrites: I was thinking of them as alternatives up until now

20:18 slava: if you parse aaaa, there are 4 possible results

20:18 rhickey: meredydd: clojure.sourceforge.net goes to clojure.org, no?

20:18 slava: a|aaa aa|aa aaa|a

20:18 jcrites: rhickey: it's broken somehow, try some google searches

20:18 slava: oh, and |aaaa aaaa|

20:19 jcrites: so all of those are returned.... oic ...

20:19 slava: ( scratchpad ) "aaaa" "a" token <*> parse llength .

20:19 5

20:19 there are 5 results there, "", "a", "aa", etc

20:19 but if we insist that it must patch the whole string

20:19 ( scratchpad ) "aaaa" "a" token <*> just parse llength .

20:19 1

20:19 'just' filters out all parse results where the 'rest' is not nil

20:20 parser combinators are all operations on lazy lists

20:20 jcrites: interesting.... the Scala versions are a deterministic version of that, I guess

20:20 slava: then they're not parser combinators

20:20 jcrites: I am not very familiar with the original Parsec

20:20 slava: they're parsing expression grammars

20:20 pegs don't backtrack after a successful match

20:21 so you either get one parse result or nothing at all

20:21 jcrites: yes, that's how they are

20:21 slava: so for example, with pegs, (a*)a will never match aa

20:21 because (a*) 'eats up' the aa

20:21 and it doesn't backtrack after that, ever

20:21 jcrites: hmmm

20:21 slava: with parsec, it fails to match the second 'a', and backtracks, then suceeds

20:21 meredydd: rhickey: Yes, but the sub-URL forwarding is borked.

20:21 rhickey: http://clojure.sourceforge.net/reference/namespaces.html (just to pick one out of the air)

20:21 jcrites: thanks for the insight; I think you have saved me from some hours of bad design decisions ;-)

20:21 slava: jcrites: parsec parsers are generally less efficient

20:22 but they're more expressive

20:22 rhickey: some of them will be as they use subdirectories which my new wiki doesn't support

20:22 slava: however i'm pretty sure anything you can parse with parsec can be parsed with peg, except sometimes the peg grammar can get convoluted

20:22 jcrites: I was actually thinking of trying to generate a greedy | non-greedy fixed implementation from the combinators or something

20:22 slava: regular expressions are very hard to embed in peg for example

20:22 jcrites: you build up a combinator expression then serialize it, compile it, or something

20:22 slava: yes, in peg, everything is 'greedy' unless you add a lot of negations

20:23 in factor, we have parsec parsers, which are built with lazy lists and are inefficient, and peg parsers which compile to machine code but are a bit harder to use

20:23 jcrites: is that generally a problem if you were going to, oh, say totally hypothetically ;-) use it to parse a simple language?

20:23 meredydd: rhickey: I don't think they even get that far - I'm getting "Unknown host 'clojure.orgreference'"

20:23 rhickey: Looks like you're missing a '/;

20:23 *'/'

20:23 jcrites: meredydd: it's missing a /

20:23 slava: jcrites: well, pegs can be used to parse very complex languages

20:23 like javascript

20:23 rhickey: jcrites: ah, just a sec

20:24 jcrites: oic, a peg is what I want then :)

20:24 slava: and they're more efficient than parsec

20:24 jcrites: however, I was trying to add something quite odd

20:24 I was trying to add a "fork" peginator

20:24 (peginator, combinator)

20:24 slava: i'm not familiar with that type of thing

20:24 i didn't implement any of this stuff, i just use it a fair bit

20:24 jcrites: which would parse (depending on how I decide to do it) either both arguments, or one

20:24 (or fail and nothing)

20:25 there are some options for the specifics, but the general idea is to parse both alternatives, sort of like a real parser combinator

20:25 except both are valid syntactically

20:25 rhickey: jcrites: ok, you'll get a 404 now

20:25 jcrites: the two valid syntax trees would be the result of the parsing operation

20:25 the goal of this being to allow simple ambiguous grammar in a programming language

20:26 I know it's crazy; I just want to try it ;-)

20:26 one could imagine a Lisp where either (a + b) or (+ a b) are accepted

20:26 you could, if you wanted, represent both expressions by the same AST

20:26 slava: what if a evaluates to a function?

20:26 jcrites: or they could be different

20:26 if there is semantic ambiguity it would be an error

20:26 slava: at run time?

20:27 sorry, i don't like ambiguity in syntax :)

20:27 jcrites: I don't know as much about dynamic languages :) I was thinking of this from a static perspective

20:27 oh I know, it might be a horrible idea

20:27 imagine reading (f g h)

20:27 who knows what that means

20:27 g(f,h)?

20:27 f(g,h)?

20:27 or maybe it even means h(f,g)! :)

20:28 I wasn't expecting this particular example to be a good use of ambiguous syntax

20:28 I was thinking maybe about using [ ] to specify a variety of similar data structures or whatnot

20:28 the two alternative ASTs might be very different and easily distinguishable by the programmer

20:28 that would be a better usage of ambiguous syntax

20:29 my goal was just to not not have that option because of technical limitations in the parser ;-) I think I can write semantic analysis algorithms to handle it intelligently if I can support nondeterministic parse trees

20:29 but only experimenting with the language feature will tell me for sure if it's a horrible idea, or one worth investigating ;-)

20:30 where the program expression mirrors a very numerical form, I might think of it as OK.... (a+b)*(c+d) or something

20:30 (in an otherwise Lisp program)

20:30 I'm also going to try ambiguous syntax by omission

20:31 you could imagine a Lisp program where many parentheses are simply omitted, inferred by other means such as indentation or newlines

20:31 in that case, one way I could implement it is via ambiguous syntax... I'm not sure exactly how to do that without an explosion of AST cases, but the general idea makes sense I think

20:40 rhickey: meredydd: links better now?

20:40 meredydd: Much, thanks :)

20:40 rhickey: thanks for the heads up

20:50 drewr: Is trunk@906 broken?

20:51 Exception in thread "main" java.lang.NoSuchMethodError: clojure.lang.RT.cons(Ljava/lang/Object;Lclojure/lang/ISeq;)Lclojure/lang/ISeq;

20:55 slava: do you guys have continuous integration?

20:56 drewr: We need to get rhickey on a dvcs.

20:57 These "interim checkins" are unnecessary.

20:57 meredydd: drewr: I dunno...I like being able to experience something he's playing with

20:57 The real problem is that svn sucks at branch-and-merge

20:57 jcrites: a dvcs is not necessary for development commits to be made on a development branch ... eh ?

20:57 drewr: You'd still get the history. He could just push it when it's functional.

20:57 jcrites: what sucks about it?

20:58 slava: we have a bunch of machines which compile and run tests, if everything passes they tag the revision they just built

20:58 dudleyf: jcrites: Merging is way harder than it needs to be

20:58 slava: so people can check in broken stuff and users can just pull the 'clean' tag

20:58 when you support 11 architectures its unrealistic to expect every developer to test on every platform before checking in changes

20:58 jcrites: what's harder about it? forgive me, I haven't used svn very often in any kind of production environment

21:00 meredydd: jcrites: You need to specify revisions manually. You can't do sensible things like "merge all changes made to meredydds-devel since it diverged from trunk"

21:00 dudleyf: svn makes you track down the branch point manually, then apply a diff, then fix any conflicts, then commit

21:00 jcrites: oh, so effectively you can only merge revisions, and not branches?

21:00 that does sound painful.

21:00 meredydd: jcrites: What you do, effectively, is get a diff between two revisions on one branch, and apply it to another.

21:01 So, I'd take the diffs between 503 (when I branched) and now, on my branch, and apply it to trunk. That sort of thing.

21:01 jcrites: I wonder what their rationale is for that kind of merge behavior .... ?

21:03 meredydd: "It's one hell of a lot better than CVS" :D

21:03 I don't think it's that bad, actually - if there were sensible automated ways of figuring out which revisions you wanted to take, I think I'd be quite happy with it

21:04 jcrites: what happens if you specify an earlier revision? does the merge get all messed up, or just fail?

21:04 (a revision already merged, say)

21:04 meredydd: Uhh...it tries as best it can.

21:05 It's exactly like applying a patch from the wrong revision

21:09 drewr: Ah, 906 isn't broken. I had to ant clean myself of the cruft of many revisions ago.

21:18 slava: ant still doesn't support automatic recompilation of dependencies? :)

21:58 pjb3: I'm working on adapting the server socket repl example from the wiki into a toy web server

21:58 http://pastie.org/215618

21:59 Weirdest thing though, if I redefine the handle-request function and just change the value of msg

21:59 the next request I make from a web browser uses the old version of msg

21:59 but the 2nd request uses the updated version

Logging service provided by n01se.net