#clojure log - May 07 2010

The Joy of Clojure
Main Clojure site
Google Group
List of all logged dates

0:09 rdsr: help

0:09 exit

0:09 quit

0:13 defn: rdsr: /quit

0:18 technomancy: who was asking about pretty representations of fn objects?

0:18 http://github.com/technomancy/serializable-fn

0:18 ^_^ check it out

0:20 defn: technomancy|away: awesome!

0:23 joshua-choi: Is there an RSS feed for Clojars?

0:36 arohner: technomancy|away: that's really clever

0:49 tomoj: hmm, I get an "Unmatched delimeter: )" trying to slime-connect to the lein swank I get in that project

0:50 remleduff: I think that's clojure-test mode or whatever the test mode is called

0:51 tomoj: oh fuck

0:51 now I can't connect to a mvn clojure:swank in my other project

0:51 hiredman: tomoj: yeah, you have to grab the latest clojure-test-mode from github

0:52 tomoj: hiredman: technomancy's?

0:52 hiredman: yeah

0:52 tomoj: hasn't changed since april it looks like

0:52 hiredman: you can just replace the clojure-test-mode you have in ~/.emcas.d/epla/ with it

0:52 tomoj: why did this problem just now happen to me, I wonder?

0:53 hiredman: I had this error too, and replacing test-mode with test-mode from git fixed it

0:53 and I got this advice from technomancy

0:53 ,Q

0:53 clojurebot: <-nil-<

0:54 tomoj: yep, fixed it, thanks

0:54 still wonder why, though..

0:57 remleduff: Going by the github commit, he was using ^ to mean (meta) in one place, which broke with the 1.2 changes

1:00 tomoj: is it totally gone now, no longer just deprecated?

1:01 remleduff: Yeah, ^ is for type hints now

1:01 tomoj: oh, like ^String ?

1:01 cool

1:01 remleduff: Yep

1:01 tomoj: I tried that earlier and it didn't work

1:01 so I must have just gotten a 1.2.0 update which caused the error

1:04 joshua-choi: remleduff: Does #^ still work in Clojure 1.2 for now? Is it deprecated?

1:05 remleduff: #^ is still ok

1:05 Only ^foo -> (meta foo) is gone

1:05 I guess #^ must be deprecated if only because it's ugly ;)

1:07 tomoj: I'm glad type hints will be prettier

1:07 I could never really remember how to do them

1:10 remleduff: I guess #^ will probably stick around for quite a while if library try to support both 1.1 and 1.2

1:23 DuneMan: darnit

1:24 something is holding onto the head again

1:24 *watches memory rise*

1:24 this is becoming a real problem....

1:24 hiredman: pastebin your code

1:24 DuneMan: So, what is the correct way to write code using infinite seqs to stay away from such things?

1:24 hired: It's spread over 7 files

1:25 and does a bunch of stuff

1:25 hiredman: DuneMan: if you can't reduce it to a simple test case there isn't much anyone can do for you

1:25 we will just get annoyed listening to you complain about something you aren't able to demonstrate

1:25 DuneMan: There are, however, best practices when dealing with such things.

1:25 hiredman: the best practice is don't hang on to the head

1:26 most likely you are inadvertently doing it

1:26 DuneMan: They can tell you what sorts of operations do this often... e.g. (apply (partial ...) infinite-sequence) ; badness

1:27 I know I am. And I'll find this specific case, again. Just wondering if there's a general form that makes it happen less often. Like doing (lazy-seq (...)) and recursing?

1:27 hiredman: it depends on the functions you partial

1:27 ,(apply first (cycle [[1]]))

1:28 clojurebot: Execution Timed Out

1:28 remleduff: When did fine grained locals clearing get added, is that 1.2 or was it 1.1?

1:28 DuneMan: actually, it doesn't depend on the function you partial. Because partial take [& more]

1:28 * hiredman punches clojurebot

1:28 hiredman: DuneMan: it does dpeend on those functions

1:28 DuneMan: the functions returned by partial*

1:28 joshua-choi: remleduff: I believe it was 1.1.

1:28 hiredman: joshua-choi: no

1:28 joshua-choi: Oh dear

1:28 Is it being added in 1.2?

1:28 hiredman: I belive it wasn't

1:28 DuneMan: I was seeing massive problems in 1.1, and going to 1.2 made things a lot better.

1:28 hiredman: yes

1:29 DuneMan: 1.2 at least made it so every case I've seen of this happening has been my fault.

1:29 hiredman: the change was invasive to the compiler and made close to the release of 1.1 so it was decided to wait until 1.2

1:29 ,(apply (fn [& _] 1) (iterate inc 0))

1:30 clojurebot: 1

1:31 hiredman: DuneMan: (partial head-holding-function 1) <-- resulting function still holds on to the head

1:32 DuneMan: *looks to see if he is still using the code that leaked memory with partial but not withour*

1:32 tomoj: are there still cases where you're holding onto the head but even if you meditated on the problem code for a while you wouldn't notice unless you had some deeper knowledge of the compiler?

1:33 hiredman: dunno

1:33 it used to be possible to do it without knowing by using destructuring in a loop binding

1:35 DuneMan: I'm no longer using the code that was head-holding with (partial...) and apparnetly I didn't do a commit with that version

1:36 too bad I'd like to see what my function was doing that was holding the head.

1:36 I fixed it by replacing (partial...) with my own function that returned a function of 1 arg instead of variable args

1:36 paraseba: Hi all. What's the idiomatic way to insert an element in a vector?

1:36 DuneMan: and using it to wrap the same function.

1:37 paraseba: I mean insert an element in the middle of a vector

1:38 DuneMan: It'd be useful to find a good tool for figuring out what was holding the head. Sorta hard with all the java tools because they assume a meaningful class name for things holding references.

1:38 any suggestions there?

1:38 joshua-choi: paraseba: Actually, it's idiomatic to not use vectors if you need to insert in the middle of it

1:38 I think

1:39 hiredman: insert can mean different things

1:39 do you mean grow a vector by inserting into the middle, that is messy

1:39 paraseba: hiredman: yes, exactly that

1:39 hiredman: don't do that

1:40 * joshua-choi wonders about finger trees

1:40 hiredman: if you are doing that you are doing it wrong

1:40 paraseba: I know it sounds weird, the fact is that I'm writing a macro that modifies the result of a function which already returns a vector

1:40 hiredman: it's not weird, it is bad

1:40 paraseba: I need to return a new vector with an added element

1:40 hiredman: vectors "grow" in one place

1:40 the end

1:42 paraseba: hiredman: vectors are short, and part of a templating system, changing vectors to other structure is not an option. This is some sort of wrapper which gets the result and add an extra element in the middle

1:42 but I guest the answer is "there is no idiomatic way ... no one ever does that stupid thing"

1:43 it makes sense ...

1:43 hiredman: paraseba: how is not an option?

1:43 joshua-choi: paraseba: If you really need to (and since it's a macro, performance doesn't matter that much anyway), I'd use subvec twice with into.

1:43 hiredman: changing vectors into other structures is done usually without thinking about it

1:43 ,(map inc [1 2 3])

1:43 clojurebot: (2 3 4)

1:44 hiredman: ,(vec (concat [1] [2] [3]))

1:44 clojurebot: [1 2 3]

1:44 joshua-choi: hiredman: Incidentally, is there a function for lazily inserting something into the middle of a seq?

1:44 hiredman: nope

1:45 paraseba: hiredman: yes, but that's exactly my question ... there are several ways in which I could transform the vector to a seq, add the element and transform the result back to vector

1:45 arohner: joshua-choi: you'd need to evaluate up the start of the insertion point

1:45 joshua-choi: arohner: If there was a way to pass on the current index, perhaps not...

1:46 hiredman: arohner: no you wouldn't

1:46 ,(let [x (iterate inc 0)] (take 10 (concat (take 3) [-1] (drop 3))))

1:46 clojurebot: java.lang.IllegalArgumentException: Wrong number of args passed to: core$take

1:46 hiredman: ,(let [x (iterate inc 0)] (take 10 (concat (take 3 x) [-1] (drop 3 x))))

1:46 clojurebot: (0 1 2 -1 3 4 5 6 7 8)

1:47 joshua-choi: Aha

1:47 hiredman: there may be one in seq-utils, or whatever it is called now

1:48 joshua-choi: .seq, I believe

1:55 LauJensen: Morning all

2:00 DuneMan: So, my infinit seq is actually just: (repeatedly #(.take queue)), and I guess since the caller is what is doing the (repeatedly...), it happens that its holding the head somehow... even though it's not doing it in a let-binding or anything. This whole sequence is consumed in a single (doseq [e es] ....)

2:01 If I replace this with passing the queue (a LinkedBlockingQueue) into my handler function, and doing (loop [] (println (.take queue)) (recur)), it indeed does not kill memory.

2:03 hiredman: DuneMan: what is es?

2:04 DuneMan: I think I can pastebin the functions up to that and it will make sense. 1 sec.

2:13 ('es' is passed into the function as an argument, (handler-function (repeatedly #(.take queue)))

2:13 http://pastebin.com/L9ubTrBq <--- no memory leak with doing the .take within the innermost function

2:18 .... and now I reverted my code and the memory usage is stable.

2:19 http://pastebin.com/1WWyccYf

2:19 not my day, apparently.

2:22 tomoj: double spaced?

2:22 wtf?

2:22 DuneMan: Maybe the memory growth was just the JVM not being able to keep up with GCing stuff... I had a rogue process on the box eating 7gb of ram and a lot of CPU... and this thing is reading in a few megs a second.

2:22 Yeah, I have no idea why pasebin did that?

2:23 tomoj: that is very disorienting to me

2:23 DuneMan: it looks aight in the text box when I pasted it

2:23 tomoj: I wonder if they do that all the time?

2:24 DuneMan: I also wonder if implementing this with agents would perform.

2:25 hiredman: you should try paste.lisp.org or gist

2:25 DuneMan: oh, queue-seq is a macro that expands to (repeatedly #(.take queue))

2:25 ah, good to know.

2:26 and now this is humming along swimmingly and the jvm+gc is getting warmed up and memory is dropping

2:26 .... maybe it was just not keeping up.... goddamn.

2:30 have any of you used agents to implement stream-type processing?

2:31 Where you are reading a stream of messages which are ordered and need to do processing on them in a potentially order-dependent manner?

2:32 the model didn't seem to fit quite right, but I'm not sure I totally understand yet :-)

3:22 sids: what is the meaning of the '-SNAPSHOT' in version numbers of clojure and other clojure projects?

3:23 spariev: sids: thats maven thing

3:24 SNAPSHOT is a special version in maven that indicates the latest code; typically TRUNK or HEAD in your source control.

3:24 With this version, maven will automatically grab the latest SNAPSHOT every time you build

3:24 leininigen follows this convention too

3:24 sids: spariev: thanks. So if I'm going to release a library version 1.0, say, I should push 1.0-SNAPSHOT to clojars?

3:25 spariev: sids: if youre planning to update this release frequently, maybe you should

3:25 sids: and bug fixes for the 1.0 release should also be pushed with the same 1.0-SNAPSHOT version? did I understand this right?

3:26 spariev: I think yes

3:27 sids: thanks

3:27 DuneMan: It seems to me that you should branch for you release versions

3:27 and also push a snapshot version of latest-version+bump

3:27 spariev: but once you have your codebase stabilized, you should drop SNAPSHOT

3:27 DuneMan: er, release-version+bump

3:29 sids: hm, this is a bit confusing. Can I update a release-version-SNAPSHOT with bug fixes or do I push a new release-version-bump-SNAPSHOT?

3:33 spariev: I would go with the release-version-SNAPSHOT if you are going to have frequent changes and bugfixes, so your users won't have to change their dependencies every time bug fix arrives

3:33 remleduff: In general for code you're making available to other people, it's better to just do released versions like 1.1, 1.2, it's not like you're going to run out of numbers

3:33 SNAPSHOT is really meant for local builds, it's being abused by our community somewhat

3:34 DuneMan: And then you should push bug fixes in bugfix bumps

3:35 I think the leiningen git page links to a recommended version style

3:35 sids: thanks everyone

3:35 DuneMan: indeed it does

3:35 http://semver.org/

3:35 sexpbot: "Semantic Versioning"

3:36 sids: yeah, I read that, but it does not mention anything about -SNAPSHOT

3:37 DuneMan: -SNAPSHOT just means that you took the current head of your repo, in your development version number, and packaged it.

3:37 it may even not work.

3:38 though probably should :-)

3:38 remleduff: It actually resolves to a timestamp-ish looking thing, like clj-json-0.2.0-20100215.165114-2

3:39 That's actually clj-json-0.2.0-SNAPSHOT from when I downloaded it

3:39 sids: I'm just publishing a lein plugin to clojars; lein uses the -SNAPSHOT, so I'm going to use it too, for now :)

3:40 remleduff: that is very informative. How did you find that out?

3:40 DuneMan: That's what will be in your /lib directory.

3:40 when you do lein deps

3:40 sids: oh, never looked there, makes much more sense now. Thanks!

3:42 remleduff: In maven world, once you release a non-snapshot version, you're never supposed to change that version again because people are supposed to be able to depend on it. SNAPSHOTS can change without warning

3:42 In clojure world, people tend not to release often enough so lots of people end up running SNAPSHOT versions

3:45 sids: makes a lot of sense, not that I understand that SNAPSHOTs are really pointers, not fixed releases

3:45 s/not/now/

3:45 sexpbot: makes a lot of sense, now that I understand that SNAPSHOTs are really pointers, now fixed releases

3:46 DuneMan: I do like that feature of irc bots.

4:18 zmila: s/do/don't/

4:18 sexpbot: I don't like that feature of irc bots.

4:18 zmila: hohoho

4:53 LauJensen: test

4:53 s/test/$(println "test")

4:53 s/test/$(println "test")/

4:53 Just wondering if he read what he wrote, and if he did, if he had access to more functions than I do :)

5:46 jwr7: So I'm trying to move my deftests out of my code and into the test/ directory (using leiningen), and I just discovered that if I do this, my test namespaces can't access private symbols defined using defn-, and so my tests won't work.

5:46 How do people get around this?

5:46 In fact, I'm not entirely sure I want to move my tests out into a separate tree — what I need is a way to run them all using lein.

5:48 AWizzArd: jwr7: can your tests go (in-ns ns-with-private-fns)?

5:48 (in-ns (quote ns-with-private-fns)) I mean

5:49 jwr7: AWizzArd: I guess so — so instead of declaring a new namespace for my tests, I could just… but then I also have to load the ns-with-private-fns before, I guess.

5:51 sids: jwr7: I create a test dir and then symlink my tests into that

5:55 jwr7: java.lang.Exception: Unable to resolve symbol: use in this context. I guess my in-ns placed me in an incomplete namespace somehow?

6:04 Ok, got past that (typo), but now lein doesn't seem to find my tests. lein test says "Total: {}".

6:10 AWizzArd: ok, that won't work. lein uses clojure.contrib.find-namespaces to find namespaces in test directories, so you actually need to define a NEW test namespace, not use an existing one.

6:10 Which brings me back to my original question — phrased differently, is there a way to :use a namespace WITH private symbols?

6:21 LauJensen: jwr7: Does it make sense to test privates only? I expect them to be wrapped by something publicly exposed, which makes more sense to test?

6:22 jwr7: LauJensen: good question, I pondered it as well. Still, those *are* supposed to be *unit tests*, right? And in my case, I very often have private functions, whose behavior is best described by tests.

6:22 If I go a level up, my tests will confuse various functionalities and become large, unwieldy and hard to maintain.

6:22 LauJensen: jwr7: if I make (defn add3numbers [x y z] (+ x y z)) - I test that with pos, neg, nil etc, but I dont separate unit-test "+"

6:23 s/separate/separately/

6:23 sexpbot: jwr7: if I make (defn add3numbers [x y z] (+ x y z)) - I test that with pos, neg, nil etc, but I dont separately unit-test "+"

6:24 jwr7: LauJensen: of course. But in case of more complex packages you often have internal functions with non-trivial functionality.

6:25 LauJensen: that cannot be validated by seeing succesful tests on the top-level fns ?

6:25 ie. where its possible to have a private regression which doesnt affect the public fn ? is that even a regression? :)

6:25 jwr7: LauJensen: as an example, think of a text search package. You'd have externally accessible 'search' and 'compute-index' functions and internal scoring functions. You don't expose the scoring functions to the external world, but you want to test them separately.

6:25 LauJensen: and I can definitely achieve lots of regressions in the private functions :-)

6:26 LauJensen: jwr7: You could be right, Im just saying I wouldnt start hammering out those tests up front :)

6:26 jwr7: I know you can - Im just saying, that'll show up in the higher levels as well

6:26 jwr7: LauJensen: oh, sure. My approach to tests is very pragmatic, I try to do as few as possible and still get the benefits.

6:26 LauJensen: ie, if you break score in a way, which still produces all the correct results, for all the tests on search, then who cares?

6:27 jwr7: LauJensen: they *will* show up in the higher levels. But then you don't have unit tests anymore. You have higher-level tests, and if they fail, you have to investigate where is the actual problem. Plus, these tests tend to get difficult to maintain (from my experience).

6:28 LauJensen: trick is, you don't want to hardcode the "correct results" on a higher level. You want them to be as close to the actual code as possible. That's the idea of unit testing, as opposed to component testing.

6:28 I'm not saying the latter isn't useful, of course.

6:28 LauJensen: Yea I see what you're saying

6:28 And I guesss the higher levels fns, often fit more into integration tests, than unit tests

6:29 jwr7: LauJensen: right.

6:29 LauJensen: as another example, I used to develop weblocks, a CL web framework. Higher-level tests always ended up being "run this thing and compare the resulting HTML to this string". Completely unmaintainable in the long run.

6:30 You would change a tiny function and then have to modify 100 test strings.

6:30 LauJensen: painful

6:31 Chousuke: hmm.

6:31 jwr7: I tend to think of unit tests as an extended docstring, of sorts — they document the corner cases. Hmm, perhaps separating them from the functions isn't a good idea after all?

6:31 Chousuke: I'm more partial to the idea of writing only regression tests :P

6:32 at least with languages like clojure

6:32 where it's easy to test every function after you've written them.

6:32 DuneMan: I am hopeful I've figured out how to write correct code with lazy seq

6:32 memory isn't growing... woo

6:33 jwr7: Chousuke: I disagree, but perhaps I tend to introduce more bugs :-)

6:33 LauJensen: Chousuke: Im with you on that

6:33 Chousuke: something like QuickCheck is useful too though.

6:33 jwr7: Chousuke: regression tests don't capture the corner cases, as those won't make it through the higher-level functions. So they don't future-proof your code, as unit tests do.

6:33 Chousuke: where you have a dumb but certainly working reference implementation and a test data generator

6:34 jwr7: unit tests don't help either if you don't write the correct ones :/

6:34 jwr7: Also, mind you, I'm not a test fanatic — I don't write tests for everything, only for non-trivial functions. My tests are essentially a saved REPL session.

6:34 Chousuke: true, but writing them helped me discover problems with corner cases. I guess it's a way of disciplining myself to check.

6:36 LauJensen: jwr7: corner cases? You know Clojure is Functional right? Which corner cases do you mean

6:36 jwr7: LauJensen: passing an empty sequence, which causes a division by zero later on

6:37 AWizzArd: Testing pure functional code is not very important (+, fibonacci, str, assoc, etc.). But the state changing parts should indeed be tested. Tests can also help to develop a useful API.

6:38 LauJensen: jwr7: (defn divme [x y] {:pre (not (zero? y))} (/ x y))

6:38 there's a time and a place for everything

6:39 jwr7: LauJensen: Oh. In my case (a scoring algorithm) I had to special-case this and return a score of 0.0. So it wasn't really a precondition.

6:39 I don't test functions with are a simple composition of clojure or library stuff.

6:40 So if all I do is reduce, map, into, etc — I don't need tests for that.

6:41 Chousuke: I suppose writing tests is an improvement; at least you're tempted to think "tests pass, ship it" instead of "it compiles, ship it"

6:41 DuneMan: It's sorta confusing writing functions that work right with lazy-seq... but I think I'm getting it. A bit convoluted to have a function that returns a lazy-seq in which is defined a function which is recursive with itself, and as a base case returns its value and then is recursive with the outer function.

6:41 but it works, and it doesn't consume tons of memory. and is ... concise.

6:41 jwr7: Chousuke: yeah, that too. But for me there are two advantages: 1) they form an extended specification and 2) they help me discover corner cases I haven't thought of. Regression testing is actually an afterthought in my case.

6:45 DuneMan: jwr77: While I agree testing can work well that way, I think it can still happen in that workflow with keeping tests in a seperate file.

6:46 I think its particularly important to have them separated when you go in and start hacking on code to change functionality.

6:46 It becomes to easy to break test code.

6:46 too*

6:47 errant find/replaces, and whatnot... the same types of bugs that break other code... the code you want the tests to be sanity-checking (in the regression case)

6:56 g'night all, thanks for the help all day.

6:59 Borkdude: defn: here?

7:22 I was searching for a nice example with -?>

7:22 But this doesn't work: http://getclojure.org:8080/examples/-?%3E

7:22 http://getclojure.org:8080/examples/-?&gt;

7:26 I was thinking smth like this:

7:26 (-?> {:a {:a2 "A"} :b "B"} :a :a2)

7:26 vs (-?> {:a {:a2 "A"} :b "B"} :b :b2)

7:26 but that's kind of what get-in is for

7:31 it's a terrible macro name to search for on the internet

7:57 sids: Borkdude: (-?> #"hello" (re-matches "hello") .toUpperCase)

7:57 Borkdude: vs (-?> #"hello" (re-matches "goodbye") .toUpperCase)

8:09 Borkdude: sids tnx

8:11 sids, I will mention your example on @cleotd ok?

8:25 jowag: If I close over a java object, and I do not keep other references to that object other than from the closure itself, will the object be garbage collected (bad) or it will be garbage collected only after I let go the reference to the closure (good)?

8:28 chouser: jowag: the latter

8:29 jowag: no worries, you're all set.

8:32 jowag: thank you

9:11 fogus: ,Q

9:11 clojurebot: <-nil-<

9:12 Borkdude: ?

9:13 cgrand: Borkdude: http://clojure-log.n01se.net/date/2010-05-06.html#16:02

9:13 sexpbot: "#clojure log - May 06 2010"

9:14 cgrand: $"http://clojure.org/&quot;

9:14 sexpbot: Command not found. No entiendo lo que estás diciendo.

9:14 cgrand: $(identity "http://clojure.org/&quot;)

9:14 sexpbot: result: http://clojure.org/

9:16 Borkdude: chouser: I like the fish skeleton

9:21 fogus: I couldn't resist adding the fish queue to the book last night.

9:22 Borkdude: clojure powered ascii art, the beginning of a new genre

9:23 fogus: LauJensen already blazed that trail. http://www.bestinclass.dk/index.php/2010/02/my-tribute-to-steve-ballmer/

9:23 sexpbot: "My tribute to Steve Ballmer | BEST IN CLASS"

9:29 Borkdude: hmm, that is ascii photographism, not as exciting as handcrafted works imho ;)

9:31 chouser: I love the stunning lack of type hints required in programs that use protocols extensively

9:33 LauJensen: Conversely I love the astounding lack of protocols, in programs which use type hints extensively

9:33 chouser: :-(

9:35 LauJensen: sorry, just kidding

9:35 But I've yet to see a problem which really leverages protocols

9:39 cemerick: cgrand: that's a horrible tale. :-(

9:40 cgrand: cemerick: what? my tweet? yeah :-(

9:40 cemerick: yeah

9:40 chouser: noooo

9:40 cemerick: I don't think there's a pithy slogan in "bring back the :use errors"

9:40 defn: Borkdude: oy, i don't think that's on the whitelist of functions for the sandbox

9:41 chouser: cgrand: he still gets a warning, right? He's just going to ignore it?

9:41 cemerick: chouser: programmers have been trained to ignore warnings since the 7th day.

9:41 cgrand: apparently yes

9:42 jfields: is there a better way to do this: (+ (or x 0) quantity) - where x could be nil, but quantity never would be.

9:43 elean: ha I discovered the stack overflow in my code.. funny thing, if you got ref on vector containing the ref itself it may not be that easy to print it

9:44 cemerick: cgrand: you should post on that on the dev list or something -- a good horror story might be a good motivator. :-)

9:44 chouser: elean: should be able to set your *print-level* to avoid problems with that

9:44 cemerick: the solution would be :as-of ?

9:45 AWizzArd: jfields: what you have is a good solution

9:45 elean: chouser: wow nice thanks

9:46 jfields: cheers.

9:47 cemerick: chouser: I've been mostly absent from the :as-of and last-var-wins discussions, but the latter definitely rubs me wrong. :as-of is version sniffing IIUC, which is always bad IMO (and wouldn't help w.r.t. third party libs?). As I understand the problem, it seems that being smarter about definitions in general is the best route, perhaps with feature-conditional evaluation (as opposed to version sniffing).

9:48 I proposed the former yesterday, but I'm guessing it's too complex to tackle for 1.2.

9:48 chouser: http://groups.google.com/group/clojure/msg/86384d7e6555c024

9:49 chouser: cemerick: yeah, I raed that message a couple times. I don't think I understand yet exactly what you're proposing.

9:51 cemerick: chouser: it's a significant complication to the reader if namespaces aren't held in refs, or something similar. i.e. "load" the namespace without def'ing anything, and see if any defs will clash with referred vars; if so, bail, otherwise, proceed. This makes it trivial to predefine all the vars in the ns, eliminating the need for declare. This was a hot issue a long while back.

9:52 If namespaces were in refs, then it gets a *lot* easier -- just create the bare ns in a ref, and attempt to load it within an alter.

9:53 If there's no error, then you're good -- otherwise, all the namespaces are left in a pristine state, so you can't get into trouble at the REPL either.

9:53 chouser: but isn't the whole reason for address this now the problems caused by simply bailing when two :use'd libs clash?

9:54 cemerick: Well, that problem's been around forever. I think the latest spate of activity is to give some relief to people whose var names are being shadowed by core in 1.2 (and later releases).

9:55 cgrand: chouser: to me if two :use clashes the user deserved it, my only concern is about clashes with core

9:55 chouser: right. that's the problem we're trying to solve, and I'm not seeing how running through the lib ahead of time helps with that.

9:55 cemerick: If you're doing some work with var name checking before actually def'ing anything, you can ensure that you don't refer in any conflicting names -- the local one can always win, without any last-first-in-the-middle-wins behaviour.

9:56 chouser: if two :use clauses clashing is the problem being solved, that's the wrong problem to solve IMO.

9:56 cgrand: cemerick: to me :as-of is not some version sniffing, it's just a shortcut to says (:refer-clojure :only [vars known at the time of writing]) because this only is tedious to maintain by hand

9:57 cemerick: I thought clashes with new vars in upstream libs was the real target.

9:57 cgrand: it's roughly the same as doing (if (= "1.2" *clojure-version*) (refer ......)) *shrug*

9:58 But, I don't have a lot of ground to criticize :as-of. It seems like a very limited mechanism though -- e.g. not useful by third-party libs.

10:00 The reordering of :use declarations though -- that's just evil. Underhanded. Sneaky. Must be killed with fire.

10:00 ;-)

10:01 cgrand: on 3rd part lib: unless they start adding :added metadata -- but anyway you shouldn't use without :only a ns you don't own

10:02 cemerick: cgrand: I mostly agree with you. c.c.core and c.c.def are exceptions IMO. Others may have different ones.

10:03 chouser: cemerick: local winning seems like a plausible goal, but if core and some other lib clash, which is "local"?

10:03 cemerick: chouser: you mean if someone is :using the other lib?

10:03 cgrand: ok, I'm going to lobby lpetit to make ccw automaintain (:refer-clojure :only []) :-)

10:04 chouser: right, isn't that what's causing these problems?

10:04 (:use [clojure.contrib.seq :only [reductions]])

10:05 cemerick: sure; I suppose in that case, "local" means "what you use and define wins".

10:06 That's not going to help someone that :uses two libs that have the same name def'ed, but again, I don't think that's a problem to be solved. Or, it's a social, rather than a technical problem.

10:07 chouser: so at the repl, it would simply be impossible to define or refer 'reductions' without first removing it from your ns?

10:08 cemerick: chouser: why does that follow?

10:08 chouser: well, how else would you handle it?

10:08 defn: cgrand: (add-classpath "http://github.com/Lau-of-DK/clojureql/raw/master/src/&quot;); github

10:09 cgrand: very cool.

10:09 chouser: you're in 'user', user/reductions already exists (points to clojure.core), and now you refer c.c.seq/reductions. what happens?

10:09 previously, you get a fatal error.

10:10 currently, last one wins (user/reductions starts pointing to c.c.seq/reductions)

10:10 cgrand: defn: this ugly hack still work?

10:10 defn: cgrand: don't know -- i thought it was a recent post because you don't date your posts you just put the time on it

10:10 cemerick: bah, I need a pre-last-var-wins repl

10:11 defn: utilities — cgrand @ 19 h 34 min

10:11 cgrand: defn: oops

10:11 cemerick: anyway, yeah, the REPL is a slightly different use-case than whole-file loading. There, last-var-wins probably should hold sway.

10:11 controlled by a var set in the REPL, presumably.

10:12 defn: cgrand: will there be a new release of enlive and moustache in the near future for 1.2?

10:13 cgrand: (on clojars)

10:13 cemerick: chouser: hrm, no, I take that back. last-var-wins is just bad.

10:14 I fiddle around with a fn for a while, get it into just the right condition, and then run a blanket use that clobbers it (and who knows what else?). No thanks.

10:16 last-var-wins changes use from a slightly icky but useful thing, into a dangerous beast. ;-)

10:17 cgrand: defn: posts have date now (no more need to look at the permalink)

10:18 and I will do releases targetting 1.2 or targetting both 1.1 and 1.2 (I haven't looked at version ranges yet)

10:19 defn: cgrand: awesome. i've been meaning to upgrade to 1.2 for a couple projects but have been holding out for a stable 1.2 release

10:19 cgrand: thanks for the cool tools. :)

10:19 elean: am I correct assuming it should do the trick of limiting print level ?: (binding [*print-level* 0] [[1 2] [2]])

10:21 chouser: hm. Namespace.mappings is already an atom. If it were a ref, most usage could be the same as before, but some situations (like 'load') could run in a transaction.

10:22 cemerick: chouser: you should turn pro someday. :-)

10:22 chouser: within the transaction, replacing a core mapping with a local def or with a new mapping could be allowed, but replacing a non-core mapping with anything disallowed.

10:22 cemerick: heh. I have no idea what that means!

10:23 cemerick: chouser: I've been big on the sports metaphors lately. And, I'm running on little sleep today. :-)

10:23 chouser: wait, why is a transaction needed?

10:23 cemerick: chouser: not needed, but using a transaction means that, if a transitive dependency four layers down fails to load, your environment is left in the state it was in before.

10:23 chouser: we used to use an atom and fail. we could just do the same, but fail in slightly different circumstances.

10:24 we don't have transitive refer mappings though

10:24 the only think changing your namespace is you.

10:24 thing

10:25 cemerick: Yeah, I don't mean for refers, I mean for loads that fail.

10:25 chouser: loading a lib doesn't change your namespace

10:25 so if it fails your namespace is still unchanged.

10:26 you might have been planning on refering some thing when it was done loading, but if it failed you probably shouldn't. :-)

10:27 cemerick: Right, I know it doesn't impact the current namespace, but load certainly impacts other namespaces. Partially loaded libs have been a source of pain occasionally.

10:27 but that's a whole separate topic :-)

10:27 chouser: this doesn't prevent you from doing something like using a fn from core, then redefining it locally and using that.

10:28 cemerick: Yeah, local defs and :usages should always win.

10:28 hrm

10:29 chouser: I can't see any way to prevent that without brining in all the pain we're trying to avoid.

10:29 cemerick: No, that's fine.

10:29 But local defs should *always* win, and a conflicting use should fail.

10:30 i.e. (def reductions 5) (:use [c.c.seq :only [reductions]]) should error

10:31 chouser: sure, I'm saying (print 123) (def print println) (print 456) would succeed, but the two uses of 'print' would do different things

10:32 cemerick: yeah, I think I'm OK with that.

10:32 clobbering referred vars is OK -- you can always recover from that. Clobbered local vars are gone forever.

10:33 chouser: so if partially-loaded libs is a different topic, I think this might be a fairly simple change.

10:33 the question is if everyone (well, the right people) would agree it's desirable.

10:33 cemerick: heh

10:34 well, last-var-wins is simply bad. Hat tip to cgrand for the penultimate example.

10:34 * cemerick better get some snapple before he gets too riled up. :-P

10:34 defn: snapple just fuels my rage

10:35 cemerick: defn: crack a chilled diet raspberry, and you'll be as relaxed as can be ;-)

10:36 defn: :D

10:36 chouser: this does special-case 'clojure.core' in a few more places. That used to only be special in the ns macro.

10:36 cemerick: chouser: does it have to, though?

10:37 chouser: to do what you've described, yes.

10:37 cemerick: ah, yeah, nevermind

10:38 chouser: the alternative would be to allow a later :use to override an earlier one, which would allow cgrand's nightmare.

10:38 cemerick: yes

10:39 it's everyone's nightmare, actually. We just don't know it yet.

10:39 chouser: It's cgrand's because he observed it. :-)

11:35 Plouj: hi

11:35 cemerick: holy crap, Array.forEach isn't standard in javascript

11:36 Just thought you'd all like to share in my misery. :-)

11:36 * cemerick trudges back to the js/web ghetto

11:38 raek: cemerick: seen http://soledadpenades.com/2007/05/17/arrayindexof-in-internet-explorer/ ?

11:38 sexpbot: "Array.indexOf in Internet Explorer | soledad penadés"

11:39 raek: I guess js is a pretty neat language, at least outside the web

11:39 cemerick: raek: yeah, I'm looking for a canonical forEach impl right now

11:40 fogus: Is is a nice language... the prototypical 20% language... but nice

11:41 cemerick: Distribution is the *only* thing it has over any other reasonable language.

11:43 fogus: cemerick: That does not sound like a positive

11:43 cemerick: fogus: oh?

11:45 raek: this looks semi-canonical and fairly complete: http://code.google.com/p/js-methods/

11:45 sexpbot: "js-methods - Project Hosting on Google Code"

11:45 fogus: The only value of the language itself is something that has nothing to do with the language itself. :p

11:45 cemerick: oh, right

11:45 yeah, I think that's fair, relative to other languages.

11:46 Its only *relative* benefit is distribution; it has some good characteristics, but those are matched or eclipsed by plenty of other langs.

11:46 fogus: I see

11:47 I'm actually excited about coffeescript. I think its symbiotic model, while not ground-breaking, is the wave of the future for js

11:50 * cemerick shrugs

11:51 cemerick: js seems like a really bad host language in general

11:51 fogus: why do you say so?

11:53 cemerick: baked-in object system...odd equality semantics...boxed everything...poor stdlib...

11:53 arohner: I'm excited about clojure-in-clojure-in-js

11:53 cemerick: definitely

11:55 There must be a good reason for it, but I wonder why an org like mozilla never exposed a lower-level bytecode layer that people could base other language impls on.

11:55 fogus: cemerick: That makes it the perfect host. All of that stuff needs to be fixed and the js language committee isn't going to do it.

11:56 cemerick: fogus: doesn't that just guarantee poor perf and a miserable life for language implementors forevermore?

11:57 fogus: Maybe if Eich had and 11th day he could have created a bytecode layer

11:57 Well, perf is a problem, but it's the job of the language implementor to be miserable... so I don't have to be (it's all about me)

11:57 cemerick: heh

11:58 arohner: in general though, the web is a terrible place for application development

11:58 and I don't see that getting any better, even if js gets a bytecode layer

11:59 cemerick: well, there's places where the lang is bad, and places where the DOM is bad. The latter is solved by jquery et al...the former is solved by having another lang :-)

11:59 fogus: Crockford said that the browser was the most hostile environment he'd ever seen... until he found out about mobile

11:59 arohner: yeah, but there's still problems with ML + CSS

11:59 cemerick: arohner: all sites will soon just be js + canvas. Problem solved! :-D

12:00 arohner: cemerick: hah. true

12:00 then we'll just be running X over HTTP, with an order of magnitude more bloat

12:01 fogus: I shudder thinking about programming for the web. We use GWT and although it's bad, it's way better than the alternative

12:01 cemerick: I can't bring myself to hack around in java anymore.

12:04 fogus: Me neither

12:04 bsteuber: is there a way to specify a specific snapshot-version in maven?

12:05 say the last pushed jar before the meta-reader-change :)

12:05 cemerick: bsteuber: yup, just use the concrete snapshot version

12:07 bsteuber: like "1.2.0-master-20100426.160114-46" ?

12:07 cemerick: right

12:07 bsteuber: cool, thx

12:07 this will avoid some sleepless nights with swank for now :)

12:07 cemerick: the versions plugin will automate this for you if you don't want to muck with them on your own

12:08 i.e. it will fix all snapshots to concrete versions

12:08 bsteuber: I see

12:08 cemerick: quite key for releasing while depending on a snapshot :-)

12:08 bsteuber: will try the simple way at first - but good to know for later research :)

12:09 Licenser: aloa

12:11 joshua-choi: chouser and fogus: I'm reading your "An In-Depth Look at Clojure Collections" article, and I'm enjoying it a lot

12:15 arohner: ok, I know I've asked this in the past, but I forgot to actually change the settings

12:15 how do I get rid of PermGen errors when developing?

12:16 chouser: from where are you getting PermGen errors?

12:16 arohner: I'll leave a repl open for a couple of days doing devel work

12:16 only happens on my dev environment, doesn't happen in production

12:17 jwr7: while we're talking about slime and swank… any idea why I'm getting NullPointerExceptions when doing C-c C-c, while C-x C-e works fine and C-c C-k works fine?

12:17 cemerick: arohner: turn on concurrent GC, and enable class and perm gen class unloading

12:17 arohner: thanks

12:17 cemerick: arohner: -XX:MaxPermSize=256m -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled -XX:+CMSPermGenSweepingEnabled are my defaults

12:18 arohner: cemerick: thanks

12:25 bsteuber: cemerick: hm, I think one of my dependencies override the clojure-version so the SNAPSHOT is still taken

12:26 will the version plugin help me for this?

12:26 cemerick: bsteuber: are you declaring the concrete snapshot dep in your pom?

12:27 I mean, in your current project's pom, as opposed to a parent pom...

12:27 bsteuber: I declare it just once in the parent

12:27 but as dependency, not management

12:28 cemerick: yeah, that should be fine

12:28 put brackets around the version number to force the issue

12:28 [1.2.0-master-dddddd]

12:28 bsteuber: ok, I'll try

12:29 cemerick: version numbers are flexible by default, so a -SNAPSHOT version might override a concrete version number.

12:30 bsteuber: cemerick: now I get an error:

12:30 Couldn't find a version in [1.1.0-alpha-SNAPSHOT, ... 1.2.0-master-SNAPSHOT] to match range [clojure-1.2.0-master-...]

12:31 cemerick: bsteuber: ok, pull out the brackets, and do.....

12:31 Plouj: what's a good introduction to clojure to read on a train? :)

12:32 cemerick: bsteuber: mvn dependency:list

12:32 Licenser: I think maven is kind of borken with dependency ranges :(

12:32 cemerick: that'll show us exactly what the dep is resolving to

12:32 Licenser: it does not enforce then to match

12:32 cemerick: Licenser: the default is loose, yes. If it weren't you'd be a lot less happy. :-)

12:33 Licenser: cemerick: I have a hard time to belive I could be a lot less happy :P

12:33 at least not today

12:33 cemerick: Licenser: without flexible version resolution, every one of your project's dependencies would have to, e.g. depend on exactly the same version of log4j, or joda time, or whatever.

12:34 bsteuber: cemerick: ok, I've done that - now I've got a list of stuff I'm using

12:34 cemerick: bsteuber: right...and what's the clojure dep resolving to?

12:34 bsteuber: org.clojure:clojure:jar:1.2.0-master-SNAPSHOT:compile

12:34 Licenser: cemerick: but version ranges and just pick stuff that does not match is a difference

12:35 I know that you can use ranges, and that is great but if maven fetches me a version of clojure and a non matching version of c.c it is just broken

12:35 bsteuber: so the snapshot, as expected

12:35 Licenser: at least clojure and clojure.contrib should be very stricktly boundled

12:36 cemerick: bsteuber: and what's the full error you got before?

12:36 bsteuber: you mean with the brackets?

12:36 cemerick: yeah

12:37 hiredman: brackets and swank?

12:37 cemerick: yeah, whatever your preferred config would be

12:37 bsteuber: cemerick: Couldn't find a version in [1.1.0-alpha-SNAPSHOT, 1.1.0-master-SNAPSHOT, 1.1.0-new-SNAPSHOT, 1.1.0, 1.2.0-master-SNAPSHOT] to match range [clojure-1.2.0-master-20100323.140115-17,clojure-1.2.0-master-20100323.140115-17]

12:37 org.clojure:clojure:jar:null

12:38 Licenser: bsteuber: I am not sure if that will work ike that

12:38 ranges have to be numerical,

12:38 as in major.minor.patchset

12:38 cemerick: bsteuber: yeah, that's not the right version

12:38 should be [1.2.0-master-ddddddd]

12:38 Licenser: [1.1,1.2] is a good one

12:39 if you use 3 didgets in a range things usually break

12:39 bsteuber: http://blog.licenser.net/2010/04/22/on-clojure-libs-and-versions

12:39 sexpbot: "lice! : On clojure libs and versions."

12:40 bsteuber: now I tried [clojure-1.2.0-master-20100323]

12:40 but this gives the same error

12:40 cemerick: right, "clojure" isn't part of the version number

12:41 bsteuber: oh

12:41 cemerick: you want the entire concrete snapshot version

12:41 bsteuber: must have sneaken somehow :)

12:41 cemerick: [1.2.0-master-20100323.140115-17] is probably what you want, given the error msg

12:42 bsteuber: hm, same problem

12:42 Couldn't find a version in [1.1.0-alpha-SNAPSHOT, 1.1.0-master-SNAPSHOT, 1.1.0-new-SNAPSHOT, 1.1.0, 1.2.0-master-SNAPSHOT] to match range [1.2.0-master-20100323.140115-17,1.2.0-master-20100323.140115-17]

12:43 cemerick: I'll give it a shot with another project here.

12:43 bsteuber: thx

12:45 maybe I should just install the version as 1.2.0.master-SNAPSHOT and remove the repository reference :)

12:46 cemerick: bsteuber: OK, I've got the same error :-)

12:48 bsteuber: ok, good to know

12:50 maybe I should beg technomancy to update the elpa swank-clojure so I can happily use the SNAPSHOT again? :)

13:38 technomancy: bsteuber: as soon as clojure 1.2 does a release candidate.

13:41 bsteuber: technomancy: great to hear - so I just have to use some hacks until then :)

13:50 technomancy: you don't need the latest swank-clojure.el to use snapshot versions of clojure though

13:51 arohner: man, I really want a macro that does to reduce what 'for' did to map

13:51 I just don't know what it would look like

13:52 DuneMan: Is there a more straightforward way to write this (reduce + partition-by??) that doesn't hold the head? http://paste.lisp.org/+24BN

13:52 ataggart: couldn't you just reduce the results of for?

13:53 DuneMan: (this code doesn't hold the head, btw, and it works)

13:54 reduce the results of for...

13:55 ataggart: my comment was directed at arohner

13:55 DuneMan: oh, okay

13:55 I was confused :-D

13:55 arohner: ataggart: I mean a different way of expressing a reduction, that improves readability

13:55 ataggart: similar to the way 'for' is a different way of expressing a map

13:56 DuneMan: I am asking because I tried to write it in the naive way yesterday and couldn't come up with something that seemed like it'd work... but then wrapped my head around using lazy-seq with inner and outer recursion and it seems to work.

13:58 meaning I've shoved 1.7 billion messages through it with no memory leakage.

13:58 ataggart: arohner: the benefits of for are cartesian product, built-in limits/filters/lets. I can see the value of such for reduce, but I still think it's end up being a for wrapped with a reduce

13:58 duneman: are you trying to make a moving-window average sequence?

13:59 DuneMan: No. In this case I'm doing sums over a jumping window.

13:59 arohner: ataggart: I'm mainly talking about how the code gets ugly when you have a large, messy, one-off reduction function

13:59 DuneMan: The moving-window is simpler.

13:59 because you need to keep all intermediate results in memory anyway.

13:59 "jumping" meaning it outputs 1 result at the end of every period.

14:00 arohner: ataggart: (reduce (fn [x y] ....) start coll). when that fn gets big, the code is ugly. you can move the fn out to a letfn, but that's ugly as well

14:00 ataggart: for me, one of the biggest benefits of for over map is the indention ,and the implicit fn creation

14:01 ataggart: ah because the fn bosy is the body of the for?

14:01 *body

14:01 arohner: yes

14:01 ataggart: cool

14:01 hm

14:02 duneman: why not use partition-by?

14:03 DuneMan: I tried to (which is why I asked about it specifically). But was confused about how it was implemented.

14:04 I tried to, but ended up having trouble writing it. Specifically because I need to loop over partitions, updating the 'end time' per partition.

14:05 Either that or move the code that does the date addition into the check...

14:05 though actually, I suppose my code is slightly wrong

14:05 ataggart: hard to say without understanding the datastructure you're working with

14:05 DuneMan: because 60-second windows will drif.

14:05 ataggart: an infinite sequence defined by: (repeatedly #(.take queue))

14:06 in the sequence are somewhat large hash maps.

14:06 ataggart: heh

14:06 the elements matter too

14:06 DuneMan: Ah, yes.

14:06 So the elements are discrete events: {name "Blah" :date "Blah" .... parameters .... }

14:06 ataggart: also I'n not smart enough to infer what you're trying to do from the code (nor the comment)

14:07 DuneMan: The code does a reduce per partition. (The inner step function recurses, applying "f" to the result of itself and the next item in the list, just like reduce does)

14:07 and creates a lazy sequence of the results.

14:08 ataggart: I'm trying to deduce what you're partitioning by

14:08 DuneMan: I feel like I should be able to use partition-by and reduce to make it work.

14:08 ataggart: I think you shoudl too

14:08 DuneMan: I'm partition by (date< time-of-current-event end-time)

14:08 ataggart: these elements aren't already sorted by time?

14:09 DuneMan: Yes.

14:09 They're real-time events, guaranteed to be sorted before they hit any of the processing threads.

14:09 I'm used to writing this kinda code in c++ :-)

14:10 They *are* already sorted by time.

14:10 ataggart: k, then I don't understand the variable naming, seems that s-exp you pasted should always yield true

14:10 DuneMan: The s-expression returns a (lazy-seq

14:10 *wonders if I posted the write code*

14:10 ataggart: this: (date< time-of-current-event end-time

14:10 seems like it would always return true

14:11 which I infer means I don't understand what it really means

14:11 DuneMan: no. Check out the function call at the end of the (lazy-seq)

14:11 it calls (step .... (date+ first-event-time length)

14:11 So, on the first call, you get an end-time in the future

14:11 at the end of each partition, we recurse with a new (future) end time.

14:12 There's the inner "recur" recursion for calculating the reduce, and then at the end of the partition, I do (cons val (<outer recursion in lazy-seq style)>)

14:13 which becomes the result of the current context, and then lazy seq pulls that value off, and executes the next chunk.

14:14 It's somewhat similar to how take-while works, with an inner recursion loop that looks like reduce.

14:16 ataggart: so you're reducing the elements that fit within a window of time, then reducing the next window of time, etc?

14:17 DuneMan: yes.

14:18 ataggart: if the window is, say 60s, shoudl that 60s start from the time of the last element in the previous window, or 60s from the first element in the next window?

14:18 DuneMan: http://paste.lisp.org/display/98915#1 <-- better variabler names

14:19 ataggart: or 60s of real time, regardless of the time values of the elements

14:19 DuneMan: ataggart: it should be at 60*<number of partitions processed so far + 1>, ideally.

14:19 60s as defined by the timestamps.

14:19 My implementation here does "60s from the first element seen"

14:19 ataggart: k

14:19 stand by

14:19 DuneMan: which is a little wrong.

14:21 I suppose partition-by would work without head holding because it uses two lazy-seqs in the background (take-while...) and (drop-while...)

14:21 I get really confused by what is actually holding onto references.

14:24 ataggart: well you can't use partition-by since your predicate doesn't just use the current element

14:24 your predicate has two variables, the current element time and the max element time for that partition

14:25 DuneMan: Yes, and I couldn't find a way to tease that apart.

14:26 The other option, of course, is to change my predicate

14:26 such that it partitions time into n-second multiples by doing a mod.

14:26 ataggart: what's the "start" value supposed to be?

14:27 DuneMan: Ah, if you go to my updated post (http://paste.lisp.org/display/98915#1), you'll see I changed its name to "initial"

14:27 it's the initial value of the reduce, e.g. 0.

14:27 start was a bad name.

14:27 nurv: Hi.

14:27 DuneMan: hi nurv.

14:28 nurv: Hi, DuneMan. :)

14:29 DuneMan: This is one of those cases where I find it very easy to reason about how to implement this imperatively, but difficult using a functional+seq style approach.

14:29 hoping to get past that soon

14:30 ataggart: k, I annotated your last paste

14:30 DuneMan: (The initial value of the reduce at the start of each partition)

14:30 aha! split-with....

14:30 ataggart: :)

14:31 DuneMan: That is so the right way of doing this.

14:31 ataggart: and I messed up the last line

14:31 DuneMan: but yes, this makes sense.

14:31 ataggart: fixed it

14:32 DuneMan: split-with is what I wanted last night when I used parition-by :-)

14:32 This is much easier to understand, thanks :-)

14:33 ataggart: ya, since you need to change the values for each window, you can only get one "partition" at a time, hence split-with

14:33 np

14:34 DuneMan: I know there was something like split-with, but my 3am coding brain couldn't find it

14:34 So I ended up with what you saw :-)

14:38 Aha... this ends up holding the head somewhere

14:38 *watches memory usage climb*

14:39 yeah, it holds onto the whole list for the length of the time window.

14:39 and then that whole chunk goes away.

14:40 Or it did that for the first 2 minutes... and now it doesnt, so it must not be.

14:41 *confused*, yes, it working.

14:43 And using a baseline of less memory for some reason, with a similar cpu profile. perfect.

14:44 bmason: when I run 'lein pom' it does not generate a <build> section... I'm wondering why

15:05 any clue on why lein pom wouldn't generate a <build> section?

15:06 I've compared my project.clj with the LabRepl project and I don't see any major differences/sections missing

15:12 sids: bmason: as I understand it, the pom generated by lein pom is mainly useful for pushing to maven repos

15:12 bmason: I'm trying out the netbeans plugin, and it seems to base the file list on the .pom...

15:13 sids: bmason: I don't think labrepl's pom.xml has been generated using lein pom

15:13 bmason: yeah, I'm thinking not, since when I erase it and regenerate it using lein it is missing the build section as well

15:14 a bit weird though, since the note at the bottom says it was generated by lein

15:15 sids: maybe the pom was generated using lein but then edited and the note was not removed

15:15 bmason: yeah

15:15 hmm... netbeans still seems to work fine with the lein generated pom

15:16 (on the LabRepl project)

15:16 not on mine :)

15:17 sids: are you sure the plugin is looking at the pom?

15:17 bmason: it doesn't detect it as a project at all without a pom in the directory

15:20 alright, probably time to read up on Enclojure docs

15:41 Borkdude: ,Q

15:41 clojurebot: <-nil-<

15:43 Borkdude: ,(type Q)

15:43 clojurebot: clojure.lang.PersistentQueue

15:46 hiredman: ,(conj Q 1 2 3 4)

15:46 clojurebot: <-(1 2 3 4)-<

15:47 chouser: ,(conj Q (symbol " "))

15:47 clojurebot: <-( )-<

15:47 chouser: ,(conj Q (symbol ""))

15:47 clojurebot: <-()-<

15:48 Borkdude: what was the print method again?

15:48 chouser: ,(pop (conj Q 1))

15:48 clojurebot: <-nil-<

15:55 Borkdude: ,(conj Q (symbol "^_^"))

15:55 clojurebot: <-(^_^)-<

15:55 Borkdude: Jonah...

15:59 DuneMan: okay! Now another question, earlier, ataggart helped me come up with a function that partitions a sequence and reduces per partition. I wanted to modify this a bit to change the partitioning

15:59 http://paste.lisp.org/+24BW

15:59 The problem is that this holds the head somehow that I don't really understand.

16:00 oh... maybe I should just...

16:00 ataggart: the arity jump

16:00 would be my guess

16:01 the call to the 4-arg holds the head then calls the 5-arg

16:01 hiredman: or either of the lets

16:02 ataggart: lazy-seq is a macro but youre not actually calling cons in the 4arg version

16:02 hiredman: I would suspect the lets more than the arity jumps

16:02 DuneMan: Exactly. So I could get around this by having a a macro that expands the function.

16:02 hiredman: no

16:02 DuneMan: expands to the function call

16:02 hiredman: no

16:02 no

16:02 DuneMan: so that I don't have to pass in the "date-end" arg

16:02 hiredman: stop flailing around

16:02 stop looking for "magic"

16:02 ataggart: sigh

16:02 hiredman: lazy-seq is not magic, so don't just stick it anywhere

16:03 DuneMan: ok.

16:03 hiredman: ,(macroexpand '(when-let [a foo] bar))

16:03 clojurebot: (let* [temp__4810__auto__ foo] (clojure.core/when temp__4810__auto__ (clojure.core/let [a temp__4810__auto__] bar)))

16:03 hiredman: ^-

16:04 it kind of depends I guess

16:04 it seems like the locals clearing should be able to handle that

16:04 but I am unsure

16:05 DuneMan: With just a normal function call inside the 3-arg version that calls the 4-arg version?

16:06 thus the whole thing returning a lazy seq?

16:06 hiredman: DuneMan: tell me why you are calling lazy-seq in the four arg version

16:07 if you don't understand what it does, you should not be doing it

16:10 DuneMan: Because I need the result to be a sequence of values resulting from a reduce on subsequent partitions. Similar to partition-by. I want to be able to treat the result as a sequence of reductions.

16:10 hiredman: what does that have to do with the call to lazy-seq in the four arg version?

16:10 (the answer is nothing)

16:14 DuneMan: I guess that is what I need to figure out exactly, my current intuition is that since each s expression isn't at all evaluated until it is requested, you can effectively make the calls to subsequent parts of the expression not hold onto a reference.

16:15 hiredman: http://paste.lisp.org/display/98924#1

16:15 DuneMan: Is that wrong?

16:16 hiredman: yes

16:16 clojure is not lazily evaluated

16:16 DuneMan: lazy-seq's documentation says explicitly that's what it does

16:17 "Each coll expr is not evaluated until it is needed."

16:17 hiredman: sure

16:17 if you are making a correct call to lazy-seq

16:17 but lazy-seq is not all of clojure

16:18 you could have said "in a lazy-seq my current intuition is ..."

16:18 and laziness has nothing to do with whether or not you hold on to the head

16:19 it just gives you more rope to hang yourself with if you do

16:19 DuneMan: That is what I meant, I was speaking in the context of your question about why it has the (lazy-seq...) calls.

16:19 Sorry I wasn't more clear about that.

16:19 hiredman: ,(doc lazy-seq)

16:19 clojurebot: "([& body]); Takes a body of expressions that returns an ISeq or nil, and yields a Seqable object that will invoke the body only the first time seq is called, and will cache the result and return it on all subsequent seq calls."

16:19 hiredman: DuneMan: so all the four arg version does is call the five arg version

16:20 so it will return what the five arg does

16:20 and the five arg returns a lazy-seq

16:20 so you effectively have (lazy-seq (lazy-seq ...))

16:20 0

16:21 DuneMan: So, the take-while and drop-while bit is exactly what split-with, does, correct? So the problem then with the 5-arg version of what I posted was the let binding?

16:21 hiredman: this kind of think leads me to believe you don't know what you are doing and should be using lazy-seq

16:21 DuneMan: (and the 4-arg version was wrong)

16:21 hiredman: thing

16:21 and should not be

16:22 :)

16:22 DuneMan: No, I should be doing this, and thus learning how it works.

16:22 It's sorta like learning how to fight by getting your ass kicked.

16:22 hiredman: nah, you should be using map + partition

16:22 DuneMan: I tried that, and failed.

16:23 hiredman: really? lets see it

16:23 DuneMan: *does a git diff*

16:23 hiredman: if you find yourself using the lazy-seq macro in preference to map/filter/reduce/etc that is a red flag

16:24 DuneMan: aaand I apparently didn't commit that. So, my initial attempt was "Okay, I can do this with reduce and partition-by"

16:26 The problematic part was that I need to update my partition-end-time after each partition. Basically, I need to know "How much time has passed". Which now that I think about it means that I can just close over the "start time" of the partition and check whether (< (- current start-time) length)

16:26 and then partition-by will work

16:26 because it will only depend on the input of the current item.

16:28 okay, I'll go try to write this with reduce + partition-by, again.

16:28 hiredman: are you sure you want reduce?

16:28 what you have there is a map

16:29 (a map of a reduce)

16:29 (map (partial reduce some-fn) (partition-with what-ever something))

16:29 DuneMan: I want to partition a sequence, reducing each partition as its generated.

16:29 hiredman: a map of a reduce

16:30 DuneMan: ... indeed.... *blinks*

16:30 hiredman: you are mapping a reducing function across a sequence of sequences

16:30 DuneMan: You are very right.

16:30 hiredman: ~hiredman

16:30 clojurebot: hiredman is lazy

16:32 DuneMan: However, I see a problem with writing the partitioning function (what-ever in your example)

16:33 Because, at least how I've thought of this, it is not always the same function, it changes depending on the number of sequences seen.

16:34 hiredman: if you cannot get by with the sequence combinators provided for you, then you should consider constructing your own

16:34 but don't try and cram everything into yours

16:34 if in this case you just need a custom partitioner, then just write that

16:34 and use (map (partial reduce some-fn) ...) across that

16:35 DuneMan: right, so I could use the split-with technique to return the partitions, and then map my reducer over that.

16:35 correct?

16:40 ataggart: sigh

16:42 again, you cannot use partition-by because the predicate function requires is multivariate

16:43 DuneMan: right, and thus the split-with technique to produce my partitions.

16:43 ataggart: yup

16:44 so I'm still at a loss as to hiredman's point

16:44 which isn't unusual

16:44 DuneMan: Well, he is pointing out that I don't quite understand what I'm doing.

16:44 and he is right.

16:44 ataggart: which is both obvious and unhelpful

16:45 if you knew what you were doing you wouldnt be asking for help

16:47 and nothing I've seen so far addresses the holding-on-to-head issue

16:47 DuneMan: that is true, heh. He did point out a very good thing, though, which is that I can split this up into two logical steps of "map the reducer over a sequence" and "make a partitioner that generates a sequence"

16:47 ataggart: yup

16:48 though it doesn't solve your problem

16:49 DuneMan: ataggart: True.... and I still don't quite understand when things are holding on to the head.... not intuitive to me yet.

16:49 Hoping to gain that in time.

16:49 ataggart: I'm still not clear on why you think you need the init value

16:51 you only need 3 args, the reducing fn, the time window, and the seq

16:51 DuneMan: ataggart: Well, a reduce can start with "0" if its doing a simple count, but if I want to do what "frequencies" does (ie return a hash-map of keys to frequencies) then I would need to init it with {}

16:52 exactly the same reason "reduce" takes an init value.

16:52 ataggart: k, I missed that the same initial is used for every call

16:53 DuneMan: ah, yes.

16:53 ataggart: erm, every reduce

16:53 DuneMan: same initial is used for each reduce

16:53 ataggart: but yeah, split out the custom partitioning

16:54 then you can do whatever

16:54 DuneMan: Maybe it will make it easier to reason about.

16:54 ataggart: using extant functions

16:54 yup

16:54 still odd about the head-holding

16:55 DuneMan: I'll split it out, and try again and see if I can pin down what's happening. Thanks for the help and understanding :-)

17:13 springify: Good evening everybody. Is there a way to make slime use the clojure version in the project's lib directory when connecting to a session started with lein swank?

17:14 (Mine seems to insist on using the one in ~/.swank-clojure

17:30 Raynes: Is there some sort of magic that must be done that I don't know about in order to get hiccup's include-css to find a css file?

17:33 springify: Hmm, doesn't include-css only render the html? Won't you still need to configure the ring middleware for serving static files?

17:34 Raynes: Uh, I don't know.

17:35 It seems to serve the page.

17:35 But it looks like the css file is just non-existent. I don't know if I'm supposed to link to it in some special way or something.

17:35 I'm completely new to this stuff.

17:35 DuneMan: interestingly enough, when I split out my partition and functions and reduce, I get a memory usage pattern that implies that the time window is being held on to, and then at the end of the period, it is reduced and all the data goes away. then it repeats.

17:35 springify: Raynes: You need to wrap your handler with the middleware for serving files.

17:35 DuneMan: So... half way there.

17:36 springify: Raynes: Do a (find-doc "wrap-file")

17:36 Raynes: Wrap your application, that is...

17:37 Let me see whether I find an example.

17:38 Raynes: springify: That worked.

17:38 springify: Thanks a lot man. :D

17:39 * Raynes notes that this stuff could use more docs.

17:39 springify: Raynes: You're welcome. The doc for ring is actually okay. There's a small example showing the wrap-* stuff, too.

17:40 Raynes: It would probably help if I had even a slight idea of what I was doing.

17:40 I'm getting there.

17:40 I think. ;)

17:40 springify: Now who can tell me how to work around the "could not find gem cucumber..." error when using cuke4duke with lein-cuke?

17:43 dakrone: can anyone tell me what's I'm doing wrong with protocols here: http://gist.github.com/394035 ?

17:52 springify: Is there a way to make slime use the clojure version in the project's lib directory when connecting to a session started with lein swank? Mine seems to insist on using the jars in ~/.swank-clojure.

18:01 lancepantz: so what would be the right way to overload a method in DynamicClassLoader?

18:01 can i do that from within clojure using proxy?

18:09 technomancy: springify: use M-x slime-connect instead of M-x slime

18:11 springify: technomancy: Great, thank you. Manually updating .swank-clojure every time I wanted to try out a different version seemed cumbersome :)

18:16 elean: why memoization works only for 144 consecutive recursive calls ?

19:01 DuneMan: ataggart: I think I've figured it out.

19:06 sbt: I want to specify a custom string representation for a struct I've defined. is there a standard way of doing this?

19:29 ataggart: sbt: (defmethod print-method ::my-type [x writer] (do-stuff...))

19:29 see http://clojure.org/multimethods

19:34 DuneMan: ataggart: I think the problem with *any* lazy seq with a body that looks like (cons (take-while.... s) (.... (drop-while ... s))) is that the whole size of the partition will necessarily be realized and in memory at once, until you've finished reading it.

19:34 hiredman: nope

19:34 DuneMan: Because the (drop...) has to hold onto the head of its sequence.

19:34 hiredman: nope

19:34 DuneMan: How does that get executed, then?

19:34 ataggart: useful response as always

19:35 DuneMan: Because that is exactly the behavior I am seeing, hiredman.

19:35 hiredman: ~def drop

19:35 bah

19:35 DuneMan: It consumes a partition, stores it in memory, and then once that whole partition has been consumed, it it goes away at once.

19:36 once per window-period (minute).

19:42 http://paste.lisp.org/+24C7 <-- so, this is how I split up the partition and reduce phases.

20:07 so, then, hiredman... if I am wrong about how this works, can you explain to me how partition-by manages to cons together the items from a partition, place this whole list on the front of another list then then does a "count" on the first list in order to force it to consume elements... without keeping the entire partition in memory between the execution of the first statement and the count?

20:07 http://github.com/richhickey/clojure/blob/ab6fc90d56bfb3b969ed84058e1b3a4b30faa400/src/clj/clojure/core.clj#L3429

20:08 it does (drop (count run) s)

20:10 joshua-choi: I just git-pulled Clojure 1.2 again, and I'm happy to see that pprint was recently promoted

20:22 DuneMan: Concerning partition-by, it looks to me like we have exactly 2 things consuming elements from the head of the list [a (take-while ... s), and a (drop n s)], and these are run sequentially. Hence, the a reference to the head of the list must exist until the 2nd item [the drop] starts consuming it. Thus, the entire contents consumed by the first one will be in memory until it stops consuming, and the 2nd item starts consuming. Thus partition-by is inefficien

20:22 the size of your partitions is big.

20:24 Am I wrong? And if so, how, and why am I seeing the behavior I would expect if that were the case.

20:24 tomoj: take-while doesn't consume anything, does it?

20:24 nor drop?

20:25 ,(let [x (drop 3 (iterate inc 1))] (println "foo"))

20:25 clojurebot: foo

20:25 tomoj: guess I just don't know what you mean

20:29 oh, I see

20:36 DuneMan: https://gist.github.com/12c68e4e194f2477313d

20:36 does that work any better?

20:39 I got a 5x speedup on some stupid test case

20:40 DuneMan: *takes a look*

20:42 Ah, that is actually how I had this implemented. And it indeed exhibits the behavior that I described above. some few hundred megs of data is read in during the minute window, and at the end of it, the partition is reduced, and the memory is freed.

20:43 split-with just gives you (take-while...) (drop-while...), so you end up with the same effect, a take consuming a bunch of data starting from head, then the drop consuming a bunch of data starting from head.

20:43 tomoj: I don't really have a clue what that means, good luck


20:43 mmarczyk: um, I might have missed a fragment of the conversation

20:43 tomoj: double-spacing on paste.lisp.org too, must be your client

20:44 mmarczyk: but tomoj: it looks like you're using partition-by inside partition-by* ?

20:44 tomoj: oh, whoops

20:44 that should've been changed

20:44 DuneMan: It means that if I am reading in 500mb of data per minute, the whole 500mb is stored in memory, even if it doesn't need to be.

20:44 I'm copy/pasting from terminal... and... when I look at paste.lisp.org its not double-spaced...

20:45 tomoj: here? http://paste.lisp.org/display/98935

20:45 that doesn't look totally fucked up?

20:45 DuneMan: nope.

20:45 tomoj: oh, I bet it's because my screen is too small

20:46 bmason: looks hard wrapped to me

20:46 probably there are endlines where there shouldn't be?

20:47 tomoj: the raw source looks fine

20:47 mmarczyk: DuneMan: is there any chance that, in processing your data, you might need to skip one partition ahead, then go back?

20:47 tomoj: it's just that most of the lines are very very long with a bunch of extra whitespace

20:47 DuneMan: mmarczyk: No.

20:47 mmarczyk: then why would you use partition-by in the first place?

20:47 remleduff: Why does when-let only allow 2 args, and not as many pairs as you might want to give it?

20:48 DuneMan: mmarczyk: I'm not using partition-by, I'm writing a partition function that does something similar. I need to generate partitions, and then reduce each of them. And I never want all of the data stored in memory.

20:48 because since each partition is being reduced, they don't need to be.

20:48 tomoj: remleduff: what would you want it do if you gave it more? only proceed if all the values were truthy? proceed if any one of them were truthy?

20:48 technomancy: remleduff: because it's not obvious what the semantics would be for multiple bindings

20:48 remleduff: and vs or

20:48 DuneMan: er it doesn't need to be*

20:48 tomoj: ..yeah, what he said

20:49 DuneMan: The partitions are defined on by slices of time, and the input is guaranteed sorted by time.

20:50 mmarczyk: that sounds like a job for reduce to me

20:50 meaning an "outer reduce", with "per-partition reduce" happening inside

20:50 remleduff: But doing (when-let [a (foo)] (when-let [b (bar)] [a b])) just seems kind of grotty

20:51 Only makes sense for "and"

20:51 DuneMan: mmarczyk: It is (map (partial reduce f initial) (generate-partition-stream .... s))

20:51 As pointed out by hiredman

20:52 bmason: anyone out there using JSwat with slime?

20:53 DuneMan: So, while investigating that, I encountered all sorts of ways to do this that realizes each partition in memory at some point. And it seems that partition-by, and the ilk, all do that.

20:53 And I am hoping to find out whether my reasoning is correct.

20:54 mmarczyk: well I'm pretty sure it is

20:54 for the reasons you've pointed out above

20:54 DuneMan: awesome! Given that, I think my initial, slightly screwy solution, is the correct one.

20:55 mmarczyk: I think it would be a bit of a pain to write a partitioner matching your specs

20:55 have you managed to write one...?

20:56 DuneMan: I managed to do it by combing the reduction and partitioning.

20:56 http://paste.lisp.org/display/98924#1

20:56 mmarczyk: right, that's my initial idea -- "outer reduce" :-)

20:56 DuneMan: This works because there is only 1 thing that actually consumes the sequence.

20:56 mmarczyk: yup

20:57 DuneMan: Well, then hiredman lead me astray... oh well, learned a lot.

20:57 maybe I was explaining myself poorly.

20:57 mmarczyk: um, not necessarily

20:58 I mean I still believe that it's possible to write a "nicer" (lazier) partitioner

20:58 DuneMan: er, sorry

20:58 that links was the wrong one.

20:59 mmarczyk: which version of Clojure are you on, by the way?

21:00 DuneMan: oh, I see, they give different urls for the same post.

21:00 I am on 1.2

21:00 Which helped a ton compared to 1.1

21:00 mmarczyk: I may try to write a nicer lazy partitioner right now.

21:01 mmarczyk: DuneMan: I think I might try my hand too :-)

21:01 (unless I fall dead straight into my bed, which might happen... it's been a long day)

21:01 DuneMan: I think the strategy with the inner recursive function is a good one... there's some library code which does similar.

21:01 mmarczyk: (but I'd hate not to have some fun at the end of it :-))

21:25 ok, I think I've got something for you

21:26 that's just about the weirdest code I've written in Clojure

21:26 just let me Gist it...

21:27 http://gist.github.com/394219

21:28 this is a partition-by which goes out of its way to discard everything that won't be needed any more

21:29 on the other hand, it simply won't work as written if you don't force each individual partition in full before moving on to the next one

21:32 (and there are bugs, not surprisingly, ouch... hopefully fixable)

21:48 DuneMan: *takes a look*

21:49 yes, each partition is necessarily fully eaten

21:52 I see... you use an atom...

21:53 that is really... funky...

21:54 don't quite grok yet :-)

21:55 so you use an atom to keep track of the head in the state of the inner function

21:56 tomoj: that sounds funky indeed

21:56 DuneMan: It is, incidentally, quite similar to what my co-worker was suggesting a bit ago.

21:56 slyphon: is there some common bit of documentation people have been working off of to grok deftype et. al.?

21:57 tomoj: I assume you mean beyond the docstrings?

21:57 slyphon: yeah

21:57 tomoj: I'd been looking for something like that for a long time

21:58 slyphon: usually there's some blog post out there that shows "Hey, here's an example"

21:58 oh, shoot

21:58 tomoj: the only other thing I ever really looked at was http://www.assembla.com/wiki/show/clojure/Datatypes

21:58 sexpbot: "Datatypes | Clojure | Assembla"

21:59 mmarczyk: DuneMan: not only is it funky, it doesn't really work! ;-)

21:59 slyphon: tomoj: ah, this is good

21:59 ty

21:59 tomoj: ,'`'foo

21:59 clojurebot: (clojure.core/seq (clojure.core/concat (clojure.core/list (quote quote)) (clojure.core/list (quote sandbox/foo))))

21:59 tomoj: anyone understand that?

22:00 I can sort of make sense of the result but I don't really understand why it's the result

22:00 DuneMan: No :-)

22:00 slyphon: you're doing some major evil w/ escaping

22:00 mmarczyk: DuneMan: but hopefully it's fixable

22:00 tomoj: I'm only doing that evil because I want to understand it

22:00 slyphon: like ~'blah in a macro

22:00 tomoj: I don't have anything like that in actual code

22:00 DuneMan: mmarczyk: hah. I'm still trying to do it without an atom... feeling... um... yeah.

22:00 slyphon: tomoj: thank god!

22:00 tomoj: :D

22:01 slyphon: tomoj: i mean, #perl is ----> that way

22:01 ;)

22:01 mmarczyk: DuneMan: you could try using deftype with a :volatile-mutable field, maybe?

22:02 DuneMan: but to me this feels even more like a vortex of utter weirdness

22:02 I mean, the atom-based approach is just a tiny vortex of utter weirdness

22:04 tomoj: have a look at SyntaxQuoteReader in clojure.lang.LispReader maybe :-)

22:04 DuneMan: My solution of shoving the reduce and partition together is looking pretty damn good

22:04 mmarczyk: DuneMan: a single reduction is always going to "just work"

22:04 no question about that

22:05 it could be nice to have a special-purpose partitioner to do some of that work

22:05 though

22:05 DuneMan: Agreed.

22:05 especially considering ... I will be doing things 'like this' again

22:06 tomoj: mmarczyk: thanks

22:06 mmarczyk: np

22:06 DuneMan: exactly

22:07 DuneMan: I like how what I needed to do fell directly into this corner-case of stuff that the clojure standard lib just doesn't do very nicely.

22:08 slyphon: "just lucky, i guess"

22:08 ;)

22:08 DuneMan: If only I could hold 1.5tb of data in memory at once...

22:08 tomoj: oh, I get it

22:09 '`'foo is '`(quote foo)

22:10 mmarczyk: theoretically `(quote foo) could just return (quote *ns*-name/foo)

22:11 tomoj: doesn't it?

22:11 ,`(quote foo)

22:11 clojurebot: (quote sandbox/foo)

22:11 mmarczyk: rather than that unearthly bundle of seq-manipulating functions :-)

22:11 well in one sense it does, in another it apparently doesn't

22:11 tomoj: well, it expansd to the unearthly bundle I guess

22:11 mmarczyk: ho-hum :-)

22:12 I like how helpful my previous full sentence was :-)

22:12 tomoj: and the unearthly bundle, when eval'd, returns (quote ns/foo)

22:12 mmarczyk: yup

22:13 it's actually somewhat unusual of Clojure

22:13 the fact that syntax-quote is not called quasiquote :-P

22:13 and also, it is not a macro

22:13 tomoj: do you really think there is another possibility?

22:13 mmarczyk: with a reader macro as a shorthand notation

22:13 tomoj: I mean, that syntax-quote could work without expanding to unearthly bundles?

22:13 I'd be surprised if rich did that for no reason

22:13 hiredman: Chousuke tried to write it as a macro but I don't think he got it fully functional

22:13 mmarczyk: in Scheme, quasiquote can be implemented as a macro -- and that's the way people do it

22:14 of course it's not the same thing

22:14 which is another part of how unusual s-q is

22:15 hiredman: interesting

22:16 tomoj: ,(eval (eval (eval ````foo)))

22:16 clojurebot: DENIED

22:16 tomoj: oh, duh

22:16 mmarczyk: anyway, I thought syntax-quote-as-a-macro was a vague plan for the future?

22:17 tomoj: I don't see how you could do it without unearthly bundles

22:17 mmarczyk: as a macro, you couldn't :-)

22:18 but then you'd get '`'foo -> (syntax-quote (quote foo))

22:18 because `'foo would not be mangled by the reader

22:18 tomoj: ah, I see

22:34 Plouj: anyone here program clojure in Fedora?

22:34 I'm wondering what's a good way to setup emacs for repl

22:35 tomoj: once you have emacs I don't think it matters which distro you're using

22:35 slyphon: http://technomancy.us/126

22:35 sexpbot: "in which are found tricks of the trade concerning clojure authorship - Technomancy"

22:35 tomoj: I think that's outdated, isn't it?

22:36 slyphon: oh

22:36 yeah :/

22:36 sorry, i know i bookmarked something

22:36 tomoj: install package.el -> install clojure-mode and swank-clojure -> done

22:36 Plouj: tomoj: well, it's still a matter of whether or not to use slime and stuff and how to get all these emacs add-ons

22:36 tomoj: yeah

22:37 do you already have slime installed and/or need to work with another lisp?

22:37 Plouj: humm, I'll try package.el

22:37 I don't have slime installed ATM

22:37 tomoj: if you install clojure-mode and swank-clojure from elpa, you'll get enough slime to work with clojure

22:37 Plouj: k

22:37 slime just sounds cool :)

22:37 tomoj: I also recommend installing the paredit beta

22:37 http://mumble.net/~campbell/emacs/paredit-beta.el that one

22:39 and then something like https://gist.github.com/28fb1aa24fcbe06e25ca

22:39 (paredit is really totally optional)

22:40 if you're working on a leiningen project, 'lein swank' and then M-x slime-connect and you should be good

22:40 slyphon: !!

22:40 paredit is *mandatory*

22:40 ...unless you like not being awesome

22:40 tomoj: heh, yeah

22:41 hugod: tomoj: did you get jclouds doing what you wanted?

22:43 tomoj: not really, but I worked around it

22:43 Ymmot: Plouj if you're trying to get slime-clojure + emacs, ignore most of the stuff you find on the internet

22:43 tomoj: the stuff I am interested in writing doesn't depend on the clojure stuff that is already there, so

22:43 Ymmot: Most of it is outdated

22:44 slyphon: wow

22:44 deftype

22:45 Plouj: boo, errors installing clojure-mode and swank-clojure: http://fpaste.org/Iz47/

22:45 tomoj: you can probably ignore the errors

22:45 try `M-x slime` to see if it worked

22:45 Ymmot: La ligne est mince entre la raison et la névrose n'est pas clairement établi, et je jupe à la limite toutes les heures de veille.

22:45 Plouj: even if it says "Cannot open load file: slime"

22:45 tomoj: it should ask if you want to install clojure, say yes

22:45 slyphon: yeah

22:45 Ymmot: whoops

22:46 slyphon: emacs always throws a ton of errors when compiling

22:46 tomoj: then `,quit` to kill that repl, use M-x slime-connect instead of M-x slime for real development

22:47 Plouj: tomoj: ok, that seemed to work

22:47 slyphon: or swank-clojure-project

22:47 tomoj: swank-clojure-project is dying

22:47 Plouj: lol, where am I connecting?? :)

22:47 slyphon: tomoj: !

22:47 tomoj: you need to start up a swank server in your project first

22:47 Plouj: nothing on localhost

22:47 oh

22:48 Ymmot: One thing I had trouble with Plouj was make sure you don't already have ~/.clojure and ~/.clojure-contrib directories when you install swank-clojure/slime etc

22:48 slyphon: tomoj: i need it for one of my projects, the classloader does stupid stuff when i run it as a separate process

22:48 tomoj: for a leiningen project that's `lein swank` in the root of the project, but you may need to add some deps for that to work

22:48 slyphon: strange..

22:48 well I think it will stay around but just be deprecated and unsupported

22:48 slyphon: tomoj: yeah, it was java badness

22:49 Atomikos TransactionEssentials being "smart" and "clever"

22:49 tomoj: "Even if it's deprecated, I will definitely leave M-x swank-clojure-project around"

22:49 Plouj: Ymmot: ok, doesn't seem to be the case for me

22:51 slyphon: sooo, deftype creates something that's like multimethods in the namespace they're defined in?

22:51 Plouj: tomoj: well, then M-x slime is what I need if I just want to play around without working on any project, right?

22:52 tomoj: yeah, that should work fine

22:52 slyphon: wow, that's bizarre

22:52 and for some reason reminds me of CLOS

22:52 tomoj: I don't think they're like multimethods

22:52 I think they're just java methods?

22:52 slyphon: well, it's a dispatch based on type

22:52 hiredman: no

22:52 slyphon: no?

22:52 hiredman: thats protocols

22:52 slyphon: http://clojure.org/protocols

22:53 the (deftype Foo ...) is what i'm looking at

22:53 hiredman: you can implement a protocol inline on a deftype

22:53 right

22:54 slyphon: what's 'x' in those examples

22:54 hiredman: and you are ignoring the protocol stuff that comes *right* before what you are looking at

22:54 slyphon: i just looked up

22:54 and saw it

22:54 :/

22:54 hiredman: well, I think the assumed reading direction in the clojure docs is top to bottom

22:55 slyphon: ...

22:55 hiredman: but I guess you could complain about the poor quality of the docs and ask for a warning at the bottom to start from the top

22:55 :)

22:55 * slyphon laughs

22:56 slyphon: ok ok

22:56 tomoj: am I wrong that you have to call deftype methods like (.foo x) instead of (foo x) ?

22:56 slyphon: i wasn't sure what was defining the (bar-me ...)

22:57 hiredman: I wonder if there is a test suite for syntax-quote

22:57 slyphon: so 'x' there is 'this'

22:57 hiredman: tomoj: you wouldn't call like that

22:57 you would use a protocol fn

22:58 tomoj: if the deftype implements an interface defining a foo method, then, (.foo x)?

22:58 instead of a protocol I mean

22:59 hiredman: yeah

22:59 tomoj: I hadn't realized protocols let you drop the .

23:00 hiredman: you can have your deftype implement an interface

23:00 then make a protocol and extend it to that interface

23:01 and no more .

23:01 slyphon: oooooh

23:01 that hawt

23:01 er, "that's hawt", even

23:04 manuel_: hi

23:04 hiredman: technomancy was talking about once ILookup (what keywords use for looking themselves up in maps) is replaced with a protocol extending ILookup to Number so stuff like (:days 2) would give you the number of seconds in two days

23:06 slyphon: oh nice

23:06 hiredman: (extend-type Number ILookup (val-at [n k] ...))

23:06 something like that

23:07 slyphon: hiredman: i did something simple that lets you do (-> 2 days from-now) or (-> 3 hours ago)

23:07 hiredman: that reads well

23:07 slyphon: yeah, i thought so

23:07 it's embarrasingly easy

23:07 hiredman: very extendable too

23:08 slyphon: hang on, i'll gist the code

23:08 tomoj: hah, (-> 3 hours ago) is brilliant

23:08 slyphon: i got the idea from rails' 3.hours.ago sugar

23:09 tomoj: yeah, but your sugar is sweeter

23:09 slyphon: that's what all the girls say....

23:09 ;)

23:09 tomoj: (-> 10 minutes and 20 seconds after midnight last Thursday)

23:10 that would be terrible..

23:10 slyphon: https://gist.github.com/5b80f2242f719fca5a07

23:10 haha

23:10 oh, duh

23:10 hiredman: tomoj: wouldn't work

23:10 20 is not callable as a fn

23:11 tomoj: aww

23:11 hiredman: (-> 10 minutes (plus (-> 20 seconds)) after midnight last Thursday)

23:11 manuel_: are there some "commercial" webapps running on clojure?

23:12 i've been using lisp for that for the last 6 years but i'm thinking of switching

23:12 but i'm wary because of the lack of frameworks and documentation

23:12 slyphon: you could also do (-> 5 megabytes)

23:12 (and other such unit conversions)

23:12 hiredman: it seems like most companies have clojure in the backend somewhere and rails or similiar on the front

23:13 slyphon: yeah, that's what we're doing

23:13 manuel_: i don't really want to delve into ruby :]

23:13 rhudson: manuel_: Flightcaster seems to be the Clojure poster child for webapps

23:13 manuel_: i don't have the time rather

23:13 flightcaster, k

23:14 the webpart is rails though

23:14 hiredman: right

23:14 slyphon: rhudson: is that the one in Cambridge?

23:14 manuel_: i have no real backend part

23:14 rhudson: nothing's perfect

23:14 manuel_: anyway, i'll stay with lisp i guess

23:15 rhudson: slyphon: Don't know where they're located

23:15 * slyphon nods

23:15 manuel_: and move my cnc stuff over to clojure first

23:15 thx people

23:15 slyphon: sure

23:15 cnc? command-n-control?

23:15 DuneMan: *stares at a memory graph and wonders wth the jvm is doing*

23:15 manuel_: computer numeric control

23:15 slyphon: ah

23:15 manuel_: milling machine and laser cutter

23:15 slyphon: OOOOH

23:15 :)

23:15 fun!

23:16 Plouj: humm, how would I compile/eval the ants.clj file I just loaded in emacs with slime. This says http://riddell.us/ClojureWithEmacsSlimeSwankOnUbuntu.html I should do C-c C-k, which does nothing for me

23:16 sexpbot: "ClojureWithEmacsSlimeSwankOnUbuntu"

23:16 Plouj: C-c C-k is undefined

23:16 DuneMan: concurrent mark and sweep GC turned on, and I still get this really odd spikes of growth with different slopes. Must be trying to figure out what to do :-)

23:16 Plouj: I didn't actually follow those ubuntu installation steps

23:16 manuel_: slyphon: "kind of" ;]

23:16 slyphon: manuel_: hahaha

23:16 manuel_: dude, you have frikkin lasers!

23:17 manuel_: i ripped one out of a dvd-rw

23:17 and put it on a plotter

23:17 slyphon: hahahahaha

23:17 manuel_: now i can cut cardboard and foam

23:17 slyphon: wow, seriously?

23:17 manuel_: yeah

23:17 got an old plotter off ebay

23:17 slyphon: daaaamn

23:17 hiredman: Plouj: well, if you didn't follow part of the instructions, why do you expect a different part of the instructions to work?

23:17 manuel_: better than getting a co2 laser for xxx$

23:17 slyphon: i didn't know a dvdrw's laser wsa powerful enough :)

23:18 Plouj: hiredman: well, I installed slime and swank with Emacs' package.el thing

23:18 manuel_: me neither

23:18 it's not perfect though

23:18 DuneMan: you controlling it with clojure?

23:18 heh

23:18 manuel_: no, with lisp

23:18 Plouj: hiredman: I'm just guessing at keys to eval things from buffers

23:18 DuneMan: fair

23:18 manuel_: but i'm getting tired of deploying lisp apps

23:18 tomoj: Plouj: http://github.com/technomancy/swank-clojure

23:18 some handy keys referenced there

23:19 hiredman: Plouj: C-c C-k is the correct keybinding

23:19 I suggest you follow some directions

23:19 tomoj: don't follow those directions though, imo

23:19 hiredman: because it sounds like you now have your emacs in some kind of indeterminate state which may make it harder once you start to try and follow directions

23:21 Plouj: humm

23:21 well, the swank doc says C-c C-k is also correct

23:21 but my emacs says it's undefined

23:21 hiredman: well, find some directions

23:21 Plouj: and C-e C-x does something partially (shows code in a mini-buffer)

23:22 mmarczyk: DuneMan: I'll continue on weird partitioning schemes next time I'm awake :-)

23:22 maybe you'll have the problem solved by then, in which case it would be nice to compare notes

23:22 tomoj: Plouj: what mode is your buffer in

23:23 DuneMan: mmarczyk: Thanks, I will too. I gave up for now and went with my reducer that I've got

23:23 Plouj: tomoj: it says ant.clj is in (Clojure)---------

23:23 DuneMan: and moved on to inserting shit huge swaths of data into mongodb

23:23 works like a charm :-)

23:23 mmarczyk: right :-)

23:23 tomoj: it needs to be in slime mode as well

23:23 M-x slime-mode

23:23 mmarczyk: see you

23:23 hiredman: Plouj: my understanding is either the slime or swank you get from the package is bogus for clojure (it's for common lisp) so you either need to reconfigure it for clojure, or delete it and follow the directions in the swank-clojure README

23:23 tomoj: sometimes I have to do this manually when working on a remote emacs, so I'm guessing there's some configuration which makes this happen automatically

23:24 Plouj: tomoj: ah, now C-c C-k says not connected, although I have slime running in a diff buffer :(

23:24 tomoj: and the repl in that buffer works?

23:24 Plouj: (I started slime with M-x slime)

23:24 yeah, repl works

23:25 but

23:25 when I just start it, it says:

23:25 Lisp connection closed unexpectedly: connection broken by remote peer

23:25 although I still see process java running and REPL works

23:26 tomoj: huh?

23:26 Plouj: http://fpaste.org/Ir4M/

23:26 tomoj: go to the repl and ,sayoonara

23:26 then start it again

23:26 if that doesn't work I dunno what is going on

23:26 Plouj: , doesn't do anything

23:26 clojurebot: java.lang.Exception: Unable to resolve symbol: doesn in this context

23:26 Plouj: I just end up typing in the whole ",sayoonara" string

23:27 tomoj: uhh

23:27 Plouj: user=> user=> ,sayoonara

23:27 tomoj: the repl you're in, what's it called and what mode is it

23:27 Plouj: java.lang.Exception: Unable to resolve symbol: sayoonara in this context (NO_SOURCE_FILE:0)

23:27 user=>

23:27 tomoj: *inferior-lisp buffer* or something?

23:27 Plouj: -UUU:**--F1 *inferior-lisp* All L15 (Comint:run)---

23:27 tomoj: yeah, that's wrong

23:27 Plouj: that's what M-x slime opens

23:27 rhudson: manuel_: There's a Clojure web framework Compojure -- haven't used it myself but it seems well thought of

23:28 tomoj: M-x slime should open an inferior lisp buffer, but when it's done loading it should put you into a *slime-repl clojure* buffer

23:28 if not, something's wrong, I guess

23:29 Plouj: I suspect something is wrong

23:29 remleduff: slyphon: When you were looking for deftype documentation did you see http://clojure.org/datatypes and http://clojure.org/protocols ?

23:29 Plouj: the only other slime buffer I have is *slime-events*

23:29 slyphon: remleduff: yeah, i found my way over there

23:29 remleduff: thanks :)

23:30 tomoj: maybe you need to restart emacs? :/ I dunno

23:30 M-x slime works fine here with no extra configuration, just the elpa packages

23:31 Plouj: I've tried restarting emacs multiple times :(

23:31 ok...

23:31 now it works...

23:32 and now it doesn't

23:32 it seems to run when I first run package-list-package

23:32 s

23:33 actually no, even then sometimes it looses the connection

23:33 tomoj: I just tried on a brand new emacs23-nox and it seems to work fine, no restarting emacs necesssary

23:33 remleduff: Plouj; Does M-x clojure-enable-slime-on-existing-buffers work?

23:35 Plouj: remleduff: well no because it's not connected to slime

23:37 tomoj: just confirming.. you installed clojure-mode and swank-clojure from package-list-packages, right?

23:38 Plouj: tomoj: yup, I even double checked that

23:39 tomoj: got no advice then, I just did the exact same thing from scratch and no problems, good luck

23:39 Plouj: http://fpaste.org/okox/

23:44 tomoj: did you put anything into your ~/.emacs for slime?

23:45 remleduff: What's the value of your slime-lisp-implementations?

23:47 tomoj: nope

23:47 didn't do anything except the package.el installation and clojure-mode, swank-clojure from elpa

23:47 then M-x slime started up fine and C-x C-f foo.clj was in slime mode and connected properly

23:48 Plouj: remleduff: how do I check that? :)

23:49 remleduff: I'm not sure the "proper" way, I do C-h v, then type it in

23:49 Plouj: ((clojure ("java" "-classpath" "/home/plouj/.swank-clojure/clojure-1.1.0-master-20091202.150145-1.jar:/home/plouj/.swank-clojure/clojure-contrib-1.1.0-master-20091212\

23:49 .205045-1.jar:/home/plouj/.swank-clojure/swank-clojure-1.1.0.jar" "clojure.main" "--repl") :init swank-clojure-init))

23:50 I suspect I need something like (setup-slime)

23:50 because if I re-download the clojure-mode and swank-clojure with package-list-packages and then M-x slime, everything works perfectly

23:50 tomoj: do you use .emacs or .emacs.d/init.el?

23:51 I think the package.el installation might dump stuff into .emacs no matter what

23:51 but if that code weren't being loaded, slime wouldn't work at all, I'd expect

23:52 Plouj: I don't have .emacs.d/init.el, so that means ~/.emacs

23:52 anyways, I'll look into this tomorrow

23:52 thanks for helping

23:52 happy clojuring(?)

Logging service provided by n01se.net