#clojure log - May 11 2011

The Joy of Clojure
Main Clojure site
Google Group
List of all logged dates

1:09 Raynes: Anybody happen to know if http://github.com/daveray hangs out on IRC and if so, what his nick is?

1:24 amalloy: Raynes: dunno. he tweets as darevay though

3:20 desertRoamer: anyone here familar with jython? i'm not really python literate, but i am looking at some python code that i'd like to convert into clojure via java, ie jython. is it possible to go back and forth between python and clojure, or should i just learn python and not bother.

3:21 shachaf: desertRoamer: "convert into clojure"? Jython isn't going to turn Python source into Clojure source.

3:21 opqdonut_: jython was pretty unmaintained when we used it (it was stuck in python 2.2 when cpython was up to 2.6 or so), so we switched to clojure

3:21 I've no idea about it's current state

3:24 desertRoamer: ok, thanks. guess i'll just "bite the python bullet"

3:24 opqdonut_: I still have no idea why you are talking about python in here

3:25 clojure and jython are two separate and unrelated languages for the java virtual machine

3:25 and jython is actually an implementation of the language called python, which also has other implementations

3:29 desertRoamer: right, just curious how seemlessly python and java (or clojure) developers might collaborate on a project, i guess is my larger question

3:30 sorry, seamlessly, i mean

3:31 OlegYch|h: if you can call jython from java, you should be able to call it from clojure

3:31 just that build process becomes complicated

4:08 rrc7cz: yesterday I had a bit of a head scratcher around converting some imperative pseudocode and I was hoping for some thoughts on my solution: https://gist.github.com/964996

4:10 it came about during the conditional mutation of two variables, with the mutation acting as a dependency for the next conditional mutation of another two variables (when seconds == 60, inc minutes and set seconds = 0, then do the same for minutes)

5:35 Vinzent: i've a macro which takes 4 args, all tests work fine while the macro is public. then i've added ^{:private true} and called it from the other ns with var-quote #' syntax and it threw Wrong number of args (4). wtf?

5:38 from the ns where macro is declared all ok after making it private. from the tests ns it takes 6 args instead of 4 somehow. Can't understand what i'm doing wrong

5:42 raek: Vinzent: you cannot use that trick for macros

5:43 when you do (#'your-marco ... ) there will not be a macro expansion. instead, it will become a call to the underlying macro function

5:44 the underlying function of a macro always take 2 extra arguments compared to the macro

5:45 Vinzent: you could try (def ^{:private true, :macro true} your-macro @#'the.ns/your-macro)

5:45 (but I haven't tested that)

5:46 ok, you don't need ":private true" here

5:49 Vinzent: raek, thanks, understood. i'll try def

5:56 yep, it works perfect. Thanks again!

7:20 fliebel: I remember someone saying defmethods are namespace local? So if I define a multimethod in ns1, and add a method to it in ns2, but use it in a function in ns1, will the new method be visible?

7:25 Apparently they do work across namespaces, yay!

7:26 logs?

7:26 clojurebot: logs is http://clojure-log.n01se.net/

7:34 raek: fliebel: yes, they are meant to work across namespaces

7:35 in contrast to other "def-" macros, the name symbol can be a fully qualified symbol that identifies a var in another ns

7:35 fliebel: :)

7:35 clgv: Vinzent: making a macro private and calling it from another namespace sound really odd to me.

7:36 raek: I assumed it was for unit testing

7:36 clgv: raek: ah ok. that is the only reason that makes sense, I guess

7:45 Leonidas: hi, I am trying to get a hang with clojure stm, so I thought of building a counter that I increment in parallel from a number of threads. once unsynchronized and once using STM.

7:45 but I tried atoms and refs and they both seem to work the same

7:46 when do I use atoms?

7:47 """Atoms are an efficient way to represent some state that will never need to be coordinated with any other""" hmm, I think I see

7:47 clgv: Leonidas: you should use atoms when you have atomar behaviour. refs should be used when you have a transactional context where you need to synchronize the access to a couple of refs

7:47 Leonidas: but how do I get an "unguarded" integer?

7:47 so for my task, an atom would be enough

7:47 clgv: yes

7:48 atoms should be faster as well since they have less overhead. you could measure it for your example

7:49 Leonidas: ok, so much for that.

7:49 and now the example without STM, how to do that?

7:49 I want to explicitly allow threads to overwrite the counter

7:50 clgv: atoms are without STM afaik. refs are clojure constructs that incorparate the STM#

7:53 raek: Leonidas: one way to emulate unrestricted mutation is to modify the counter like this: (reset! a (inc @a))

7:54 (it's highly unideomatic to write code like that, but I guess you're already aware of that :) )

7:55 Leonidas: raek: yes, of course. I just want to demonstrate the problem to people and therefor I need an example that does it "wrong"

7:55 *therefore

7:55 raek: is there also a version of reset! for refs?

7:56 raek: the code intriduces the race condition you are looking for because the read and the write is not done atomically

7:56 Leonidas: ref-set

7:56 Leonidas: but that can't go wrong, since you can only change a ref in a transaction

7:57 Leonidas: raek: exactly, this is what I want to demonstrate. the stm example on the clojure page is a bit hard to grasp for people who don't know lisp

7:57 raek: and the transaction makes the read and the write atomic again

7:58 Leonidas: raek: so I can't create that race condition with refs at all?

7:58 raek: Leonidas: exactly

7:58 Leonidas: ok, so I'll do the race condition example with atoms

7:58 raek: there is something called write scew which can happen, but that is not a race condition IIRC

8:00 Leonidas: one of the videos here have such an example: http://www.pluralsight-training.net/microsoft/OLT/Course/Toc.aspx?n=clojure-concurrency-tutorial

8:01 (and yes they're evailable in non-silverlight format too)

8:04 Leonidas: Will look into this

8:10 heh, I'm unable to build a race condition :|

8:18 clgv: Leonidas: but you should be able to build an inconsistent execution. e.g. let each thread increment an int or something like that

8:19 Leonidas: clgv: yeah, I'm trying.

8:24 fliebel: What? You are trying to *make* a race condition?

8:25 clgv: fliebel: for demonstration purpose he said ;)

8:25 fliebel: I missed that...

8:32 &(let [a (atom 0) f #(dotimes [_ 1000] (reset! a (+ @a 1)))] (pcalls f f f f f) (Thread/sleep 100) @a)

8:32 sexpbot: ⟹ 1271

8:32 fliebel: meh, that is silly

8:32 _fogus_: Raynes: I added you as a commiter to Marginalia. You no long need to wait around for me to merge. :-)

8:37 fliebel: _fogus_: I encountered your SO answer again :) I know more of these now than I knew then. http://stackoverflow.com/questions/3958630/what-are-important-languages-to-learn-to-understand-different-approaches-and-conc

8:39 edw: In SLIME, where does *out* go? I'm looking at the terminal where I'm running my swank server, the *slime-events* buffer, and the *slime-repl clojure* buffer, and (println "foobar") doesn't produce any discernable output.

8:44 raek: edw: I get mine in *slime-repl clojure*, except when I do it from another thread than the repl thread. in that case I get the output in the swank-clojure terminal (*lein-swank* with Durendal)

8:45 _fogus_: fliebel: Great! There are so many more that I wanted to add, but I didn't want to cross over into the realm of absurdity. :-)

8:46 edw: raek: Thanks. Hmm.

8:47 Fossi: qi sounds interesting

8:48 raek: edw: you can use (intern 'clojure.core '*out* *out*) to use the current *out* in all threads

8:48 edw: raek: Odd, did a ",disconnect" and then reconnected: all is well.

8:48 (Right before your message...)

8:50 Oh, wait: I did a (spit *out* <LIST OF LIST OF STRINGS>) and my *out* went away...

8:50 fliebel: _fogus_: Which ones would you add? AM I in for a dive into obscurity, or just more interesting concepts?

8:52 raek: edw: maybe spit closes the "file" when it's done

8:52 edw: you can use pr to print any clojure form to *out*

8:53 fliebel: _fogus_: Erlang, Haskell, Prolog and Io are all in "seven languages in seven weeks"; I did some Forth on my own.

8:55 _fogus_: I would add a defunct language like T (among others), probably Eiffel, Delphi, Joy...

8:56 Pure too

8:59 fliebel: _fogus_: ghehe, okay. I heard about Joy, Eiffel and Delphi before. Can you program all these languages? I'd love to know why you think these are interesting, but that is probably going to take you some time to explain.

9:00 clgv: fliebel: I dont think Delphi is really interesting if you have done C++ ;)

9:00 fliebel: Well, I havn't ;)

9:08 clgv: I learned programming with pascal & delphi ;)

9:09 _fogus_: Delphi is especially interesting if you know C++. It's much cleaner in any case.

9:09 Leonidas: fliebel: that looks nice :)

9:09 clgv: _fogus_: that's true. but there are no really new concepts ;)

9:16 edw: raek: I was testing SPIT; yeah, probably closes it. That makes me feel better. "It's not a bug, it's a *feature*."

9:16 * fliebel is reading about stack based languages

9:16 Fossi: pfft. who needs delphi when you can have pascal for windows? ;)

9:18 my dad's worst ever investment in my coding skills :D

9:18 Leonidas: fliebel: stack based languages are fun.

9:19 fliebel: i played with PostScript, Joy and Factor and the #concatenative channel is cool

9:19 fliebel: Leonidas: But are they useful as well? I keep hearing complaints about leaving the stack n a weird state

9:19 Fossi: lua metaprogramming is funky, but not too far from 7 language's ruby

9:20 forth is so much fun an an olpc

9:20 Leonidas: fliebel: well, Postscript certainly is. The point is, once you grasp the combinators you don't need to think so much about the stack.

9:20 Fossi: because all the drivers are "natively" written in it

9:20 and you can inspect a whole lot of stuff

9:21 Leonidas: fliebel: just like in lisp, where once you grasp map and fold, you don't need to implement everything recursively but have higher level abstractions.

9:22 fliebel: Right, I guess I'll hang around in #concatenative for a bit :)

9:22 Leonidas: fliebel: and i can highly recommend the factor online help. its a bit like wikipedia, where you click though the defintiontions and start learning a lot fancy tricks.

9:22 fliebel: yay

9:23 Leonidas: take a look how 'max' is implemented. i would've used if with 2 clauses, but they do cool stuff, the implementation blew my mind

9:23 although I still don't know why they implemented it this way :>

9:28 fliebel: strange, when exacuting, I don't get any race condition with your code

9:36 bhenry: how can i add a list to the end of a list of lists. cons and conj both go to the beginning on lists

9:37 chouser: bhenry: it is not possible to efficiently add to the tail of an immutable linked list

9:38 bhenry: chouser: how much overhead is it to vectorize the list of lists as a vector of lists?

9:39 chouser: bhenry: you can build a lazy sequence that walks one list and then the other using concat, and then build a new list out of that seq if desired

9:39 bhenry: pouring a list of anything into a vector is O(n)

9:39 but then adding a single item to the right-hand-side of that vector is essentially constant time

9:40 bhenry: thanks

9:50 mec: Is there a way to do short-circuit permutations? for example if I want to (first (filter #(apply > %) (permutations [1 2 3 4 5]))) even tho the first test is wrong it will stell go thru 12354 12435 12453 12534 12543 etc

9:59 chouser: mec: permutations looks like it returns a lazy seq. I'm not quite sure what you're asking for.

10:00 mec: like know that (> 1 2) is false, rather than running thru 12345, 12354, 12435... you can skip straight to 21345

10:01 chouser: ohhhh. hm.

10:02 bhenry: mec is your usage bigger or more complicated than that example?

10:03 mec: no i think thats a fair simplification

10:05 fliebel: mec: Have a loot at core.logic, I think that'll be able to do it.

10:09 dnolen: mec: fliebel: you would probably need to use conda, fliebel has a permo, but yeah that might work, not sure about perf in this case if that matters. seems like another good problem for a fast constraint solver...

10:10 fliebel: dnolen: :) yea, not sure mine works correctly.

10:10 cemerick: Is there any canonical discussion for the hinting of arg vectors?

10:11 chouser: cemerick: why read a discussion when you can create a new one? :-D

10:12 cemerick: chouser: I'm about to, but figured I'd poll the channel for references that might help me avoid sounding like an idiot. :-D

10:14 mec: fliebel: dnolen: I'm not at all sure how to use core.logic, is there a clojure constraint solver or is that something you have to write specifically per problem?

10:15 dnolen: mec: you can write logic program to generate the search space and cut of those branches that don't matter.

10:22 chouser: dnolen: will you be presenting at clojure-conj?

10:22 * fogus`away hopes so

10:22 chouser: me too

10:22 dnolen: chouser: I need to submit abstracts :D procrastination ...

10:23 fogus`: dnolen: I am stoked to hear what you have to say

10:23 chouser: I'd like to submit some abstracts as well, but I'm really fumbling for good topics

10:24 monads? DSLs? lazy seqs in javascript?

10:24 fogus`: chouser: cinc is still in play

10:24 pdk: an abstract about your journey to find a good abstract topic

10:24 cemerick: My problem is I'm just not 133t enough.

10:24 chouser: pdk: heh. cute, but so dull! :-)

10:24 pdk: just embellish a few parts

10:25 like the part where the king sent you to slay

10:25 um

10:25 stroustrup i guess?

10:25 chouser: dnolen: I'd love to hear from you about logic programming, esp. related to assertions about clojure code at compile time

10:25 fogus`: (inc chouser)

10:25 sexpbot: ⟹ 6

10:27 dnolen: chouser: yeah that's what I'm most excited about anyhow. I would love to see core.logic be fast enough to replace more common operations, but that's a whiles a way (if ever). But I think it's useful for compile time stuff now.

10:27 chouser: whichever talk gets accepted I'll talk about it anyhow :)

10:27 chouser: dnolen: excellent. But that removes logic programming from my list of potentials to talk about. :-)

10:27 dnolen: chouser: heh.

10:28 chouser: I only have two talk ideas - predicate dispatch and logic programming.

10:29 chouser: Ah, I'd love to hear about either. Well, both actually.

10:29 but either will do

10:29 cemerick: The latter would definitely be a crowd-pleaster.

10:29 pleaser*, that is

10:30 chouser: fogus`: cinc is still interesting to me. But what is there to talk about?

10:31 mefesto: dnolen: those would be awesome topics :)

10:31 so fwiw (inc)

10:33 fogus`: chouser: not sure.... I was hoping your talk would enlighten me. :-(

10:35 chouser: heh

10:35 edw: Is there a "State of Clojure in Clojure" written somewherw?

10:37 I mean, anything more than the end of the following slide deck? http://www.slideshare.net/fogus/clojure-11-and-beyond

10:39 dnolen: mec: your question is interesting to me, I'll try to cook up an example today of a solution.

10:39 gotta run.

10:40 mec: dnolen: cool thanks, im trying to implement it directly with a permutation generator :x

10:40 fogus`: edw: I don't think so. chouser's blog post is still the go to source for me

10:41 chouser: that was written back when rhickey talked to us in here

10:49 hugod: for the intrepid, swank-clj 0.1.0 is out - https://github.com/hugoduncan/swank-clj - with slime debugger support

10:59 zippy314: I'm getting an error I don't understand: "Thrown class java.lang.InstantiationException" in the context of macros. Here's a gist that makes it happen: https://gist.github.com/966613 Can anyone tell me what this means?

11:00 The basic macro works fine, i.e. you can use it create objects by passing in a symbol name or a function.

11:01 The problem is when I use the macro within another macro.

11:11 cemerick: Comments welcome, especially if I'm totally missing the boat somewhere: http://groups.google.com/group/clojure/browse_frm/thread/2b72bd8be27f0f72

11:12 xian: Hello. This might come close to a stupid question, but how do I tell slime that I want square-brackets highlighted when typed in the slime-repl (just like regular parentheses)?

11:12 (They're highlighted perfectly in clojure-mode, just not in the slime-repl.)

11:12 mec: zippy314: (let [k :test] (make-obj (symbol (str (name k))) :foo)) gives the same exception, so the problem is somewhere in make-obj

11:14 chouser: zippy314: eval in a macro is messy

11:15 mec: ah of course, eval doesnt have the scope

11:15 chouser: exactly

11:17 zippy314: Hmm. So then how does one implement the intent there that a parameter is evaluated in some cases but no in others (i.e. if it's a symbol)

11:20 whidden__: zippy314: i would split the cases for early evaluation into one pile and those for late evaluation into another pile and if it is appropriate combine at a higher level of abstraction.

11:21 zippy314: How's that for a non-answer? :)

11:21 zippy314: whidden__: heh

11:23 jarpiain: zippy314: (let [name (if ... `'~n n)] `{(keyword ~name) ...})

11:25 zippy314: jarpiain: yah, that makes sense. Thanks.

11:27 mec: zippy314: i think you can just do (defmacro make-obj [n & args] `{(keyword ~n) [~@args]})

11:28 hmm maybe not

12:10 dnolen: cemerick: re: type-hints, I kind of fail to see how it would be possible make it work given Clojure's design.

12:10 with real type-inference maybe, but that's a big ol' project.

12:11 cemerick: dnolen: which part, my naive ideal?

12:13 dnolen: cemerick: could be completely wrong of course. I'd brought this up before myself with rhickey as I was suprised that it didn't work the way you're describing.

12:13 cemerick: I didn't *think* I proposed anything difficult or novel.

12:15 Perhaps return hints really do need to be on arg vectors for fn* to do its work, but fn should be able to lift them from the arity-body parens as needed.

12:16 The unification of :tag metadata and the prim interfaces seems more like grunt-work and maybe some cat-herding to make sure things are aligned as necessary for tooling purposes.

12:16 s/parens/lists

12:22 dnolen: cemerick: fn's only return Object. :tag is just a cast. to do what you propose it means the compiler now has to track all the signatures internally to know which cast to apply in a given situation. but perhaps you're right, just grunt work.

12:23 well fn's only Object or one of the support prims.

12:24 gfrlog: does it sound correct to say that some functions cannot be made point-free without changing their performance characteristics?

12:25 cemerick: dnolen: it's already tracking signatures; otherwise, it would have to fall back to reflection for calls to different arities that have different return types

12:25 TimMc: gfrlog: Have an example in mind?

12:25 gfrlog: TimMc: yes

12:25 cemerick: or, I should say, it's tracking signatures thanks to the prim IFn interfaces

12:25 gfrlog: (fn [x y] (if (< x y) (* x x) (* y y)))

12:26 TimMc: that might be a bad example actually

12:26 dnolen: cemerick: that's what I said.

12:26 cemerick: but a non-primitive type-hint is just a cast, something completely different from a real IFn sig.

12:27 gfrlog: TimMc: I'm more thinking of (if) expressions and the problem of preventing unnecessary evaluation

12:31 cemerick: dnolen: True, but those different things are already handled well by the compiler.

12:31 TimMc: gfrlog: Yeah, the compiler specials might be tricky.

12:31 gfrlog: Might be able to circumvent that with lazy seqs...

12:31 gfrlog: TimMc: I guess the everything-can-be-done-point-free principle applies to possibility but not performance

12:32 TimMc: I thought of that too. It might work, but it'd feel hacky

12:32 TimMc: *point-free* feels hacky

12:34 dnolen: cemerick: what different things are handled well by the compiler? it looks like tags are oriented around exprs, there's no notion of an expression having multiple possible tags based on known primitive types in a context.

12:36 cemerick: not saying this work shouldn't be done, but I don't see how it doesn't boil down to type-inference. if it means type-inference does that mean type-inference should be limited to the scope you're suggesting.

12:37 cemerick: dnolen: :tag metadata and fn class interface implementations; in the example where the var is hinted as ^String but a particular arity is hinted as ^double, there's no reflection at all because the compiler appears to disambiguate return types based on arity without difficulty.

12:38 dnolen: cemerick: it does no such thing. that works because there 300+ interfaces.

12:39 cemerick: All I was suggesting is that (a) fn/var metadata should be the canonical source of type information and (b) hinting position should be consistent regardless of category of return type

12:40 dnolen: That's where the compiler is getting the non-:tag return types, but it sure looks like it's disambiguating based on arity.

12:41 dnolen: cemerick: correct me if I'm wrong, prior to primitive support, fns could have only ONE tag.

12:41 cemerick: True.

12:41 dnolen: regardless of arities.

12:42 cemerick: so what you're describing simply is not true of the current compiler behavior.

12:43 cemerick: dnolen: then I'm confused about this result: https://gist.github.com/a2247da58bf5a971d7e7

12:43 mec: xian: any luck with your slime repl question so far? I've been trying to figure that one out too

12:46 cemerick: dnolen: Right — so, you're right, it's not as smart as I thought, but now I don't know what to make of it at all: https://gist.github.com/a2247da58bf5a971d7e7

12:47 dnolen: cemerick: there are no smarts in it *at all*.

12:48 cemerick: dnolen: Just a pinch, given the reflection warning there -- it's checking the implemented prim interfaces, and would seem to be finding a long return for arity 1.

12:48 dnolen: cemerick: I reiterate, what you want is type inference :) now who's gonna work on that?

12:49 cemerick: In any case, how does that impact how return information is represented in the metadata and the syntactic placement of hints?

12:50 dnolen: cemerick: I think you misunderstand how it was before. There was never any return information. fn vars were hinted.

12:50 raek: xian: mec: I think durendal can do it: durendal-enable-slime-repl-font-lock

12:50 hiredman: from what I've read it sounds like hm has problems with oop type systems (like the jvms)

12:50 mec: raek: thanks

12:51 raek: xian: mec: install it with package.el (M-x package-list-packages, install "durendal")or get it from https://github.com/technomancy/durendal

12:52 cemerick: dnolen: which trickled down to the metadata of fns themselves

12:53 * cemerick is waving at the difference between :tag and "proper" return types

12:53 hiredman: cemerick: what makes you say that?

12:53 dnolen: cemerick: uh not as far I understand

12:53 hiredman: the compiler grabs the tag from the var

12:54 in the case of vars

12:54 cemerick: hiredman: true enough

12:54 hiredman, dnolen: https://gist.github.com/5eb24e0d2b3525b04601

12:55 hiredman: the other case is hints on forms, like ^String (foo 1) which are only used in a local context, does

12:56 cemerick: the arg lists are wrong in the second one, are you sure you are looking at the right thing?

12:57 cemerick: hrm, that does look wonky

12:57 * cemerick kicks his REPL

12:59 dnolen: cemerick: in my repl, the fn has no tag information at all, only the var.

12:59 hiredman: there are two type hint paths, type hinting on forms, and type hinting on vars, I imagine defn copies the type hint on to the fn form so that recursive calls via the lexical binding of the fn name have access to the type hint that way

12:59 singe (defn foo [] ...) expands to something like (def foo (fn foo [] ...))

13:02 dnolen: cemerick: https://gist.github.com/966873

13:03 cemerick: dnolen: yeah, I'm getting that now too -- but redefining the same var leads to odd results: https://gist.github.com/248246a3ac545a20a7b3

13:03 Speaking of which, I seem to recall a bug along these lines…

13:04 hiredman: something about copying metadata?

13:06 cemerick: Yeah, IIRC there was some tension around whether the var's info should be dropped or merged -- though I don't ever remember seeing fn values taking on prior var's metadata…

13:12 hiredman: could also be something sort of racey, like the fn taks the vars metadata, but does it before the new metadata gets added to the var

13:14 dnolen: cemerick: so to unify tags and primitive type-hints, exprs now might have multiple possible tags which need to be selected based on context. This code will need to account for arity *and* types.

13:16 cemerick: dnolen: That's more ambitious than I'm proposing -- all I'm saying is that primitive returns should be in var metadata too.

13:16 dnolen: cemerick: what good would that do?

13:17 cemerick: (and get that meta onto the fn's)

13:18 dnolen: make things simpler for tooling or anyone else that cares about return types at runtime

13:19 jarpiain: cemerick: the compiler does look at the primitives metadata on (:arglists (meta #'foo))

13:19 gfrlog: TimMc: I take it back. I think you can do efficient point-free ifs without using laziness

13:21 dnolen: cemerick: ah, so you just want extra information to flow through.

13:21 cemerick: jarpiain: so it does; I had forgotten that Long. also takes a String in https://gist.github.com/a2247da58bf5a971d7e7

13:22 dnolen: Right, a consistent representation of returns. I especially don't think that anyone should ever have to touch the IFn$XXXX interfaces.

13:23 It's been suggested that those interfaces could be eliminated entirely and generated as needed, anyway.

13:23 (also eliminating the 4-arg limit)

13:24 dnolen: cemerick: who suggested that?

13:24 cemerick: dnolen: I don't recall.

13:24 hiredman: anyone can suggest things

13:25 dnolen: cemerick: what I mean was, did rhickey think that could actually work or some random person :)

13:25 hiredman: I might suggest that tomorrow we make the sun orbit the earth

13:25 seems unlikely

13:25 cemerick: dnolen: I think this was post-rhickey-in-#clojure :-)

13:26 dnolen: cemerick: well I don't see how that could work.

13:26 * gfrlog likes to think the phrase "rhickey-in-#clojure" means something similar to "clojure-in-clojure"

13:31 cemerick: dnolen: it doesn't seem entirely absurd. Similar to proxy class generation, perhaps.

13:34 fliebel: dnolen: Do you know about a levenschtein distance in prolog, minikanren or logos?

13:37 dnolen: fliebel: none of the above :)

13:38 fliebel: dnolen: do you think it'd be easy, tasty and short, or just stupid?

13:40 dnolen: fliebel: what benefit do you see over just doing it in pure Clojure?

13:47 xian: raek: Thanks a lot.

13:50 TimMc: gfrlog: Try doing recursive fib or fact in point-free, then.

13:54 gfrlog: TimMc: that is a terribly good example. I don't have anything at the moment, but hopefully soon.

13:56 xian: raek: Unfortunately it doesn't quite work. M-x durendal apropos didn't list durendal-enable-slime-repl-font-lock at all. I set it to t anyway in my .emacs, but square brackets are still not highlighted.

13:56 mec: Have you perhaps had more luck with durendal?

13:57 mec: xian: havn't tried it yet

14:38 raek: xian: oh, sorry. looks like that function is not an interactive one. also, the version of durendal that is in the marmalade package.el repo is old and does not have it

14:39 xian: if you have the most recent version installed, I think you enable it with by having (durendal-enable) in your emacs configuration

14:43 ok, this did not apparently work...

14:48 manutter: Does durendal work with emacs 23? I had it enabled but started having weird issues, and just panicked and bailed out.

14:49 raek: I have used the version from the marmalade repo regularly without weird issues

14:49 technomancy: durendal needs a hero to champion it

14:50 manutter: though in fairness to durendal that could also have been the result of my other clueless attempts to "improve" my .emacs file

14:50 technomancy: anyone in here named roland?

14:50 manutter: lol

14:50 raek: ok, it worked. just my "method of installation" that was a bit flawed

14:50 manutter: raek: what's the marmalade repo?

14:50 technomancy: oh hey, this guy is: https://github.com/rosado

14:50 raek: manutter: a package.el repo, like ELPA

14:51 manutter: Ah, I'm coming from elpa

14:51 eh, I'll look it up later, don't have time to mess with it now anywho

14:51 tks

14:51 raek: emacs apparently used the byte-compiled version of the old file, so that's why I didn't see the changes

14:51 manutter: http://marmalade-repo.org/

14:51 for future reference

14:52 manutter: cool, I should be able to remember that one. Tks again

14:52 * raek is still learning how installation of emacs "packages" in all their forms work

14:54 gfrlog: ,((ancestors (class [])) clojure.lang.ISeq)

14:54 clojurebot: nil

14:56 gfrlog: I can recur from a multimethod and it will hit up the dispatch function again?

15:02 raek: (macroexpand-1 '(defmethod foo :bar [x y] (+ x y))) --> (. foo clojure.core/addMethod :bar (clojure.core/fn [x y] (+ x y)))

15:02 technomancy: gfrlog: I don't think so

15:02 cemerick: gfodor: no, recur's target is always the inner-most function head or loop

15:02 raek: gfrlog: probably not

15:02 joly: have to look it up via get-method?

15:02 gfrlog: oh dang

15:03 it's okay, I don't need tail call anyhow

15:03 manutter: seems like you could use trampoline for something like that...

15:04 gfrlog: prolly cood

15:04 could

15:04 man that was a weird mispelling

15:04 manutter: hey, it rhymes with "good" :)

15:04 * gfrlog wonders if he spelled "mispelling" correctly

15:05 gfrlog: yeah but I don't think I've done that the entire time I've been alive

15:05 strange to start now

15:06 manutter: IPD -- Intermittent Phonetic Dyslexia, the occasional substitution of a spelling that would sound the same. (See: LOLCATS)

15:06 gfrlog: maybe it'll happen to me once every few decades

15:07 s/once/wunse

15:07 sexpbot: <gfrlog> maybe it'll happen to me wunse every few decades

15:07 manutter: apparently it gets worse over time...

15:08 :)

15:09 gfrlog: dang gray matter

15:09 why does having a brain have to be so unfathomably strange?

15:14 raek: anyone know where the magic &env and &form macro parameters are documented?

15:17 manutter: ,(find-doc "&env")

15:17 clojurebot: nil

15:17 manutter: eh, worth a try

15:20 TimMc: raek: What's this now?

15:21 _fogus_: raek: http://blog.jayfields.com/2011/02/clojure-and.html

15:24 raek: fogus`away: thanks

15:27 TimMc: I guess it's not an official feature.

15:29 raek: found it here: https://github.com/clojure/clojure/blob/1.2.x/changes.txt

15:30 "Macros now have access to implicit arguments: * &form - the macro form * &env - the local bindings"

15:34 (defmacro captor [] `(zipmap '~(keys &env) (list ~@(keys &env))))

15:34 (let [x 1, y 2] (captor)) --> {x 1, y 2}

15:34 interesting...

15:37 mec: how does (list ~@(keys &env)) produce the values from &env

15:38 raek: mec: the (let [x 1, y 2] (captor)) call expands into (let [x 1, y 2] (zipmap '(x y) (list x y))

15:38 )

15:39 so &env is not used to get the values (they don't exist until the code is evaluated)

15:39 mec: ah i did macroexpand and just got (clojure.core/zipmap (quote nil) (clojure.core/list))

15:40 which now makes sense

16:00 markoman: how can I get an unique list of this [{:id 1}{:id 2}{:id 1}] -> [{:id 1}{:id 2}]

16:01 amalloy: &(map first (vals (group-by :id) [{:id 1}{:id 2}{:id 1}]))

16:01 sexpbot: java.lang.IllegalArgumentException: Wrong number of args (1) passed to: core$group-by

16:01 markoman: maps has other keywords too but i need to filter by id

16:01 amalloy: &(map first (vals (group-by :id [{:id 1}{:id 2}{:id 1}])))

16:01 sexpbot: ⟹ ({:id 1} {:id 2})

16:02 amalloy: but if you need to preserve other keys, your question is not fully-defined

16:02 raek: markoman: what should happen when to maps have the same id?

16:03 markoman: yes, I need to preserve other keys and any of the maps could be picked, rest just thrown away

16:04 amalloy: then there you go. my snippet should work fine

16:05 raek: ,(let [ms [{:id 1, :x 1} {:id 2, :x 2}{:id 1, :x 3}]] (->> ms (map (juxt :id identity)) (into {}) (vals)))

16:05 clojurebot: ({:id 1, :x 3} {:id 2, :x 2})

16:08 amalloy: interesting take on the problem, but isn't your map/juxt/into just group-by?

16:10 raek: at least, it does something very similar

16:10 markoman: yeah, both work, they give different map for :id 1 but it doesnt matter

16:19 zippy314: newbie q: In this code: (defn- write-json-fancy-type [x #^PrintWriter out] (write-json-string (str x) out)) what does the #^ construct mean in the parameter list?

16:19 gfrlog: type hint

16:20 zippy314: oh!

16:20 gfrlog: by "hint" I believe we mean that it's not strictly necessary, but it may help optimize

16:20 zippy314: its a hint for out, I assume.

16:20 gfrlog: yes

16:20 zippy314: tx!

16:20 gfrlog: np

16:20 amalloy: zippy314: #^ is deprecated now, btw; if you write new code just use ^

16:21 gfrlog: anybody know why this blows the stack? https://gist.github.com/967259

16:22 amalloy: gfrlog: haven't looked yet, but i'ma go with "too much recursion"

16:22 specifically, "loop/recur on a lazy-seq" is very popular

16:22 gfrlog: is neither

16:22 no such thing in the code

16:22 amalloy: man. if i were the compiler i would crash your computer just to get you to go away

16:23 gfrlog: ah that might be what's happening

16:25 maybe if I pay it more...

16:27 Mladej: gfrlog: Eh, are you really sure it can't be rewritten in more readable way?

16:28 gfrlog: but I pretty-printed it!

16:28 Mladej: :)

16:29 TimMc: raek: Does that example with "captor" imply one could write a macro that expands differently when run vs. macro-expanded?

16:30 raek: TimMc: it does indeed imply that the context (which local variables are present) can affect the macro

16:31 TimMc: That could make testing of $env-using macros tricky.

16:31 &env, rather

16:31 sexpbot: java.lang.Exception: Unable to resolve symbol: env in this context

16:38 dnolen: any opinions on pattern matching as it exists in Scala for those people that have used it?

16:44 Mladej: I've used pattern matching only in F#, so I don't know how it works in Scala. But try to look at http://www.brool.com/index.php/pattern-matching-in-clojure

16:44 is it what you are trying to find?

16:44 hiredman: Mladej: dnolen is writing pattern matching, not using it

16:45 Mladej: aha, sorry

16:45 dnolen: Mladej: yeah I'm more interested in a experience report

16:45 * gfrlog liked it in erlang

16:46 dnolen: even better would be someone who understands how it works in SML/OCaml/Haskell

16:46 TimMc: dnolen: type inference next plz k thx :-P

16:46 dnolen: gfrlog: does Erlang support guards on patterns?

16:47 gfrlog: dnolen: it's been a while but I'd give that a pretty confident 'yes'

16:47 dnolen: TimMc: heh that's a big project, hopefully we can drum up some community interest.

16:47 gfrlog: TimMc: my first hack at the factorial function is blowing the stack for no apparent reason

16:48 TimMc: dnolen: Where community == people who can hammer on clojure itself?

16:49 dnolen: TimMc: Clojure compiler foo not necessary as far as I can tell, familiarity with logic programming, constraint programming very useful.

16:49 s/foo/fu

16:49 sexpbot: <dnolen> TimMc: Clojure compiler fu not necessary as far as I can tell, familiarity with logic programming, constraint programming very useful.

16:51 dnolen: gfrlog: ah yes, Erlang has pattern guards.

16:53 gfrlog: dnolen: most of my hesitation was not knowing if "guard" meant something different in the other languages

16:53 dnolen: gfrlog: yeah it's just a test to be run after the pattern match initially succeeds.

16:59 gfrlog: does it make sense for one of my tests to throw a stack overflow error, yet the stack trace shows no user code at all? (not even clojure.test)

17:00 pdk: stack traces tend to get absurdly long with clojure core calls

17:01 gfrlog: yes I don't mind that it's long. I mind that I can't tell where it came from.

17:04 dnolen: damn I'm wondering if I'm wandering down a rats nest here, http://scala-programming-language.1934581.n4.nabble.com/Possible-scala-partial-function-pattern-matching-bug-td3029632.html#none

17:04 one dude works on the Scala pattern matcher, no ones ever tried to help him w/ patches to the pattern matcher.

17:04 hiredman: well, it is scala

17:06 dnolen: hiredman: possibly, understanding efficient pattern matching is not hard, but it does require reading some esoteric papers to see how it's done. w/o prior context I imagine deciphering a pattern matching code base would be quite difficult.

17:07 hiredman: dnolen: sure, and the scala language is complex as it is

17:11 gfrlog: TimMc: nevermind, doesn't work.

17:13 * dnolen thinks maybe he should write the predicate dispatch library with marginalia to encourage patches.

17:20 markoman: hmh.. im pulling data from database by search criterias and I need to filter result. I got help already to get unique items only, but my next problem is to find items, that are available on all sets (and clause)

17:21 gfrlog: markoman: can you give an example of the data you have and what you want to do with it?

17:21 raek: markoman: can't you solve the uniqueness problem in the database query?

17:22 markoman: [[{:id 1}{:id 2}{:id 3}] [{:id 1}{:id 2}{:id 4}] [{:id 1}{:id 2}{:id 3}]] -> [{:id 1}{:id 2}]

17:23 im using google datastore, that doenst allow to bind tables in a way sql does, so this needs to be done programic way

17:24 gfrlog: markoman: you could create sets with those, so you would have [#{1 2 3} #{1 2 4} #{1 2 3}]

17:24 then you could take the union of the sets

17:24 ,(reduce clojure.set/union [#{1 2 3} #{1 2 4} #{1 2 3}])

17:24 clojurebot: #{1 2 3 4}

17:24 markoman: oh, I still need to preserve the content of maps

17:24 gfrlog: wait

17:24 intersection I mean

17:25 I always get that backwards

17:25 markoman: is the content of one {:id 1} the same as the other {:id 1}?

17:29 markoman: hmh, let me think a bit

17:33 this is a bit more complicated I guess

17:33 gfrlog: :)

17:37 markoman: I need to make a pastebin, hard to explain here on one line

17:43 http://pastebin.com/hzgiAgQt this could give better idea of the problem

17:45 so :id I told earlier is same as :parent

17:47 gfrlog: markoman: these are some kind of java objects?

17:47 TimMc: gfrlog: No luck on pointless fact?

17:47 gfrlog: TimMc: funny you say that, I just succeeded a second ago; trying to make a pastie now

17:48 if I can figure out why (with-out-writer) doesn't exist

17:48 I just wanna call (pprint) and have it go to a file...

17:49 markoman: these are entities from datastore. types I guess

17:49 raek: gfrlog: use binding: (binding [*out* some-print-writer] ...)

17:50 gfrlog: raek: I just found it in c.c.duck-streams, which is faster than thinking about how to get a print-writer for a file

17:50 this is repl code, so I don't need it to be good, just effective :)

17:50 raek: gfrlog: use clojure.java.io instead.

17:51 (binding [*out* (PrintWriter. (io/writer ...))] ...)

17:51 gfrlog: raek: next time, next time...

17:51 raek: most functions of duck-streams have been transfered to clojure.java.io

17:52 markoman: but it simplifies to the next form I think: [[#<Key Form("form5")/SavedForm(130)>, #<Key Form("form5")/SavedForm(131)>] [#<Key Form("form5")/SavedForm(130)>] [#<Key Form("form5")/SavedForm(130)>]] -> [['a 'b] ['a] ['a]] -> ['a]

17:52 raek: some even to clojure.core

17:52 markoman: and if [['a 'b] ['a] ['c]] -> []

17:52 gfrlog: TimMc: https://gist.github.com/967464

17:53 I even did it in under 250 lines :)

17:53 markoman: so in the case of the symbols, you can go back to the set intersection idea

17:54 ,(let [xs [['a 'b] ['a] ['a]]] (->> xs (map set) (reduce clojure.set/intersection)))

17:54 clojurebot: #{a}

17:54 * TimMc looks at gfrlog's paste and cries

17:54 gfrlog: TimMc: next is to make it a little smaller maybe

17:55 TimMc: Can you paste your point-ful original as well?

17:55 gfrlog: yep one sec

17:56 markoman: ,(reduce clojure.set/intersection [#{'a 'b} #{'a} #{'c}])

17:56 clojurebot: #{}

17:57 gfrlog: TimMc: I added it as a comment on the gist

17:58 markoman: how do you turn result back to list , with (into [] #{'a}) ?

17:58 gfrlog: markoman: that could work, although most of the time a set will work just as well

17:58 raek: markoman: [] means vector, () means sequence or list

17:59 gfrlog: clojure: I still can't decide if it's confusing or not

17:59 TheMoonMaster: gfrlog: It is

17:59 gfrlog: ah, good to know

17:59 raek: all clojure's sequence functions (e.g. filter, map, ...) will call (seq ...) on the argument, so pouring the result into another data structure is often not needed

18:00 ...unless you want to use one of its features, like constant time random access for vectors or constant time membership test for sets

18:00 gfrlog: TimMc: this could be the beginnings of a code obfuscation library

18:01 markoman: I see. I may have tried to get vectors unnecessarily. and I think problem solved easily at the end, thanks

18:08 oh, one more thing, how do you make a set from this: [[{:id 1}{:id 2}{:id 3}] [{:id 1}{:id 2}{:id 4}] [{:id 1}{:id 2}{:id 3}]] ?

18:08 -> [#{1 2 3} #{1 2 4} #{1 2 3}]

18:08 gfrlog: welp...

18:09 (for [rec-vector [.....]] (->> rec-vector (map :id) (set)))

18:09 I think that should work

18:10 amalloy: gfrlog: ##(set [1 2 3])

18:10 sexpbot: ⟹ #{1 2 3}

18:10 amalloy: dang it. okay, you're right

18:10 gfrlog: amalloy: HA

18:10 amalloy: i always forget which functions return #{1 2 3} and which return #{[1 2 3]}

18:10 gfrlog: $findfn 1 2 3 #{1 2 3}

18:10 sexpbot: [clojure.core/hash-set clojure.core/sorted-set]

18:10 gfrlog: ah that one

18:10 amalloy: yeah, i know

18:10 gfrlog: I was about to question if anything did that

18:10 cause I've never used it

18:11 but probably could have

18:11 amalloy: which makes it hard to turn something into a sorted-set. you have to go like ##(into (sorted-set) [2 5 7 12 50])

18:11 sexpbot: ⟹ #{2 5 7 12 50}

18:11 gfrlog: amalloy: do you use sorted sets often?

18:11 amalloy: no

18:13 gfrlog: amalloy: I guess that's the advantage to only knowing one of the two versions -- I was sure of what it did :)

18:14 markoman: ,(reduce clojure.set/intersection (for [rec-vector [[{:id 1}{:id 2}{:id 3}] [{:id 1}{:id 2}{:id 4}] [{:id 1}{:id 2}{:id 3}]]] (->> rec-vector (map :id) (set))))

18:14 clojurebot: #{1 2}

18:15 markoman: I think it works :) thanks

18:15 gfrlog: woohoo

19:14 TimMc: gfrlog: OK, so you're doing explicit recursion.

19:15 (re: the fact gist comment)

19:42 gfrlog: TimMc: if that's what you call explicit...

19:42 I would call that "indirect", but I'm not in charge of words

19:46 pdk: okay stab in the dark here

19:46 but is anyone here familiar with imap

19:47 gfrlog: that is like a normal map function but made by apple?

19:47 pdk: sigh

19:47 the email protocol :p

19:48 tl;dr trying to figure out if an imap search query i'm trying to write is valid since the server isn't taking it

19:54 TimMc: gfrlog: "indirect"... yes, that's the word

19:54 "explicit" is ambiguous

19:55 gfrlog: TimMc: for normal recursion you would say "direct"?

20:13 TimMc: I suppose so.

20:13 Or just "recursion".

20:22 amalloy: for normal recursion i would say "yes please"

20:24 * gfrlog takes a long sip from a warm mug of normal recursion

21:28 technomancy: so has anyone used jgit?

21:28 is there a version of it that doesn't return nil for every operation?

21:28 because that would be super.

21:29 alandipert: technomancy: folks are still catching on to the whole expression vs. statement thing :(

21:30 ChristianMarks: Noob with paste here: http://paste.lisp.org/+2M3K

21:31 technomancy: alandipert: heh... if only

21:31 this actually claims that every repository simply has no commits in it.

21:32 ChristianMarks: I know neither Java nor Clojure, but decided to learn both by translating the abysmal button demo from Java to Clojure, and learn Leiningen while doing it. I suppose the connoisseurs frown on such undertakings.

21:33 alandipert: ChristianMarks: not at all, i admire your tenacity

21:33 ChristianMarks: It took me a couple days.

21:34 The paste at http://paste.lisp.org/+2M3K can be mercilessly criticized.

21:37 alandipert: ChristianMarks: no need for (if (nil? x) ..., can just be (if x .. and then swap your then/else forms

21:37 ChristianMarks: Ah -- evidence of non-functional noobery right there, thanks.

21:39 tomoj: CamelCase is unpopular

21:39 e.g. create-image-icon and img-url would be more common

21:39 alandipert: ChristianMarks: which language are you coming from?

21:40 TheBusby: Hmm, (read-lines "1GB_file.tsv") is initially consuming 2+GB of memory. Is that a big red flag here?

21:40 tomoj: looks like you're calling the result of the System/err println

21:41 ChristianMarks: I've toyed with common lisp and Haskell. Mostly python, C, C++ various scripting languages.

21:41 OCaml

21:41 tomoj: System/err doesn't have a println anyway, does it?

21:42 ChristianMarks: Don't know. The point is I don't know Java or Clojure. This is the result.

21:43 tomoj: seems (clojure.contrib.io/with-out-writer System/err (println "foo")) works, but ugh

21:44 gfrlog: that is strange isn't it?

21:44 maybe they stopped once they got the Hello World bootstrapped

21:44 ChristianMarks: There is a System.err.printlin

21:45 tomoj: ah, yes

21:45 I was trying to use it like the println function

21:45 gfrlog: maybe they kept going once they got the Hello World bootstrapped

21:45 tomoj: so the . special form is unpopular as well

21:46 prefer (.println System/err "foo") to (. System/err (println "foo"))

21:46 ChristianMarks: OK, I'll make the requisite changes.

21:46 tomoj: especially unpopular are lonely parens :D

21:46 gfrlog: ,(println ())

21:46 clojurebot: ()

21:47 ChristianMarks: This will elevate my status from low-grade moron to imbecile. Thank you.

21:48 tomoj: imo that you're here means you're probably quite a ways beyond those :)

21:49 this is not really a criticism, but it is common to use _ for a binding when you ignore it, like (fn [_] ..) for event handlers that ignore the event

21:49 (fn [e] ..) is fine too I'd say, though

21:49 gfrlog: ,((fn [_ _] (+ _ _)) 3 4)

21:49 clojurebot: 8

21:49 ChristianMarks: OK

21:50 That's a common convention in functional programming

21:50 * gfrlog isn't sure why he didn't expect that

21:51 ChristianMarks: I think the clojure version is better--warts and all--than the Java original

21:53 alandipert: gfrlog: that is a bit of a mind bender

21:53 tomoj: initialize-buttons semantics seems kind of unclojurey, but it's java so..

21:54 gfrlog: alandipert: the bindings are done in order, so the second overrides the first

21:54 tomoj: (I don't mean the way its code looks, I mean the way it side-effects on its arguments instead of taking some data and returning buttons)

21:56 ChristianMarks: You're right.

21:56 Maybe I should stick with Standard ML

21:56 tomoj: but the stuff it actually does is so arbitrary that any way you write it its gonna look funny I think

21:57 ChristianMarks: That's Java for you: anti-modular and anti-parallel

21:58 amalloy: someone commented already on the use of (nil?), but the ((....)) after it looks way wrong

21:58 alandipert: ChristianMarks: stuart sierra has a series of posts on his blog about clojure + swing, and using macros to alleviate pain... have you run into them?

21:59 ChristianMarks: A few of them.

21:59 The code shows that they haven't completely penetrated my skull.

21:59 amalloy: ChristianMarks: if-let will make you happy

22:00 (if-let [img-url (...)] (ImageIcon. img-url) (.println System/err ...))

22:00 ChristianMarks: That's cool

22:00 gfrlog: there's a convenience function for just about everything except checking if an item is in a collection

22:01 amalloy: i'm not sure what your intent was with the ((. construct, but it's definitely wrong. if you can tell me what you meant it to do, i may have a suggestion

22:01 ChristianMarks: I can see that if-let is superior

22:01 The supernumerary parentheses?

22:01 amalloy: yes

22:02 see also the ##(doc doto) macro, which is CRAZY AWESOME for interop

22:02 sexpbot: ⟹ "Macro ([x & forms]); Evaluates x then calls all of the methods and functions with the value of x supplied at the front of the given arguments. The forms are evaluated in order. Returns x. (doto (new java.util.HashMap) (.put \"a\" 1) (.put \"b\" 2))"

22:02 ChristianMarks: They should be removed

22:02 amalloy: &(macroexpand '(doto b1 (.setText blah) (.setSize bar)))

22:02 sexpbot: ⟹ (let* [G__11708 b1] (.setText G__11708 blah) (.setSize G__11708 bar) G__11708)

22:02 ChristianMarks: I'm using doto

22:03 amalloy: oh you are, in -main

22:03 use it in initialize-buttons too :P

22:03 * gfrlog goes off to write a be-done-to-by macro

22:03 amalloy: gfrlog: big fan of the passive voice?

22:03 TheBusby: I apologize for interupting, but I can't seem to load a 1GB TSV file into a repl with 8GB of heap. Any pointers to blogs/etc for handling data and memory?

22:04 amalloy: TheBusby: do it lazily? don't try to load it all at once

22:05 TheBusby: amalloy: unfortunately I need the entire data set available

22:05 amalloy: then uh. don't load all the *text* at once. build up a data structure as you parse it?

22:05 if you need all the data at once, and all the data doesn't fit in memory, not a lot you can do

22:05 TheBusby: here is what I'm doing now that doesn't work, http://pastie.org/1891378

22:06 1GB < 8GB

22:06 gfrlog: amalloy: who have strunk and white been thought that they are anyhow?

22:07 amalloy: TheBusby: don't use read-lines

22:07 line-seq

22:07 not that this will really help, probably

22:07 TheBusby: I should use line-seq, or avoid read-lines because it uses line-seq?

22:08 amalloy: use line-seq

22:08 i forget what the deal is with read-lines but it's deprecated

22:08 TheBusby: k, thanks. I checked readlines and it appears to be consuming about 2.2GB of RAM to load the 1GB file.

22:09 gfrlog: gotta have one byte for the char and 8 bytes to point to it

22:09 TheBusby: if that was the only consumer if memory I could deal with it, but the parsing or storage is blowing that up quite a bit

22:09 amalloy: TheBusby: re-split is probably causing your problem

22:09 * gfrlog is gone away

22:10 amalloy: maybe

22:10 TheBusby: k, can check that

22:10 amalloy: it is probably doubling memory consumption

22:10 TheBusby: ah, yeah that did it

22:11 amalloy: i'd also verify that your repl *actually* has 8gb of memory

22:11 TheBusby: since it's hard to determine memory used by the JVM

22:11 I've been changing the max heap size, and if the process dies due to GC I'm just figuring it is out of mem

22:12 Everything works fine with 16GB heap, but dies at less than 8GB

22:12 amalloy: and put some doalls in front of parse-line and/or map, perhaps, since i can imagine it's keeping everything in memory twice

22:12 TheBusby: what do you mean?

22:13 amalloy: TheBusby: (zipmap fields (parse-line %)) is going to return a map whose values are all lazy-seqs closing around the original input data

22:14 TheBusby: ahh

22:14 great catch, thank you for pointing that out

22:15 amalloy: how many columns are in each row of the tsv?

22:15 TheBusby: looking at 20-30

22:16 though I'd like to use this function with up to 200 or so... :)

22:17 amalloy: i mean, you're storing a shedload of duplicated header data in each item

22:17 gotta be like half a kilobyte spent on pointers and trees and stuff to make the map of keywords to data

22:17 TheBusby: was wondering about that, was hoping keys resolved to something smaller

22:18 amalloy: TheBusby: the keywords themselves will be tiny. just a pointer

22:18 TheBusby: a 64bit pointer is much larger than an 8bit ID though...

22:18 amalloy: but a pointer is eight bytes, and you have twenty of them, and they have to fit into a tree structure...

22:18 TheBusby: so worth making a customer structure then I guess

22:18 amalloy: not necessarily!

22:19 seems much nicer to return (a) a seq of seqs and (b) a function for looking up some field in a column-seq

22:19 TheBusby: I love the fact you can do (map :key data) you see

22:19 amalloy: that's what my first thought was

22:20 just use the header to determine the offset in each line

22:20 amalloy: right

22:20 TheBusby: would need to do something like (foo key data) to get at the data, but that's not bad either

22:21 amalloy: if you wanted you could (let [get-foo (partial lookup-fn :foo)] (get-foo data-item))

22:22 or, perhaps better, have (lookup-fn :foo) return #(nth % 10)

22:22 TheBusby: oh, that's a great idea

22:23 will give that a try and see what memory consumption looks like. Thank you *very* much!

22:23 amalloy: nothing you can't solve with another level of indirection

22:23 TheBusby: Er, I wouldn't say that in regarding memory consumption though. ;) Not unless you're counting JNI

22:24 amalloy: hey, it worked here. we added a level of indirection to the lookup function, and in return saved a bunch of memory on the data structures

22:24 TheBusby: Excellent point

22:29 ChristianMarks: I believe I have taken all of your suggested revisions (at least those I was able to absorb): http://paste.lisp.org/+2M3K/1

22:29 TheBusby: Just FYI, read-lines and (line-seq (reader)) show the same memory usage

22:30 amalloy: TheBusby: indeed. but line-seq is less broken in other ways

22:31 ChristianMarks: lookin' pretty good, there. the tabs under the if-let are a bit weird given you've indented everything else with spaces

22:32 ChristianMarks: Hairy arm principle

22:32 TheBusby: amalloy: thanks again for the tip ;)

22:33 amalloy: any tips on what I should be looking for to replace re-split?

22:34 amalloy: TheBusby: not really. there's re-seq, but it will have similar issues

22:35 TheBusby: I vaguely remember java having memory problems with other string operations. Is it work falling back to java.lang.String or something else?

22:35 ChristianMarks: the add-action-listener functions could be cleaned up with another function to set the booleans in order

22:35 But it's still imperative code

22:37 I could nest defns I believe, so that I could use the closure provided by the outer defn...

22:37 amalloy: ChristianMarks: you might like some of the changes at https://gist.github.com/967834

22:37 or you might not; sorta a style thing

22:38 errrr, and the doseq should be OUTside of the let-bindings area. i was improvising a bit

22:38 datka: does aleph work with clojure 1.3?

22:39 ChristianMarks: amalloy: that's great. Intuitively obvious.

22:39 datka: I'm thinking about switching my app over

22:39 ChristianMarks: Same thing for setting the button booleans

22:39 amalloy: if you adjust initialize-buttons to take an array of buttons, you can even replace [b1 b2 b3 :as buttons] with just buttons

22:40 technomancy: so good: http://memegenerator.net/instance/7741684

22:40 amalloy: technomancy: haha that kills me

22:41 technomancy: amalloy: yeah, ztellman's beard is getting pretty distinguished

22:42 amalloy: i guess i'm glad i don't depend on his libs, then?

22:42 tomoj: datka: tests fail

22:42 appears potemkin is broken

22:43 (at least)

22:44 datka: ok, then that would explain what I'm seeing

22:44 I've never been a big fan of the immegrate-like functionality

22:46 tomoj: I patched it to not fuck up M-. and try to pretend it's not there :/

22:53 amalloy: ChristianMarks: you might (defn enable [b] (.setEnabled b true)) and similarly for disable

22:54 ChristianMarks: That's better than (for [[button flag] button-flags (,setEnabled button flag)) ...?

22:54 I meant . not ,

22:54 no macro intended

22:55 amalloy: up to you

22:55 ChristianMarks: left out ] after button-flags too

22:55 Early onset senility

22:55 I liked the destructuring bind

22:55 amalloy: go for it

22:55 but save yourself, use doseq instead of for

22:55 for would make you cry

22:56 &(let [s (range 5)] (for [x s] (println x)) 'done)

22:56 sexpbot: ⟹ done

22:56 amalloy: &(let [s (range 5)] (doseq [x s] (println x)) 'done)

22:56 sexpbot: ⟹ 0 1 2 3 4 done

22:58 ChristianMarks: you can use a destructuring bind within the doseq...

22:58 amalloy: doseq and for have identical syntax

22:58 ChristianMarks: OK

22:58 amalloy: but for is lazy and returns a result, while doseq is eager and returns nil

22:59 ChristianMarks: I prefer eager functions (lazyness leads to monadification, which tends to make functional programs look imperative)

22:59 amalloy: &(doseq [[a b] (partition 2 1 (range 5))] (println a b))

22:59 sexpbot: ⟹ 0 1 1 2 2 3 3 4 nil

22:59 tomoj: what is monadification?

22:59 amalloy: tomoj: an irrational fear, it sounds like. but we'll give him a little wiggle room; he's new to laziness

22:59 dnolen: ChristianMarks: laziness in Clojure is not like in Haskell. laziness in Clojure means lazy sequences

23:00 ChristianMarks: OK

23:00 tomoj: amalloy: adrenaline rush when I read it.. :)

23:00 amalloy: hahaha

23:00 dnolen: ,(map inc [1 2 3 4 5])

23:00 clojurebot: (2 3 4 5 6)

23:00 dnolen: ChristianMarks: ^ no fearsome monadification going on here ;)

23:01 ChristianMarks: Good. Robert Harper wrote a post on that on Existential Type

23:01 amalloy: i <3 (partition 2 1 ...)

23:01 dnolen: ChristianMarks: I like his posts thohis SML fanboyism is quite plain, I like SML a lot too.

23:02 at least Haskell is evolving at a clip, can't say that much about SML.

23:03 ChristianMarks: I'm buying his fanboyism. He's a good type theorist. Anyone who follows Voevodsky on homotopy type theory can't be completely mistaken...

23:03 tomoj: hmm

23:04 dnolen: ChristianMarks: but he's no Simon Peyton Jones, who is a great hacker. You need both kinds of thinking in this game.

23:06 tomoj: seems (partition 2 1 x) is 2-3x slower than (map .. x (next x))

23:06 but that probably hardly ever matters

23:07 ChristianMarks: Look, I use xmonad myself. I agree with you, actually. I'd like to see how far Harper's ideas go. He says Haskell's type system is fundamentally broken.

23:07 * technomancy suspects you read all that up in Pinckwerts; the notion that homotopy functioned eugenically was exposed long ago by Glumpe.

23:08 dnolen: ChristianMarks: type systems are fundamentally broken according to Alan Kay. The truth is probably somewhere between these viewpoints.

23:09 amalloy: tomoj: well, partition is allocating new structures

23:09 * technomancy cackles maniacally and re-reads http://enowning.blogspot.com/2010/02/g.html

23:09 dnolen: ChristianMarks: but perhaps you'd like to help a implement a type system / inferencer for Clojure - hint hint nudge nudge

23:09 ChristianMarks: Not a bad idea

23:10 tomoj: amalloy: I tried + and list in the map

23:10 + seemed unfair to map but it was probably unfair to partition, yeah

23:10 dnolen: ChristianMarks: I've been thinking about it a lot, there's definitely interest from the core team - a la carte type systems.

23:11 amalloy: i much prefer the flexibility it gives me, though; destructure the result in a for/doseq instead of creating a new function for use with map, for example

23:11 tomoj: well, you can destructure (map list c (next c)) the same way, right? but it's definitely uglier

23:12 amalloy: &(time (dorun (partition 2 1 (range 1e6))))

23:12 ChristianMarks: I'm in the middle of some NSF grant proposals (completely different area: environmental science) but after I could check in again

23:12 sexpbot: ⟹ "Elapsed time: 3209.967174 msecs" nil

23:12 ChristianMarks: (took a break to have my novice code exposed)

23:12 amalloy: &(time (dorun (apply map list ((juxt identity rest) (range 1e6)))))

23:12 sexpbot: ⟹ "Elapsed time: 1123.962417 msecs" nil

23:13 amalloy: i *also* love (juxt identity foo)

23:13 dnolen: ChristianMarks: cool, I'll be here trying to convince unsuspecting newbies to build type inferencing for Clojure.

23:14 amalloy: ~source partition

23:15 tomoj: I think more interesting questions about partition's performance arise with different values for n and step

23:15 arohner: can swank-cdt pause the process and just print the current stack?

23:15 ChristianMarks: It's a great idea really, because you could use the type inference engine to perform access control, contain side effects...

23:16 tomoj: though maybe the tests I remember running weren't actually testing partition's performance, I think I was dorunning the whole thing and each partition, which automatically is at least O(n*(count coll)) with step=1 (?)

23:16 amalloy: tomoj: from the source it looks like partition is already dorunning the partitions

23:17 (= n (count p))

23:18 tomoj: I think that is ok

23:18 for some reason it scared me

23:19 dnolen: ChristianMarks: yes, a lot of cool things could fall out of it. It could also potentially be more powerful and certainly easier to extend / experiment w/ since it's not hard coded into Clojure itself as most type systems are.

23:20 ChristianMarks: Exactly. It's a great idea.

23:22 Do you plan on modifying the reader (whatever correspond to the lisp function...)

23:23 amalloy: ChristianMarks: the clojure reader isn't very extensible, so that wouldn't work well with the plan of experimenting

23:24 dnolen: ChristianMarks: no, the missing bit is a hook to the compiler, would need to start a discussion as to what that hook would be.

23:24 ChristianMarks: OK -- could lead to horrendous code if it were

23:27 I agree with that approach. I wanted to implement justification logic in clojure -- kind of on hold now.

23:28 I have to split

23:28 Thanks for the advice ... invaluable.

23:47 sorenmacbeth: I wonder if anyone might be able to help me with a problem

23:47 I have a seq like so: ("7" "7 elements" "7 elements landing" "7 elements landing page" "add")

23:48 I want to collapse it such that if the first string is contained within the second string, remove it

23:48 and so on down the seq

23:49 so in my example, I'd want to end up with ("7 elements landing page" "add")

Logging service provided by n01se.net