#clojure log - May 09 2013

The Joy of Clojure
Main Clojure site
Google Group
List of all logged dates

0:00 xeqi: add a BufferedReader. to the end of your reader building

0:00 tomoj: isn't that what io/reader does?

0:01 shriphani: but i have a bufferedreader

0:01 and an io/reader both in there...

0:01 and I still cannot read past the 4300th line.

0:02 tomoj: I don't see the BufferedReader. - did you change the code?

0:02 I tried your refheap with a million lines of "yes" and got a million lines

0:02 probably a bad test case..

0:02 shriphani: tomoj: https://www.refheap.com/paste/14333

0:03 xeqi: tomoj: ah, your right

0:03 tomoj: yeah that gives me 1000000 too

0:03 xeqi: it is what io/reader does

0:04 shriphani: are you reading with the right encoding?

0:04 mthvedt: it does seem problematic that you sometimes need to construct java readers to use clojure stuff that is supposed to abstract those

0:04 shriphani: xeqi, I am guessing no at this point.

0:04 I didn't specify one while writing to the file.

0:05 but the lines it reads in are ok...

0:06 tomoj: mthvedt: there is no abstraction, clojure.java.io just has some helpers

0:06 of course it doesn't have every conceivable helper..

0:06 shriphani: so it possibly reads EOF somewhere where zcat doesn't….

0:06 mthvedt: well, s/abstraction/more accurate word

0:07 tomoj: so what seems problematic, that clojure.java.io doesn't contain every conceivable java.io.* helper? O_o

0:08 mthvedt: that some helpers produce stuff that can't be consumed by other helpers

0:08 without doing the appropriate java rain dance

0:09 tomoj: like what?

0:09 shriphani: and unix tells me line #4301 is not unnatural.

0:10 brehaut: mthvedt: it is java.io, expectations of being java-free are based on incorrect fact gahter

0:10 gathering*

0:10 mthvedt: i'm not talking about java-free

0:13 say you open a file with io/reader

0:13 then you try to use 'read on the result

0:13 ClassCastException java.io.BufferedReader cannot be cast to java.io.PushbackReader

0:14 hiredman: pushbackreaders can by their nature be problematic

0:14 brehaut: back up a moment, read is core, not java.io. a) read isnt a helper b) its in a different module

0:14 hiredman: they can end up reading a character then pushing it back in to an internal buffer, so if you auto wrap with pushback readers you can lose a character

0:15 tomoj: shriphani: I got no clue, I've never had trouble with code like that..

0:16 (without the BufferedReader. since io/reader already returns one)

0:16 hiredman: shriphani: I'd suggest taking a step back and starting from scratch verifying each step

0:17 shriphani: hiredman… I only have 1 function that I am calling from a repl….

0:17 hiredman: we use clojure and gzipinputstreams and gzipoutputstreams and clojure.java.io extensively at work

0:17 so I know they work fine, so the only variable is your setup

0:17 shriphani: I guess my file is corrupt in some way

0:18 hiredman: shriphani: have you restarted your repl? do you have some bogus state in there somewhere?

0:18 shriphani: hiredman, yes

0:18 hiredman: shriphani: gzip can be a little more tolerant of errors than the stream classes, but in my experience in prints warnings if that is the case

0:18 shriphani: so I ran it on another file and it returned with the right number of lines.

0:19 that one had 316k lines in it.

0:19 it is just the 8 million lines file that claims it has 4300...

0:19 and throws no errors..

0:20 hiredman: shriphani: have you checked the result when the file is uncompressed?

0:21 shriphani: I'm trying that right now.

0:24 hiredman, so that gzip file was obtained by doing a cat on two gzip files. I ran a cat the other way round and tried the routine on it and it reports 3200 lines on that one.

0:25 i.e. instead of cat file1.gz file2.gz I did cat file2.gz file1.gz > test_out.gz and I get 3200 on that one… so the files are wonky and hit some odd edge case.

0:33 tomoj: shriphani: hmm, that doesn't work

0:33 shriphani: tomoj ?

0:33 tomoj: GZIPInputStream doesn't expect multiple concatenated gzip streams

0:34 shriphani: tomoj… I the cat of two gzip files is a valid gzip file...

0:35 tomoj: `cat file1 file2 | gzip > test_out.gz` should work

0:36 shriphani: tomoj, this is from the gzip manpage: ADVANCED USAGE

0:36 Multiple compressed files can be concatenated. In this case, gunzip will extract all members at once. For example:

0:36 tomoj: yeah I'm looking at that too

0:36 `gzip -c file1 file2 > foo.gz` is also readable by GZIPInputStream

0:36 just not `cat file1.gz file2.gz > foo.gz`

0:36 shriphani: ....

0:37 wow...

0:37 tomoj: (which I don't see in the manpage despite the "Multiple compressed files can be concatenated"

0:38 http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4691425

0:40 shriphani: umm that bug report says Resolved...

0:40 so they just closed the ticket ?

0:40 tomoj: well as a wontfix..

0:40 shriphani: fantastic.

0:40 tomoj: http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4763158

0:40 that being the 'real bug'

0:40 shriphani: This assumption isn't documented within any of the J2SE documentation as far as I can tell.....

0:41 man that is just so awesome...

0:47 logen: In order to update a session in ring, I have to pass the request into my view function and assoc my values back into the :session keyword, right?

0:49 tomoj: so, uh, speaking of IO abstractions

0:49 rich has said "we need reducible IO" or something

0:50 consider ZipInputStream

0:50 most natural thing that fell out when I started writing was a reducer where the values are the ZipEntry's

0:50 Apage43: logen: yep

0:50 if you're using the session middleware that ships with ring

0:51 tomoj: well, clearly that won't work if all we have is a ZipInputStream

0:51 hiredman: tomoj: that doesn't track

0:52 logen: Apage43: I keep ending up needing stuff in the request object...is it bad form to just pass the whole request into view functions?

0:52 tomoj: if we have a ZipFile we can get a reducer of entries and use the entries to get whatever an InputStream is as a reducer

0:52 hiredman: track?

0:53 hiredman: no

0:53 you can get entries from a zipinputstream

0:54 Apage43: logen: well it won't break anything. Depends on how concerned you are about separation of concerns. Is there a particular subset of things you tend to need or could it be anything?

0:55 tomoj: yeah, but then you have to have the ZIS still around (and not disturbed further) to read the actual contents of the entry?

0:55 hiredman: which you can do just fine

0:55 in fact, it is that kind of resource handling that makes reducers based io appealing

0:56 logen: Apage43: I need to know if the user is logged in, in order to show a Log In link or show their username if already logged in. POSTs need post params. It seems like most of my view functions end up needing the request.

0:56 hiredman: https://gist.github.com/hiredman/4075459

0:56 shriphani: tomoj, was that ZIJ comment for me ?

0:56 tomoj: shriphani: nope

0:56 shriphani: ah.

0:56 Apage43: logen: sounds like they typically want :session and :params

0:57 logen: Apage43: true, but both are already in request.

0:57 tomoj: hiredman: ByteSource makes sense to me

0:57 hiredman: tomoj: so you would do the same thing with an zipinputstream

0:57 ZipEntrySource or what have you

0:58 Apage43: well yes that's where you'd get them. Passing in the request isn't a big deal, as long as you're passing them to something you wouldn't expect to want to ever use outside of that context

0:58 tomoj: so the user of the ZipEntrySource must have a reference to the ZipInputStream

0:59 shriphani: I am surprised.. with the number of sugar daddies that java has, how does it still manage to not have the resources to fix such bugs ?

0:59 hiredman: tomoj: ideally the user would have see the zis

0:59 Apage43: if you wanted to be able to separately test those fns with different sessions/params, it could get bothersome later if you have to construct a whole request map to do that

0:59 hiredman: just create a zipentrysource and reduce over it

0:59 tomoj: getting ZipEntry values which are useless?

0:59 hiredman: they are not useless

1:00 logen: Apage43: Thanks. I'm trying to learn and do at the same time. The docs for ring + sessions is sparse.

1:00 hiredman: tomoj: if you r/map over them that r/map happens while the stream is still open

1:00 Apage43: logen: mhm. I often just go look at the code https://github.com/mmcgrana/ring/blob/master/ring-core/src/ring/middleware/session.clj :)

1:00 tomoj: but you have to have to see the ZIS to use them

1:00 ..I assumed you meant "should not have to see the ZIS"

1:00 Apage43: ah wait

1:00 https://github.com/ring-clojure/ring/blob/master/ring-core/src/ring/middleware/session.clj

1:01 the mmcgrana repo is old, don't look at that one

1:01 hiredman: tomoj: ok, so you don't return a entry exactly, you return something like an entry with its own internal reference

1:02 tomoj: and now the 'values' coming out of the reducer self-destruct

1:02 hiredman: in what way?

1:02 tomoj: if you don't use it and save it away instead, it's now broken

1:02 hiredman: it isn't

1:02 tomoj: like if you did (into [] the-reducer-of-entry-like-things)

1:03 well it can either read all of the contents for you and save them away so that there's no problem

1:03 hiredman: yes

1:03 tomoj: or it lets you decide which entries' contents to read, in which case you're fucked

1:03 hiredman: or you can just hand on to the unreduced reducer

1:03 which is a promise to do the work, and manage the resources when you want it

1:03 logen: Apage43: Yeah, a lot of time it would just be quicker to read the source than google and find a bunch of old blog posts. I'm trying to get to the point where source doesn't send me off consulting clojure docs every other line.

1:04 hiredman: tomoj: if you run r/filter the filtering happens while the zis is still open

1:05 tomoj: right, OK, so the ZipEntrySource's values contain like byte-arrays with the entire contents of each entry, and you skip the ones you don't care about?

1:05 hiredman: maybe, maybe not depending on what you want and how you do it

1:06 tomoj: what I'm trying to avoid is the requirement that a value coming out of the reducer be used during the reduction

1:07 only seemingly sane way I can see so far (besides always reading everything into memory) is to have the reducer make a copy of the input stream in a temp file so that we can go back and to the entries later

1:07 Apage43: logen: if you're using an editor with some integration, having hotkeys for jump to the source of this function and show me the docs for this function makes exploring much nicer

1:07 hiredman: you can do it in stages (r/map make-valuies (r/filter values-i-want zip-entry-source))

1:08 tomoj: yeah, but only if you have a ZipFile, afaict

1:08 rather than just a ZipInputStream

1:08 hiredman: no, that is just not correct

1:08 tomoj: oh, right, your example works

1:09 hiredman: and you can even do more transforms on the results there

1:09 tomoj: but say (into [] (r/filter non-values-i-want zip-entry-source))

1:09 those aren't values, right?

1:09 hiredman: it depends on how you do it, but sure they may not be

1:10 the source has a lot of control, for example it could look at the result and reify it as a value on the way out

1:11 tomoj: oh, interesting. I hadn't thought of that, will take me a while to process I think. thanks

1:12 hiredman: hmmm, I dunno if that is practical though

1:12 that could get really nasty fast

1:13 tomoj: I don't see what you mean actually - 'result' here is a reducef return value?

1:14 hiredman: yes

1:14 you would have to walk the value

1:14 "that could get really nasty fast"

1:15 tomoj: I suspect my (still nebulous) goals probably cannot be satisfied without saving the input somewhere

1:20 totimkopf: do most web frameworks extend ring?

1:22 tomoj: I wonder what rich meant, we need impure reducer IO?

1:24 I guess for now I'll ignore the impurity since it should work OK. but that can't be what we need..

1:24 I mean, sometimes maybe, but that's not what I want to work with most of the time

1:27 dpwright: hello, can somebody point me in the right direction of a more idiomatic approach to building up a list from repeated calls to a function

1:27 Basically, I have a function (lanterna.screen/get-key) which returns the last pressed key if there was one, or nil otherwise

1:28 right now I'm using loop/recur with an accumulator to put all these keys into a list and drop out when it returns nil

1:28 but I feel like there must already be a function which does that -- runs a function repeatedly, conj/consing the result into an accumulator, until some predicate has been met

1:30 tomoj: (->> f repeatedly (take-while (complement pred))) ?

1:32 dpwright: tomoj: Ooh, a couple of things I didn't know about there.. I've used -> but not ->>, and I've not used take-while

1:32 tomoj: you successfully decoded the ->> I guess?

1:32 dpwright: same as -> but at the end?

1:32 tomoj: indeed

1:33 lanterna.screen/get-key sounds sorta insane

1:34 I mean what does (take-while (complement pred) (repeatedly lanterna.screen/get-key)) even mean

1:34 er, (take-while identity ..)

1:35 it's something like "if the user is pressing keys faster than I can handle them, keep adding to this seq I'm building, otherwise the seq ends"?

1:35 dpwright: yeah, or pressing two keys at once

1:36 although I'm not actually sure whether it does work like that

1:36 it's last key pressed, not all keys currently down

1:37 tomoj: oh, and, I see, it's buffering them up instead of just dropping them

1:37 dpwright: but yeah, I suppose if you were to press two keys precisely simultaneously they'd come in at the same time

1:38 well... actually I'm not sure that it is doing that, heh

1:38 that's what I wanted it to be doing

1:38 but looking back at the documentation, it might not be doing that

1:38 tomoj: I just read "Each call to get-key pops one character off the input buffer and returns it. If there isn't anything on the buffer, it returns nil."

1:39 which makes me not quite want to say 'insane' but still seems nuts

1:39 dpwright: oh good, I'm not going mad though

1:40 it makes sense if you want to treat each key as an event and act on it straight away, I suppose

1:40 probably not all that "functional"?

1:42 this question in particular was more general anyway -- this is not the first time I've used loop/recur in this way... it seems quite common that there's some non-pure function that is pulling in outside information, like new keypresses or incoming network packets or whatever, and I just want to buffer them up

1:47 tomoj: looks like it's lanterna's fault (not clojure-lanterna's)

1:47 reminds me of datomics tx-report-queue though which apparently stu and rich are fine with (?!)

1:49 dpwright: would you mind explaining why it's nuts? I feel like I'm missing something obvious

1:52 tomoj: seems like you have to have exactly one place in your program which reads input

1:52 and then distributes that input out to the multiple places that need it

1:52 dpwright: ah

1:53 tomoj: or you just go full imperative style and write get-key all over the place - then how do you make sure you don't consume an input someone else wants?

1:53 dpwright: yeah, that is why I'm having to do this pulling it into a list thing

1:54 tomoj: I guess a counterargument is that get-key and tx-report-queue are more raw, and they couldn't make the right choice for everyone, so you just suck it up and make the right choice for you in that one place in your program..

1:55 dpwright: but... what's the alternative? otherwise you'd have to have some way of, I don't know, clearing it or something.. or putting timestamp information along with the keypress so you can work out if it's interesting to you or done with, or...

1:55 tomoj: yeah the alternative I really want doesn't belong in lanterna nor datomic (this is why the counterargument occurred to me)

1:56 a more lightweight and common alternative would be like (add-input-listener! terminal (fn [char] ...)) or something

1:56 dpwright: ah right, yeah

1:57 I guess as you say, that's slightly higher level and easy enough to implement on top of lanterna/whatever yourself

1:58 tomoj: yeah I wouldn't expect rich to put that in datomic!

2:00 dpwright: anyway, thanks -- rephrasing the loop/recur into a combination of repeatedly and take-while works and makes the code a lot nicer to read too :-)

2:02 tomoj: actually though

2:02 it does belong in datomic

2:03 hopefully in 1.6

2:03 * tomoj crosses fingers

2:37 tomoj: hiredman: https://www.refheap.com/paste/c85b347e704d5d6ce70a31bef :/

2:44 https://www.refheap.com/paste/caa9c0785111b142ae1c81e91

2:45 seems pretty weird

4:30 should r/map preserve metadata?

5:56 rodnaph: i'm having a strange problem where on my build server leiningen isn't fetching a plugin dependency (lein-bin). it seems to fetch all the others... i'm stumped as to how to debug it, can anyone help?

5:57 fine locally, fine on a VM image which should (should) be identical. but not on the server. it's running inside a mock RPM build root.

6:06 broquaint: Is something purporting to satisfy the lein-bin dependency and thereby obviating lein from pulling it down?

6:08 rodnaph: broquaint: hmm... possibly, how could i check? and why would it work in one env and not another?

6:10 broquaint: I'm not sure, rodnaph, if you open a mock shell you can poke around and see what's what.

6:10 That's my usual approach anyway :)

6:11 rodnaph: ok, i'll see if i can get in. thanks

6:11 noidi_: rodnaph, do you have a ~/.lein/profiles.clj on either machine?

6:12 rodnaph: there shouldn't be... if lein-bin was specified in there would it not get pulled down?

6:12 noidi_: I don't know, that's just the first thing that came to my mind that might be different between the Leiningen installations

6:13 there's also the #leiningen channel, maybe someone over there can help you

6:15 rodnaph: yah will check now (it's usually the same ppl in #leiningen as #clojure though, hehe)

6:22 tomoj: I wonder why the pedestal app model has both :value and :attrs

6:22 is :attrs analogous to metadata?

6:23 if not, why isn't the :value just a map instead?

6:23 svedubois: Do you know any "hello world" tutorial about Prismatic/dommy?

6:36 tomoj: https://github.com/pedestal/samples/blob/master/square-root/src/square_root.clj how does this even?

6:37 {:fn sum :input #{:guess :divide}} {:fn half :input #{:sum}}

6:37 (defn sum [state inputs] ...)

6:37 (defn half [state input-name old new] ...)

6:44 https://github.com/pedestal/pedestal/blob/master/app/src/io/pedestal/app/util/adapters.clj#L42

6:44 OK, so that's deprecated. good.

6:47 https://github.com/pedestal/pedestal/blob/master/app/test/clj/io/pedestal/test/app.clj#L476

6:54 noidi_: Pedestal seems quite daunting at a first glance

6:56 noidi: there's a lot to more to take in than in a typical "data goes in, data comes out" Clojure library

7:07 callen: noidi: seems like a big prank being pulled on the clojure web community.

7:07 "snicker snicker, lets see if they actually think this stuff is good. snicker snicker"

7:43 Okasu: callen: What stuff?

7:44 muhoo: it feels to me like there's a better way to do this: (assoc m :durations (-> m :durations duration-fix*))

7:44 TimMc: callen: Shush, unless you're going to give an actual critique.

7:45 muhoo: i want to swap the value of :durations with the application of a function to it

7:46 i am golfing tho. this is prolly fine

7:50 Anderkent: muhoo: (update-in m [:durations] duration-fix*)

7:50 muhoo: update-in, of course, thanks

7:51 tomoj: dpwright: I think I stumbled upon what you were talking about earlier

7:51 hmm, nevermind

8:03 muhoo: heh, and after no sleep, i write stuff like this (apply cons ((juxt (comp keys first) (partial map vals)) ms))

8:03 * muhoo passes out

8:04 learner: Hello Gurus. A newbie question here. I am reading Clojure STM stuff and I understand a transaction is retried if there is a state change. Is there any way I can print a simple println statement everytime a transaction is retried due to a conflict/state change? Thanks for your help in advance. cheers

8:05 TimMc: learner: Sure, just throw a (println "whoops!") at the top of the dosync.

8:09 learner: TimMc: Hmm, it gets printed for every run even though there is no conflict. Isn't it?

8:14 TimMc: Derp. Yeah, it does. Uhh... you'll need an atom or something.

8:15 (let [tries (atom 0)] (dosync (when-not (zero? @tries) (println ...)) (swap! tries inc) ...))

8:15 That works if there isn't a containing dosync.

8:35 learner: TimMC: I slightly modified example given @ http://clojure.org/refs by adding new atom and printing the total count at the end. And I can see the diff. Thanks for the hint. cheers

8:35 TimMc: I slightly modified example given @ http://clojure.org/refs by adding new atom and printing the total count at the end. And I can see the diff. Thanks for the hint. cheers

9:47 SpindleyQ: Is it just me, or is there no clojure function or library for writing EDN?

9:49 bbloom: SpindleyQ: pr and prn will do it, but you just need to validate yourself that the output will be EDN: you can do that with = and clojure.edn/read

9:51 SpindleyQ: bbloom: but I'd like to use tagged elements to eg. serialize a queue

9:53 bbloom: SpindleyQ: for your own types, you can override the printing mechanisms, but you shouldn't do that for other people's types. beyond that: i agree, it's a problem that tagged literals came after all the core print methods were already defined

9:53 SpindleyQ: bbloom: afaict the way to override pr for different types is with print-dup? But print-dup with even a simple map gets me things like "#=(clojure.lang.PersistentArrayMap/create {:x 3})" which is not helpful

9:54 bbloom: you can defmethod print-method, but you really should not change the print behavior for queue or anything like that

9:54 for your own types, you can just define print-method & it will work

9:58 jtoy: if i have code in project and i want to make a version that is a standalone dameon and a version that runs in a batch mode, would i use the same project? or seperate them?

9:58 so basically build 2 different executables

9:58 tcrayford: jtoy: imo that's purely up to you. You could use lein's profiles to build two different executable uberjars

9:59 jtoy: tcrayford: ah ok, my confusion was how would i build 2 seperate jars, i didnt know about lein profiles, ill try to use that

10:00 tcrayford: see https://github.com/technomancy/leiningen/blob/stable/doc/PROFILES.md for more on that

10:01 bbloom: jtoy: do you really need to separate jars and two separate builds?

10:01 jtoy: could you just have one jar with two different bash scripts to start different main functions?

10:02 tcrayford: jtoy: alternatively you could use a command line switch with your main function

10:02 jtoy: bbloom: yeah, i could do that, i've never written java before clojure though, so im not sure how all the jva parts work

10:02 bbloom: jtoy: in general, java doesn't really like to be run as a normal unix process

10:02 tcrayford: jtoy: if you don't need different jars, I would say just one build with a bash script makes more sense

10:03 jtoy: i dont really need different jars, i could use the same one

10:03 bbloom: jtoy: when you run java code, you tell it which class to look in for the main function. jars can specify in metadata where to look for that

10:04 jtoy: you can either compile two main functions or compile one main function that switches on command line parameters

10:04 the lein folks would need to weigh in on if/how you can get that to play nice with `lein run`, since i haven't messed with that before

10:11 Anderkent: bbloom: you can always lein run -m main.namespace , or give a default main and declare some alias in the project.clj

10:11 bbloom: jtoy: ^^ yeah what Anderkent said

10:11 Anderkent: :run-aliases {:alt [alternative.namespace "arg1" "arg2"]}

10:43 shriphani: hi. I have a question about doall. I have a large file that I read in line-by-line and then apply a function to each line. I currently do: (doall (map f (line-seq in))). Now, is there a way to not force the entire list to be evaluated in case I just need something like (first (routine-that-reads-line-by-line …)) i.e. eval lazily.

10:50 jjttjj: shriphani: can't you just take off the doall?

10:50 shriphani: jjttjj, what if I want the first 100 lines ?

10:51 won't in close ?

10:51 tcrayford: shriphani: combining laziness and io is typically a bad idea

10:51 and it might well close ;)

10:52 shriphani: tcrayford, I just want to be able to do (first *sequence-of-processed-strings*) and not have to wait 20 mins...

10:54 tcrayford: shriphani: yeah, that's actually a hard problem. Ensuring the file isn't closed with laziness in the picture is somewhat tricky.

10:55 I'm assuming from what you said above that this is a *very* large file, such that you can't just read it into a string?

10:56 (that's a dumb solution, but it can work in some cases)

10:56 shriphani: tcrayford, yes.

10:57 tcrayford: yeah, you're pretty fucked if you wanna use first and have laziness and file closing. In haskell I could point you at like 6 competing libraries that solve this problem, but they don't exist in clojure (and afaik on the jvm)

10:58 mmitchell: leiningen question -- anyone know if it's possible to set the active "profile" with a plugin? I'd like to hook into an existing task (lein mijde), and set the project profile to "test" -- doable?

10:58 tgoossens: If I would like to ask rhickey a question. What medium should I use. (It's not urgent at all (ie: i am not planning to spam). I would just like to ask him some questions about his study (I'm about to choose my masters degree) ) ? If I get no answer, then that's also okay.

11:03 Anderkent: mmitchell: not sure if you can do that with a plugin, but if you're making a custom task you can do it easily - just look at what with-profile does, and do the same thing

11:07 mmitchell: Anderkent: ok thanks, i'll do that

11:12 shriphani: tcrayford, so if you had to implement the unix head command in clojure, you would need to load the entire file ?

11:13 tcrayford: shriphani: I'd be kind tempted to do head completely imperatively, if I was doing that. You don't need to load the entire file, it's just if you want laziness in there and streaming, and good resource finalization, you need something more complex.

11:13 justin_smith: shriphani: I don't think it is that bad, the problem is that if you want to use the file handle to generate a lazy structure, you need to have a strictness boundary before you leave the scope of the file being open

11:14 jtoy: if i am making a basic server that gets a request and spits out json, what server would you use?

11:14 justin_smith: laziness and resource handling can be a tricky combo in general

11:14 jtoy: i want something very small and simple

11:14 justin_smith: simple in its implementation, or simple for you to use?

11:15 jtoy: simple for me to use

11:15 Anderkent: jtoy I think ring + compojure is probably your best bet

11:15 justin_smith: ring is straightforward for that

11:15 yeah

11:15 Anderkent: assuming you're talking http

11:15 jtoy: i dont have to talk http, but that probably is the easiest?

11:15 tcrayford: yeah, I'd say so

11:16 jtoy: the server is only serving one request, so i dont a full web service framework

11:16 justin_smith: OSC is easier, but not everything talks OSC :)

11:16 jtoy: what is OSC? open soud control

11:16 justin_smith: yeah

11:16 sends arbitrary structured data over udp

11:17 very simple

11:17 but it is udp, so it is also lossy

11:17 (potentially - I trust it over a wire but not on wifi)

11:18 Anderkent: So guys - I have a server running somewhere and a repl on it. Some other threads have defined functions there and are using it. I want to modify one of these, so that the modification propagates to all the threads. Do I use alter-var-root? Do I just do (defn function-name ...)?

11:19 shriphani: justin_smith, how about I ditch with-open and open and close the handles myself ?

11:19 I tried doing a .close on the reader and it complained....

11:20 justin_smith: shriphani: I think it would require using java interop

11:20 but then you have some trouble getting a lazy structure out?

11:21 shriphani: one option is to explicitly leave the scope of the with-open

11:21 shriphani: justin_smith, how do I do that (sorry - I'm a noob).

11:22 justin_smith: (with-open ... (for [...] ...)) for has ways of describing an "exit condition", so you have for drive the reading of the data, and tell for you are done and have it return the accumulated data when apropriate

11:23 shriphani: ah.

11:23 but then say I do (nth (get-lines-lazy …) 5).

11:24 justin_smith: you could similarly use take-while

11:24 any construction that goes through a part of a lazy seq then returns

11:25 TimMc: cemerick: Will you be at the Boston Clojure Meetup tonight? If so, would you mind exchanging key signatures?

11:25 cemerick: TimMc: Nope, missing this one.

11:25 shriphani: justin_smith, I meant how versatile would the code be if I had to express an :exit condition.

11:26 justin_smith: shriphani: I think the trick would be to put the function that actually decides how much of the lazy seq you want directly inside the with-open

11:26 shriphani: one sec. if my :exit did a readline and got a nil and I checked and finished on that condition, would I still be able to do first, nth etc. with it ?

11:27 justin_smith: it would return a lazy seq up to the point you said it was done

11:27 unless you tell for not to collect the results somehow

11:27 erm... maybe not lazy? ugh

11:27 shriphani: and I could obviously do first, nth etc and only a partial amount of the file would be read in right ?

11:28 justin_smith: (with-open "file" (doall (for ...))) would be the safe bet, dunno how strict for is off the top of my head

11:28 shriphani: justin_smith, wouldn't doall force eval?

11:28 i.e. wouldn't it cause the entire list to be loaded ?

11:28 I am not opposed to using interop.

11:28 justin_smith: shriphani: yes, the trick is that for is deciding how much to generate

11:30 easier example than for: (with-open [rd (io/reader "file")] (take 10 (line-seq rd)))

11:30 that is unix head

11:30 or a simplified version

11:30 it only ever reads 10 lines

11:31 the trick is that you need to know, before exiting the with-open block, how many lines to read

11:31 shriphani: yeah. I will probably use interop for this.

11:31 justin_smith: but with (for ...) you can decide which line is the last

11:32 notice how with-open uses (io/reader ...) - can't you just directly call io/reader and close the result when done with it?

11:32 or did you say that already failed to work?

11:33 shriphani: justin_smith, I found something on s/o that used a lazy-seq to recursive read lines and called .close at the end.

11:33 justin_smith: cool

11:34 shriphani: but it gave me this: No matching field found: close for class clojure.lang.LazySeq%

11:34 so I will probably need to avoid using lazy-seq

11:34 Anderkent: you must call .close on the input stream, not on the lazy seq that is returned

11:34 shriphani: yeah I figured.

11:34 justin_smith: shriphani: even if you use a lazy-seq to read from, don't you still have the value returned by io/reader?

11:35 I really don't think you need to use java primitives for this

11:35 shriphani: justin_smith, yes.

11:35 Anderkent: aren't java streams closed on garbage collection anyway?

11:36 you could just figure that since the file is private to the lazy seq, it will be closed eventually once the lazy seq is used and discarded. As long as you're only reading, that shouldn't be an issue?

11:36 justin_smith: (def rd (io/reader "project.clj")) (take 1 (line-seq rd)) ...

11:37 (.close rd)

11:37 works!

11:37 shriphani: yeah.

11:37 actually can I wrap this in a macro myself?

11:37 justin_smith: shriphani: those three calls are all you need (plus some looping logic I imagine)

11:37 Anderkent: well if you know how many lines to take you can just do (with-open [rd (io/reader filename)] (doall (take n (line-seq rd)))

11:38 justin_smith: he was looking for something more flexible than with-open

11:38 TimMc: cemerick: Won't somebody think of the children^Wweb of trust!

11:40 justin_smith: shriphani: also, repeated calls to (take n (line-seq rd)) return the next n lines

11:41 Anderkent: any convenient way to read all the content from a inputstream/reader?

11:41 justin_smith: Anderkent: line-seq will try to return the whole thing if you don't use take to restrict it

11:42 Anderkent: as multiple strings though

11:42 i just want the contents of a stream

11:42 justin_smith: slurp

11:42 Anderkent: that's on file not stream?

11:42 justin_smith: slurp works on anything returned by io/resource in my experience

11:42 which is not neccissarily a file

11:43 Anderkent: oh, thanks

11:43 shriphani: justin_smith, I finally have: https://www.refheap.com/paste/14348

11:43 justin_smith: it calls reader on the arg, so you can see clojure.java.io/reader to see what args work with it

11:44 shriphani: there is a recursive call there. would that be a problem (i.e result in stack-overflow if I read a lot of lines?)

11:44 justin_smith: shriphani: well you could refactor to a version that can use recur

11:45 arrdem: /join #libcello

11:45 justin_smith: by adding an accumulator argument to the function, so that call can be a tail call

11:45 shriphani: ok done.

11:45 justin_smith: shriphani: also, (.close rdr) already returns nil, no need for a do there

11:46 also, I think io/resource can get you a stream from a gzipped file, and would be much simpler

11:47 I know I use it to open files inside zips transparently

11:47 (inside file.zip, not clojure zippers)

11:48 so lazy-file-lines always returns all the lines of the file, correct?

11:50 shriphani: justin_smith, yeah

11:50 but it isn't eager.

11:51 justin_smith: then you could do (with-open [rdr (io/reader (io/resource "something.gz")) rdr] (line-seq rdr))

11:52 if you only read 3 elements of the list, it would only read 3 lines of the file

11:52 then the file would be closed

11:52 I thought you were doing something more complex

11:53 err.

11:53 never mind!

11:53 I see the problem now

11:53 hah

11:53 shriphani: justin_smith, did I do something stupid…..

11:54 justin_smith: no, I did!@

11:54 I now realize, laziness makes that not work

11:54 you need to know, inside the with-open, how many lines you need

11:54 shriphani: also, recur expects 0 args here: https://www.refheap.com/paste/14349 …. why is that

11:55 justin_smith: zero args? that is weird

11:55 shriphani: CompilerException java.lang.IllegalArgumentException: Mismatched argument count to recur, expected: 0 args, got: 2, compiling:(reddit_crawler/read_posts.clj:16)...

11:56 justin_smith: I would imagine some weird macro interaction

11:56 where that is actually inside a hidden zero arg function

11:57 ,(macroexpand '(lazy-seq (+ 1 1)))

11:57 clojurebot: (new clojure.lang.LazySeq (fn* [] (+ 1 1)))

11:57 justin_smith: see, lazy-seq creates a thunk

11:57 so recur is not possible there

11:59 shriphani: I am not even sure how to debug some of the things I see….

12:00 justin_smith: that hidden function call thing makes more sense when you start writing macros

12:00 as does the instinct to check macroexpand

12:00 shriphani: so I can't use recur there ?

12:01 justin_smith: it will need to be refactored in order to be able to recur

12:02 shriphani: ugh..

12:02 justin_smith: maybe it could be turned into reduce?

12:02 bbloom: shriphani: you need to realize that recur compiles down to a loop. if you have laziness, you need to break out of evaluation, so recur can't jump back into a loop

12:02 shriphani: you can use explicit recursion, which doesn't have the memory guarentees as recur

12:03 justin_smith: yeah, my suggestion of using recur was off base here, sorry!

12:03 edbond: shriphani, you are trying to get lines from gzipped file?

12:03 shriphani: edbond, yeah...

12:03 bbloom: shriphani: but that's OK, b/c the explicit recursion will be inside a thunk, so the stack won't nest, some calling code walk the lazy seq and evaluate the thunks in a flat manner, so the stack won't blow up

12:04 so in that paste you made, just replace recur with helper

12:04 shriphani: ok.

12:04 I was thinking otherwise to do a (take-while identity (.readLine rdr)).

12:05 bbloom: there is almost always a general function for any major use of lazy-seq you can think of

12:05 shriphani: umm (repeatedly $(.readLine rdr)).

12:06 bbloom: generally, i like to avoid laziness in my IO code....

12:06 shriphani: uh-oh that is not lazy anymore..

12:06 justin_smith: shriphani: how does (repeatedly #(.readLine rdr)) differ from (line-seq rdr)

12:07 shriphani: it doesn't..

12:08 bbloom, If I move the helper call to a non-tail position, it will still be evaluated without a stack-overflow ?

12:10 edbond: shriphani, see https://www.refheap.com/paste/14350

12:10 bbloom: shriphani: without looking at your particular example: laziness involves replacing tail positions with thunks and then letting somebody else execute those thunks. when that happens, the tail position returns a Seq object with the thunk in it and that somebody else is going to execute that thunk in a different stack context. if that thunk then produces ANOTHER thunk, it's not nested in the first one… so there is no stack overflow possibl

12:13 shriphani: edbond, that works….

12:14 justin_smith: that relies on gc to close the fd, correct?

12:14 shriphani: yes I think so.

12:15 irctc092: I have a problem which matches the bin packing problem - http://en.wikipedia.org/wiki/Bin_packing_problem . Can anyone help me translate this into a clojure algo ?

12:15 justin_smith: you could easily modify it to return the handle for further usage or closing

12:17 irctc092: which algo?

12:17 first fit, best fit decreasting, first fit decreasing?

12:17 I would reduce, with an array of objects and an array of bins

12:19 an object being a number (size) a bin being an arry, with (apply + array) telling you its current volume

12:19 well I guess you need hashes, since you also want to know a max capacity

12:19 (apply + (map :capacity bin))

12:19 irctc092: best fit decreasing is what I am looking for

12:20 justin_smith: so, each bin should be something like {:capacity 100 :contents [1 2 3]}

12:20 where contents are the sizes of the objects it has in it

12:21 irctc092: yup and also the objects themselves

12:21 justin_smith: so (apply + (:contents bin)) tells you how much it has in it and (:capacity bin) tells you how much it can hold

12:21 then your objects are an array of numbers

12:22 irctc092: so :contents [ [3 "abc"] ...] , where 3 is the volume

12:22 justin_smith: then you reduce across the array of objects/numbers with the bins as your accumulator

12:22 oh, you could do {:size 3 :id "abc"} for the objects

12:22 yeah

12:24 so (reduce (fn [bins object] ...) [{:capacity 100 :contents []} {:capacity 44 :contents []}] [{:size 6 :name "abc"} {:size 33 :name "hugo"}])

12:24 now you just want to add the object to a bin, and return the new set of bins, in the reducer function

12:25 oh, and best fit decreasing (reverse (sort-by :size objects))

12:26 irctc092: how does sorting them by size help ?

12:26 justin_smith: that is how you do best fit decreasing

12:27 before iterating on them you sort reverse size

12:27 so you find places for biggest first

12:28 (reduce (fn [bins object] ...) [{:capacity 100 :contents []} ...] (reverse (sort-by :size [{size 6 :name "abc"} ...])))

12:28 right?

12:28 clojurebot: flatten |is| rarely the right answer. What if your "base type" is a list

12:30 irctc092: so if I have added a number of items to the bin, and it still has some space left. Now comes another item which does not fit bcoz its vol is greater than that left. However if I manually look at the bin, I can replace one of the items and have an exact fit for my volume, how do I aceieve that?

12:31 justin_smith: sounds like you would need a loop with object and bins as args

12:31 if you find a potential swap, you recur with the current obj swapped out with an element from one of the bins

12:32 the trick being how do you make sure that loop does not keep swapping objs out of bins indefinitely?

12:32 irctc092: yup that is what I am struggling with ?

12:33 I assume this should be an algo / pattern which is established, but cant find one ...

12:33 justin_smith: maybe you only do a swap if obj swapped in is larger than the one you are swapping out? that will stop eventually

12:34 because the obj swapped will be smaller every time

12:34 base case smallest object has no home

12:35 irctc092: didnt understand the base case

12:35 justin_smith: so the danger of infinite recursion is if you just keep swapping because there is no fit findable by the algo

12:35 if you only swap with a smaller object, the smallest object will not recur

12:36 irctc092: yup that should work ... let me code that

12:36 thanks justin

12:36 justin_smith: np!

12:36 more fun than stinky orm web ui crap I should be doing

12:36 lol

12:45 edoloughlin: Is it possible to use type hints to help destructring for multiple function bodies? https://gist.github.com/edoloughlin/5548643

12:46 I get an error: " Unable to resolve symbol: & in this context"

12:47 hiredman: edoloughlin: because you aren't doing multiple function bodies correctly

12:47 edoloughlin: oh.

12:47 hiredman: edoloughlin: keep in mind, destructure is not pattern matching, the only thing that determines which body you get is arity

12:48 edoloughlin: ok. thanks. Will

12:48 hiredman: your function wouldn't work if it had the correct syntax because the last arity can accept the same number of args as the first

12:49 edoloughlin: So, there's no way to do what I want?

12:49 hiredman: a good rule of thumb is "varargs, every function gets one"

12:49 bbloom: edoloughlin: you're trying to accomplish overloading by type?

12:50 edoloughlin: I just want a different body executed if I have a Throwable in there

12:50 hiredman: edoloughlin: in computer programming there is always a way to get what you want, the real question is "is it worth it?"

12:50 edoloughlin: Guess I'll take the path of least resistance. Thanks.

12:50 bbloom: edoloughlin: so clojure can only dispatch on normal functions by arity

12:51 edbond: use (if (instance? <class> <obj))

12:51 bbloom: edoloughlin: there are more complex dispatch strategies, but none that directly match the optional infix argument switched by type

12:51 edoloughlin: so yeah, the simplest thing to do is to have the signature [error-key & args] and then parse args

12:52 edoloughlin: Ok. Thanks. Will do.

12:52 bbloom: (let [[cause & args] (if (exception? (first args)) args (cons nil args)))

12:52 something like that

12:52 edoloughlin: Thanks. Gotta run.

13:06 justin_smith: is there a way to introspect on an fn and see its source? we have a hash of hooks and want to verify the contents at runtime

13:07 bbloom: justin_smith: not generally

13:08 justin_smith: the `source macro can show you code as declared in files, but it won't work for local functions or ones entered at the repl or what not

13:09 justin_smith: but if it was an anonymous fn from a file, would source help there?

13:09 bbloom: ,(source source) ; it's not as scary as it looks :-)

13:09 clojurebot: Source not found\n

13:10 bbloom: heh. well that works in a real repl

13:10 justin_smith: what namespace is source in?

13:10 it is not bound here

13:10 bbloom: clojure.repl

13:10 should be loaded by default by lein

13:10 justin_smith: ok, I did not have clojure.repl loaded

13:11 I am running swank manually from inside a webapp request handler thread

13:11 dabd: how can I translate the following imperative idiom to functional clojure? https://gist.github.com/anonymous/5548875. The first 'for' computes some values of an array but it stops as soon as somePred returns false. But the values of the array are used in the following for cycle. It seems clear I need to use reduce to accumulate the array but how do I make it stop as soon as somePred returns false?

13:11 bbloom: easier to just dig into ##(-> #'clojure.repl/source meta :file)

13:11 lazybot: ⇒ "clojure/repl.clj"

13:11 justin_smith: so clojure.repl is not there automatically

13:12 bbloom: dabd: ##(doc reduced)

13:12 lazybot: java.lang.RuntimeException: Unable to resolve var: reduced in this context

13:12 bbloom: shesh, lazybot is not cooperating with me today

13:12 ,(doc reduced)

13:12 clojurebot: "([x]); Wraps x in a way such that a reduce will terminate with the value x"

13:12 bbloom: that's the most general answer

13:13 but you can map & take-while

13:13 justin_smith: or (take-while ... (reductions ...))

13:13 bbloom: ,(->> (range 10) (map dec) (take-while #(< % 5)))

13:13 clojurebot: (-1 0 1 2 3 ...)

13:14 bbloom: damn print limit… i am sucking at clojure bots today

13:14 ,(->> (range 10) (map dec) (take-while #(< % 2)))

13:14 clojurebot: (-1 0 1)

13:15 bbloom: and then if you want to index into that value, use a vector:

13:15 ,(->> (range 10) (map dec) (take-while #(< % 2)) vec)

13:15 clojurebot: [-1 0 1]

13:15 justin_smith: ,(take-while (partial < -20) (reductions - (range)))

13:15 clojurebot: (0 -1 -3 -6 -10 ...)

13:15 justin_smith: that would have ended at -15

13:15 bbloom: now you can index into it: ##([-1 0 1] 0)

13:15 lazybot: ⇒ -1

13:15 bbloom: now you can index into it: ##([-1 0 1] 2)

13:15 lazybot: ⇒ 1

13:16 dabd: ok thanks for the ideas, i can't find how to use reduced

13:16 out of curiosity it seems interesting

13:16 bbloom: dabd: reduced is new in 1.5

13:16 dabd: forget i mentioned it :-P

13:18 justin_smith: you can use reductions exactly the way you would reduce (it just returns a lazy seq rather than the final result, last element of that seq is the result)

13:22 edbond: dabd, take a look at http://stackoverflow.com/questions/15625341/reduce-a-lazy-sequence-like-a-loop-with-a-condition-in-clojure for reduced example

13:23 dabd: edbond: thanks!

13:24 irctc260: (def items (take 20 (repeat {:desc "abc" :weight (int (rand 100))})))

13:25 dabd: reduced seems the simplest solution and apparently it is faster in 1.5 too

13:25 irctc260: I am trying to create a list of 20 diff items, however the above returns a list of 20 same items

13:26 (def items (take 20 (repeat {:desc "abc" :weight (int (rand 100))})))

13:26 justin_smith: that only calculates the rand once

13:26 you want repeatedly with a thunk

13:27 (def items (take 20 (repeatedly (fn [] {:desc "abc" :weight (int (rand 100))}))))

13:28 gfredericks: "the heap of flour fell to the ground repeatedly with a thunk"

13:28 justin_smith: heh

13:29 irctc260: yup thanks

13:31 justin_smith: why doesnt this work ? (def items (take 20 (repeatedly #({:desc "abc" :weight (int (rand 100))}))))

13:31 gfredericks: you're calling the map

13:31 justin_smith: that tries to call the map as an fn

13:31 gfredericks: the #() syntax is bad for returning a literal

13:31 justin_smith: there is #(do ...)

13:32 edbond: #(hash-map :a 4)

13:32 justin_smith: but (fn [] is only one extra char

13:32 that is even more extra chars!

13:32 gfredericks: there's like 4 ways to do it, and (fn []) is probably the one that takes the least thought and indicates intent the clearest

13:32 edbond: :)

13:33 irctc260: yup that makes sense...

13:33 dabd: anyone knows why nrepl-quit won't kill the java processes on win?

13:42 sritchie: do you guys have any suggestions for a directed graph library in clojure?

13:42 bacwn and contrib.datalog both have the same graph.clj file,

13:42 but I'm hesitant to lift the thing out...

13:44 bbloom: sritchie: depends on what you want to do with it

13:44 sritchie: bbloom: I'm rewriting the cascalog datalog implementation

13:45 so, I want to compile a set of datalog rules down into a directed graph of computations that I'll feed into MapReduce

13:45 into Cascading, specifically

13:45 bbloom: sritchie: without knowing anything about cascalog's datalog impl….. do you need a large library of graph functions? or are there a particular small number of algorithms you need?

13:45 considering how easy it is to represent a graph with a clojure data, it seems unwise to bring in a large/complex library of representations

13:46 amalloy: bbloom: it's not *that* easy to represent a graph in clojure, if you want to have any cycles

13:47 arrdem: amalloy: use node ids and that's easy

13:47 bbloom: amalloy: it's pretty easy to represent any graph as a spanning tree if you assign names/identities to nodes

13:47 sritchie: I don't need much, just a few functions for getting inbound edges, outbound edges, etc -- it's easy to code, but if a small library existed I wanted to avoid the duplication

13:47 amalloy: i know. but that's another layer of indirection you'd rather the graph library deal with

13:48 arrdem: sritchie: there's loom on github, I can vouch for the fact that it works and is pretty usable

13:48 sritchie: here's the bacwn graph file, lifted from the old contrib datalog: https://github.com/fogus/bacwn/blob/master/src/clojure/fogus/datalog/bacwn/impl/graph.clj

13:48 amalloy: wait, how do spanning trees enter into it, bbloom?

13:49 sritchie: arrdem: great, good tip

13:49 irctc260: I want to sort these - (def items (take 20 (repeatedly (fn [] {:desc "abc" :weight (int (rand 100))})))). (sort :weight items) doesnt work. How do I define a java.util.comparator ?

13:49 bbloom: amalloy: you flatten your graph into a tree… that tree touches all the vertexes of the graph

13:49 arrdem: sritchie: used it to build a type hierarchy so I only really touched the directed graph parts of it, not sure how well the weighted graph bits work tho

13:49 justin_smith: irctc260: you want sort-by, not sort

13:50 sort-by takes :weight from each of the two, and does a standard comparison

13:50 sort, as you notice, needs a true comparitor (taking two objs, returning -1 0 or 1)

13:50 irctc260: thanks

13:50 amalloy: bbloom: i know what a spanning tree is. turning a graph into a spanning tree either loses a lot of information, or makes it very hard to work with, if you want to do general graph stuff to it

13:52 bbloom: amalloy: ok maybe spanning tree isn't precisely the right thing to call it when you flatten all the vertices into a list

13:52 amalloy: you got a better name?

13:53 arrdem: bbloom: how are you representing edges in this structure?

13:54 amalloy: arrdem: the same way you suggested, he's just not very good at verbalizing it :)

13:54 bbloom: arrdem: depends on if you need to to associate data with your edges. depends on if your edges are directed. depends on if they can be traversed in both directions

13:54 arrdem: amalloy: ah. yeah that's how loom does it, but it makes sense for clojure especially

13:55 bbloom: i'm sorta anti-graph-library b/c there are too many representational considerations that by the time you've picked all the choices, you might as well have written the 5 functions you needed for your use case

13:56 and i don't think that it's a good idea to always go with the most general version; something like a graph database with propertied nodes & edges

13:59 hugod: bbloom: what are your plans regarding a fipp release? I used it here https://github.com/pallet/robot-crab/blob/master/src/robot/crab/print.clj and was thinking about how to get my project released.

13:59 bbloom: hugod: you just need a non snapshot release?

14:00 hugod: bbloom: right

14:00 I only saw snapshots on clojars

14:00 bbloom: what version are you using now?

14:00 if you say it works, i'll consider that good enough and release a version :-)

14:01 hugod: hey! "0.3.0-SNAPSHOT" from a few of days ago

14:01 justin_smith: I have a hobby project to take declarative data structure describing a dag and translating that to a running synthesizer via csound

14:01 I should look into one of those proper graph libs

14:02 nodes = modules, edges = patch cables

14:02 bbloom: hugod: i'll make a 0.3.0 release

14:03 hugod: bbloom: thanks! no hurry

14:03 bbloom: oh now i remember why i didnt

14:03 apparently to make a non snapshot release, you need to set up gpg and all that jazz

14:03 :-/

14:04 hugod: I think signing is still optional

14:04 bbloom: is there a way to bypass it it quickly?

14:04 arrdem: bbloom: no

14:04 technomancy: yes

14:04 justin_smith: gpg is easy to set up

14:04 technomancy: but yeah, if you're going to be releasing software you really should learn this

14:05 bbloom: technomancy: *sigh* fiiiiiiinnneeeee

14:05 hugod: you'll have to wait until i figure that out

14:08 hugod: bbloom: np

14:09 justin_smith: gpg --gen-key

14:09 it is interactive

14:10 technomancy: `lein help gpg` too

14:11 bbloom: thanks guys. i've got phone meetings until 4pm… if i have the energy i'll try it then :-)

14:17 irctc260: justin_smith : how do I write the fn to decide which item to swap from the bin ? Given a bin {:capacity 100 :items [ {:w 20} {:w 30} {:w 40}] } and an item {:w 50} f(bin, item) => {:w 40}

14:26 dabd: after trying to use compare on a lazyseq i got an error ClassCastException clojure.lang.LazySeq cannot be cast to java.lang.Comparable clojure.lang.Util.compare (Util.java:153)

14:26 How do i convert a lazy seq to a seq that can be used with compare?

14:26 edbond: irctc260, isn't this a knapsack problem? http://rosettacode.org/wiki/Knapsack_problem/0-1#Clojure maybe you can get some ideas there.

14:27 dabd, how do you want to compare sequences?

14:28 dabd: how to compare lexicographically two sequences (compare [1 2] [1 3])

14:28 I want to compare*

14:28 the problem is that the sequences i am giving to compare are lazyseqs and it complains

14:29 a simple solution is to use (vec s) but isn't this O(n)?

14:30 amalloy: dabd: overly simple answer, but...write a function to do it. that's really all there is. walk the two sequences, and stop when you see a difference

14:31 dabd: so using (vec s) is inefficient?

14:31 (compare (vec s1) (vec s2))

14:35 Pupnik-: (compare (count s1) (count s2)) should give the same behavior as that

14:35 which saves you from allocating new vecs probably, im no clojure expert

14:35 ,(compare [1 2 3] [5 6])

14:35 clojurebot: 1

14:36 Pupnik-: ,(compare (count [1 2 3]) (count [5 6]))

14:36 clojurebot: 1

14:37 matthavener: comparing the counts?

14:37 dabd: i want to compare the sequences lexicographically, comparing the counts doesn't make sense

14:37 matthavener: if they're the same length you can do something like (every? #(apply = %) (map vector list1 list2))

14:37 Pupnik-: in that case you cant use the first one either

14:37 because thats what compare does with vecs

14:37 SegFaultAX: Pupnik-: You didn't read what he asked for.

14:38 matthavener: compare with vecs just compares the length?

14:38 oh, you want the lexographic -1, 0, 1 output?

14:39 SegFaultAX: matthavener: No, I don't think it does.

14:40 justin_smith: (any > v1 v2) ?

14:40 amalloy: it *starts* by comparing the length

14:40 justin_smith: I mean some

14:40 Pupnik-: (map compare s1 s2) ?

14:40 ,(map compare [1 2 3] [2 1 9])

14:40 clojurebot: (-1 1 -1)

14:41 amalloy: ,(map compare [1 1 1] [1 1 1 2])

14:41 clojurebot: (0 0 0)

14:41 SegFaultAX: The problem isn't comparing vecs or lists, the problem is he's trying to comapre a partially realized lazy seq.

14:42 justin_smith: (some #{1} (map compare v1 v2)) will stop comparing as soon as one element in v1 is greater than the corresponding in v2

14:43 SegFaultAX: Vectors already short circuit during comparison.

14:44 matthavener: yeah if you know they're the same length (or will eventually be unequal), you could do (first (drop-while #{0} (map compare l1 l2)))

14:44 justin_smith: actually make that some #{-1 1} - then it returns the comparator, and short circuits

14:44 dabd: what is the meaning of #{1}?

14:44 matthavener: justin_smith: or that, yeah

14:44 justin_smith: returns nil if input is not 1

14:44 matthavener: dabd: its a set that contains one element, 1

14:44 SegFaultAX: dabd: A set with a single element 1

14:44 justin_smith: #{-1 1} returns nil if input is not in the set, otherwise that element

14:45 dabd: ok

14:45 a set is also a function

14:45 justin_smith: yes

14:45 SegFaultAX: dabd: As are hashs, vectors, keywords, etc.

14:47 dabd: but it won't work in the case ,(some #{-1 1} (map compare [1 2] [1 2]))

14:47 ,(some #{-1 1} (map compare [1 2] [1 2]))

14:47 clojurebot: nil

14:47 SegFaultAX: dabd: Use ## for in-line forms

14:47 dabd: ty

14:47 SegFaultAX: Just coalesce the value to 0

14:47 clojurebot: Gabh mo leithscéal?

14:48 dabd: what do you mean?

14:48 SegFaultAX: ,(or (some #{-1 1} (map compare [1 2] [1 2])) 0)

14:48 clojurebot: 0

14:48 justin_smith: (or ,(some #{-1 1} (map compare [1 2] [1 2])) 0)

14:48 oops!

14:48 he beat me to it anyway

14:48 amalloy: fwiw, guys, IMO (-> (some #{-1 1} (map compare [1 2] [1 2])) (or 0)) is a bit easier on the eyes, since it keeps the 0 together with the or

14:49 justin_smith: amalloy: excellent point

14:49 amalloy: <3 ->

14:50 dabd: i need to start learning how to use the threading macros

14:50 SegFaultAX: (fn lazy-compare [& s] (-> (apply map compare s) (or 0)))

14:50 justin_smith: ,(->> (map compare [1 2] [1 2]) (some #{-1 1}) (or 0)) ; is even better imho

14:50 clojurebot: 0

14:50 SegFaultAX: Probably could use a when in there to make sure they're all seqs or something.

14:50 amalloy: justin_smith: aside from not working at all, it's fine

14:51 justin_smith: oh, oops

14:51 amalloy: that will return 0 on any arguments at all, even non-collections

14:51 justin_smith: the or 0

14:51 SegFaultAX: Oh whoops

14:51 justin_smith: yeah

14:51 SegFaultAX: Pasted the wrong code

14:51 justin_smith: defining lazy-compare is better anyway

14:52 amalloy: also, so far nobody has proposed a solution that handles collections of different lengths

14:52 SegFaultAX: (fn lazy-compare [& s] (-> (some #{-1 1} (apply map compare s)) (or 0)))

14:53 amalloy: This wouldn't be an issue at all if we already knew the length.

14:53 Since vectors already short circuit during comparison.

14:53 (And use count)

14:53 amalloy: I think the OP was asking about comparing a lazy seq.

14:54 amalloy: he was...so you have to handle lengths properly. i don't see what you're getting at

14:55 he wants a function that acts like strcmp for sequences. you can't just ignore lengths

14:56 justin_smith: amalloy: does that mean forgoing map for an explicit loop?

14:56 SegFaultAX: amalloy: I don't think he wanted to realize the entire seq up front.

14:57 Otherwise, I agree with you.

14:57 dabd: this issue came when trying to translate python code to clojure, since python < operator works on sequences

14:57 SegFaultAX: Oh I see, nevermind. You're 100% correct

14:58 dabd: if one sequence is shorter it compares only up to the shorter sequence

14:58 Pupnik-: what if you convert the sequences to string then use compare?

14:58 SegFaultAX: Pupnik-: Wut.

14:58 slagyr: I'm seeing some weird behavior with Var binding in Clojure 1.5.1:

14:58 speclj.config/*parent-description*: #<Unbound Unbound: #'speclj.config/*parent-description*>

14:58 (bound? #'speclj.config/*parent-description*): true

14:58 (thread-bound? #'speclj.config/*parent-description*): true

14:58 (.getThreadBinding #'speclj.config/*parent-description*): #<TBox clojure.lang.Var$TBox@769a90b9>

14:59 SegFaultAX: ~refheap

14:59 clojurebot: refheap is forget ~paste

14:59 SegFaultAX: ~paste

14:59 clojurebot: paste is not gist.github.com

14:59 SegFaultAX: slagyr: Please refheap multi-line pastes.

14:59 slagyr: Anyone got a clue why the value of the var is Unbound, yet bound? returns true?

14:59 dabd: actually i am wrong is one sequence is shorter it does not evaluate up to the shorter sequence

14:59 if one*

15:00 ppppaul: ,(doc ->>O)

15:00 hiredman: sure

15:00 ppppaul: ,(doc ->>)

15:00 clojurebot: It's greek to me.

15:00 "([x form] [x form & more]); Threads the expr through the forms. Inserts x as the last item in the first form, making a list of it if it is not a list already. If there are more forms, inserts the first form as the last item in second form, etc."

15:00 ppppaul: ,(doc :>>)

15:00 clojurebot: #<ClassCastException java.lang.ClassCastException: clojure.lang.Keyword cannot be cast to clojure.lang.Symbol>

15:00 hiredman: (declare x) (def *parent-description* x)

15:00 slagyr: SegFailtAX: Sorry. https://www.refheap.com/paste/14358

15:00 ppppaul: can someone explain this example to me? http://clojuredocs.org/clojure_core/clojure.core/condp#example_44

15:00 wtf is :>>

15:00 i tried it in my repl and it works

15:01 SegFaultAX: ppppaul: A man with 2 beards.

15:01 amalloy: slagyr: one way you could get into that scenario is (def ^:dynamic foo) (binding [foo foo] ...)

15:01 justin_smith: ppppaul: a magic keyword in condp

15:01 amalloy: ppppaul: (doc condp)

15:01 ppppaul: SegFaultAX, its

15:01 magic keywords

15:01 magic

15:01 magnets

15:02 i guess i should have looked at the source first

15:08 Pupnik-: ,(compare (str [1]) (str [1 1]))

15:08 clojurebot: 61

15:08 Raynes: @fogus on twitter: "OOP does model the real world! Things at the top of the hierarchy seemingly do nothing but tell those at the bottom what do do."

15:09 Fogus gold!

15:16 mpenet: nice

15:25 gfredericks: is there anything smelly about creating a background query function with java.jdbc that returns a lazy-seq that is eagerly realized in a background thread as the resultset is read?

15:26 amalloy: gfredericks: i guess the only smelly thing is that you're not just using seque

15:27 gfredericks: that thing I haven't remembered about for a year at least?

15:27 amalloy: i'm not actually sure if it would be easy to apply here

15:27 but it sounds like the right ballpark

15:28 gfredericks: the docs aren't clear about what the 1-arg version does

15:28 how does "up to n items" apply if there is no n?

15:28 amalloy: &(doc seque)

15:28 lazybot: ⇒ "([s] [n-or-q s]); Creates a queued seq on another (presumably lazy) seq s. The queued seq will produce a concrete seq in the background, and can get up to n items ahead of the consumer. n-or-q can be an integer n buffer size, or an instance of java.util.concur... https://www.refheap.com/paste/14361

15:28 gfredericks: also I'm not sure how well that would fit with the "I have to close the resultset" paradigm

15:30 SegFaultAX: amalloy: I didn't even know that was a thing. Thanks!

15:30 amalloy: SegFaultAX: i was recently accused of breaking it, so it's fresh in my mind :P

15:31 gfredericks: the background is single-threaded, eh?

15:31 SegFaultAX: Will the seque always try and keep the buffer filled?

15:32 Like if I give it 10 then consume 5 items, will it produce another 5 in the background?

15:38 dabd: could someone help me translate the following python function to clojure? https://gist.github.com/anonymous/5549946 I'm having a hard time with the recursion since there is a 'del' instruction after the recursive call.

15:39 SegFaultAX: dabd: This is only called for its side effects.

15:40 dabd: yes i tried to emulate the for with a map over to lists (range start 52) (iterate butlast cards) but it is not working

15:40 two lists*

15:40 SegFaultAX: The issue here isn't really recursion. It's that you're mutating some global state.

15:41 Also, you're mutating a list that you're iterating over which is universally considered an awful idea.

15:41 In Python or otherwise.

15:42 Oh I take that back, you're just pushing it on and popping it off repeatedly.

15:43 edbond: dabd, is't this just a combinations? http://clojure.github.io/math.combinatorics/#clojure.math.combinatorics/combinations

15:43 amalloy: SegFaultAX: yes, it does try to keep the buffer filled

15:44 dabd: edbond the whole code is here: http://stackoverflow.com/questions/3829457/generating-all-5-card-poker-hands

15:44 SegFaultAX: amalloy: Hmm, maybe I don't understand. It appears to return the rest of the seq if the blocking queue will block.

15:45 amalloy: SegFaultAX: fill returns the seq, yes, but that's just what goes into the agent

15:45 dabd: edbond: it's the code from the second answer. It is generating all isomorphic combinations of 5 card poker hands

15:46 amalloy: then next time you try to realize an element, you call drain, which starts the agent up again on the leftover seq, and it .offer()s again

15:46 SegFaultAX: Ah, cool!

15:47 slagyr: amalloy: Thanks for the help. The var was indeed being bound to itself.

15:47 edbond: dabd, implement canonical and filter (filter canonical (combinations cards 5))

15:48 * gfredericks binds a var to itself just to see what happens

15:48 Bronsa: heh

15:48 then you can do @@@@@@@@@var

15:48 amalloy: &(binding [*out* #'*out*] @@@@@@@@@*out*)

15:48 lazybot: java.lang.SecurityException: You tripped the alarm! pop-thread-bindings is bad!

15:49 gfredericks: Bronsa: that was the conclusion I was just coming to

15:49 amalloy: well, whatever. also, if you bind it to itself while it's unbound, you get the funny behavior slagyr asked about

15:49 dabd: i managed to translate the function but it is awfully slow and ugly. Any suggestions for improvement? https://gist.github.com/anonymous/5550022

15:50 edbond: dabd, I'm not good in cards ;)

15:51 dabd: edbond filter canonical combination is the way to go! the power of lazy evaluation

15:53 edbond: dabd, I would try to translate rules from comments and skip python version.

15:53 I mean doesn't translate word to word

15:53 dabd: it's done already. The key is the canonical? function. https://gist.github.com/anonymous/5550057

15:54 SegFaultAX: I stole your lazy-compare :-)

15:54 justin_smith: ,(def cards (set (for [suit [:hearts :clubs :spades :diamonds] face (concat [:ace :jack :queen :king] (map inc (range 10)))] [suit face])))

15:54 clojurebot: #<Exception java.lang.Exception: SANBOX DENIED>

15:54 justin_smith: (take 5 (shuffle cards))

15:54 done

15:55 err, it should not be a set, but...

15:55 dabd: justin_smith it's not that simple read the link to understand why some poker hands are isomorphic

15:57 Pupnik-: which part is making it slow dabd ?

15:58 dabd: the last version i posted seems to be ok but i'm sure canonical? can be improved

15:59 justin_smith: dabd: sorry, I should have actually looked at the link

15:59 the shuffle pun was too good to resist though

16:12 dabd: is there any way to make this faster? https://gist.github.com/anonymous/5550057 The python version is so much faster

16:22 justin_smith: dabd: you could use bit opts like they do, bit twiddling can be ugly but it is fast

16:23 dabd: the correct solution is the answer no2 and there is no bit twiddling

16:25 i think the problem is that the python solution generates only the isomorphs while clojure generates all combinations and filters isomorphs

16:26 this is lazy (filter canonical? (c/combinations (range 52) n)) but it still has to test all combinations right?

16:27 gazbox: That's better :)

16:27 dnolen: dabd: what n are you passing to generate isomorphs?

16:27 dabd: yes

16:27 dabd: the number of cards

16:28 dnolen: dabd: but what specific n?

16:28 dabd: if i want to generate 5 card poker hands i pass 5, if i want 2 cards i pass 2 :-)

16:28 gazbox: Hi, I'm a new clojure programmer and I have some trouble understanding something

16:28 I have a test case on pastebin

16:28 I'll paste it :D

16:28 http://pastebin.com/SZ8NAar6

16:28 dnolen: dabd: you have to test 2,598,960 possibilities for n=5

16:28 dabd: dnolen: yes that is the problem

16:29 gazbox: I'm trying to use iterate to produce a function that's called recursively to n deep

16:29 justin_smith: but the ranking follows rules, so you could generate the valid rankings from the input set without iterating every possibility

16:29 gazbox: Not call the function but create an instance of it from an infinite set

16:30 sequence I mean

16:30 dabd: generating only isomorphs is a bit more complicated i've read a bit about it

16:30 AimHere: gazbox > every time you call 'op', it just calls itself indefinitely

16:31 gazbox: doesn't op return an lambda?

16:31 AimHere: erm, oops, maybe so

16:31 dnolen: dabd: if you have the python code it shouldn't be hard to translate

16:31 gazbox: :D

16:31 the code runs in the repl

16:32 just does't do what I expect it to

16:32 justin_smith: gazbox: just a pedantic point, there is no such thing as a "pointer to a function" in clojure, it is just a function

16:32 gfredericks: except in the sense that near everything on the jvm is a pointer

16:32 gazbox: justin_smith: I thought someone would say that :D I'm a c++ coder originally :D

16:32 I'm getting over it though ;)

16:32 mthvedt: a lot of people use pointer and reference interchangeably talking about the jvm

16:33 justin_smith: but is #() a reference?

16:33 it is a clojure function

16:33 gazbox: yeah true, the function would be constructed

16:34 if that's the right terminology :)

16:34 gdev: can't core.logic-koans to run in windows; i think the bat file isn't concatenating the classpath correctly and its looking in .\lib for jars despite there not beng a lib folder

16:34 *can't get

16:34 gazbox: but yeah the code I posted doesn't ever call fun

16:35 and I thought the (m) bit would

16:35 call op five times and then call fun

16:36 gdev: it was saying it couldn't find clojure.main so I hardcoded the classpath in the bat file and now it is saying it can't find the koan engine

16:36 justin_smith: gazbox: there is no situation where op would call its arg as you have defined it

16:37 gazbox: oh it should return the function and call it first>

16:37 justin_smith: all it does is pass the arg - it never calls or returns it

16:37 callen: justin_smith: are you from Ohio?

16:37 justin_smith: no

16:37 callen: good.

16:37 justin_smith: why?

16:37 clojurebot: why is the ram gone is <reply>I blame UTF-16. http://www.tumblr.com/tagged/but-why-is-the-ram-gone

16:38 callen: justin_smith: I knew an awful person by your name from Ohio.

16:38 justin_smith: there are lots of us

16:38 lol

16:38 * callen shrug

16:38 justin_smith: (awful justin smiths from various places)

16:39 gazbox: got it working :D

16:39 Cheers terrible people fron ohio :D

16:40 I'm really enjoying clojure, managed to get a nice setup where I live code in emacs. It's amazing

16:42 justin_smith: emacs is a process of continuous discovery

16:42 today I discoverd C-u M-x vc-diff

16:42 shows diff of current file in two arbitrary branches

16:42 (neither has to be current)

16:42 gazbox: I've been in Sublime for ages and had to take the plunge so I could use nrepl

16:43 learning curve like a cliff but it's pretty cool

16:43 callen: vc-annotate is my preferred prelude to murder.

16:43 justin_smith: so, gazbox was your goal to make a function that called itself recursively 5 times and then evaluated its argument?

16:44 gazbox: yeah, the op thing will be calling stuff to set the OGL matricies

16:45 so I can do some recursive geometry stuff

16:47 muhoo: i find it frightening not only that i wrote this, but that it worked on the second try, and that i understand it: https://www.refheap.com/paste/14366 is that too obtuse? am i getting too golf-y?

16:49 justin_smith: golfier- switch out sort #(compare ... with (sort-by :date

16:49 well, shorter but actually more readable too

16:50 muhoo: not too golfy, dense but not obfuscated imho

16:51 muhoo: oh, i'd forgotten about sort-by, thanks

16:52 gazbox: The only thing I find hard about clojure so far is there's not a lot of filler

16:53 every line counts

16:53 you have to concentrate a bit more :)

16:53 gfredericks: type less think more

16:53 gazbox: yup

16:53 gfredericks: good for your fingers

16:53 gazbox: well

16:53 I do a lot of deleting :D

16:53 gdev: i tap my fingers when i think so it's a zero-sum game for them

16:57 bbloom: gazbox: that's why people often think lisp is hard to read

16:57 i argue that it's actually EASIER to read in that you gain more knowledge per unit of time & require less total time for more total knowledge

16:57 justin_smith: well that is half of it, also you have to learn how not to read parens

16:57 bbloom: but it means you read much slower in terms of lines per unit of time

16:58 justin_smith: information theory tells us there is some optimal measure of redundency for optimum communication, the parens provide the redundancy :)

16:58 Glenjamin: hrm, thats an interesting idea

16:58 shannon entropy to measure language expressiveness

16:59 gfredericks: just gzip it and see how well it does :)

16:59 justin_smith: yup! as a language gets more powerful, it gets harder to tell a bug from an interesting but obscure construct

16:59 bbloom: & hence you need more documentation… and no javadoc style per function comments do not count

16:59 lazybot: java.lang.RuntimeException: Unable to resolve symbol: hence in this context

17:00 rbxbx: ha

17:00 gfredericks: &(hence soforth wherewithal)

17:00 lazybot: java.lang.RuntimeException: Unable to resolve symbol: hence in this context

17:03 gfredericks: I'm having issues with my tests hanging

17:03 there's a thread-pool thing I'm doing, and the tests only hang if I do it at least three times or something bizarre like that

17:04 testing any single namespace finishes successfully o_O

17:04 justin_smith: my solution to race conditions is get a healthy head start so I win

17:04 :P

17:04 hiredman: gfredericks: have you tried testing all the namespaces in random orders

17:05 gfredericks: hiredman: not sure how to do that with clojure.test?

17:05 hiredman: `lein test` then a shuffled list of test namespaces

17:05 gfredericks: ah very good

17:05 I'm on it

17:07 hiredman: clojure.test can also behave very oddly if if you do something bad with fixtures, I am trying to remember what, I forget if the symptoms are anything like tests hanging though

17:07 gfredericks: definitely using fixtures

17:08 justin_smith: example of entropic terseness in clojure code: a controller helper in a webapp that extracts the id of the current user's permission role: (defn role-id [{{{role-id :role-id} :user} :session}] role-id)

17:08 four of the size names in that definition are the same

17:08 *six

17:09 gfredericks: hiredman: still hangs on first random run

17:10 the idea is to check if the hangin is caused by running A before B?

17:10 hiredman: yeah

17:10 gfredericks: I guess I should reverse it then rather than doing more randoms

17:10 hiredman: maybe you shutdown a threadpool and don't restart it or something

17:11 gfredericks: hmmm...that one hung early o_O

17:13 rodnaph_: hey - i'm thinking of using core.cache in my application - does anyone have experience, and can give some pros/cons? cheers

17:13 hiredman: gfredericks: the with fixtures is due to the way they compose, if you swallow exceptions in a fixture thinking "oh, I never care about exceptions here" you can endup swallowing excetions from fixtures you care about

17:13 maybe not relevant here

17:14 the thing with

17:15 justin_smith: code that unexpectedly catches my exceptions bedevils me

17:15 hiredman: yes, it can be a bit of a bugger

17:17 Glenjamin: justin_smith: isn't that example an example of too much entropy in the code?

17:17 justin_smith: yeah I think so!

17:17 Glenjamin: (defn role-id [req] (get-in req [:session :user :role-id]))

17:17 justin_smith: I would not have called it entropic if it was ideal

17:17 gfredericks: hiredman: okay, I don't think I expect any exceptions in my fixtures

17:17 justin_smith: yeah, I like destructuring, but I overdo it

17:18 Glenjamin: ,((defn role-id [req] (get-in req [:session :user :role-id])) {:session {:user {:role-id 1}}})

17:18 clojurebot: #<Exception java.lang.Exception: SANBOX DENIED>

17:18 Glenjamin: ,((fn [req] (get-in req [:session :user :role-id])) {:session {:user {:role-id 1}}})

17:18 clojurebot: 1

17:18 justin_smith: yeah

17:18 gfredericks: ,(assoc-in {} [:session :user :role-id] 1)

17:18 clojurebot: {:session {:user {:role-id 1}}}

17:18 justin_smith: what I like about destructing is that you can read it backwards to see the structure the funciton needs

17:19 gfredericks: ,(assoc-in {} (repeat 10 :x) :y)

17:19 clojurebot: {:x {:x {:x {:x {:x {:x {:x {:x {:x {:x :y}}}}}}}}}}

17:19 Glenjamin: heh

17:21 ,((fn role-id [{{{role-id :role-id} :user} :session}] role-id) nil)

17:21 clojurebot: nil

17:25 mabes: it doesn't look like nrepl.el/ritz has a trace function helper like the swank version had.. is that right, or have I missed it?

17:37 Glenjamin: bah, why does get-in work on vectors, but not lists

17:37 gfredericks: vectors are peculiarly map-like. lists are not at all.

17:37 get-in is for mappy things, same as get

17:37 Glenjamin: mm, i guess i was cheating a bit

17:38 had a seq with one deep map

17:38 was trying to do get-in [0 :key :key2]

17:38 instead of first (get-in [:key :key2]

17:38 gfredericks: you could also (-> % first :key :key2)

17:39 Glenjamin: mm, but that's not null-safe

17:39 and this lib has back-compat to 1.3, so i can't use some->

17:39 gfredericks: not null-safe?

17:39 ,(-> nil first :key :key2)

17:39 clojurebot: nil

17:39 Glenjamin: oh right

17:39 gfredericks: or your keys are nil?

17:39 Glenjamin: i dunno why i thought that

17:40 much better, thanks

17:52 shriphani: hi. Is there a way to pipe output to less (or something like that) in a repl? pprinting leads to something that can't fit in 1 screen-buffer.

17:57 rlb: Should (set-stroke chart :width 5) affect a bar chart in incanter when going to pdf?

17:59 vijaykiran: shriphani: not aware of anything like that - may be you should write the pprint to a tempfile?

18:00 amalloy: or use something smarter than a command-line repl, like an emacs nrepl connection

18:02 n_b: shriphani: Not from the default clj REPL; either spit it out and view it in your editor or switch to emacs/vimclojure/what-have-you

18:03 shriphani: amalloy, the repl is running on a remote machine and I am using sshfs.

18:04 vijaykiran: tramp using emacs

18:24 Glenjamin: can you increase the screen buffer?

18:29 justin_smith: shriphani: another option is the script command, makes everything you do in the shell also print out to a file

18:29 script is a shell command you would run before opening the repl

18:32 gfredericks: I have three namespaces such that `lein test A B C` hangs but removing any of A, B, or C completes

18:32 Raynes: I tend to break leiningen.test.

18:33 If I broke this it must have been broken for quite a long time.

18:34 _{^_^}_: when do you guys decide to use a n-tuple to represent an entity vs a hashmap? e.g. {:name "tyler" :age 29} vs ["tyler" 29]

18:35 gfredericks: only when haxing

18:42 Apage43: _{^_^}_: I tend to only do 2-tuples

18:42 once I get to 3 I start forgetting which element is which too often

18:42 and i have to debug less if I just give them names

18:43 I get them mixed up often enough with 2-tuples, but if I'm just playing around and trying to bang something out quick i do it to save typing

18:44 but you can use https://github.com/Prismatic/plumbing and nearly get the best of both worlds

18:47 omaciel: given the following sequence ("Blue" "Yellow" "Red") how do I check if a "Blue" is in this sequence?

18:48 amalloy: ~contains

18:48 clojurebot: contains is gotcha

18:48 amalloy: well, true enough. don't use contains

18:48 justin_smith: ,(some #{"Blue"} ["Blue" "Yellow" "Red"])

18:48 clojurebot: "Blue"

18:48 omaciel: hmmmm

18:48 hiredman: amalloy: you can use the java method just fine

18:49 the real question is, shouldn't you be using a set instead?

18:49 omaciel: is it possible to get back a boolean?

18:49 wait

18:49 I don't have a vector

18:49 justin_smith: ,(some #{"Blue"} '("Blue" "Yellow" "Red"))

18:49 clojurebot: "Blue"

18:50 hiredman: omaciel: why don't you keep the colors in a set instead?

18:50 omaciel: ,(some #{"Black"} '("Blue" "Yellow" "Red"))

18:50 clojurebot: nil

18:50 omaciel: ,(boolean (some #{"Black"} '("Blue" "Yellow" "Red")))

18:50 clojurebot: false

18:50 hiredman: sets are much better for membership testing

18:50 omaciel: ,(boolean (some #{"Blue"} '("Blue" "Yellow" "Red")))

18:50 clojurebot: true

18:51 omaciel: hiredman: hmmm I may change the function that is returning that seq then

18:51 justin_smith: ,(boolean ((set '("Blue" "Yellow" "Red")) "Blue"))

18:51 clojurebot: true

18:51 hiredman: :(

18:52 patchwork: Best practice: iterate over a string or split it with #"" ?

18:52 hiredman: don't build a set out of a list just for that

18:52 patchwork: (to do something to each char in the string)

18:52 justin_smith: hiredman: yeah, good point

18:52 omaciel: hiredman: what do you recommend?

18:53 justin_smith: I was just showing the list could easily be a set

18:53 if the list is just generated once, then the set could be generated once and reused

18:53 patchwork: ,(clojure.string/split "hello world" #"")

18:53 clojurebot: ["" "h" "e" "l" "l" ...]

18:54 patchwork: Seems wasteful

18:54 justin_smith: ,(map identity "Hello World")

18:54 clojurebot: (\H \e \l \l \o ...)

18:54 patchwork: Yeah but those are chars, not strings

18:54 ,(map str "hello world")

18:54 clojurebot: ("h" "e" "l" "l" "o" ...)

18:55 patchwork: That may be best

18:55 justin_smith: as long as you know you can't do it with chars, sure

18:55 patchwork: The algorithm expects strings yeah

18:57 amalloy: &(.equals [] ())

18:57 lazybot: ⇒ true

18:58 amalloy: hm. i guess i expected that to return false, since there was some work done on making clojure collections obey the "broken" java APIs and instead use equiv or something for "useful" equality

18:59 technomancy: I think that was just for numerics?

18:59 dabd: how can I make sure int-array will create an array of 32 bit integers?

19:00 tieTYT2: i've often got this problem when I develop clojure code that I assume the types that are input/output and I assumed wrong. But the errors aren't very helpful in figuring out what is really going on. How do you guys usually debug these issues? EG: an arity exception is only sometimes helpful. It doesn't tell you what you passed in, just the amount you passed in

19:00 amalloy: dabd: call it. how could it ever do anything else?

19:00 tieTYT2: another eg: I have a function I think will return a vector of x, but it really returns a vector of vectors of x

19:00 dabd: i was searching the docs but nowhere it says it won't create a 64 bit integer

19:00 amalloy: &(doc int-array)

19:00 lazybot: ⇒ "([size-or-seq] [size init-val-or-seq]); Creates an array of ints"

19:01 amalloy: $google java int

19:01 lazybot: [Primitive Data Types (The Java™ Tutorials > Learning ... - Docs Oracle] http://docs.oracle.com/javase/tutorial/java/nutsandbolts/datatypes.html

19:01 tieTYT2: amalloy: why do you use & there?

19:01 ,(doc int-array)

19:01 clojurebot: "([size-or-seq] [size init-val-or-seq]); Creates an array of ints"

19:01 tieTYT2: same thing?

19:01 amalloy: did you notice anything different that happened in response to my message vs yours?

19:02 (hint: if the answer is no, keep looking)

19:02 tieTYT2: yeah that little => thing

19:02 amalloy: anything else...?

19:02 tieTYT2: not that I see

19:02 amalloy: say, that two different nicks responded?

19:03 tieTYT2: oh ha, right

19:07 patchwork: What is the name of that function that chooses a random element from a list? I thought it was `choose`, but I was wrong

19:08 Or I can just write it again

19:08 amalloy: $findfn [1 1 1 1 1] 1

19:08 lazybot: [clojure.core/second clojure.core/last clojure.core/peek clojure.core/first clojure.core/fnext clojure.core/rand-nth]

19:08 pppaul: woah

19:09 that's like smalltalk stuff

19:09 $findfn {} []

19:09 lazybot: [clojure.core/lazy-cat clojure.core/sequence clojure.core/vec clojure.core/concat clojure.core/seque clojure.core/drop-last clojure.core/reverse clojure.core/cycle clojure.core/rest clojure.core/lazy-seq clojure.core/flatten clojure.core/sort]

19:09 justin_smith: is findfn in clojure.repl?

19:09 pppaul: $findfn {} [1]

19:09 amalloy: $google clojure findfn

19:09 lazybot: [Raynes/findfn · GitHub] https://github.com/Raynes/findfn

19:09 patchwork: amalloy: Badass

19:10 justin_smith: ahh

19:10 lazybot: []

19:10 pppaul: $findfn {:a 1} []

19:10 lazybot: [clojure.core/drop-last clojure.core/rest clojure.core/flatten]

19:10 pppaul: today i almost found a use for empty

19:11 it was related to displaying the shape of data

19:21 tieTYT2: i have these java junit tests that test my clojure code (because they were written before the clojure code was used). Is there a way to run the tests in a repl?

19:21 i guess I can do it with a doto that calls the setup

19:21 hiredman: tieTYT2: sure, just figure out how junit runs tests and do that from clojure

19:22 https://github.com/sonian/Greenmail/blob/20247759a0c6b1782e4307e4f143e700f057bd39/test/greenmail/test/zzz.clj

19:23 or something

19:24 tieTYT2: i see, thanks

19:41 patchwork: Because I know this is what you are all looking for: https://github.com/prismofeverything/zalgo

19:43 justin_smith: patchwork++

19:43 (inc patchwork)

19:43 lazybot: ⇒ 1

19:43 justin_smith: ;oops

19:51 muhoo: hu, theres' no way to get array types like org.postgresql.jdbc4.Jdbc4Array into a db using clojure jdbc is there?

19:51 tieTYT2: why am I getting this error? https://www.refheap.com/paste/6b946c5e313795f048f8ac253

19:51 is it because the 2nd and 3rd are both lists as input?

19:52 hiredman: destructuring is not pattern matching

19:52 tieTYT2: i see

19:52 hiredman: [a] and [[a]] are both one argument functions

19:52 the second just treats the one argument as a list and binds a name to the first value

19:52 tieTYT2: ok that's what I figured. What should I do that will still be concise?

19:52 muhoo: i can get them OUT, but not in. (-> (jdbc/query db ["select * from array_test"]) first :ips .getArray set) works great in the query direction. insert, not so much.

19:58 arrdem: hum... so my project explicitly requires ring 1.1.8, lein deps :tree shows 1.1.8 loaded but even after restarting nrepl can't find ring.util.request

19:58 ideas?

19:59 weavejester: arrdem: ring.util.request was only added in 1.2.0-beta1

19:59 arrdem: weavejester: that'd do it :/

19:59 weavejester: Speaking of which, I need to sort out RIng this weekend and release an RC

20:06 arrdem: weavejester: is com.cemeric/friend usable?

20:07 weavejester: arrdem: I know people who've used it

20:07 As far as I know it is

20:07 arrdem: weavejester: ok thanks

20:08 tieTYT2: i've got a long threading macro, I've got a bug in one of the steps. How can I trace that piece?

20:08 it's currently doing: (map update-algorithm)

20:08 I want to see the end result of that

20:09 dabd: could someone help me trying to read a gzip file of ints into an int-array. So far I got this https://gist.github.com/anonymous/5551539. But java IO is not so simple and I don't get why instead of an int array it shows a ByteBufferAsIntBufferB instead of an int-array.

20:09 justin_smith: insert (first (juxt identity pprint/pprint)) if you are using ->>

20:09 err

20:09 not quire

20:09 ((juxt identiy pprint/pprint)) first

20:10 that works with either style of threading

20:10 prints anyhere in the midle

20:10 *middle

20:10 tieTYT2: why is the first outside of everything?

20:10 justin_smith: (-> foo bar baz)

20:11 (-> foo bar ((juxt identity pprint/pprint)) first)

20:11 replicates data flow while printing

20:11 tieTYT2: i see

20:11 justin_smith: oops, should have kept the baz at the end

20:11 but you get the idea, I hope

20:11 tieTYT2: ok I think I get it, thansk

20:13 callen: Stuff like juxt makes me so happy. FP lets you avoid so much boilerplate.

20:14 justin_smith: <3 juxt

20:14 technomancy: pprint doesn't need juxt though

20:14 (doto x pprint)

20:16 justin_smith: doto does not work as nicely with ->> though

20:16 ((juxt x y)) works with both threading styles

20:17 technomancy: oh yeah, true

20:23 tieTYT2: this is horrible, how do I make this more concise?

20:23 https://www.refheap.com/paste/02e9f3d0ee4e0d3a474bf9607

20:24 justin_smith: the nested ifs could be a cond?

20:25 tieTYT2: i'll try that

20:25 this destructuring is not making things easier for me

20:26 dnolen: tieTYT2: seems convoluted this looks like map

20:27 SegFaultAX: tieTYT2: (if (and c1 c2 (= (.getStartTime c1) (.getStartTime c2))) ...

20:27 dnolen: where the sequence is pairs of values that aren't neseted

20:27 nested

20:27 SegFaultAX: Also, why is there a nullary clause at all?

20:28 justin_smith: [nil] as input?

20:28 tieTYT2: I'm getting nil as input because of the &rest

20:28 justin_smith: that is how c1 could be falsey

20:28 tieTYT2: which I didn't expect

20:28 this would be easier for me to think about with pattern matching

20:29 dnolen: tieTYT2: I don't see how, you're just mapping over a sequence

20:29 write it as a map

20:29 tieTYT2: but I'm reducing

20:30 (perform-update! c1 c2) may return c1 or c2

20:30 dnolen: mapcat

20:30 tieTYT2: hrm, I'll look into that

20:31 but the INPUT is one collection, not a collection of collections

20:31 it's the output that's a collection of collections

20:32 justin_smith: tieTYT2: also (partition 2) may help get your pairs

20:32 tieTYT2: my newb brain looks at mapcat and thinks it's only for the latter

20:32 justin_smith: tieTYT2: an extra level of [] fixes that

20:32 dnolen: (apply concat (map (fn [[c1 c2] ...) (partition 2 xs))))

20:33 sans typos

20:33 justin_smith: ,(mapcat identity [[:a] [[[:b]]] [[:c]]])

20:33 clojurebot: (:a [[:b]] [:c])

20:33 tieTYT2: ok but if the input is [1 2 3 4], won't that only map on [[1 2] [3 4]]?

20:34 dnolen: ,(partition 2 [1 2 3 4])

20:34 clojurebot: ((1 2) (3 4))

20:34 tieTYT2: I need to call the fn on 1 2, take the result of that and call the fn on it and 3

20:34 dnolen: ,(map list (map first (partition 2 [1 2 3 4])))

20:34 clojurebot: ((1) (3))

20:34 justin_smith: your code as stands does not do that

20:34 tieTYT2: seems more like a reduce to me

20:34 dnolen: ,(apply concat (map list (map first (partition 2 [1 2 3 4]))))

20:34 clojurebot: (1 3)

20:34 tieTYT2: justin_smith: oh you're right!

20:34 i move on to the next one

20:34 shoot

20:35 justin_smith: that is why I thought partition would be apropriate - bug for bug compatibility

20:35 tieTYT2: haha

20:35 see my real long term problem here is I don't know how to peer into my code to figure out that bug

20:35 dnolen: tieTYT: ok, rewrite it as a real reduce

20:36 tieTYT2: I realized that after you said it, but I don't know the tools to figure it out on my own

20:36 dnolen: er tieTYT2

20:36 tieTYT2: i've already got it that way

20:36 i'll show you what i do

20:36 i thought that the direction I was going would be cleaner

20:37 https://www.refheap.com/paste/a701a23918dd942500eae55be

20:37 justin_smith: regarding "knowing how to peer into the code" - the more code you read and write the easier that is

20:38 tieTYT2: yeah but this doesn't seem like a challenge in java. I'm biased, but it seems like a compiler would catch a lot of my issues (not this one)

20:38 callen: dabbling with and reading a lot of open source code has a way of inducing familiarity with alien concepts that might not otherwise happen with writing code in your comfort zone.

20:38 dnolen: tieTYT2: it's not a challenge in Clojure either once you're thinking functionally

20:38 justin_smith: yeah, other people's code is key

20:38 tieTYT2: that algorithm above does exactly what I want. It passes all my tests

20:39 dnolen: well I came from haskell actually

20:42 anyway thanks for the help. I gtg now

20:59 nkoza: (type ...) returns the :type metadata, but there is no reader macro shorcut to set it? You can only set it using ^{:type :mytype} {...} ?

21:00 ivan: linkedin spam suggests that Apple is looking for a Clojure/Java consultant

21:09 dnolen: nkoza: putting types on maps like that isn't so useful, kind of a holdover from the days of structs. If you want typed maps might as well use records.

21:32 technomancy: ivan: and twitter spam and email spam and probably lots of other forms of spam; geez =)

21:33 ivan: someone must have secretly written a mission-critical pile of Clojure

21:36 xeqi: oh, are they being public about it now?

21:36 gdev: what happened?

21:45 nkoza: dnolen: but how you dispatch using multimethods if the map isn't tagged? or the common idiom is to use also defrecord's for them?

21:46 bbloom: nkoza: you can dispatch on a key in the map

21:47 like the clojurescript compiler uses the dispatch-fn :op and has maps like {:op :let …} and {:op :fn …}

21:47 nkoza: ok, then using {:type xxx ...} in maps is obsolete, you usually dispatch by another key

21:48 (I'm only trying to get what's the common idiom)

21:48 bbloom: nkoza: i think dnolen was trying to say that maps with METADATA of :type was obsolete

21:48 nkoza: ah ok

21:48 bbloom: generally, you'd prefer to avoid naming a key :type since that implies a JVM or other host type

21:49 if you're going to switch by type, you can name the key the same way you'd name an enum in C or something like that

21:49 so like if you were to have a server that can respond to 5 different kinds of messages, you might enum MessageType { Connect, Chat, Disconnect} or whatever

21:50 nkoza: you do :message-type :connect , et al

21:50 bbloom: in that case, you can use a :message key, so you'd have {:message :connect …} {:message :disconnect ...}

21:50 yeah

21:50 something like that

21:50 nkoza: thanks, I think I need to read more clj code :)

21:51 bbloom: if you need strict dispatch by host type (for speed, interop, custom semantics, whatever) then it is preferable to use protocols than multimethods

21:51 but as a general rule, start with a multimethod & maps and move to custom types & protocols when you need them

21:52 (which should be a lot less often than you'd expect it is, coming from more type-ful languages)

21:54 hiredman: the other thing is sometimes a map can have different types depending on what operations you are doing

21:55 bbloom: hiredman: yup, good point

21:55 hiredman: {:storage :s3 :send :broadcast} might be something that in one context has a "type" of SavedOnS3

21:55 gfredericks: Using ::foo must be smelly if you ever use the expanded version as well, eh?

21:55 bbloom: gfredericks: huh?

21:55 hiredman: and in another context has the "type" BroadcastableMessage

21:56 bbloom: hiredman: that's a pet peeve of mine

21:56 gfredericks: bbloom: due to the possibility of changing the namespace of the implicit one without thinking to change the explicit ones

21:56 bbloom: i've seen sooo many projects with app/models/user.py

21:56 where that's 10X larger than all the other model code combined

21:56 there is no way it could possibly be a good idea to have PRECISELY ONE representation of a user

21:56 er s/.py/.rb/

21:57 gfredericks: oh, yeah, ::foo has burned me a few times

21:57 gfredericks: but it seems like a nice sugar if you _don't_ have any explicit usages

21:57 bbloom: yes

21:58 after dnolen taught me that you can use aliases ::foo/bar, then i decided that i'd never use ::bar and ::full.path.to.foo/bar together, instead spelling it out for ::bar

21:58 so yeah, exactly what you are saying :-)

21:58 it's too bad you can't assign a local alias for your current namespace...

21:58 like (ns [full.path.to.foo :as foo] ...)

22:00 hiredman: ,(ns foo (:refer foo :as bar))

22:00 clojurebot: nil

22:00 hiredman: I wonder if that actually works

22:00 bbloom: heh

22:00 * hiredman forgets how :refer works

22:00 bbloom: it's a grand mystery to me every time i need to write a namespace....

22:00 hiredman: really?

22:01 I :require all day long

22:01 sometimes :load

22:01 bbloom: *shrug* i eval whole files or individual forms from vim with cpr and cpp

22:01 i basically cargo cult copy paste namespaces every time i make a new 1 :-P

22:02 and i even understand them

22:02 i just fail at the syntax every time

22:02 hugod: so i think i have time to figure out code signing

22:03 hugod: but i'm looking at your code & realizing that you are overriding protocol implementations… isn't that, um… bad form? if anyone else loads your library, they will break their other uses of fipp

22:41 technomancy: :require :as and :refer-clojure :exclude are basically all I ues

22:42 oh, and :require :refer :all for tests

23:24 gdev: 55 issues tagged "newbie" and here I am wondering what I'm going to do with the rest of my night, here we go

23:27 nevermind, those are all closed; only 4 open newbie issues; false alarm

23:35 technomancy: gdev: is the link wrong?

23:36 gdev: technomancy, I think i was already in the issues section, clicked on the left nav tag, and it showed all the closed ones =/

23:47 n_b: I'm in the process of building a language model for Danish by parsing over wikipedia dumps, and part of that is the semi-canonical MapReduce example of getting a count of each word that appears in a corpus

23:48 However, I'm trying to avoid all the Hadoop overhead since this is really such a simple example, and not that large of a dataset, and curious what data structure you would suggest for computing it in memory

23:49 A trie is probably how I'd do it with a mutable data structure, but I'm not sure exactly what approach would be best in Clojure

23:51 Or should I just use a transient map and persist it at the very end?

23:51 mthvedt: n_b: the standard pattern is to (reduce processing-fn my-data-structure my-input-data)

23:51 if performance becomes an issue, you can swap in transients but the code shape for transients is the same (on purpose)

Logging service provided by n01se.net