#clojure log - Mar 04 2015

The Joy of Clojure
Main Clojure site
Google Group
IRC
List of all logged dates

0:01 ddellacosta: justin_smith: thanks, that's the conversation I needed to have. Much appreciated

0:01 justin_smith: np

0:07 rhg135: i miss clojure so much D:

0:08 ob_: what happened?

0:08 rhg135: after writing 3000 lines of java for an app that still only half-works i feel sad

0:08 ob_: :(

0:09 rewrite in clojure, problem solved!

0:09 rhg135: android

0:09 i guess cljs exists, but glue code

0:10 arrdem: lein-android exists..

0:10 codestorm: would be cool if android had a first class FP option

0:11 arrdem: there's a GSoC project proposed to build a Clojure -> ART compiler..

0:12 ob_: welcome your new overlords android and ios

0:12 and js

0:12 arrdem: eh compiler targets :D

0:13 Guthur: there has always been overlords they just get toppled now and then

0:18 cemerick: arrdem: do you have a link for that beyond what's @ http://dev.clojure.org/display/community/Project+Ideas

0:18 ?

0:18 arrdem: cemerick: neg.

0:20 rhg135: on the upside i atleast am using clj_ds

0:20 justin_smith: I won't ask what that is and just imagine that's a clojure that targets the nintendo ds

0:20 rhg135: but no literals make rhg135 sad

0:25 i know i enjoy reading code so here in case anyone wants to read java: https://github.com/rhg/derpyirc

0:27 * justin_smith can't find the part where it derps.

0:27 Guthur: I have a template, and i would like to run a shell command when doing lein new

0:27 is there any best practice for that

0:28 justin_smith: if lein lets you do that, it would be a very interesting way to root people's boxes

0:28 rhg135|zzz: it's called so mainly due to beaing a temp name, but it's quite fitting if you read the code

0:28 cast all the things!

0:29 johnmendonca: in the repl, how do I look up docs for something like clojure.lang.Namespace?

0:29 justin_smith: ,(apropos "javadoc")

0:29 clojurebot: ()

0:30 justin_smith: hrmph

0:30 Guthur: justin_smith: well the template seems to be just a function that lein calls which "renders" the files using what ever name is passed in

0:30 so presumably one could invoke a shell command there

0:30 justin_smith: johnmendonca: clojure.repl/javadoc can do that for you

0:30 Guthur: but i was wondering if there were some other approach

0:31 justin_smith: Guthur: yes, and given that no OS is without local privelege escelation bugs, it would be a way to get root on dev boxes

0:31 Guthur: yeah, but then everyone is careful about code they run on their boxes, right

0:31 hehe

0:31 this is for myself so I not really worried

0:32 johnmendonca: justin_smith: thanks javadoc is working, I thought I tried it already...

0:32 justin_smith: "wget http://i-own-ur-box | sudo sh" is always the first line of my install guide

0:33 TEttinger: justin_smith, quick give me a simple transducer example! I'm trying to show off what clojure does

0:33 johnmendonca: though when i run (javadoc clojure.lang.Namespace) it opens a google tab with two not useful result

0:33 justin_smith: johnmendonca: wow, that's weird

0:34 TEttinger: ##(sequence (comp (take 10) (map inc) (filter even?)) (range))

0:34 lazybot: ⇒ (2 4 6 8 10)

0:34 justin_smith: not the greatest example - what's cool there isn't really visible from the syntax

0:35 TEttinger: (inc justin_smith) ; thanks, I can understand it

0:35 lazybot: ⇒ 204

0:37 justin_smith: Guthur: in all seriousness, I am sure you can fit a call to clojure.java.shell/sh into your driver function. But I also feel sketched out about the fact that it would be so easy.

0:37 engblom: It seem like Gitorious has been the prefered place for Clojure code. As it will close on June 1 because Gitlab aquired Gitorious, lets hope all projects move to the same place so you do not end up searching many sites for suitable libraries and stuff.

0:38 justin_smith: engblom: what's on gitorious? I don't know of any clojure projects there.

0:39 engblom: most of us are using github, no? I have a paid account, I would think they would have told me if they were shutting down.

0:40 engblom: justin_smith: You are right. I must have slept too little as when I woke up this morning and saw the news about Gitorious I mixed it with Github.

0:40 justin_smith: oh, OK

0:41 * engblom should not comment on stuff before he is fully awake

0:41 justin_smith: engblom: anyone who hangs out here has seen me have sillier mixups

1:01 Niac_: what is about the side-effect-free in clojure?

1:11 numberten: is the idiomatic way to create directories in clojure just java interop?

1:11 arrdem: clojure.java.io is your friend

1:16 l3dx: how do I get clojure to call a method with one required arg + varargs instead of the overload with a single arg of a different type?

1:17 specifically http://docs.oracle.com/javase/7/docs/api/java/nio/file/Paths.html#get(java.lang.String,%20java.lang.String...)

1:17 numberten: arrdem: thanks :)

1:17 l3dx: insetead of the one taking a URI

1:21 I'm trying to call get(String)

1:22 cemerick: l3dx: (java.nio.file.Paths/get "foo" (into-array String []))

1:26 Eremox: Why is there no official standard on clojure? Is it to young?

1:26 l3dx: cemerick: thanks :)

1:27 arrdem: Eremox: there is no standard library or parser spec.

1:27 we are entirely implementation defined

1:28 TEttinger: there are 3 main implementations, all of which have small platform differences in how some parts of the language work. CLJS is the most different, and even that is still pretty close to clojure

1:29 (inc cemerick) ; thanks, that's handy

1:29 lazybot: ⇒ 18

1:30 mercwithamouth: hey is there a way for me to tell leiningen to use an older version of a template? like i'd like to use luminus...and JUST purchased a book that uses the web app that was published last month. however 2 days ago luminus updated to a new version that's completely different

3:15 dysfun: mercwithamouth: hrm, what's changed?

3:15 hrm, they appear to have added ragtime

3:17 hrm, they've changed a few things

3:19 mercwithamouth: dysfun: hey. lib-noir is gone

3:19 quite a few things it seems

3:21 dysfun: noir-exception is still there

3:21 i don't use lib-noir or noir-exception in my stuff

3:21 i have been using the luminus template to give me a base and then heavily customising it

3:21 looks like i'm going to have to just go build my own template now

3:27 mercwithamouth: dysfun: yeah. i just bought clojure web development essentials and now it looks like i won't be able to follow it as it

3:47 dysfun: mercwithamouth: that's unfortunate. but you can check out one of my projects based on the old luminus if it helps

3:47 let me see what i've got that i didn't touch much after generation

3:49 oh, i've just found one that i generated randomly and didn't use

3:49 you should be able to follow your book with that

3:53 the state of web development in clojure is such that i keep finding cool libraries i want to integrate

3:53 so you'll probably move on from old luminus quite naturally

5:49 devll: Hi,how to splice a lazy seq in macro?

5:50 (defmacro splice-test [col]

5:50 `(println ~@con))

5:50 (splice-test (range 10))

5:50 (splice-test (doall (range 10)))

5:50 none of all working.

5:52 am I being muted for asking too easy questions?

5:52 Bronsa: devll: that doesn't make sense, you're trying to use the runtime value of (range 10) at compile time

5:52 devll: oh

5:53 Bronsa: devll: what exactly is that you want to do?

5:53 devll: I wanted to do mysql batch insert

5:54 (j/insert! db-spec :fruit

5:54 {:name "Pomegranate" :appearance "fresh" :cost 585}

5:54 {:name "Kiwifruit" :grade 93})

5:55 Bronsa: devll: can't you justs iterate over a collection with map/reduce/doseq?

5:55 devll: slow

5:55 too slow

5:55 Bronsa: that's not true

5:56 doseq expands to a loop, it's hardly going to be too slow

5:56 devll: yes. doseq is not slow.

5:57 inserting one by one is.

5:57 wagjo: devll: (apply j/insert db-spec :fruit maps)

5:57 Bronsa: gotcha, then you didn't explain yourself. use apply

5:58 devll: sorry.My English sucks.so does my programming.I am working on it.

5:58 thanks.

5:59 (inc wagjo)

5:59 lazybot: ⇒ 4

5:59 Bronsa: no worries, try what wagjo said

5:59 wagjo: you're welcome

6:08 enaden: sp

6:11 egli: it appears that I can't call juxt on macros

6:11 I would like to ((juxt .getMessage .getLineNumber) exception)

6:12 wagjo: egli: juxt works with functions, and macros are not functions

6:13 egli: wagjo: I figured as much, thanks

6:13 wagjo: egli: try -> macro

6:13 egli: I want to do (zipmap [:error :line :column] ((juxt .getMessage .getLineNumber .getColumnNumber) exception))

6:13 instead of {:error (.getMessage exception)

6:13 :line (.getLineNumber exception)

6:13 :column (.getColumnNumber exception)}

6:14 but maybe it's not worth the hassle

6:15 wagjo: -> probably doesn't work in this case

6:15 wagjo: egli: yep, won't work for your case

6:15 Bronsa: egli: .getMessage & co are neither function nor macros

6:16 egli: Bronsa: ah .getMessage are macros that expand to a special form

6:16 Bronsa: they are sugar for . forms that only make sense in a call position, you can't use .foo as a higher-order method call

6:17 egli: Bronsa: ok all clear now. Thanks

6:18 wagjo: egli: maybe you can use memfn

6:20 ,((juxt (memfn isEmpty) (memfn length)) "foo bar")

6:20 clojurebot: [false 7]

6:22 egli: wagjo: ah, thanks. Just what the doctor ordered

6:22 Bronsa: wagjo: that code requires reflection. I don't think it's worth using memfn in egli's case

6:23 wagjo: yes the reflection is a big tradeoff here

6:23 Bronsa: (let [x (x.)] {:foo (.foo x) :bar (.bar x)}) vs (zipmap [:foo :bar] ((juxt (memfn ^x foo) (memfn ^x bar)) x))

6:23 I would never write the latter, there's really no advantage

6:28 wagjo: egli: Bronsa: how about using bean for this?

6:29 ,(try (/ 1 0) (catch Exception e (bean (aget (.getStackTrace e) 0))))

6:29 clojurebot: wagjo: Titim gan éirí ort.

6:29 wagjo: ,(try (/ 1 0) (catch Exception e (bean (aget (.getStackTrace e) 0))))

6:29 clojurebot: wagjo: Titim gan éirí ort.

6:33 Bronsa: wagjo: that might be a good idea however I remember bean not caching the values, maybe I'm just mis-remembering though

6:34 egli: wagjo: seems like a clever idea. Maybe too clever though, i.e. border line obfuscating for such a simple use case

6:35 Bronsa: ,(def a (Exception. "foo"))

6:35 clojurebot: #'sandbox/a

6:35 Bronsa: ,(def b (bean a))

6:35 clojurebot: #'sandbox/b

6:35 Bronsa: ,(seq (:stackTrace b))

6:35 clojurebot: (#<StackTraceElement sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)> #<StackTraceElement sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)> #<StackTraceElement sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)> #<StackTraceElement java.lang.reflect.Constructor.newInstance(Constructo...

6:35 Bronsa: ,(.setStackTrace a (into-array StackTraceElement []))

6:35 clojurebot: nil

6:35 Bronsa: ,(seq (:stackTrace b))

6:35 clojurebot: nil

6:35 Bronsa: yeah, I really don't like bean because of this

6:36 ,(instance? clojure.lang.IPersistentCollection b)

6:36 clojurebot: true

6:36 Bronsa: but it's actually mutable

6:36 wagjo: heh

6:36 clojurebot: It's greek to me.

6:37 minora: ciao

6:37 !list

6:39 |list

6:39 Bronsa: minora: that won't work

6:40 minora: why dont work?

6:41 Bronsa: minora: that's not a command implemented in any bot here. are you sure you're in the right channel? this is about the clojure programming language.

6:41 minora: ok thanks you

6:42 bye

7:55 noncom: i have some questions regarding google guava

7:55 1) is using google guava eventbus a good practice for clojure?

7:55 2) does it have any advantages/disadvantages to core.async?

7:56 3) does anyone have actual experience with that?

7:56 agarman: never used guava in Clojure. Used it a lot in Scala & Java.

7:56 mavbozo: i think stuart halloway and rich hickey found guava good enough to use in datomic

7:56 but i don't know if they use guava's eventbus

7:57 noncom: datamic is too, mainly a java program afaik

7:58 justin_smith: the fact that guava eventbus uses annotations makes it a bit less convenient from clojure

7:59 tcrayford____: noncom: datomic is *mostly* clojure from my understanding

7:59 justin_smith: annotations can be done from clojure, but it's a little awkward

7:59 tcrayford____: noncom: I use guava for yellerapp.com (just the caching though). It's been good so far

7:59 (my api's 99th percentile response latency is ~300 microseconds, primarily because of guava caching)

8:00 noncom: tcrayford____: so, you say it is fast?

8:00 tcrayford____: noncom: well, the cache is fast enough for my purposes at least ;)

8:01 noncom: no judging *at all* on the rest of the library, I use like 1% of it

8:01 noncom: ok, so yes, eventbus is based on annotations, which would look strange in clojure i think (never done them like that)

8:01 tcrayford____: is yellerapp.com part where you use them written in clojure or java?

8:02 tcrayford____: I think many folk would recommend core.async or looking at the j.u.c queues (which are the queues datomic uses internally)

8:02 noncom: I do *not* use eventbus, just guava's caching. It's all clojure

8:02 noncom: ah, got it. just caching :)

8:02 justin_smith: tcrayford____: exactly - core.async for convenience / clarity, j.u.c queues when you need more perf

8:03 tcrayford____: or the disruptor or whatever. There are a lot of options for in memory java queues with different tradeoffs

8:03 justin_smith: right

8:04 is disruptor really usable from clojure? it has some weird constraints to go with that blazing performance

8:04 tcrayford____: (http://research.neustar.biz/tag/disruptor/ covers a buncha research around them [disclaimer, from a friend's company])

8:04 justin_smith: it is, afaik. No annotations there, just a *lot* of mutability

8:04 justin_smith: gotta love ringbuffers :)

8:06 noncom: how much lower performance can i expect from core.async vs j.u.c ?

8:06 (if it is answerable at all)

8:06 tcrayford____: depends entirely on your use case, machine configuration, jvm etc

8:07 they're both pretty quick though

8:07 justin_smith: noncom: I'd say don't worry about it until you have a bottleneck. Sadly I don't have a concrete measured comparison. I should make one one of these days.

8:07 tcrayford____: iirc core.async benches at something like 12m channel transfers/s or so (but that's from memory, and I didn't verify the benchmarks like I do these days)

8:07 (and those benches were likely not on the machines you're gonna be running in prod either)

8:08 noncom: oh, well, that far covers my needs, i think :)

8:08 tcrayford____: likely most apps don't need anything like that though haha :)

8:08 yeah. I'd just use core.async, it's an easy reach for most clojure devs, is well adopted etc

8:09 noncom: yes, i just want to choose a mechanism for messaging inside the app, that won't be even a 100 channels and no big data sent over it, just small messages

8:09 i guess, i'll pick core.async again for that :)

8:09 tcrayford____: neat!

8:09 glad we could help :)

8:09 noncom: thanks :)

8:09 justin_smith: tcrayford____: I've noticed performance loss from using core.async. Maybe it's related to the core.async state machine and hotspot? not sure. There's more going on than just channel throughput anyway I think.

8:11 noncom: justin_smith: what was your scenario?

8:11 tcrayford____: justin_smith: the go macro will most likely prevent inlining because of the code bloat it generates if you put a lot of code in it

8:12 noncom: but again, the JVM is *very* fast, and it's likely that ain't gonna matter at all for your use case

8:12 noncom: yeah, my case is definitely the "light" one

8:13 i would go with j.u.c on something heavyweight then

8:13 justin_smith: noncom: core.async coordinating requests / caching on an image resize server

8:14 it was more about latency, even for cached result

8:14 s

8:14 also, I should do some more focused benchmarks, because it could have been something else stupid in my implementation logic :)

8:25 arav93: Hi! :)

8:26 dysfun: i need to maintain some dependency information. is there a library that aids in performing cycle detection?

8:27 Bronsa: dysfun: tools.namespace has a namespace that handles dependency graphs

8:27 dysfun: yes, which uses 'component'

8:27 but i'm struggling to reason about the behaviour when you create cycles

8:27 Bronsa: tools.namespace doesn't use component

8:27 dysfun: sorry, 'dependency'

8:28 https://github.com/stuartsierra/dependency

8:29 ah, it throws an exception. i hadn't spotted that

8:29 stuartsierra: The 'dependency' is just a copy of the dependency graph from tools.namespace

8:29 dysfun: that's acceptable

8:30 stuartsierra: Both are just a small adaptation from Clojure's hierarchy graph functions.

8:30 dysfun: *nod*

8:30 thanks

8:33 one of the things i love about clojure incidentally is that you randomly bump into the people who write the libraries you use on irc. it's quite a nice thing :)

9:07 noncom: i am trying to find the best way to write the macro: https://www.refheap.com/98107

9:07 please could someone answer questions 1 and 2, written in the comments?

9:08 dysfun: you're worried about performance loss using apply?

9:08 noncom: the aim of the macro is to accept a [] of functions and apply the args to all of them, filtering out nils

9:08 dysfun: i would suggest not to be in the general case

9:08 noncom: dysfun: yeah, also, using apply implies using (into []), at least the way i have managed to make it work with apply. can i avoid (into [])

9:08 ?

9:11 Bronsa: noncom: http://sprunge.us/MSOD?clj

9:12 noncom: Bronsa: wow

9:13 so actually macros can have several `() places..

9:13 did not know that

9:14 Bronsa: or even http://sprunge.us/UQUF?clj

9:14 noncom: but that's logical..

9:14 Bronsa: noncom: ` has nothing to do with macros

9:14 noncom: yes, i know.. but i just could not think of it this way :)

9:15 so, list* is even better..

9:38 justin_smith: noncom: why not ((apply juxt (filter identity funs)) args)

9:40 ,((apply juxt (filter identity [nil + - nil * /])) 5 3)

9:40 clojurebot: [8 2 15 5/3]

9:46 justin_smith: even if you decide a macro is better, juxt can eliminate your apply / into

9:47 eg. fns-apps `((juxt ~@fns) ~@args)

9:49 noncom: justin_smith: hmm, not really familiar with juxt much :) but i knew that a moment would come to get to know it better..

9:50 justin_smith: noncom: the above should replace your fns-apps binding in the let block. The difference is it returns a vector of all the results, so you would want a `(peek ~fns-apps) at the end instead of the do

9:50 (if you only wanted to return the result of the last function)

9:51 dm3: is there a way to sidestep git tag signing on lein vcs tag?

9:52 noncom: yeah, juxt returns a vector.. well, in my particular usecade all fns are pure side-effects (it is an IO stuff), but as i make this macro into my library of useful things, i guess, juxt is a nice thing to have since it is more general in returning a vector, not making assumptions on fns being side-effetless

9:52 justin_smith: yeah, if you are typically ignoring the return value, may as well just return the vector if you use juxt

11:23 nsjph: what's a good file-backed key value store i can use with clojure?

11:24 aside from just spitting out EDN to disk

11:34 noncom: hsjph: probably none.. widely known.. but why not edn?

11:35 justin_smith: this makes me wonder if anyone has done a simple edn -> directory structure mapping

11:36 nsjph: noncom: i don't feel like thinking about thread-safety and database-like resiliency for a flat file

11:36 i also dont feel like using sqlite here

11:37 justin_smith: numbers, strings would become files - the tricky part would be hash-maps with composite keys...

11:37 nsjph: so i was wondering what key-value stores with file-based backends (ala sqlite) are about

11:37 justin_smith: nsjph: there's mongo, if you are aware of the pitfalls and don't mind them

11:37 congomongo

11:37 nsjph: justin_smith: oh, and i dont want to install a database server

11:37 noncom: nsjph: maybe http://www.h2database.com/html/faq.html#database_files ?

11:37 nsjph: particularly mongo ;)

11:38 justin_smith: well, for sql, embedded h2 is an option, yeah

11:38 nsjph: i should probably just bite the bullet and use either sqlite or h2

11:38 justin_smith: with a file backend

11:38 nsjph: couchdb?

11:39 nsjph: no network server :)

11:39 noncom: orient db also. they claimed to provide a local db mode

11:39 nsjph: i've also used couchdb, didn't mind it

11:39 not a bad db

11:39 hiredman: http://hiredman.github.io/Amontillado/

11:40 nsjph: interesting

11:41 justin_smith: (inc hiredman)

11:41 lazybot: ⇒ 74

11:41 justin_smith: that looks great

11:43 noncom: justin_smith: so, what did you say about mapping edn to filesystem?

11:43 what is the use about this?

11:45 justin_smith: noncom: it would mean that the persistent data structure operations could be mapped to file system operations

11:46 including cheap immutable update

11:46 theoretically

11:46 maybe it's a silly idea

11:46 noncom: justin_smith: wow, why did not someone just made that? are there any problems with that?

11:46 justin_smith: there's probably some pitfall that I am overlooking

11:47 eg. something that works in the jvm but not on a file system level

11:47 maybe it's just the added difficulty of making it work on more than one OS (because they don't use the same filesystems, so they have different rules regarding locking etc.)

11:48 noncom: java.nio.Files seems to target just that - working flawlessly on any fs

11:48 (available from java 7 and up)

11:48 justin_smith: clojure targets 1.5

11:48 err, 5 by that system :)

11:48 noncom: but we can use any java version from inside clojure anyway...

11:57 sdegutis: Is there any chance for a ClojureScript compiler that doesn't depend on the Google Closure Compiler or Library at all?

11:57 I'd love to just write Clojure and let it compile down into pure JavaScript.

11:59 justin_smith: sdegutis: what about something like scriptjure? https://github.com/arohner/scriptjure

12:00 sdegutis: justin_smith: not a bad idea

12:00 thanks

12:00 justin_smith: it doesn't do the clojure semantics, it's just a syntax translator

12:00 sdegutis: That's fine, I don't mind JS semantics.

12:00 I'm just tired of managing commas.

12:01 justin_smith: sdegutis has forsaken immutability

12:02 sdegutis: justin_smith: "If a tree falls in the woods, does it make a sound? If a pure function mutates some local data in order to produce an immutable return value, is that ok?" - Rich Hickey

12:02 justin_smith: :)

12:02 sdegutis: justin_smith: I just use _.cloneDeep(json) for immutability

12:02 where _=lodash

12:02 justin_smith: check out https://github.com/sdegutis/mybudget/blob/master/database.js btw, it was designed inspired by clojure's immutability

12:03 justin_smith: cool

12:03 sdegutis: I love that the immutability aspect made undo/redo into one-liners

12:03 On the other hand, it's probably ridiculously inefficient compared to using something like Immutability.js

12:03 (which I assume does data-structure sharing etc)

12:04 justin_smith: yeah, proper persistent data structures

12:19 puredanger: justin_smith: clojure targets Java 1.6, not 1.5

12:19 justin_smith: puredanger: is that a recent change?

12:19 puredanger: Clojure 1.6

12:19 justin_smith: oh, TIL, thanks

12:20 (inc puredanger)

12:20 lazybot: ⇒ 34

12:20 puredanger: Clojure 1.7 will also target Java 1.6+

12:20 danielglauser: puredanger: what Java version does Clojure 1.7 target?

12:20 justin_smith: puredanger: still supports the point I was making about 1.7 features :)

12:20 puredanger: Clojure 1.8 will likely change to target Java 1.7+

12:21 Java 1.7 public updates end next month

12:21 http://www.oracle.com/technetwork/java/eol-135779.html

12:22 Java 1.6 public updates ended 2 years ago

12:22 if you want to use 1.7+ stuff but fall back to something else, search for the compile-if macro in Clojure's source

12:35 eflynn: So I’m writing a scheme interpreter in clojure, and i’m wondering how i should handle mutable state

12:36 I was thinking something like (eval exp env) -> (exp, env2)

12:36 where changes in the environment are returned

12:36 any thoughts

12:50 justin_smith: eflynn: one advantage of that (rather than an implicit global env, or env represented as a bunch of mutable objects) is that it is flexible enough to map cleanly to the other alternatives

12:51 eflynn: justin_smith: map as in rewrite?

12:52 justin_smith: eflynn: you can implement a global env object, or separate mutable objects for each variable, based on the explicit passing of state

12:52 the translation does not work as well going the other way

12:52 you can't really model explicit state passing using a global env or mutable objects, at least not cleanly

12:53 eflynn: explicit state passing with a global env wouldn’t make sense

12:54 justin_smith: eflynn: right, I am saying that given the implementation of explicit state passing, you can implement the others

12:54 but not visa versa

12:54 so it's a superset of the others

12:54 eflynn: ok

12:55 justin_smith: it would involve a wrapper function most likely

12:55 but it's harder to write an unwrapper function :)

14:33 dbell: long time no see clojurists...is there a way to get the # of args a fn takes? I've dug around in the source a bit but haven't found anything

14:34 dnolen: dbell: you can only do that via the meta of a var

14:34 ,(-> #'first meta :arglists)

14:34 clojurebot: ([coll])

14:34 dnolen: dbell: ^

14:35 dbell: how about in cljs?

14:35 what w/no vars

14:36 dnolen: dbell: the will work in CLJS too

14:36 s/the/that

14:41 justin_smith: ,(defn inane-arities [f] (keep #(try (apply f (repeat % nil)) % (catch clojure.lang.ArityException e nil) (catch Exception e %)) (range 1024)))

14:41 clojurebot: justin_smith: Pardon?

14:41 justin_smith: :P

14:41 anyway, that code is stupid, but it returns all argument counts the function accepts below 1024

14:41 don't use it, just a proof of concept

14:45 dbell: justin, ha, yeah, i thought of that too, but it felt, uh, well, you know

14:45 justin_smith: a little dumb?

14:45 dbell: ha, yeah

14:45 david, i must have misunderstood, but i can't seem to access vars in cljs

14:46 justin---i might take that and use it to make a lookup table

14:47 justin_smith: haha, oh man...

14:59 and thanks to that silliness I learn...

14:59 ,(conj)

14:59 clojurebot: []

15:02 justin_smith: (conj Double/NaN)

15:02 ,(conj Double/NaN)

15:02 clojurebot: NaN

15:31 amalloy: dnolen: anything i should be doing to make http://dev.clojure.org/jira/browse/LOGIC-167 more palatable? it's not like you to leave a patch unmerged for longer than ten minutes

15:31 dnolen: amalloy: nah nothing to do just been busy with other stuff

15:31 amalloy: okay

15:31 dnolen: other people have pinged me for another core.logic release since various fixes are still sitting around

15:31 will cut a release on Friday

15:33 Beamed: I hope you guys don't mind me asking, but I was having trouble with a random 4Clojure problem and was hoping to ask for some advice

15:33 dbell: fire away

15:34 Beamed: basically, it's an implementation of interpose, and my solution is just (fn [val & seq]( reduce #(concat %1 val %2) seq) )

15:35 but for some reason, that just returns the passed sequence, rather than concatenating it with val :/

15:35 for example, given 0 [1 2 3] it should provide [1 0 2 0 3] , but it just returns [1 2 3]

15:36 justin_smith: that's odd, because (concat 1 0 2) shouldn't work

15:36 amalloy: Beamed: you probably mean [val seq]

15:36 Glenjamin: ,(concat 1 0 2)

15:36 Beamed: I ran it through the lein repl and it does the same justin

15:36 clojurebot: #<IllegalArgumentException java.lang.IllegalArgumentException: Don't know how to create ISeq from: java.lang.Long>

15:37 Glenjamin: ah, i see why it doesn't error

15:37 amalloy: justin_smith: seq here is [[1 2 3]], not [1 2 3] as intended

15:37 justin_smith: Beamed: it worked by accident, because of the issue amalloy points out

15:37 Beamed: amalloy - I think that's why it's not crashing; I was a dope and made seq a variadic param. Doy

15:37 amalloy: right

15:37 Glenjamin: the & and missing initial case combine

15:38 Beamed: thanks, I knew it was something like that.

15:40 amalloy: Beamed: happy 4clojuring

15:50 Beamed: hah, and then I see it was as simple as combining but-last and interweave >_<

15:51 amalloy: Beamed: well. the thing about 4clojure is, you solve it in whatever way advances your understanding best

15:51 you wanted practice using reduce, so you used reduce

15:52 that kind of flexibility is one of my favorite things about it

15:52 blake_: This is very true. You can go back and solve the same problem multiple times to gain insight into different mechanics.

15:54 amalloy: for example when i solved 4clojure problems i already felt quite comfortable with combining built-in functions like map, filter, reduce, whatever, so i probably would have easily seen a solution like the second one you mentioned. but i wanted more practice with manual recursion, so i solved the problems that way instead

15:56 Beamed: Yeah, you're right - I'll comb through ti again when I know more about all the built-in functions to see how simple and elegant I can re-do it - for now, I'll try learning them at all.

15:56 Thanks!

15:56 puredanger: as always, http://clojure.org/cheatsheet is a great resource

16:00 amalloy: is (some-> x (f)) a better or worse way to write (when x (f x))?

16:01 justin_smith: amalloy: I would usually express that as (and x (f x)) but maybe that's weird

16:01 puredanger: the when is clearer to me at a glance

16:01 dbell: i would do what justin would do

16:02 but some-> isn't well known

16:02 so maybe not such a bad thing

16:02 amalloy: justin_smith: i don't like the and at all

16:02 ToxicFrog: amalloy: I'd write that as (some-> x f) or (and x (f x)), but I prefer the former.

16:02 amalloy: but i guess i haven't mentioned whether f is for side effects or values (it's side effectts)

16:03 justin_smith: oh yeah, I wouldn't use and for side effects

16:03 amalloy: justin_smith: another thing is, to use when or and you have to actually name the thing; with some-> you can get away with never naming it

16:04 justin_smith: well, you could generate it in context the same as you did with some->

16:04 or am I missing something there?

16:04 ahh, right

16:04 don't mind me, I get it now

16:04 yes, that's a clincher for some->

16:08 ToxicFrog: ,(doc when)

16:08 clojurebot: "([test & body]); Evaluates test. If logical true, evaluates body in an implicit do."

16:08 ToxicFrog: Oh that's handy.

16:08 noncom: i am writing an app that has a controlling messaging system in it, realized with core.async. i am thinking about using vectors for messages, where the first kw is the message id and then goes some parameters.

16:09 my wuestion is: will constructing a vector for each message be an expensive operation?

16:09 amalloy: noncom: are you writing the firmware for a pacemaker?

16:10 noncom: basically, a multimedia app. for example, i plan to use such messages for spreading mouse cursor parameters into the app

16:10 justin_smith: noncom: depending what you do with it, the vector overhead isn't likely to be a big deal. Couldn't hurt to benchmark vector vs. list, it shouldn't be hard to write your code so they could be swapped out

16:10 noncom: or to facilitate messages between ui elements

16:10 think of like a complex 3d game level scripting

16:11 amalloy: noncom: it was a rhetorical question, meant to imply: "unless you are writing software where the delay might kill somebody, focusing on microoptimizations like trying to reduce the number of vectors you allocate is pointless folly"

16:11 puredanger: how are you going to *use* the data (indexed access, etc)?

16:12 amalloy: additionally there are zillions of vector allocations you can't do anything about going on all the time, so removing some of yours won't have any substantial impact

16:12 noncom: oh :D

16:13 yes, data in the messages are access and some params

16:13 more like bash commands with params. a similar kind of load/idea

16:14 puredanger: if you need indexed access, then the cost of that (or not getting that) is probably more important than constructing the vector. but if you're really trying to push memory/perf, you could also consider arrays.

16:15 in general, any "which data structure" question in clojure is often better approached first from the "how will it be used" angle

16:15 amalloy: (inc puredanger)

16:15 lazybot: ⇒ 35

16:16 noncom: well, like generating and destructuring 1000 []/s is ok?

16:17 puredanger: sure :)

16:17 (I have no idea really)

16:17 why guess when you can test?

16:18 noncom: oh sure :)

16:18 amalloy: noncom: put an abstraction layer in between your logic and the data structure it manipulates, and then you can change it if you end up needing to do something else

16:18 noncom: right, that's nice !

16:20 amalloy: personally my default choice would be to build more objects, not fewer, something like {:cmd "ls" :args ["/var/lib"]}. but it'll all depend on how you want to use your data, like puredanger says

17:38 imanc_: is there a way to break into the repl in some executing code similar to python's import pdb; pdb.set_trace() ?

17:45 justin_smith: imanc_: you can get close with jdb (but it's not clojure aware, you only see the jvm level stuff) or with cursive (not sure how extensively the debugging is implemented, but I know it is present)

17:46 and I meant to add, the debug tooling in cursive is more extensive than with any other clojure environment I know of

17:49 imanc_: OK, cursive sounds like the way forward :)

18:39 farhaven: asdffffff

18:39 .

18:50 cfleming: imanc_: With Cursive you can set breakpoints and when it stops you can evaluate expressions

18:50 imanc_: It's not quite a full REPL (yet) but it's pretty useful

18:51 imanc_: And you can break on uncaught exceptions etc

18:51 imanc_: I'll be talking about it and demoing it at Clojure/West actually

18:54 the_danko: greetings

18:55 anyone know how to do a "ctrl-R" style command history search in the cursive REPL?

18:56 cfleming: the_danko: Greetings. You're looking for Tools->REPL->Search REPL History

18:57 the_danko: sweet, thanks

18:57 will remap that guy

18:57 cfleming: the_danko: Narrows down on typing, when you select it'll copy the command to the REPL editor or shift-enter will execute immediately

18:57 the_danko: also, what does this load forms in repl namespace fellow do?

18:58 cfleming i was hoping you would be around!

18:58 cfleming: the_danko: You mean Run top form in REPL?

18:58 the_danko: I saw the IRC bat-signal

18:58 the_danko: ha

18:59 i mean in settings

18:59 cfleming: the_danko: Run top form in REPL finds the top-level form that your cursor is in, and sends that to the REPL for execution

18:59 the_danko: under load out of date files transitively

18:59 cfleming: the_danko: Ah

18:59 the_danko: That controls the namespace that's used when executing forms from the editor.

19:00 michaniskin|BNC: is there a library that will return #'clojure.core/ex-info, given the string "clojure.core$ex_info" (but for the java class name of any clojure function)?

19:00 the_danko: cfleming so on this subject, i do load file in repl a lot as i am working.

19:00 cfleming: the_danko: So you send a form from the editor using Run top form or whatever - if that setting is checked, the form will be executed in the current namespace of the REPL. If it's not checked, the form will be run in the namespace of the file the form came from.

19:01 the_danko: cleming that is idomatic, no? but then i have to do ('using editor.namespace)

19:01 idiomatic

19:01 am i doing something wrong?

19:02 use

19:02 not using

19:02 cfleming: the_danko: I'm not sure what you mean - you're talking about when you load a file in the REPL?

19:02 the_danko: yeah so i have a file cljlearn.core

19:02 and my repl in the user namespace

19:03 and when i change a function in cljlearn.core and reload it

19:03 sorry i mean when i add a function to cljlearn.core

19:03 i have to do (use 'cljlearn.core)

19:03 again

19:03 justin_smith: the_danko: that won't help

19:03 because you have loaded the ns already

19:04 the_danko: well if i add something to the namespace

19:04 a new function

19:04 then i have to reload it

19:04 justin_smith: maybe, but use won't reload

19:04 the_danko: basically, i want to know what the best practice is as far as what namespace to keep the REPL in

19:04 relative to the file i am working on in the editor

19:05 cfleming: Yeah, in that case you will have to, because the symbols are all interned at the point you (use...)

19:05 So you can just switch to that namespace and work in there

19:05 the_danko: cfleming is that what you do?

19:05 cfleming: Or you can create an alias to it, and use (alias/function ...)

19:05 the_danko: (in-ns 'cljlearn.core)

19:06 cfleming: the_danko: Yeah, or there's a Cursive command to switch to the current ns

19:06 the_danko: cfleming and then (use 'clojure.core) and a bunch of other sh*t?

19:06 cfleming: the_danko: It depends what I'm doing. I usually use aliases

19:06 the_danko: so in the repl you set up an alias

19:06 justin_smith: the_danko: (clojure.core/refer-clojure) because use won't work if clojure isn't referred yet

19:07 the_danko: ha yeah i mean so that seems like a pain

19:07 justin_smith: the_danko: but if the ns has already been created properly, you won't need to do either

19:07 cfleming: Cursive auto-requires in the REPL, so if I'm typing (str/tr and I complete it to str/trim, Cursive will auto-require clojure.string :as str, as long as there's an example of that somewhere in your project

19:08 the_danko: interesting

19:08 cfleming: the_danko: But I'm mostly working on Cursive itself, which is a relatively massive project, so my use case is different to working in a single namespace on a smaller project

19:08 the_danko: yeah i bet

19:08 cfleming: the_danko: For that, aliases work best

19:08 the_danko: for which?

19:08 the large project?

19:09 cfleming: For bigger projects

19:09 For a smaller one, I'd probably just switch to the ns and work there

19:09 the_danko: well my project won't always be small

19:09 ok thanks

19:09 cfleming: the_danko: Sure, but there's nothing to stop you changing how you work at that point

19:10 the_danko: good poiont

19:10 point

19:10 cfleming: the_danko: If you really want to work in user, I'd recommend aliases, I think

19:10 the_danko: so i open up the repl

19:10 cfleming: the_danko: But I just do whatever's easiest for what I'm working on at the time

19:10 the_danko: so what all is in user by default?

19:11 core

19:11 that i have to add when i work in a given namespace

19:11 nevermind, i'll figure that out myself

19:11 not a high value question

19:11 cfleming: core is always there, there's a few other things like pprint

19:11 One sex

19:11 Oooops

19:11 One sec, rather

19:11 That's a freudian slip if ever I saw one

19:12 amalloy: cfleming: i was quite embarrassed as a teenager when i wrote "afk for a sex"

19:12 even worse than yours

19:13 michaniskin|BNC: a "freudian cock" as we say in the land of recursion

19:14 the_danko: so cfleming, do you use the debugger at all?

19:14 cfleming, like one does in java?

19:16 cfleming: the_danko: (apply require clojure.main/repl-requires) will set any ns up as user is

19:16 the_danko: cfleming awesome, thanks!

19:16 cfleming: amalloy: When I was at school in the US while my folks were on sabattical, I was 14 I think, I asked a girl in my class for a rubber

19:16 amalloy: cfleming: that's neat, i didn't know that

19:17 cfleming: ha, a classic cross-pond mixup

19:17 cfleming: amalloy: Which in the civilised world is an eraser

19:17 amalloy: i hope you asked for a fag at some point during a visit as well

19:17 cfleming: amalloy: I also had a similar "bush" incident

19:17 amalloy: cfleming: have you checked to see if you are in a romantic comedy?

19:18 justin_smith: cfleming: a friend of mine went to australia from the US, did not know why everyone thought her story about falling on her fanny was so funny

19:18 cfleming: amalloy: I hope so, I'm hanging out for the happy ending

19:18 justin_smith: Another classic

19:18 justin_smith: pants is another good one

19:18 amalloy: thong

19:19 cfleming: the_danko: Yeah, I use the debugger all the time

19:19 amalloy: really it's a wonder we can talk to you folks at all

19:19 cfleming: yeah, no doubt - especially as teenagers

19:19 the_danko: cfleming sweet, look forward to trying that

19:19 cfleming i had to buy these http://www.amazon.com/gp/product/B003Y95IVU/ref=oh_aui_detailpage_o01_s00?ie=UTF8&psc=1

19:20 cfleming for my all black das in order to keep up with your keyboard shortcuts.

19:20 cfleming ha, definitely worth it though.

19:20 amalloy: the_danko: huh? do you have an azerty layout or something?

19:20 cfleming: the_danko: Ha, yeah, paredit with a das would be a challenge

19:20 the_danko: i know where the letters are

19:21 but i dont have great memory yet for } ] f12

19:21 etc

19:21 as cfleming says, it's a little tough flying blind

19:21 cfelming finally i was able to admit i needed help. it took a long itme.

19:21 cfleming: That's always the first step in any recovery

19:22 blake_: cfleming: The auto-require is just wonderful.

19:22 cfleming: blake_: Yeah, I love that - I use it all the time

19:23 blake_: cfleming: Money's just burning a hole in my pocket, dude...

19:23 cfleming: blake_: Arg, I know

19:24 blake_: cfleming: heheh...I'm being very selfish here...I really want to encourage continuing development. =P

19:25 cfleming: blake_: hehe, yeah - development is continuing!

19:25 blake_: cfleming: Groovy.

19:25 cfleming: blake_: Development speed isn't limited by money as much as by only having 24 hours a day, and having a baby daughter

19:26 blake_: cfleming: Yeah, I get that. They're cute and exhausting and don't really care about your projects. =P

19:27 cfleming: blake_: I commented to a friend that kids will soak up all the time you let them. He said yeah, they're like Farmville

19:27 He's a fountain of wisdom

19:27 blake_: cfleming: lol...yeah...a regular Dr. Spock.

19:54 CaptainLex: Is there a good resource out there for relative beginners on how to use the REPL to debug? I hear it's very powerful and I think I'm in need of that power

19:54 I'm actually working in CLJS but it'd be good to know for later CLJ projects

19:56 bridgethillyer: I found this useful: http://blog.jayfields.com/2014/01/repl-driven-development.html

19:56 Morgawr: mmm.. I recently updated clojurescript for my project and now it's telling me a lot of warnings about cljs.core/update being shadowed by some functions I have (which are called 'update')

19:57 the problem is... where does update come from? It's not in the clojure.core library...

19:57 or at least I can't find it. I see it in clojurescript

19:57 is this something new that I missed?

19:58 michaniskin|BNC: Morgawr: https://github.com/clojure/clojurescript/blob/master/src/cljs/cljs/core.cljs#L4142

19:59 Morgawr: michaniskin|BNC: Yes, I know, as I said in my question :P

19:59 I see it in clojurescript, but where does it come from?

19:59 it's not in clojure

20:00 looks like a wrapper around using assoc with mutable javascript data.. ?

20:00 CaptainLex: michaniskin|BNC: Thanks

20:00 !

20:01 enaden: t

20:11 tvanhens: is there any way to execute a series of jdbc requests in a transaction without using a macro in the jdbc library?

20:12 amalloy: tvanhens: i mean, anything you can do with a macro can be done without a macro, by writing out whatever the macro expands to

20:13 tvanhens: good point. was wondering if there was a function option in that lib before I start hacking with that approach

20:14 hiredman: what would you expect the function to do?

20:15 amalloy: tvanhens: db-transaction* looks like the thing to use?

20:15 it takes in a database and a thunk, and executes that thunk while inside a transaction

20:15 hiredman: the bounds of jdbc's transaction macro go a long towards delineating a good transaction

20:16 amalloy: if tvanhens wants that there is also the option of (fn [db fun] (with-db-transaction [foo db] (fun foo)))

20:16 amalloy: true

20:17 hiredman: if your code does fit that shape, you might end up having a bad time with transactions

20:17 doesn't

20:17 tvanhens: thanks guys the problem with that approach hired is all my query definitions are in data. I'm trying to create all their jdbc calls with generated code so the macro is kinda throwing a wrench in things

20:19 db-transact* looks promising

20:19 hiredman: the thing to keep in mind is transactions can fail, so how do you encapsulate the execution of the code for the transaction in a package that is retryable on failure

20:20 lvh: Does anyone have any reccomendations for getting useful error messages out of core.test? as opposed to actual: false :/

20:20 core.logic's featurec does almost what I want, but also won't give me a useful error message.

20:21 hiredman: lvh: the messages you get are a result of what you put in

20:21 lvh: hiredman: Sure, that's why I'm asking for suggestions on how to make it better :)

20:21 hiredman: if you are comparing two large structures using =, you get a single value back, true/false

20:22 if you compare different keys or positions or whatever by themselves you get more information (multiple values)

20:22 (is (= a b)) vs (is (= (get a :foo) (get b :foo))

20:23 lvh: ah; so just stuff multiple is'es into a testing block?

20:23 hiredman: that works for comparisons, but if you do some large calculation outside of the is, and then hand it to the is, all the is has is that value

20:24 amalloy: someone at geni wrote some kind of macro that converted (is (= x {:a 1 :b 2 :c 3})) into three different calls to is

20:24 hiredman: https://github.com/pjstadig/humane-test-output is also a thing (limited by the information you give to the is macro) you can also look at using are instead of is

20:24 amalloy: which was kinda handy

20:25 hiredman: amalloy: if I remembered the syntax to are, I think you could do that pretty quick

20:25 amalloy: yes

20:26 justin_smith: (is (= [nil nil] (take 2 (data/diff a b)))

20:26 hiredman: (are [x y z] (= (get y x) (get z x)) [:foo a b] [:bar a b] ...)

20:26 amalloy: justin_smith: why take 2?

20:26 justin_smith: amalloy: the third value is the items the two had in common

20:27 less likely to need to assert anything about that

20:28 unless you expected a specific difference I guess?

20:29 but of course the advantage there is that the failed test shows exactly the differences that are present but not expected

20:29 amalloy: the way 'are works makes me sad. the clojure.walk calls don't understand shadowing or quoting

20:30 i wish we could get macrolet as part of the compiler, rather than clojure.tools.macro needing to reinvent it

20:33 Morgawr: I'm trying to wrap my head around testing in clojurescript... I run "lein cljsbuild test" and it says 1 test was run containing 1 assertion (which is (is 1 2) to make it fail) which doesn't fail... and I need to ctrl+c the commad to kill it because it doesn't terminate

20:33 am I doing something wrong? (answer is very yes)

20:34 oh, I guess is needs (is (= 1 2))

20:35 now I only need to figure out how to make the command exit at the end...

20:36 scottj: Morgawr: maybe it is watching the files to rerun when they change?

20:38 Morgawr: scottj: weird, I just tried using rhino instead of phantomjs and it exits properly

20:38 with phantomjs it just hangs and waits

20:39 oh

20:39 I fixed it, I was missing :runner in the test command

20:39 now it works :)

21:24 vas: Hi I'm using datomic and trying to get relevant entities by supplying an author or eid ... but every time I query the db I get a :db.error/too-few-inputs ... https://www.refheap.com/98117

21:28 scottj: vas: I haven't used datomic, but I would guess ":in $ ?eid" is not what you want and is causing it to want an arg more than just the db connection, maybe?

21:28 vas: maybe you want :in [$ ?eid]

21:29 puredanger: that's correct

21:29 I mean it's correct that :in [$ ?eid] is going to expect another arg after the db

21:30 so pass eid at the end

21:32 owl-v-: hi

21:33 what's the math library I can use for matrix operations in clojure?

21:34 scottj: owl-v-: core.matrix afaik

21:35 and/or vectorz-clj

21:36 owl-v-: thanks

21:37 enn: If I want to use a local version of a (lein) project dependency, I can makes a checkouts directory. Is there any way to do something similar across projects, for a dependency that is defined in my profiles.clj?

21:48 gfredericks: enn: I can't tell how the second thing you described is different from the first thing

21:52 amalloy: it sounds like enn wants to have every project on the machine use a modified checkout

21:53 gfredericks: oooh I missed the profiles.clj part

21:53 amalloy: which is just not a good idea: publish the modified version, or at least install it locally, under a different groupid; then you can depend on it like a proper person

21:53 gfredericks: I read "project.clj"

22:27 enn: yeah, I guess it is time for me to figure out how to deal with maven locally

23:02 rufoa: I'm trying to turn two sequences [:k1 :k2 :k3] and [:v1 :v2 :v3] into a map {:k1 :v1 :k2 :v2 :k3 :v3}

23:02 is there a neater way than this? (into {} (mapv vector [:k1 :k2 :k3] [:v1 :v2 :v3]))

23:03 Wild_Cat`: rufoa: zip?

23:03 amalloy: zipmap

23:04 but also if you were going to do it your way, mapv is silly compared to just map

23:06 rufoa: ah i'd forgotten about zipmap! and you're right about mapv of course yes

23:06 thanks!

23:14 enn: Since str returns something like "clojure.lang.LazySeq@e93c3" for a lazy sequence, why does it realize the sequence?

23:26 amalloy: e93c3 is the hashcode, which is based on the sequence's items

23:31 enn: ah

23:37 nicola1: i am trying to figure out why my clojure program won't exit after the last statement; any pointers on how what to do to figure this out?

23:37 i put a println statement as the very last thing, it gets executed, so i'm looking for connections left open or things like that

23:38 godd2: nicola1 its possible that you're println'ing an infinite range?

23:38 nicola1: all i could find online is related to pmap/shutdown-agents, but i'm not using that

23:38 tomjack: kind of makes me sad when I need ~30G ram to chew a 1.6G file: https://www.refheap.com/e04a09f241c8c63bcd7b052ee

23:38 justin_smith: nicola1: shutdown-agents

23:38 godd2: I couldn't say without seeing the code

23:38 tomjack: wonder if there are obvious problems there besides, uh, using seqs

23:38 justin_smith: nicola1: if you have used futures, agents, etc. at all clojure won't shut down immediately unless you call shutdown-agents

23:38 tomjack: (whoops, s/soft/foo/ :( )

23:39 nicola1: i tried calling shutdown-agents at the end, didn't help :(

23:39 i'm basically ingesting data from avro files into cassandra

23:40 (let [connection ...] (doall (map #(ingest-into-cassandra connection %) filenames)))

23:40 justin_smith: tomjack: looks like a great use case for transducers rather than nested map / filter etc, if you can use 1.7

23:40 tomjack: yes, I just didn't want to write yet another reducers version of line-seq :(

23:41 justin_smith: nicola1: oh, I wasn't talking about pmap/shutdown-agents, just the regular shutdown-agents

23:41 tomjack: this run just finished, and luckily this box has 250G ram, but I'll try transducers next time

23:41 justin_smith: tomjack: you can keep the line seq part, but turn the rest into reducers, that will reduce your heap usage

23:41 tomjack: hmm. good idea

23:41 justin_smith: s/reducers/transducers that is, of course

23:41 nicola1: justin_smith: yes me too, i shouldn't have written "/" since it's clojure syntax - sorry about that

23:44 justin_smith: nicola1: ahh, yeah. shutdown-agents is needed not just for pmap, but if any code touches the pool, which includes futures, agents, pmap, reducers...

23:46 nicola1: justin_smith: i was just trying again to make sure, that's not it unfortunately

23:49 i'll keep poking, thanks for your help

23:50 tomjack: justin_smith: well, it ran ~75% faster this time, and htop didn't show my heap growing (though I don't trust that _too_ much)

23:50 thanks

23:51 15min vs 3min ish, big difference!

23:51 next time I'll try to dig up a lines reducer

23:52 justin_smith: or transducer

23:52 ahh, wait

23:52 75% faster with no change?

Logging service provided by n01se.net