#clojure log - Dec 03 2014

The Joy of Clojure
Main Clojure site
Google Group
List of all logged dates

0:00 kenrestivo: so (assoc feh :meh "stuff!") makes more sense, you'll get {:meh "stuff!" ...}

0:00 TEttinger: lazybot?

0:00 clojurebot: lazybot is a lazybut

0:00 kenrestivo: hahaha

0:00 justin_smith: kenrestivo: right, and the ->> macro is supplying that arg

0:00 kenrestivo: see my code above which was the subject of that exchange

0:00 TEttinger: you meant lasericus I hope, but he quit

0:00 kenrestivo: oh, well.

0:07 dviramontes: hi anyone know what the equivalent of swap is for om ?

0:07 swap!*

0:08 dnolen_: dviramontes: transact!, you can also swap! the atom, not you'll miss out on some things - may or may not matter future.

0:11 dviramontes: @dnolen_ thanks!

0:15 @dnolen_ pretty new to cljs / om. When i do om/transact! my updates stop working - when i switch back to swap! it works again. Could it be because the atom is too simple of a data structure ? meaning : (def app-state (atom {:value "text"})

0:16 dnolen_: dviramontes: how are you using transact! ? the signature is unfortunately different from swap!

0:16 dviramontes: ohhh that might be why

0:17 update function looks like this : (om/transact! app-state assoc :value data)

0:18 dnolen_: dviramontes: transact! doesn't take an atom, it takes a cursor into the app-state

0:18 dviramontes: I really recommend going through the tutorial, it will be much easier going

0:19 dviramontes: lol, yeah you're right. Thanks for the help!

1:27 kenrestivo: are there any special gotchas when making function calls outside of a go-loop?

1:54 andyf: I?ve not used core.async, but I thought all of the gotchas were inside of the go loops. Well, and that some core.async-specific calls can only be made inside of go loops

1:59 kenrestivo: it's been fascinating. i'm now considering using async chan so serialize access to a resource instead of using agents

2:00 s/so/to/

2:08 yeah, i'm making calls from inside a go-loop, to functions outside the go-loop, and i have a feeling this could end in tears.

2:33 dysfun: anything involving asynchronicity will usually end in tears. could be worse, you could be using a language that does it less well :)

3:01 Empperi: really?? https://github.com/vbauer/lein-jslint/blob/master/src/lein_jslint/core.clj#L71

3:01 default configurations override what you define?

3:01 man

3:03 rritoch: kenrestivo: I haven't yet run into an opportunity to use core.async but it seems that if your concerned about an external call it should be possible to setup a function which sets up an atom change listener which writes the change back to the channel and then launching your extern function in a "future" to write the results back to the atom. At least that is what I'd try.

3:04 kenrestivo: yeah using atoms/agents/listeners is what i'm used to. this async method is new to me.

3:05 i already see a few places where i might rip the async channels out and go back to agents/atoms. but others i'll keep.

3:08 rritoch: kenrestivo: In the "early-days" of asynchronous programming (LPC) long proceses (over 2 seconds) would automatically end in a fatal error so loops would need to monitor time and maintain state variables and a "call later" queue, so if a loop doesn't have time to finish it could resume later when the next "heartbeat" came. All IO would need to be no-blocking though.

3:09 kenrestivo: Now you can assume most machines have multiple processors, so sending blocking or potentially blocking code into it's own execution thread is fairly reasonable.

3:09 echo-area: rritoch: When was the "early-days"?

3:09 kenrestivo: core.async is only a year old :-)

3:09 ddellacosta: anyone know how to do a prepared statement in clojure.java.jdbc w/an in where clause?

3:10 like, where id in (1, 2, 3, 4) ...I would ideally not have to take my vector and turn it into a string, but if that's the only way...

3:10 rritoch: echo-area: This was about 20 years ago, but as far as I know LPC was the first "engine" that utilized asynchronous programming

3:13 http://www.mars.org/home/rob/docs/IntermediateLPC/chapter2.html#set_heart_beat

3:14 dysfun: ddellacosta: clojure.java.jdbc is a bit low level. are you sure you don't want something like korma?

3:14 ddellacosta: dysfun: I am using HoneySQL, which does not support the "create table ..." sql I want to use, unfortunately

3:14 dysfun: and I'm not going to use Korma just for this (or for any reason)

3:14 rritoch: That link is the core of how it was done back then and may provide some clues of easier ways to do things now. These engines were run on machines with less speed than todays smartphones, so any code that worked then, would be high-performance today.

3:15 ddellacosta: alas, I just ended up doing join and turning it into a string

3:15 hopefully that will scale...

3:16 probably will be fine

3:18 dysfun: "hopefully that will scale". why would joining strings not scale?

3:18 i mean it's hardly nice, but *shrug*

3:22 rritoch: Per the copyright notice that documentation was written exactly 21 years ago. I am not familiar with any asynchronous code before then, but that doesn't mean it doesn't exist.

3:51 andyf: rritoch: if one?s definition of asynchronous code includes the kinds of programs written in Erlang, it appears early experiments in Erlang go back to the late 1980s.

3:58 rritoch: andyf: Until you mentioned it I've never heard of Erlang, but from what I can see Erlang was used in a lot of proprietary systems. It would be helpful code if any of it has been open sourced. All of LPC was open source though so it is a better refrence. The developers of LPC forbid it's use via the copyright for business purposes so much of the code written in it was published open-source.

3:59 whodidthis: im sure yesql supports even create table, it's 100% goodtimes

3:59 andyf: There are open source Erlang implementations and significant libraries, but still closed-source applications developed using it, as there are closed-source applications developed in most languages.

4:06 fairuz: Hi guys

4:07 andyf: howdy

4:10 fairuz: I have (into [] (map X list-A)). But I need to do dissoc to the result of X. How can I do this?

4:11 rritoch: andyf: I found a decent article comparing Erlang to Go, core.async is based on Go. http://joneisen.tumblr.com/post/38188396218/concurrency-models-go-vs-erlang , As far as I can tell it appears that LPC used functionality more similar to Go as LPC only had one thread, and no virtual threadding.

4:12 andyf: fairuz: You could do (into [] (map (fn [o] (dissoc (X o) :key-to-dissoc)) list-A))

4:12 Or some people prefer using comp to compose two functions and create a new one, e.g.

4:12 razum2um1: is clojure hyphen(-) is preffered to name variables, but in database likely have convention to use underscores - how do you stick with it? (e.g. for example in ruby-land there are usually converters from camelCase in xml to underscores as readability matters more than speed)

4:13 andyf: (into [] (comp #(dissoc % :key-to-dissoc) X) list-A))

4:13 scottj: razum2um1: https://github.com/qerub/camel-snake-kebab maybe

4:14 fairuz: andyf oh yeah. I'll go to the former first

4:14 the second is still a bit alien to me :)

4:15 andyf: The comp form makes me stop and think a little longer, since it still isn't natural for me after seeing it many times.

4:24 razum2um1: scottj: thanks

4:28 kral: namaste

4:46 rritoch: andyf: It took me a good amount of time to find asynchronous examples in erlang. Erlang is multiprocessing so asynchronous patterns would only be used for optimizations. http://krondo.com/?p=2692 , As far as I can tell Erlang asynchronous development was identical to what I suggested to kenrestivo, launching separate processes from the asynchronous code.

4:57 clgv: how do you create a single instance of a typed-struct with vertigo?

5:03 rritoch: Does rich hickey (the developer of clojure) ever come to this channel? I don't believe I've ever seen him here

5:03 SagiCZ1: rritoch: he used to

5:03 clgv: rritoch: very seldom, haven seen him lately

5:03 SagiCZ1: if you checked out very early ~lazy-logs you can see he was pretty active

5:03 clgv: $seen rhickey

5:03 lazybot: rhickey was last seen quitting 76 weeks and 1 day ago.

5:03 rritoch: Hmm

5:04 SagiCZ1: what drove him away? ... guys?

5:04 clgv: being busy? ;)

5:04 wink: I wouldn't call him "the developer" though. creator, yes :P

5:05 rritoch: Well, I did a lot of research into this problem of running multiple runtimes. I discovered that my OSGi code wasn't creating multiple runtimes no matter what I did since the "import" functionality of OSGi leads to any refrences to RT being resolved by the same classloader

5:05 SagiCZ1: wink: so do we believe in Clojure creationism?

5:05 wink: SagiCZ1: that's a good question. afaik it didn't evolve from another language ;)

5:06 rritoch: I found some solutions to the issue, all of which involved making custom classloaders based on functionality in clojure's core that isn't guaranteed to persist into future versions

5:06 clgv: rritoch: you could use an approach similar to classlojure but be aware that there might be some issues with memory leaks left (maybe they got all fixe dwith 1.6, didnt check)

5:06 SagiCZ1: wink: no? i thought it was heavily inspired by lisp

5:06 clgv: SagiCZ1: lol, but it was created, right?

5:06 rritoch: Yes, I see everyone ends up with probelems when it comes to data sharing

5:07 wink: SagiCZ1: I think you should read up on genetics ;)

5:07 clgv: rritoch: does OSGi support external processes?

5:07 SagiCZ1: wink: it's a hobby of mine, so i read quite a lot.. i am also fascinated by evolutionary algorithms and genetic programming

5:07 clgv: rritoch: having the different clojure instances as external processes might be beneficial performance wise as well

5:08 rritoch: clgv: I suppose, but only in the ways that the Java runtime does, by using Pipes, or sockets.

5:09 Anyhow, as far as I can tell the need for multiple runtimes boils down to one key feature of clojure, the clojure.lang.Namespace object

5:09 wink: SagiCZ1: just saying that if you want to have that limping analogy, I think it needs to be a fork or at least share common roots, not ideas.

5:09 SagiCZ1: wink: i see what you mean, yeah that makes sense

5:10 clgv: rritoch: you cant have different versions of clojure within the same classloader neither

5:10 rritoch: I'm not sure what the "cost" would be, but if clojure.lang.Namespace was refrenced as an object, and the object associated to the threads via a dynamic variable, I believe it would become possible to have multiple runtimes (each with their own namespace) that would still be able to share data without any problems.

5:11 clgv: rritoch: sharing data is not a big issue. you can easily share through edn serialization

5:11 rritoch: This would avoid any classloader trickery, and create multiple (virtual) runtimes, each with their own namespace.

5:11 clgv: rritoch: I am not convinced how this should work

5:12 rritoch: clgv: Basically, every function in clojure, such as future, which starts a new thread, would "copy" the namespace object refrence into the new thread.

5:13 clgv: So starting a new runtime would simply require creating a new namespace object, and replacing the current namespace object refrence with the new one, and any threads spawned from that thread would then use the new namespace object.

5:14 clgv: As far as I can tell, vars can have a default value, so any threads without a namespace object refrence could fallback to a root (master) namespace.

5:15 So to get to the point, even if I were to make something like that it would "fork" clojure which is why I'd like to bring up the idea to rich hickey and see what he thinks about it.

5:15 clgv: rritoch: you are talking of one "namespace" object all the time? do you mean something like a "namespace registration" that holds all defined clojure namespaces?

5:16 rritoch: clgv: Yes

5:16 SagiCZ1: rritoch: i think there are ways to reach him with such idea

5:16 rritoch: clgv: Currently namespace registration is implemented via static calls to the namespace class.

5:16 clgv: rritoch: but you wont be able to support different clojure versions anyway, because of the java implementation part

5:18 rritoch: clgv: That is true, it would only have one version of clojure available, but you could still have multiple "namespaces" available to provide multiple runtime environments (of the same version).

5:19 clgv: Having multiple versions of clojure communicating with eachother directly would always lead to complications anyhow

5:19 clgv: I'm simply looking for a way to isolate modules so they don't have conflicts with namespaces from other modules.

5:20 clgv: rritoch: I think if the classloader apporach does not suite you, the multiple process approach using EDN to pass data is the simplest/easiest

5:20 rritoch: immutant uses the classloader approach for that

5:21 rritoch: clgv: My only problem with the classloader approach is that there's no guarantee of compatiblility with future versions of clojure, and eventually the classloader would end up filled with logic controls based on the clojure version being loaded.

5:21 clgv: rritoch: why should that be?

5:23 rritoch: have a look at how immutant does it - afair there is no clojure version specific stuff

5:23 rritoch: except from maybe cleaning up memory leaks of previous clojure versions

5:24 rritoch: clgv: immutant uses shim, I looked into that already, and am considering that as a solution but all shim does is juggle the classloaders

5:25 clgv: rritoch: yes, but independently of which clojure version is used

5:25 rritoch: clgv: This creates a lot of overhead since each runtime ads 1 second overhead, and you end up with multiple copies of the entire clojure runtime in memory. I believe the footprint of simply having separate namespace registries would be MUCH smaller.

5:27 clgv: rritoch: but is pretty unlikely to be introduced anytime soon

5:27 rritoch: clgv: As of now I took the easy way out, and avoided the issue all-together. I'm maintaining a registry of classloaders for each module, and switching to the appropriate classloader manually each time a call needs to cross a module boundary.

5:27 clgv: rritoch: except you fork clojure and do it yourself

5:27 with all the problems associated to that

5:28 rritoch: you probably need security managers as well, right?

5:28 rritoch: clgv: Exactly why I'd like to talk to Rich, if I could establish a fork which does this without any new overhead, would he be willing to merge it into the language.

5:30 clgv: rritoch: you could ask on the mailing lists as well. lots of clojure devs read it

5:31 rritoch: clgv: I'm not sure about the security managers. I haven't yet run into a situation where I needed to use one.

5:32 clgv: OSGi provides a list of all modules, and I'm registering the namespace objects of the modules as services so the process is to use OSGi to locate the appropriate model, switch to the classloader registered for that module, and call the functions in that registered namespace that need to be called.

5:33 clgv: With this structure though, any module can manipulate namespaces from other modules as soon as those namespaces have been loaded, which is much less than ideal.

5:34 clgv: Namespace collissions between modules would be catostrophic.

5:37 clgv: My main application is exporting clojure, so no modules need to contain their own runtime, reducing load times significantly, but I realized that because of this the bundle classloader is falling back to the application classloader, so using a shim alone woudln't solve the problem. I would also need a custom classloader to take ownership of all classes required by the clojure RT & Compile

5:38 fairuz: hye guys. I have something like this. (if-not (nil? A) (B C D) (B X D)). Can I simplify this better? since the only difference is the C and X part

5:38 on nvm

5:38 andyf: fairuz: You can also use (B (if-not (nil? A) C X) D)

5:38 fairuz: I can just put the test inside

5:39 yeah andyf. sorry

5:39 rritoch: From what I've seen so far, assuming you don't need multiple versions of clojure, the most effecient means of managing it would be to change how namespaces are registered. Currently they're registered as static calls to the clojure.lang.Namespace classs.

5:41 andyf: rritoch: The problem you are trying to solve is making it more memory-efficient to run Clojure code that uses conflicting versions of common namespaces?

5:41 rritoch: Technically switching environments could be achieved via a simple hack of clojure.lang.Namespace, removing "final" from the declaration of the namespaces static property.

5:42 andyf: Yes, basicaly, but I would also want modules to have all of their namespaces hidden from other modules, other than those which hey specifically decide to export.

5:43 Either way, simply removing final from clojure.lang.Namespace would still require juggling that object so it seems using thread-local memory for the namespace registrations makes the most sense.

5:47 Removing the final from the namespaces property also creates the potential of making an extremly unstable environment, since any module and/or process could freely change it. Containing changes of the registrations to thread-local memory is what makes the most sense to me.

5:58 Where is clojures official mailing list?

6:00 hyPiRion: https://groups.google.com/forum/#!forum/clojure

6:00 rritoch: hyPiRion: Thanks :)

6:00 hyPiRion: np =)

6:24 clgv: switching from strings to keywords was a great solution to save memory thanks to interning

6:24 and since keywords are fine for all the associated use cases

6:26 rritoch: clgv: What about keywords with spaces? That is a common case where I've had problems with keywords and http forms (clj-http)

6:29 clgv: What I find odd is that clojure happily makes keywords out of them, But other than using ##(keyword "My Keyword") in each usage there doesn't seem to be any way to properly handle them.

6:29 lazybot: ⇒ :My Keyword

6:30 clgv: rritoch: there are no spaces in this scenario. otherwise I would run into trouble with further analysis

6:30 rritoch: clgv: Here is where If ind the problem....

6:30 ,(read-string (pr-str (keyword "My Keyword")))

6:30 clojurebot: :My

6:30 clgv: rritoch: the missing checks in the `keyword` function are very likely a performance tradeoff

6:32 rritoch: clgv: Well, either way, I didn't realize keywords were more memory efficient, I just use them because my boss prefers them, but I constantly run into problems when there are spaces or reserved symbols invovled

6:32 ,(keyword "WebArray[]")

6:32 clojurebot: :WebArray[]

6:33 clgv: rritoch: they use interning and hence each keyword exists only as one instance

6:34 rritoch: you can achieve the same via fly-weight pattern, but then you have to implement that manually - though it is not that had but it is an additional effort -

6:34 *hard

6:49 rritoch: clgv: That would be a good solution to memory management issues. I'm just not a big fan of how keywords deal with various characters. Persionally I think a reversable process of generating keywords would have been better, such as URLEncoding the string, replacing all underscores with %5F, and then replacing all % with underscores.

6:51 clgv: I've avoided that because I didn't see any advantage to it, but I recently became aware of the interning, and since there is a memory advantage to it I think I may need to make my own wrapper to the keyword function to provide more complete coverage of all possible strings, since it provides a memory management benefit.

6:51 clgv: Unfortunatly it still won't help me much when interfacing with third party libraries.

6:53 I ran into interning when trying to figure out ways to produce multiple runtimes environments

6:56 clgv: rritoch: well the keywords with spaces and similar are not valid keywords according to the "spec"

6:57 rritoch: `keyword` just doesn't fail for them

6:57 rritoch: clgv: Well, now that you mention it, a lot of the articles I saw about using classloading tricks to get multiple runtimes running, they ended up having problems with the keywords. One went as far as forking clojure to resolve the issue.

7:23 clgv: rritoch: which keyword problems did they have?

7:26 rritoch: clgv: I can't find the original article about it, but I found where they posted the fork to the mailing list https://groups.google.com/forum/#!topic/clojure/0AgUIiY8BQ8

7:26 clgv: I'm not sure if that has been merged in or not

7:29 Ok, that is the article, but you have to expand the original email which explains the problems with keywords

7:30 clgv: rritoch: well, there is one point in the beginning you do not have to share the data structure implementations or their interfaces to share data - as long as the overhead of serialization to EDN (binary with fressian) is acceptable

7:33 rritoch: if really large amounts of data need to be shared, you probably need a database service anyway, right?

7:35 rritoch: clgv: I suppose that depends, a common case, for me at least, would be web based chat system, if for example it supported file transfers, the file transfers may be provided by its own module in a separate runtime.

7:36 clgv: It would be more efficient to stream the data between the users than to cache it in the filesystem or a database.

7:44 clgv: In this case keywords would often be used as chat commands, and the user commands would need to be understood by any module processing the transaction. After seeing these issues though it still seems that having separate namespace registrations would be more efficient, even if it does require some hacking into clojures core.

7:45 clgv: I'm also now considering the possiblity of simply extending clojure to provide those features, a solution similar to the shim, accept taking over the operations of all core classes and modifying clojure.core to use the new class names.

7:47 clgv: Ideally no clojure apps should be directly manipulating clojure core classes, so technically it should work.

7:49 clgv: In theory at least, I for one have code which manipulates clojure core classes to squeeze out new functionality, but that code was never expected to be upgrade-safe.

7:50 clgv: rritoch: as long as the big effort pays off... I guess I'd stop at this point

7:50 rritoch: clgv: If I was in it for the money I would have quit a long time ago :)

7:50 justin_smith: rritoch: wouldn't using strings instead of keywords save a lot of problems?

7:51 clgv: rritoch: ah so this is hobby project?

7:52 rritoch: clgv: Not exactly, I hope to eventually be able to use this for production environments, and clients

7:52 clgv: But there's no real loss if it's not successful, since I can always go with alternative platforms, such as ruby

7:55 clgv: rritoch: when you are working with the namespaces you could take care of transactional loading of namespaces - which clojure does not

7:56 rritoch: just in case you need to require namespace dynamically at runtime from more then one thread ;)

7:59 rritoch: clgv: Hmm, so you want namespaces to automatically uninstall after the transaction completes?

7:59 clgv: rritoch: no. just that they can be loaded safely ;)

8:00 rritoch: the process of loading clojure namespaces is not thread safe

8:01 rritoch: clgv: I see, I've never run into a problem with that, but I always use on-demand namespace loading, I first check find-ns before using require in cases of runtime namespace loading.

8:02 justin_smith: rritoch: require already checks if a namespace is loaded, and that method is still vulnerable to race conditions

8:04 joegallo: yeah, iirc, the ns exists pretty early in the require process, but before all the code has necessarily been read and loaded into it

8:05 yguan: hi genius. is there a parallel filter like pmap?

8:05 joegallo: so thread a could still be loading code into the relatively empty ns, while thread b is off to the races trying to execute against vars that aren't there...

8:07 vijaykiran: yguan: http://clojure.org/reducers

8:08 yguan: vijaykiran: thanks!

8:08 vijaykiran: yguan: I think more appropriate question would be why do you want a parallel filter ?

8:10 yguan: vijaykiran: i assume if each test is indepedent to each other, they can do the job in parallel. plus if i don't care the order but only the value returns true?

8:10 justin_smith: vijaykiran: it would make sense if you have a massive sequence of items, and only want to keep a small percentage of them

8:11 otherwise the coordination of threading will be more expensive than the gains of going parallel

8:13 vijaykiran: justin_smith: yes, that's why I was more interested in the problem yguan is solving

8:15 rritoch: There are multiple ways to load namespaces though, some go through RT some through Compiler so I'm not entirely sure there would be a thread-safe way of loading namespaces and still maintain clojure as dynamic and relatively decent speed.

8:15 yguan: vijaykiran: i'm given a vec as pixel of image, try to apply a function to find pixels with interesting feature.

8:17 rritoch: You can also have various threads doing crazy things, such as constructing namespaces at runtime using in-ns.

8:17 clgv: rritoch: that wont safe you as far as I know

8:18 vijaykiran: yguan: looks like it fits into "when to use" on the reducers page :)

8:19 clgv: rritoch: the problem is, that a namespace is found or is marked as existing as soon as the ns-declaration is finished. but at this point its variables do not necessarily exist or do not necessarily refer to the functions they are supposed to

8:20 rritoch: clgv: But that will always be the case since you can always add and remove symbols to/from the namespace at runtime

8:20 yguan: vijaykiran: ok, first two checked, i'm not sure around 2000000 pixel is "large"...

8:21 rritoch: clgv: It should be the namespace-user's responsiblity to ensure the symbols they need are available.

8:21 clgv: rritoch: sure. but you'd expect to be able use the functions defined in that namespace file after `require` returns - which is not the case when loading dynamically in parallel

8:22 rritoch: clgv: So basically you want a version of require that hangs, possibly forever, until the symbols it needs are available?

8:23 clgv: ex. (require 'some-ns ['sym1 'sym2 ...])

8:25 clgv: rritoch: no.

8:26 rritoch: when you use require to load a namspace dynamically only from one thread, you have the guarantee that as soon as require returns you can use all variables of the namespace

8:27 rritoch: and this is not true anymore when you do it in parallel. I have a small lib that fixes that already

8:27 rritoch: clgv: Than don't use require, use load

8:28 clgv: rritoch: that is no solution

8:28 rritoch: then I get different class incarnations with the same name for deftypes and defrecords

8:29 rritoch: it's true that there are not that many use cases for loading namespaces dynamically in parallel - but I had one...

8:33 rritoch: clgv: Well, no matter how you slice it, clojure namespaces are dynamic so they won't be thread-safe. Verifying the existance of the symbols you need I believe is the only sure-fire solution. Using require without :reload-all doesn't provide any guarantees

8:33 clgv: A in-ns statement, before a require, should be enough to ensure that the symbols you want never get loaded.

8:34 justin_smith: verifying the existence doesn't suffice if they use declare

8:35 clgv: rritoch: in my scenario the namespaces to load are all static. there is nothing added or removed at runtime

8:35 rritoch: clgv: Do you have control over the source code that is being loaded?

8:36 clgv: rritoch: yes. it's deployed as jar

8:36 rritoch: well, the problem is solved by making require transactional

8:36 rritoch: clgv: If so I suppose you could alter the metadata as the last part of the source file, then after your require enter an infinite loop that waits for that metadata to be available.

8:37 clgv: no need to

8:41 rritoch: clgv: Wouldn't making require transactional drastically increase the load time of any application with a lot of dependencies?

8:43 clgv: rritoch: with the my current fix there is is only a small overhead if only one thread is loading these namespace. I should probably make a benchmark for it

9:47 justin_smith: $ping

9:47 lazybot: justin_smith: Ping completed in 0 seconds.

9:49 otfrom: I'm trying to use flambo 0.3.3 and 0.4.0-SNAPSHOT and I get the following error: #<CompilerException java.lang.ClassNotFoundException: flambo.function.Function, compiling:(flambo/function.clj:56:1)> works fine in 0.3.2. Any flambo/spark people here today with an example project using > 0.3.3?

9:52 clgv: $ping justin_smith

9:53 otfrom: you are trying to use both in the same project?

9:54 otfrom: clgv: more than one version of flambo? No, just one, but I'd like to use the latest release or the latest snapshot if possible.

9:54 clgv: are you thinking I have a classpath clash problem?

9:56 clgv: otfrom: you can check via "lein deps :tree"

9:57 craigglennie: Good morning all. I have a function that takes a URL, downloads a file to disk, and returns the path to the file. Is it okay (in terms of best practices / style) to use a function with side-effects like this one in a “let”?

9:58 clgv: otfrom: it is using gen-class to create flambo.function.Function - so you need to AOT flambo.function

9:58 justin_smith: craigglennie: absolutely fine. let is a procedural construct.

9:59 otfrom: clgv: only clashes are with commons-codec

9:59 craigglennie: justin_smith: Cool, thanks

9:59 clgv: otfrom: see above

9:59 otfrom: so in my client lib I need to aot the namespace with the main in my project and flambo.function?

9:59 * otfrom always gets bitten by aot

10:00 clgv: otfrom: complain to the devs ;)

10:00 otfrom: lol

10:00 I believe the appropriate response to my complaint would be: submit a PR. ;-)

10:00 noncom|2: what is the best library for xml for clojure ?

10:01 i find that clojure.data.xml can't parse/unparse xml elements as clojure datastructures.. is there any lib that does that ?

10:02 justin_smith: noncom|2: I have never seen clojure.data.xml output anything but clojure datastructures. Maybe I misunderstand you.

10:02 clgv: otfrom: https://github.com/yieldbot/flambo#aot

10:03 noncom|2: justin_smith: ymm, i guess you're right.. i mixed things up, it really does :)

10:03 otfrom: clgv: giving that a try now

10:07 clgv: that did it. Thanks!

10:07 * otfrom goes off to try 0.4.0-SNAPSHOT after 0.3.3

10:07 mumrah: how can i point leiningen at a local jar?

10:07 or alternatively, how can i change the location of maven's ~/.m2 dir?

10:08 justin_smith: mumrah: easiest is lein install

10:08 mumrah: lein install is akin to mvn install, yes?

10:08 justin_smith: mumrah: or you could explicitly add the jar's location to the classpath via project.clj

10:08 mumrah: yeah, it will put the artifact into .m2, and can generate a pom for it if needed

10:09 mumrah: justin_smith: have an example of adding a jar to the classpath handy?

10:09 that sounds like my ticket

10:10 * otfrom thinks he should add "or it might be an aot problem" to "It is always a classpath problem"

10:11 * otfrom is very happy that clgv got him going all the way up to flambo 0.4.0-SNAPSHOT. Thx! (❁´◡`❁)

10:11 justin_smith: mumrah: you should be able to add the full path to the jar to the :source-paths vector

10:11 mavbozo: leiningen generates command like "/usr/bin/java -classpath :/Users/mavbozo/.lein/..."

10:11 how can i see the generated commands?

10:12 justin_smith: mavbozo: what OS? with linux/osx you can see the full command line via ps x, or you can just see the classpath generated with lein cp

10:13 mavbozo: justin_smith: windows, linux, osx

10:13 justin_smith: no leiningen command to debug?

10:13 justin_smith: mavbozo: "lein cp > class_path; java -cp $(cat class_path) clojure.main" is a fun startup time hack

10:14 to just see the classpath there is "lein cp" but thanks to plugins especiallly lein can do a lot more than just create a classpath

10:14 there is also :eval-in pprint

10:14 mavbozo: justin_smith: i want to see the full 'java -jar ... ' command

10:14 justin_smith: mavbozo: in my experience the full command does not use -jar at all

10:15 it is java -cp [classpath] main-class

10:15 where that classpath has many jars, of course

10:17 and main class will be clojure.main -m your-main-ns

10:17 mavbozo: the problem is, i get lots of "Unable to resolve symbol" or "Class not found" in some of my projects in windows

10:18 justin_smith: mavbozo: I am sure there is some way to look at the command line of a running process on windows. I just down't know what it is, I haven't had a windows machine since win98

10:18 pepijndevos: How do I write fast and working Clojure these days? (debugging, profiling, etc.)

10:18 justin_smith: *don't

10:18 mavbozo: i suspect those problem happens because the command generated by leiningen exceeds windows cmd max string length

10:18 mikerod: ,(clojure.reflect/reflect (fn []))

10:18 clojurebot: #<ClassNotFoundException java.lang.ClassNotFoundException: clojure.reflect>

10:18 mikerod: I found this error interesting

10:19 If I do (very ugly in one line):

10:19 ,(let [f (fn []) r (clojure.reflect/->JavaReflector (.getClassLoader (class f)))] (refl/do-reflect r (class f)))

10:19 clojurebot: #<CompilerException java.lang.ClassNotFoundException: clojure.reflect, compiling:(NO_SOURCE_PATH:0:0)>

10:19 noncom|2: justin_smith: what is the recommended way to decend a parsed xml tree in data.xml ? i find that on every node i have to ask for the :tag before going down inside the content ?

10:19 mikerod: oh boo

10:19 no clojure.reflect loaded

10:19 mavbozo: because when I move the project directory to C:\ (shorter path name), the problem gone

10:19 justin_smith: pepijndevos: jvisualvm works good for profiling, the cursive plugin for intellij idea probably has the most complete debugging / profiling integration in the environment

10:19 puredanger: pepijndevos: I've been pretty happy with Cursive for debugging lately. I use YourKit and Criterium for profiling (depending what I'm doing)

10:20 justin_smith: &(require '[clojure.reflect :as reflect])

10:20 lazybot: ⇒ nil

10:20 noncom|2: justin_smith: also, while trying to traverse the tree of elements, i always get ClassCastException clojure.data.xml.Element cannot be cast to java.util.Map$Entry

10:20 so the returned structure is *not* a native clojure data structure

10:20 mikerod: So to demonstrate without clojurebot help: `(clojure.reflect/reflect (class (fn [])))` ;= ClassNotFoundException <some-ns-here>$eval130048$fn__130049 java.net.URLClassLoader$1.run (URLClassLoader.java:202)

10:21 So the issue is that clojure.reflect/reflect defaults to using the context class loader

10:21 justin_smith: noncom|2: is that in a clojure.walk/postwalk call?

10:21 pepijndevos: Do people actually pay hundreds of dollars for YourKit?

10:21 justin_smith: mikerod: I just demonstrated how to load clojure.reflect above btw

10:21 puredanger: pepijndevos: I have, several times. It's worth it.

10:21 noncom|2: justin_smith: no.. actually i just try to replace one tiny value in an xml file

10:21 mikerod: However, if I make my own JavaReflector: `(let [f (fn []) ] (clojure.reflect/->JavaReflector (.getClassLoader (class f))))`

10:21 justin_smith: pepijndevos: I think most of us use the open source license

10:22 mavbozo: so, I suspect if the command line generated by leiningen exceeds OS's command length limitation, the leiningen command fails to run

10:22 puredanger: pepijndevos: actually, jvisualvm is pretty good too for basic method profiling

10:22 noncom|2: justin_smith: there is no similar problems with json, through cheshire.. it just goes very natural.. however, this xml thing is strange

10:22 justin_smith: noncom|2: OK. I never ran into that because I was just reading from the xml, not trying to modify it at all.

10:22 mikerod: I can use (clojure.reflect/do-reflect r (class f)) to get some reflection.

10:23 justin_smith: I didn't see you post about that

10:23 justin_smith: ,(require '[clojure.reflect :as reflect])

10:23 clojurebot: nil

10:23 pepijndevos: I can't claim any of my clojure things have an active community...

10:23 mikerod: justin_smith: I didn't know we could use something like `require` here

10:23 ,(reflect/reflect 1)

10:23 clojurebot: {:bases #{java.lang.Comparable java.lang.Number}, :flags #{:public :final}, :members #{#clojure.reflect.Method{:name toOctalString, :return-type java.lang.String, :declaring-class java.lang.Long, :parameter-types [long], :exception-types [], ...} #clojure.reflect.Method{:name compareTo, :return-type int, :declaring-class java.lang.Long, :parameter-types [java.lang.Object], :exception-types [], ......

10:24 mikerod: nice

10:24 noncom|2: justin_smith: alright.. well, clojure.data.xml.Element is a (defrecord).. aren't they supposed to behave like {}s when being queried?

10:24 justin_smith: pepijndevos: well, jvisualvm works (I am sure it is not as fancy as yourkit though)

10:24 mikerod: ,(reflect/reflect (class (fn [])))

10:24 clojurebot: #<ClassNotFoundException java.lang.ClassNotFoundException: sandbox$eval120$fn__121>

10:24 pepijndevos: ok :(

10:24 mikerod: ,(let [f (fn []) r (clojure.reflect/->JavaReflector (.getClassLoader (class f)))] (refl/do-reflect r (class f)))

10:24 clojurebot: #<CompilerException java.lang.RuntimeException: No such namespace: refl, compiling:(NO_SOURCE_PATH:0:0)>

10:24 justin_smith: noncom|2: yes, they should, what were you doing to it that provoked that error?

10:24 mikerod: ,(let [f (fn []) r (reflect/->JavaReflector (.getClassLoader (class f)))] (reflect/do-reflect r (class f)))

10:24 clojurebot: {:bases #{clojure.lang.AFunction}, :flags #{:public :final}, :members #{#clojure.reflect.Method{:name invoke, :return-type java.lang.Object, :declaring-class sandbox$eval171$f__172, :parameter-types [], :exception-types [], ...} #clojure.reflect.Constructor{:name sandbox$eval171$f__172, :declaring-class sandbox$eval171$f__172, :parameter-types [], :exception-types [], :flags #{:public}}}}

10:24 noncom|2: justin_smith: i did (keys) on it

10:25 justin_smith: also (:key element) always returns nil

10:25 justin_smith: noncom|2: yeah, I would think keys would work on a defrecord

10:25 xeqi: pepijndevos: I had a company buy a license and used it many years ago. Saved a bunch of time in that case

10:25 justin_smith: noncom|2: well, you want (key element) not (:key element)

10:25 if the element is a hashmap entry

10:25 xeqi: today I'd try jvisualvm as a first step though

10:26 noncom|2: from the source: (defrecord Element [tag attrs content])

10:26 pepijndevos: xeqi, yea, I assume that once you make serious money using the JVM, yourkit can make you some extra.

10:26 noncom|2: so i try (:tag my-element)

10:26 and it returns nil

10:26 justin_smith: xeqi: pepijndevos: yeah, seems like it wouldn't be hard to convince your company that better profiling would pay off fast

10:26 noncom|2: whatever i try, it is always nil

10:26 and (keys my-element) just fails

10:26 puredanger: are you sure my-element is actually an Element ?

10:27 pepijndevos: At the moment I have not much comany to speak of.

10:28 noncom|2: puredanger: wow! (type my-element) returns lazy-seq!

10:28 justin_smith: noncom|2: hah, so it was a different datatype than you expected

10:29 puredanger: so it's probably a sequence of Element or something like that

10:29 noncom|2: but (keys my-element) says ClassCastException clojure.data.xml.Element cannot be cast to java.util.Map$Entry

10:29 aaaaaaaaand the source says "(defrecord Element [tag attrs content])"

10:29 so, where is the truth out there? :)

10:29 puredanger: keys expects a seqable thing (a sequence of MapEntry)

10:29 Bronsa: ,(keys (1))

10:30 clojurebot: #<ClassCastException java.lang.ClassCastException: java.lang.Long cannot be cast to clojure.lang.IFn>

10:30 justin_smith: noncom|2: here is my universal trick - in a repl I start with (-> unknown type) then based on type I up-arrow and do (-> unknown keys) or (-> unknown first), then I do (-> unknown :first-key type) etc. etc.

10:30 noncom|2: that way I typically find what I am looking for very fast

10:30 noncom|2: justin_smith: yea, i just basically just fololowed in your steps a couple of secs ago..

10:30 justin_smith: it works!

10:30 noncom|2: for some reason it (first) returns the element

10:30 oh well..

10:39 pepijndevos: When do you use reducers or transducers?

10:40 puredanger: reducers are particularly useful if you want to parallel fine-grained data transformation

10:41 transducers can really be used anywhere you use sequences today but offer better performance in many cases, eager evaluation (with optional lazy evaluation via sequence), clearer scope for resources, and the ability to stay "in collections" (vector -> vector, etc)

10:42 eventually there will be a parallel story with transducers too, most likely in 1.8

10:46 pepijndevos: great.

10:46 visualvm just crashes when profiling my code.

10:47 justin_smith: weird, I have never seen that

10:47 is it the same visualvm version as the jvm version you are using?

10:48 pepijndevos: I don't know... just downloaded it from their site

10:48 justin_smith: pepijndevos: jvisualvm ships with the jdk

10:48 use the same one that came with the jdk that you are using

10:48 pepijndevos: huh.. so where is it?

10:49 (mac)

10:49 justin_smith: pepijndevos: depends, where did your OS put your jdk install?

10:50 pepijndevos: in /usr/bin/java :D no idea where the actual classes and things live

10:50 justin_smith: hmm, if you have the jdk via a standard method on the mac, jvisualvm should end up in your path in a shell (regardless of where it put the actual files)

10:50 pepijndevos: ah, it does

10:52 okay, this works so far

10:53 uuuhm, nope, hanging again...

10:56 Glenjamin: the jvisualvm you can download is a bit newer/shinier than the bundled one though

11:00 justin_smith: Glenjamin: I was just guessing that the one that comes with your vm would be less likely to hang or crash

11:20 LizardSquad: You Guys Ready to rumble

11:20 Lizard Squad Will strike today

11:20 And all of you will suffer

11:21 sd

11:21 sd

11:21 sdsd

11:21 d

11:21 sd

11:21 s

11:21 asd

11:21 sa

11:21 sd

11:21 sd

11:21 asd

11:21 ds

11:21 sd

11:21 das

11:21 sd

11:21 ds

11:21 sd

11:21 sd

11:21 d

11:21 sds

11:21 adsa

11:21 sd

11:21 sd

11:21 s

11:21 ds

11:21 dsssssssssssssssssss

11:21 sd

11:21 sdsdd

11:21 dsasd

11:21 asd

11:21 asd

11:21 sda

11:22 jonathanj: is there a clojure.java.jdbc command to generate "CREATE INDEX" ddl?

11:23 justin_smith: jonathanj: definitely. One minute I'll see if I can look it up.

11:24 jonathanj: if nothing else you could create the statement freeform and put it in a db-do-commands call

11:25 jonathanj: i found this http://dev.clojure.org/jira/browse/JDBC-62 but it doesn't seem to correlate with reaility

11:25 benmoss: yeah, was looking at that too

11:25 jonathanj: (or that reality has since dissolved)

11:26 justin_smith: then i have to manually figure out quoting and stuff, is there a way i can use ? placeholders and specify the values literally?

11:26 justin_smith: jonathanj: so you are doing ddl based on user input?

11:26 benmoss: ah, https://github.com/seancorfield/jsql is what you ant

11:26 https://github.com/seancorfield/jsql/blob/master/src/java_jdbc/ddl.clj#L54

11:27 jonathanj: justin_smith: not in this case, no, but i don't see any reason to assume i will never make a mistake

11:28 benmoss: thanks, that looks useful

11:30 benmoss: no prob

11:30 otfrom: anyone doing any pre-clojurex gettogethers tonight?

11:31 pepijndevos: I'm failing at transducers... https://www.refheap.com/94360

11:33 justin_smith: pepijndevos: comp on transducers is backwards

11:33 puredanger: also why the eduction in there?

11:34 justin_smith: try (comp words numbers node-group)

11:48 pepijndevos: justin_smith, ahhhh, why on earth... well

11:48 gdeer81: trying to get clojure introduced at work by doing all the testing in Clojure. gradle is killing my efforts. I can't seem to find a mapping of which versions of gradle are compatible with clojuresque 1.6

11:49 jonathanj: hrm, so clojure.java.jdbc/db-do-commands wraps each command in a transaction by default

11:49 SagiCZ1: gdeer81: is testing non-clojure code in clojure easier?

11:50 puredanger: pepijndevos: transducers are a stack so you compose them inside-out

11:50 pepijndevos: puredanger, not sure about the eduction, but I map a mapping function, because it's a seq of seqs.

11:51 gdeer81: SagiCZ1, testing java with clojure is pretty fun

11:51 puredanger: pepijndevos: I get that, but eduction will re-apply the transformation on each use. there seems to be no reason for that here?

11:52 pepijndevos: So what would I do instead?

11:52 SagiCZ1: gdeer81: is there some clojure api that focuses on that?

11:53 gdeer81: SagiCZ1, no, the ability to test java is available in core...interop

11:53 puredanger: pepijndevos: you're basically just mapping right? so anything that uses reduce there. (into [] (map to-int) %) or something ?

11:54 pepijndevos: hmmm, maybe...?

11:54 puredanger: if split was reducible, you could compose that into the splitting transformation which would be pretty nice

11:54 jonathanj: hrm, i'm running a CREATE TABLE and a CREATE INDEX in a bare db-do-commands form

11:55 my CREATE INDEX had a typo in it, so it failed to execute but the CREATE TABLE had already executed

11:55 fixing the typo and running the commands again results in an error about my table already existing

11:56 puredanger: pepijndevos: into has the benefit of utilizing a reducible source (not valid here) and using transients to produce the output collection. you could also just map around your split.

11:56 jonathanj: which is weird because i expected the failing CREATE INDEX to rollback the transaction

11:58 pepijndevos: I don't get that (into [] (map to-int) %)

11:58 puredanger: pepijndevos: so words could be (map (map (fn [s] (Integer/parseInt s)) (s/split % #" ")))

12:00 basically combine the words and numbers steps together. I'm not really sure which path yields the fastest result here; would require some testing with real data.

12:01 but I don't think there's any good reason to use eduction here

12:02 pepijndevos: ok

12:07 What does partition-by do as a transducer? Does it produce a transducer of transducers, of a transducer of collections?

12:09 puredanger: I'm not sure that question makes sense

12:09 pepijndevos: hehe

12:10 puredanger: it produces a transducer that has a reducing function that takes elements and invokes the next reducing function with vector partitions

12:11 pepijndevos: (into [] (comp (map #(what is %)) (partition-by odd?)) (range 100))

12:11 right, so it produces vectors

12:11 puredanger: yes, which differs from the seq version

12:12 pepijndevos: Is there any useful way to transducify the to-map part in https://www.refheap.com/94360 ?

12:14 edw: After the following is evaluated, will the storage for the :b key and its associated value be retained indefinitely? (def x (-> {:a 1 :b 2 :c 3} (dissoc :b) (assoc :d 42)))

12:15 puredanger: pepijndevos: well you could rewrite it as a single reduce where the merge is done on the accumulator

12:16 pepijndevos: edw, uuuhm, maybe because interning, but otherwise they would be GC'd, no?

12:17 puredanger: edw: as things are "removed" from the map, if no one else is referring to them, they are eligible for GC

12:18 that includes something holding a reference to the "before" version of the map

12:19 here you are actually storing a function on a literal map, so it's kind of a different case than in normal code

12:56 leyluj: yoooooo clojurerrs?

13:06 timpani: Anyone else have trouble trying to port install leiningen on a mac? I get ":error:build org.macports.build for port maven-ant-tasks returned: command execution failed" and, being clueless about ports (and maven), am not having luck with the suggestions on https://trac.macports.org/ticket/30060. Any help would be appreciated, thanks.

13:06 kenrestivo: why not just install the script?

13:06 joegallo: yeah, seriously. i know this sounds like flippant advice, but i promise it's not. just install the script yourself, don't use macports for this.

13:06 arrdem: it is strongly encouraged to install leiningen directly via the posted script rather than via a package manager.

13:07 and by strongly I mean technomancy will just tell you that all the packages are broken, unofficial and unsupported.

13:07 kenrestivo: i'm usually super-conservative about sticking to package managers (i'm a debian guy, ffs), and have only a handful of things in /usr/local, but lein is one of them

13:12 timpani: Script advice worked like a charm. Thanks, all.

13:14 justin_smith: lein is one of the few self-managing dependency managers that is guaranteed to be better at managing itself than what your OS provides

13:15 TimMc: justin_smith: Except it doesn't tell you when there are updates available.

13:15 justin_smith: TimMc: that's true

13:15 TimMc: Other than that I'd agree.

13:16 justin_smith: I didn't think of that because I prefer to explicitly check my dependency manager for updates

13:24 TimMc: justin_smith: I run Debian Stable so I pretty much only get security updates, and I'd rather be prompted for those. :-)

13:35 justin_smith: fair enough

14:15 mich: hey

14:16 michaelr525: anyone knows why iterate-table from cassaforte repondes as expected to (take) but hangs when used with (doall)?

14:17 kenrestivo: wow this ecosystem is amazing. it's eerie, 10 minutes after i realize i need something, before i can write it, i do a quick search and someone has like an hour ago released a library or posted an example which does it

14:17 not just that everything i need is already there, but it was written anywhere from hours to days before i need it

14:20 ppppaul: what is the lighttable channel?

14:27 arrdem: #lighttable

14:27 ppppaul: ^

14:27 ideas for tokenizing a ~500 word subset of english?

14:27 opennlp looks like massive overkill

14:28 amalloy: michaelr525: if iterate-table is anything like iterate, it produces an infinite sequence, and running doall on that is not a recipe for success

14:29 ppppaul: ^_^

14:29 amalloy: arrdem: slap all the words together with |, and then call re-pattern + re-seq or s/split?

14:41 edw: ,42

14:41 clojurebot: 42

14:53 gfredericks: question about component

14:53 sentence fragment

14:54 um so a lot of IO libraries will e.g. take a connection as the first arg in most of their API

14:54 which goes well with component to an extent

14:54 arrdem: amalloy: yeah I'll give that a shot

14:55 gfredericks: but for some ways of structuring your code you end up wanting to write a wrapper for the API that can take your Database record (say) as the first arg and will pull out the connection and then call the real API

14:55 amalloy: arrdem: you might map Pattern/quote over the words too, just in case there are some metacharacters in there

14:55 gfredericks: one way to "solve" that would be to change the library to have an IConnectionHaver protocol

14:55 which the record could implement

14:56 is this good? or bad? or what?

15:00 michaelr525: what's a faster serializer/reader nippy or transit?

15:04 timpani: newbie(-ish) question: why can't I instantiate a type defined in a namespace I use? Namespace N defines type T, but the constructor (T.) doesn't seem to visible in the repl.

15:05 arrdem: timpani: did you require N?

15:05 * or otherwise load N

15:06 hiredman: timpani: what is returned when you evaluate *ns* in the repl?

15:06 what repl are you using?

15:07 have you loaded the namespace?

15:07 pw4ever: is there a way to call Java base class's consturctor (i.e., "super(args)" in Java) with "proxy"?

15:07 hiredman: (if you have loaded the namespace code, what did you do that makes you think you did?)

15:07 razum2um: is there any progress library like clj-progress which indicates mapping over some lazy-seq? (i.e. it just counts steps every N, show time since start and draws progress like wget does when content-length is undefined) ?

15:07 timpani: arrdem: yes. hiredman: *ns* returns another namespace, but one that uses N.

15:08 hiredman: timpani: so?

15:08 timpani: hiredman: I thought require would make the type available.

15:08 but it doesn't seem to be, not even when prefixed with its namespace.

15:09 hiredman: timpani: I suggest not using defrecord/deftype until you have your sea legs with the langauge and the tools

15:11 there can be some subtle issues with defrecord and deftype that are challenging to debug via irc, so it would be easier if you were further along and could self debug to some extent

15:19 pw4ever: the answer to proxy super class args is right there in the doc. my bad.

15:28 pepijndevos: What is the most efficient way to update a bunch of record fields?

15:29 Just create a new record with the old fields?

15:29 mgaare: pepijndevos: assoc and update-in both work on records

15:30 pepijndevos: mgaare, but doing more than one update creates a bunch of intermediate records, right?

15:30 So in that case I might be better of using a new record

15:32 mgaare: pepijndevos: benchmark it both ways and see if there's any performance gain

15:32 pepijndevos: will do

15:45 EvanR: whats the preferred way to run an IO function on a list of items, and get the corresponding list of results, and force it to happen right now, not later via lazy seq

15:45 arrdem: EvanR: mapv could do that..

15:46 stuartsierra: timpani: defrecord/deftype create both Java classes and Clojure Vars. The Vars are the constructor functions `->RecordName` (both deftype and defrecord) and `map->RecordName` (only defrecord).

15:46 EvanR: ok

15:47 mikerod: Is there anyway to prevent macros from throwing exceptions about negative arity at macro expansion time?

15:47 arrdem: EvanR: it may be more idiomatic to manually loop with an accumulator here... using map for side effects isn't great.

15:47 stuartsierra: timpani: `(ns … (:require [… :refer …])` or `(ns … (:use …)` can make the Vars visible in another namespace, but not classes.

15:47 EvanR: arrdem: why?

15:49 bbloom: EvanR: doseq if you want effectfully use the results, mapv if you want to just hold on to them.

15:49 EvanR: doseq returns nil

15:50 bbloom: EvanR: then mapv is fine

15:50 amalloy: mapv for side effects doesn't make me happy. it doesn't make it clear to the reader that running it now for side effects is important; it might be that you need a vector because it's the right data structure, or because it's faster and performance matters

15:50 EvanR: well "functions" for side effects dont make me happy, but i dont think theres any way around this one either

15:51 also im trying to write a front-effect, rather than a side effect

15:51 bbloom: mapv is fine if it's clear from context, but in general it doesn't really make that much sense to linearly execute a bunch of side effects for a bunch of incremental results

15:51 amalloy: pepijndevos: making a new record with the old fields is really the only way to make a new record, period, of course

15:51 bbloom: i haven't run in to that situtation... ever really...

15:51 EvanR: eh?

15:51 stuartsierra: I always advise not to mix sequence functions with side-effecting operations. Laziness and chunking can lead to confusing effects.

15:52 pepijndevos: amalloy, hehe... true

15:52 bbloom: EvanR: why do you need the sequence of results?

15:52 EvanR: stuartsierra: hence my question, mapv is the answer

15:52 stuartsierra: or doall

15:52 EvanR: doall drops the results

15:52 stuartsierra: no, that's dorun

15:52 bbloom: (doc doall)

15:52 clojurebot: "([coll] [n coll]); When lazy sequences are produced via functions that have side effects, any effects other than those needed to produce the first element in the seq do not occur until the seq is consumed. doall can be used to force any effects. Walks through the successive nexts of the seq, retains the head and returns it, thus causing the entire seq to reside in memory at one time."

15:53 EvanR: oh well, (doall (map ... might be what i want then

15:53 amalloy: pepijndevos: https://github.com/flatland/useful/blob/develop/src/flatland/useful/datatypes.clj#L87 has a macro that does this in an efficient way with a semi-reasonable syntax; it's not my favorite, but it could be clearer than writing (Foo. (f (:x foo)) (g (:y foo))) yourself

15:54 bbloom: i always forget about doall... i don't really use it in my programs b/c, again, i haven't really felt the need to get intermediate results effectually unless i need to thread a result through, in which case reduce is what you want

15:54 pepijndevos: ah.. useful :)

15:54 amalloy: using it looks like https://github.com/flatland/useful/blob/develop/test/flatland/useful/datatypes_test.clj#L46-L51

15:54 bbloom: and even if i do want each of the individual results, i almost always do want to ALSO calculate some aggregate value, so you can just use reduce and do both the aggregation and "logging" at once

15:54 EvanR: bbloom: in general, an IO action has some result. for instance http-post could give a numeric status code, read-file gives you a string. now if i have a list of urls or filepaths, i might want to execute them all and get the list of results, this comes up a lot

15:55 bbloom: EvanR: but map really isn't appropriate for IO in any way

15:55 EvanR: but in clojure so far, i have gotten bitten by lazyseq, so maybe doall

15:55 bbloom: EvanR: instead of doing the IO in your map function, why not create a sequence of "instructions" and then do the IO at the top level?

15:56 EvanR: what?

15:56 clojurebot: http://paste.lisp.org/display/74305

15:56 bbloom: basically instead of mapping over a bunch of side effectful calls, map over raw data that describes the side effects to have

15:56 then do the side effects in a doseq or a reduce

15:56 amalloy: clojurebot: forget what |is| <reply> http://paste.lisp.org/display/74305

15:56 clojurebot: I forgot that what is <reply> http://paste.lisp.org/display/74305

15:56 bbloom: the advantage of that is you can also do a dry-run sort of thing

15:56 ,(def x (atom 0))

15:57 EvanR: bbloom: so, use body of an implementation of map?

15:57 clojurebot: #'sandbox/x

15:57 bbloom: ,(doseq [f [inc dec inc]] (swap! x f))

15:57 clojurebot: nil

15:57 bbloom: x

15:57 ,x

15:57 clojurebot: #<Atom@ef1280: 1>

15:58 amalloy: man, clojurebot has a lot of totally useless facts mapped to "what"

15:58 bbloom: EvanR: note that the sequence is data which describes which operations to perform, rather than performing the operations itself

15:58 EvanR: if you could imagine a foreach loop in an imperative language, but one that gives you the array of results rather than having to manually accumulate them in the body of the loop, this is the same advancement from the equivalent manual mapping strategy in such languages

15:59 bbloom: yeah, and the sequence of operations is constructed with something like map post urls

15:59 bbloom: EvanR: i understand what you want, and either mapv or doall will get the job done for you, but i'm telling you why i don't particularly feel a need for that -- i prefer effects be representable as printable data

15:59 EvanR: doseq would drop the results, and so doall would be better in this case

16:00 bbloom: EvanR: right but you can have {:method :post :url "http:/...} instead of #<some-opaque-function>

16:00 EvanR: or '(:post "http:///...")

16:00 bbloom: or that

16:00 EvanR: which is good but is this idiomatic

16:01 or warranted in the middle of obviously IO code

16:02 on the subject of when IO makes sense at all, im also sequencing my IO actions using the let binding form. [a (dox) b (doy x) c ...]

16:02 to me this has the same issues as map

16:02 er

16:03 (let [a (do-x) b (do-y a) c (do-z b) ...

16:03 in a pure context no one would expect a let binding to do IO

16:17 sdegutis: Is it by any means possible to delete/destroy a var so that referencing it once again results in an exception?

16:18 llasram: sdegutis: you mean remove it from an NS, as via `ns-unmap`?

16:18 sdegutis: Right! Thanks!

16:18 You win llasram.

16:18 * llasram does a victory lap around the channel.

16:20 EvanR: rigging a var to explode if disturbed ;)

16:20 assign TNT

16:22 sdegutis: I am using intern and eval and ns-unmap to run a Clojure form that has access to free-floating "variables".

16:23 EvanR: i cast "floating variable"

16:27 sdegutis: So I can do this: (eval-template '(inc foo) {:foo 2}) and it'll return 3.

16:27 Great, right?

16:30 amalloy: sdegutis: {:foo 2} seems like a terrible interface choice for that: i'd much prefer '{foo 2}

16:31 obviously it's just nitpicking, but implicit conversions from keyword to symbol make me sick to my stomach

16:32 pepijndevos: Where can you place type hints?

16:32 other than the argument vector...

16:33 amalloy: pepijndevos: that is a very broad question that sounds like it comes from a much more specific problem

16:33 and the correct answer ("almost anywhere") is unlikely to help you much

16:34 pepijndevos: Well, can I just put them anywhere, to hint at the return of a function call, or do I need to let that valueQ

16:34 ?

16:35 amalloy: welllll, if it's actually a function call, you can hint the function invocation

16:35 tbaldridge: you type hint the return value of a function via the vector

16:35 amalloy: but you're better off let-binding it and hinting the name

16:36 tbaldridge: pepijndevos asked how *else* to do it

16:36 pepijndevos: hhmmm, seems to work... yay

16:36 amalloy: and you can't actually hint a return value of anything but Object/long/double on the argvec

16:37 tbaldridge: amalloy: really? I seem to remember doing stuff like ^IFoo

16:37 amalloy: tbaldridge: sure, you can *do* it, but it has no effect

16:38 hinting the argvec is for passing and returning primitives, not for hinting what reference type to expect

16:38 hiredman: amalloy: I think there was some work to unify that, but I vaguelly recall it not being very thorough

16:39 amalloy: hm, maybe it does do something now. interesting

16:40 it didn't in 1.4, when hinting the argvec was introduced

16:40 tbaldridge: actually I just tried it, it doesn't work. I think it does for protocol methods though

16:40 that's probably why I was confused

16:40 amalloy: tbaldridge: really? i just tried it and it works

16:41 oh, interesting

16:41 (defn f ^String [x] "true") works, but (defn f ^String [^long x] "true") doesn't

16:41 tbaldridge: this gives a reflection warning in 1.6

16:41 ,(do (set! *warn-on-reflection* true) (defn foo ^String [] "foo") (.toUpper (foo)))

16:41 clojurebot: #<IllegalStateException java.lang.IllegalStateException: Can't change/establish root binding of: *warn-on-reflection* with set>

16:42 amalloy: tbaldridge: because .toUpper doesn't exist

16:42 tbaldridge: Well then......

16:42 ppppaul: anyone here have an exceptionally good experience with lighttable and cljs?

16:42 hiredman: it is kind of a mess

16:42 tbaldridge: lol, okay so it does work.

16:42 amalloy: tbaldridge: well as i said, it *kinda* works - look at my two examples

16:43 tbaldridge: The nasty bit is that IIRC something with java interop demands that the hint by on the symbol.

16:43 *be on

16:44 amalloy: anyway, this is why i recommend let-binding the result and typehinting that name

16:48 dbasch: ppppaul: I think most people here haven’t been very impressed with lighttable

16:48 bbloom: dnolen_: (or other cljs folks) is there a reason deftype* doesn't emit named constructor functions?

16:48 would be nice for debugging if no other reason to be able to (.. x -constructor -name)

16:49 dnolen_: seems like a simple extra (munge t) would do the trick, but i'm not sure if there's any fallout from that

16:51 EvanR: how i learned to stop worrying and learn to love reflection

16:52 ppppaul: i worry about learning how to stop worrying

16:53 also, i'm going to start using emacs again... spent a week with light table, and i can't get anything done with it in cljs

16:54 also, EvanR... how did you learn to stop worrying? did you add more carrots to your diet? i've always thought i haven't been eating enough carrots...

16:55 EvanR: by struggling with the type hints

16:55 ppppaul: but, what if they lie?

16:56 can one trust a type hint in this day and age?

16:57 EvanR: dynamic types never lie

16:58 sdegutis: amalloy: thx for advice

16:59 dnolen_: bbloom: haven't bothered w/ it since the best debugging environment (Chrome) just does the right thing in this case usually.

17:00 atyz: Hey all. I've written a macro and feel like there must be a better way to write it. Anybody willing to look at it?

17:00 https://www.refheap.com/94370

17:00 dnolen_: bbloom: FireFox too

17:00 weavejes_: atyz: What's the reason for making it a macro? Would a function do?

17:02 weavejester: atyz: You could write: (some (fn [f] (f entity)) fields)

17:07 bbloom: dnolen_: what does "does the right thing" mean?

17:08 dnolen_: anyway, here's a patch for it: http://dev.clojure.org/jira/browse/CLJS-902

17:08 dnolen_: bbloom: the debugging tools know

17:08 bbloom: dnolen_: really? i'm not seeing that, but i'm also trying to print string representations of things, rather than mouse over etc in the console

17:09 amalloy: weavejester: you have to be careful about chunking and such if you do it that way

17:09 dnolen_: bbloom: ctors have always been printable via (type x)

17:09 amalloy: obviously it's still right to do it as a function, but more care than just using 'some is warranted

17:09 dnolen_: bbloom: not the constructor function itself of course, but like I said Chrome/FF display this

17:10 bbloom: thanks for the patch will think about it

17:10 bbloom: dnolen_: i don't understand how the `type` function helps? it returns the ctor function, but that doesn't toString right

17:10 dnolen_: bbloom: I don't know what you are doing so I don't yet care :)

17:11 bbloom: dnolen_: ooooooh you're pr-str works

17:11 you're saying*

17:11 on the type

17:11 hm

17:11 (set! (.-cljs$lang$ctorPrWriter ~t) (fn [this# writer# opt#] (-write writer# ~(core/str r))))

17:11 i didn't realize that

17:12 dnolen_: yeah

17:12 I got sick of this problem like 1 1/2 years ago :D

17:12 bbloom: dnolen_: ok, that's helpful, but still doesn't 100% solve my problem since i'm using js libs that don't respect pr-str

17:14 dnolen_: one other thing is that if you don't want to give the ctor a name, you can also just stick ac omment string in the generated code -- /* some.ns/SomeType */ will print in the tostring, but surely get removed in advanced optimizations

17:14 dnolen_: thanks though

17:22 arrdem: amalloy: heh after some munging to figure out my language, the huge generated regex works a treat. cheers.

17:28 dmahanta: from this ({:a 1 :b 2}{:a 3 :b 4}) how do i get jus the values of a (1,3)

17:28 amalloy: dmahanta: (map :a ...)?

17:28 dmahanta: amalloy thanks now it makes sense

17:29 AeroNotix: Using the clojure-metrics library, what do I want to get "How many of a particular event in the last 60s" ?

17:50 david234567: Hello! I am having problems with core.async. What I'm trying to do is have two go routines read and write to a socket depending if channels have stuff in them, but it doesn't seem to be working right

17:50 Here's the code:

17:50 https://gist.github.com/iMultiPlay/e4fa0258c9be16f24248

17:50 The function starts on line 36

17:52 thank you :3

17:53 amalloy: david234567: "<this giant code dump> doesn't work" is not a problem that is easy to solve. what doesn't work? what happens instead? can you find a smaller program that exhibits the same problem?

17:54 dbasch: also, it’s “receive”

17:55 fix that spelling mistake and the program will magically work

17:55 kenrestivo: i could say, here's code dump that works, why not use this instead? http://crossclj.info/ns/com.gearswithingears/async-sockets/0.1.0/project.clj.html

17:55 amalloy: (inc kenrestivo)

17:55 lazybot: ⇒ 2

17:56 dbasch: one thing that’s very suspect: two nested “while true” loops

17:57 amalloy: dbasch: well, they're not exactly nested

17:57 dbasch: true, one is in a go loop, never mind

17:57 still looks strange

17:58 david234567: emacs crashed, so I guess the receive thing worked lol. The double while (one in go) was a desperate hack to get it to work

17:58 kenrestivo: i tried to write my own too. i gave up and just used asyncsockets instead.

17:59 it works, and it's fun to use.

18:01 caution though: i found a bug, submitted a pull request, and saw another pull request fixing another bug sitting there, apparently both are still not merged, so not sure if it's abandoned or not

18:03 david234567: that library seems promising, thank you! (it's not very serious so I'll just live with the bugs)

18:05 kenrestivo: quick community question: what's the "right thing to do" if i need a library, need to fix bugs in it, want to deploy the bugfixed version to production (which means clojars), but the author isn't respondin to pullrequest/issues? just fork it? do i change the artifact to my own name? what's the polite thing to do?

18:07 i mean, do i want to pollute clojars with [kens.bufixed.fork/some-library "0.1.clash-with-future-upstream-version-number"

18:07 ?

18:08 {blake}: I'm trying to pass a "true" into a Java routine. I'm assuming Clojure's "true" maps. Is that correct?

18:08 dbasch: {blake}: correct

18:08 of course true without quotes

18:08 {blake}: dbasch: Dammit. That would've been an easy fix if I had been wrong.

18:09 dbasch: ,(type true)

18:09 clojurebot: java.lang.Boolean

18:09 {blake}: Yeah, I debated denoting it with some other symbol or TRUE or whatevs. But I knew you'd know.

18:10 OK, so maybe it's this: I've translated <<System.setProperty("poi.log.level", POILogger.INFO + "");>> into <<(System/setProperty "poi.log.level" (str POILogger/INFO ""))>> but...meh, that seems to be right.

18:12 david234567: Sorry for all the questions, but it seems that the async-sockets library refuses to be found on the class path (added it to profiles.clj and ran lein deps and restarted repl)

18:13 amalloy: kenrestivo: yes, pollute clojars with an artifact that has a groupid you own

18:15 CaptainLex: What's the best way to do a grid-type data structure in Clojure? I'm storing binary data about a square grid

18:15 Retrieval time is exponentially more important than update time

18:15 Nested vectors?

18:15 amalloy: yes

18:16 dbasch: unless it’s sparse

18:16 CaptainLex: dbasch: It'll be sparse in 1s, but dense in data. No undefs

18:16 amalloy: also, i'd just like to register a complaint about "exponentially more" being used as a vague intensifier

18:16 CaptainLex: amalloy: Noted :)

18:17 dbasch: yes, it’s only constantly more important

18:18 CaptainLex: dbasch: Is that the kind of sparseness you meant?

18:19 dbasch: if you need all the 0s and 1s and are not bothered by memory usage then sure, nested vectors is the way to go

18:19 it it were sparse I’d have a map of maps with the set values

18:20 CaptainLex: dbasch: Okay, good to know. Actually I might end up needing to encode grids differently based on their use case

18:20 It's an AI project

18:20 amalloy: you could also just have a single vector and address it with multiplication, like the olde days in C; that probably gets you a slight speedup due to the increased structural sharing, but nothing too exciting

18:20 CaptainLex: there'll be the "objective" grids that describe the terrain

18:21 and then grids that describe what the agent has learned about the terrain

18:21 EvanR: CaptainLex: or a 2d java array

18:21 CaptainLex: It may end up the latter grids would be sparse, and best encoded as maps like dbasch suggested

18:22 EvanR: which might be exponentially more clucky to use from clojure

18:22 klunky

18:22 kenrestivo: there are exponentially a few matrix libraries for clj and java

18:22 EvanR: lol

18:23 CaptainLex: kenrestivo: Oh yeah oops!

18:23 I was originally writing this for CLJS

18:23 kenrestivo: today is Exponential Abuse day

18:23 CaptainLex: because I want to incorporate it into a personal project

18:23 but I decided for time's same I'll do in Clojure and port it later

18:23 amalloy: kenrestivo: i had an exponentially burrito for lunch

18:23 CaptainLex: so I could definitely use core.matrix or whatever

18:23 kenrestivo: burrito... cubed!

18:24 dbasch: I had an Indian burrito, which is exponentially more exponential than a regular one

18:24 kenrestivo: yeah core.matrix would be the first place i'd look

18:24 EvanR: does core.matrix work in cljs?

18:25 CaptainLex: EvanR: There's rumbling of an attempt to port, but not at present

18:25 kenrestivo: https://github.com/mikera/core.matrix/issues/81

18:26 which leads to http://0fps.net/2013/05/22/implementing-multidimensional-arrays-in-javascript/, which leads me to quit procrastinating on irc and try to get some work done

18:30 CaptainLex: Oh wait

18:30 before I write too much code and commit myself to Clojure

18:30 performance monitoring is a part of my project

18:30 AeroNotix: CaptainLex: riemann and graphite

18:30 CaptainLex: AeroNotix: Thanks!

18:33 supersymmetry: lol

18:41 EvanR: can you implement a protocol on another protocol

18:41 danielcompton: EvanR: IP -> TCP -> HTTP

18:42 EvanR: so funny i forgot to laugh

18:43 amalloy: i don't think i've heard "so funny i forgot to laugh" since middle school. what a surprising throwback

18:43 EvanR: i know

18:44 CaptainLex: Last time I heard it was from the mouth of Pee Wee Herman

18:50 {blake}: OK, "return string if not blank otherwise return nil"--I think I can do this without an "if", right?

18:50 Or do I need to write my own func?

18:51 supersymmetry: Sigh... such a shame Clojure is too unfamiliar here in the Netherlands

18:51 EvanR: ,(not-empty "")

18:51 clojurebot: nil

18:51 supersymmetry: Near impossible to find any work in programming clj anyway,...

18:51 EvanR: ,(not-empty "abc")

18:51 clojurebot: "abc"

18:52 supersymmetry: ... I'll be damned to take up 'PHP' for a profession though, there's simply just lines one does not cross

18:53 EvanR: professional PHP, specifically

18:54 {blake}: Thanks, EvanR!

18:54 supersymmetry: Amsterdam Clojurans meetup has about 20 people .. that is, the entire country - yay. Amazing how many 'Are you HOT for PHP'-like vacancies too ("Erh. No! And why would anyone?")

18:55 CaptainLex: supersymmetry: Try being a front-end developer and slowly convincing your superiors to let you use CLJS

18:55 supersymmetry: EvanR: yeah although I wonder if there ever is a real difference: seen too many 'professional' coders omit { and } for that

18:55 EvanR: PHP is currently like the bud light of programming jobs, deployment

18:56 CaptainLex: That's what I'm trying

18:56 arrdem: (inc EvanR)

18:56 lazybot: ⇒ 6

18:56 arrdem: EvanR: it's still piss at any temperature :P

18:57 EvanR: its cheap, ubiquitous, and terrible

18:57 hyPiRion: supersymmetry: hey, you actually have a meetup at least

18:57 supersymmetry: CaptainLex: Indeed, it would be easier too smuggle in. Or learn e.g. Seeshaw and go that way

18:58 CaptainLex: I must side with hyPiRion; the USA is too big for us ALL to meet in one place, the place where I live hasn't got a group at all

18:58 arrdem: I've got one... :D

18:58 may even make it to the meeting this monday..

18:58 hyPiRion: CaptainLex: I'm not located in the USA

18:58 supersymmetry: which makes me ask the question (since I'm not too deep in on the low level bindings stuff): does clojure generate Java besides gen-class or only bytecode?

18:59 hyPiRion: where were you from again?

18:59 CaptainLex: hyPiRion: I was just siding with you on how having a small one is better than not having one

18:59 hyPiRion: Norway

18:59 CaptainLex: ah, right – yeah indeed

18:59 arrdem: supersymmetry: Clojure never generates Java code. Everything goes straight to bytecode.

18:59 supersymmetry: true...and although other side of the country (not that big heh) I'll go and check it out next time

18:59 TimMc: supersymmetry: Clojure doesn't generate any Java, just JVM bytecode.

18:59 supersymmetry: they had new+veteran match up and do coding contests which is fun + educational I guess

19:00 right

19:02 lol@bud-light btw... so what does that make clj? Sam Adams? Thought that was a nice brew when I was in the states last time :)

19:03 CaptainLex: My understanding from my beer-snob friends is that any beer as good as Clojure

19:03 probably isn't well-known or mass-produced

19:06 ajmccluskey: Clojure is any beer brewed at Firestone in San Luis Obispo.

19:07 Not only is their beer some of the tastiest I've ever had, the design and thought they put into their brewing is impressive.

19:07 Very Clojurian :p

19:09 supersymmetry: CaptainLex: true

19:10 TEttinger: ajmccluskey: firestone, the SUV-killing tire manufacturer?

19:11 ajmccluskey: TEttinger: haha, I very much hope they're not related. AFAIK they're just a boutique brewery that happen to share the same name.

19:11 TEttinger: actually, I think their full name is "Firestone Walker", so almost certainly not

19:11 TEttinger: firestone driver would be more of a concern

19:19 supersymmetry: .. and I know perl. Damn. I feel like Neo sometimes after learning Clojure ^^

19:19 2.5 hours to fully grok it

19:22 also said one of the best paying skills to have, perl, yet extremely little vacancies here... it seems to me a lot of these *nix skilled jobs are just done with Puppet and such as much as they can

19:22 and companies using sharepoint or asp (backs away quickly)

19:35 wildnux: hi,

19:35 TEttinger: hey wildnux

19:35 wildnux: how can we accumulate values in a vector that is outside the when-let block?

19:36 TEttinger: you mean mutating the vector? you may be looking for a non-clojure data structure

19:36 wildnux: say we have (let [acc []) (when-let [s (even? (range 0 10)] (.....??.... acc s)) acc)

19:37 so after it iterates through the when-let values, the values are accumulated in acc

19:37 TEttinger: uh, there's a lot of problems there

19:37 the let already ended by the time you start using acc

19:37 wildnux: TEttinger: actually that is a typo, i just typed it here only

19:38 TEttinger: oh ok

19:38 ,(even? (range 0 10)

19:38 clojurebot: #<RuntimeException java.lang.RuntimeException: EOF while reading>

19:38 wildnux: TEttinger: but, you got the idea right, acc is outside when-let, and i want to add values to it so that after iterating throught the range, i have them accumulated,

19:38 TEttinger: ,(even? (range 0 10))

19:38 clojurebot: #<IllegalArgumentException java.lang.IllegalArgumentException: Argument must be an integer: clojure.lang.LazySeq@9ebadac6>

19:39 wildnux: ,(even? (range 10))

19:39 clojurebot: #<IllegalArgumentException java.lang.IllegalArgumentException: Argument must be an integer: clojure.lang.LazySeq@9ebadac6>

19:39 TEttinger: ok, well the idiomatic way is to return a new list

19:39 or new vector, or whatever

19:39 what is this code supposed to accomplish?

19:40 like what's a typical return

19:40 EvanR: wildnux: you cant mutate a vector, as far as i know

19:40 wildnux: TEttinger: basically, i want to iterate through a list/sequence, and for each element, i want to accumulate the result of applying some funtion to that element, only if it meets certain critera

19:40 TEttinger: sounds like a reduce

19:41 EvanR: you might be thinking of using a var or an atom to hold the current accumulator, then something can replace it with a side effect

19:41 wildnux: to simplify what is the idiomatic way to write the following in clojure?

19:42 TEttinger: ,(reduce #(if (even? %2) (conj %1 (inc %2)) %1) [] (range 10))

19:42 clojurebot: [1 3 5 7 9]

19:42 wildnux: List<Object> l = new ArrayList(); for (Object o : objectsFromExternalSources) { if (satisfiesSomeCond(o) { l.add(o);}}

19:43 amalloy: TEttinger: better, (reduce conj (map inc (filter even? xs)))

19:43 well, (reduce conj [] ...), rather

19:43 TEttinger: right, that works better, amalloy

19:43 amalloy: and actually, why are we reducing at all if we are conjing

19:44 wildnux: that is just a filter

19:44 (filter some-cond some-objects)

19:44 TEttinger: yeah, that seems like the best way then

19:44 EvanR: filter and map can both be implemented with reduce, both are more clear when thats what youre trying to do

19:44 reduce is more general

19:45 wildnux: (filter #(someCond %) objectsFromExternalSource)

19:45 ?

19:45 TEttinger: (filter someCond objectsFromExternalSource) also works

19:45 amalloy: wildnux: #(some-cond %) is just some-cond

19:46 wildnux: amalloy: :) i see it now, rookie mistake :P

19:54 kenrestivo: i really should be using a database, but just for grins, how would i flatten out a nested map {:k1 {:k2 {:k3 1 :k4 2}}} to be: [[:k1 :k2 :k3 1] [:k1 :k2 :k4 2]] ? was thinking i could abuse reductions for this purpose, but that felt kinda hacky to me

19:56 EvanR: what is going on... im calling a function of 2 args, with 2 args. the error says "passed 1 arg". so i changed it to pass 3 args. error "3 args". 4 args "4 args". back to 2 args "1 arg"

19:58 kenrestivo: EvanR: source?

19:58 pretty sure i've seen that, it was pilot error, for sure.

19:59 EvanR: ,((fn [a b]) 1)

19:59 clojurebot: #<ArityException clojure.lang.ArityException: Wrong number of args (1) passed to: sandbox/eval25/fn--26>

19:59 EvanR: ,((fn [a b]) 1 nil)

19:59 clojurebot: nil

20:02 kenrestivo: right, a b are the two args, you passed it 2 args, 1 and nil, no problem.

20:02 EvanR: kenrestivo: https://www.refheap.com/f4ff22de5030335b7d194abc1

20:03 kenrestivo: that postwalk callback is the one giving you trouble?

20:04 maybe [[k v]] not [k v] ?

20:04 EvanR: ok, im misinterpreting where the error is coming from

20:04 the tracebook in the postwalk isnt pretty

20:04 back

20:04 kenrestivo: clojure tracebacks are.... an acquired taste.

20:05 EvanR: trying your fix

20:05 kenrestivo: also acquired taste: putting closing parens all on one line.

20:05 EvanR: when i do its hard to put something in between them in the right place

20:06 (with vim)

20:06 kenrestivo: i'm pretty sure vim supports paren matching, even completion

20:06 EvanR: i need to put something somewhere in a ))))))))

20:06 for example

20:07 it will show me the matching paren on the left, but its either-or, not clear

20:07 also known as i suck at lisp

20:07 kenrestivo: everyone sucks at everything until they get good at it.

20:09 ajmccluskey: EvanR: paredit mode in Vim is decent for managing parens. That and paren highlighting are enough for me. Obviously takes a little time getting used to it, but Vim/Emacs users are used to that.

20:10 EvanR: im using vim-sexp which i gather is the cousin to paredit we dont talk about

20:12 well my problem was in the arguments of the postwalk function

20:12 ajmccluskey: Hmm, I've never looked at vim-sexp. Think I'm too used to paredit now to change without a compelling reason.

20:14 It might well be true, but I keep having to tell myself I'm not stupid despite how much trouble implementing a basic graph in Clojure is giving me.

20:14 kenrestivo: yeah, i didn't look closely at the code but that kind of popped out at me. a common error i make: [k v] instead of [[k v]], and the "two arguments supplied" error pointed at that

20:14 fowlslegs: Say I accidentally started an infinite loop in a REPL. Is there a way to kill it w/o closing the REPL?

20:14 ajmccluskey: Ctrl-C works for me

20:16 EvanR: ctrl-c, if youre lucky

20:16 fowlslegs: ajmccluskey: Do you run that in the terminal you started lein from?

20:16 EvanR: ajmccluskey: a basic graph. a map from node id to node id?

20:17 or, set of node ids

20:17 fowlslegs: Didn't work for me.

20:17 ajmccluskey: fowlslegs: I hit that in the repl that's infinite looping

20:17 fowlslegs: if that fails, I kill the java process that's chewing CPU more than the others :p

20:18 fowlslegs: Does killing that java process kill Leiningen as well?

20:18 ajmccluskey: EvanR: Maybe it's slightly more than basic. It's a directed graph, and I have to collapse equal nodes at some point

20:19 EvanR: equal nodes?

20:19 justin_smith: ajmccluskey: look at the adjacency-list representation, that's the right way to do it with immutable values

20:19 ajmccluskey: fowlslegs: it'll kill the `lein repl` session that you started

20:19 justin_smith: ajmccluskey: don't try to do it as nested data structures with clojure data structures, that's doomed

20:20 ajmccluskey: EvanR: nodes that have an equal value. It's a bioinformatics problem and I need to collapse all nodes with the same value onto one node, while cumulating all of their edges.

20:20 justin_smith: ajmccluskey: http://en.wikipedia.org/wiki/Adjacency_list

20:20 fowlslegs: ajmccluskey: I just ran `exit` in the REPL since the process was sent from CIDER. Is there a ctrl+c equivalent in CIDER?

20:20 justin_smith: fowlslegs: C-c C-c

20:20 ajmccluskey: justin_smith: I think I got burned trying that last time (and this time). I'll go back to thinking harder. Thanks.

20:21 fowlslegs: justin_smith: danke

20:21 justin_smith: ajmccluskey: got burned doing what?

20:21 ajmccluskey: justin_smith: last time I was trying to store my graph as a map containing a list of nodes, and each node a list of its incoming and outgoing edges

20:22 justin_smith: ajmccluskey: yeah, that is provably impossible with immutable data structures if the graph has any cycles

20:22 also, adjacency lists are easier to work with even if you don't have cycles

20:23 ajmccluskey: justin_smith: I think I got it working, but as the algorithms course got harder my graph got very slow and unruly. I gave up and moved to Java after that.

20:23 justin_smith: in conclusion: adjacency lists!

20:25 ajmccluskey: justin_smith: I thought I was going for an adjacency list :(

20:26 pataprogramming: ajmccluskey: aysylu/loom implements basic graph data structures. Looking at the internals may be helpful if you're spinning your own.

20:27 ajmccluskey: my design was a vector with nodes referenced by their index in the vector, and each node containing 3 items - value, indexes of nodes pointing to it, indexes of nodes its pointing to

20:28 pataprogramming: thanks. will check it out. Given it's for a course and I'm trying to get more Clojure under my belt, I'm spinning my own, but something to look at would be helpful.

20:48 tuft: i used loom on a school project. pretty handy

20:48 ajmccluskey: pataprogramming: loom is perfect for me to look at. Thanks again. I feel vindicated that our designs seem to be in the same ballpark

20:54 arrdem: alexyakushev: ping

20:59 alexyakushev: arrdem: sup

21:01 kenrestivo: i hate this function very much, but it works, and i'm not sure how to approach making it less ugly: https://www.refheap.com/943773

21:01 arrdem: alexyakushev: two things.. first is "getting started" reading on skummet's internals? realized we're wasting effort getting the same place two ways.

21:02 amalloy: kenrestivo: that is a 404?

21:02 you appear to have meant https://www.refheap.com/94377

21:02 alexyakushev: arrdem: indeed, I was saying that back in spring:)

21:03 kenrestivo: https://www.refheap.com/94379

21:03 arrdem: alexyakushev: hehe so was I but oh well.

21:03 kenrestivo: actually https://www.refheap.com/94380

21:04 amalloy: kenrestivo: so like, the most basic improvement is that all these nested fors and apply concats should just be a single for

21:04 pataprogramming: ajmccluskey: Cheers.

21:04 kenrestivo: amalloy: i'm not following. how could that be possible?

21:05 because, that'd be a huge improvement, maybe could just call it done at that point

21:05 amalloy: kenrestivo: vaguely like https://www.refheap.com/426385e142aa84e8a86424855

21:06 kenrestivo: oh cool, abusing :when

21:06 amalloy: abusing????

21:06 kenrestivo: ok, properly using, in a way i've never seen before

21:06 amalloy: kenrestivo: how is it any different from the way you were already using it?

21:07 kenrestivo: i didn't know you could nest the bindings like that.

21:08 that's really neat, thanks

21:09 (inc amalloy)

21:09 lazybot: ⇒ 200

21:09 amalloy: you're welcome

21:09 kenrestivo: hey! round number!

21:09 amalloy: 256 or bust

21:09 kenrestivo: -127 or bust THIS IS JAVA

21:09 CaptainLex: So out of curiosity, why is numeric-tower in contrib? Why is exponentiation not in clojure.core?

21:10 amalloy: CaptainLex: because it's in java.lang.Math

21:10 if you want to use doubles, anyway. and if you want to use ints, it's trivially easy to do yourself: (apply * (repeat n b))

21:10 CaptainLex: amalloy: So you mean best practice is actually to access it through that most of time?

21:11 amalloy: sure, why not?

21:11 CaptainLex: Yeah exponentiation is easy but I want square roots

21:11 No reason not

21:11 just making sure

21:14 gfredericks: ,(Math/sqrt Math/PI)

21:14 clojurebot: 1.7724538509055159

21:20 FriedBob: This snippet was taken from a demo/tutorial on zippers, but I was wonder if this is generalyl considered the best way to load and parse an XML file? https://gist.github.com/llowder/a58fb6aa92251f6858f0

22:08 gfredericks: &(map int (.doFinal (doto (javax.crypto.Cipher/getInstance "AES/CBC/NoPadding") (.init javax.crypto.Cipher/ENCRYPT_MODE (javax.crypto.spec.SecretKeySpec. (byte-array 16) "AES"))) (byte-array 16)))

22:08 lazybot: ⇒ (-121 -74 -27 113 108 -81 112 116 -43 -19 103 -34 -70 -73 -3 10)

22:08 gfredericks: &(map int (.doFinal (doto (javax.crypto.Cipher/getInstance "AES/CBC/NoPadding") (.init javax.crypto.Cipher/ENCRYPT_MODE (javax.crypto.spec.SecretKeySpec. (byte-array 16) "AES"))) (byte-array 16)))

22:08 lazybot: ⇒ (83 -65 15 -94 97 -123 3 127 73 -38 -71 -43 -86 10 103 -77)

22:09 gfredericks: does anybody know enough about javax.crypto and/or AES to guess why ^that's nondeterministic?

22:11 justin_smith: gfredericks: It's an intentional feature of AES that two encryptions of the same plaintext with the same key don't give you the same result. Otherwise an attacker can make rainbow tables.

22:11 It's all about making attempts to break your crypto as CPU expensive as possible.

22:12 If you are testing an encrypted / hashed result, the right thing is to verify the hash, not compare two cyphers.

22:12 gfredericks: I learned AES back in the day and that doesn't sound familiar; and now I'm having trouble imagining how this can even work

22:12 justin_smith: s/cyper/cyphertext

22:13 gfredericks: I thought a block cipher was supposed to be a permutation

22:13 jneen: is there an obvious reason i'm missing that leiningen might ignore ~/.lein/profiles.clj ?

22:13 justin_smith: gfredericks: right, but there are multiple outputs that decrypt / verify identically

22:13 jneen: i added a profile there and it claims it doesn't exist

22:14 gfredericks: justin_smith: isn't the output block the same size as the input block?

22:14 fowlslegs: gfredericks: Simple answer is that for symmetric encryption algorithms multiple messages are not secure in the presence of an eavesdropper when the algorithm is deterministic.

22:14 justin_smith: jneen: what happens if you try to load the file from a repl, any errors?

22:14 gfredericks: it is not about size, it's about the contents of the output being deterministic or not

22:14 gfredericks: as I said, otherwise you end up with rainbow tables

22:14 gfredericks: right but if you have to be able to get the input from the output then there has to be room for nondeterminism

22:15 if they're the same size then decryption would be ambiguous, no?

22:16 fowlslegs: gfredericks: The nonce used to encrypt each block is given as part of the ciphertext.

22:17 justin_smith: also, the fact that it is called a nonce makes me happy

22:17 gfredericks: so Cipher is internally generating some random bits?

22:17 fowlslegs: If you encrypt the same block with the same cipher and nonce, then the ciphertext will be the same.

22:17 gfredericks: I don't see any avenue for setting the nonce

22:17 fowlslegs: Yeah, this depends on the mode AES is in.

22:17 gfredericks: in the API

22:17 I also don't understand why that gives me 16 bytes of output then -- seems like it's the same size as the input

22:18 jneen: justin_smith: welp it was missing a delimiter

22:18 (which you'd think lein would warn you about...)

22:18 but i fixed the missing delimiter and it's got the same behavior

22:19 justin_smith: jneen: indeed. silently swallowing errors is a plague.

22:19 jneen: so there must be some other error it's silently swallowing?

22:19 justin_smith: jneen: odd. can you share the file?

22:20 gfredericks: oh I see it's .getIV

22:20 I had tried that before but probably before calling .init so it came back nil

22:20 fowlslegs: gfredericks: I know nothing about the Java implementation, so I can't help you any more with that. I just know how it works abstractly. And like I said, there are different modes of operation for block ciphers and different methods to generate nonces, so that's another variable.

22:20 gfredericks: justin_smith: fowlslegs: thanks

22:21 fowlslegs: gfredericks: glad I could help. Usually I'm the one getting help from you.

22:21 gfredericks: (inc fowlslegs)

22:21 lazybot: ⇒ 2

22:22 jneen: justin_smith: https://gist.github.com/jneen/5c5925ed3cc77707d9e1

22:22 ; lein with-profile foo pprint

22:22 (Warning: profile :foo not found.)

22:23 fowlslegs: gfredericks: Didn't know about that bot feature and thanks. Is this value actually stored somewhere?

22:23 gfredericks: $karma fowlslegs

22:23 lazybot: fowlslegs has karma 2.

22:23 gfredericks: fowlslegs: in lazybot's database I assume

22:23 justin_smith: fowlslegs: it's in a mongodb store on lazybot's server

22:23 fowlslegs: $karma gfredericks

22:23 lazybot: gfredericks has karma 112.

22:24 gfredericks: amalloy made it to 200 today I Think

22:24 justin_smith: fowlslegs: lazybot is open source, and I did a huge refactoring to up some lib versions recently if you have any more specific questions about the impl

22:24 $karma amalloy

22:24 lazybot: amalloy has karma 200.

22:24 justin_smith: nice!

22:24 gfredericks: not even for making a joke

22:24 justin_smith: $google github raynes/lazybot

22:24 lazybot: [Raynes/lazybot · GitHub] https://github.com/Raynes/lazybot

22:25 justin_smith: ^^ official repo

22:25 jneen: justin_smith: i'm literally copying from the docs at this point

22:26 justin_smith: jneen: I even scanned for hidden unicode chars that may be throwing it off, but it looks clean

22:27 and the file is in ~/.lein/profiles.clj?

22:27 jneen: ; ls ~/.lein/

22:27 profiles.clj

22:28 TimMc: lein version?

22:28 jneen: latest

22:28 just ran lein upgrade

22:28 to see if that would fix it

22:28 so that'd be 2.5.0

22:28 TimMc: Try lein upgrade 2.4.3 and re-try.

22:29 jneen: i had 2.4.2 before

22:29 TimMc: ah, OK

22:29 fowlslegs: justin_smith: Looks great! Don't have time to check out the code right now, but I starred it for later.

22:29 jneen: i can try 2.4.3 for lulz but i can't imagine it'd be much different

22:29 TimMc: I'm grasping at straws; 2.5.0 has some profiles weirdness, but it shouldn't affect this...

22:30 jneen: yeah me too

22:30 TimMc: yeah, don't bother

22:30 jneen: yeah 2.4.3 as well

22:31 is there some sort of like, "find the lein directory somewhere else" configuration i may have set accidentally?

22:31 my lein script is at ~/.local/bin/lein

22:32 and i've got a ~/.local/lib/lein that's got a few things in it

22:32 ; ls ~/.local/lib/lein

22:32 repl-history self-installs/

22:32 justin_smith: jneen: try putting profiles.clj in that dir

22:33 jneen: giving that a go

22:33 justin_smith: because those things usually are in ~/.lein

22:33 so maybe it thinks that is your LEIN_HOME or something...

22:33 jneen: oh hey would you look at that

22:33 ; echo $LEIN_HOME

22:33 /home/jneen/.local/lib/lein

22:33 w e l p

22:33 justin_smith: well that would explain that!

22:33 jneen: thanks past jneen

22:35 justin_smith: sorry I didn't think of LEIN_HOME earlier

22:35 jneen: bam

22:35 thank you all

22:43 TimMc: fascinating

22:43 I didn't even know that was a thing, but it makes sense.

22:57 bridgethillyer: jneen: phew!

23:18 jneen: bridgethillyer: ohai :)

23:18 bridgethillyer: hi!!!

23:25 justin_smith: http://i.imgur.com/v3KlJoc.gif perfect metaphor for my first ignorant attempts to use clojure

Logging service provided by n01se.net