#clojure log - Nov 25 2015

The Joy of Clojure
Main Clojure site
Google Group
IRC
List of all logged dates

9:39 jonathanj: how do i create a brand new element with enlive?

9:41 ystael: Anybody know whether compojure.api.swagger/swagger-docs can be told to combine routes from several route tables into one big doc json? Right now if I have a #'sub-routes reference in my top level routing table, the (swagger-docs) in that top level table doesn't summarize the API in sub-routes.

10:42 nam: how do I convert a string to a list of int? lik "31321" to '(3 1 3 2 1)

10:44 ARM9: ,(map Integer/parseInt "31321")

10:44 clojurebot: eval service is offline

10:47 ARM9: nvm a string is treated as a seq of chars, try (map #(Character/getNumericValue %) "31321")

10:54 nam: ARM9, thanks the second solution works

12:04 jjttjj: if i keep implementing the same wrapper functions over Monger for various collections, ie find-one, find-all, add, etc, what's the best way to abstract this away so i can just magically have these functions by passing the collection name to something? I vaguely recall seeing a gist that did that but can't find it

12:07 sdegutis: Is it inappropriate to use core.async within a Component?

12:20 Will a namespace that specifies `:gen-class` also generate classes for anything it requires in the `(ns (:require ...))` form?

12:21 arrdem: Is there a good CIDER hook for running javac and reloading a class?

12:24 sdegutis: arrdem: within a .java file?

12:35 muhuk: jjttjj: are you asking how to implement higher-order functions?

12:36 Bronsa: arrdem: have you seen https://github.com/ztellman/virgil?

12:37 jjttjj: muhuk: i mean is there any way to make this: https://github.com/flyingmachine/arenaverse/blob/master/src/arenaverse/data_mappers/db.clj easy without having to stick a bunch of defns in macros?

12:43 muhuk: jjttjj: I see. Another way might be to pass a ns to some function and manually add fns there. But this sort of thing is, I dislike. Someone might have better idea.

12:44 jjttjj: I would simply put these fns into a ns and call like normal people do. They are not special in any way that I can see.

12:47 jjttjj: muhuk: cool thanks. just tempted to be extra lazy and get rid of the repeating of the same string across a NS

12:47 muhuk: jjttjj: yw. Too much magic IMO.

12:51 ToBeReplaced: ,doc intern

12:51 clojurebot: eval service is offline

12:51 ToBeReplaced: jjttjj: see "intern"

13:06 sdegutis: ,(inc 2)

13:06 clojurebot: 3

13:06 sdegutis: (doc intern)

13:06 clojurebot: "([ns name] [ns name val]); Finds or creates a var named by the symbol name in the namespace ns (which can be a symbol or a namespace), setting its root binding to val if supplied. The namespace must exist. The var will adopt any metadata from the name symbol. Returns the var."

13:06 sdegutis: ,doc

13:06 clojurebot: #error {\n :cause "Can't take value of a macro: #'clojure.repl/doc"\n :via\n [{:type clojure.lang.Compiler$CompilerException\n :message "java.lang.RuntimeException: Can't take value of a macro: #'clojure.repl/doc, compiling:(NO_SOURCE_PATH:0:0)"\n :at [clojure.lang.Compiler analyze "Compiler.java" 6704]}\n {:type java.lang.RuntimeException\n :message "Can't take value of a macro: #'clojure....

13:06 sdegutis: ,doc intern

13:06 clojurebot: #error {\n :cause "Can't take value of a macro: #'clojure.repl/doc"\n :via\n [{:type clojure.lang.Compiler$CompilerException\n :message "java.lang.RuntimeException: Can't take value of a macro: #'clojure.repl/doc, compiling:(NO_SOURCE_PATH:0:0)"\n :at [clojure.lang.Compiler analyze "Compiler.java" 6704]}\n {:type java.lang.RuntimeException\n :message "Can't take value of a macro: #'clojure....

13:06 sdegutis: Oh.

13:11 justin_smith: ,(defn ☃ [& args] :❄❄❄❄)

13:11 clojurebot: #'sandbox/☃

13:13 justin_smith: ,(☃ "⛷")

13:13 clojurebot: :❄❄❄❄

13:15 tjmaynes: ,doc println

13:15 clojurebot: #error {\n :cause "Can't take value of a macro: #'clojure.repl/doc"\n :via\n [{:type clojure.lang.Compiler$CompilerException\n :message "java.lang.RuntimeException: Can't take value of a macro: #'clojure.repl/doc, compiling:(NO_SOURCE_PATH:0:0)"\n :at [clojure.lang.Compiler analyze "Compiler.java" 6704]}\n {:type java.lang.RuntimeException\n :message "Can't take value of a macro: #'clojure....

13:15 tjmaynes: clojurebot is so cool

13:15 justin_smith: ,(doc println)

13:15 clojurebot: "([& more]); Same as print followed by (newline)"

13:15 tjmaynes: ,(doc get)

13:15 clojurebot: "([map key] [map key not-found]); Returns the value mapped to key, not-found or nil if key not present."

13:16 mavbozo: ,(def ∇ [& args] :nabla)

13:16 clojurebot: #error {\n :cause "Too many arguments to def"\n :via\n [{:type clojure.lang.Compiler$CompilerException\n :message "java.lang.RuntimeException: Too many arguments to def, compiling:(NO_SOURCE_PATH:0:0)"\n :at [clojure.lang.Compiler analyzeSeq "Compiler.java" 6891]}\n {:type java.lang.RuntimeException\n :message "Too many arguments to def"\n :at [clojure.lang.Util runtimeException "Util.jav...

13:16 mavbozo: ,(defn ∇ [& args] :nabla)

13:16 clojurebot: #'sandbox/∇

13:16 mavbozo: ,(∇)

13:16 clojurebot: :nabla

13:29 zenoli: g=Spam

13:29 Whoops, sorry.

13:30 tjmaynes: (def m {:username "sally"

13:30 :profile {:name "Sally Clojurian"

13:45 justin_smith: tjmaynes: is that from clojurebridge?

13:50 sdegutis: I'm confused. How do you keep a production system Component running without exiting the app, considering aren't systems always async?

13:50 justin_smith: what's async about a system?

13:50 and why would that make the app exit?

13:51 sdegutis: Like, if all you have is a jetty-component, then it starts up in another thread. Then system/start will exit. And if that's the last call in your -main function, your program exits, right?

13:51 justin_smith: sdegutis: if that was true, we wouldn't need

13:51 ,(doc shutdown-agents)

13:51 clojurebot: "([]); Initiates a shutdown of the thread pools that back the agent system. Running actions will complete, but no new actions will be accepted"

13:52 sdegutis: Okay.

13:52 justin_smith: if you have threads running, the vm sticks around for them - I think there is some magic to declare some threads are ones that shouldn't prevent shutdown but I forget how any of that works.

13:52 sdegutis: I see.

13:52 amalloy: justin_smith: Thread/setDaemon

13:53 justin_smith: amalloy: thanks!

13:53 amalloy: but don't do that to threads from clojure's threadpools

13:53 justin_smith: right, that seems like it would be a bad thing to do

13:53 presumptuous, downright rude

13:55 sdegutis: justin_smith: Oh cool!

13:55 justin_smith: So yeah my (finally) clause was just getting called. Phew!

13:57 tolstoy: I add a shutdown hook to cleanup that kind of running service (or whatever).

13:58 (doto (Runtime/getRuntime) (.addShutdownHook (Thread. (fn [] (http-server/stop ref)))))

13:58 Or, basically, to call component/stop. ;)

13:59 And use a promise to keep the app running, then a shutdown hook to (deliver lock :release) and the app falls out.

13:59 justin_smith: tolstoy: that's very elegant

14:00 tolstoy: I can't figure out a way to make it simpler. When using "lein trampoline run" you can see your shutdown messages for each components. For some reason, that gives me confidence.

14:01 justin_smith: tolstoy: it's nice to be able to go find that output in the server logs too

14:02 when I'm reading a log, often the first thing I look for are where the components stopped, where they started, as the delimiters of bringing the app up

14:02 tolstoy: I wonder, does anyone configure their logs (logback, say) to output clojure maps?

14:02 justin_smith: tolstoy: I talked with gfredericks about doing this with json, but since then I've had experience with transit which makes me think outputting transit instead wouldn't be so bad

14:03 tolstoy: I worked on a legacy app that use Tanuki to wrap stdout to a log file.

14:03 justin_smith: tolstoy: also, kafka is effectively a machine parsible log (as well as being a messaging service) -- the messages are persisted on disk, so it becomes a very nice way to track events

14:03 tolstoy: Each line of a stack trace got its own timestamped line. Oy.

14:03 justin_smith: ouch

14:04 tolstoy: Yeah. ;)

14:04 We needed to do just the things you were saying. When were the restarts? Etc.

14:04 justin_smith: tolstoy: the nice thing with kafka has been not only can we send messages that act as logs of events, but also the replayable event sequence is very helpful when you want to know where things broke down / what input made the server barf / etc.

14:05 sdegutis: Finally got Component installed in our web app! All it does though is use the jetty-server component, but hey it's a start!

14:05 tolstoy: I wrote a "de tangler" to parse it all into Clojure maps, then a web app to display / search them. Mostly kinda worked. Helped to parse out thread-dumps as well.

14:05 justin_smith: nice

14:05 tolstoy: So, why not just output to a key/value data structure in the first place! I tell you, even if everyone else has had the idea, it still feels fun to have it.

14:08 Example main with shutdownhook and component: https://gist.github.com/zentrope/0bdc313c32193cb66743#file-main-clj-L68-L90

14:09 Nowdays I add a UncaughtExceptionHandler by default. Handy.

14:10 I need to add the "when you see an OOM, tank the app" clause.

14:13 justin_smith: tolstoy: * dump all stack traces of all threads, and tank the app *

14:13 tolstoy: Yeah.

14:14 justin_smith: actually I am seriously considering a loop that sleeps and sends all app stacktraces to a kafka channel on a periodic basis

14:14 for diagnostics

14:14 tolstoy: Had a case where one of the components just stopped with an OOM (due to the Linux OOM killer -- argh), but the app kept running, so the monitor had no idea.

14:14 justin_smith: I wonder how hard it would be to also throw in "used memory" of the runtime...

14:14 tolstoy: Hm.

14:15 Shell out and run a non-interactive "top" thing?

14:16 justin_smith: there's totalMemory and maxMemory but I don't see usedMemory - but this must exist, because you can get it from profiling data...

14:16 tolstoy: Or something that pulls via JMX. I don't know how you figure out how much non-heap memory a JVM is using.

14:16 (From within the JVM.)

14:16 amalloy: justin_smith: totalMemory-freeMemory

14:16 justin_smith: ahh

14:16 so there is a freeMemory, nice

14:22 OK, I just had a brainstorm, and am going to make a standalone lib that runs a ScheduledExecutorService that dumps monitoring data to kafka

14:22 woot

14:31 sdegutis: justin_smith: never heard of brainstorm as a noun before.. I like it!

14:32 justin_smith: why SchedExecService?

14:32 justin_smith: is it so that it can run every so often?

14:48 justin_smith: sdegutis: precisely - the idea is I find out something is messed up, I find out when the symptom is seen, then I go and read the metrics for the five minute span before and after the reported problem

14:48 looking at memory usage, which code is being run, etc.

14:49 and using kafka allows this to hook into our existing messaging system and lets kafka take care of stuff like storage and cleanup and combining the data from all our instances in one convenient place etc.

14:49 sdegutis: justin_smith: nice!

15:05 jonathanj: what's the most efficient way to turn a seq of strings into an input-stream?

15:06 (input-stream (.toBytes (str the-seq))) is pretty concise but i don't know how efficient it is

15:07 arrdem: Bronsa: nice I hadn't seen that

15:07 jonathanj: it's kind of weird that there's no method in enlive to emit to a stream

15:09 oh i guess it should be (apply str the-seq)

15:22 is it considered good etiquette to have quite large -> blocks or is it better to use (let) and assign some intermediate values?

15:23 AimHere: My guess is whichever of the two is more readable

15:34 jonathanj: some mixture of the two produces pretty nice results

15:34 thanks, AimHere

15:35 arrdem: good lord java.util.Queue involves a lot of crap

15:36 TEttinger: arrdem, are you reading the source?

15:37 arrdem: TEttinger: no I'm rolling an LL(1) parser by hand, did a Reader that tracks like, coll, file offset and tried to do a wrapper that looked like a queue for peek/pop

15:37 s/like/line/g

15:38 TEttinger: mmmm. queue being a mess of interfaces would be trouble then.

15:38 arrdem: yeah. wound up just ignoring the queue interface and writing peek/pop

15:38 otherwise would have had to implement a dozen abstract methods as UnsupportedOperationExceptions

15:40 sdegutis: Is there a rule of thumb to know what to avoid when trying to write code that's reload-friendly?

15:40 I think the rule is to avoid "anything that composes functions into a new function".

15:41 Like partial and comp.

15:41 But that doesn't explain why defroutes doesn't respect function redefinition.

15:42 Oh I think I understand why: it's because it captures the value of the inner function when you use an existing defroutes inside a defroutes, not the var.

16:35 whodevil: hello, I just created a project using the compojure template that is in lein. I'm not sure what the dev-resources directory is for. Does that somehow get combined with my regular resources during the dev profile or something? For example, should my js tests, and sass stuff live in there?

16:43 justin_smith: whodevil: resources that aren't code, and are not part of the app. Things like data used during testing.

16:44 so yeah, I could see those things going there - as long as you don't want them to ship when you deploy the jar but still want them in the project

16:44 whodevil: justin_smith: ok, so should I put my js tests in there? or am I just doing js totally wrong...

16:44 justin_smith: whodevil: if you are using cljs and cljs.test that can go in /test, but for js stuff, that would make sense to put it there

16:45 whodevil: hmm, I wonder if this will screw up my jest tests if the tests are in a totally different location.

16:45 justin_smith: but this is all kind of open ended, as long as you remember that things in dev-resources are not meant to be in the deployed code

16:45 whodevil: justin_smith: that makes sense to me

16:45 justin_smith: I'm sure it can be configured somewhere (I have no idea how your testing is set up of course)

16:45 kenrestivo: i vaguely remember their being a way to have an nrepl running in a uberjar along with an app, but i long since forgot how to do it

16:46 whodevil: I would dig on using cljs but using clojure for my work is a stretch at this point, they want me to stick to the rivers and lakes that I'm used to when it comes to UI

16:46 kenrestivo: i'd like to be able to connect to a running app to debug it from the repl, but it's deployed as a uberjar

16:46 justin_smith: kenrestivo: make clojure.tools.nrepl a dep, and start up an nrepl server (either based on getting a hit on some endpoint, or on app startup)

16:46 kenrestivo: justin_smith: thanks

16:46 justin_smith: it's something like clojure.tools.nrepl.server/start-server

16:46 it's easy, the readme shows how to do it

16:47 just make sure you don't allow access to the port except from localhost, and use ssh to tunnel your connection in

16:47 since access to nrepl means effectively having root on your box and all

16:47 (ever OS has local root holes)

16:47 *every

16:47 sdegutis: Is it possible to get a java.io.File from a resource inside a jar?

16:47 s/inside my own jar/

16:48 justin_smith: sdegutis: no, but you can get resource, which should be good enough unless you planned on writing to it

16:48 sdegutis: I'm using an API that requires a java.io.File :(

16:48 justin_smith: using clojure.java.io/resource - you can read / slurp from that

16:48 sdegutis: tell that API it's stupid

16:48 sdegutis: I'll let Amazon know :D

16:48 justin_smith: sdegutis: you can also use the resource in order to write to a temp file

16:48 sdegutis: Ahh hmm.

16:48 justin_smith: but yeah, dumb

16:49 sdegutis: Right on.

16:49 I was kind of hoping Java had an in-memory subclass of File.

16:49 Like java.io.MemoryFile

16:49 That would basically wrap StringStream

16:49 justin_smith: even then, you would need the resource first

16:49 sdegutis: or whatever java calls it

16:49 Yeah I have the resource just fine. I get it using io/resource

16:54 kenrestivo: is there any more elegant way to pass a map to a function that wants args as (some-func :foo 1 :bar 2) than (apply some-func (apply concat some-map)) ?

16:55 sdegutis: kenrestivo: nope

16:55 kenrestivo: :(

16:55 sdegutis: kenrestivo: a PR to the author is usually best at that point

16:55 kenrestivo: haha agreed

16:55 arrdem: Fix some-fn to not take varargs and take a real options map? :P

17:01 amalloy: ~mapply

17:01 clojurebot: You have to do something like (defn mapply [f & args] (apply f (apply concat (butlast args) (last args)))), which just goes to show why unrolled keyword args are a bad idea

17:01 arrdem: agreed

17:43 OscarZ: hi guys.. i was checking out the core.async walkthrough here https://github.com/clojure/core.async/blob/master/examples/walkthrough.clj

17:45 at line 57 it seems that the go block itself returns a channel.. and then a blocking take (<!!) is made.. im just wondering why its done like that

17:47 justin_smith: OscarZ: go blocks always return a channel

17:48 OscarZ: and I think that example is set up like that just to show that fact

17:51 OscarZ: justin_smith, i mean this part: (<!! (go (<! c)))

17:51 justin_smith: right

17:52 they do a blocking read of the result of a go block

17:52 which is silly in real code, but in that example, it demonstrates that a go block gives you a channel and you can get data from it

17:53 OscarZ: why is it that inside go block they use the non-blocking <! read.. and then outside again a <!!.. its just a bit difficult to understand why it is done like that

17:53 justin_smith: (silly because they could have just done a blocking read on c, of course)

17:53 OscarZ: inside a go block one should use <!, and outside you can't use <!

17:53 <! only exists inside go blocks

17:53 OscarZ: yes, that i gathered from the docs..

17:54 im just wondering if theres a legit use for that kind of structure?

17:54 justin_smith: if you already know that, I don't understand your question

17:55 OscarZ: i already knew (at a superficial level) that you can only use <! inside a go block.. <!! can be used outside.. i guess i dont really understand their difference

17:55 justin_smith: OscarZ: it's a toy example that like most toy examples isn't the way you would use those things in real code - obviously there would be a simpler way to get the string "hello" if that's all you wanted, the point here is to show how to read a channel outside a go block, how to read a channel inside a go block, how to read the channel that is returned by a go block

17:56 OscarZ: <!! blocks

17:56 <! lets core.async run different code and it will wake you up when there is something on that channel, and it doesn't block a thread

17:57 but core.async is only able to do that feature inside a go block

18:00 tolstoy: Huh. Empty stacktrace with an NPE. That's new (for me).

18:00 kenrestivo: thanks, a little boilerplate and it's in. very nice https://www.refheap.com/112076

18:01 justin_smith: kenrestivo: cool

18:01 OscarZ: ok.. i guess i understand that.. i was just wondering if that (<!! (go (<! c))) part is some idiomatic way to do some black magic inside a go block and then return the results back to "real world" :)

18:01 justin_smith: OscarZ: no, like I said, it's a toy example for the walkthrough, that's not normal at all

18:02 OscarZ: sure ok.. thanks for clarifying that..

18:02 justin_smith: if that was all you had in your go block, of course it would be better to just do (<!! c)

18:02 that gives the same result, in a much simpler way

18:03 OscarZ: that stuff is somehow very difficult to understand.. im just thinking where the execution really goes heh..

18:03 tolstoy: OscarZ: I've used that feature as a kind of "futures" thing where I create a bunch of go blocks (in a vector). Each go block does some work, but when it's done, returns :done. I then "alts!" on the vector of go blocks to wait for it to finish.

18:04 spieden_: OscarZ: go returns a channel that eventually receives the last expression evaluated inside the go block and then closes

18:05 sdegutis: Does (close! chan) not work from the sender's side?

18:06 OscarZ: are there some programming concepts etc. that are especially helpful to be familiar with when learning stuff like core.async?

18:07 justin_smith: OscarZ: you could look at CSP - it's a very simple formal model that core.async implements

18:07 OscarZ: https://en.wikipedia.org/wiki/Communicating_sequential_processes

18:07 OscarZ: i have some exposure with functional programming.. and some with asynchronous programming.. its mostly from javascript stuff though

18:08 sdegutis: Hmm.

18:08 It seems (close! chan) doesn't work from /either/ side.

18:08 justin_smith: check out the wikipedia for CSP I linked above, it might at least make it clear what you were missing

18:08 OscarZ: its mostly surviving in the callback hell

18:08 justin_smith, ok thanks.. ill check it out

18:10 does core.async have something in common with something like FRP, i've looked at a javascript library called Bacon.js which has some interesting abstractions

18:10 justin_smith: both have to do with message queues on a very abstract level, but CSP is not the same as FRP

18:12 OscarZ: ok.. ive listened to some podcasts about them, not understanding maybe all, but both seem to address "callback hell" which is all too familiar to me

18:13 justin_smith: yes, callbacks are a somewhat clumsy way to handle asynchronous operations, and CSP and FRP are both usable approaches to async that are less convoluted

18:14 OscarZ: sure ok.. ive read a bit about FRP, event streams and stuff, it makes sense to me.. i should learn about CSP :)

18:18 can you think like FRP event stream is like a core.async channel or are there some fundamental differences between these?

18:19 these are still a bit vague for me..

18:20 both seem to abstract some kind of "pipe" where you put stuff

18:20 justin_smith: OscarZ: true, and both have immutable contents

18:21 OscarZ: but frp has the concept of a "continuous" stream of data, and also of laziness where data is only calculated if something downstream uses it

18:21 core.async doesn't really have either

18:23 that laziness is also known as "pull based" - core.async is all push based

18:24 OscarZ: interesting.. im not sure what you mean by "continuous" here though..

18:24 justin_smith: OscarZ: it's an illusion - but the concept in FRP (as originally described) was that the sampling rate was an implementation detail, and this datatype was a continuous curve where you could get a value at any point in time

18:25 eg. wind speed, or sound pressure, or flow - things that have values at any point in time, and the consumer decides how to sample them

18:25 as opposed to events, which are not continuous, they happen at specific times

18:26 OscarZ: another difference between FRP and CSP is that CSP doesn't have the same strong concept of time, or events propagating at a specific time in the system

18:29 OscarZ: cool explanation.. in bacon.js it really seems the idea is that you set up the recipes and then the callbacks you provide will get the results.. so it really is pull based

18:39 i guess in FRP (im thinking Bacon.js), you have "properties", things that have state in time.. do you think this could be part of core.async somehow or is it mixing up things ?

18:40 i know the clojure philosophy is not to mix concepts if they are really separate

18:40 justin_smith: OscarZ: I'm not sure - but yeah, I think those properties are likely a version of that original idea of continuous streams

18:41 maybe there's a way to combine core.async and FRP though, I'd have to do some reading and thinking before I could answer that I'm sure

18:53 OscarZ: justin_smith, Bacon.js has this idea of "flatMapLatest".. if you have some async requests or something and you dont know when the results will come, but you always want to react only to the latest result and scrap the earlier ones..

18:57 justin_smith: yeah, that sounds cool

18:57 almost like a pull version of a debounce

18:57 OscarZ: i guess if the stuff that gets put in core.async channels had timestamps with them, you can do stuff like that.. and it could be useful if the library would provide you that

18:58 justin_smith: OscarZ: I think you can do similar with a dropping buffer

18:58 OscarZ: the channel would just drop old messages each time you send, so each pull from the channel only gets the latest event

18:59 OscarZ: my mistake, that's sliding buffer (dropping-buffer drops the newest, sliding-buffer drops the oldest)

19:00 OscarZ: https://clojure.github.io/core.async/#clojure.core.async/sliding-buffer

19:00 tolstoy: Isn't core.async a lower-level library? You'd build FRP on top of it, right?

19:00 justin_smith: tolstoy: they aren't totally compatible I don't think - I could be wrong though

19:00 OscarZ: cool ok.. im not familiar with either :) but i guess the buffer has to keep track of time right?

19:00 justin_smith: tolstoy: but yes, CSP is a lower level thing than FRP

19:01 tolstoy: I can'

19:01 t give citations at the moment, but what I seem to recall is that FRP allows operations that are not allowed in CSP, and visa versa

19:01 and the disallow of these things ensure the absence of deadlocks / bad reads

19:02 tolstoy: Ah. I'm probably a little confused given that "reactive" as a term has become kinda broadly applied.

19:02 justin_smith: tolstoy: I think FRP is a special kind of Reactive Programming, and the latter might be compatible with CSP? Still not certain though.

19:03 OscarZ: tolstoy, i feel like that too... that you could build stuff like in Bacon.js on top of core.async.. but i dont really know if it can be done

19:04 justin_smith, i see.. it would be interesting to hear where they will clash

19:04 tolstoy: OscarZ: I use core.async mostly as "queues". For instance, people click something in a browser, I plop that into a channel, then process it back at the main "event loop".

19:05 OscarZ: tolstoy, cool, do you only use it client-side with clojurescript in your app ?

19:05 tolstoy: My guess is you have to take core.async on its own, forgetting every other framework. Either you end up not needing that other pattern, or you begin to see how it all ties together in a family.

19:05 OscarZ: I also use it server side. For instance, handling web-sockets.

19:06 OscarZ: tolstoy, cool :)

19:06 justin_smith: OscarZ: I use sente, which is a wrapper for websockets where each message from the server becomes a core.async message I can consume / propagate to the app.

19:06 and same on the server side, the reply from the client also shows up as core.async message sends

19:06 tolstoy: I used to use Sente, but ended up fighting it too much. But what I do is basically the same thing.

19:07 justin_smith: tolstoy: things went swimmingly for me, until I got messages big enough that I had to split them, and couldn't get at the right internals to do splitting right ... still have not solved this and might end up dropping sente and just using websockets and core.async directly

19:07 OscarZ: ive been thinking to experiment with things like that.. nice to hear it can be done :)

19:08 tolstoy: justin_smith: My issue was with authentication. I didn't want to have to use a normal REST route to do that, so I piggybacked on the web-socket "protocol" for that.

19:09 OscarZ: I've also used to to stream database queries over the web.

19:09 OscarZ: You create a channel and return that to the caller which blocks "reading" from it. Then in a go block, you run your cursor-based jdbc query, dumping results into the channel.

19:09 OscarZ: i havent done any stuff with websockets yet.. but what benefits does using core.async bring you there?

19:10 tolstoy: OscarZ: On the web side, you output "{", then read each row from the channel and output it to the output stream, then "}" at the end.

19:10 justin_smith: OscarZ: it gives a convenient way to have a multi-step computation that requires interaction between client and server at multiple stages without callbacks getting insane.

19:10 tolstoy: OscarZ: This allowed me to serve up large data sets without blowing up a 512MB virtual server.

19:11 (Requires an amenable JSON format, of course.)

19:11 OscarZ: sounds cool :)

19:12 tolstoy: justin_smith: I think the web-socket stuff is simple enough that a large, generalized lib isn't all that necessary.

19:12 justin_smith: OscarZ: like, I can describe a process where the client asks the server what's available, the server offers a selection, the client puts that selection on the screen, the user clicks one of them, and sends the result to the server. And instead of these being a spaghetti or nest of callbacks, it can just read as a linear sequence of steps.

19:12 tolstoy: justin_smith: Worst case (for me, anyway), is that I end up really understanding what I need, and can then leverage a library more appropriately.

19:12 justin_smith: each communication is a channel send followed by waiting on a channel

19:13 tolstoy: yeah, I'll probably end up using websockets directly eventually

19:14 OscarZ: justin_smith, oh.. i see.. i thought it was about freeing up threads etc. resources..

19:16 is any of stuff you guys are describing in github or something?

19:17 justin_smith: OscarZ: on the js side there are no threads to clear up :P

19:18 OscarZ: sorry to say the interesting stuff on my side is all in a closed source app

19:18 OscarZ: justin_smith, heh.. yes.. i was thinking about server-side :)

19:19 justin_smith: OscarZ: the real benefits of core.async on the server side are things like being able to control backpressure (eg. limit the number of tasks a given server takes on at a time)

19:19 and also freeing up threads for parked blocks of course

19:19 tolstoy: OscarZ: My clients are kinda like game apps. Every event (socket data, http request, user click) goes into a single channel. I read the msg, update the state, which triggers a rebuild of the UI. That's it.

19:20 OscarZ: ok.. cool

19:22 tolstoy: OscarZ: It's kinda dated, but here's a "pong" game I did in CLJS and CLJ: https://github.com/zentrope/pong-2.

19:23 OscarZ: what if on server side i had to do things like A, B1, B2... B1 has to be done before B2, but A has doesnt have restrictions like that.. it can be done any time.. could you use core.async for that sort of thing ? i guess typically they are just done sequentially in one thread

19:23 tolstoy: OscarZ: I think the idea is that each player has a browser that is just the paddle, and another browser that shows the game of pong itself. Can't remember. ;)

19:24 OscarZ: http://swannodette.github.io/2013/07/12/communicating-sequential-processes/

19:25 OscarZ: tolstoy, that seems awesome, so conscise :) you got the working version of pong somewhere?

19:26 tolstoy: It's not deployed.

19:26 I think it even has "sessions". You go to it, it provides a number. That's how two people hook up to play the same game.

19:26 OscarZ: ok, thanks anyway.. ill be sure to study it

19:28 tolstoy, out of interest, what languages/techstacks did you use before clojure/clojurescript?

19:29 tolstoy: Hm. Pascal -> Perl -> PHP -> Python -> Java -> Common Lisp -> Erlang -> Clojure -> Scala -> Clojure. ;) (Lots of overlapping Java of course.)

19:30 Hm. I didn't even print out the port that app runs on.

19:30 OscarZ: interesting.. Erlang would be cool to check out too

19:33 tolstoy: Heh. That pong-2 thing still works!

19:34 OscarZ: where is it? :)

19:34 tolstoy: Just running locally.

19:36 OscarZ: for me the languages were: Basic (C64) -> C (very badly) -> MC68k assembler (Amiga) -> Java (for years, OO was the most fundamental structure of the universe lol)

19:37 now i've had plunges at Haskell, Clojure, Scala..

19:37 tolstoy: I'd love to do Haskell, but get stuck at JSON. ;)

19:38 OscarZ: yeah.. Haskell is really cool

19:38 justin_smith: OscarZ: I'm re-plunging into haskell myself lately (not that I'm done with clojure, but I want to really know how to program in haskell, and sometimes I get frustrated with problems in part of my clojure app that haskell would handel much better)

19:38 tolstoy: did you check out Aeson?

19:39 tolstoy: Nope. It's been awhile.

19:39 OscarZ: back then i chose to try to learn Haskell to learn functional programming, it was good in that sense.. great community too here in IRC, really friendly

19:39 justin_smith: tolstoy: the idea with Aeson is that there is a data type which can potentially be of any of the valid json datatypes, and then the compiler makes sure that your code won't fall over and die if the wrong kind of value is found under your key...

19:40 OscarZ: in real world im "doomed" to JVM environments.. i need to have at least some kind of interaction even to propose using something like that

19:41 justin_smith: OscarZ: yeah, I think that's a big part of Clojure's niche

19:41 tolstoy: OscarZ: There's frege....

19:41 justin_smith: tolstoy: how usable is frege?

19:41 tolstoy: No idea.

19:41 I think seancorfield has some experience with it.

19:41 OscarZ: justin_smith, thats cool.. i know some hardcore Haskell dudes that have high regards of Clojure :) so its not a one way street

19:43 i guess its mostly because of the huge static/dynamic differences of Haskell/Clojure .. surely some applications benefit from

19:43 being static / dynamic..

19:45 i myself feel that i like the dynamic "data-in-your-face" approach of Clojure more though..

19:46 justin_smith: I don't think there's hash-map and set literals for haskell

19:47 OscarZ: sorry.. i was actually thinking more like Java/OO vs Clojure there...

19:47 Haskell is pretty data-oriented too

19:48 justin_smith, oh.. thats pretty bad if thats so. It's been a while :(

19:49 amalloy: literals for such things are a lot less important if you never manipulate source code as data

19:49 fromList [1,3,5] is one common way to make a set

19:49 justin_smith: amalloy: fair enough

19:49 amalloy: which is obviously more typing, but works okay for haskell

19:50 justin_smith: yeah, you make up for it with missing delimeters easy, if one keeps score

19:50 amalloy: in clojure that wouldn't work because having set literals that macros can manipulate as sets is important

19:50 tolstoy: OscarZ: devtrope.com:3001/p2

19:50 OscarZ: gotta go for a while.. thanks for the info on core.async etc.. guys :)

19:50 amalloy: justin_smith: we are doing a weekly book club on LYAH at work

19:50 tolstoy: OscarZ: and devtrope.com:3001 for the main "pong" display.

19:50 amalloy: did SICP fir a couple months

19:51 justin_smith: amalloy: very cool; I did the same at my last workplace (same book even). I am checking out bitemyapp's book right now

19:51 it's actually good, much less abrasive than one might expect

19:51 amalloy: well, i've read it before. i do find it a little cloying, but still pretty good and better overall than being dry

19:52 justin_smith: yeah, I mean, we started using LYAH but only had a few sessions before that situation fell through, and only a little of it stuck for me.

19:52 amalloy: there has been some attrition as we get further into the book

19:52 justin_smith: I'm still very much a neophyte with hs, but want to have it as a tool in my toolbox when it's apropriate

19:53 amalloy: not at all surprised

19:54 tolstoy: bitemyapp's book is out?

19:55 justin_smith: tolstoy: still preview, I did the option where I pay now, read as I go, get the final version when it comes out (and any updates in between)

19:56 amalloy: is his book as abrasive as he was in here?

19:56 i didn't even know he had a book

19:56 justin_smith: amalloy: "16:51 < justin_smith> it's actually good, much less abrasive than one might expect"

19:56 :P

19:57 amalloy: i can't be expected to keep up with every little thing that gets said in a room with so many people talking at once

19:57 justin_smith: amalloy: the book is actually good, very clear and accurate, the co-author has a background in education, so it shows

19:57 amalloy: ha

19:58 justin_smith: *and it shows

21:02 simon1234: Hi there! I have a question about java interop. I'm defining a deftype implemeting an interfaces that has overloaded method with same arity, same name but different type. I specified the type but I get a "Mismatched return type: write, expected: void, had: java.lang.Object" ; and I'm not sure it is going to work if I remove all type hints... any ideas? Thanks a lot!

21:04 amalloy: different return type but same argument types?

21:04 i don't think that's possible

21:05 simon1234: amalloy: no different argument types, but same return type (void)

21:05 (thanks for your answer!)

21:05 amalloy: then you have to hint the return value as void too

21:05 if you hint everything, you have to hint everything

21:06 er, if you hint anything

21:09 simon1234: amalloy: Ah thanks, that was it! I had to type the return value too, thanks a lot!

21:16 Malnormalulo: What is Clojure's equivalent of Java's instanceof?

21:16 tolstoy: instance?

21:16 Malnormalulo: danke

21:17 nornagon: So, which of '[clj-http http-kit http.async.client aleph] should I use?

21:24 tolstoy: Wow, I don't know. Those all seem reasonably current, eh? http-kit is nice and simple and has no dependencies. Aleph lets you interact with web-sockets even on the server side.

21:42 nornagon: Yeah, hence the confusion :) clj-http looked initially like the best-liked one, but on closer inspection it seems not to have any async support. http-kit seems newer but more powerful.

21:42 I'm using just the client-side stuff, not writing a server (atm.)

21:43 tolstoy: Maybe just put your usage in a namespace, then you can swap out if it seems necessary?

21:44 nornagon: I guess so. It's nice when a language has a clear recommended option for something as simple as executing an HTTP GET though :) I'm surprised there's no clojure.http

21:51 tolstoy: I rarely need to make requests, and almost never in an async context. But I like Aleph mainly because the whole stack ztellman provides has tcp, udp, interesting "deferred" and "stream" implementations that interoperate with core.async, etc.

21:52 I think it requires a bit of conceptual investment, though.

21:52 http-kit hasn't been updated in a while.

21:55 copycat: hi, i'm looking at code with a function that accepts an argument of [{:keys [snake pills] :as world}]

21:55 could anybody help me interpret this?

21:56 the function is called somewhere else later as (render world)

21:56 tolstoy: copycat: that's destructuring a map.

21:57 justin_smith: ,((fn [{:keys [snake pills] :as world}] (str "snake" snake "pills" pills "world" world)) {:snake "scary" :pills "healthy" :status "OK"})

21:57 clojurebot: "snakescarypillshealthyworld{:snake \"scary\", :pills \"healthy\", :status \"OK\"}"

21:57 justin_smith: oops! that's ugly

21:57 tolstoy: Heh.

21:57 justin_smith: ,((fn [{:keys [snake pills] :as world}] (println "snake" snake "pills" pills "world" world)) {:snake "scary" :pills "healthy" :status "OK"})

21:57 clojurebot: snake scary pills healthy world {:snake scary, :pills healthy, :status OK}\n

21:57 justin_smith: copycat: ^ that make any sense?

21:58 tolstoy: If you call that function with {:snake "slither" :pills "pop"}, the body of your function will have values named snake, pills and world. World is the whole map, and snake/pills are bound to the values of those keys in the map.

21:58 justin_smith: the destructuring makes a concise way to get values out of data

21:58 copycat: so world is a map, containing the keywords snake and pills?

21:58 tolstoy: Yes.

21:59 (At least.)

21:59 justin_smith: right, plus any others you would like to provide

21:59 yeah

21:59 copycat: and in your example, the value snake contains the value "slither"?

21:59 tolstoy: Yes.

21:59 It's like (let [snake (:snake world) pill (:pill world)] ..... )

21:59 copycat: okay, so i guess the enxt step in reading the code is to see where the author sets the value of snake and pills in world

22:00 alright, thanks guys :)

22:02 tolstoy: What I like about that destructuring style is that it's also a map: {:keys [a b] :as c}.

23:28 kenrestivo: nrepl is amazing. it's fantastic to have a deployed uberjar, and be able to nrepl into it and debug it live

23:29 tolstoy: kenrestivo: I think they've added it natively in 1.8.0.

23:30 kenrestivo: or rather replaced it with a different socket thing, IIRC

23:30 tolstoy: Right. socket repl. Not the same thing.

23:31 kenrestivo: as long as i can reach it from emacs, i'm happy :)

23:31 tolstoy: http://dev.clojure.org/display/design/Socket+Server+REPL

23:32 I love that kind of thing. I even made a "web repl" thing once that served up a web app into your running code.

23:32 I think I did that to demo some internal whatever as a way to overcome the crushing boredom that sprint reviews are for me.

23:33 kenrestivo: nice

23:33 tolstoy: Let me light some black candles and hang a goat's head from the light fixtures and tell you about this exciting logging change we accomplished this week.

23:54 arrdem: hoboy

Logging service provided by n01se.net