#clojure log - Feb 16 2015

The Joy of Clojure
Main Clojure site
Google Group
List of all logged dates

1:38 julianleviston: Hi, question about core async patterns. I have a fanning pattern. Each time my ‘retrieve’ function is called, it gathers a set of ids, starts a go block (therefore the main function call finishes), which runs a ‘retrieve associated’ function on its results, which then calls back into ‘retrieve’ for each of those items (ie recurses). I’d like to pass a ‘done chan’ into the main retrieve function, but not

1:38 have it wait around until all its “children” are done… so is there a way I can start another go block all it does is wait around unti the n children have finished and send “:done” back to the done chan? Will that pattern work ok, do you think?

1:41 mostly I don’t care if the fanning operation is done, but very ocassionally I need to sync up some work at the end, and so I’d wrap the done chan put in a check-for-presence conditional

1:42 I might go try it out and see how I go… lol… thansk for being a sounding board, channel :)

1:58 Kneiva: no prob =)

2:07 julianleviston: Kneiva: lol

4:27 ro_st: there's no point to using tomcat to host a jar if that jar doesn't accept http connections, right?

4:27 for that, you'd simply java -jar app.jar inside an upstart script or whatever?

4:37 cmdrdats: ro_st: unless you already have tomcat going and don't want to manage another app's lifecycle

4:39 ro_st: new nodes, so wondering if it's worth the bother

4:39 does it make sense to use tomcat for non http apps?

4:40 tomcat is to java as apache is to php, no?

4:41 Empperi: ro_st: kinda, but not quite

4:41 and in my opinion if you don't need HTTP server functionality then you don't need Tomcat

4:41 and you shouldn't use it

4:41 cmdrdats: if you're not using any of the servlet deployment stuff, then probably pointless to use tomcat..

4:41 snap :)

4:42 Empperi: comparing java and php is kinda pointless anyway

4:42 ro_st: cool. no tomcat then. we already use upstart to java -jar wossname.jar so we'd continue to do that

4:42 Empperi: let's not go there :-)

4:42 Empperi: php has been designed from ground up to be used in applications which are very short lived

4:42 and java is the opposite

4:42 on average a php program lives as long as the http request which is handled by apache/nginx/whatever

4:43 ro_st: i know.

4:43 Empperi: but yeah, you'd just complicate your life with Tomcat :)

4:43 ro_st: cool. thanks

4:46 dysfun: does cemerick get on irc much these days?

4:48 ro_st: it's sleepy time in his timezone

4:49 dysfun: ah

4:49 * dysfun thought everyone left irssi running on a screen session somewhere these days

4:58 txus: hi there!

5:02 jballanc: txus: howdy!

5:02 dysfun: weechat with a bouncer for me...but at least that sets /away when I disconnect

5:03 dysfun: i've thought of using a bouncer, but i *like* irssi anyway

5:03 when i had a client who used irc, i used colloquy. a bouncer would have been nice for external networks

5:04 (mostly because colloquy supports growl and it was useful for mentions)

5:05 zacts: I'm going to try znc for myself

5:05 scottj: you could setup irssi to do that, even over ssh and screen. fnotify is the irssi module to look at.

5:05 zacts: mainly so I can have snappy keyboard response time

5:05 dysfun: yeah, i know, but i'm lazy :)

5:05 zacts: there is mosh though

5:06 dysfun: tbh i usually don't notice any lag on ssh to this box

5:06 if the internet is having a bad day, maybe it's a little annoying, but it's rare

5:06 zacts: well mosh is cool for this

5:06 dysfun: mosh doesn't do tunnelling, which i need for access to some databases

5:07 zacts: ah

5:13 txus: yay first time in the clojure channel

5:13 how are y'all??

5:13 lazybot: txus: Uh, no. Why would you even ask?

5:16 hyPiRion: oh lazybot, why so mean

5:17 lazybot answers anyone ending their sentences with ??? or ??

5:17 lazybot: hyPiRion: Definitely not.

5:24 noncom: i have just stumbled upon method resolution issue. there is a method ` java.nio.file.Paths.get(String first, String... more) ` , and if i call it like (Paths/get "/some/path"), clojure will fail to identify it correctly

5:24 it will confuse the method with ` java.nio.file.Paths.get(URI uri) `

5:25 solution is to call the required method by (Paths/get "/some/path" (into-array String []))

5:25 then it will find the one i really need

5:25 so, is this behavior ok? is it considered normal? or should i probably file an issue on jira for that?

5:31 dysfun: this is curious: java.io.FileNotFoundException: /Users/dysfun/tmp_clj/resources/templates/ayouts/main.tp (No such file or directory)

5:31 the l's have been removed from the path

5:31 some of them

5:32 oh wow. it's a selmer thing

5:32 i had {% extends layouts/main.tpl %} so it clearly took 'l' to be the delimiter

6:48 michaelr`: hey yo!

6:52 what's a good place to start a blog? I have two requirements: under my own domain name and export feature.

6:53 slipset: michaelr`: jekyll? github.io dns alias?

7:01 michaelr`: slipset: thanks.. looks like exactly what I'm looking for

8:08 * bruno`

8:08 * bruno`

8:52 vas: Hi, I want to associate a username with the session using Ring. It seems like there is a simple way to do this...

8:53 augustl: vas: are you using the ring-session middleware?

8:56 vas: augustl: yes, well, wrap-session.

8:57 augustl: vas: what is it that doesn't work in your code?

9:00 vas: i'm not sure how to associate the email with the session. i got an error for the number of args to assoc..

9:01 augustl: vas: how did you invoke assoc?

9:02 vas: generally the idea is to return a response map that contains a "session" key, then you'll get that back in the request the next time, via the session

9:04 vas: Oh I see. my invokation of assoc works now... so session is essentially for subsequent requests. that makes sense

9:05 does it make sense to do something like (GET "/page" [arg1 arg2 :as session] ... ) ?

9:05 i'm under the impression that that will cause the whole request map to be stored :as session

9:07 augustl: vas: yep, but it's probably better to name it "req" or "request" since :session is one of the many keys in the request

9:08 vas: augustl: ah okay, thank you. so if i wanted to persist a username it would be appropriate to store it in :session? does that make that key a map itself?

9:10 augustl: vas: only the things you put in :session will be put in the session

9:10 not sure whether or not it's appropriate in your case :)

9:15 vas: augustl: thanks :] so basically i'm going to set some part of :session to have the user's email, and somehow it'll have to check the session for every request to other pages to make sure the user variable is set. this is fun, learning about middleware by hacking together my own implementation. do you know how cookies fit into this picture?

9:29 sdegutis: Hi, I'm having trouble porting this line to ClojureScript: new NW.Menu({ type: 'menubar' })

9:30 justin_smith: in clj that would be (NW$Menu. {:type "menubar"})

9:30 Glenjamin: orly, i always thought you had to do (-.Menu js/NW)

9:30 justin_smith: Glenjamin: that's probably the right way in cljs

9:30 Glenjamin: you need a #js in there too justin_smith i think

9:31 justin_smith: Glenjamin: I said "in clj", but yeah

9:31 Glenjamin: oh, i see

9:31 i misread

9:47 justin_smith: noncom|2: yes, that's the right way to call a varargs method with 0 of the optional args

9:47 noncom|2: justin_smith: oh well :)

9:48 justin_smith: noncom|2: "varargs" are a fiction of javac that has no existence on the bytecode level, on the bytecode level it's just "array as the last arg"

9:48 noncom|2: yeah, that is how i guessed to put that (into-array) there..

9:49 stepped twice upon the thing today..

9:49 justin_smith: so it doesn't recognize the varargs method without the empty array there

9:53 sdegutis: The solution I got from #clojurescript does not work: https://gist.github.com/sdegutis/03c30f1a8a5427d6e78b

9:55 justin_smith: sdegutis: hmm, that looks like what Glenjamin suggested. It doesn't work?

9:55 Glenjamin: sdegutis: does (new (.-Menu nw)) work?

9:55 sdegutis: Glenjamin: that is a compile-time error, because the second arg to new must be a symbol

9:55 Glenjamin: eugh

9:55 sdegutis: (that was my first attempt)

9:57 Perhaps it's time for me to just bite the bullet and learn JavaScript.

9:58 Glenjamin: i've found cljs to be a bit clunky at interop when dealing with JS that uses objects like hashmaps

9:58 but a few macros/helpers and it's not too bad

10:07 sdegutis: welp, my hope in clojurescript is all gone

10:07 gonna try purescript again

10:20 nonrecursive: in case anyone’s interested in checking out Boot, I wrote an article that covers the concepts behind it http://www.flyingmachinestudios.com/programming/boot-clj/

10:21 agarman: @nonrecursive you have a quick boot v lein pitch?

10:22 mnngfltg: nonrecursive, thanks, I'll read it

10:23 nonrecursive: agarman: erm not really, I just think boot’s fun to at least learn about because it’s built on well thought out abstractions that I haven’t seen in other build tools

10:26 zerokarmaleft: boot's logo is cuter. lein's logo has a more gentlemanly moustache.

10:26 justin_smith: agarman: I thinkthe big selling point of boot over lein is theoretically going to be lower overhead / better startup time. If you have tried starting things up with / without lein you'll see a pretty big difference.

10:27 agarman: yeah, lein repl is slower than old school java script is slower than nailgun

10:27 vas: So I am using crypto.password.scrypt to make an authentication token... unfortunately sometimes it generates forward slashes in the hashed result string... which is no fun when playing with ring destructuring syntax... i was thinking about just wrapping this up with yet another layer of encoding, perhaps there is a better easier simpler stronger faster kanye west better stronger idea?

10:27 agarman: I'd be inclined to use something other than lein if it handles deps, is extensible and integrates with nailgun :-)

10:29 nonrecursive: agarman justin_smith yeah, the speed is definitely one thing I’ve been enjoying about boot. You can also load dependencies without having to restart your repl

10:30 justin_smith: that can be done via pomegranate or alembic/still with lein, but cool to know you can do it directly with boot

10:36 vas: i realized i can just replace the offending characters with a known string ^.^

11:00 justin_smith: vas: since it is being used in a URL, just URL-encode it ##(java.net.URLEncoder/encode "a/b/c.d%&e" "UTF-8")

11:00 lazybot: ⇒ "a%2Fb%2Fc.d%25%26e"

11:01 vas: justin_smith: way to be helpful and boring at the same time, I was translating it to "EEPAFORWARDSLASH" lol. thank you for your help ^.^

11:01 justin_smith: heh

11:02 vas: presumably i have to url-decode it as well?

11:02 justin_smith: vas: depends how you are using it

11:02 if you generate the token just to recognize the same token again, you can just register the encoded version

11:02 but you can also decode it if you prefer - I don't see how that would help though

11:03 I assume you save the token to the session, you can just save the version that is already encoded

11:04 vas: i'm stumped on how to save it to the session, but i'm learning about it now

11:04 justin_smith: vas: (assoc-in request [:session :super-secret-special-token] "token string")

11:05 vas: thank you. :D

11:05 justin_smith: and pass that forward as the new value of the request from your middleware

11:20 Integralist: Does anyone know how you can reference a function from another namespace when that namespace uses a macro to create the function?

11:20 vas: Integralist: do you want to reference the macro?

11:21 justin_smith: Integralist: the way the function is created doesn't matter

11:21 Integralist: if it has been created at the time your code is compiled, you can refer to it as usual

11:22 Integralist: if it is created at runtime, you'll need to wait until runtime to resolve it

11:22 the resolve function helps with this

11:22 (doc resolve)

11:22 clojurebot: "([sym] [env sym]); same as (ns-resolve *ns* symbol) or (ns-resolve *ns* &env symbol)"

11:22 justin_smith: ,(resolve '+)

11:22 clojurebot: #'clojure.core/+

11:22 justin_smith: what happens here is unlike directly using +, it doesn't have to exist yet when your form is compiled

11:23 though if the var doesn't exist at runtime, this will be a runtime error

11:36 Integralist: justin_smith: thanks I'll take a look at the resolve function.

11:37 justin_smith: effectively I have a macro that's called at run time which generates a function. but I'm getting an error to say the function that's generated doesn't exist

11:39 justin_smith: Integralist: right, because you reference it before it exists

11:39 ,((resolve '+) 2 2)

11:39 clojurebot: 4

11:39 justin_smith: that's how resolve is used

11:39 pretty simple

11:39 since resolve only sees a symbol, there is no compile time error if that symbol can't be resolved yet

11:40 you are also forgoing the basic error checking of the compiler of course - the advantage of not using resolve is that a typo in the function name would be caught

11:40 (and also resolution of a var is better done once at compile time rather than on every call, if you have the choice)

11:41 since it has an overhead

11:41 Integralist: justin_smith: so that didn't work for me. I get the error "Can't refer to qualified var that doesn't exist". I'll throw up a gist in a second to demonstrate what I'm doing

11:42 justin_smith: Integralist: what was your arg to resolve?

11:44 Integralist: justin_smith: here's a gist that shows three namespaces: the first https://gist.github.com/Integralist/b347b6f19d77af095550#file-macro-issue-clj-L1

11:44 the second https://gist.github.com/Integralist/b347b6f19d77af095550#file-macro-issue-clj-L21

11:44 the third https://gist.github.com/Integralist/b347b6f19d77af095550#file-macro-issue-clj-L30

11:45 justin_smith: in the first ns I def a macro that creates a function.

11:45 justin_smith: Integralist: instead of 'utils/endpoint spell out the full ns name 'spurious-aws-sdk-helper.utils/endpoint

11:46 that's pretty much what the error message was telling you

11:47 Integralist: justin_smith: can I still use `:as utils` (for the other places in the ns that use code from that ns)

11:47 justin_smith: of course

11:47 it's just that resolve doesn't like the shorthand when resolving something that doesn't exist yet

11:48 next trick of course is making sure that endpoint is created before the funciton that resolves/calls it is called

11:48 (clojure can't ensure that for you)

11:49 Integralist: justin_smith: hmm, so I'm getting an error "Can't refer to qualified var that doesn't exist" for line 10 of core.clj (which is the reference to the macro def-endpoint)

11:50 justin_smith: I've tried using the full ns for that as well

11:50 justin_smith: does def-endpoint exist at the time that is compiled?

11:50 also, btw, (do) inside defn body is not needed

11:52 Integralist: justin_smith: oh! I thought `do` was needed for multilines of a function. That's good then :-)

11:52 justin_smith: so at the moment I'm in a repl and I'm calling `(use 'spurious-aws-sdk-helper.core :reload-all)`

11:52 justin_smith: so, def-endpoint is going to expand to a definition of "endpoint" inside the ns where it is called

11:52 is that what you want?

11:54 Integralist: justin_smith: if we look at https://gist.github.com/Integralist/b347b6f19d77af095550#file-macro-issue-clj-L23-L28 effectively I expect nothing to happen as I've not executed any code to execute the `configure` function which would subsequently execute the call to `defmacro` - unless this is another case of `def` when `defmacro` is evaluated immediately (the problem with that would be the defmacro reli

11:54 es on an arg provided by the user).

11:55 justin_smith: Integralist: it might simplify things if instead of putting things inside functions, you use delay, and then use @ to force the delay and deref it

11:56 Integralist: justin_smith: how would that work in this instance? sorry if that's a stupid question; I guess I'm still not clear on the use of delay

11:57 justin_smith: hmm, taking another look to make sure that would actually help here...

11:59 Integralist: justin_smith: looking at delay docs it seems that you say "do X the first time I'm called, and then do Y for every subsequent call"

12:00 tomjack: "then be Y"

12:00 (where Y is the result of doing X)

12:01 justin_smith: Integralist: a delay is like a var or atom that can only change value once - from "unbound" to the result of evaluating its body

12:01 sdegutis: What's a superlative technique for branching off based on predicates for a given expression such as string?, vector?, seq?, nil? etc

12:01 Right now I'm doing this: (cond (string? thing) ... (vector? thing) ... etc)

12:01 justin_smith: sdegutis: I'd hardly call multimethods superlative, but they are nice enough for me

12:02 sdegutis: Do multi-methods work the same in ClojureScript?

12:02 justin_smith: yeah, they should

12:02 sdegutis: Great, thanks.

12:02 I'll re-learn them.

12:02 justin_smith: multi with dispatch on type of the first arg

12:03 Integralist: justin_smith: just not sure if delay fn would help in this example because if I call `(@utils/def-endpoint type)` the first time it would do something (not sure what) and then it would do what it was supposed to do (which is generate a function) but I'm not sure when I would call @ to deref it again to execute actual function created.

12:03 justin_smith: sdegutis: they aren't hard, and are much cleaner than a condp on type usually (which is what I would use instead of a cond)

12:04 sdegutis: Thanks justin_smith.

12:04 justin_smith: Integralist: no, this isn't a case for delay - though it could be with a much more radical restructuring I think

12:04 Integralist: it just feels like a big tangle of runtime binding that could be simplified

12:05 a defn with a form inside it that creates another defn is usually a bad sign

12:06 Integralist: justin_smith: the idea was that the user calls one function and passes in a 'strategy' of sorts, in this case they pass in X or Y. If it's X, then the subsequent function that gets called using a different map structure to what would be used if they passed Y

12:06 justin_smith: Integralist: in that case we have multimethods and protocols that are designed for that kind of usage

12:06 Integralist: justin_smith: and I didn't want a function that ultimately had an if statement inside of it so I assumed a macro would be better

12:07 justin_smith: by multi-method are you referring to multi-arity?

12:07 justin_smith: the lib would define a multimethod or protocol, and the app would pass in something fulfilling that

12:07 (doc defmulti)

12:07 clojurebot: "([name docstring? attr-map? dispatch-fn & ...]); Creates a new multimethod with the associated dispatch function. The docstring and attr-map are optional. Options are key-value pairs and may be one of: :default The default dispatch value, defaults to :default :hierarchy The value used for hierarchical dispatch (e.g. ::square is-a ::shape) Hierarchies are type-like relationships that do not depend upon type inheritance. By default Clo

12:08 justin_smith: $grim clojure.core/defmulti

12:08 lazybot: http://grimoire.arrdem.com/1.6.0/clojure.core/defmulti

12:08 justin_smith: Integralist: see the examples in the above link

12:08 I think it will simplify your code a lot

12:09 Integralist: justin_smith: cool, looking at multimethod now

12:11 justin_smith: Integralist: protocols are similar in usage, but only dispatch based on type (which should be fine in your case) and allow defining multiple methods, all part of one overall protocol

12:17 dysfun: #<CompilerException java.lang.NoClassDefFoundError: javax/servlet/http/HttpServletRequest, compiling:(ring/middleware/multipart_params.clj:46:5)>

12:17 anyone know why i'm seeing that?

12:17 b

12:18 justin_smith: dysfun: are you trying to aot compile a war file?

12:18 dysfun: no, that's just when i type 'lein repl'

12:18 Lewix: if I understand correctly, vars in clojure are kind of like constant right?

12:19 justin_smith: Lewix: they are mutable - though in practice we only mutate them during development via the repl

12:19 in sane code at least

12:21 Lewix: justin_smith: so does it mean they are used as constants however they behave like variables defined at the top level

12:22 justin_smith: great book between. I ordered it

12:24 apprently people define vars as such: vars are not variables. great! but what are they then?

12:24 justin_smith: Lewix: yeah, in sane code you can assume they are constant. You can capture them in let bindings or function args (equivalent) to ensure they are only resolved once and are then truly constant in that scope

12:24 Lewix: well, vars actually are variables. Though they do have some features for coordination of their mutation

12:25 Lewix: justin_smith: programming clojure keep on saying they're are not variables

12:25 justin_smith: hmm...

12:25 noonian: Lewix: they are the thing that binds free variables to functions or values at the top level (free variables being anything not a formal parameter or let bound local)

12:26 they are mutable but that is mainly for interactive development so you can redefine things, but in your actual program you normally shouldn't redefine top-level vars

12:27 Lewix: I see . thanks guys

12:27 justin_smith: Lewix: so perhaps we can pedantically say, contradicting "programming clojure" "vars are variables, but shouldn't be treated that way"

12:28 Lewix: justin_smith: i like that

12:29 justin_smith: noonian definition was pretty good

12:30 justin_smith: it's definitely clearer than just saying. they're not variables

12:33 justin_smith: oh yeah, noonian's explanation is quite good

12:33 (inc noonian)

12:33 lazybot: ⇒ 13

12:44 sritchie: DomKM: looks like my hack fails on unmatch, unfortunately

14:38 DomKM: sritchie: Uh oh :(

14:39 sritchie: DomKM: just because unmatch turns the seq of routes into a map

14:39 so duplicate keys get knocked

14:39 worked around it for now

14:42 vas: is there a reason why ring response would prompt a file download (containing the response string ) instead of just printing it to the browser?

14:42 DomKM: sritchie: Ah, sorry about that. Thanks for working around it for now. :)

14:43 justin_smith: vas: you need to set the content-type

14:43 vas: a browser prompts for a download if it doesn't know if it can display it (common cause of that, not setting the right content-type)

14:43 vas: justin_smith aha. so response in the barebones http talkback.

14:44 thank you justin

14:44 justin_smith: np

14:45 vas: general idea is {:headers {:content-type "whatever"} :body (generate-body-string) :status 200 :session {some session-data}}

14:45 there are middleware to help with automatic content-type setting, but sometimes you just need to set it by hand

14:46 vas: very cool. i don't think i have ever been more intimate with http than i have working with clojure

14:46 justin_smith: this is a much saner version of http :)

14:46 vas: so what would an idiomatic way to set the content type of a response look like?

14:47 justin_smith: vas: associate :headers {:content-type "text/html"} to your response map

14:47 or whatever the apropriate content-type is

14:47 noonian: if you have your routes organized into page-routes, api-routes etc you could put it all in middlewares and apply them to the different handler sets

14:48 justin_smith: noonian: that's an excellent point

14:49 vas: yeah. that will be cool and fun to do. i'm starting to see where the layering of my app is happening..

14:54 thank you for the suggestion noonian

14:55 noonian: vas: no problem, justin_smith did most of the helping :P good luck and enjoy learning Clojure!

14:56 vas: :D i just got the login/session functionality to work, ecstatic.

14:57 on behalf of this successful mission i'd like to thank justin smith, tupac shakur, rich hickey, the wutang clan...

14:58 justin_smith: nice

15:01 vas: so on a bunch of pages i'm going to be verifying the contents of the :session values... rather than doing it individually for each route, i can make a middleware that applies that step to given routes?

15:01 justin_smith: of course

15:01 the generel outline of a middleware is (defn mid [handler] (fn [request] (handler (frob request))))

15:02 *general

15:02 so inside your frob you could redirect to an error page

15:02 or invalidate the session

15:02 or redirect to a login

15:02 or whatever

15:02 in fact, you can avoid calling the handler altogether and just hijack the whole thing

15:03 (if that's called for)

15:03 noonian: i'd recommend making the middleware do the verify logic, and then only apply it to specific handlers you want to do the verifying

15:03 instead of the middleware figuring out which routes to verify

15:03 justin_smith: indeed

15:03 kaplan_: what is a middleware?

15:04 justin_smith: kaplan_: a function that wraps another function - usually in ring as a way to add a feature to a website

15:04 SegFaultAX: kaplan_: A function that is applied before, after, or around your handler.

15:04 kaplan_: ah ok

15:04 sobel: when i was into AOP, those were called 'advice'

15:05 or advisor functions

15:05 iirc

15:05 SegFaultAX: sobel: They're just normal functions. There's nothing special going on here.

15:05 sobel: SegFaultAX: yeah, they don't have to be special in lisp

15:05 SegFaultAX: You could also call them "decorators" if you wanted to make them sound even more magical.

15:05 justin_smith: SegFaultAX: sure, but it's not like advice functions are that special

15:05 haha

15:05 SegFaultAX: :)

15:06 sobel: SegFaultAX: ugh, haha, i remember that set of terminology...much hated :)

15:06 SegFaultAX: justin_smith: Yea I know, that's my point. It's all just functions.

15:06 vas: i don't suppose a tutorial for such an endeavor would exist? (=

15:06 SegFaultAX: vas: For what?

15:06 justin_smith: right, one more "design pattern" that's just business as usual once you are in fp world

15:06 sobel: the invocation of advisors was usually magical

15:06 same as annotations

15:06 justin_smith: which is why i'm here. in a nutshell.

15:06 justin_smith: sobel: right, because they don't have our magic they have to invent their own flavor

15:07 SegFaultAX: Well annotations (as in Java) *are* different.

15:07 justin_smith: sure, but we can do most of what annotations are used for

15:07 SegFaultAX: Because they can be expanded and/or take effect at different times (compile time, run time, etc)

15:07 vas: for making a middleware than will check my session values .. and applying the middleware to only a subset of routes.. the principle is clear but i'm not sure what it looks like in practice

15:07 justin_smith: without needing to directly use that feature. And yeah they are weird.

15:08 SegFaultAX: justin_smith: I don't know if this is true, but intuitively I'd guess that macros are more or less equivalent to annotations.

15:08 Or at least, you can recreate the effects of annotations with macros.

15:08 sobel: something like that

15:08 justin_smith: SegFaultAX: some of the features of annotations are also available via metadata

15:08 in terms of functionality at least

15:08 SegFaultAX: Yea, hmm.

15:09 justin_smith: but, of course, macros can put metadata on things at expansion time

15:09 so it still fits

15:10 noonian: vas: here's an example of a simple middleware to print requests and responses, the one you want will be inspecting values in request (the :session key in particular) https://www.refheap.com/97320

15:11 amalloy: yeah, annotations look a lot more like compiler-directed metadata to me

15:11 vas: noonian: wow thanks kind sir

15:11 justin_smith: noonian: that's a lot of characters to spell out "frob", hehe

15:12 noonian: a middleware is just a function, so before you pass your handlers to jetty or w/e just have something like (def routes-with-middleware (map spy-middleware my-handlers)) and pass routes-with-middleware to jetty or w/e

15:12 benhuda: hi guys

15:12 justin_smith: hello

15:12 noonian: justin_smith: heh

15:12 benhuda: so im sketching out a sort of ad-exchange bidding proxy - get a request, and then make N requests to N services in real time - and return a selected best offer back to a user.

15:13 i'm wondering how Clojure is fitting this problem space. I can understand Node.js and Go will be good at this because they're both by default using async IO

15:13 is there anything in Clojure that can supply an equivalent answer?

15:13 sobel: Clojure is also pretty useful for async IO

15:14 SegFaultAX: benhuda: The JVM has first class support for asyncio through nio

15:14 sobel: go routines and channels are part of clojure, if you're familiar with those from Go

15:14 SegFaultAX: And netty is pretty great, there are a few wrappers for it.

15:14 (Well, the netty /API/ is complex and sometimes hard to understand. But performance wise, it's awesome)

15:14 benhuda: SegFaultAX: to be honest - as a matter of personal taste i would like such a thing to come as a facility of the language or a tightly knit library (rather than touch netty API or nio API directly)

15:15 justin_smith: benhuda: nio is tightly nit - interop in clojure is trivial

15:15 *knit

15:15 and you don't need to touch it's api, there are wrappers using ring

15:15 sobel: benhuda: you should search the web for examples of async functionality in clojure.

15:15 benhuda: how easy will it be to create such a reverse-proxy load balancer?

15:16 sobel: i'm aware of clojure async - i just am not sure it's async io

15:16 sobel: benhuda: what else would it be?

15:16 benhuda: a way to abstract threading? a syntactic sugar around promises?

15:16 justin_smith: benhuda: we have actual threads - we can do async even when the io can't be async. But via nio / netty we have async io

15:16 SegFaultAX: sobel: core.async is not about async io at all.

15:17 benhuda: sobel: ^^

15:17 noonian: benhuda: core.async is a way to coordinate asynchronous stuff

15:17 benhuda: yes, right. thats why i'm wondering out loud regarding async/io (without really touching nio or netty)

15:17 noonian: so it could be useful in managing your logic but the actual async io would be performed by different libraries/apis

15:18 SegFaultAX: benhuda: What do you mean "without really touching nio". That's the JVMs facility for async io.

15:18 justin_smith: benhuda: if you use aleph or http-kit you don't have to touch the netty or nio apis to get async request handling

15:18 SegFaultAX: Prior to nio, all JVM IO was sync/blocking.

15:18 benhuda: justin_smith: ah! right, aleph was the library i couldnt recall

15:18 justin_smith: SegFaultAX: right, but he doesn't have to write code that interacts with the nio api directly

15:18 he can just use the ring api

15:19 SegFaultAX: Yup.

15:20 justin_smith: benhuda: most ring servers will assign one thread per request (which allows parallelism but not the most efficient possible usage of cores, of course), with aleph or http-kit you get async handling using multiple threads, to make the most of your resources, and best possible response times

15:20 benhuda: yes

15:20 justin_smith: but from your handler code's point of view, it's still straightforward

15:21 no need to write things in terms of callbacks or whatever

15:21 benhuda: so now im asking myself - if i were to solve this problem - think of building a load balancer, which is also a reverse proxy, is there a way i can do the calls with async io?

15:21 tbaldridge: there's also pedestal that accepts core.async channels for async responses

15:21 justin_smith: benhuda: sure, we have a couple of good http async libs

15:22 benhuda: and there's no danger using those libs on ring which runs on, say, Jetty ?

15:22 tbaldridge: well ring is sync, so you can't really run async code on it at all

15:23 benhuda: so i guess i still have a block of blurriness for these use cases in clojure

15:23 maybe i can rephrase

15:24 is there a smooth/easy way to do event driven I/O in clojure? like Ruby's EventMachine for example

15:25 tbaldridge: sure, aleph, http-kit (to some degree) and pedestal all offer that for HTTP, and other libraries offer it for other types of IO

15:26 benhuda: im curious - what other types of IO?

15:26 noonian: the answer to 'can you do X in Clojure' is usually 'Yes' unless X needs to startup fast or run somewhere without a JVM

15:26 justin_smith: benhuda: disk IO for example

15:26 or raw socket IO

15:26 benhuda: async IO for files? only windows supplies that

15:27 justin_smith: ?

15:27 arohner: benhuda: no, JVM NIO can do it as well

15:27 justin_smith: that's patently false

15:27 arohner: on unix

15:27 benhuda: is there a kernel facility for async file IO?

15:27 arohner: yup

15:27 there are several

15:27 tbaldridge: yeah, stuff like Vert.X offers that sort of stuff in an abstracted way

15:27 justin_smith: benhuda: it's a system call called "select" that has existed longer than windows has

15:27 arohner: on linux

15:27 benhuda: select works on files too?

15:27 arohner: yup

15:28 justin_smith: yes, it was originally designed for watching multiple file descriptors

15:28 on unix everything is a file descriptor as far as the apis are concerned

15:28 benhuda: hm. someone on the nodejs channel twisted my knowledge then!

15:28 tbaldridge: (not surprising)

15:28 benhuda: wait ill try to find the log

15:28 * tbaldridge ducks

15:28 benhuda: ah crap. no logs, i had restart

15:28 xemdetia: select is for big data

15:28 justin_smith: benhuda: on a linux system even memory regions or the audio device can be accessed as if they were files

15:28 or the mouse

15:28 etc.

15:29 benhuda: ok well good to know my memory is still good. i should remember to doublecheck people on IRC myself :)

15:29 justin_smith: xemdetia: select is used for low level code where you have many potential sources of data in one thread

15:29 vas: noonian: thanks man. here's what i have so far, https://www.refheap.com/97322 ... i'm not sure how to align the area by (def app ... ) so that it invokes the middleware "routes-with-sesh-verification"

15:29 xemdetia: justin_smith, I was making a bad joke

15:29 justin_smith: xemdetia: ahh, ok :)

15:29 xemdetia: I am on a conference call of sadness and this is all I have going for me

15:29 arohner: xemdetia: :-)

15:29 justin_smith: haha

15:29 (inc xemdetia)

15:29 lazybot: ⇒ 2

15:30 benhuda: im really tempted to run this new venture all-clojure

15:30 i just want to make sure i have an async-io escape hatch

15:30 justin_smith: vas: that's not quite it

15:31 benhuda: i remember from my last clojure project (2013) - i didn't find anything that were nice to work with

15:31 tbaldridge: benhuda: what are you trying to build?

15:31 justin_smith: vas: you need to create the handler (you are probably doing this via compojure) then wrap logged-in-verify around the handler

15:31 noonian: vas: you have a couple issues here. logged-in-verify looks pretty good, but in the if check if things are good thats where you should put the (let [response (ring-handler request] ...) or just (ring-handler request) and in the else put your error response.

15:32 like justin_smith said your middleware expects a handler so you need to write a function that takes a request and returns a response and thats what you wrap with logged-in-verify

15:32 benhuda: tbaldridge: one part data science, one part user tracking, one part ad server, one part ad exchange.

15:33 tbaldridge: i know there are a couple startups in this space that are all clojure so that's nice. i just have no clue how they do scalable async IO, if they do at all. i know at this field i need to be anal about latency and performance

15:38 noonian: vas: i updated your refheap a little bit to try and make it clearer: https://www.refheap.com/97323

15:40 jlewis_: is there a way to write an inline macro like you can write an inline function?

15:41 amalloy: jlewis_: by inline i suspect you mean local or lexical?

15:41 there is clojure.tools.macro/macrolet

15:41 michaelr`: benhuda: sounds like most of the heavy lifting could happen in the tools you'd use and not necessary in your clojure code..

15:42 jlewis_: that works, i was wondering about a #() equivalent

15:42 amalloy: no

15:42 sobel: michaelr`: yeah, i was thinking, this has sql/openresty written all over it, from my perpsective

15:46 jlewis_: i'm trying to abuse the for macro to give me all permutations from a set of sets - like from [1 2] [2 3] i get [[1 2] [1 3] [2 2] [2 3]]

15:47 i'm wishing i could generate the binding vector inline, but i need to make a macro for it above my for expression

15:47 (right?)

15:48 dmbennett: jlewis: kind of like find all the subsets of a given set?

15:48 findign*

15:48 finding*

15:48 wow, I hate this keyboard

15:48 jlewis_: sort of, but each element in my output vector is drawn from a different input set

15:49 dmbennett: interesting

15:50 vas: noonian: thanks it looks beautiful! =)

16:30 ajmagnifico: anyone: is there a clever way of doing inner and full outer joins in incanter? Googling around provided some solutions that just looked kind of hackish. Does incanter really not support full outer and inner joins between two DAtasets?

16:34 nicferrier: justin_smith: thanks for your help.

16:36 gfredericks: jlewis_: ##(for [a [1 2], b [2 3]] [a b])

16:36 lazybot: ⇒ ([1 2] [1 3] [2 2] [2 3])

16:36 gfredericks: ^ is that not what you wanted?

16:36 or is the problem that you have a variable number of sets?

16:37 jlewis_: yeah variable number of sets

16:37 gfredericks: you're describing a cartesian product I think

16:37 jlewis_: right there you go

16:37 gfredericks: which is in the math.combinatorics lib

16:37 and also pretty easy to write recursively

16:40 ,(defn cartprod [xss] (if (empty? xss) [[]] (let [things (cartprod (rest xss))] (mapcat (fn [x] (map #(cons x %) things)) (first xss)))))

16:40 clojurebot: #'sandbox/cartprod

16:41 gfredericks: ,(cartprod [[1 2] [2 3]])

16:41 clojurebot: ((1 2) (1 3) (2 2) (2 3))

16:41 gfredericks: ,(cartprod [[1 2] [2 3] [:a :b :c]])

16:41 clojurebot: ((1 2 :a) (1 2 :b) (1 2 :c) (1 3 :a) (1 3 :b) ...)

16:48 justin_smith: nicferrier: oh, np

16:49 nicferrier: so you got your logging thing sorted out?

16:49 nicferrier: justin_smith: yep, although it's a bit hacky. it works.

16:50 currently investigating using a classloader in leiningen to make that faster.

16:52 justin_smith: nicferrier: wait, a classloader in leiningen to make what faster? if you mean overall app startup, you should probably be using lein uberjar to make a runnable jar that doesn't use lein at runtime

16:53 nicferrier: justin_smith: no. I don't mean that. I mean starting lein and clojure in a jvm and then having compilation and run and package and all that occur in a classloader. so you can throw it all away all the time.

16:53 apparently the reason it wasn't done like that was "jetty".

16:53 justin_smith: ahh

16:53 nicferrier: but it seems like it would be a good thing to do.

16:54 it would certainly reduce a lot of nrepl madness.

16:54 justin_smith: nicferrier: see also lein trampoline

16:55 jlewis_: ,(defmacro m [sets] (#(let [syms (repeatedly (count sets) gensym)] `(for [~@(interleave syms sets)] [~@syms]))))

16:55 clojurebot: #'sandbox/m

16:55 nicferrier: justin_smith: that doesn't use classloaders tho I think. it somehow saves the state of the vm?

16:55 jlewis_: ,(m [[1 2 3] [4 5]])

16:55 clojurebot: ([1 4] [1 5] [2 4] [2 5] [3 4] ...)

16:55 jlewis_: gfredericks: ^^ heh

16:55 justin_smith: nicferrier: it just avoids starting yet another vm

16:56 nicferrier: justin_smith: right. didn't seem as efficient as using a classloader to constantly throw away the project on top of lein.

16:56 jlewis_: er, don't need that #() either

16:57 nicferrier: justin_smith: well. presuming lein has the clojure that the project wants.

16:57 jlewis_: ,(defmacro m [sets] (let [syms (repeatedly (count sets) gensym)] `(for [~@(interleave syms sets)] [~@syms])))

16:57 clojurebot: #'sandbox/m

16:57 jlewis_: ,(m [[1 2] [4 5]])

16:57 clojurebot: ([1 4] [1 5] [2 4] [2 5])

16:57 nicferrier: justin_smith: I'm not really clear why it hasn't been done this way.

16:58 justin_smith: nicferrier: have you looked at boot at all? it's a fresh approach to this stuff (I haven't used it yet myself, but one of their stated goals is faster startup)

16:58 nicferrier: justin_smith: going to look now.

17:00 justin_smith: interesting... but also... sigh.

17:00 was this the thing that is just programs?

17:00 that was the latest build thing a while ago. no more declarative! just programs!

17:01 justin_smith: yeah, imperative rather than declarative tasks

17:01 right

17:01 nicferrier: right. now I'm looking at their bootstrap site rather than their github I see that.

17:02 * nicferrier starts a fashion of referring to developer websites as "bootstraps"

17:03 gfredericks: jlewis_: that only works when the collections are literals though, which means you still have to know the number of collections at compile-time and may as well have written the for by hand

17:04 nicferrier: justin_smith: thing is ... I need windows and I don't need exe's.

17:05 is a nice idea though.

17:06 does feel like they're onto something.

17:19 jlewis_: gfredericks: mm that is true

17:19 cespare: is there an idiomatic way to do required kwargs?

17:21 justin_smith: cespare: I'd say prismatic/schema if you want real data validation, or roll your own doseq / assert if you don't need schema's other features

17:21 hyPiRion: required kwargs? I'd just make them args.

17:22 cespare: hyPiRion: i've got some functions with enough args that I'd rather not

17:22 hyPiRion: Depends on context, of course. If you want to have a similar api then it's probably better to do assertions or something.

17:22 cespare: it's nasty plumbing stuff like passing connection options around

17:23 anyway thanks justin_smith, i'm doing the doseq/assert right now so I'll probably just stick with that

17:23 thanks

17:23 justin_smith: (doseq [required [:a :b :c]] (assert (contains? opts required) (str (pr-str opts) "does not contain" (pr-str required)))

17:24 cespare: or i could (defmacro defn+ ... ) to allow {:keys [...] :required [...]} ;)

17:24 jk

17:24 justin_smith: well, you could. I don't know if you need to go that far with it, but it is possible

17:25 cespare: ime you have to exercise a lot of good judgement around the what's good vs. what's possible boundary to write good clojure :D

17:28 justin_smith: cespare: (defmacro is-required [m ks] `(let [m# ~m] (doseq [k# ~ks] (assert (contains? m# k#) (str (pr-str m#) " should contain " (pr-str k#))))))

17:28 then you can do (is-required opts [:x :y :z])

17:29 actually, that doesn't even need to be a macro

17:30 ,(defn is-required [m ks] (doseq [k ks] (assert (contains? m k) (str (pr-str m) " should contain " (pr-str k)))))

17:30 clojurebot: #'sandbox/is-required

17:30 justin_smith: ,(is-required {} [:a])

17:30 clojurebot: #<AssertionError java.lang.AssertionError: Assert failed: {} should contain :a\n(contains? m k)>

17:31 justin_smith: ,(is-required {:a 0} [:a])

17:31 clojurebot: nil

18:31 crash_ep: why is defonce used so much more frequently in clojurescript than in clojure?

19:01 AeroNotix: ,(doc defonce)

19:01 clojurebot: "([name expr]); defs name to have the root value of the expr iff the named var has no root value, else expr is unevaluated"

19:13 justin_smith: "No one loves you, for love does not exist. Nothing matters. Enjoy Valentine's Day.

19:13 Your local Arby's awaits."

19:13 oops, wrong window, sorry

19:14 TEttinger: classic

19:14 gfredericks: justin_smith: kind of a cumbersome password if you ask me

19:14 justin_smith: Friday the 13th eve: test your luck at Arby's. feed the abyss. die anyway. Luck's a pointless abstraction, as is life. Arbys: God hates you.

19:15 TEttinger: nihilist sandwich shop

19:15 justin_smith: my new favorite twitter account

19:15 https://twitter.com/nihilist_arbys

19:17 gfredericks: actually, compared to most passwords that hard to guess, it would be quite easy to memorize

19:18 So I only use it for things that need to be super secure, of course

19:18 like IRC

19:18 arrdem: clojurebot: ping

19:22 Lewix: Note that let is implicitly used anywhere locals are required

19:22 what does that mean

19:22 justin_smith: anything that creates locals is either going to call let, or have a macro that expands to a call to let

19:23 that would be my interpretation of that statement at least

19:24 Lewix: justin_smith: oh I get it now. functions parameter use let implicitly to be bound as locals

19:25 tomjack: they don't, though, do they?

19:25 Lewix: justin_smith: do you use clojure in your day to day job

19:25 tomjack: apparently yes

19:25 justin_smith: tomjack: well, they do use let for destructuring

19:25 Lewix: yes

19:26 tomjack: ah, yeah

19:26 justin_smith: Lewix: I work on backend web stuff, using clojure

19:26 Lewix: justin_smith: I might job hunt for a job that uses clojure to learn faster

19:26 justin_smith: Lewix: sink or swim can work, yeah :)

19:27 Lewix: justin_smith: However, there's no many in Canada =)

19:27 justin_smith: what part? I know of one company up there

19:27 but it's a big country

19:30 gfredericks: sort of

19:31 justin_smith: gfredericks: regarding let/ fn? yeah, not all function bindings translate to let of course

19:31 gfredericks: no I meant "big country"

19:31 justin_smith: ahh

19:32 Lewix: justin_smith: toronto

19:33 a big country geographically , a small country population wise

19:33 justin_smith: right, geography is the sense I meant

19:34 Lewix: https://gist.github.com/6ewis/ea8867dab84127495831 can someone explain to me line 2 ( _ )

19:34 justin_smith: I think living social has an office not far from you

19:34 Lewix: by convention, _ is a name used for values you won't actually reference again

19:34 it's a way of saying "a binding has to go here, but I won't access that binding"

19:34 informally

19:35 Lewix: right, but why is it needed at all in this case

19:35 can't we just right it without it?

19:35 justin_smith: Lewix: because otherwise it wouldn't be a valid clause in a let binding

19:35 because let always needs binding / value pairs

19:35 how would it line things up properly otherwise?

19:35 Lewix: ahhhh

19:35 thanks justin_smith

19:38 justin_smith: Also, I'm checkin living social now

19:56 theman: hi

20:02 gfredericks: are there any obvious use cases for clojure.core/pr being dynamic?

20:05 amalloy: gfredericks: none that are obvious to me

20:12 justin_smith: weirdo custom printing code that dynamically rebinds pr?

20:12 I mean - it makes that potentially possible, but I don't know that anyone would really want to...

20:23 gfredericks: the question is if calling pr on e.g. a collection will end up calling it on the elements

20:23 or if once you hit print-method you stay in print-method

20:23 in the latter case it seems less useful

20:49 ,(doc *agent*)

20:49 clojurebot: "; The agent currently running an action on this thread, else nil"

20:49 gfredericks: ^ I've never once used that

20:50 can't think of a use for it

20:53 justin_smith: gfredericks: violating abstraction?

20:55 gfredericks: since you are around, what's the name for an algorithm that would take, eg. a uniform distribution as input, and output a bell curve distribution of values

20:55 gfredericks: it can be done with interpolation and table lookup, but I don't know what it is actually called - nonlinear mapping maybe?

20:57 if the input is -1 to 1, it can be calculated by raising it to some odd power

20:57 gfredericks: justin_smith: I don't know nuthin about names of such things

20:58 justin_smith: OK

20:58 gfredericks: so gaussificator seems reasonable

20:58 justin_smith: gausifier

20:58 "the baby's getting flat again honey, get him is gaussifier"

20:59 a healthy baby will of course have a nice bell curve of bodyfat

20:59 gfredericks: j.u.R has a gauss thing

21:00 justin_smith: it can also be calculated via a polynomial - you take your initial curve, interpolate to a polynomial, then use that polynomial to do the mapping

21:01 I hate having like 1/4 of a grasp on the concept, but none of the right keywords

21:01 benmoss: is there a way with clojure.test/run-tests to exclude tests based on metadata?

21:01 justin_smith: benmoss: this is something lein test will do

21:01 benmoss: this library im looking at has (deftest ^:benchmark ...) on some

21:01 justin_smith: benmoss: but you might have to use lein's code, or write it yourself

21:01 benmoss: bleh, i hate the slowness of lein test

21:03 justin_smith: benmoss: you could use a fixture that checks the metadata and decides whether to run the test or not based on said metadata

21:03 but that would be per-ns

21:03 gfredericks: I'd investigate how lein test does it

21:04 justin_smith: or wait, I think fixtures actually get the function itself as an arg, not the var (which would have the metadata)

21:05 gfredericks: ha software.

21:05 benmoss: meh ill just comment out all the tests and restart my repl

21:06 justin_smith: it's faster to just conditionally unmap test vars that have a given metadata

21:06 and then reloading the file would bring them back again if you want a different set

21:06 (doc ns-unmap)

21:06 clojurebot: "([ns sym]); Removes the mappings for the symbol from the namespace."

21:06 justin_smith: that's how one deletes vars

21:07 * amalloy deletes vars with a flamethrower

21:07 justin_smith: heh

21:08 "this shuts down the computer"

21:08 gfredericks: M-x flamethrower

21:16 amalloy: gfredericks: you made me write https://www.refheap.com/bf192c465dae4b9c01fe0a698

21:17 gfredericks: makes sense

21:17 justin_smith: haha

21:17 that's maybe more like a shotgun

21:18 I would imagine a flamethrower singing edges (ie. taking random chunks of the end of each line)

21:18 bonus points if it also makes flame-themed syntax highlighting

21:18 amalloy: it made me wonder: is this method of choosing a random subregion good, or would it be better to uniformly choose two numbers within the region and then treat one as the start and one as the end

21:18 the second one appealed to me more, but what i actually did was easier

21:19 justin_smith: what you have now is biased toward the end of the document

21:19 amalloy: justin_smith: that is what i thought too, but i couldn't actually be sure

21:19 justin_smith: I have used a similar algo for random placements of sonic events

21:19 amalloy: instead, pick a width first, and then randomly place the center within the constraints of the width

21:20 that will actually be uniform

21:20 (and of course calculate bounds based on that center)

21:20 amalloy: justin_smith: i think my suggested other algorithm is just as good, isn't it?

21:20 justin_smith: amalloy: yeah, that would actually work too I think, and be simpler

21:21 but probabilites are weird, so we should second guess things of course...

21:21 amalloy: yeah, i guess the problem with the algorithm i actually used is that the start point averages around the center, which means the midpoint averages 75% of the way through

21:21 justin_smith: exactly

21:22 amalloy: flamethrower don't care

21:23 justin_smith: with common workflows, stuff toward the end of the document is more likely to be missed with a random deletion, right?

21:24 amalloy: maybe?

21:25 justin_smith: here would be a fun one: M-x entropy

21:25 picks a random char, and swaps it with a random neighbor

21:25 N iterations of this

21:25 brownian motion within a document

21:26 do it just a few times and it is hard to distinguish from typical typos

21:28 amalloy: justin_smith: i bet i could write an algorithm to distinguish it from real, human typos would work like 90% of the time, if you gave me a file with either 5 typos or 5 real transpositions and told me which two characters were mis-ordered

21:28 justin_smith: hmm

21:28 yeah, you probably could :)

21:30 amalloy: the algorithm would be, if the two characters are typed with the same finger, it was a computerized transposition, and if typed with different fingers it was a real typo

21:30 benmoss: bleh, is there any way to see a macroexpansion of a deftype's protocol implementation?

21:30 i bet that is not super clear let me provide an example

21:31 https://github.com/ztellman/manifold/blob/master/src/manifold/deferred.clj#L349-L351 implements deref via a macro, https://github.com/ztellman/manifold/blob/master/src/manifold/deferred.clj#L241-L257

21:32 having a problem with it, trying to debug it, but don't know what I can call macroexpand on here

21:32 amalloy: benmoss: you can just (macroexpand-1 '(deref-deferred timeout-value time TimeUnit/MILLISECONDS))

21:32 the fact that it's inside a protocol definition has no bearing on how it expands

21:33 benmoss: indeed

21:33 thanks

21:41 justin_smith: amalloy: https://www.refheap.com/97339

21:41 it works!

21:44 updated (there was an off-by-one bug at the end of the buffer)

21:45 it's oddly soothing to watch a namespace decay into gibberish this way

21:46 TEttinger: what on earth is that, justin_smith?

21:46 that refheap

21:46 justin_smith: TEttinger: it randomly swaps to characters in your current buffer

21:47 brownian motion!

21:47 TEttinger: hahaha

21:48 justin_smith: I need to make it respect prefix arguments by doing N iterations

21:52 gfredericks: justin_smith: how do you loop it?

21:52 justin_smith: now prefixed and looped https://www.refheap.com/97339

21:53 maybe it would be good to put a delay in too...

21:53 just a small one

21:56 gfredericks: now it is animated, and will stop if you enter any editor command https://www.refheap.com/97339

21:56 amalloy: justin_smith: M-100000 M-x entropy really wrecks a file

21:56 justin_smith: wow, OK, I might make an infinite version of that

21:57 I bet!

21:57 amalloy: i ran it on my .emacs

21:57 justin_smith: nice

21:57 try the animated version if you haven

21:57 't yet

21:57 huge improvement I think

22:01 amalloy: justin_smith: sit-for is interesting. i think you mean (sit-for 0); (sit-for 0 100) is weird

22:02 justin_smith: amalloy: first arg is seconds, second is ms

22:02 I definitely don't want to wait more than a second between redraws

22:03 oh wait, that's sleep-for

22:03 amalloy: justin_smith: sit-for says that form is deprecated and you should just use one fractional arg

22:03 justin_smith: right

22:03 but it also waits for input, which I don't strictly want

22:03 I want the loop to keep going

22:04 amalloy: have you tried it? they both seem to behave the same when given input

22:04 justin_smith: amalloy: ahh, that's much better

22:04 thansk!

22:05 updated

22:06 interesting that the algorithm keeps the cursor on a given character

22:14 amalloy: justin_smith: i am still thinking about the randomness of these various algorithms, and i can't see how we conclude that my first algorithm is biased. it chooses uniformly from among all possible start points, and then uniformly among all possible endpoints given that start point

22:15 so it ought to be selecting uniformly from all possible intervals

22:18 justin_smith: amalloy: think of adding up all stary poimts from which a given end point can be calculated

22:19 amalloy: i think i see what is going on

22:19 justin_smith: *starting points

22:20 also you can do ot empirically with frequencies

22:20 amalloy: yeah, i did

22:20 it's obviously not uniform

22:21 the problem is that there are only 1 or 2 possible intervals starting at character 10000, and there are 9000 intervals starting at character 1000, but each are as likely to be selected as a start point

22:22 so the 1 or 2 starting at 10000 get selected far too often

22:22 justin_smith: writing a typo simulator right before leaving the house adds a new layer to the typos I do on mobile

22:23 amalloy: justin_smith: i tried my suggested other algorithm by the way, and it is biased in a way we didn't think of: empty intervals are half as common

22:24 because there are two ways to get, say, (0 1), but only one way to get (0 0)

22:24 but it's fine if you just remove those

22:25 justin_smith: oh, so 0,1 is empty by that method

22:25 amalloy: no, i mean 0,0 is empty

22:25 justin_smith: ok.... I think I get it

22:26 pick size, pick center should be a fair method though

22:27 djames: Data structure question: one (naive) way to keep keys and values is with a hash, e.g. {1 true 2 true 3 true 4 true 5 false 6 true 7 true}

22:28 Is there a name for a data structure where the above is represented as {[1 4] true [5] false [6 7] true}?

22:28 justin_smith: djames: are you looking for a set?

22:28 amalloy: justin_smith: i dunno, i think it will run into the same problem as my original algo

22:28 justin_smith: oh, n/ m

22:28 djames: justin_smith: :) implementing this is easy, naively. just curious if there is a name for such a thing

22:28 justin_smith: I misread ay first

22:28 amalloy: there are 1000 possible sizes for a buffer of size 1000, but for eg size 999 there are only two possible intervals

22:29 so you will over-select those two

22:29 djames: no problem, it was a two part question

22:30 justin_smith: amalloy: mobius world would fix this (and sldo rescue the initisl slgorithm)

22:30 amalloy: mobius?

22:30 justin_smith: looped

22:31 amalloy: PS i just tried your algo empirically and it was biased in the way i predicted

22:31 justin_smith: modulus style

22:31 amalloy: i think the only reasonable way to do it is my second way, picking two independent numbers and choosing the smaller as the start

22:32 justin_smith: amalloy: cool, I should have thought of that

22:32 amalloy: randomness is hard

22:32 justin_smith: yeah it is

22:33 djames: is that skip-list encoding?

22:35 djames: justin_smith: I wasn't intending that. When I last reviewed skip lists, they were probabilistic data structures. I am not saying skip lists are not a possibility, but I don't think they capture the essence of what I need.

22:36 I wouldn't be surprised if what I'm talking about isn't particular interesting theoretically speaking. :)

22:36 But I think it probably comes up in practical situations fairly often; e.g. calendar applications and time-based applications. One doesn't want to have to keep a hash map with ALL the keys.

22:37 justin_smith: yeah, I should have googled first, I get lazy on mobile

22:40 djames: I would define a protocol with insert and delete, then try defining a deftype using a list of pairs to represent the domain

22:41 then you can easily compare the map baded version (ehere insert snd delete ar assos / dissoc maybe?

22:41 djames: justin_smith: right now, I'm thinking at this level: http://cstheory.stackexchange.com/questions/29516/data-structures-to-compress-similar-values-in-a-map-from-adjacent-keys

22:42 basically, I just want to see what is out there in the same rough problem area

22:44 justin_smith: ahh, so falsr and not present are distinct here

22:46 djames: justin_smith: right

22:46 http://en.wikipedia.org/wiki/Interval_tree is interesting but not quite what I want

22:47 actually, it could be what i want

22:49 dnolen: if you're feeling adventurous just landed Google Closure Module support into master https://github.com/clojure/clojurescript/wiki/Compiler-Options#modules

22:52 justin_smith: djames: interval tree was actually the thing I was trying to remember

22:53 djames: justin_smith: I think a binary search tree will work fine -- I put that as my answer. Maybe someone will come along with a better idea. Back to coding!

22:54 l1x: hey

22:54 djames: dnolen: thanks for the recent work on the 'logistical' aspects of CLJS. very helpful.

22:55 dnolen: djames: np, long out standing enhancement, glad to be finally getting to them.

22:55 l1x: what is the idiomatic way of waiting on a channel operation with a timeout in Clojure?

22:55 djames: e.g. less "sexy" but valuable

22:55 l1x: do you mean with core.async? or some other reference type?

22:55 l1x: yes, core.async

22:56 i found only this -> [[result source] (alts! [stat-chan (timeout channel-timeout)])]

22:56 djames: l1x: i was just about to point you to http://clojure.github.io/core.async/#clojure.core.async/timeout

22:57 but I think there are other ways... thinking

22:57 l1x: that is the function definition that i am aware of

22:57 i am curious about its idiomatic use

23:00 djames: l1x: I skimmed some online discussions. have you seen this? http://stackoverflow.com/questions/26487118/more-elegant-way-to-handle-error-and-timeouts-in-core-async

23:00 l1x: djames: thanks, i went through all of these

23:01 djames: l1x: are you sure the timeout needs to be handled at the core.async level? what kind of timeout are you handling? waiting on an external service?

23:01 benmoss: yeah, what comes to my mind would just be some deref

23:02 l1x: djames: exactly waiting on a queue to send a message

23:02 djames: l1x: as in, you are waiting on an external message queue?

23:02 l1x: yes

23:03 ok, here is how i use it now

23:03 https://gist.github.com/l1x/e573595307667dfb968f

23:03 arrdem: $seen bbloom

23:03 lazybot: bbloom was last seen quittingQuit: Textual IRC Client: www.textualapp.com 4 days and 4 hours ago.

23:03 djames: anyhow, I'm not an expert on core.async timeouts, but it seemed like the SO question made sense.

23:03 l1x: if you found a better way let me know

23:03 thanks

23:03 djames: l1x: that seems reasonable -- and IIRC is a normal pattern for alts! and timeout

23:04 l1x: djames: thanks bro!

23:04 i was hoping that somebody collapsed it into a macro

23:05 djames: l1x: possibly, but your style is certainly used by others. see https://github.com/search?l=clojure&q=alts%21+timeout&type=Code&utf8=%E2%9C%93

23:06 l1x: interesting

23:38 https://github.com/halgari/clojure-conj-2013-core.async-examples/blob/master/src/clojure_conj_talk/core.clj

23:38 this is so far the best in the subject

23:43 easiest to do something like this, for blocking writes with timeouts -> (alts!! [[channel message] (timeout 150)])

23:44 the return value is [true|nil channel]

Logging service provided by n01se.net