#clojure log - Feb 11 2016

The Joy of Clojure
Main Clojure site
Google Group
IRC
List of all logged dates

0:18 tolstoy: justin_smith: Remember that "node.js sockets on windows" discussion a few days back? Turns out (™), I had to execute those sorts of things in the "main" process, not in the "render" process. Why it worked great on OSX and Win7, I've no idea.

1:30 renl: Hi may i ask for gen-class, can the class be compiled into .java instead of bytecodes so I can check what code did the gen-class create?

1:43 MasseR: renl: you can always decompile the class

1:43 renl: k thanks :)

1:44 is :state in gen-class fields in a java class?

2:25 anyone used Coinor Ipopt on clojure? or for that matter are there any nonlinear programming library for clojure out there?

2:38 Voxnihili1618: hey

4:07 python476: hi, I get stack overflow with a function of two signature http://pastie.org/10717536

4:07 calling (f) overflow, calling (f <default>) runs fine

4:08 is it misguided to define a fun (defn f ([] (f <default>)) ([a] <real-logic>)) ?

4:09 opqdonut: no that's normal

4:10 the problem is you're calling "(rl [i 32])"

4:10 but you meant "(rl i 32)"

4:10 do you get the differences?

4:11 python476: wtf do I

4:11 I did

4:12 opqdonut: I often do this typo .. I need to rest

4:14 thanks a lot donut

4:37 CStorm: Building a very small api in clojure. I grab some data from an API and saves/replace it to a db ever x hour. Then an app can get that data from my API. I am new to clojure, so which database should i use for this simple project?

4:42 schmir: CStorm: sqlite or H2 may be suitable.

4:42 or just keep it in memory

4:43 hiredman: the sqlite options for the jvm are not very good, h2 is very odd about multithreading, use derby

4:43 CStorm: maybe i should start doing it en memory and then just move it to a db after

4:45 and what about deployment, which should i go for? heroku, digitalocean? were do most deploy their clojure?

4:45 schmir: I didn't run into any problems with sqlite.

4:45 CStorm: if i dont wanna roll my own.

4:45 wrote all three possiblities down, schmir and hiredman.

4:52 qsys: CStorm: I usually use H2 for that kind of apps

4:52 CStorm: thanks.

4:52 qsys: I used to think to store it as plain datastructures (compressing with nippy)

4:53 but I don't see the point of not using a db

4:53 and I would use 'lightweight datomic' if available :p

4:53 datascript might be a kind of an option, but that's very browser-oriented

4:53 so I prefer H2 so far...

4:59 CStorm: thanks.

5:01 hiredman: http://www.h2database.com/html/features.html#multiple_connections explains that h2 synchronizes (serializes via locks) access to databases by default, and if you read further the setting to change that is considered experimental (and has been for I think years now)

5:04 CStorm: Where should i deploy it all if i just want to get something going quickly?

5:05 qsys: true, but never had any issues with it so far, with H2 for small applications. Since v >1.4, it uses MVCC, so no locking

5:05 hiredman: qsys: that is, as far as I know, not true

5:11 WickedShell: is there a way to change where lein is using for scratch space? it keeps filling up disk C (on windows atm sadly) which causes all sorts of problems. I have tons of scratch space on other drives, but C: optimistically has 500mb at the start of the day...

5:12 wmealing: WickedShell: good question, does it put it in temp ?

5:14 WickedShell: wmealing, no clue, I'm assuming, although I'm looking, this is the error I see: http://pastebin.com/cw2ZRSy1

5:14 and of course the fact that there are now 0 bytes free...

5:14 wmealing: ok, lets have a look

5:15 i assume its https://github.com/technomancy/leiningen/blob/master/src/leiningen/javac.clj

5:15 line 12 makes me think that

5:15 its probably

5:15 (let [options-file (File/createTempFile ".leiningen-cmdline" nil)]

5:15 line 64

5:16 WickedShell: seems reasonable, although I'm working off a different disk

5:16 wmealing: ok

5:16 yep

5:16 it seems to call javas interpretation

5:16 java.io.File

5:16 i'm thinking, and i haven't confirmed yet

5:17 is that java looks at the windows environment variables

5:17 to find where to store tmp files

5:17 so, stackoverflow suggests its %TEMP%

5:18 start->run->%TEMP%

5:18 WickedShell: hmm yeah that's on C

5:18 wmealing: ok

5:18 so, again windows isnt my thing.. i think you can set that

5:18 http://answers.microsoft.com/en-us/windows/forum/windows_7-files/change-location-of-temp-files-folder-to-another/19f13330-dde1-404c-aa27-a76c0b450818?auth=1

5:18 see the answer given there

5:19 starting with "hi mike"

5:19 WickedShell: yeah I can change that, (ironically my temp is only 70 megs :P)

5:19 wmealing: righto !

5:19 btw, might be time to clean up C

5:19 if you're using a modern windows partition

5:19 sorry, not partition

5:19 version

5:19 WickedShell: 64GB SSD was not enough....

5:19 wmealing: do you realise that symlinks work ?

5:19 WickedShell: the whole windows side by side store thing is awful :P

5:19 wmealing: ie, to make a directory on another disk appear on another drive

5:20 WickedShell: Yeah I have a fair amount of the My Documents etc pointed elsewhere, didn't realize it had proper symlinks

5:20 wmealing: yeah, me neither till not long ago

5:20 WickedShell: a couple of programs (pcb design tools etc) require ~2GB's a pop on C and wouldnt install anywhere else...

5:20 was really annoying

5:21 * wmealing nods

5:21 wmealing: what i did, was install it… then copy it to another disk

5:21 then make the symlink

5:21 to that new disk

5:21 this doesn't work for system things though

5:21 only for non core.. stuff

5:25 WickedShell: do you need to reboot for that to work ?

5:26 WickedShell: no, usually need to reload any command prompts/enviroments there but otherwise no

5:26 wmealing: oh ok

5:28 WickedShell: thanks

5:28 wmealing: WickedShell: working ?

5:29 WickedShell: yeah

5:29 wmealing: WickedShell: awesome ! have bucketloads of fun

5:30 thankyou for flying clojure airlines.

8:18 jweiss: what's the recommended way to factor out common code from a defn where the different arities share code? private function in the same ns?

8:22 opqdonut: jweiss: if you can't just delegate to the more specific arity, then yeah private function seems apropos

8:29 Kamuela: Private functions are achieved how?

8:30 opqdonut: (defn- foo ...)

8:30 which just does (defn ^:private foo ...)

8:30 luma: (defn ^:private foo ...) is the preferred way

8:31 opqdonut: is it?

8:31 why?

8:31 clojurebot: why is Why

8:32 ridcully: Why?

8:32 clojurebot: Why is the ram gone

8:32 ridcully: so true

11:10 sdegutis: Is there a name for this kind of transformation? [[:a] [:b :c] [:d]] => [[:a :b :d] [:a :c :d]]

11:11 luma: cartesian product

11:17 sdegutis: Thanks luma.

11:17 Kah0ona: quick question about testing with Midje. when i (use 'midje.repl), and then run some test, it seems my test environment variables (using environ.core) aren't picked up, instead it picks the 'dev' environment vars

11:18 anyone any idea where I should specify it? I do `lein repl` -> `(use 'midje.repl)` -> `(autotest)`

11:23 ah start the repl with-profile +test...

11:23 duh. :)

11:48 benjyz1: hello. I have a question about protocols and methods

11:48 I have what in Java would be an Interface. I have several of them, and a wrapper around it, i.e. a facade pattern

11:49 what would this be in Clojure? I would define the interface as a protocol?

11:50 it is not a protocol, but it doesn't apply to different types.. perhaps someone has an idea

11:55 justin_smith: benjyz1: in clojure protocols are needed in order to have methods, even if only one type implements it (unless you use gen-class or an existing interface)

11:59 benjyz1: what you want is precisely a protocol - that is how we do facades in clojure

12:01 benjyz1: any different between records and types? I guess in OO everything being a class mixes many different things together

12:02 *difference

12:02 justin_smith: benjyz1: defrecord and deftypes both create classes

12:02 benjyz1: clojure doesn't hide the jvm class system, it just provides opinionated tools that simplify it

12:03 rhg135: A record is merely a class with some implementations

12:03 benjyz1: yes. so far I've taken the approach to avoid types and classes altogether. which works well on a small scale

12:03 justin_smith: benjyz1: there are no types

12:03 rhg135: Importantly, Associative

12:03 justin_smith: deftype and defrecord create classes.

12:03 defrecord automatically implements clojure's hash-map methods

12:04 (in the generated class)

12:04 benjyz1: ok, I see. I think generally class and object and type are now very precise concepts

12:04 *not

12:04 justin_smith: benjyz1: the jvm has classes and primitives

12:04 benjyz1: we do not implement anything in clojure that isn't a class

12:05 benjyz1: contrast to scala which enforces abstractions that don't exist in the bare vm, we don't attempt this

12:05 benjyz1: ah, that's where the mess in scala comes from ^^

12:06 justin_smith: haha

12:07 Glenjamin: wouldn't protocols fall into that category?

12:08 benjyz1: is it a bad idea to have deftypes with the same name as a namespace?

12:08 justin_smith: benjyz1: deftypes should use CamelCase like you would use for Classes

12:08 (as should defrecords)

12:23 rhg135: Not really, Glenjamin, the builtin nashorn engine does it too,

12:23 justin_smith: Glenjamin: do protocols enforce any restrictions their underlying Interface would not?

12:23 Glenjamin: we add functionality but not restrictions, is my impression

12:26 Glenjamin: i see, they're effectively liskov substitutable for interfaces

12:27 justin_smith: I think so

12:28 rhg135: Not exactly. If you pass one extended dynamically the method calls fail

12:33 justin_smith: rhg135: wait, since you can't dynamically extend Interfaces, doesn't this still fit liskov?

12:33 maybe I'm confused

12:34 rhg135: you may be right.

12:34 I thought it meant b must work where a does.

12:35 justin_smith: my intent was that protocols can do all things interfaces do, while offering other features as well

12:36 rhg135: Yeah, they're nice.

12:36 justin_smith: under the umbrella of clojure adding functionality but not adding extra restrictions to what the vm offers (unlike eg. scala)

12:36 agreed!

12:37 or would a scala expert counter that the jvm level features will all work and the restrictions are only on scala specific extensions in the same way? I don't know scala well enough to address that, but I wonder.

12:38 rhg135: I think #scala exists

12:39 justin_smith: yes, I might just have to go ask that question

12:39 rhg135: Good luck!

12:51 justin_smith: #scala says "using scala from other langs is a pain in the ass, but can be possible if you are careful to implement all features as java interfaces in the scala code"

12:51 so there we go

12:51 no attempt at the liskov thing there

12:59 rhg135: I guess by pita they mean more than RT.resolve(...).invoke()

13:01 justin_smith: right, they mean it isn't even safe unless the scala is restricted to a java interface

13:01 rhg135: Oh dear. That's bad

13:01 Really bad

13:02 justin_smith: there are certain constraints that the scala compiler understands that aren't even represented at the vm level

13:02 yuung: i'm writing a small blog application with clojure. coming from a rails background, i'm very used to the MVC pattern. i was wondering what your thoughts are regarding how I should implement models with clojure/how to handle db interactions?

13:02 justin_smith: so you can try it outside scala if you want but who knows what you get? haa

13:03 yuung: or if i should even stick with the MVC paradigm with a functional language

13:03 the only library i'm using is ring, and handlers sound a lot like controllers to me

13:04 rhg135: Isn't that like protocols? You get java features but if you use more it won't work

13:04 justin_smith: yuung: well, the typical preference in clojure is to use vanilla clojure data like hash-maps and lists and vectors, and even something as simple as clojure.java.jdbc will easily extend that to the db

13:04 yuung: justin_smith, yeah; i've got jdbc working. just wondering if there was an established pattern

13:05 justin_smith however i'm happy to use poco :)

13:05 sdegutis: I don't think that's what it's really called.

13:05 yuung: i guess plain old clojure objects is kind of a contradiction... let's say pocd

13:05 sdegutis: It's almost like transposing.

13:07 rhg135: pod

13:08 justin_smith: data :)

13:09 rhg135: But, acronyms are cool

13:09 yuung: ikr ;p

13:10 justin_smith: yuung: you can get pretty far with a record in db becoming a hash-map in clojure, and we have existing tools to put hash-maps into classes when we need to attach custom behaviors for OO (via defrecord).

13:10 yuung: justin_smith what should i google for the hash-map -> classes thing? also doesn't that kind of break the philosophy of fp?

13:11 rhg135: Isn't that a subset of ORM?

13:11 justin_smith: yuung: we use OO unashamedly

13:11 clojurebot: Pardon?

13:12 stoopkid: hi is clojure a total functional programming language?

13:12 justin_smith: stoopkid: no

13:12 rhg135: I don't think of records as OO

13:12 More like optimization

13:13 justin_smith: yuung: when you do (defrecord Foo [field field2 ...] SomeInterface (someMethod [this ...] ...)) it creates a function map->Foo which turns a hashmap into a Foo

13:13 rhg135: they tie behaviors to data, that's OO

13:13 rhg135: You could always use anonymous classes

13:13 justin_smith: it's not the shitty part of OO

13:13 but still OO

13:13 rhg135: True

13:14 They're hardly OO though

13:15 * rhg135 mainly considers inheritance oo

13:17 justin_smith: rhg135: I consider inheritance a parasite on programming that primarily latched onto OO

13:18 rhg135: Lol

13:19 But lately I've found that I get many code-induced nightmares due to things other than inheritance

13:24 * justin_smith just bought his tickets to clojure/west.

13:24 rhg135: When's that?

13:26 justin_smith: april 15-16 but I am going a day early for the datomic training

13:26 also they billed my opportunity grant donation as if it was a ticket which is slightly odd

13:27 rhg135: That's not too far off

13:27 I'm so excited

13:28 justin_smith: well they asked me who the opportunity grant was for so I clicked "I don't know yet" so they said it was snoozed or something

13:30 rhg135: I wonder what the talks will be about

13:31 justin_smith: an ex-coworker of mine got the official approval for a talk about learning clojure as a junior programmer, so her talk is probably just going to be about how I'm a huge jerk.

13:32 rhg135: Lmao, I need to watch that one

13:39 sdegutis: Does this make sense? (= [[:a :b :d :e] [:a :c :d :e] [:a :b :d :f] [:a :c :d :f]] (f [[:a] [:b :c] [:d] [:e :f]]))

13:40 Or https://gist.github.com/sdegutis/6bcbafa56cebb07a9da3 for pretty-printed version.

13:40 I have no idea what this kind of transformation is called.

13:44 Oh yeah cartesian-product does look right actually.

13:46 rhg135: A deduplicated one

13:47 amalloy: sdegutis: in haskell that function is called sequence. in clojure we don't really have it built in, but it's not too ahrd to write with a (for ...) that calls itself recursively

13:47 binjured: asked my wife we could name our first child amalloy, she said no and also we don't want children remember. i tried man!

13:47 amalloy: haha

13:47 sdegutis: amalloy: apparently it's clojure.math.combinatorics/cartesian-product

13:48 amalloy: yes, that too

13:48 sdegutis: Except it results in the middle two collections being swapped.

13:48 amalloy: Hmm thanks for the idea of recursive (for)s. I would like to do it that way preferably.

13:48 However I don't know how to implement it. I never took any CS classes or went to university.

13:49 I went to junior college and all I learned there was Visual Basic .NET

13:49 binjured: that's basically lisp *ducks*

13:50 sdegutis: Hmm the implementation of clojure.math.combinatorics/cartesian-product uses (loop).

13:51 amalloy: ,((fn cart [coll] (lazy-seq (if (empty? coll) '(()) (for [more (cart (rest coll)), x (first coll)] (cons x more))))) '((a b) (c) (d e) (f)))

13:51 clojurebot: ((a c d f) (b c d f) (a c e f) (b c e f))

13:51 amalloy: recursive for is an important pattern to get the hang of. it's everywhere, if you start viewing more things as trees

13:51 sdegutis: Hmm neat implementation.

13:51 rcassidy: amalloy: doesn't one use of 'for' get you what you want?

13:51 or am I missing something

13:52 amalloy: try it and see

13:52 sdegutis: amalloy: why is this a lazy collection?

13:52 amalloy: why wouldn't it be?

13:52 rcassidy: ,(for [w [:a] x [:b :c] y [:d] z [:e :f]] [w x y z])

13:52 sdegutis: It seems like an optimization, right?

13:52 clojurebot: ([:a :b :d :e] [:a :b :d :f] [:a :c :d :e] [:a :c :d :f])

13:53 sdegutis: rcassidy: yeah that was my initial solution, but I couldn't figure out how to do it with a variable number of sequences.

13:53 amalloy: rcassidy: that's fine if you have four different collections

13:53 rcassidy: ahhhhhh OK

13:53 justin_smith: rhg135: but that only works if you know ahead of time how many args

13:53 amalloy: try it with one list of lists

13:53 rcassidy: macros, maybe?

13:53 :p

13:53 justin_smith: err

13:53 amalloy: sure, get back to me when your macro can know at compile-time what data you get at runtime

13:53 rcassidy: hence the maybe. lol.

13:54 sdegutis: Maybe I can macro-expand the for-loop though and use the same logic in that but with variable number of sequences.

13:55 WHOA, never mind!

13:55 rcassidy: that's what I was thinking

13:55 but I'm not familiar with clojure macros yet

13:55 sdegutis: https://gist.github.com/sdegutis/953cf7cee228b755fd97

13:56 rhg135: Oh my.

13:56 That's a lot of code

13:57 rcassidy: lol

13:57 yikes

13:57 amalloy: for does a ton of stuff that's either (a) for performance, or (b) for handling weirdo keywords like :let or :when in the middle

13:57 sdegutis: ahh

13:57 amalloy: if you wrote for yourself it would be slower and less expressive, but much shorter

13:58 (exercise: implement for)

13:58 rcassidy: found some implementations with for and recur but oh man that's weird

14:01 rhg135: ,(macroexpand '(defrecord X [field]))

14:01 clojurebot: (let* [] (clojure.core/declare ->X) (clojure.core/declare map->X) (deftype* sandbox/X sandbox.X [field __meta __extmap] :implements ...) ...)

14:01 WorldsEndless: Can anyone verify that you CANNOT provide contingent definitions in an :or portion of an arguments definition? (unlike let, which allows you to draw on earlier-defined values)

14:02 rhg135: Macros are awesome

14:04 WorldsEndless: Here's code that's not working; I'm probably breaking more than one Clojure rule

14:04 https://www.refheap.com/114716

14:05 Contingent :or definitions, and nested -> statements

14:06 amalloy: maps aren't ordered, sot he :or clause could get reordered

14:06 nothing wrong with nested -> in principle, if used tastefully

14:06 WorldsEndless: Thanks for the verification. I figured

14:09 And if I don't provide an :or for some the :keys, and they go un-provided, I'll get an error?

14:15 amalloy: nil

14:16 justin_smith: ~amalloy |is| basically a repl.

14:16 clojurebot: Ack. Ack.

14:28 sdegutis: ~amalloy

14:28 clojurebot: That's *the* amalloy to you

14:28 sdegutis: hey *the* amalloy how's it going

14:29 rhg135: Is it just me, or is partial too long a name

14:29 sdegutis: rhg135: it's 7 chars longer than Haskell's version

14:30 About once a year I find myself thinking Haskell looks way super awesome, and then not caring anymore and going back to being productive in Clojure.

14:30 rhg135: I'm starting to think we need to reinstate the rule about haskell

14:31 It'd be nice to have a shorter name, it'd delay the #() explosion

14:33 It's not like having a clear name makes the meaning obvious

14:34 * rhg135 also likes car/cdr. *ducks*

14:37 justin_smith: rhg135: I think if partial, identity, and constantly had shorter names we'd get complaints from newcomers and much cleaner code thanks to actually using those things more often

14:37 of course it's easy enough to make ones own shorter name for all of them but it's not the same is it

14:38 rhg135: (def P partial)

14:42 There's something like that in flatland/useful

14:44 justin_smith: rhg135: regarding your earlier "wondering what the clojure/west talks will be out" the clojure/west account has been tweeting about the speakers https://twitter.com/Clojurewest

14:45 mikerod: Someone who feels strong with compiler/jvm semantics perhaps

14:45 https://github.com/clojure/clojure/blob/master/src/clj/clojure/core_deftype.clj#L35-L36

14:45 is this guaranteed that the class dynamically generated by `gen-interface` won't get GC'ed before the `import` happens?

14:45 hiredman: yes

14:45 mikerod: from what I can tell, only a SoftReference to that class exists

14:45 for a very short time

14:46 (non-AOT)

14:46 we actually had a very hacky situation where we un-imported a Class that was created via definterface, and it actually got GC'ed on us before a JIT loaded namespace tried to import that class later

14:46 admittedly this is not a good or common use-case. my question is specifically these 2 lines though

14:47 To me, for these to be safe, you'd hvae to hold a local reference to the Class returned from `gen-interface` until the `import` is complete

14:47 hiredman: specifically, it is a let instead of do there to avoid the class being gc'ed

14:47 mikerod: hiredman: I was suspicious of that!

14:48 but I have no idea why the `let` is causing a reference to be held still

14:48 I guess I need to explore that more

14:48 WorldsEndless: I find myself writing something like, (topic-keywords :file file :model model ...) for a bunch of values. I seem to remember hearing, isn't there an easier way

14:48 mikerod: I would have expected it to need to be `(let [cl (gen-interface :name ~cname :methods ~(vec (map psig sigs)))] (import ~cname))

14:48 WorldsEndless: ?

14:49 mikerod: WorldsEndless: do you mean destructruing a map?

14:49 hiredman: if it was a do, it would be a top level do, so each expression within the do would be hoisted in to its own top level form

14:49 mikerod: (let [{:keys[file model]} my-map] <etc>)

14:49 justin_smith: WorldsEndless: the solution is to never write functions that take & {:keys ...} args

14:49 and just use hash-maps instead

14:49 mikerod: justin_smith: good one :)

14:49 (inc justin_smith)

14:49 hiredman: and each toplevel form, when it is compiled, basically ends up in its own function like ((fn [] form)) for reasons

14:49 mikerod: hiredman: ok, fair enough on top-level forms. However, what if GC stops the world between `gen-interface` and `import`

14:50 can it not do that?

14:50 maybe that is where i don't understand well enough

14:50 perhaps there is some safety in unused return value references as long as it happens on within the same call stack frame or something

14:50 happens within*

14:51 I am aware that do's hoist into top-level forms though

14:51 hiredman: exactly

14:51 mikerod: interesting

14:51 hiredman: the let means it all happens within the same method

14:51 so you have a live reference on the method stack

14:51 mikerod: yes, I was thinking the return value of `gen-interface` would still have pushed that value to the stack

14:52 so then I thought - maybe it is still "live" from a GC point of view

14:52 I just wasn't sure of the guarantees there

14:54 hiredman: if the gen-interace and the import where both top level forms, the return value of gen-interface would be thrown away, between forms, and that return value is the reference to the class

14:54 mikerod: yep, that makes sense

14:55 I just didn't know that return values are safe to rely on when you don't have any reference to them right away. however, I guess that wouldn't make sense from a JVM-level since you have to push the reference to a local after it has been created

14:55 local or global etc

14:57 (inc hiredman)

14:58 I guess that doesn't work anymore - or changed :P

14:58 hiredman: it is kind of precarious

14:58 justin_smith: mikerod: bot is off the job

14:58 mikerod: I see

14:58 hiredman: I would do something like (let [a (gen-interface ...)] (import ...) a)

15:00 binjured: is there a way to reference a function from another ns in a way that doesn't lose docstring and stuff?

15:01 like `alias` but for a particular function, i guess

15:01 hiredman: you are confused

15:02 binjured: usually

15:02 hiredman: so your question doesn't make much sense

15:03 what are you doing and why do you think you are losing docstrings?

15:03 mikerod: binjured: yeah, doc strings and that sort of information is stored on the var typically if you are thinking of like (defn my-function "my docs [x] )

15:03 so you get it from the var object, not the function #'my-function

15:03 specifically (meata #'my-function)

15:03 ugh

15:03 (meta #'my-function)

15:04 binjured: just trying to provide a public api to some functions in deeper namespaces, basically

15:04 mikerod: if you want to transfer that, you can, but I'd probably use something more robust for it like potemkin

15:04 https://github.com/ztellman/potemkin#import-vars

15:04 Looks like Zach Tellmen had the same thoughts as you

15:05 binjured: ah, cool, thanks!

15:05 mikerod: I do like to use import-vars for a few things. mostly when we need to hide a dependency from some ns we don't want to expose publicly

15:05 hiredman: gross

15:05 binjured: haha

15:05 mikerod: hiredman: hmm, I mean you can do it yourself. however, the idea of needing to hide some of your namespaces from the public API has value

15:05 hiredman: nope

15:05 mikerod: and if you do it yourself, you'll wind up nearly at import-vars

15:06 jcorneli: hello, I have a question about how to override global variables

15:06 This isn't working

15:06 (set! *print-length* 10) (defmacro print-all [body] (with-redefs [*print-length* nil] `(do ~body)))

15:06 (print-all (my-function)) will only print ten items

15:06 mikerod: I have a great use-case. we have a library, we have a unbounded ton of consuming code that will use this library. if we want our consumers to use some other clj lib, like schema for example, we need to hide it behind a layer in case they make any api changes.

15:07 otherwise we can essentially never upgrade

15:07 because we cannot go fix every consumer of our lib

15:07 so we want very minimal dependency exposure to these other projects

15:07 and they are small and do not need unlimited access to the world of all clj libs

15:07 there are times you don't want to expose namespaces

15:07 hiredman: mikerod: shuffling around names in namespaces doesn't do that

15:07 mikerod: not to mention, just general code organization and movement breaks everything

15:08 hiredman: mikerod: there is no such thing as a private namespace

15:08 rhg135: That just seems like a hack around documentation

15:08 mikerod: it isn't private, you just expose a limited view of an api that comes from elsewhere

15:08 and you call it your own name, in case they have breaking changes

15:08 then you move it out of import-vars and make a pass-through that deals with the breaking change in a non-breaking way

15:08 but you have to start off with import-vars since at first, you are the same

15:09 hiredman: you are better off starting without import-vars with your own function

15:09 mikerod: allows a slower roll out of changes, whlie still allowing the dependencies to be uplifted

15:09 that part I don't get

15:09 it mostly transfers metadata

15:09 I mean, we could write it. but it is a tiny lib, so why rewrite it?

15:09 rhg135: So a layer of indirection

15:10 hiredman: don't transfer the metadata

15:10 mikerod: you can have a var point to anotehr var

15:10 then your 2 deep in calls though

15:10 I guess that's fine

15:10 hiredman: if you are creating an api over another api, do the work and create it

15:11 mikerod: Create schema over again?

15:11 hah

15:11 honestly, I see very little weirdness in what import-vars is doing

15:11 hiredman: mikerod: no, you say you are carving out some specific subset

15:11 and exposing that as a new api

15:11 mikerod: yes, like - everyone use schema stuff that we have exposed

15:12 nothing else in case they change a name or something on us in a newer release

15:12 (this has happened)

15:12 and not only in schema

15:12 hiredman: if that is what you are doing, you should write functions that do that (which may call schema) with their own docstrings describing the semantics of your api

15:12 mikerod: it is identical

15:12 so we are just copy/pasting

15:12 import-vars = automate that

15:12 not to mention there is an entirely different argument here

15:12 to be said for the goals of import-vars

15:13 that your api structure doesn't need to be your code organization structure

15:13 I haven't really went down that path, but I think there is value in that idea

15:13 hiredman: what you have created is just a worse form of vendoring all your deps

15:13 mikerod: it is like vendoring, I'd agree with that part

15:13 hiredman: so vendor

15:13 mikerod: but I don't see the "worst" argument

15:13 we'd rather shade it in and rename all namespaces

15:14 so they don't conflict with consumers versions etc

15:14 but remapping namespaces is risky in clj I'd say

15:14 hiredman: because you don't actually have your own copy of the code that you have control over

15:14 mikerod: so our consumers don't have a dep on schema at all, for example

15:14 is how it works to import-vars over it

15:15 so we control the version, we control introduction of breaking changes regardless of the version we pick up

15:15 and if we use another lib, that also uses schema, and we need to get to the same version to be compatible between our lib and theirs, we can. while still not breaking our consumers

15:15 since our consumers don't see these breaks immediately

15:16 Perhaps there are other paths to fixing it. import-vars seemed pretty straight forward.

15:17 and maybe it is a specialized case in general

15:17 basically though, I was going to write similar to import-vars, then I saw it was written already and used quite a bit across other projects

15:21 hiredman: a lot of bad things are done in a lot of projects

15:21 mikerod: Ok, but I still haven't been very convinced of the pitfalls of this approach.

15:22 I mostly gather that you don't like it and think it is weak vendoring of some sort.

15:22 So I'm always open to good suggestions that explain the pro's and con's. I just am not hearing them.

15:22 hiredman: I think what you are doing is

15:22 (weak vendoring)

15:22 I don't think all uses of import-vars are

15:23 mikerod: I said I see 2 entirely separate camps of usage of it. (1) decoupling code organization from API organization.

15:23 hiredman: in generally, they seem to be the result of poor code organization, and/or the unwillingness to tackle large refactorings

15:23 mikerod: (2) isolating dependencies

15:23 I guess you can call it unwilling, but the reality is its consumers out of "our control"

15:24 old things have to keep working until some end-of-life

15:24 it is always fun and nice when you can go uplift the world whenever you want, but not always practical

15:24 pullo: does anyone know why, when I 'lein deploy' to a private repository (just a folder on my local drive, different from ~/.m2), the task also creates an empty (i.e. no jar) artifact in my ~/.m2 repo?

15:25 mikerod: and the reality is, we like schema. our consuming projects should use schema. we can't risk being in conflict of versions between us and our consuming libs on schema.

15:25 pullo: I currently have to delete the empty artifact in my ~/.m2 if I want to resolve dependencies from the private repo without lein crapping out.

15:25 mikerod: going with the schema theme

15:26 hiredman: mikerod: sure, if you are committed to providing a compatible api to a legacy schema version, then commit and create a namespace with the set of functions you support, with the doc strings spelling out the semantics you support, and then call out to schema

15:27 mikerod: hiredman: so just have to do that for like 50 vars or more

15:27 just sounds like manual work, where a machine can do it just as well

15:27 hiredman: if a schema docstring says "look at website example.org for how feature X works"

15:27 mikerod: when we have an exceptional case, a wrapper, we use our own new var to do it, with its own doc string

15:27 we don't import-vars if we have changed it

15:27 hiredman: and a consume bookmarks that website and uses it as their reference

15:27 mikerod: that wouldn't make snese

15:28 Ok, that is a good point on how a doc string could become misleading

15:28 if it was specific to that version, then they don't realize other vars they are using are different

15:28 hiredman: what happens when you use whatever editors "view source" on your facade?

15:28 mikerod: although they are different namespace, so you really can't just assume they aren't different

15:29 with import-vars, looks to jump to the original source

15:29 I understand that your cross reference can become weird from doc strings

15:29 that is a good point on where it has a pitfall

15:30 "this function uses function-name-x"

15:30 however, we don't expose function-name-x because it was a breaking change, so we still have function-name-y instead

15:30 so you'd hvae to look at function-name-y's doc string, which would say "this is a pass through to function-name-x but left for compatibility with previous versions of lib X"

15:30 but still, yes I could see some confusion

15:32 hiredman: unless you are truely vendoring (copying and renaming namespaces) your consumers (and any of their dependneices) could replace the version of schema on the classpath at any time

15:36 jcorneli: I'm still stuck with rebinding *print-length* in a macro

15:36 the latest non-working version I've come up with is:

15:36 (defmacro print-all [& body] `(do (binding [*print-length* nil] (do ~@body))))

15:36 justin_smith: jcorneli: what is the outer do for?

15:36 or the inner one, even

15:36 pullo: ~'*print-length*

15:36 clojurebot: I don't understand.

15:37 hiredman: jonathanj: there is no call to print anything in there, why would binding print length do anything?

15:37 mikerod: hiredman: yes they can, and that would break things. it isn't consumers of arbitrary libraries int hat sense

15:37 so yes, specialized

15:37 jcorneli: the call to print is in the body

15:37 or in the ~@body, rather

15:37 rhg135: justin_smith: her tweet about the talk didn't even mention that job. I think you are safe :P

15:37 justin_smith: rhg135: it's just jokes, yes

15:38 hiredman: jcorneli: what does it look like?

15:39 jcorneli: The inner function is meant to be "any code"...

15:39 it could be '(1 2 3 4 5 6 7 8 9 10 11)

15:40 hiredman: jonathanj: (def f (print-all (fn [] (prn (range 20))))

15:40 then (f)

15:40 jcorneli: there is no call to println in that quoted list

15:40 jcorneli: But *print-length* tells how many items to print from a list

15:41 Given (set! *print-length* 10), then (print-all '(1 2 3 4 5 6 7 8 9 10 11)) returns (1 2 3 4 5 6 7 8 9 10 ...)

15:41 or rather, that's what it prints to the REPL

15:41 hiredman: jcorneli: yeah, but unless a call to print is in the dynamic extent of the body of the binding the binding doesn't exists

15:42 jcorneli: the repl prints results after they have been evaluated, not inside some arbitrary scope in the evaluation

15:42 jcorneli: ok, think I see what's going on

15:43 hiredman: think of each expression you type in to a repl as a function body, that is called, and then the repl prints the result

15:43 jcorneli: It's interesting because (defn save-local [] (binding [*print-length* nil] (spit "data.clj" (prn-str @node-store)))) works as expected

15:43 But that's not printing to the REPL but rather to a file

15:43 hiredman: nothing "prints to the REPL"

15:43 jcorneli: Well, CIDER does :-)

15:43 hiredman: the repl evaluates an expression, then prints the result

15:44 justin_smith: jcorneli: you aren't relying on the implicit print behavior there, the implicit print of the repl happens outside any of your dynamic scope

15:44 hiredman: the repl is driving the printing

15:44 jcorneli: yes, this is dawning on me now

15:44 justin_smith: unless you started said repl inside your dynamic scope

15:45 jcorneli: Thanks for the help guys

15:50 amalloy: hiredman: i would say that (print 5) is "printing to the repl", because it's printing to a *out* which is configured to go straight to the repl

15:51 hiredman: semantics

15:52 "printing to the same output stream as the repl"

15:54 if you embed some kind of repl in your app a common thing is you end up wanting to limit the printing, and that can often start out as creating a custom evaluator that sets the bindings, which is of course terrible, you should create a custom printer that sets them

15:54 sorry, not a custom evaluator

15:54 wrapping the whole repl in the bindings

16:16 sdegutis: What's that function again that lets you count the numbers of things in a collection, assuming there are duplicates?

16:16 It's some obscure word that I almost remember but can't quite.

16:17 occurrences?

16:17 TEttinger: distinct

16:17 sdegutis: TEttinger: That just removes duplicates without counting them.

16:17 TEttinger: maybe

16:17 sdegutis: TEttinger: The function I'm thinking of counts them. It's kind of like group-by in that sense.

16:17 Hmm let me look for that Clojure cheat sheet, maybe that has it.

16:18 frequencies!

16:18 Phew. Thanks TEttinger.

16:19 TEttinger: ohhh

16:20 yeah I was about to suggest frequencies, it didn't sound like what you wanted

16:20 I thought you meant #(count (distinct %))

16:20 jcorneli: justin_smith hiredman: I believe I've found a simpler solution that actually works: (defn print-all [body] (let [result (do body)] (binding [*print-length* nil] (print result))))

16:20 thanks again for the guidance

16:21 hiredman: that is nonsense

16:22 it is trivially made simpler and better

16:23 (let [x y] x) is just x

16:26 justin_smith: *(let [x (do y)] x)

16:28 sdegutis: hiredman: you mean y right?

16:38 ,(let [y 2] (= y (let [x y] x)))

16:38 clojurebot: true

16:38 sdegutis: Anyway, programming is hard.

16:38 justin_smith: sdegutis: you left out the do

16:38 sdegutis: justin_smith: Correct.

16:39 justin_smith: ,(do 'b (do 'b (do))) ; sinatra

16:39 sdegutis: My original CSS stuff was very incorrect, I'm trying to rewrite it to be correct, and it's a *very* recursive problem to solve. So I'm struggling to think recursively right now to try to figure that out.

16:39 clojurebot: nil

16:39 sdegutis: justin_smith: hahaha hhaha

16:39 justin_smith: sdegutis: I am afraid of CSS

16:40 sdegutis: justin_smith: CSS itself isn't great, but what I'm struggling with isn't inherent to CSS itself. It's just a transformation of one Clojure structure into another.

16:41 The input can either be a string, a hash-map, or a list containing any of these three things. The output needs to be a re-structured version of it that's not too dissimilar.

16:41 So my natural idea was to create a recursive function that does a cond, with list? string? and hash-map? as its only three conditions.

16:42 But that's where I'm stuck. When it encounters a string, it needs to "add" that to some kind of list, via a return value, which presumably appends to a collection obtained via a parameter one level up.

16:43 Arghh, recursion is hard.

16:44 Why after a decade of professional programming is recursion still so hard?

16:46 amalloy: understanding recursion is only hard because you haven't yet understood the rest of recursion

16:47 pbx: http://www.badum-tish.com/

16:48 justin_smith: amalloy: the hardest part is figuring out when you can escape learning recursion.

16:51 lockdown: sdegutis: because you didn't do "The Little Schemer"

16:51 sdegutis: amalloy: whoa

16:51 justin_smith: oh man

16:51 this is too intense

16:56 rcassidy: what step of the design recipe are you on? /s

16:58 (but seriously, it helps with recursion)

17:06 bendlas: Hi people, any news on the missing latest source packages?

17:06 GET https://commondatastorage.googleapis.com/chromium-browser-official/chromium-48.0.2564.109.tar.xz 404s

17:07 justin_smith: bendlas: are you looking for google closure?

17:07 bendlas: justin_smith: ooh sorry

17:20 TimMc: rcassidy: was gonna say it but didn't :-)

17:25 xemdetia: why are we writing faces

17:25 sorry my brain is broken please ignore

17:30 justin_smith: xemdetia: ¯\_(ツ)_/¯

17:45 xemdetia: justin_smith, I have that as a macro in st

17:45 for emergencies

18:14 cortexman: what's the right / easy way to read an edn file off disk

18:15 justin_smith: cortexman: do you want the string or the clojure data?

18:16 for just string, slurp, for clojure data (read-string (slurp f))

18:16 if you don't trust the file (maybe it came from uncertain source) use clojure.edn/read-string

18:23 amalloy: if you do trust the file, use clojure.end/read-string anyway

18:27 justin_smith: yeah, I guess edn/read-string is the better one to use all around now

18:28 rhg135: for edn? yeah always

18:29 it's kinda like reading json with eval. it technically works but eventually it'll mess you up

18:45 sdegutis: Is there some simple way to copy a string to an output stream that I'm missing?

18:46 justin_smith: sdegutis: into an existing stream or create a new one based on that string?

18:46 sdegutis: Into an existing output stream.

18:46 justin_smith: and .getBytes isn't doing it?

18:46 sdegutis: The only thing I can think of is to convert the string into a UTF-8 thing using .getBytes

18:47 rhg135: ,(do (require 'clojure.java.io) (doc clojure.java.io/copy))

18:47 clojurebot: "([input output & opts]); Copies input to output. Returns nil or throws IOException. Input may be an InputStream, Reader, File, byte[], or String. Output may be an OutputStream, Writer, or File. Options are key/value pairs and may be one of :buffer-size buffer size to use, default is 1024. :encoding encoding to use if converting between byte and char streams. Does not close any streams except thos...

18:48 sdegutis: rhg135: when copy receives a string, it assumes it's a filename or URL.

18:48 rhg135: I'd need to convert a string into an input stream or something first to use it with copy.

18:48 rhg135: StringReader yo

18:48 sdegutis: Which isn't a half-bad idea I guess.

18:48 justin_smith: or just getBytes

18:48 rhg135: or that

18:49 sdegutis: Heck yeah I just created a zip file in Clojure.

20:02 jack_rabbit: I'm reading core.async docs. If I use >!! or <!! in a go block, what happens?

20:03 I assume it blocks the thread executing the block rather than yielding?

20:04 amalloy: justin_smith: getBytes :(

20:05 it's fine if you need bytes, but converting from characters to bytes in order to process a string should be a sin

20:25 sdegutis: How do you write an InputStream to a file?

20:25 I tried (spit "filename" istream) but no luck.

20:27 rhg135: io/copy

20:28 (io/copy in (FileWriter. name))

20:28 io/copy is the answer. always.

20:34 sdegutis: Thanks rhg135.

20:35 rhg135: np

20:36 I wish something like it existed for node/cljs

20:36 but I'm not masochist enough to write it myself

20:38 sdegutis: This is so strange.

20:39 I'm copying from a non-empty byte-array to a file, and it's turning out an empty 0-byte file.

20:39 I'm baffled.

20:57 Okay whatever it works now good enough.

20:58 amalloy: you're probably not closing the writer

21:06 rhg135: maybe I'll write it for nashorn/cljs where it's basically a sed

21:08 justin_smith: amalloy: but outputstreams use bytes not strings

21:17 amalloy: i mean, when you have characters and you're writing to a byte-oriented stream, you need to at least consider encoding. just slapping utf-8 on it is not a solution

21:18 you need to make sure whatever's on the other end is prepared to receive your string as utf-8, or else when you get a user in asia your io code will just break

21:21 noncom: hi!

21:22 can somebody advice a way to organize a file download in compojure?

21:22 or give a link to an example..

21:22 i can't find a working example of this

21:22 i have a byte array for the file

21:39 rhg135: if we all just used utf-8 we wouldn't have that problem, but we'd have the problem of noobs in weakly typed languages using bytes as mutable strings

21:41 noncom: i have found a solution

21:42 rhg135: huh? it's the first time i hear of that

21:43 rhg135: I saw it all the time in python2

21:43 it had no byte type

21:44 it was nightmare fuel

21:44 noncom: omg

21:44 justin_smith: multi-byte-character? what's that? who ever heard of such a thing?

21:45 wmealing: nobody ever.

21:45 all those chinese people.. they never heard of it

21:45 rhg135: it's funny how much they protested at python3's sanity

21:45 wmealing: if you dont do foreign languages, its pretty easy to miss

21:45 ie, not see the need for

21:45 justin_smith: wmealing: I mean, for a multi byte character to be needed, that would require more than 256 symbols

21:46 and I mean there's gotta be less than 100 symbols, total

21:46 rhg135: "we don't need no bytes! it's too hard!"

21:46 wmealing: justin_smith: :)

21:46 rhg135: "now we have to be explicit"

21:47 * rhg135 recites the zen of python

23:19 nikki93: do people like "clojure in action" 2nd ed?

23:39 skjljsk: https://github.com/adzerk-oss/boot-test boot test break if there is a custom source file with build tasks?

23:39 clojure.lang.Compiler$CompilerException: java.io.FileNotFoundException: Could not locate boot/core__init.class or boot/core.clj on classpath., compiling:(build/tasks.clj:1:1)

23:43 anyone else faced this?

Logging service provided by n01se.net