#clojure log - Jun 28 2015

The Joy of Clojure
Main Clojure site
Google Group
IRC
List of all logged dates

7:05 mmeix: just found https://atom.io ... looks interesting. Unfortunately can't get the nrepl package to do anything so far ...( https://atom.io/packages/nrepl ) - anyone tried this?

7:11 kwladyka: mmeix, light table could be also great but it looks like author abandon this project? who know what will happen with atom.io... i don't like immature projects anymore :)

7:24 mmeix: Light Table I'm just using now

7:34 ifies: Hello #clojure! I'm trying to insert something into a vector at a certain position. I want to transform [\a \c] into [\a \b \c]. I'm trying with (assoc [\a \c] 1 \b), but that only 'changes' the \c into a \b (returning [\a \b]). In Javascript I would (have to) write something like `myArr = ['a', 'c'] ; myArr.splice(1,0,'b') ; return myArr;`. Is it not possible to to this in a Clojure vector?

7:51 rritoch: ifies: There are many ways of doing it, one way which isn't too painful is (reduce into [(subvec myvec 0 idx) [invalue] (subvec myvec idx)])

7:52 ifies: Where myvec is the starting vector, invalue is what your trying to insert, and idx is where you want to insert at.

7:55 profil: is it possible to merge two core.async channels' output and input? I want to combine the input from channel 1 with the output from channel 2. If I take from my combined channel, it should take from channel 1, and if I put it should put to channel 2.

7:56 Does this exist or do I have to use reify to get what I want?

8:06 kwladyka: https://www.refheap.com/104534 <- which function should i use to solve that?

8:06 in most elegant way :)

8:08 rritoch: profil: How is that any different than pipe? I think I just misunderstand your question.

8:09 profil: http://clojure.github.io/core.async/#clojure.core.async/pipe

8:12 profil: rritoch: Internally I have read-ch and write-ch. read-ch is written to when data is incoming on a websocket connection, write-ch is used to send data over the websocket. I want to combine read-ch and write-ch into ch so that when i do (<! ch) it runs (<! read-ch) and when I do (>! ch) it runs (>! write-ch)

8:15 rritoch: I am thinking about solving it like this, https://www.refheap.com/104535

8:15 ops, it should be a WritePort in there aswell

8:17 rritoch: profil: You know, that is a difficult challenge in any language. I think your going to end up using reify because you need your socket using one API, and your socket user using a different API. At least that's how I've solved it in other languages. Like users call read/write but the socket listener/writer calls readLow/writeLow

8:32 profil: rritoch: I did it like this, https://www.refheap.com/104536

8:32 just need to test it :)

8:35 rritoch: Anyhow, I'm heading out, I just wanted to let clojure dev's know that the Java docs are apparently not so accurate. >> https://github.com/openjdk-mirror/jdk/blob/jdk7/jdk7/master/src/share/classes/java/lang/Double.java#L507-L522 Last I saw clojure source it was using valueOf boxing for optimization purposes, but it turns out that the optimization claimed by the documentation doesn't exist

8:35 At least not in openjdk.

8:36 It was recently pointed out to me when someone asked why I was using the valueOf method.

8:36 Bronsa: rritoch: the documentation doesn't guarantee the optimization to exist

8:37 rritoch: Bronsa: Sure, but it's VERY misleading, since calling the constructor directly should technically be faster on any implementation that doesn't have it optimized.

8:39 Bronsa: It should at least have a cache of 0.0 since it is the most common value... anyhow. It doesn't, so optimizing based on the documentation seems to be futile.

8:40 I've been running into similar issues with tomcat, where the documentation says one thing, and the code does something completely different.

8:41 Bronsa: rritoch: http://code.metager.de/source/xref/openjdk/jdk8/hotspot/src/share/vm/ci/ciMethod.cpp#1305

8:45 rritoch: so it might not be cached but it seems to be used for certain unboxing optimizations

8:47 ifies: rritoch: Thank you! That is just what I needed. I didn't know the subvec function, thanks for showing me! Imho the code you wrote should be in clojure.core...

8:47 Bronsa: rritoch: hotspot will take care of inlining the /valueOf method call when necessary anyway so there's really no point in thinking about it

8:47 rritoch: ifies: lol, thanks. Been down that route before though, I'd rather pull out my own teeth with pliers than make another code submission to clojure.

8:49 Bronsa: Well it is certainly interesting. I've run into many cases where opimizing large amounts of doubles is important, such as video processing.

8:55 drwin: dnolen: FYI playing with cljs-bootstrap to give me ASTs for arbitrary files, want to use it in my experimental structural cljs editor: https://github.com/darwin/quark/blob/master/cljs/src/base/quark/cogs/editor/analyzer.cljs

8:56 looking forward to have some equivalent of 'analyze-file', right now it gives me only very basic ASTs, no requires work for me

8:56 dnolen: drwin: cool!

8:57 rritoch: Bronsa: My big question thats unanswerd regarding array optimization and isn't really documented as far as I can tell is how much memory does an object pointer use in a typical implementation. If the object pointer is 64 bits, and your cached double is 64 bits, than the object pointer and the native value use the same amount of ram so there's really no potential for optimization via caching values

8:57 dnolen: drwin: not sure about supplying analyze-file as it will look differently everywhere

8:57 io, file resolution, etc. there not a good way to make this work consistently across all JS targets

8:58 and I'm not really interested in it myself.

8:58 since once you have a bootstrapped thing you can go sort it out on your own.

8:58 rritoch: Bronsa: But if the underlying system is using 32 bit object pointers, than caching double objects in an array should use less ram.

8:58 drwin: dnolen: I'm happy to reimplement it for my editor-use-case, just wanted to ask if analyzer can work if I implement it, I'm not fully aware of the state of analyzing in cljs

8:59 dnolen: drwin: what do you mean "state of analyzing in cljs"?

9:00 TMA: rritoch: sun-jdk has X-bit object pointers in their X-bit versions. I assume openjdk and other JVMs do the same

9:01 drwin: I understand that you are in the process of adapting compiler infrastructure to be bootstrapped in js, my code will be cljs-only, so I need to analyzer to work without clojure. I'm not sure if it is possible at this point

9:02 I could rewrite analyze-file and friends (dependency resolution using nodejs fs apis, etc.), but I'm not sure if don't hit wall because some parts of clojurescript analyzer are not yet ported to cljs

9:02 TMA: rritoch: as a rule of thumb: a 64-bit JVM instance takes approximately double the amount of RAM for the same program as the 32-bit ones

9:03 drwin: I'm not interested in emitting at this point, just analyze

9:04 dnolen: drwin: bootstrapping isn't involving changing anything about the analyzer and the compiler

9:04 it is the exact same compiler, exact same analyzer

9:04 drwin: everything works

9:04 rritoch: TMA: Thanks. That is very helpful, it implies that caching is only useful when the amount of cached data > architecture size, so caching complex numbers or Big* numbers would save ram, but anything smaller than architecture size is better as native values.

9:04 dnolen: the only place things get tricky are the bits that need I/O

9:04 Bronsa: dnolen: cljs-bootstrap -- no need for your t.r fork. the CLJS-1303 branch in t.r should work

9:05 dnolen: analyzing dependencies right now, loading files

9:06 drwin: dnolen: ok, I'm happy to hear, so in theory if I dive into it I could be able to get ASTs from any cljs file

9:06 dnolen: drwin: my test case is analyzing the entire standard library

9:06 so that better work :)

9:07 Bronsa: noted, I couldn't remember which branch it was

9:08 drwin: dnolen: when I eval more complex code I'm getting: "Cannot use 'in' operator to search for 'defn' in null", didn't investigate it yet, will have to first try to understand how namespaces are resolved and recorded in the compiler env

9:10 dnolen: drwin: right the way you have to deal w/ macros at the moment is a bit manual not generalized

9:10 I would hold off unless you're brave enough to dig into how the analyzer/compiler works and how the bootstrapped thing has to be different accordingly in certain areas.

9:13 drwin: should I store their defs in related.namespace.name$macros? I saw some new switch logic

9:13 I have to start somewhere, I'm gonna try, in the worst case I'll just fail :)

9:16 dnolen: drwin: yes we rewrite macro namespaces so that macro nses and runtime nses can appear to reuse symbol names.

9:25 rritoch: Is it possible to disable the clojure security manager? Unlocking some of the final values in clojure's core via reflection would be a way that I could run clojure customizations without needing to run an actual fork of clojure.

9:27 Alternativly I was thinking of just loading clojure with a custom classloader that strips all finals which would open up unlimited customization potential, but for my purposes right now just unlocking finals would be enough.

9:28 Bronsa: rritoch: what is clojure's security manager?

9:32 mmeix: (not to disturb the high-level discussion: do we have a bot in #clojure-beginners, for demonstration purposes? like this:

9:32 , (filter even? (range 7))

9:32 clojurebot: (0 2 4 6)

9:33 mmeix: (maybe with another start symbol?)

9:33 rritoch: Bronsa: I honestly don't know, I haven't dug that far into the issue. I just know that clojure's security manager yells at me if I try to access finals.

9:33 ImportanJoe: ,((comp (partial apply +) reverse (partial filter even?)) (range 7))

9:33 clojurebot: 12

9:35 rritoch: Bronsa: Nevermind. I guess I was just doing it wrong

9:36 ImportanJoe: ,((comp (partial apply + reverse filter even?) (range 7))

9:36 clojurebot: #<RuntimeException java.lang.RuntimeException: EOF while reading>

9:36 ImportanJoe: ,(comp (partial apply + reverse filter even?) (range 7))

9:36 clojurebot: #object[clojure.core$comp$fn__4493 0x660bbc9a "clojure.core$comp$fn__4493@660bbc9a"]

9:36 rritoch: Bronsa: THis worked... (let [x (.getDeclaredField String "value") y "foo"] (.setAccessible x true) (.set x y (.get x "bar")) y) so I just need to be more careful with the types when accessing finals.

9:52 Bronsa: Actually, I was able to fire the security manager

9:53 Bronsa: With (let [x (.getDeclaredField clojure.lang.Namespace "namespaces") y (java.util.concurrent.ConcurrentHashMap.) (.setAccessible x true) (.set x nil y))

9:54 Bronsa: There was a ] after the ConcurrentHashMap.)

9:55 Bronsa: But I think this may still be a typing issue, since I didn't create a ConcurrentHashMap<Symbol, Namespace> but I only created a regular ConcurrentHashMap

9:59 Bronsa: That one literal "final" in that one file is the only reason I stopped using clojure. If I can't isolate namespaces in a way that's compatible with OSGI, I can't use clojure.

10:00 Clojure is going to have lots of fun with Java 9 which is likely to include OSGI features.

10:07 But then again, maven is probably going to have equal troubles when people have dependencies provided by modules. I haven't yet tried the new features because the documentation is sparce, but I'm fairly sure java 9 is going to break half my libraries.

10:07 sparce=sparse

11:08 sharms: If I am destructuring a map, is there a form which lets me add a prefix to those keys? ex { :mfg "something" :version "1" } - I want to do a let [{:keys [:mfg :version]} something] and instead of mfg, I want to alter it to say something/mfg

11:09 justin_smith: sharms: don't use :keys then

11:09 sharms: well there are like 12 fields, so I dont want just a giant let statement

11:09 justin_smith: ,(let [{some-a :a some-b :b} {:a 0 :b 1}] some-b)

11:09 clojurebot: 1

11:09 justin_smith: also, you can't create a binding in a let that has a / in the middle

11:10 sharms: they would all be keywords like :newprefix/mapkey

11:10 I am just trying to insert a map into datomic with the proper namespace

11:10 justin_smith: ,(let [{:keys [a]} {:whatever/a 0}] a)

11:10 clojurebot: nil

11:10 justin_smith: hmm

11:10 clojurebot: Excuse me?

11:10 crocket: hi

11:11 justin_smith: ,(let [{:keys [whatever/a]} {:whatever/a 0}] a)

11:11 clojurebot: 0

11:11 crocket: If a transaction refers to a ref named 'hobbit', does the transaction obtain the snapshot of hobbit as of the start of transaction or the first read of hobbit in the transaction?

11:12 sharms: at the end of that statement, 'a', I want :newprefix/a to be that

11:12 justin_smith: sharms: so you want to generate a new keyword, not bind the keyword's value?

11:12 sharms: I want the value which was in the map bound to it

11:13 justin_smith: you can't bind values to keywords, closest you can get is use them as keys with the val attached

11:13 sharms: (defn add-hardware [hardware]

11:13 @ (d/transact conn [{:db/id (d/tempid :db.part/user)

11:13 :hardware/mfg (:mfg hardware)

11:13 :hardware/model (:model hardware)

11:13 :hardware/version (:version hardware)}]))

11:13 that is basically what I am trying to do

11:13 justin_smith: sharms: don't do that

11:13 use a pastebin

11:14 crocket: justin_smith, ^^

11:14 sharms: but hardware has like 12+ fields, so I didnt know if there was an easy way to remap it all to :hardware

11:14 justin_smith: sharms: OK, this has nothing at all to do with destructuring then, you just want to rename a bunch of keywords in a hashmap

11:15 mmeix: couldn't you use something like {:hardware {:model ... :version ...}} ?

11:15 justin_smith: ,(into {} (map (fn [[k v]] [(keyword "hardware" (name k)) v]) {:a 0 :b 1 :c 2})

11:15 clojurebot: #<RuntimeException java.lang.RuntimeException: EOF while reading>

11:15 justin_smith: ,(into {} (map (fn [[k v]] [(keyword "hardware" (name k)) v]) {:a 0 :b 1 :c 2}))

11:15 clojurebot: {:hardware/a 0, :hardware/b 1, :hardware/c 2}

11:15 crocket: What is the read point of a transaction?

11:16 documentation at Clojure.org states "All reads of Refs will see a consistent snapshot of the ‘Ref world’ as of the starting point of the transaction (its ‘read point’)."

11:16 justin_smith: ^^ sharms that's what you want (probably with a select-keys then merge)

11:16 crocket: the transaction locks the refs

11:16 sharms: mmeix - tried that and it only has the keywords present, but nothing bound to them

11:16 justin_smith: crocket: other parts of your code have to wait until the transaction is done

11:16 crocket: at the start of a transaction? or at the first read of a ref in a transaction?

11:17 sharms: justin_smith: thanks I will give that a shot

11:17 crocket: I want to know when a ref is locked in a ref.

11:17 at the start of a transaction or at the first read of a transaction.

11:17 justin_smith: crocket: the transaction locks all refs and agents that it accesses

11:17 at the start

11:18 crocket: the compiler makes a note of all refs and agents that are accessed when compiling the transaction form

11:18 crocket: If a transaction locks all refs and agents in the beginning in advance, it means the transaction knows which refs and agents it accesses in advance. Right?

11:18 Very cool

11:18 justin_smith: right, because it compiles the form

11:18 clojure is not an interpreted language (though you can fake it with things like trampoline)

11:19 crocket: Thanks

11:20 sharms: justin_smith: that did work thanks, that really helped with respect to understanding destructuring vs manipulation of a map

11:21 crocket: Is it ok to apply commute and then alter to a ref in a transaction?

11:22 Will it lead to infinite retries?

11:22 justin_smith: oh, wait - do transactions lock or retry?

11:23 crocket: I was wrong http://clojure.org/refs#Refs%20and%20Transactions this is the real info

11:24 crocket: ??

11:24 clojurebot: ? is suddenly

11:25 crocket: ok

11:25 * laxask suddenly

11:25 justin_smith: crocket: transactions do not lock, they retry, it doesn't matter how many alterations or commutes are inside your transaction, the retry happens iff any of those refs changed outside the transaction while you were working

11:25 crocket: If I apply commute and then alter to a ref in a transaction, I encounter an exception.

11:25 justin_smith: crocket: interesting, you should be able to do both

11:25 crocket: I can't alter a ref after applying commute to a ref.

11:26 Clojure programming says "The unique semantic of alter is that, when the transaction is to be committed, the value of the ref outside of the transaction must be the same as it was prior to the first in-transaction application of alter. Otherwise, the transaction is restarted from the beginning with the new observed values of the refs involved."

11:27 Thus, commute should be able to break alter...

11:28 justin_smith: crocket: when the transaction is commited

11:28 if both are inside the transaction, the commit hasn't happened yet

11:29 crocket: I think "clojure programming" got it wrong.

11:29 justin_smith: crocket: no, it's correct. It's talking about inside the transaction vs. outside, both modification happen inside the transaction, that should work

11:30 crocket: justin_smith, Anyway, I can't alter after commute.

11:30 Alter wants to be the first one to alter a ref.

11:31 If you apply alter, commute, and alter in the exact same order to a ref, you encounter an exception.

11:31 "can't set after commute"

11:32 can't alter after commute

11:34 justin_smith: interesting, yeah, commute does something weird there...

11:34 but that's not because of the out-of-transaction value changing

11:35 crocket: justin_smith, Are you sure the value of every ref is snapshot at the start of a transaction?

11:35 justin_smith: crocket: absolutely, that's what drives retries

11:36 crocket: ok...

11:39 justin_smith: crocket: it compiles the transaction form, captures the values of all refs accessed, then runs the transaction (but none of its changes are visible outside the transaction yet). If none of the refs changed in value in the meantime, all the changes to the refs are applied atomically

11:39 crocket: justin_smith, http://ibin.co/26o26cO39gvZ fooled me.

11:39 In the figure, t2 reads the value of a ref in the middle.

11:40 justin_smith: crocket: yes, and that a1 != a2 shows the retry condition being calculated

11:41 crocket: t2 is not reading, it is changing a2

11:41 crocket: The figure makes it sound as if there would be no conflict if t1 read a after t2 commited.

11:41 The figure makes it sound as if there would be no conflict if t1 read 'a' after t2 commited.

11:41 justin_smith: t1 does all its reads at once (at the start)

11:41 crocket: So does t2?

11:42 It seems the author fooled a lot of people.

11:42 justin_smith: right, but t2 started and completed it's alteration of a while t1 was still running

11:42 which means t1 retries

11:44 crocket: the official answer to the issue about commute https://groups.google.com/forum/#!msg/clojure/NGiWsqheUjo/9lWI-JBqXo8J

12:05 crocket: going home

13:01 jonathanj: what is the return value of schema.coerce/coercer when there is a coercion/validation failure?

13:02 apparently it's some value of type `schema.utils.ErrorContainer` but there's nothing that describes how to deal with this, apparently?

13:23 justin_smith: jonathanj: my instinct would be to check for and handle that type, but maybe there is something more official

14:05 nano-: Is it possible to configure ring to do http2?

14:07 justin_smith: nano-: ring is a protocol that various http servers are extended to support, your best bet would be to find out what java / jvm http server supports http2, then look for a ring adapter for that server

14:07 nano-: Oh, thought ring was a jetty wrapper. I guess I have some reading to do.

14:08 justin_smith: nano-: well it's a "wrapper" for any http server on the jvm - it's a protocol that makes http servers more convenient to use from clojure

14:08 nano-: cool, thanks for clearing it out.

17:19 Reefersleep: Hello everyone

17:20 I'm trying to get vim-fireplace working with ClojureScript

17:20 I've started a lein repl in my project root, I've :Connect'ed to it, I've lein figwheel dev from root, and then, when I do :Piggieback, I get

17:20 ClassNotFoundException cemerick.piggieback java.net.URLClassLoader$1.run

17:21 I've included piggieback and nrepl from my project.clj as per the instructions at the piggieback github repo

17:21 So I can't really figure out what I'm doing wrong :/

17:21 I've tried :Piggieback [7888] (the port for my figwheel nrepl) and gotten the same message

17:21 So I figure something's wrong with my piggieback dependency

17:21 but I can't figure out how to clean it

17:23 currentoor: Has any one here used Datascript on the server?

17:25 DomKM: currentoor: Try asking here https://gitter.im/tonsky/datascript

17:26 currentoor: Thanks!

17:40 Reefersleep: I think I cleaned it properly just with lein clean

17:40 But now I'm having trouble with the :Piggieback invocation

17:40 I don't understand the doc - what are you supposed to write after :Piggieback ?

17:46 lodin_: I noticed that with-meta forces evaluation of the head of a LazySeq.

17:46 ,(let [c (atom 0) _ (with-meta (map (fn [_] (swap! c inc)) (range 100)) {})] @c)

17:46 clojurebot: 32

17:47 lodin_: Is that strictly necessary?

17:48 gfredericks: I think so

17:49 I mean not strictly no

17:49 but I think the alternative would add a bunch of checks somewhere

17:49 justin_smith: lodin_: what if (range) were empty? you can't put meta on nil

17:49 gfredericks: ,(range 0)

17:49 clojurebot: ()

17:49 gfredericks: justin_smith: I don't think that's an issue

17:49 justin_smith: oh, right

17:49 gfredericks: I'm trying to remember what the real issue is

17:49 oh right

17:50 lodin_: I looked at the source, and seq() is called, but at least superficially it seems like you could create a new LazySeq that has meta != null.

17:50 gfredericks: it's that forcing a lazy seq is a coordinated effort

17:50 you don't want to have two LazySeq objects holding the same thunk

17:50 because then they would have to coordinate about who gets to force it

17:51 so the behavior you're seeing is in service of keeping one LazySeq per thunk-fn

17:51 lodin_: gfredericks: Ah. I think I understand what you mean.

17:52 gfredericks: which is probably good otherwise every single dang step of realizing a lazy seq would involve making sure nobody else is in the middle of realizing it

17:53 you might be able to come up with a high-perf solution to the problem but it would probably be more complex than the current impl

17:56 Reefersleep: I tried to follow the guide here: http://ofnir.net/posts/make-the-figwheel-repl-work-in-vim-fireplace.html

17:57 But then, when I do the :Piggieback part, I get

17:57 CompilerException java.lang.ClassNotFoundException: figwheel-sidecar.repl-api, compiling:(/private/ var/folders/h0/7srrpf416l57gzg3hx9w879c0000gn/T/form-init4543993028774803874.clj:1:83)

17:57 :/? Anyone got any ideas?

17:58 But then, when I do the :Piggieback part, I get

17:58 oh sorry :)

17:59 lodin_: gfredericks: Btw, have you done any more work on stateless random numbers?

18:03 gfredericks: I integrated the thing into test.check; now I'm mostly just thinking about other test.check stuff

18:03 I think the randomness part works as well as it's likely going to

18:04 I guess it could get moved to a standalone library

18:04 which would entail better API design

18:10 lodin_: gfredericks: I saw that you made four a couple of years ago, but no plans on updating it then I guess?

18:11 I wrote my own lib that is very similar to four a while back, before I found four, because I really don't like "uncontrolled" randomness. :-)

18:13 gfredericks: hmm

18:14 four was definitely trying to include the immutable RNG thing, but also functions mirroring clojure's random functions but with more control

18:14 nooga: Anyone used ztellman/gloss? I'm having small problems with it and the doc/src doesn't help :C

18:16 gfredericks: lodin_: there's a jira ticket around the latter problem, and I'm imagining maybe a contrib library for the immutable stuff

18:44 nooga: amalloy_: got a minute?

21:19 TEttinger: gfredericks: I'm curious about a better random lib for clojure too

21:21 gfredericks: TEttinger: mutable or immutable?

21:22 TEttinger: hm.

21:22 with immutable, you would need to pass state around, right?

21:23 gfredericks: generally yeah

21:23 TEttinger: I can see that being good too...

21:23 gfredericks: doesn't have to be a full state monad though if your generator is splittable

21:24 and I'd say immutability isn't worth much otherwise

21:24 TEttinger: I've written or copied several RNGs in a java game lib I intend to use eventually

21:25 (like an RNG that will start generating higher results if it "rolls low" repeatedly)

21:26 gfredericks: oh so funny distributions

21:26 TEttinger: yes

21:26 they only get funny once a threshold is reached

21:34 gfredericks: a bit like a 1-bit-generator than never returns the same result more than seven times in a row

21:36 I guess it's more than a funny distribution, since the successive results are independent

22:05 tomjack: really?

22:08 I imagine taking (repeatedly #(< (rand) 0.5)) and going left-to-right, not'ing an element when all the previous k generated elements match it

22:09 then (nth xs i) and (nth xs (inc i)) are independent?

22:09 TEttinger: you would need to realize it to find what was before, right?

22:09 or retain head or whatever the term is?

22:12 tomjack: just trying to see if I understand the claim "the successive results are independent"

22:38 gfredericks: tomjack: oh maybe I typo'd that and meant "aren't"

22:39 sorry for the confusion :(

22:39 s/maybe/definitely/

23:16 l34ch: anyone working on any interesting open source projects?

Logging service provided by n01se.net