#clojure log - Jun 30 2017

The Joy of Clojure
Main Clojure site
Google Group
List of all logged dates

0:05 TEttinger: random_numbers: SecureRandom does that

0:05 random_numbers: TEttinger: I see.

0:06 TEttinger: SecureRandom is, as it probably should be, massively slower than j.u.Random

0:10 kenrestivo: /dev/random vs /dev/urandom

0:11 random_numbers: Makes sense.

0:19 TEttinger: there are a ton of advantages to SplittableRandom's algo though.

0:19 you get leap-ahead/leap-back in constant time

0:20 SplittableRandom may not expose it? but anything it does can be done by you

0:22 if you normally called the version I gave like splitMix64(state += 1337) , then you could jump forward 23 steps and get the result from there by doing splitMix64(state += 1337 * 23)

0:22 you could jump back 23 steps by doing splitMix64(state += 1337 * -23)

0:25 what makes a generator cryptographically secure anyway, random_numbers?

0:26 I think the first criteria needs to be inability to predict future outcomes from previous ones

0:26 I think it might be good if the output is indistinguishable from true randomness with a low margin of tolerance

0:26 random_numbers: TEttinger: Hm, can't recall off-hand. Just recalled that reusing seeds can be unadvisable.

0:27 TEttinger: depends

0:27 for testing you need to reuse seeds

0:27 either you're testing possibly every input, or you need to reproduce failure cases

0:54 xrash: I have a vector of maps in the following format: [{x:2, ...}, {x:2, ...}, {x:3, ...}, ...] and I want to traverse this vector creating another vector in the format: [{x:2, items: [every item with x = 2 from the original vector]}, {x:3, items: [every item with x = 3 from the original vector]}]

0:54 I am struggling to get this working in a functional way.

0:55 I cannot not think about this as a for loop collecting stuff. :(

0:55 random_numbers: TEttinger: I see.

0:57 jeaye: xrash: reduce can certainly do this.

0:57 TEttinger: or for I think

0:57 xrash: Ill take a closer look on those.

0:57 TEttinger: where you say every item, do you mean the whole map {x:2, ...} or just the ... there ?

0:58 reduce is definitely powerful but it can be tricky to get it to work cleanly, and there's often a different way

0:58 xrash: I think this will make no difference.

0:58 TEttinger: still not sure what an item is

0:59 ohhh

0:59 xrash: Item is the whole map, but if every map from the result vector will have the value of x, it will make no difference to me whether the resulting item will also contain nested value of x or not.

0:59 TEttinger: merge-with could be handy

1:00 ,(doc merge-with) ;; checking for myself

1:00 clojurebot: "([f & maps]); Returns a map that consists of the rest of the maps conj-ed onto the first. If a key occurs in more than one map, the mapping(s) from the latter (left-to-right) will be combined with the mapping in the result by calling (f val-in-result val-in-latter)."

1:01 xrash: This seems to have some relation with my case, however I will also explore reduce and for anyway as an exercise.

1:01 TEttinger: for may be useful in similar cases

1:02 I think merging is the general type of thing we're looking at

1:03 xrash: However this merge-with function seems to receive several maps as different parameters, and I have a vector. Will it work with vectors?

1:04 And I want a vector as a result, not a single map.

1:06 jeaye: xrash: https://gist.github.com/jeaye/829d0944c39de52e17214da25f08e307

1:06 You want something like that?

1:07 Oh, updated to show what grouped is. You can refresh.

1:08 xrash: At first glance it seems exactly like it, but I will have to read a little bit to understand.

1:08 Thanks!

1:08 jeaye: You _could_ do this with one reduce, but I wouldn't without better reason.

1:09 xrash: I recommend taking it out of the gist and formatting it in your editor, to help understand what's going on. Feel free to ping me with questions about it.

1:14 TEttinger: ,(mapv (fn [[k v]] {:x k :items v}) (group-by :x [{:x 2, :item "hooray"}, {:x 2, :item "yay"}, {:x 3, :item "yippee"}]))

1:14 clojurebot: [{:x 2, :items [{:x 2, :item "hooray"} {:x 2, :item "yay"}]} {:x 3, :items [{:x 3, :item "yippee"}]}]

1:14 TEttinger: hooray

1:14 I was off a bit on merge, i should have thought of group

1:15 I was looking for a not-reduce solution because heavy uses of reduce tend to be harder to read

1:15 jeaye: Wait, is that what he wanted?

1:15 TEttinger: (I'm not sure)

1:15 jeaye: We have quite different outputs.

1:15 TEttinger: they're similar enough that they could be changed to fit

1:16 jeaye: :)

1:16 TEttinger: I think the specific request was that it be a vector of maps

1:16 jeaye: You may be more on point then.

1:16 TEttinger: I think a map where you could look up 2 to get all items where x was 2 would be nice

1:17 jeaye: I wonder if specter has something for this.

1:17 TEttinger: :D

1:21 xrash: TEttinger: thx, I will take a look on that too!

1:23 huawei: hi

1:25 i have a newbie question

1:40 dysfun: well don't stick around long enough to ask it, will you?

1:41 xrash: jeaye: I see that you create the "grouped" map before actually constructing the final vector. Is the order of the "grouped" map keys guaranteed? I can test it and it seems to work, however can I trust it?

1:42 dysfun: group-by will come out in order provided the collection was ordered

1:43 jeaye: xrash: I'm using (fnil conj []) so the order is maintained.

1:43 conj onto a vector pushes onto the end.

1:44 xrash: Naturally, you'd probably want a (let []) and not the (def grouped ...) that I had in the repl.

1:44 xrash: I'm still understanding that fnil thing, but I mean after the grouped map is created, when you apply the last `map` on it, is it guaranteed that it will traverse the "grouped" map in the order you have constructed it?

1:45 Because maps may not maintian the key order.

1:45 Depending on implementation, I suppose.

1:45 Yeah, I am already familiar with let.

1:45 justin_smith: yeah, in practice the shuffle happens after the 16 item threshold - usually

1:46 jeaye: xrash: Oh, I see what you mean. You can use a sorted map or somethin'. At that point, I think the group-by approach may be nicer.

1:46 xrash: As for fnil, it takes a fn and some values and returns a fn. When you call that fn, any arguments which are nil are replaced with the values you gave up front.

1:47 ,(pr-str nil 5)

1:47 clojurebot: "nil 5"

1:48 jeaye: ,((fnil pr-str 33) nil 5)

1:48 clojurebot: "33 5"

1:48 jeaye: The order/place is kept though.

1:48 xrash: Yeah, good explanation. (By the way, the description of reduce in clojuredocs.org is a beast :P)

1:48 jeaye: ,((fnil pr-str 33 44 66) nil 5 nil)

1:48 clojurebot: "33 5 66"

1:48 jeaye: 44 doesn't go in, since 5 is there in the second place.

1:49 justin_smith: ,((fnil / 1 1) 2 nil) ; I didn't realize I could do this...

1:49 clojurebot: 2

1:49 justin_smith: very cool! thanks jeaye

1:49 I had only ever used fnil with one arg

1:50 jeaye: :)

1:53 ,((apply fnil pr-str (repeat 3 :unbound)) nil 5 nil)

1:53 clojurebot: ":unbound 5 :unbound"

1:54 jeaye: It's limited to 3 though, which is a bummer. Otherwise, you could use a lazy repeat and just have any nil arg be substituted.

2:06 xrash: jeaye: I think I understand your whole solution now, really kool, thanks!

2:07 ,(mapv (fn [[k v]] {:x k :items v}) (group-by :x [{:x 2, :item "hooray"}, {:x 2, :item "yay"}, {:x 3, :item "yippee"}]))

2:07 clojurebot: [{:x 2, :items [{:x 2, :item "hooray"} {:x 2, :item "yay"}]} {:x 3, :items [{:x 3, :item "yippee"}]}]

2:07 xrash: Now this one seems to be a bit higher level, let me check it. :P

2:11 Well, group-by seems to magically do it all.

2:14 TEttinger: yay

2:15 bdevel_: (print "hello world")

2:16 jeaye: (no)

2:20 TEttinger: ,(print "needs a comma at the start")

2:20 clojurebot: needs a comma at the start

2:20 justin_smith: TEttinger: is there a clojurebot quine?

2:20 TEttinger: I think it ignores itself

2:21 justin_smith: but is there an input that returns itself, complete with comma?

2:22 jeaye: No, since the comma is just whitespace.

2:22 justin_smith: but you can make the bot emit a comma surely - I know this must be possible, just speculated someone had one handy

2:24 TEttinger: ,(print ",{}")

2:24 clojurebot: ,{}

2:25 TEttinger: ,",()"

2:25 clojurebot: ",()"

2:26 TEttinger: drat this will be a time waster

2:26 jeaye: haha

2:26 justin_smith: haha, sniped

2:40 ridcully_: ,(symbol ",(symbol \",\")")

2:40 clojurebot: ,(symbol ",")

2:41 ridcully_: hmm

3:26 osfameron: TEttinger: ah, interesting. Don't think I care about leap-ahead/back (didn't know the concept even ;-) but will bear it in mind

3:26 TEttinger: heh, osfameron I just made use of it now that you brough it to mind

3:27 dysfun: this sounds interesting, but i can't find the original exchange

3:27 TEttinger: I use it in a simple, not-professionally-secure method to obscure things like save files for games on phones

3:28 one thing I do is to shuffle the input string and apply some XOR to each character, up to the first 5 bits for reasons. this is totally reversible with jump-back or leap-back or whatever you can call it

3:29 the length doesn't change, so the number of random bits produced is the same encoding and decoding

3:29 osfameron: dysfun: started with 01:44 <osfameron> can you generate repeatable random numbers?

3:29 I'm UTC+1

3:29 dysfun: ah

3:29 TEttinger: and it turns out there's some simple methods that pass randomness tests and perform well on the JVM

3:30 SplittableRandom is 3 lines, plus an update that you can control yourself

3:30 dysfun: well any DRBG is repeatable given the same seed

3:30 osfameron: I'm actually not doing anything especially interesting with random numbers (was just thinking about implementing Candy Crush in clojure for lols)

3:30 TEttinger: except clojure doesn't allow seeds in rand-int and such

3:30 osfameron: and was disappointed that clojure rand doesn't have seeds, so would be impossible to do & test in FP stylee

3:31 TEttinger: it is kinda a strange omission, maybe related to using ThreadLocalRandom

3:31 which is also non-seed-able

3:31 dysfun: still, you can use java's other random stuff

3:32 osfameron: yeah, that's fine and not too hard to wrap, but definitely an odd omission

3:32 TEttinger: indeed. java.util.Random, whatever SecureRandom is in, java.util.SplittableRandm on newer Java

3:32 Rovanion: The implementation of rand is simple, just copy it and add one argument: https://stackoverflow.com/questions/17445813/random-seed-math-random-in-java

3:33 And replace Math.random with an instance of java.util.Random.

3:34 TEttinger: oh you mean to replace clojure.core/rand ?

3:34 Rovanion: Make a new function rand'

3:34 Or call it random.

3:35 TEttinger: java.util.Random has some helpful tweaks above its LCG implementation type, but i don't know how the quality really is. it just discards a bunch of repetitive bits from a 64-bit random value

3:36 one bit it really is good that it discards it since I think it alternates 1, 0, 1, 0

3:38 Rovanion: Perhaps I'm not understanding the issue correctly. But I thought we just wanted repeatable randomness for a game java.util.Random seems to be capable of it.

3:39 Eh, missed an 'and' there between game and java.util.Random.

3:42 TEttinger: Random is capable of it yeah but I wouldn't rely on it just by default. it's very slow for what it does

3:43 SplittableRandom I consider an improvement all around

3:43 * dysfun is just using UUID/randomUUID, has no idea what that uses under the hood

3:43 osfameron: I'll worry about that when I need the bare metal raw speed required for Candy Crush ;-)

3:44 dysfun: it only needs bare metal raw speed because it's hard work animating all those GIVE US YOUR MONEY screens

3:44 * osfameron wraps the function with a (Thread/sleep 100) just to prove that he doesn't care about performance

3:44 osfameron: dysfun: yeah, good point

3:44 TEttinger: but the main thing you may want is the ability to get the current random state, for save files

3:45 osfameron: they've got worse in the last 2 years (last time I got my android tablet out with CC installed it was much much less laggy)

3:45 TEttinger: Random I don't think allows that

3:45 osfameron: (defn randfn ([] (randfn (java.util.Random.))) ([r] #(.nextDouble r)))

3:45 is what I saw suggested

3:45 * dysfun hasn't played CC in years

3:45 osfameron: you pass it a seed and it generates a rand function

3:46 which is mutating, but at least predictable

3:46 TEttinger: that's not what that does?

3:46 osfameron: might look at SplittableRandom though

3:46 TEttinger: I guess you could consider a Random object a seed

3:46 it may be serializable in some way

3:47 osfameron: yeah, you call e.g.: (def source2 (randfn (java.util.Random. 37)))

3:47 TEttinger: oh ok

3:47 osfameron: oh I see, yeah, it's a Random object which is the seed

3:47 TEttinger: that's reasonable as long as there's a way to save the object seeds for later

3:48 osfameron: well I only really need the seeds for testing

3:48 TEttinger: cool

3:48 then probably fine :D

4:08 Rovanion: Does anyone know why the spec macros like assert don't keep track of predicate names? I made a stupid little modification of assert to demonstrate: https://gist.github.com/Rovanion/10b5af393498500472c84cab53b1ff02

4:09 dysfun: goddamnit cognitect take this man's patch

4:10 any other massive improvements you feel like making?

4:15 Rovanion: Pff, I don't think the word massive is necessarily appropriate here. Just a tiny little change which may be a bad idea because of X.

4:15 dysfun: well i have spent rather a long time in the last week building stuff against the spec api and it's exhausting

4:15 so to me, anything that makes spec explain itself better is a massive improvement

4:59 TEttinger: can't spell cognitect without tic con get

5:01 dysfun: git tec con

6:32 warrshrike: hey

6:33 dysfun: hello

6:33 warrshrike: someone suggested a good windows jdk thing once

6:33 i forgot the name

6:33 dysfun: zulu?

6:33 warrshrike: dysfun: hey man how you been

6:33 dysfun: yup thats it

6:33 dysfun: yeah not bad and you?

6:34 warrshrike: dysfun: same old same old. lifes better now that i dont have to fight js-cljs interop constantly ~~ working on jdk clojure now

6:35 dysfun: yeah me too. turns out the jvm is much more fun than js

6:35 lxsameer: is zulu e jdk ?

6:35 s/e/a/

6:36 dysfun: yes

6:36 warrshrike: dysfun: Amen to that brother

6:36 dysfun: i'm also doing kotlin

6:36 but at the minute most of the libraries for kotlin are java ones

6:37 so i'm using clojure and kotlin together

6:37 warrshrike: Everything works much more 'organically' in jdk

6:37 yeah kotlins cool

6:38 but any particular reason for using it with clj?

6:38 couldnt you do clojure only?

6:38 dysfun: i could, but i'm not doing

6:38 shocking i know, but there are places where a type system is useful

6:39 lxsameer: dysfun: what's the difference between zulu and openjdk ?

6:39 js sucks by the way

6:40 dysfun: zulu is polished up and engineered to pass the JCK. those patches will eventually go back into openjdk, but at any given point in time, openjdk isn't guaranteed to pass the conformance test suite

6:40 so essentially it's the less mystery meat version of openjdk

6:40 lxsameer: dysfun: ow, good to know

6:45 warrshrike: dysfun: yeah makes sense. static typing improves tooling immeasurebly

6:46 kotlin comes off as a more portable and platform independent Swift

6:46 dysfun: yeah, there are swift influences

6:46 also c# influences. c# is quite a better language than it used to be

6:46 osfabibisi: c# looks mildly interesting

6:47 I keep meaning to play with linq, but not enough to suffer working on windows

6:47 warrshrike: dysfun: haven't ever used c sharp...but i always thought it was java-ey

6:47 java itself has improved much btw

6:47 osfabibisi: dysfun: so you're doing the core components in kotlin and stitching them together in clojure ?

6:48 dysfun: something like that, yes

6:49 all the business logic is written in clojure and the high performance parts are written in kotlin mostly, but that's because i haven't gotten as far as the problems where types really help out

6:50 oh also the bits where clojure gets really fugly are written in kotlin

6:50 some things are just easier to express in a java-like language

7:07 Hanonim: dysfun: how has been your experience with zulu ? you've been using it for a while i believe

7:08 dysfun: it works as described, what can i say?

7:09 Hanonim: nothing weird to declare

7:09 dysfun: oh weirdnesses of course, but nothing i can pin down to being its fault

7:10 Hanonim: relative to oracle ?

7:10 :q

7:10 dysfun: the purpose of the JCK is prove it works

7:10 both have been subjected to it

7:10 there should be no difference from a user perspective

7:19 ridcully_: i have recently replaced an oracle jdk with zulu in production -- nothing odd there

7:20 Hanonim: they make money only out of consultancy ?

7:21 ridcully_: dont they also have their own jdk (or fork) for some border cases?

7:26 dysfun: they make money primarily off consultancy and their zing jvm which will basically get you low pause time garbage collection for free and a bunch of other things

7:26 they also produce (produced?) some coprocessors for java garbage collection

7:26 Hanonim: wow, i've never even heard about such coprocessors

7:27 dysfun: oh and they've been doing an IOT java build as well

7:27 Hanonim: yes i've been experimenting with it on the rpi but not enough to have a firm opinion

7:28 but it looked like it was behaving almost as fast as the oracle jvm optimized for arm, that looked promising

7:29 dysfun: oohj

7:29 Hanonim: anyway i'll use it in production quite soon

7:30 dysfun: is this the free one or the paid one?

7:30 Hanonim: free

7:30 dysfun: cool

7:30 Hanonim: openjdk isn't optimized at all for arm, it is about 10x slower than oracle

7:30 and oracle... well, you know...

7:31 dysfun: https://www.azul.com/products/zing/falcon-jit-compiler/ # fascinating. i followed the open source project do this year for a while

7:31 s/year/

7:33 Hanonim: is it mostly for perf ? or does it add something new ?

7:33 dysfun: just perf

7:34 but add it to pauseless GC and eliminating the warmup phase when restarting and it starts to look quite compelling

7:35 Hanonim: zing does look compelling, yes

7:36 dysfun: i don't fanboi for proprietary software, but if it were open source...

7:38 Hanonim: i wonder how much better is their GC in real life

7:41 dysfun: the first step is identifying that you have a GC problem, otherwise it's pointless

8:10 warrshrike: dysfun: what gets really fugly in clojure?

8:15 dysfun: anything involving abstract things for a start

8:18 Hanonim: that's broad

9:20 is there an obvious way to add metadata to the symbol rather than the value (def sym x) ?

9:20 besides alter-meta

9:21 dorian: hey random question: is there any way to start a cider repl without being in the context of a project?

9:27 ragepanda: dorian: cider-jack-in doesn't work?

9:27 dorian: pretty sure it complains about not being situated in a project

9:27 one sec, checking

9:28 oh, weird, there it is

9:28 ragepanda: it worked?

9:28 dorian: yeah, but in my head i tried that before

9:28 and it was like wah wah where's the lein project

9:29 ragepanda: that's weird. are you sure you're not in a project right now?

9:29 dorian: yep

9:30 maybe i'm misremembering something or had triggered some weird behaviour before and it just stuck in my memory

9:30 ragepanda: https://cider.readthedocs.io/en/latest/up_and_running/#launch-an-nrepl-server-and-client-from-emacs

9:30 doesn't mention it running without a project, let me try it on my machine

9:31 ah, it prompts you. "Are you sure you want to run a Clojure REPL without a project?"

9:31 dorian: ha okay that's funny because it was totally silent when i just tried it

9:32 weeeeird

9:32 0.13.0snapshot

9:32 assuming that's not the newest

9:34 ragepanda: 0.15.0snapshot for me

9:37 Rovanion: I like the ragepanda moniker.

9:37 ragepandemic: rage against the pandas

11:57 osfabibisi: we quite often have threading macros with this sort of thing in them: (#(assoc % :key (message-key (:value %))))

11:57 is there a nicer way to do this?

11:58 e.g. we want to assoc a key to the threaded hash-map, based on a value in the entire hash-map

11:58 e.g. it's not the same as an update, which assoc's a key based on the value of *that* key

11:58 justin_smith: osfabibisi: there's as->

11:58 osfabibisi: ah yes, lemme have a look at how to use that

11:59 justin_smith: (as-> some-hashmap m (assoc m :key (:value m)))

11:59 it pays of more with more clauses, but even that is nicer than the #() version imho

12:00 though as-> is just a shorthand for a let block that always uses the same binding symbol in every clause

12:00 (it's helpful to read the macro source)

12:00 osfabibisi: yeah, it's not all that nice really

12:00 you could think of it as: (as-> % (assoc % :foo (:bar %))) ;-)

12:01 but I guess there's not much you can do about that

12:01 justin_smith: except we get to use a more meaningful symbol, and multiple clauses

12:02 and you were mentioning threading - you can put as-> inside ->, it's designed for that

12:02 (-> foo (frob) (as-> the-foo (assoc the-foo :x (:y the-foo))))

12:03 osfabibisi: yes indeed

12:04 I wonder if I want to write a macro that would let me write that as: (-> m (assoc-by :foo :bar)) though

12:04 justin_smith: it's close to being clojure.set/rename-keys...

12:05 though that doesn't match your original example

12:05 osfabibisi: sorry, the :bar there is a function

12:06 rather than a key name. sorry, I'm mixing my examples

12:07 justin_smith: I'm also fond of the plain-old procedural let block

12:07 osfabibisi: sure

12:07 in fact the answer might just be: (-> m (some-func-that-encapsulates-that-awkward-thing-I-was-whining-about))

12:08 justin_smith: this may even be one of the rare uses of letfn... ?

17:39 {blake}: How does cljsbuild determine what to compile? I've had this project for over a year, and now it's ignoring everything but my "main" cljs.

17:45 It detects when I make a change to a non-main file, and that triggers the recompile but never seems to compile it.

Logging service provided by n01se.net