#clojure log - Sep 30 2014

The Joy of Clojure
Main Clojure site
Google Group
List of all logged dates

19:08 ambrosebs: aconbere: it looks like you triggered a bunch of namespace checks that weren't there before

19:08 aconbere: ambrosebs: yeah... but I'm not sure why :'(

19:08 dbasch: marchdown: of course in reality you’d just split it and use ‘frequencies'

19:08 marchdown: okay.

19:09 I’d like to reimplement it just because I’m stuck and that exposes a lapse in my understanding.

19:09 So far I’ve got (defn account-for-a-word [w d] ; word -> dict -> dict

19:09 (assoc d w + (#(get d % 0) w) 1)))

19:09 (Should I use pastebin or are two-line pastes okay here?)

19:10 gfredericks: technomancy: the web page disappears after 0.75 seconds for some reason; but I can read it painstakingly by repeated refreshes

19:10 justin_smith: marchdown: that's not how assoc works

19:10 dbasch: two are okay, three pumps are excessive celebration

19:10 aconbere: just starting with the first error I'm off to a bad start :)

19:10 ambrosebs: Type Error (yesql/generate.clj:33:1) Must pass HeterogeneousSeq to clojure.lang.PersistentHashMap/create given (clojure.core.typed/Option (clojure.core.typed/NonEmptyASeq clojure.core.typed/Any))

19:10 https://github.com/aconbere/yesql/blob/reorder-imports-for-health-and-wellness/src/yesql/generate.clj#L32

19:10 justin_smith: marchdown: unless you meant to use the function + as a key in your map

19:10 aconbere: that's that function

19:11 marchdown: umm, how do I change hashmap entries then?

19:11 justin_smith: update-in is good for that

19:11 joshuafcole: assoc is the right tool for adding values to keys, but it just takes direct values, e.g.

19:11 marchdown: but isn’t breaking the pretense of immutability?

19:11 TEttinger: ,(assoc {:a "a"} :whee "yay")

19:11 clojurebot: {:whee "yay", :a "a"}

19:12 joshuafcole: sniped

19:12 amalloy: marchdown: none of these suggested methods actually change any maps anywhere

19:12 justin_smith: marchdown: not at all, it returns a new map

19:12 joshuafcole: they just return new maps based off the input map

19:12 amalloy: (update-in m [k] f) is just (assoc m k (f (get m k)))

19:12 justin_smith: ,(assoc {} :a 0 :b 1 :c 2)

19:12 clojurebot: {:c 2, :b 1, :a 0}

19:12 Bronsa: ,(let [a {:a 1}] [(assoc a :b 2) a]) ;; marchdown

19:12 clojurebot: [{:b 2, :a 1} {:a 1}]

19:12 TEttinger: not strictly new, it does share structure for efficiency reasons with the initial hash-map

19:12 gfredericks: amalloy: technomancy: there now the world is a better place CLJ-1542

19:12 ambrosebs: aconbere: hmm I'd expect a line number 32:1

19:13 aconbere: ambrosebs: that's just because I have an extra line in my current env :-/

19:13 sorry about the confusion

19:13 if you look at the travis ci error the line numbers line up

19:13 :-/

19:13 amalloy: marchdown: also, in order to be convenient to use from reduce, you'd rather have the signature be dict -> word -> dict, rather than word -> dict -> dict

19:13 ambrosebs: aconbere: ok, do you want to fix the errors or suppress them like before?

19:14 marchdown: Oh, okay, thanks. I’ll go fiddle around with that.

19:14 aconbere: ambrosebs: I AM TORN!

19:14 ambrosebs: it seems most useful to start by suppressing them

19:14 and them giving the authors a chance to add the checks as they see fit

19:14 ambrosebs: add {:core.typed {:collect-only true}} in problematic namespaces

19:14 as ns metadata

19:15 (ns yesql.asdf {:core.typed ...} (:require ...))

19:15 or just wrap everything in a tc-ignore

19:15 it expands to a `do`

19:16 Bronsa: ambrosebs: has CLJ-130 ever affected core.typed?

19:16 ambrosebs: Bronsa: I'm sure it will at some point

19:16 Bronsa: right now I just reread the ns form if I need it

19:17 aconbere: ambrosebs: gotcha! well that quiets the errors, now I guess I can at least work through them one at a time with the authors

19:17 thanks :)

19:18 ambrosebs: so in general is the process with core stuff to apply your own annotations to them?

19:18 ambrosebs: aconbere: if core.typed is missing them yea

19:19 aconbere: k

19:19 are the annotations namespaced?

19:19 ambrosebs: if core.typed adds them, you'll get a duplicate annotation error

19:19 no

19:19 aconbere: k

19:19 okay... I might be able to make some progress on this ...

19:19 ambrosebs: no-check annotations probably should be namespaced

19:20 but not today

19:31 gfredericks: amalloy: I wonder if part of the reason rich was reluctant to talk about the return value is because of the bizarre codepath involved in delivering a promise

19:31 amalloy: i.e., it's actually a detail of the promise function, not the deliver function

19:32 amalloy: gfredericks: those two are symbiotic enough that i think documenting them "as a whole" makes sense. you can't just say that all behavior of deliver is undefined because you don't want to couple yourself to the behavior of promise

19:32 hiredman: also promises now contain an atom and a countdown latch

19:33 oh, I guess they always must have had both

19:33 gfredericks: and the atom initially holds the countdown latch as a sentinel; funny.

19:38 amalloy: that's a fun property of a few clojure.core functions: they just use whatever objects happen to be floating around as sentinels. see seque, for example

19:39 arrrms: I'm wondering what the functional, idiomatic way to do something is... I've got a vector of maps. Each map has name and status properties. I want to search the vector for a map with a particular key, change its status, then save it back into the vector in its original position. Do I need to search the vector, find the index of the map I want to update, split the vector apart, operate on the map in question, and then

19:39 reassemble the entire vector?

19:40 dbasch: arrrms: why does it need to be a vector?

19:41 arrrms: I assume a vector is the best container for a group of maps. Should I be using a list? Something else entirely?

19:41 justin_smith: arrrms: you can assoc onto a vector by numeric index

19:41 dbasch: arrrms: there is no best, it depends on what you want to do with it

19:42 amalloy: arrrms: the "split apart" and "reassemble" phases sound confused: you probably just want to use update-in instead. but also, as dbasch is no doubt going to finish saying, you don't want a giant vector to hold things so you can do linear scans over them: much better if you can have a map indexed by whatever you plan to look up by

19:42 justin_smith: but why search rather than have a map keyed by the property to look up>

19:42 arrrms: I guess a list would make more sense, since I don't necessarily need random access to it. I'd always be looping through it

19:42 justin_smith: arrrms: are there multiple search functions used?

19:42 dbasch: what amalloy said I was going to say :P

19:42 amalloy: a list is suuuuuuper bad for this. a vector could be right, depending on your access patterns

19:43 arrrms: justin_smith: sorry, I don't know what you mean. I'm *very* new to Clojure

19:43 justin_smith: arrrms: when you search, do you always look for the item by name?

19:43 arrrms: oh, yes

19:43 dbasch: arrrms: he means that you can update the vector “in place”

19:44 justin_smith: then just group-by :name

19:44 (doc group-by)

19:44 clojurebot: "([f coll]); Returns a map of the elements of coll keyed by the result of f on each element. The value at each key will be a vector of the corresponding elements, in the order they appeared in coll."

19:44 justin_smith: then you can just look up by name

19:44 no need to search at all

19:44 then use update-in

19:44 (doc update-in)

19:44 clojurebot: "([m [k & ks] f & args]); 'Updates' a value in a nested associative structure, where ks is a sequence of keys and f is a function that will take the old value and any supplied args and return the new value, and returns a new nested structure. If any levels do not exist, hash-maps will be created."

19:44 arrrms: hmm... so no need for maps at all?

19:44 justin_smith: map as in the data structure :)

19:44 amalloy: justin_smith: i think you're going too fast here, suggesting solutions before he understands the problem

19:44 justin_smith: OK

19:44 arrrms: oh, yes, I meant the data struct and not the func :)

19:45 justin_smith: group-by returns a map

19:46 amalloy: arrrms: the idea is, instead of having a giant list of maps, which you scan over one item at a time, you can have a map of maps

19:46 indexed by name, or whatever else is your primary identifier

19:46 then when you need to find a map with a particular name, you just look it up in that index, without having to scan over any other maps

19:47 dbasch: arrrms: you can have a "directory structure" of maps within maps if it fits your problem

19:47 amalloy: justin_smith is suggesting that group-by would be a good function to help turn your current data structure into the proposed new one

19:47 arrrms: amalloy: ah, the map of maps makes a lot more sense

19:48 amalloy dbach justin_smith: thanks! you've given me a good idea of how to proceed (and some new functions to read up on)

19:48 amalloy: this suggestion only "works" if you have one primary characteristic which you always look things up by, such as :name

19:48 arrrms: dbasch*

19:49 yeah, that should do the trick. This is a basic to-do app, so I'll be looking tasks up by their name

19:49 amalloy: great. well, have fun!

19:50 marchdown: How do I match whitespace including commas? I would expect #",\s+" to work, but it doesn’t.

19:50 amalloy: #"[,\s]+"

19:50 that's a regex question, of course, not a clojure one at all

19:50 hiredman: arrrms: if you are building an in memory database checkout clojure.set

19:50 marchdown: Oh wait, or does it. I might have fudged something.

19:51 Thanks.

19:51 arrrms: hiredman: for now I plan on writing/reading from a JSON file, but will definitely check out clojure.set

19:51 amalloy: marchdown: #",\s+" means "exactly one comma, followed by at least one whitespace character"

19:51 hiredman: https://clojure.github.io/clojure/clojure.set-api.html#clojure.set/index

19:51 amalloy: so it will coincidentally work in some cases

19:51 marchdown: amalloy yep, got it. thanks.

20:02 gfredericks: okay tonight is NREPL-53 or bust

20:03 hiredman: I have a hard time with that bug, I mean, isn't it just a topo sort? is that so difficult it needs so much back and forth? why was it not a topo sort to begin with?

20:04 gfredericks: my problem with it is I haven't yet understood the code well enough to see what it's doing wrong; but yeah that's my intuition is well

20:04 so I haven't decided whether to push on trying to fix the existing code or rewrite it under the risk that it was complicated for some important reason I missed

20:10 hiredman: rewrite

20:11 gfredericks: clojurebot: what |can| possibly go wrong

20:11 clojurebot: Roger.

20:11 technomancy: clojurebot: chesterton's fence?

20:11 clojurebot: No entiendo

20:11 technomancy: huh, I thought I had added that

20:12 maybe I'm thinking of #emacs

20:12 hiredman: honestly, I still don't see the benefit of including a weird depedency mechanism in nrepl

20:12 technomancy: clojurebot: chesterton's fence is <reply>The more modern type of reformer goes gaily up to it and says, “I don’t see the use of this; let us clear it away.” To which the more intelligent type of reformer will do well to answer: “If you don’t see the use of it, I certainly won’t let you clear it away. Go away and think. Then, when you can come back and tell me that you do see the use of it, I may allow you t

20:12 clojurebot: Alles klar

20:13 gfredericks: (inc technomancy) ;; except for his truncation

20:13 lazybot: ⇒ 136

20:13 hiredman: at best it should be middleware

20:13 technomancy: aw shucks; where'd it get cut off?

20:13 gfredericks: ~chesterton's fence

20:13 clojurebot: The more modern type of reformer goes gaily up to it and says, “I don’t see the use of this; let us clear it away.” To which the more intelligent type of reformer will do well to answer: “If you don’t see the use of it, I certainly won’t let you clear it away. Go away and think. Then, when you can come back and tell me that you do see the use of it, I may allow you t

20:13 technomancy: clojurebot: forget chesterton's fence

20:13 clojurebot: excusez-moi

20:14 gfredericks: there's no going back now

20:14 technomancy: dang it

20:14 ~chesterton's fence is The more modern type of reformer goes up to it and says, "I don’t see the use of this; let us clear it away." To which the more intelligent type of reformer will do well to answer: If you don’t see the use of it, I certainly won’t let you clear it away. Go away and think. Then, when you can come back and tell me that you do see the use of it, I may allow you to destroy it.

20:14 clojurebot: Roger.

20:15 gfredericks: you forgot <reply>

20:15 technomancy: gfredericks: yeah, trying to trim it

20:15 I guess I should have just linked out

20:15 hiredman: technomancy: let me tell you, there is lots of software with literally no use

20:15 technomancy: hiredman: but how do you tell it apart from the rest of it?

20:15 gfredericks: clojurebot: every software |has| a use

20:15 clojurebot: c'est bon!

20:16 hiredman: most software has a utility below zero

20:16 technomancy: well, a good first step is "does it run?"

20:16 second is "does it run correctly"

20:16 technomancy: hiredman: but but... my scheme interpreter

20:16 gfredericks: software was endowed by its creator with certain unalienable uses

20:16 technomancy: hiredman: "it's art"

20:16 hiredman: the code in question for nrepl-53 fails both those tests

20:17 gfredericks: no I think it at least runs

20:17 hiredman: gfredericks: sometimes it runs

20:17 dbasch: hiredman: did the author get paid for writing it? was it enjoyable?

20:17 gfredericks: when does it not run?

20:17 technomancy: ok, you caught me. I have no context and was only looking for an excuse to quote G.K. Chesterton.

20:17 as one does.

20:17 hiredman: gfredericks: ok, maybe I misread the ticket

20:17 "is it understandable"

20:17 (nope)

20:18 gfredericks: 1/3

20:18 hiredman: kill it with fire

20:18 gfredericks: sounds like run-of-the-mill legacy code by those three metrics :)

20:19 hiredman: and a lot of that could use a rewrite, let me tell you

20:19 dbasch: software with intentional random crashes and delay loops to make the developer/mantainer look good when needed

20:20 hiredman: I may still be sore over chas deciding to use bencoded data for the default nrepl transport over s-expressions

20:22 technomancy: hiredman: I don't think I will ever understand that.

20:22 dbasch: hiredman: bencoded as in bittorrent-bencoded?

20:22 hiredman: dbasch: more or less

20:23 gfredericks: so if, while you are mucking about in nrepl, and chas is sleep deprived, you happen to switch the default transport to something s-expression based, that would be super

20:24 gfredericks: I'll add a comment to the ticket I'm sure it'll be fine

20:27 hiredman: https://github.com/clojure/tools.namespace/blob/master/src/main/clojure/clojure/tools/namespace/dependency.clj even has a CA'ed topo sort

20:29 cfleming: technomancy: how would you feel about a patch that changes how lein uses memoize internally so that lein can be used embedded?

20:29 technomancy: Either allowing memoization to be disabled entirely, or memoizing in an external map that could be cleared.

20:30 justin_smith: cfleming: sounds like a good case for using core.cached

20:30 technomancy: cfleming: yeah, that would be fine

20:30 justin_smith: core.cache is actually a bit of a nightmare

20:30 cfleming: justin_smith: could well be, I'll take a look at it.

20:30 justin_smith: OH, OK

20:31 technomancy: cfleming: it's on my list, just haven't gotten around to the lein issue backlog recently

20:31 cfleming: technomancy: ok great, I'll take a look at it. Currently it's quite painful. I'll see if I can come up with a patch for that.

20:31 technomancy: justin_smith: I mean it sounds reasonable, but core.cache has problems with AOT and plugins: https://github.com/technomancy/leiningen/issues/1563

20:31 as far as I can tell it's because records are terrible?

20:32 justin_smith: oh, never mind that then

20:32 cfleming: technomancy: Any strong feelings about disabling entirely vs a clearable map? Or some other solution I haven't thought of?

20:32 technomancy: cfleming: if the memoization included the timestamp of the file that would do it, but I haven't looked at the code to see if that's reasonable

20:32 hiredman: technomancy: well, it is AOT is terrible

20:33 technomancy: hiredman: sure

20:33 hiredman: but 20s boot times are worse

20:33 hiredman: sure

20:33 technomancy: cfleming: if not, then I'd lean towards a *memoize?* var you could rebind or alter-var-root of

20:33 cfleming: technomancy: The issue there is that often what's memoized is calculated several fns deep and involves lots of files.

20:34 hiredman: the issue is lein and its deps are transitively compiled, but plugins are bringing in their own core.cache, which isn't, so now you have generated class files on disk fighting with dynamically generated class files

20:34 cfleming: technomancy: e.g. what bit me yesterday was .lein/profiles and profiles.d - all the profile calculation is memoized.

20:34 hiredman: AOT is terrible :/

20:35 technomancy: hiredman: works fine with things that aren't defrecord

20:35 * technomancy shrugs

20:35 hiredman: technomancy: sort of

20:35 gfredericks: hiredman: the tools.namespace tip is interesting; do you think nrepl might be too fundamental for dependencies, or did you mean to paste it?

20:35 hiredman: gfredericks: I have said before I don't understand why nrepl has a dependency system in it

20:35 gfredericks: hiredman: no I mean maven deps

20:36 TimMc: technomancy: That sure isn't the only reason.

20:36 hiredman: gfredericks: oh

20:36 gfredericks: e.g., nrepl requires tools.namespace and now nobody can use any tools.namespace that's too old to have the topsort

20:36 hiredman: gfredericks: I would just copy and paste, rewrite out all the protocol stuff

20:36 technomancy: cfleming: yeah, in that case probably just a flag to disable is the way to go

20:36 gfredericks: hiredman: gotcha

20:38 hiredman: technomancy: I if lein depended on library X, and lein was AOT compiled on date D, but a new version of library X was released on D-1 (sometime before D), and a plugin tried to include the new version of X, they would uncoditionally get the old version from the precomiled class files

20:38 precompiled

20:39 because the class files would be newer than the source files, and that is the heuristic clojure uses

20:40 technomancy: hiredman: yeah, but in lein you get that behaviour regardless of AOT because of how we use the bootclasspath

20:41 hiredman: technomancy: are plugins not also on the bootclasspath?

20:41 technomancy: no, plugins aren't known till runtime

20:42 turns out the JVM really, really doesn't want you to write CLI programs

20:42 hiredman: oh, of course, right you have to parse the plugin stuff from project and profiles and such

20:43 I also have a topo sort, without the protocol stuff, which is not warrantied for any purpose https://gist.github.com/hiredman/5623801

20:48 amalloy: hiredman: that definition of merge-sorted doesn't handle nil very well

20:48 Bronsa: I feel so good now that I know why CLJ-130 happens

20:49 I've been wanting to fix that for years

20:51 gfredericks: Bronsa: looks like it was fun :)

20:51 rplevy: a question for people who are privy to the datomic source: is the squuid intended to adhere to the type 1 time-based UUID standard, or does it differ in some way?

20:53 cfleming: technomancy: are you aware of any cases where performance would be significantly degraded without memoization? I was surprised that the profile stuff was memoized - it doesn't look performance intensive and should only be called once for a "normal" lein invocation as far as I can tell.

21:00 gfredericks: A rewrite would be nice just to get rid of this "insert into the middle of a vector" function

21:00 * gfredericks stares at (into [] (concat before [x] after))

21:04 technomancy: cfleming: I think it used to be called a bunch of times, but the profile code has probably been rewritten since then. if it's only a handful of times and the perf penalty is negligible we could remove the memoization.

21:06 cfleming: technomancy: Ok, I'll take a look and see how it looks.

21:06 hiredman: amalloy: I bet

21:09 rritoch: Hi, does anyone know how to set the classloader used by (eval)? I have some code running from OSGi that's breaking with class not found (clojure.lang.Util). I checked the source for eval on github for clojure/core.clj but I can't tell how it works as it seems to call itself.

21:11 raspasov: hey guys, has anyone used Akka from Clojure, any opinions/experiences welcome

21:11 hiredman: rplevy: I have no special knowledge, but my guess is no, they are not intended to implement the time based standard, they are to provid ordered random ids that have good sorting for data that may be accessed in a time based way

21:12 amalloy: rritoch: eval probably uses its own dynamic classloader

21:14 hiredman: https://gist.github.com/hiredman/6214648 is an example of eval with a custom classloader

21:17 gfredericks: hiredman: I just deleted this questionable looking function and wrote a topological sort and I think it worked

21:17 whoops just kidding all the tests are failing now

21:17 rplevy: hiredman: hmm ok. As a side note it would be more convenient if these supporting functions such as squuid were in a separate datomic utils library, so that they could be used without including datomic as a dependency.

21:19 I mean really it would be awesome if datomic was open source, but I guess this is Rich Hickey’s day job.

21:19 dbasch: they cannot be type 1 UUIDs based on this doc http://docs.datomic.com/javadoc/datomic/Peer.html#squuid()

21:20 hiredman: rplevy: I suspect the implementation is tied to datomic

21:20 dbasch: (it conflicts with this definition http://wiki.apache.org/cassandra/TimeBaseUUIDNotes )

21:20 hiredman: rplevy: e.g. each peer may be assigned an id by the transactor that it can use to mint ids, etc

21:21 amalloy: ah, that old "count of 100-nanosecond intervals since 00:00:00.00, 15 October 1582" gets me every time. i get why it's useful, but it seems silly at first glance

21:21 rplevy: dbasch, hiredman: thanks

21:23 rritoch: hiredman: Thanks, though I'm not entirely sure what I'm looking at, it looks like a utility for making a class loader. Can I use the push-thread-bindings to bind the classloader? The evals are executed within clojure agents so if I can bind the thread to the OSGi classloader, that would probably solve my problem(s).

21:30 hiredman: rritoch: it is a reimplementation of clojure's eval such that you can use a custom classloader (in this case it captures the bytecode)

21:31 it is old, so the compiler may have changed, so clojure.lang.Compiler/eval may do something different now, but that is what is used to do

21:34 rritoch: hiredman: Thanks. I was hoping to use clojure's built-in eval, but a custom eval would probably work. I just found the java code for eval and it's making it's own classloader to use.

21:35 ddellacosta: hiredman: hey, can you give me access to clojurebot? Thank you

22:27 myguidingstar: I have problem pushing package to clojars

22:27 com.jcraft.jsch.JSchException: Auth fail

22:27 my leiningen is latest version

22:27 amalloy: clojars doesn't take ssh uploads anymore, with the bash vulnerability

22:27 myguidingstar: I even run 'lein keygen' and add new key to clojars' profile

22:28 well

22:28 clojurebot: Cool story bro.

22:28 myguidingstar: then what should I do now amalloy ?

22:28 amalloy: and i think that error comes from uploading via scp, right?

22:28 you should be using lein deploy, which will use maven or something

22:29 myguidingstar: I'm not sure, I just use 'lein push'

22:29 okay, i'll try

22:30 amalloy: i think lein push is one of those super-old things that's been deprecated since before lein 2

22:30 $google leiningen deploy

22:30 lazybot: [leiningen/DEPLOY.md at master · technomancy/leiningen · GitHub] https://github.com/technomancy/leiningen/blob/master/doc/DEPLOY.md

22:31 myguidingstar: oh I didn't see any warning for that

22:31 amalloy: which links to https://github.com/technomancy/leiningen/blob/stable/doc/TUTORIAL.md#publishing-libraries

22:34 myguidingstar: for your convenience, the deprecation notice: https://github.com/ato/lein-clojars#clojarsorg-leiningen-plugin

22:34 i guess it would have been nice of him to release a new version of it that prints "dude stop using me", but then not everyone updates to the latest anyway

22:35 myguidingstar: many thanks amalloy

22:50 devn: Offtopic: Wow. The Princeton Companion to Mathematics is a fantastic book.

22:54 gfredericks: hiredman: I take it back again; it worked fine once I got the topological sort straightened out

22:55 it's not a totally generic sort because nrepl wants to push the independent middlewares to the end for some reason

22:59 devn: I don't think it would be an exaggeration to say it's astounding, startling in its depth and breadth. I can an of course tell you this with great confidence, for I have reached the depths of page 70.

23:00 s/an//

23:01 anyone here have a chance to check out onyx?

23:03 gfredericks: devn: interesting

23:12 xeqi: (inc amalloy) ; for dealing with clojars deployment fun

23:12 lazybot: ⇒ 171

23:12 amalloy: xeqi: you're welcome. was i right that you're still not taking scp uploads?

23:13 xeqi: amalloy: correct

23:13 there should be a notice now for scp command line attempts

23:13 no idea on the old lein-clojars plugin

23:14 technomancy: it might just discard stderr =\

23:22 amalloy: that would be the easiest thing to accidentally do

Logging service provided by n01se.net