#clojure log - Feb 29 2016

The Joy of Clojure
Main Clojure site
Google Group
IRC
List of all logged dates

3:16 prohobo: 666 nicks..

3:56 jonathanj: are there any clojure libraries to *produce* markdown from a clojure datastructure?

3:57 programmatically producing markdown is a pretty unpleasant experience, i was hoping to not write code to do it

4:00 J_Arcane: jonathanj: huh. I don't seem to be able to find one.

4:00 Mayhaps that'd be a good library project.

4:01 I've tinkered with some markdown generation in the past (my resume used to be generated from a custom MD generator in Racket). How would that look though?

4:24 jonathanj: J_Arcane: I dunno, something like hiccup maybe?

4:25 J_Arcane: Yeah. I was thinking that, maybe.

4:25 Last time I did it was with composable functions.

4:25 That just all resolved down into strings of MD-formatted text.

4:26 amalloy: why do you want to generate markdown? markdown is sorta intended to be human-editable source that turns into html

4:26 jonathanj: amalloy: sure, tell that to like everything on the planet

4:26 amalloy: huh?

4:26 jonathanj: amalloy: for example, Slack's "rich" content is some subset of Markdown

4:27 which is how you interact with it both as a human and as a machine

4:27 J_Arcane: amalloy: Well, in my last use case, I had a set of data (resume entries) that I wanted in Markdown so I could just host it on Github, which automatically gives you a rendered view of any .md file.

4:28 dacechavez: I’m modeling a simple friending app and a user is a hash-map {:name “John” :friends []}. How should I add a friend to John’s vector of friends? By using swap! on an atom or by making a pure function that returns a new hash-map with a friend added to the vector?

4:29 J_Arcane: dacechavez: That would rather depend I think on where the friend lives? Ie. is it being passed as a value, or kept in an atom?

4:29 amalloy: dacechavez: i mean, the former is something you do in additino to the latter

4:29 you can't really get away from writing that pure function; once you have, you can optionally use it as an atom's swap! function

4:30 jonathanj: J_Arcane: with the bunch of functions approach, it seems pretty hard to know when you can't do something, since the context is encoded in the markdown string you're building instead of the data you're building the string from

4:31 like when some syntax cannot be present inside some other syntax

4:32 dacechavez: hmm Ok I see..

4:33 jonathanj: as a more general complaint, i don't think anyone has solved this problem: html is way too general for most purposes, so you have to build in all kinds of white/black listing to avoid XSS etc; markdown (and most other human-readable markups is awful to produce programmatically

4:39 dacechavez: J_Arcane: i don't know.. I kept it as a value until I started testing the add-friend part, where I tried to conj something to the vector but data is immutable obviously. That's when I thought maybe I need to wrap the hash-map in an atom..(?)

5:04 Kneiva_: ,(update {:name "John" :friends []} :friends #(conj % "Bob"))

5:04 clojurebot: {:name "John", :friends ["Bob"]}

5:12 Bronsa: Kneiva_: you can just do:

5:12 ,(update {:name "john" :friends []} :friends conj "bob")

5:12 clojurebot: {:name "john", :friends ["bob"]}

5:13 Kneiva_: cool

5:28 J_Arcane: ,(update-in {:name "John" :friends []} [:friends] conj "Steve")

5:28 clojurebot: {:name "John", :friends ["Steve"]}

5:28 J_Arcane: dacechavez: Like that?

5:29 dacechavez: J_Arcane: Yea I guess. I had this urge to change the original hash-map but maybe thats not the Clojure-way

6:12 kungi: I f* love clojure

6:12 Implementing a "mocking framework" in 33 lines of code ... :-D

6:13 mokuso: :)

6:16 tdammers: plot twist: line 33 is this: ))))))))))))))))))))))))))))))))))))))))))))))))))))))))))

6:16 kungi: tdammers: :-) No actually it's )))

6:17 https://gist.github.com/Kungi/1e1b396db629272749b6

6:25 jonathanj: anyone have opinions on truss?

6:25 https://github.com/ptaoussanis/truss

7:32 benjyz1: Hi, I'm using timbre for logging. any idea for how to pipe logs from libraries?

7:32 jetty webserver spits out a lot of lines and I want to filter them

7:33 I guess .. :ns-whitelist []

8:27 kungi: Wow the cider debugger is great

10:00 shem: kungi: yes it is! just today i was stepping all over my functions just for the joy of it

10:30 justin_smith: benjyz1: the problem with ns-whitelist is that jetty doesn't have any namespaces in it

10:30 it has packages and classes...

10:30 benjyz1: trying to filter the logging output. harder than I thought

10:30 justin_smith: benjyz1: or is :ns-whitelist just a weird name and it can manage output from java crap too?

10:31 benjyz1: I have logback and XML filters, but that's just painful

10:31 justin_smith: benjyz1: the jvm logging expects filtering by classes / packages right?

10:31 benjyz1: don't know. log4j and logback are absurdly complex

10:32 I mean in logback you can write filters which reference classes ... ha :D

10:32 tdammers: average java applications log absurd amounts of uncaught exceptions, too

10:32 benjyz1: so XML => class filter.. genius

10:40 jonathanj: is there a name for (first (filter pred xs))?

10:40 justin_smith: jonathanj: that should be a FAQ if we had a FAQ and the answer is no, not in core

10:41 jonathanj: what does one normally call that?

10:43 ,(some even? [1 1 nil 2 4])

10:43 clojurebot: #error {\n :cause "Argument must be an integer: "\n :via\n [{:type java.lang.IllegalArgumentException\n :message "Argument must be an integer: "\n :at [clojure.core$even_QMARK_ invokeStatic "core.clj" 1372]}]\n :trace\n [[clojure.core$even_QMARK_ invokeStatic "core.clj" 1372]\n [clojure.core$even_QMARK_ invoke "core.clj" 1367]\n [clojure.core$some invokeStatic "core.clj" 2592]\n [clojure.co...

10:44 jonathanj: not quite.

10:54 Rick77: Hi! Anyone knows how to declare static fields with gen-class? I know how to declare static (or not static) methods, but I found no information about the fields...

11:05 never mind, found a simpler solution involving leiningen and java-source-path: ;-)

11:16 justin_smith: jonathanj: some is different in that it returns what the predicate returns

11:17 jonathanj: justin_smith: true

13:19 puredanger: justin_smith: btw, there is now a faq: http://clojure.org/guides/faq

13:19 and would be happy to have a PRs expanding it on https://github.com/clojure/clojure-site

13:35 jonathanj: nice FAQ

13:46 justin_smith: puredanger: oh cool, I was thinking a FAQ for the IRC channel (which would be different in tone / target than the clojure.org FAQ maybe?) but good to know the clojure.org FAQ is there and accepts PRs, thanks

14:18 jweiss: is there a library that tracks changes of data (branching, merging etc) basically the way git does with files?

14:18 justin_smith: jweiss: not that I know of, though clojure.data.diff/diff is cool

14:18 ,(require 'clojure.data.diff)

14:19 clojurebot: #error {\n :cause "Could not locate clojure/data/diff__init.class or clojure/data/diff.clj on classpath."\n :via\n [{:type java.io.FileNotFoundException\n :message "Could not locate clojure/data/diff__init.class or clojure/data/diff.clj on classpath."\n :at [clojure.lang.RT load "RT.java" 456]}]\n :trace\n [[clojure.lang.RT load "RT.java" 456]\n [clojure.lang.RT load "RT.java" 419]\n [clojur...

14:19 justin_smith: err

14:19 jweiss: justin_smith: yeah, i've used that a bit before to diff changes using watchers. afaict there's no branching or merging though

14:19 justin_smith: right, right

14:20 ,(require 'clojure.data)

14:20 clojurebot: nil

14:20 justin_smith: ,(clojure.data/diff {:a 0 :b 1 :c 2} {:a 1 :b 2 :c 2})

14:20 clojurebot: ({:b 1, :a 0} {:b 2, :a 1} {:c 2})

14:20 jweiss: could definitely use clojure.data/diff to help write a merge function though

14:21 justin_smith: yeah, I imagine a thing that only looks at top level differences and then recurs on changed subtrees

14:21 but then maybe its easier to do your own one level diff at that point

14:23 jweiss: i think i'd have to make a graph (using loom or something) and traverse it up to the common ancestor, apply all the diffs of each branch after that (my merge conflict resolution is much simpler than git's)

14:24 the only conflict it would resolve is both branches adding different items (they'd be merged like a map does it)

14:25 justin_smith: jweiss: why walk back up again when clojure does structural equality?

14:25 and might even be using the same underlying k/v pair to represent that part of the data if they came from the same origin

14:27 jweiss: justin_smith: hm. i had planned on having each node contain changes to the ancestor node, but i may be able to just store the resulting value and let clojure's persistent data structures deal with the memory management

14:28 justin_smith: oh, OK, I guess I misunderstood your plan... interesting

14:31 jweiss: justin_smith: it may be as simple as writing a merge function that takes the common ancestor value, and both branch values, and just returns the merged value. i can't think of any situation where not knowing the intervening history would be a problem.

14:41 justin_smith: jweiss: in that case you might even be able to do some magic with (or variation on) the existing merge-with

15:35 sdegutis: You know how sometimes you end up with a chain of data processing using ->> ?

15:36 Well, I've got one of those, and I'm trying to figure out a nice way of accessing the data in the middle of the chain while still using it.

15:36 Normally I'd split the chain into two chains, the first one bound to a variable-name inside a (let), so that I can just access it via that variable.

15:37 But I'm betting there's probably a smarter way.

15:38 Well I guess another way is to extract a custom function that wraps the thing I'm trying to do. But that's just an extended version of the (let) solution.

15:41 tolstoy: Would the as-> macro help?

15:42 kenrestivo: how did people check for exceptions in clojure.test prior to 1.8?

15:42 there's no thrown? in 1.7 apparently

15:42 sdegutis: tolstoy: that was my first though, but it's not useful in a ->> chain because the argument goes at the end whereas as-> needs it at the beginning.

15:42 tolstoy: in fact Clojure doesn't have an elegant way of accessing or destructuring the "last" argument, whereas it does have it for the first.

15:43 tolstoy: Oh, I thought you could put "name" in any position of subsequent expressions.

15:43 sdegutis: kenrestivo: I thought it was?

15:43 luma: kenrestivo, it's been there always

15:43 kenrestivo: really?

15:44 clojure.lang.Compiler$CompilerException: java.lang.RuntimeException: No such var: clojure.test/thrown?, compiling:(/tmp/form-init2006229078634769762.clj:1:680)

15:44 ,(use clojure.test)

15:44 luma: well, it has never been a var, the is macro just checks if there's a symbol 'thrown? in there

15:44 clojurebot: #error {\n :cause "clojure.test"\n :via\n [{:type clojure.lang.Compiler$CompilerException\n :message "java.lang.ClassNotFoundException: clojure.test, compiling:(NO_SOURCE_PATH:0:0)"\n :at [clojure.lang.Compiler analyze "Compiler.java" 6688]}\n {:type java.lang.ClassNotFoundException\n :message "clojure.test"\n :at [java.net.URLClassLoader$1 run "URLClassLoader.java" 366]}]\n :trace\n [[ja...

15:45 kenrestivo: oh

15:46 amalloy: kenrestivo: there was never a thrown? in any version

15:46 not as a var that exists

15:46 rather, it's a symbol that's recognized by the 'is macro

15:46 kenrestivo: amalloy: luma pointed out it's not a var, it's just a symbol the macro checks for

15:47 amalloy: oh, i see. i missed it between your clojurebot chatting

15:47 kenrestivo: sorry for the noize

15:47 amalloy: note though that it's not actually a special symbol that's hard-coded into 'is: there's a multimethod used by 'is, and 'thrown? is one of the methods

15:47 you can add your own if you want

15:47 kenrestivo: dispatches on it? great, thanks.

15:48 it's fine, just trying to check for a specific exception in a test case.

15:57 Shayanjm: justin_smith: when using clj-kafka, how do you test your producer-consumer interactions? It feels like I almost need to have 2 REPLs going since the data is streamed...

15:57 I seem to be able to produce just fine, but i'm wrestling with the consumer

16:05 justin_smith: Shayanjm: what kind of managing do you need between your producer and consumer?

16:06 Shayanjm: justin_smith: for right now - I'm just trying to test that I can produce messages that my consumer can then parse into some sort of data structure

16:06 justin_smith: Shayanjm: my consumer just runs in a thread, always looping and looking for the next message

16:06 so you could set this up via core.async (what I used) or just threads and queues

16:07 Shayanjm: but you can also produce, and then consume, one after the other in a repl

16:07 that should work with what you are describing just fine

16:07 * sdegutis sighs

16:07 Shayanjm: justin_smith: I actually tried that and didn't get the result I was expecting

16:07 let me try again

16:08 justin_smith: Shayanjm: what I found useful was first figuring out how to make everything work with kafka-console-producer.sh and kafka-console-consumer.sh

16:08 eg. just the two of those interacting

16:08 Shayanjm: justin_smith: Sure, I've got those two interacting nicely

16:08 justin_smith: and then after that I introduced my clojure stuff on oneside, leaving the shell one on the other end

16:08 Shayanjm: Yeah that's exactly my approach. I've been producing via clojure & consuming via shell

16:09 justin_smith: that isolates the errors (did I produce wrong? consume wrong?)

16:09 Shayanjm: let me try doing one after the other

16:09 justin_smith: Shayanjm: the most likely error with producing then consuming in one repl is that you set your offset to latest, which explicitly ignores messages from before the start of your consumer, and is usually a default

16:10 Shayanjm: ahhhhhh

16:10 justin_smith: instead of latest you can ask for "oldest"

16:10 iirc

16:10 Shayanjm: so in that case it's best practice to create a consumer, emit messages, and then (def msgs (messages c "topic"))?

16:10 justin_smith: Shayanjm: well, that would be one way to do it, sure

16:10 or start the consumer in a loop on a thread putting messages in a data structure

16:10 whatever makes sense

16:12 Shayanjm: justin_smith: is there a 'best' way to start an ever-consuming background process using core.async?

16:13 one that I can preferably manage (start/stop at will)?

16:17 justin_smith: unless you are already a core.async user, it's likely easier to loop in a future, reading and putting the result in a java.util.concurrent.ArrayBlockingQueue

16:18 and then if you have a handle to the future, you can use future-cancel and the wait to read from kafka allows that cancellation (it's one of the small set of things future-cancel reliably halts)

16:19 with core.async this would be done with alts! and a cancellation channel but no need to go into that if you aren't already using that tool here

16:28 Shayanjm: awesome thanks! I'm going to play around with it a bit

16:29 justin_smith: Shayanjm: also for testing you could just open two repl connections to the same lein repl actually

16:29 so you don't need two actual instances of your app running, just two repl connections, one reading and the other producing

17:08 dbasch: this is not a clojure question, but please indulge me. I need to give a 20-min talk at work titled “What is functional programming” and the audience is mostly junior devs, many of them without any formal CS training (coding school, Ruby, js). I want to make it fun and interesting, looking for ideas about how to start with a good hook.

17:10 amalloy: dbasch: https://docs.google.com/presentation/d/1mGihUBBIKMQn5Uz-5DvQ1Vu24qSuk3o4yUoPoBZW3tI/edit is a presentation i put together that you might steal from

17:11 dbasch: (inc amalloy) ;; if this still works

17:12 I was thinking about going “functional?” “programming?”, then going into the mathematical concept of a function, then asking “ok, but what does this have to do with programming”

17:13 amalloy: that probably takes longer than 20 minutes, especially for junior devs without the theoretical background

17:13 i think i gave this talk in 35-40 minutes, but you could easily cut out half of it

17:14 dbasch: yeah, I don’t think I want to spend much time on laziness and concurrency. I have to mention them, but it would be too much.

17:14 amalloy: incidentally, https://cdsmith.wordpress.com/2012/04/18/why-do-monads-matter/ is a cool article on monads that follows the approach you mention

17:19 al80: what is the best way to do TDD (London style) with Clojure?

17:20 clojure.test is it ok? Is there a lib that provide mocking, expectations?

17:20 and is everything runnable from a repl?

18:07 justin_smith: al80 left but clojure has mocking via with-redefs, and clojure.test works great, though refactoring code slightly for testability is often win-win regardless of which testing setup you use

18:07 and yes it works from a repl

18:59 pilne: now to convince the friend whose project i committed to that clojure and clojurescript are far superior to java for the jvm... *twirls my non-existant villan mustache

19:01 (java and javascript, lol... too busy plotting to be smart i guess... lol)

20:06 simplesam: I have a java class that has a public interface: public class AA { public interface ILog { void log(String logStr); }}

20:06 how can I implement ILog in Clojure ?

20:06 amalloy: (reify AA$ILog ...)

20:07 justin_smith: simplesam: the jvm doesn't actually have nested classes or interfaces, it just creates a silly name in the bytecode

20:07 the name amalloy mentioned

20:07 simplesam: i've tried using deftype without success.

20:07 ah ok. thanks. i'll give that a go

20:16 cortexman: blow my mind

Logging service provided by n01se.net