#clojure log - Feb 05 2013

The Joy of Clojure
Main Clojure site
Google Group
List of all logged dates

0:05 ivan: apparently my Xbootclasspath/a:clojure/target/classes was causing my truncated class problems

0:05 maybe JDK7u13 broke it

0:10 * ivan stops being a dummy and puts the .jar there instead, which works

1:05 callenbot: ivan: yes, blame Oracle.

1:15 ivan: that should always be the first step

1:22 xeqi: ugh, arbitrary code execution found

1:23 ivan: hope nobody runs master in production

1:24 xeqi: ... now you made me wonder what the responsible disclosure would be

1:24 was planning on a reply in the ml

1:28 tomoj: https://www.refheap.com/paste/0edb0432d917a9f7e73e3e570 :/

1:28 at least it works

1:35 sshack: Does anyone know if the in canter project is still alive?

2:04 nonuby: is there a version of -> macro that works right to left, i want to use it with hiccup (otherway-> with-header with-container with-normal-body (get-article 1))

2:05 ivan: perhaps comp

2:06 nonuby: ivan, thanks, will check docs on that

3:18 luxbock: anyone here using ac-nrepl for Emacs? for some reason ac-nrepl-popup-doc works fine for me in the nrepl-buffer, but if I try to use it in a clojure-mode buffer where I'm editing something, then it complains:

3:18 ac-nrepl-documentation: Wrong type argument: arrayp, nil

3:21 bdash: luxbock: fwiw, I saw the same thing when I tried it last week

3:22 luxbock: hmm

3:23 amalloy: nonuby: that's just parentheses. (with-header (with-container (with-normal-body (get-article 1))))

3:28 nurettin: I wish clojure could be written like [1 1 1] reduce +

3:28 it would look even better than ruby

3:40 nonuby: just playing in repl if do just want map to be a no-op how do I do this (map #(%) [1 2 3])

3:43 thorwil: nonuby: use identity

3:43 nonuby: ah do work

3:43 s

3:43 'do' still seems nasty, http://pastebin.com/fjaQLTPP

3:43 ignoring the css != script mistake

3:45 thorwil thanks

3:46 thorwil: nonuby: looking at your paste, why don't you use defhtml?

3:47 Ember-: http://www.javacodegeeks.com/2013/02/java-8-from-permgen-to-metaspace.html

3:48 nonuby: thorwill, will revisit the docs, admittedly i rushed into this with only cursory read of frontpage readme.md

4:13 drorbemet: Hello, may I ask questions on how to get started with Clojure-Projects on Eclipse?

4:14 clgv: drorbemet: just do. you already have Counterclockwise installed?

4:18 drorbemet: Yes, I have Eclipse, Counterclockwise and Leiningen installed. I build up the startingclojure project successfully. But now I am having trouble with importing projects from github.

4:19 clgv: drorbemet: describe your problem in more detail: what do you want to do? what do you expect? what happens (error messages?)?

4:36 desertmonad: anyone used the kindle version of Clojure Programming?

4:37 pimeys: has anybody done the SICP excersises with Clojure?

4:38 clgv: pimeys: there are one or two resources where they try to convert all SICP examples to clojure

4:38 $google sicp clojure

4:38 lazybot: [SICP in Clojure] http://sicpinclojure.com/

4:38 drorbemet: I am unsure about the order in which I have to set up the project template with Leiningen and importing a project from github.

4:39 I want to import e.g. clj-file-browser from github into Eclipse with Leiningen support. If I set up a new Leiningen-Project with name "clj-file-browser" first and then import https://github.com/xiaonaitong/clj-file-browser.git - I end up beeing asked to set up a "New Clojure Project" again. But if I import from github first I have no Leiningen support.

4:39 clgv: drorbemet: do you want to modify the github project or do you just want to use it?

4:39 pimeys: clgv: it's pretty easy to do the exersises with clojure, just some of them are not so relevant anymore

4:39 for some reason lisps are quite easy to understand, even between the different versions

4:40 just wondering the relevance of chapter 2 now, doing the message passing style

4:40 having abstractions over basic sum etc.

4:40 and using lists to keep the value type

4:40 clgv: drorbemet: you can enabled leiningen support rightclicking the project

4:41 drorbemet: I want to work with it as an exercise, but without pushing up to github at the moment.

4:45 clgv: drorbemet: when you only want to use it you just add it to your project.clj

4:45 drorbemet: Exactly, that's what I am missing when I rightclick on the project. I have an other project in Eclipse, that has Leiningen in the context menue.

4:46 I think I have to do it on the shell

4:47 clgv: drorbemet: you need a project.clj in that project and then in the context menu there is an entry "Convert to leiningen project" under "Configure"

4:48 drorbemet: but again: if you only want to use it in another project, add it as dependency

4:49 pimeys: oh, there it is http://sicpinclojure.com/?q=sicp/2-5-3-example-symbolic-algebra

4:49 drorbemet: Ah, thanks a lot, ... Now I found it :-)

4:49 pimeys: does anybody in the lisp world use that way of building software?

4:49 doing the put functions etc.

4:49 keeping the functions and types in tables

5:02 drorbemet: pimeys: which projects would you suggest for exercise with clojure?

5:03 AWizzArd: Hello regexperts, I currently have this:

5:03 ,(re-seq #"(.)\d+" "(and &10 &20 $30 &40)")

5:03 clojurebot: (["&10" "&"] ["&20" "&"] ["$30" "$"] ["&40" "&"])

5:04 clgv: drorbemet: 4clojure.com

5:04 AWizzArd: Is it possible (maybe with look-around assertions?) to get instead ==> (["10" "&"] ["20" "&"] ["30" "$"] ["40" "&"])

5:04 So, without matching the & or $ symbol, but still collecting it as group?

5:05 clgv: AWizzArd: you can use a range [$&]

5:05 drorbemet: Ok, yes 4clojure is very helpfull, I am doing that from time to time :-)

5:06 pimeys: drorbemet: do your own lisp :)

5:06 it's the most fun way of digging into lisps

5:06 metacircular evaluator

5:08 clgv: drorbemet: usual exercises from computer programming courses might serve as well especially from functional programming

5:08 drorbemet: pimeys: yet I am starting to get there after reading a lot :-)

5:09 pimeys: drorbemet: doing the evaluator is not at all hard

5:09 when you get it, it's kind of an explosion of your mind :)

5:09 there's a reason that there is thousands of lisp implementations

5:10 Ember-: AWizzArd:

5:10 ,(re-seq #"(?<=$|&).\d+" "(and &10 &20 $30 &40)")

5:10 clojurebot: ("10" "20" "40")

5:10 Ember-: like this?

5:11 AWizzArd: nearly

5:11 Ember-: oh, you wanted the & and $ also

5:11 but not with the decimals

5:11 AWizzArd: I like that it now matches 10 20 30 40, but I would like to get the $ or & too

5:11 so I know that it was the & in front of the 10

5:11 ["10" "&"]

5:12 Ember-: ok, but what's the problem of just matching to "$10"

5:12 clgv: ,(re-seq #"(.)(\d+)" "(and &10 &20 $30 &40)")

5:12 clojurebot: (["&10" "&" "10"] ["&20" "&" "20"] ["$30" "$" "30"] ["&40" "&" "40"])

5:12 Ember-: ,(first "$10")

5:12 clojurebot: \$

5:12 AWizzArd: clgv: hmm yes, this works

5:12 pimeys: for me the hardest thing with lisps are the macros

5:12 Ember-: (second ["10" "$"])

5:12 ,(second ["10" "$"])

5:12 clojurebot: "$"

5:12 pimeys: I still don't get the syntax

5:13 clgv: AWizzArd: the first entry is always the whole match and after that you get the groups

5:13 Ember-: but anyway, you can as easily get the first character to check for $ or &

5:13 pimeys: I read The Joy of Clojure, everything else is very simple

5:13 drorbemet: pimeys: Yes in Lisp I see the relevant patterns more clearly I think.

5:13 AWizzArd: Ember-: yes of course, I can easily transform my output. But would be nice of the regexp to give me what I want, if possible ;)

5:13 pimeys: but oh my god the macros :P

5:13 and I really want to learn them

5:13 Ember-: AWizzArd: well you'll end up fetching stuff from the second index of the grouped list anyway

5:13 AWizzArd: clgv: yes good, I can do it as you suggested by having two groups. Good idea.

5:14 Ember-: but glad you got your solution

5:16 jballanc: nurettin: (re: alt syntax) check out Io. It works sorta like that...

5:23 desertmonad: pimeys: This video helped me a fair bit with macros in clojure (http://www.infoq.com/presentations/Clojure-Macros)

6:06 nurettin: jballanc: yes, it looks almost like that: list(1, 2, 3) reduce(+) http://iolanguage.org/scm/io/docs/reference/index.html#/Core/Core/List/reduce

6:07 io library itself looks surprisingly rich

6:44 juxovec: hi, I have function abc and list of params [1 2 3] and want to call (abc 1 2 3) instead of (abc [1 2 3]), how can I do this?

6:47 malesch: (apply abc [1 2 3])

6:47 juxovec: thanks

6:48 matrix: power script

6:50 deg: I'm getting a "reference to field ToString can't be resolved warning", apparently from this line: (defmacro current-function-desc [] `(-> (Throwable.) .getStackTrace first .toString unmangle))

6:51 I'm not sure where to put the hint; my naive attempts are all failing.

6:51 (don't worry about the defn of unmangle... my utility)

6:57 Bronsa: Deece: (defmacro current-function-desc [] `(-> (Throwable.) .getStackTrace ^Object first .toString unmangle))

6:57 ops

6:57 deg ^

6:57 or you could just call str instead of .toString

6:57 clgv: deg: those threading macros are problematic in terms of type hints. I often failed to get the hint to work although I tried all logical position

7:14 deg: Bronsa: Thx, str solved the problem. Brilliantly obvious in hindsight.

7:16 hyPiRion: Wait what

7:16 The Clojure compiler isn't able to deduct that every value can use .toString?

7:17 Bronsa: right.

7:17 hyPiRion: That's kind of silly.

7:17 Not completely, but kind of.

7:18 pimeys: I'm kind of wondering how much of the java stuff and methods you really need in daily life with clojure

7:18 clgv: hyPiRion: cries for a jira ticket ;)

7:18 pimeys: the syntax is not the pretties

7:18 t

7:18 to do (.fooBar (Foobar. "value"))

7:19 Bronsa: hyPiRion: w/e, I just always use str

7:19 pimeys: me too

7:20 Bronsa: it's better looking, can HOF, and removes the need for hinting

7:20 hyPiRion: Bronsa: what about .equals or .hashCode? They should suffer from the same thing.

7:20 pimeys: is there an eq? function like in scheme?

7:20 hyPiRion: It's a minor detail, and they can be replaced by = and hashcode, but well.

7:21 pimeys: I don't like to think about objects when doing clojure

7:21 deg: The sad thing is I also always use str, with this one exception. This was a piece of code that I copied from somewhere back when I first started using Clojure a few months back. Guess if I was really motivated, I'd find the blog page and see it is open for comments.

7:21 pimeys: but am I then using the wrong language :)

7:22 ivaraasen: type hints are a bit annoying

7:22 wasn't able to get them to work properly in macros

7:23 hyPiRion: pimeys: I agree, but the compiler itself should be able to infer that a cast to an object will never fail (except for, woo, null)

7:23 cemerick: hyPiRion: if a call only resolves to java.lang.Object, the compiler assumes it's a reflective call; isn't related to .toString in particular

7:24 Probably a reasonable enhancement.

7:25 pimeys: does the clojure still use the STM when modifying java objects?

7:25 clgv: hyPiRion: there is one exception: primitives

7:25 pimeys: ok, objects themself are not so dangerous, I'm more worried about class variables

7:25 cemerick: pimeys: Clojure never uses STM implicitly.

7:25 hyPiRion: clgv: But wouldn't they be boxed unless explicitly casted?

7:25 pimeys: so if I build my concurrent application on top of the clojure agents and futures, can I be sure no roque library will cause any trouble with the threads

7:25 clgv: hyPiRion: yeah they would^^

7:25 pimeys: this is the biggest reason for me now to learn clojure

7:26 clgv: at least when passed as parameter to a function

7:27 pimeys: I'm doing a lot of work with ruby and threads, and I want a language that has nice interface for doing parallel stuff without any worries

7:27 arcatan: Haskell!

7:27 pimeys: so it's either clojure, haskell or erlang, and I love lisp syntax

7:28 the immutability of clojure is a big thing for me, but the only thing I'm not sure of is the java interop

7:29 ivaraasen: pimeys: Java interop isn't a problem usually, but YMMV

7:29 hyPiRion: It may sound like I'm advocating casting whenever Object-methods are called, but it's not a big issue. I'm just saying that it should be possible. If it's a trivial and simple enhancement in the compiler, then I may consider a ticket.

7:29 pimeys: yeah, that's what I thought

7:29 if you're instantizing inside your threads and modifying the state there

7:30 at least the situation is not as hellish as with ruby

7:30 hyPiRion: pimeys: Java interop is very, very easy if you need to work with Java objects.

7:30 pimeys: it is, just how threadsafe that is?

7:31 do we have the same protection mechanism as with refs for example?

7:31 hyPiRion: pimeys: Just as threadsafe as java is. You can use monitors and stuff if you need to.

7:32 clgv: pimeys: you get pure java in interop

7:32 pimeys: so you can do a class variable and mutate it?

7:32 clgv: pimeys: but there are some convenience macros for several access pattern

7:32 pimeys: sorry, my java experience is from 10 years ago

7:32 alexnixon: pimeys: you'll want to be careful combining mutable Java objects and transactions

7:33 pimeys: and what about clojure libraries, that use java libs behind the scenes, how easily they do something dangerous that will mess up my app when using threads?

7:33 I mean, with ruby, it's almost half of the libraries

7:33 I don't want the same situation when I'm building my next thing with some toolset :)

7:34 hyPiRion: pimeys: The most used wrappers take care of that, from my experience.

7:34 pimeys: if it's plain clojure, I'm really fine with that, great ideas in there

7:35 hyPiRion: The big issue is how you wrap java libraries where there's mutating objects.

7:35 pimeys: of course

7:35 hyPiRion: Usually you'd prefer some other alternative if it exists.

7:35 pimeys: how big of an issue this is with clojure libraries

7:35 is it still always a good idea to read the whole source code first :)

7:36 hyPiRion: I haven't had any issues with it yet, but I haven't been a heavy library user

7:36 ivaraasen: pimeys: I believe most wrappers take care to use the ! convention for functions that mutate objects

7:36 pimeys: ach so

7:36 nice :)

7:37 hyPiRion: Maybe cemerick has some input here, since he did a talk about it.

7:37 or well, the bad parts of Clojure.

7:37 pimeys: is it downloadable somewhere?

7:37 hyPiRion: pimeys: It's watchable, let me find the link

7:37 ivaraasen: hyPiRion: yeah, that talk is great

7:37 pimeys: I'm still not sure what lisp to use for my next big thing, clojure feels like the best choice for now

7:38 there's leinegen, there's good community and really good ways of doing parallel computing

7:38 cemerick: hyPiRion: on what?

7:38 ivaraasen: pimeys: I dabble in SBCL and Racket from time to time, but I find Clojure more practical. also, nice data structures

7:39 cemerick: sorry, haven't been following

7:39 hyPiRion: cemerick: Wrapping java libraries

7:39 pimeys: http://www.infoq.com/presentations/What-Sucks-about-Clojure-and-Why-You-ll-Love-It-Anyway

7:39 cemerick: hrm, I didn't do a talk on that

7:39 oh, that

7:39 yeah, there's ~5m on that topic in there

7:39 hyPiRion: cemerick: Well, you did mention it at least, so I assumed you had some experience re that issue.

7:40 cemerick: tl;dr: be careful about which libraries you choose

7:40 pimeys: cemerick: that I was asking

7:40 should I be careful on selecting clojure libraries, like I have to be when doing ruby

7:40 hyPiRion: pimeys: What are you going to build, by the way?

7:41 pimeys: a callback system, which interacts with thousands of different servers

7:41 and has 1000-2000 operations per second

7:41 investigating my toolset for that

7:41 ivaraasen: intense

7:41 pimeys: the current system is ruby + beanstalkd + threads and works fine for this traffic

7:41 but I see the sore points and when I need to refactor that, I might change the language

7:41 Bronsa: I'm confused.

7:42 clojure-1.5.0-beta7 has just been released

7:43 hyPiRion: pimeys: I'd look at either Clojure or Erlang for that system at least. Erlang is designed for those things if I've understood the language correctly

7:43 pimeys: hyPiRion: I know, but I have this not so reasonable lisp devil living inside me :D

7:44 hyPiRion: :D

7:44 pimeys: maybe with elixir erlang would be better

7:44 cemerick: pimeys: of course, you always need to be careful about which libraries you use, regardless of language. Choosing and using Java libs from Clojure deserves special consideration though, insofar as the former can be wildly stateful and unpleasant to use with the latter.

7:45 hyPiRion: I haven't worked with elixir yet, will do so when I know the language better.

7:45 pimeys: how often you find a need to use java libraries with clojure?

7:46 hyPiRion: It seems like you should be fine with agents for the concurrency parts.

7:46 ivaraasen: hyPiRion: would probably need message queues to make it concurrent as well

7:46 pimeys: my libraries with that project might be korma and clj-http

7:47 and I like this clojure approach with agents better than the erlang processes

7:47 where you can send functions instead of plain messages

7:48 hyPiRion: ivaraasen: Yeah, you got lamina for that I think

7:49 Bronsa: huh, maybe Rich decided that the RC should go away since there are new changes (*read-eval* discussion going on)

7:51 pimeys: neither clj-http nor korma has mutability issues. Though well, there has been some discussion about Korma's way of wrapping stuff. See https://github.com/korma/Korma/issues/64 and other issues before you decide to go that route.

8:21 luxbock: (reduce #(and %1 %2) [... bools ...]) and (every? bools) do the same thing, right?

8:21 hyPiRion: luxbock: almost.

8:22 luxbock: what's the difference?

8:22 hyPiRion: ,(reduce #(and %1 %2) [true true])

8:22 clojurebot: true

8:22 hyPiRion: ,(reduce #(and %1 %2) [])

8:22 clojurebot: #<ArityException clojure.lang.ArityException: Wrong number of args (0) passed to: sandbox$eval60$fn>

8:22 hyPiRion: ,(every? boolean [])

8:22 clojurebot: true

8:22 hyPiRion: ,(every? boolean [true true])

8:22 clojurebot: true

8:22 luxbock: ah I see

8:22 hyPiRion: ,(every? boolean [true false])

8:22 clojurebot: false

8:22 hyPiRion: ,(reduce #(and %1 %2) true []) ; should fix it for you.

8:22 clojurebot: true

8:24 clojure-newb: hey guys, my compojure.route/files stops serving static resources once i have a route like "/some-path/:id", any ideas on a strategy that works across all sorts of different routes ?

8:25 I understand that its relative.. just wondering what people do to solve this kind of thing

8:25 pimeys: in which order you have your routes?

8:25 if I remember correctly the order matters

8:25 not an expert here yet, just guessing

8:26 clojure-newb: pimeys: I see you think having the "/some-path/:id" in a different order to the "/some-path" could fix things ?

8:27 pimeys: not really

8:27 somebody is overwriting your files path

8:27 but maybe somebody who knows better can explain

8:28 clojure-newb: if I place a '/' as a prefix to the css href attribute it works, but then obviously breaks when trying to load straight from disk into a browser

8:29 I'm trying to have the best of both worlds

8:30 bburns777: hello, anyone out there ever use storm's clojure del?

8:30 clojure DSL rather

8:36 rhickey: beta7 should be out, looking for experience reports

8:43 cemerick: rhickey: tests running now

8:45 clojure-newb: is it advisable to use composure to serve static files in production ? wondering how it performs compared to the apache's etc

8:45 xumingmingv: &(defn f1 [a] (-> a meta))

8:45 lazybot: java.lang.SecurityException: You tripped the alarm! def is bad!

8:45 xumingmingv:

8:46 I cannt define a function in the sandbox? lazybot???

8:46 lazybot: xumingmingv: Oh, absolutely.

8:46 xumingmingv: I cannt define a function in the sandbox? lazybot!!!

8:46 AWizzArd: Where is beta7? I see 1.5.0-RC6 @maven.org

8:49 rhickey: AWizzArd: it takes a while to show up

8:52 cemerick: AWizzArd: you can add the sonatype releases repo to your project.clj to get it right away

8:53 xeqi: AWizzArd: https://www.refheap.com/paste/9475

8:58 AWizzArd: Oki great, thx

9:04 Btw, could it make sense at some point that print-dup and print-methods become protocol functions?

9:11 clgv: oh a beta in-between release candidates. is that because of the *read-eval* patch?

9:14 stuartsierra: clgv: I think that was Hudson having a spasm.

9:14 rhickey: we are back in beta

9:18 clgv: what happened?

9:18 stuartsierra: Oh, ok.

9:18 Our new contrib for tools.reader had some issues with Hudson releases last night, so I thought it might be related.

9:23 augustl: what data structures are good for map-like structures where both keys and values are unique, and I can look up values both ways (get the value of a key, and the key of a value, in hashmap terminology)?

9:23 clgv: augustl: sounds like you need to combine two maps

9:25 augustl: clgv: it's the same map, it holds a map between internal database values (a namespaced symbol) and publicly used values in the API (a string)

9:25 I need to look up both ways, depending on whether I look at incoming data or internal data

9:26 clgv: well you have to add each pair twice (both vals as key and value) then

9:27 augustl: if I use a normal map, then yes, I was hoping there was an alternative to that :)

9:27 cemerick: clgv: arbitrary code execution vulnerabilities are front-of-mind for many of us. Hoping to see Clojure buttoned up from that perspective ASAP.

9:33 saalaa-folio: hey guys, anyone has had issues with vim-foreplay and recent versions of vim?

9:33 recent as in 7.3.754

9:34 clgv: cemerick: so that is the reason for dropping back to beta?

9:36 cemerick: clgv: appears so, yes

9:44 tpope: saalaa-folio: I haven't tried. what kinds of issues?

9:46 saalaa-folio: I'm not sure, I'm cleaning my setup so that I don't make useless noise

9:48 seems like there's an error in classpath#detect line 64

9:51 tpope: saalaa-folio: what error? and do you have https://github.com/tpope/vim-classpath installed or are you just using the one that ships with foreplay?

9:51 saalaa-folio: it's installed

9:51 but this error occurs even without vim-classpath

9:52 tpope: once again, what's the error?

9:52 check :messages if you can't see it

9:52 saalaa-folio: and the error occurs when I open any .clj file in a project

9:52 here's the full text: "project.clj" 6L, 264C

9:52 Error detected while processing function classpath#detect:

9:52 line 64:

9:52 E484: Can't open file /tmp/vD3yrRQ/6

9:52 tpope: check :set shell?

9:53 saalaa-folio: I'm using fish

9:53 hum

9:53 tpope: oh, fish will give you all sorts of trouble

9:53 add set shell=/bin/bash to your vimrc

9:54 saalaa-folio: yeah, it's working now

9:55 thanks a lot tpope!

9:55 tpope: ;)

10:19 shriphani: hello everyone. I am struggling to find info on how to run a script (i.e. write a main function and so on). Is there a definitive way to do this ? (I see source code using *command-line-args*, main [& args]. What is the definitive way of doing this?

10:20 nDuff: shriphani: main [& args] is closer to what the language will do for you automatically.

10:20 shriphani: *command-line-args* is a var that's presumably being populated somewhere -- ie. someone is actually setting its value themselves. That doesn't make it a bad idea.

10:21 shriphani: i see.

10:21 nDuff: (indeed, as a dynamic var, its values can be overridden during individual tests during test suite execution, which maybe makes it a _good_ idea)

10:21 shriphani: and the [& args] format gives you an array with command line args in it?

10:21 nDuff: that expands all remaining arguments into a vector called args

10:21 it happens, for something that's a conventional main, that that's what you want.

10:22 shriphani: ah I see.

10:22 nDuff: ...it's nothing specific to command-line arguments, but using the normal destructuring mechanism that works everywhere else as well.

10:24 shriphani: I am coming from racket to clojure (libraries). I am using clj <scriptname> <arg> to run my script and I am doing a println of the args vector. I don't see anything on stdout. Am I doing something wrong ?

10:25 borkdude: shriphani what is clj, a script? what's in that script

10:25 * nDuff has never heard of this ''clj'' tool.

10:26 nDuff: What I was talking about, with respect to building a main, is for if you're compiling to a traditional/conventional Java main

10:26 This ''clj'' sounds like it does something else.

10:26 shriphani: https://gist.github.com/shriphani/4715130 <-- that is my clj

10:26 I found it in one of my tutorials.

10:26 nDuff: Ahh, so it's arguments passed directly to clojure.main

10:27 borkdude: shriphani the most easy way I would say is make it a leiningen project and call "lein run args"

10:27 nDuff: shriphani: ...for the future, getting accustomed to using Leiningen really _is_ the right thing -- you'll need it when you start dealing with 3rd-party libraries.

10:27 Ahh. *command-line-args* is something set by clojure.main

10:28 Didn't know that because, well, clojure.main isn't very often used.

10:28 ...so, if you _are_ going to use clojure.main, *command-line-args* is the right thing.

10:28 TimMc: stuartsierra: I'm really glad to see this read-eval stuff going into 1.5. Thoughts on backporting?

10:28 shriphani: I just ran it with lein run and I get this: No :main namespace specified in project.clj.

10:28 nDuff: shriphani: so edit your project.clj and tell it which namespace you put your main in.

10:28 stuartsierra: TimMc: I don't understand.

10:29 TimMc: stuartsierra: 1.4.1, 1.3.1, 1.2.2 with dead-eval bound to false by default

10:29 *read-eval :-P

10:29 stuartsierra: Erg. If you're volunteering...

10:29 borkdude: shriphani add ":main foo.core", if foo.core is the namespace you put the -main function in to project.clj settings

10:30 shriphani oh yeh, the standard name of the main function is -main

10:30 shriphani so not the minus

10:30 shriphani note

10:32 cemerick: TimMc: FWIW, it looks like *read-eval* will not be false in 1.5.

10:32 TimMc: cemerick: D-:

10:33 I suppose I should check the mailing list ever.

10:33 shriphani: borkdude and nDuff thanks for the help. I got it to work.

10:33 borkdude: :-)

10:46 TimMc: cemerick: If read is moved to unsafe-read, that's fine by me.

10:47 Anyway, it's not like Clojure hasn't made breaking changes before in second-segment version bumps...

10:47 shriphani: borkdude: I am trying to do this : is this incorrect ? (map (fn [s] (println s)) (line-seq rdr))

10:48 borkdude: shriphani (map println …) would also work I think

10:48 shriphani: This happens : Exception in thread "main" java.io.IOException: Stream closed

10:49 borkdude: shriphani ah, this means the stream already has been closed before consuming the entire lazy seq

10:49 shriphani so you would have to wrap it in side the (with-open ...)

10:49 shriphani: I have it in with-open

10:50 rplaca: shriphani: you probably want doseq rather than map there

10:50 shriphani: rplaca: would it be lazy then ?

10:50 borkdude: shriphani rplaca is right, because map is also lazy and if you consume this result outside the with-open, it also would be closed before read

10:50 rplaca: no, but if you have side effets, you probably don't want laziness

10:51 *effects

10:51 shriphani: borkdude: but the map is inside the with-open

10:51 rplaca: shriphani: but the map is lazy

10:51 borkdude: shriphani yes, but the result is probably not consumed, and map is lazy

10:51 rplaca: so it's not executed until later

10:52 shriphani: so i should use doseq ?

10:52 rplaca: shriphani: if you want to print lines from a stream, yes

10:53 it guarantees that all the work is done by the end of the call *and* it's more idiomatic in this case

10:53 shriphani: rplaca: now it wants a vector. line-seq isn't working.

10:55 vijaykiran: shriphani: what wants a vector ?

10:56 shriphani: Caused by: java.lang.IllegalArgumentException: doseq requires a vector for its binding in seedscore.core:27

10:56 borkdude: shriphani (doseq [l (line-seq (clojure.java.io/reader "/tmp/Foo.java"))] (println l))

10:57 shriphani or: (dorun (map println (line-seq (clojure.java.io/reader "/tmp/Foo.java"))))

10:57 rplaca: borkdude: beat me to it. My nine year-old came in to ask me about nodejs :)

10:58 but I would claim the doseq version is clearer than the dorun version here. (Though it's a matter of preference)

10:58 borkdude: rplaca and no intermediate collection

11:01 shriphani: rplaca: racket has file->lines primitive I can use for something like this: (filter (lambda (s) (string-is-ok s)) (file->lines s)). Is there an as-clean way of doing this ?

11:01 sorry (file->lines filename) *

11:01 borkdude: rplaca maybe there should be a switch in 1.5: *lazy* true/false :P

11:01 nDuff: shriphani: there's a line-seq

11:01 rplaca: borkdude: ugh!

11:02 borkdude: omg, I take that back… and also I haven't said anything about significant whitespace

11:03 rplaca: shriphani: no, things are simpler in this case here because racket isn't lazy so you don't have to worry about the deferred evaluation

11:03 borkdude: shriphani you can get the lines into a vector of course

11:03 rplaca: shriphani: you can always force execution with doall in these cases and get a non-lazy seq

11:03 ,(doc doall)

11:03 clojurebot: "([coll] [n coll]); When lazy sequences are produced via functions that have side effects, any effects other than those needed to produce the first element in the seq do not occur until the seq is consumed. doall can be used to force any effects. Walks through the successive nexts of the seq, retains the head and returns it, thus causing the entire seq to reside in memory at one time."

11:04 nDuff: ...why is laziness a problem for shriphani here?

11:04 * nDuff feels like something was missed.

11:04 rplaca: nDuff: cause he's in a with-open block

11:05 nDuff: Ahhh; I missed that.

11:06 luxbock: could this function be simplified somewhat? I have a really hard time wrapping my head around what's going on

11:06 https://www.refheap.com/paste/9477

11:06 borkdude: shriphani (-> "/tmp/Foo.java" slurp clojure.string/split-lines)

11:06 luxbock: it's from here: https://github.com/abengoa/clj-gametheory/blob/master/src/gametheory/core.clj

11:07 I've been trying to fix this into a more readable format following the clojure style guide and trying to understand it at the same time

11:08 S11001001: luxbock: the style guide mentions indentation, right?

11:08 ah, I suppose

11:08 luxbock: yes, here's what I've done so far: https://www.refheap.com/paste/9478

11:08 it's not my code originally

11:09 borkdude: shriphani filter all lines from Foo.java for which count is more than 1: https://www.refheap.com/paste/9479

11:09 S11001001: luxbock: well first, (map f (map g x)) = (map (comp f g) x)

11:09 luxbock: I can follow the rest of the code alright but that one function makes my head hurt

11:10 S11001001: luxbock: this is called fusion, and you can apply it to combine the 3 outermost maps into 1

11:11 luxbock: yeah that helps a little bit

11:11 S11001001: luxbock: also, comp is a variadic associative operator, so (comp (comp f g) h) = (comp f (comp g h)) = (comp f g h)

11:11 then you should have (map (comp ??? ??? ???) all-strategies), and you should be able to fuse those 3 functions into a single function relatively easily

11:12 luxbock: yeah that sounds like a good excercise

11:13 S11001001: I do recommend doing it mechanically like this, so you can see the transformations

11:13 borkdude: hmm, makes me wonder, is there such a thing as higher order slurp which takes a filter fn ;)

11:13 S11001001: e.g., because you're working purely, you can then do SICP chapter 0 style argument substitution to actually do the fusion

11:15 luxbock: I'm learning Clojure without much of a background in any other languages (besides for a little bit of Python)

11:15 borkdude: luxbock nice choice :)

11:15 pimeys: it's then much easier, I'd say :D

11:15 S11001001: yeah

11:15 luxbock: would it be a good idea for me to go through SICP on my own using Scheme or Clojure to get a hang of things?

11:15 pimeys: yes

11:15 it's teaching from very scratch

11:16 and the exersizes are easily converted to clojure

11:16 although some of the stuff seems irrelevant for clojure, but still it's a good read

11:16 S11001001: luxbock: I'd at least do chapter 0, but no need to bog yourself down so much that you lose motivation for working on what you really want to work on.

11:16 luxbock: can I combine it with learning Clojure at the same time, or should I just stick to the original format and use Scheme? I know there's that one SICP in Clojure project but it's not quite ready yet

11:16 pimeys: chapter 4 is also good

11:16 luxbock: I'm learning clojure with sicp currently

11:17 so it's doable

11:17 luxbock: ah cool

11:17 pimeys: but some stuff, like the end of chapter 2 seems a bit outdated for clojure

11:18 and when they're talking about tail recursion, instead of calling the iterative function recursively, use the loop...recur construct

11:18 TimMc: cemerick: The idea is that unsafe-read wouldn't do binding, and could be used in libs, but that read would bind *read-eval* to false before calling unsafe-read?

11:18 luxbock: I'm learning programming so I can learn about other stuff that's hard to think about without the mental tool box that programming provides you with

11:18 AWizzArd: Is there a good reason for this? ==>

11:18 ,(identical? 'and 'and)

11:18 clojurebot: false

11:18 cemerick: TimMc: That's my minor proposal, yes.

11:18 S11001001: AWizzArd: metadata

11:19 TimMc: AWizzArd: Symbols aren't interned.

11:19 luxbock: this one game theory library library I was able to find hits pretty close to what I want to be doing, but it's a lot of things to learn at once

11:19 AWizzArd: So both are fresh and different symbols.

11:19 While for keywords `identical?` works fine.

11:19 Oki, understood, thanks.

11:19 pimeys: luxbock: well it takes another 30 years to be a great programmer, so be patient :D

11:19 TimMc: Yep, keywords are interned.

11:20 rhickey: cemerick: which accomplishes nothing, since unqualified call to read are then not useful in libraries, and the I accidentally called unsafe read moves to I accidentally called this library which called unsafe read - how dare they!

11:20 AWizzArd: I think they are interned up to a certain limit. At some point older interned keywords get overwritten, so that there is no memory hole?

11:20 Bronsa: FWIW tools.reader's read now disallows read-eval by default

11:20 you have to use unsafe-read and unsafe-read-string explicitely

11:21 TimMc: rhickey: The library maintainer would have a responsibility to document their call to unsafe-read, yes.

11:21 rhickey: Bronsa: that's bad design

11:21 TimMc: again ,for what purpose? Basically there will be no libraries that read and compose

11:21 S11001001: AWizzArd: they're in soft refs so can get GCed; a cleanup running every n ticks drops empty refs. It would violate the semantics of keywords to drop keywords that are still reffed

11:22 TimMc: rhickey: That was ambiguous... You're saying that this would block composition, or that reading libs don't exist?

11:22 S11001001: AWizzArd: the java code in Keyword.java that implements this is pretty straightforward, short, and worth reading if you're interested.

11:22 you know, for java code

11:22 TimMc: rhickey: My understanding is that unsafe-read would not perform any binding.

11:22 more like "maybe-unsafe-read"

11:23 Bronsa: rhickey: I'm sorry but if find this behaviour saner

11:23 rhickey: You can't put a hardwired choice at the bottom of a library

11:23 TimMc: It wouldn't be hardwired.

11:23 unsafe-read would respect the current binding

11:24 Bronsa: TimMc: that's what I'm doing in tools.reader. unsafe-read allows read-eval to be true or false

11:24 rhickey: TimMc: but who will use it in their library and risk being responsible for someone reading crap from the internets?

11:24 TimMc: I would. With a big ol' warning.

11:24 rhickey: looking forward to a world where everything is prefixed by unsafe-

11:25 Bronsa: ,unchecked-inc

11:25 clojurebot: #<core$unchecked_inc clojure.core$unchecked_inc@4633f685>

11:25 rhickey: because if not, then the hungarian notion safety system fails

11:26 notation

11:26 AWizzArd: S11001001: :)

11:28 rhickey: (yourlib/foo ...) is it safe?

11:28 If I have to read docs, I might as well read the docs for read

11:29 TimMc: True.

11:29 But it still depends on the dynamic scope anyhow.

11:29 rhickey: And then I would understand that the safety is up to me, not your library, and how to ensure it

11:29 TimMc: I can't tell the safety of that call in isolation.

11:30 rhickey: But if I take responsibility for myself, I am always explicit about safety

11:30 TimMc: I'd still have to document that mylib/foo calls read-string.

11:30 ANd the user would have to read it.

11:30 S11001001: TimMc: in core.typed you can add an alias for the Identity monad, Unsafe, which would check percolation of unsafety :)

11:31 rhickey: no, it's often obvious that a library is I/O oriented, and one can presume uses read. In any case, one could explicitly setup the safe binding in self-defense

11:31 the point is, defaults accomplish nothing here

11:32 and to the extent they lull people into not being explicit, are doubly dangerous

11:32 pjstadig: rhickey: no one is arguing that defaults would save you from cases where someone explicitly established a binding

11:32 those are two separate circumstances that should be considered separately

11:33 even if the default changes, there's still a need to program defensively, and educate people to do so themselves

11:33 rhickey: read is not and has never been safe. you can;t superimpose safety on old code that calls read by changing what it means

11:33 * ohpauleez is tempted to just make a core.reader.edn contrib, which solves a large portion of this concern

11:33 rhickey: and, that old code might use binding, which would be disabled

11:33 headshot: ah, rhickey. thanks for clojure

11:34 </end-gushing-fan>

11:34 rhickey: ohpauleez: at which point you will prove my point, a safe reader is explicitly so, with no options for otherwise

11:34 like safe-read

11:34 pjstadig: rhickey: isn't it the case that the changes on master also break code?

11:34 if someone is doing something with #= that isn't in the whitelist

11:34 borkdude: wouldn't nested bindings also cause danger?

11:34 rhickey: headshot: you're welcome

11:35 borkdude: for example, a library designer could always do (binding [*foo* false] ….) no matter what you bind *foo* to?

11:35 ohpauleez: agreed - if that's what we want, and that's what people want (it's sure what I want in my code), let's just make the most direct solution

11:35 pjstadig: borkdude: yes, but that is orthogonal to what is the default value for *read-eval*

11:35 ohpauleez: then say, "if you're using it for just data, grab the contrib"

11:35 rhickey: borkdude: but that would be a bad library, a library could fire missles too

11:36 borkdude: or drones, so I heared

11:36 rhickey: ohpauleez: I don't disagree, the point is people are looking to break read without getting the benefit they seek

11:37 I'd be happy to have an edn reader

11:37 pjstadig: someone can always bind *read-eval* whether it defaults to true, false, or :default, but that is an entirely separate issue as "what should the default binding be"

11:37 rhickey: reusing the reader for interop is an act of extreme convenience

11:37 pjstadig: and the default ensure what for you?

11:37 pjstadig: the change of the root value of *read-eval* to :default will still break code

11:38 so the question isn't whether to break code, but how much code to break

11:38 rhickey: pjstadig: It might, and an option is to go back to the black/white world, but if we do, the default will still be unsafe, as it must be for all the reasons I've given

11:39 pjstadig: no, the non-breaking option is to add safe-read and docs

11:40 pjstadig: rhickey: in my mind having a 'safe' default doesn't ensure anything, it's just a backstop

11:40 i don't disagree with you

11:40 it would not be a be-all-end-all solution

11:40 rhickey: So, if you were a fan, would you sit behind a backstop with gaping holes?

11:41 I wouldn't

11:41 pjstadig: what gaping holes?

11:41 rhickey: But the semblance of a backstop, and people blogging about it now being a safe default might very well cause people to not be explicit

11:41 pjstadig: binding

11:42 pjstadig: rhickey: as soon as you use binding, you've stepped into a different situation

11:42 the changes on master also have "gaping holes"

11:42 someone could bind

11:42 rhickey: The [roblem is when someone else uses binding around you

11:42 pjstadig: and that problem has nothing to do with the default

11:42 i see your argument that perhaps having a 'safe' default lulls people into thinking they are safe

11:43 rhickey: pjstadig: yes it does, if the default is unsafe I never rely upon it for safety

11:43 pjstadig: but i don't think the solution there is to throw them to the lions

11:43 and *read-eval* as :default is still a "lulling" situation

11:43 someone could still bind

11:43 rhickey: bind arounf safe-read? so what?

11:43 pjstadig: no

11:43 read

11:43 safe-read may be a good addition either way

11:44 but having *read-eval* as :default for read is just as much lulling people as having it false

11:44 and the problems with composing bindings still exist

11:44 rhickey: look, there is not going to be a safe read given whatever default, of a function that has dynamic control, that is not explicit. At the point it is explicit it doesn't care what the default nor surrounding bindings are

11:45 pjstadig: as I said, it could go back to full eval by default

11:45 pjstadig: full eval with safe-read?

11:46 rhickey: old code, add safe-read, docs, make record reading subject to *read-eval*

11:46 pjstadig: what i mean is read would to full eval, but we would have a new safe-read function

11:46 right

11:47 yeah, there would be no expectation of safety there, sure

11:47 but the composition problems still exist

11:47 rhickey: what composition problems?

11:47 pjstadig: someone could still decide to use safe-read for me and i wouldn't be able to control it

11:47 or they could bind below me

11:47 rhickey: library code calls read, application code ensures safety

11:47 pjstadig: (in the case of using read)

11:48 ohpauleez: Personally, I really like the changes in master + an EDN reader contrib. I think that covers all the bases

11:49 rhickey: pjstadig: they can't bind around a code path that's your responsibility below you

11:50 pjstadig: if i call someone who binds *read-eval* to true, i cannot effect that

11:50 ohpauleez: cemerick: Want to jump in on this real quick?

11:50 pjstadig: if they decide to use safe-read, but i want read with *read-eval* true, then I cannot effect that

11:50 rhickey: pjstadig: If they are doing that they own that, they could launch missles too

11:50 borkdude: backward-bind: the most outside binder wins :P

11:50 pjstadig: but as i said those are orthogonal to defaults

11:51 those situations come about no matter what the default is

11:51 rhickey: pjstadig: you are wrong about that, if read's defaults are characterized as safe

11:51 cemerick: ohpauleez: apparently I missed the action, was off writing an email :-P

11:52 pjstadig: rhickey: yeah, i mean that's fine if that's the tack you want to take

11:52 rhickey: It's not a tack, it's basic privilege narrowing

11:52 pjstadig: rhickey: and if that's the tack you want to take then you probably don't want what's on master

11:52 clgv: rhickey: I like the whitelitst idea. but one little question why bind *read-val* to :default instead of :whitelist?

11:53 rhickey: pjstadig: it depends on how it is sold - I still characterize it as unsafe

11:53 and thus safe-read

11:53 pjstadig: in my mind it is as much about establishing a safe default as setting *read-eval* to false

11:53 and it still breaks code

11:53 possibly less, but it will

11:54 rhickey: but if we are worried about code written with complete lack of awareness, perhaps it reduces the risk footprint

11:54 not changing read is least disruptive

11:55 pjstadig: there is not such thing as a safe, dynamically rebindable, default

11:55 no

11:55 pjstadig: rhickey: but when you talk about binding you have stepped outside of default

11:56 someone can always bind

11:56 what should the root value be

11:56 rhickey: default means - if I do nothing else explicit, this is what I get. A default value of a dynamic var doesn't give you a default semantic for read

11:56 pjstadig: iff someone binds

11:56 rhickey: pjstadig: the point is the root value doesn't convey any semantic, better to choose one that doesn't make people complacent

11:56 aroemers: pjstadig: breaking a little code for a reduced risk isn't that bad, is it?

11:57 pjstadig: rhickey: i understand that is what you are saying, but i don't think everyone sees it that way

11:57 rhickey: and i don't think you want what is on master in that case

11:58 aroemers: it depends on what the goal is

11:58 TimMc: rhickey: I think I see what you mean about read vs. unsafe-read -- it might connote that read is always-safe.

12:01 pjstadig: TimMc: but if safe-read gets renamed to read and always binds *read-eval* to false, then it would be always-safe, no?

12:01 hyPiRion: One way of solving this "*read-eval* should be true/false"-discussion would be to make *read-eval* unbound by default. It will break everything, though.

12:01 aroemers: pjstadig: well, I think a lot of Clojurians are not aware of the read-eval binding. As Clojure gains more popularity, I think it is good to have some risk reduction, while indeed not implying that read is defaults to safe (as rhickey states).

12:01 egghead: all this talk about safe reading... no one ever told me reading was dangerous!

12:02 pjstadig: aroemers: yeah, well i'm in favor of having *read-eval* default to false, which would break code for reduced risk

12:02 rhickey: egghead: reading from the internet is dangerous, any developer that doesn't know that should be fired

12:02 egghead: :)

12:02 pjstadig: haha

12:02 hyPiRion: oh TimMc, did you see that I finished the quicksort?

12:02 technomancy: ring used to call read-string on session values until very recently

12:03 cemerick: ohpauleez: sorry, what were you asking about? safe-read + a canonical non-side-effecting reader impl?

12:03 ohpauleez: yes

12:04 cemerick: well, that was the email I was writing earlier, on the ML now

12:04 ohpauleez: ahh

12:04 I'll read now

12:04 TimMc: pjstadig: Oh right, never mind.

12:05 aroemers: hyPiRion: how feasable is this idea of having an unbound *read-eval*? How much would it break?

12:05 cemerick: I don't think anyone has said why #= is so critical to maintain. rhickey is right that dynamic scope ensure that the default cannot be relied upon as an absolute, but I disagree that it's effectively meaningless.

12:06 pjstadig: yeah it's a good question, some cases of #= can be covered with reader literals

12:06 records have their own syntax now

12:06 matthavener: even with *read-eval* false, is read-string even safe from other reader attacks? what would be the uses of a safe-read?

12:06 hyPiRion: aroemers: Everyone who doesn't bind *read-eval* and attempts to use read-* will end up getting a "*read-eval* is unbound" error.

12:07 If *read-eval* is unbound by default, that is.

12:07 aroemers: hyPiRion: I understand that, and I actually like that idea, but I cannot foresee how much it breaks. Maybe you have a better view on it.

12:07 cemerick: egghead: thank you for representing the vast majority :-)

12:07 re: "all this talk about safe reading... no one ever told me reading was dangerous!"

12:08 rhickey: Clojure's code-as-data extends to Java data - classes, Java collections etc. All of that can be intermixed with Clojure data but not all can be represented as Clojure data

12:08 reconstructing it requires calling ctors, static methods etc

12:09 it is unlikely that will ever be replaced by tagged literals except via a tagged literal as dangerous as #=

12:09 hyPiRion: aroemers: Well, it will most likely be an easy fix by binding *read-eval* at a program's entry point. However, that's without threads. As far as I know, binding is only thread-local, so a new binding has to be set up for each thread (or one must use boundfn if one wants *read-eval* to be passed on to that thread)

12:10 rhickey: at least, as dangerous as the #= default in patch

12:10 TimMc: rhickey: The vast majority of the time, full serialization is completely unnecessary.

12:11 rhickey: Here's a thought experiment - imagine we have a startup option that sets the default value of *read-eval* - how would that change your code or your recommendations to others on the shape of their code?

12:11 hyPiRion: I think the main issue here is that people aren't aware of *read-eval* and what it does. People should know that it exists and what may potentially happen if they don't bind it.

12:11 TimMc: rhickey: alter-var-root?

12:12 Oh, an option, I see.

12:12 rhickey: IMO, you should code as if set to the worst, regardless of the default

12:12 hyPiRion: TimMc: More like clojure --read-eval=false or something, I think.

12:12 rhickey: the point is not the option, it's that the option doesn't change good code

12:12 aroemers: hyPiRion: so the fix is indeed not that complicated. But would many projects/libraries need such a fix? (I have no idea where and how much read is used)

12:12 rhickey: and doesn't fix bad code

12:13 pjstadig: it would possibly break bad code if --read-eval=false

12:13 does that code deserve to be broken?

12:13 rhickey: what bad code?

12:13 cemerick: rhickey: except, AFAICT, that capability is unused, undocumented (rightly so IMO), and necessarily makes Clojure a conduit for exploits.

12:14 pjstadig: code that doesn't explicitly set a binding before calling read?

12:14 hyPiRion: aroemers: For libraries, you'd only have to bind for testing (and potentially replace some `fn`s with boundfn) the code. Only applications should have "an issue" with it.

12:14 rhickey: pjstadig: and reads what?

12:14 hyPiRion: Though, that being said, I suspect that the world is a bit more complicated than that.

12:14 pjstadig: rhickey: user generated data

12:14 rhickey: pjstadig: if edn, will be silent

12:15 pjstadig: rhickey: what are you thinking of when you use the term "bad" code?

12:15 rhickey: computers can't smell user generated data

12:15 pjstadig: um, i wasn't saying they would?

12:15 pimeys: I smell the ruby's YAML hell when seeing this :)

12:15 technomancy: pimeys: heh, that's exactly why we're having this conversation

12:15 rhickey: bad code is code that needs to protect itself and doesn't setup safe read

12:16 unsafe readers need do nothing

12:16 i.e. readers of trusted sources

12:16 pjstadig: right, so good code is unaffected by the default value for *read-eval*

12:16 technomancy: pimeys: the rubygems.org exploit happened because people didn't understand that YAML.parse was a dangerous operation

12:16 rhickey: right, and good code is necessarily explicit

12:16 pimeys: I know

12:16 pjstadig: but "bad" code could be broken if you set *read-eval* to false by default

12:17 pimeys: we've been hot-fixing like hell in my workplace recently

12:17 rhickey: pjstadig: and I contend good code is less likely to exist

12:17 pjstadig: rhickey: possibly, but in the end if there's a high profile "rubygems like" incident it will reflect on the whole clojure community

12:18 technomancy: pimeys: fun, innit

12:18 rhickey: pjstadig: no, bad code (fails to set anything) won;t be exposed until bad things are sent to it

12:18 pimeys: sad, I think

12:18 rhickey: pjstadig: It's fine to be afraid of that, but important to solve it, not just make gestures

12:18 pimeys: but will happen everywhere if people are not careful

12:18 pjstadig: rhickey: but when something bad is sent it will throw an exception instead of exposing some security risk

12:19 TimMc: rhickey: A library that just calls read, is that bad code? And a caller of that library that doesn't bind, is that bad code?

12:19 rhickey: we have a different scenario than python/ruby with the dynamic bindings

12:19 pjstadig: only if not wrapped in another context - I contend the default does nothing

12:19 pjstadig: i tend to fall on the side of "good code" is unaffected, "bad code" deserves to be broken because if it's not then people will cast a bad light on the community as a whole

12:19 rhickey: pjstadig: bad code will be undetected

12:20 cemerick: should sql query parameters be explicitly escaped?

12:20 pjstadig: rhickey: it may be a different scenario, but i think anyone analysing an incident would deride *read-eval* defaulting to true

12:20 rhickey: there's no substitute for taking responsibility for yourself

12:21 pjstadig: and when it was false and still happened?

12:21 pjstadig: rhickey: they we say, "we had a safe default and you changed it"

12:21 it's no substitude for personal responsibility; agreed

12:21 rhickey: pjstadig: like you told me I had to in order to do X, Y, and Z

12:22 hyPiRion: cemerick: SQL libraries should provide you with a sane default and explicitly tell you that default.

12:23 aroemers: bad code would indeed go undetected, until something happens to it, in that case the results of that bad code would be less severe, which is good for the Clojure community IMHO.

12:23 pjstadig: aroemers: right

12:24 rhickey: you are just playing the odds, not really safe

12:25 aroemers: Still, I think rhickey is right to, in the sense that read should still be documented as being unsafe, and a safe-read in that case is very welcome.

12:25 *too

12:26 craigbro: well

12:26 even a "safe-read" that allows macro characters has issues if I can get data to disk

12:26 technomancy: craigbro: if you can get it to the classpath, you mean?

12:26 pjstadig: rhickey: having *read-eval* default to :default is still playing the odds by reducing the chances someone wants to tweak the value of *read-eval*

12:26 TimMc: A default of [*read-eval* false] fails-safe, a default of true fails-deadly.

12:26 pjstadig: i'm not saying that's bad

12:27 rhickey: craigbro: as I said on the list, read is not going to be made secure near-term, whatever we do right now

12:27 pjstadig: it's hard to judge odds here

12:27 technomancy: craigbro: in which case you have plenty of more appealing attacks available to you than the reader

12:27 TimMc: You can get "failure" either way, but one of those probably won't cost you a month to fix.

12:27 craigbro: yah, but that's priv escalation

12:27 pjstadig: i don't think anyone is really analyzing frequencies

12:27 craigbro: rhickey: I agree, I bring it up as a point that the issue is more than just default binding

12:28 hyPiRion: pjstadig: ##(frequencies (repeatedly 345 (partial rand-nth [:for :against])))

12:28 lazybot: ⇒ {:for 160, :against 185}

12:28 ruff_: hello my friends i first with #clojure how to runing #clojure on #node.js server no lag?? sorry my english

12:29 rhickey: craigbro: sure, someone could probably OOM you just be feeding indefinitely much data inside a collection

12:29 craigbro: of feeding a anon fn to the right values

12:29 s/of/or

12:29 hyPiRion: or a reified object, for that matter.

12:30 craigbro: hyPiRion: hmm, not clear on how that would happen

12:30 or I can leak values using @deref

12:30 cemerick: hyPiRion: My broader point was that saying "you need to be explicit" is a poor option. Even the brightest, most careful among us make mistakes; missing a checkbox to opt into not being exploited is inevitable.

12:31 Frozenlock: As a bad person, I often try functions I don't really understand until 'it works'. If I can use 'read-string' and it works, I probably won't think of checking anything else.

12:31 rhickey: cemerick: making the mistake of not calling safe-read on internet-sourced data is one for firing

12:31 jkkramer: to write good (explicit) code, one has to actually know that reading is unsafe. newcomers generally don't know that

12:31 hyPiRion: craigbro: ##(= (reify Object (equals [this _] (println "alert!"))) :foo)

12:31 lazybot: java.lang.NullPointerException

12:32 rhickey: this whole culture of the lang/lib/service/whatever is going to take care of me is gross

12:32 cemerick: rhickey: not "take care of me"; "don't go implicitly executing code"

12:32 rhickey: jkkramer: agreed, no argument on docs needing to become much more explicit there

12:33 cemerick: it's what the reader has always done, it is not fundamentally a tool for reading the internet

12:33 enquora: has anyone seen or attempted a clojure pdf generation library?

12:33 thickey: it's very interesting problem. the one's at risk are always the unaware. you are only safe if you are explicit.

12:33 matthavener: cemerick: i agree, even with *read-eval* false, read-string still doesn't "take care of you" (OOM/DoS type attacks are still possible)

12:34 rhickey: people are using it for that because no one has taken the time to write the tiny bit of code needed for an edn reader. Everyone wants reader for free

12:34 wangtd: //names

12:34 cemerick: rhickey: The history is what it is; that doesn't meant that it needs to or should remain

12:35 rhickey: If there was an edn reader right now in Clojure, who would advocate using read for internet data?

12:35 It wouldn't be called core/read

12:35 it would have to be a different call (like safe-read, erm edn-read)

12:35 cemerick: I think people should be forgiven for using the Clojure reader to read Clojure data.

12:36 craigbro: cemerick: no way

12:36 technomancy: https://github.com/ring-clojure/ring/commit/7028d12759ababdcd

12:37 dnolen: cemerick: hmm I don't know, people used to abuse eval in JS for JSON parsing. This spawned proper JSON parsers - now no one uses eval for JSON parsing.

12:37 jkkramer: enquora: there's http://www.clojuresphere.com/clj-pdf/clj-pdf. I ended up shelling out wkhtmltopdf, which provided much better control/flexibility

12:37 craigbro: cemerick: I'm not saying we be vindictive, but since the days of CL web dev and before, lisps do not use read to get data

12:37 technomancy: any guesses as to how long that affected every cookie-store-using Clojure web application?

12:37 enquora: jkkramer: that's based on iText, afaik, which is a non-starter

12:37 cemerick: dnolen: except no one is using clojure.core/eval to parse clojure data that they wrap in a (quote ...) form.

12:37 Frozenlock: technomancy: 24 days ago? Shi...

12:37 `fogus: technomancy: since the beginning of time?

12:38 rhickey: cemerick: there's no substitute for a hardened interop-oriented read - Clojure's reader isn't that, and changing a default won't make it that

12:38 dnolen: cemerick: read-string is a bad as eval is my point

12:38 * Frozenlock updates his servers

12:38 dnolen: cemerick: people should use EDN reaaders

12:38 cemerick: dnolen: indeed, read-string is as bad as eval now; what I've been driving at is, it doesn't have to be.

12:38 rhickey: Clojure's reader is meant to server the language, and devs looking to fully exploit their environment

12:39 hugod: I like the idea of a separate read that is unaffected by dynamic vars

12:39 rhickey: cemerick: just use safe-read, wtf

12:39 cemerick: indeed

12:39 enquora: jkkramer: that's exactly what we do now. our needs push html layout beyond its limits though :-(

12:39 ohpauleez: cemerick: I'm with dnolen on this - tempted to cook one up in the spirit/API of clojure.data.json

12:39 dnolen: cemerick: agree w/ rhickey here. read language support is being hijacked out of convenience.

12:40 rhickey: I don't see changing defaults as being better than calling safe-read and advocating to everyone else to do the same with untrusted data

12:40 hyPiRion: Okay, so the issue here is that we don't have a proper `read-edn` function yet?

12:40 rhickey: safe-read

12:40 safe-read is a slight superset of edn

12:40 enquora: need something that is optimized for technical report generation. Am reconciled to creating something from scratch, but hoping to find something existing as a starting point that at least provides font-metrics and pdf primitives

12:40 rhickey: allowing records, but disallowing JAva

12:41 hiredman: enquora: your best bet is to find a java library for that

12:41 enquora: haven't had much luck there, either

12:42 hiredman: ~google java reporting library

12:42 clojurebot: First, out of 920000 results is:

12:42 JasperReports Library | Jaspersoft Community

12:42 http://community.jaspersoft.com/project/jasperreports-library

12:42 hiredman: seems like a solid google hit

12:42 rhickey: how about we call safe-read edn-read, and consider it slightly but harmlessly broken, to be made conformant by 1.6?

12:42 enquora: hiredman: by which I mean something that is appropriate and I'd like to use ;-)

12:43 `fogus: ohpauleez: The CLJS reader is a good first approximation. Needs work to get the whole way.

12:43 craigbro: my vote is for seperate edn reader, designed from start to be safe and deal with public data

12:43 enquora: hiredman: layout concepts nowhere near what we need

12:43 TimMc: rhickey: How about moving eval and read into clojure.compiler or whatever, since they aren't for general purpose use?

12:43 craigbro: I think they still are for general purpose use

12:43 rhickey: and everyone who thinks this is important direct their energies off the list and on patches for it?

12:43 rplaca: Jasper is a crazy complicated system. It used to belong to Sun/MySQL, but I don't know what they're up to now

12:43 hiredman: `fogus, ohpauleez: I am surprised neigher of you mention the recently contribized blind

12:43 craigbro: there are tons of times when I am readingwriting dat that I control

12:44 nightfly: Eval and read not for general use? Is this not a lisp?

12:44 TimMc: Well, that's the message I'm getting.

12:44 rhickey: TimMc: I don't see the point, given better documentation and an alternative edn-safe-read mentioned in same

12:44 `fogus: hiredman: I thought that was intended to provide the same capability as the Reader code.

12:44 enquora: we need to produce *very* dense technical/engineering reports with extremely tight typographic control. unfortunately, there isn't much out there that meets this goal.

12:45 rhickey: All we're doing by futzing with read is breaking things and delivering nothing

12:45 hiredman: enquora: generate latex

12:45 rhickey: but making ourselves feel better

12:46 `fogus: hiredman: But yes, its code could be used as a basis for a true CLJ EDN reader

12:46 rhickey: craigbro: I believe most serious use of read is dominated by controlled code and sources, outside of webapps

12:46 ivaraasen: hyPiRion: so I think I finally managed to decouple the basic functionality of array-utils from doubles. only slightly slower, which is nice.

12:46 craigbro: TimMc: general use does not include reading tainted strings 8^)

12:46 enquora: hiredman: have done that in the past. it's a bit like using regex to parse html - in this case. ConTeXt is at least a bit better.

12:46 hyPiRion: I would support changing the name of safe-read, as it doesn't convery why it's "safer" than normal read.

12:46 ivaraasen: sweet

12:47 hiredman: oh god, yeah, imagine a reader without syntax quote in it

12:47 hyPiRion: call it edn-read or read-edn (plus records), as that's what it really does.

12:47 enquora: hiredman: need to generate these in javascript in the browser, too. am thinking about something that can be compiled using clojurescript

12:47 rboyd: +1 vote for renaming read to hackme-read

12:47 Frozenlock: As a side question: Isn't read and eval two seperate functions usually? (R.E.P.L.)

12:48 ohpauleez: hiredman: `fogus - yeah cljs.reader or the new clojure reader are both solid candidates.

12:48 hyPiRion: Not that I really care much about it, but edn-read converys what data format we read -- safe-read doesn't.

12:49 jonasen: +1 for edn-read. And leave the current read as is (with a pointer to edn-read in its docstring)

12:50 craigbro: https://github.com/search?l=Clojure&q=read-string&ref=advsearch&type=Codehttps://github.com/search?l=Clojure&q=read-string&ref=advsearch&type=Code

12:50 doh, sorry, repeated url

12:50 anyways, there ya go bois, get to work patching all those 8^)

12:51 hyPiRion: Hmm.

12:52 jkkramer: edn-read is also probably unused in the wild. there are codebases that already have a safe-read

12:52 cemerick: dnolen, ohpauleez: a separate standard reader library is certainly welcome. FWIW, I was trying to eliminate a source of inevitable error, not maximize convenience.

12:53 aroemers: Would edn-read or safe-read then be the same in functionality as read with *read-eval* to false?

12:54 hiredman: ,(- 28 16)

12:54 clojurebot: 12

12:54 ohpauleez: craigbro: https://github.com/search?l=Clojure&q=safe-read&ref=advsearch&type=Code

12:55 fwiw

12:55 dnolen: cemerick: communicate to not read/read-string (docstring, IRC, ML, Twitter). add a kibit rule etc.

12:55 I honestly don't understand why the huff now after 5 years of unsafe read

12:56 ToBeReplaced: i'm a bit lost... shouldn't it always be opt-in-for-danger? if you just create safe-read, someone will read somewhere... i liked the idea of moving eval, read, etc. to a separate namespace.

12:56 rhickey: dnolen: agreed

12:56 technomancy: dnolen: because three high-profile vulnerabilities caused by the same problem just surfaced last month

12:56 craigbro: dnolen: HN echo of rails catastrophe

12:56 matthavener: dnolen: i think people (include myself) assumed that read-string couldn't eval... but imho a lack of understanding on my part doesn't require a change on clojure's part

12:56 TimMc: dnolen: Apathy.

12:57 ibdknox: matthavener: it does actually, when that lack of understanding spans the community

12:57 rhickey: technomancy: that doesn't mean the answer is change read, the answer could be - use something else

12:57 Frozenlock: matthavener: That was my initial insight. There's a `read' and 'eval' function.

12:57 craigbro: ohpauleez: awesome, I did not mean to cast aspersions on clojure coders with that url, just thought github search was kewl and an easy way to track this stuff and talk about how clojure code gets written in the wild

12:58 ToBeReplaced: rhickey: would you say that the answer is "make sure everyone is taught that there is an issue here" or "make sure that no one hits the worst-of-the-issues unless they explicit say it's okay"

12:58 matthavener: ibdknox: but is our goal to please the community or make clojure easy for the community? or to build a good/simple lang?

12:58 `fogus: Where oh where is clojure.tools.edn?

12:58 technomancy: I don't think it would be productive to get involved in this conversation further, but that's why it's being discussed now.

12:58 ibdknox: matthavener: there is no good/simple lang without a community :)

12:58 cemerick: dnolen: what everyone said above, plus I was around leaving cemerick-shaped holes in walls, per usual.

12:58 matthavener: ibdknox: agreed, there's a balance there somewhere :P

12:59 rhickey: thanks everyone for your input, I'll be back later with an answer I hope :)

12:59 ibdknox: I'm amazed this is even contentious, to be honest

12:59 cemerick: ibdknox: oh?

12:59 technomancy: "YAML.load didn't look dangerous to me" <- sound familiar?

13:00 ibdknox: cemerick: any large organization having found an insecure default?

13:00 know what we did at MSFT?

13:00 lol

13:00 TimMc: You laughed out loud, is what?

13:00 ibdknox: or we dropped a few million dollars to fix it in the next couple days

13:00 Bronsa: `fogus:, dnolen would a tools.reader.edn subnamespace be more acceptable than defaulting to disabling read-eval?

13:01 dnolen: Bronsa: yes that's what I would suggest

13:01 aroemers: Rich shouldn't 'wft' dear Chas next time he comes for input though! ;)

13:01 ohpauleez: Bronsa: dnolen I'd rather see data.edn

13:01 jonasen: `fogus: shouldn't that be clojure.data.edn?

13:01 ohpauleez: to match data.json

13:01 since it's a data language

13:02 dnolen: ohpauleez: no argument there.

13:02 aroemers: *wft, that is

13:02 ohpauleez: I'm with you jonasen

13:02 aroemers: **wtf...!

13:02 ibdknox: if we took a straw poll on how many people know about #= I imagine it's only the obvious people

13:02 jonasen: ohpauleez: heh

13:02 Bronsa: right, I would include it in tools.reader only to avoid code duplication :)

13:02 ohpauleez: Bronsa: are you cooking this up right now? I was just going to use the CLJS reader, since it's nearly purely EDN as is

13:03 Bronsa: ohpauleez: tools.reader is already feature-complete

13:03 `fogus: jonasen: Maybe. But data.edn is just Clojure. ;-)

13:03 technomancy: the funny thing is the YAML vulnerability was actually much less dangerous than the reader since it only exposed a single []= method; it had to be paired with a class that called eval inside such a method to be exploitable

13:03 Frozenlock: ibdknox: I've seen #= more than 10 times since yesterday and I still don't know what it does...

13:03 ohpauleez: Bronsa: totally, but you need to not pick up Records, etc

13:04 ibdknox: Frozenlock: it allows you to execute any arbitrary code at read time.

13:04 ohpauleez: no eval, no records, but data literals, reader literals, and datatypes

13:05 ibdknox: I'm still laughing at the term, "the obvious people"

13:05 ibdknox: :)

13:05 Bronsa: ohpauleez: I'm working on it

13:05 `fogus: Bronsa: Sweet!

13:05 ohpauleez: Bronsa: Solid! I will stop my work then!

13:05 TimMc: Frozenlock: (read-string "#=(eval (System/exit 0))")

13:05 ohpauleez: let me know how I can help

13:05 Frozenlock: TimMc: naughty

13:05 I might have tried that.

13:05 TimMc: Frozenlock: Long story short, use (binding [*read-eval* false] ...) around everything.

13:06 cemerick: technomancy: All I can think of now is a phrase from an email of yours early in one of the threads, "tribal knowledge"

13:06 technomancy: cemerick: honestly I don't know how anyone gets anything done in Clojure without hanging out in this channel anyway

13:06 sshack: How do you install a jar file into the local maven repo? I did this a few days ago but forgot the command.

13:06 ibdknox: haha

13:06 Frozenlock: ,(read-string "(+ 1 2)")

13:06 clojurebot: (+ 1 2)

13:06 ibdknox: technomancy: it is a wonder

13:06 Frozenlock: ,(read-string "#=(+ 1 2)")

13:06 clojurebot: #<RuntimeException java.lang.RuntimeException: EvalReader not allowed when *read-eval* is false.>

13:07 Frozenlock: So the point of #= is to screw us?

13:07 TimMc: Yes.

13:07 cemerick: lol

13:07 ohpauleez: haha

13:07 Frozenlock: I mean... I would just do (eval (read-string "(+ 1 2)")) if I wanted to eval code.

13:07 `fogus: technomancy: That's odd because I don't know how anyone gets any work done /while/ hanging out in this channel :p

13:07 TimMc: But since we're all mega-rockstar programmers who never screw up, it will never bite us in the butt.

13:07 ohpauleez: it's why it loosely resembles a...

13:07 technomancy: Frozenlock: only in undocumented ways, don't worry

13:07 rbxbx: cemerick the tribal knowledge situation in this community isn't nearly as bad as, for example, Rails.

13:07 cemerick: rbxbx: Oh, good, thanks for that. :-P

13:07 rbxbx: then again Clojure is younger, so we might end up there eventually :|

13:07 TimMc: technomancy: It's a good thing that black-hats never read the docs, am I right?

13:08 ohpauleez: haha

13:08 technomancy: Frozenlock: I get the feeling you're going to pull out the c-word

13:08 rbxbx: cemerick just saying, I feel like you can read a few books and blog posts and be a moderately effective Clojure developer. I don't see that in Rails at all.

13:08 ibdknox: Would anyone actually complain about making read non-side-effecting?

13:08 technomancy: (not that c-word, I mean complected)

13:09 rbxbx: (and yes, language vs framework, but meh, point stands ––– it's really community comparison, not tooling)

13:09 ohpauleez: technomancy: haha

13:09 cemerick: ibdknox: it is quite off the table

13:09 rbxbx: technomancy: I kept seeing c-word on the mailing list and thought that perhaps wasn't the best choice of phrasing.

13:09 ibdknox: cemerick: I read your response that said that, but I didn't catch why

13:10 Frozenlock: technomancy: I don't get it.. :(

13:10 ibdknox: Frozenlock: read currently does more than one thing

13:10 Frozenlock: like you said, you expected to use (read (eval ..)) if you needed such a thing

13:11 craigbro: reverse that 8)

13:11 Frozenlock: Indeed.

13:11 mattmoss: But can you put #= inside #= ?

13:11 Frozenlock: So clojure doesn't have a repl, it has a RPL.

13:12 ohpauleez: mattmoss: You're talking about the 5th dimension of eval? I've only heard the rumors

13:12 cemerick: ibdknox: I think they closest we'll come is @ http://clojure-log.n01se.net/#12:08

13:12 technomancy: Frozenlock: maybe more like a reepl

13:12 ibdknox: cemerick: I see

13:13 mattmoss: ohpauleez: C'mon... everyone knows the 5th dimension is love... or... wait: that's the 5th element.

13:14 craigbro: welll

13:15 cbp: any way for nrepl to terminate the jvm it started?

13:15 craigbro: clojure code is data, clojure data is code -- that mantra should at least clue you in that reading data as clojure is not something you do with any old input

13:15 cbp: so i don't end up with 10 jvm hogging 3 gb of ram at the end of the day? :p

13:15 cemerick: ibdknox: long before edn and tagged literals, there was a desire to be able to represent Java objects within Clojure data, e.g. #=(java.util.ArrayList. [1 2 3])

13:15 technomancy: craigbro: that mantra is true of other lisps, and they don't have this problem.

13:15 Frozenlock: cbp: (System/exit 0) might do it...

13:15 craigbro: technomancy: yes, they do

13:16 dnolen: technomancy: I would find that *very* suprising

13:16 cemerick: thus print-dup, etc.; apparently, #= existed already?, but I don't recall that prehistory

13:16 technomancy: craigbro: so... how would you exploit an Emacs instance via read-from-string?

13:16 ohpauleez: C-x C-x M-a C-t

13:17 see what I did there?

13:17 dnolen: technomancy: heh, was thinking about production Lisps, not text-editor lisps

13:18 `fogus: Do a Google search for "Red Team vs the Agents"

13:18 Frozenlock: Doesn't common lisp require an eval function?

13:18 technomancy: dnolen: package.el calls read-string on untrusted data, so it's still an issue

13:18 read-from-string; sorry

13:19 `fogus: http://www.websecuritywatch.com/cve-2012-3479/

13:19 technomancy: dnolen: I haven't used racket; is it a safe operation there?

13:20 `fogus: https://gist.github.com/fogus/4716440

13:20 ohpauleez: `fogus: Thank you

13:20 craigbro: technomancy: in elisp case, it appears that #N for circular obects is the only special reader macro

13:21 technomancy: so off the top of my head, only a DOS is possible, but I have not read emacs source

13:21 dnolen: technomancy: I don't know, but I wouldn't be surprised if there were exploitable corners given Racket's flexibility

13:21 craigbro: technomancy we want a read with reader macros, ability to read/output java or non-native objects etc..

13:22 technomancy: craigbro: who is "we"?

13:22 craigbro: technomancy: so once you have thos things, read is prolly not what you want

13:23 technomancy: most people are happy with reader literals

13:23 craigbro: technomancy: err, the people who wrote clojure, those who campaigned for reader macros, those who want to be able to write out their clojure data, and read in their clojure data when it contains non-native objects etc...

13:23 technomancy: so... people who don't want to bother with reader literals for some reason?

13:24 craigbro: technomancy: when I said reader macro above, I was refereing to reader literals, sorry, I confused the terms because of the use of the word macro character on the clojure reference page that talks about them

13:27 TimMc: I think "reader literal" and "reader macro" are mostly the same thing.

13:28 craigbro: TimMc: well, with the caveat that reader macros ala CL can modify your read table on the fly, and do a whole lot more in terms of controlling input.

13:28 TimMc: *Clojure* reader macros

13:28 craigbro: reader literals just get prelexed forms

13:28 technomancy: yeah, in my mind "reader macros" refers to the much more flexible CL flavour, right

13:28 aroemers: AFAIK, reader literals, apart from some default ones, need to be made available to the system explicitly right?

13:28 `fogus: TimMc: Unless you hack it, the reader macros are not intended for extension

13:29 TimMc: There's some confusion here with the #tagged stuff.

13:29 craigbro: yah CL reader macros have access to the stream being read and all kinds of neat stuff

13:29 aroemers: If I am correct, then that is a lot safer than #=()

13:29 technomancy: aroemers: yes, it's basically always a better choice

13:30 craigbro: BTW, my support for safe-read or edn-read or whatever, is orthogonal to make *read-eval* default false. however, I also think you will not get read to be safe in the present of reader literals

13:30 Frozenlock: It could lead to some interesting expression. "Ah man, I got #=ed by this deal."

13:31 craigbro: I mean, looking at the rails sploits, it was a case of something similiar to reader literals, being abused because some complicated reader literal fns were hooked up by default

13:31 aroemers: craigbro: not even if reader literals are the _only_ way to go, as in #=() is not available?

13:32 craigbro: aroemers: "not even"? I don't undertand, restate.

13:32 devn: Is it impossible to do this in straight Clojure? https://github.com/danielribeiro/conjcraft/blob/master/java/mod_Conjcraft.java

13:33 aroemers: craigbro: I mean, wouldn't that make read a lot safer? If it does not support #=() at all?

13:33 devn: (because of the @Overrides)

13:34 technomancy: craigbro: I don't think it's fair to compare it to a bad reader literal; the problem was YAML.load itself

13:34 rodnaph: core.logic - if there is no :else on conde, is there an alternative?

13:35 gfredericks: does it sound plausible that `lein new` doesn't add `.lein-repl-history` to the .gitignore?

13:35 dnolen: rodnaph: you don't need :else in conde, every branch is tried anyway

13:35 proger79: How to do the same aliasing if instead of println I want to use MessageBox/Show in .NET?: (def out println) (out "hi!") . When using (def out MessageBox/Show) (out "hi!") the error is given: "Unable to find static field: Show in System.Windows.Forms.MessageBox".

13:35 technomancy: gfredericks: it does in 2.0.0, but it's a recent-ish addition

13:36 gfredericks: technomancy: upgraded! all better now.

13:37 craigbro: technomancy: I am refering to the XML containing YAML part of the rails exploit, not the YAML.load !class representation

13:37 technomancy: craigbro: oh, gotcha; sure

13:37 craigbro: aroemers: i think it would make it safer, and this it's a reasonable default, however it's not safe, nor is it solving the problem

13:37 technomancy: I was tracking the rubygems.org exploit more closely

13:38 craigbro: aroemers: I personally thing having it default to false would be good

13:38 aroemers: however, I would also never use read-string even then to read tainted strings

13:39 the Rails issue is really not about defaults in the anguage

13:39 it's about defautls in the framework. that same set of decisions that led to it, would not be changed by read-eval being false

13:39 it's the decision to "hey, let's make this more flexible and powerful" without evaluating costs, or being explicit about what exactl you are doing

13:40 Frozenlock: craigbro: like a read that does eval at the same time? :)

13:40 callenbot: technomancy: just saw the seattle hacking locations post, cool stuff. How'd you come by that standing desk?

13:40 rodnaph: dnolen: https://gist.github.com/rodnaph/4716586 trying to implement flatten (got my hands on TRS now) - but the else part seems not to be firing correctly... it seems to be an extra clause is needed here to either say that s is a "single" or that it's not null or a pair... ?

13:41 (thanks too btw - you always seem to answer my silly questions)

13:42 technomancy: callenbot: it's just a basic ikea model; nothing fancy

13:42 craigbro: Frozenlock: as I said before, I think it's reasonable for read to be able to do tht, and to have reader literals (I want more, but I'l settle)

13:42 technomancy: I have room to keep a recliner by the standing desk, which means I don't need an adjustable model. I couldn't stay on my feet all day.

13:43 craigbro: Frozenlock: because the mantra code = data in a place where your data has to interact with non-native data, you need that extra step.

13:43 callenbot: technomancy: I've heard having a soft mat to stand on helps with standing desks.

13:43 technomancy: I have a carpet. it's my knees that get worn out. may have inherited week knees.

13:44 craigbro: technomancy: standing desk destroyed my knees for a few months too.

13:44 technomancy: with mat, with shoes, without etc...

13:44 `fogus: technomancy: Relevance has these extra padded floor mats that work wonders for the knees

13:44 craigbro: I did not get a fancy gel mat tho

13:44 * `fogus trying to find

13:44 technomancy: `fogus: huh; I assumed they were more to deal with pain in the feet

13:44 callenbot: technomancy: you need more than carpet.

13:44 technomancy: you really need thick, soft padding

13:45 craigbro: i have a floor desk, a standing desk and a recliner, and rotate thru them

13:45 Frozenlock: technomancy: if you have some money to spare: geekdesk.com. You can easily switch between standing/sitting and even on-your-knees.

13:45 `fogus: technomancy: It's all connected. ;-)

13:45 callenbot: technomancy: the knees are probably from the leverage caused by your height :)

13:45 craigbro: the floor desk I use a zafu, and sit zazen-style at

13:45 technomancy: Frozenlock: I have enough room in my lab that I don't need all those in a single desk =)

13:45 craigbro: technomancy: work from home?

13:45 technomancy: craigbro: home and around town =)

13:46 craigbro: work from bars

13:46 technomancy: https://secure.flickr.com/photos/technomancy/tags/laboratory and http://technomancy.us/156

13:46 heh

13:46 and https://secure.flickr.com/photos/technomancy/tags/remoteoffice/ in the summer

13:47 craigbro: x100e?

13:47 technomancy: craigbro: the thinkpad? it's an X200s

13:47 dnolen: rodnaph: I don't think you need that last clause anyhow - it's handled by the first one

13:48 Frozenlock: technomancy: https://secure.flickr.com/photos/technomancy/4397554484/ really? :P

13:48 dnolen: rodnaph: and that flatteno doesn't do what you think it does - it won't return a flattened list after 1 run.

13:48 technomancy: Frozenlock: don't knock it till you try it =D

13:48 Frozenlock: Ever dreamed of trying https://en.wikipedia.org/wiki/File:DataHand_Professional_II_Keyboard-Right.jpg?

13:48 technomancy: Frozenlock: I don't use it all the time, but it lets you keep your arms in a completely neutral position

13:49 rodnaph: dnolen: ok thanks - i shall take another look.

13:49 technomancy: Frozenlock: I would love to see that adapted for mobile use. I suspect it wouldn't be as big of a win vs a full keyboard.

13:50 rodnaph: dnolen: ah yes i see now! i was misguided. back to the drawing board, heh

13:53 craigbro: you know

13:53 I didn't realize edn basically has reader literals

13:54 so I defeintly support setting read-eval to false by default

13:54 hehe

13:55 of course, theissue of controlling the binding of that var still applies

13:55 hence the need for a edn-read or something, that will never ever eval

13:56 Frozenlock: I want a function named `read' that does that.

13:58 craigbro: Frozenlock: hehe 8^)

14:04 mpenet: technomancy: About the knee issue with standing desks, a pair of worn out running shoes does it for me (I have that $20 ikea standing desk setup).

14:05 ivaraasen: hyPiRion: just pushed the decoupled version. wanna have a look at it? wondering if I'm using the right abstractions for decoupling

14:24 rhickey: edn reader done - name game time - edn-read + edn-read-string, or what?

14:25 pjstadig: read-data?

14:25 unless you want edn in there

14:25 ohpauleez: but it's still a str

14:25 rhickey: read reads data too

14:25 rplaca: rhickey: I go for the original name: safe-read (+ -string)

14:25 craigbro: +1 edn-read end-read-string

14:25 rhickey: only reads edn subset

14:25 * rplaca shows up late to the party

14:25 rhickey: safe implies same as read but + safeness

14:25 ohpauleez: +1 edn-read end-read-string

14:25 mpenet: read-edn? matches read-string: read-<something>

14:25 ohpauleez: also

14:26 rhickey: mpenet: read-string-edn?

14:26 rplaca: rhickey: ahh, ok, vote withdrawn

14:26 rhickey: read-edn-string

14:26 pjstadig: read reads data too, but we shouldn't use it to read data

14:26 jeremyheiler: +1 for read-edn and read-edn-string

14:26 pjstadig: what is the use for read outside of the compiler

14:27 rhickey: pjstadig: stop being ridiculous - serious programs use read to read data from trusted sources all day long

14:27 rplaca: rhickey: then I would lead with edn: edn-read (+ -string)

14:27 craigbro: pjstadig: dumping a computation state out between trusted programs

14:27 pjstadig: wow ok

14:27 i didn't realize i was being ridiculous

14:27 craigbro: pjstadig: aka, any time I am moving data between trusted programs

14:27 mpenet: rhickey: read-edn-string sounds good. I prefer the shorter read-edn, but I understand the need to be more specific

14:27 rhickey: read can read data that includes java data structures

14:27 craigbro: pjstadig: join the club!

14:27 pjstadig: dnolen says read is being "hijacked" to read data

14:28 craigbro: pjstadig: I don't agree with that

14:28 read is for reading clojure data

14:28 rhickey: pjstadig: you are overstating things in a manner that suggests not being serious - it's quite obvious that read has a lot of utility - if that's not apparent to you, then just stay out of it

14:28 craigbro: you don't read clojure data from strangers, cause remmeber, data = code in clojure 8^)

14:28 pjstadig: rhickey: you're being entirely unkind

14:28 ohpauleez: I like leading with edn ala edn-read-string

14:28 pjstadig: i wasn't over stating i was asking a question

14:29 rhickey: "pjstadig: read reads data too, but we shouldn't use it to read data"

14:29 I'm tireds of that crap

14:29 pjstadig: should have been a question mark at the end of that

14:29 rhickey: I'm trying to work here

14:29 pjstadig: i'm trying to understand the landscape here

14:29 we're going to have read, safe-read, read-edn

14:30 rhickey: user=> (doc read)

14:30 -------------------------

14:30 clojure.core/read

14:30 ([] [stream] [stream eof-error? eof-value] [stream eof-error? eof-value recursive?])

14:30 Reads the next object from stream, which must be an instance of

14:30 java.io.PushbackReader or some derivee. stream defaults to the

14:30 current value of *in*.

14:30 Note that read can create arbitrary Java objects etc (controlled by *read-eval*),

14:30 and as such should be used only with trusted sources.

14:30 For data structure interop use edn-read

14:30 no safe-read

14:30 pjstadig: ok

14:30 rhickey: read and edn-read

14:30 craigbro: I think I may chang my vote to read-edn and read-edn-string

14:31 ohpauleez: cool

14:31 rhickey: our big problem was the overloading of read - the internet use case needs something that can be compromised

14:31 edn-read has no capabilities to do eval, java etc

14:31 ohpauleez: thanks rhickey, it's appreciated

14:31 rhickey: can't be compromised :)

14:32 craigbro: it is also explicit about the format, which includes reader literals, and the security implications of that

14:32 aka, control your *data-readers* map

14:32 rhickey: I like read-edn and read-edn-string, but switching code to the latter might be more of a hassle (says the guy who can't use editors and regexes)

14:32 craigbro: i like the read-edn because when I do tab completion it is shown 8^)

14:33 rhickey: craigbro: yes

14:33 craigbro: anyways, the fact that *data-readers* are there also rules out the "safe-" naming convention IMO

14:34 rhickey: craigbro: unless your program is perfect - nothing is safe

14:34 ohpauleez: +1

14:35 rhickey: but you need never add anything to data-readers

14:35 and we might be able to add control over what data readers are active for a read-edn - oh nooo...

14:35 craigbro: rhickey: read-json in that case is what I do

14:36 rhickey: you do throw out sets and a few other things when you do that

14:36 rhickey: control == responsibility

14:37 craigbro: rhickey: also, the ability to pass in the *data-reader* map to the read-edn* fns worth considering?

14:37 aka, not relying on the dynamic binding

14:37 a stretch...

14:37 rhickey: ok - seems like edn-read edn-read-string vs read-edn read-edn-string - vote?

14:38 jonasen: read-edn

14:38 rhickey: craigbro: could be, but read has that arity overload

14:38 TimMc: Keep the read-* prefix.

14:38 jeremyheiler: i vote for read-* because it groups them in documentation and easier for auto-complete

14:38 craigbro: read-*

14:38 jkkramer: +1 read-edn / read-edn-string

14:38 amalloy: i like edn-read. splitting read-string in the middle to produce read-edn-string looks confusing

14:38 TimMc: Newbies are more likely to find read-edn*

14:38 saolsen: edn-*

14:39 normanrichards: read-edn*

14:39 rhickey: amalloy: read-string-edn is possible but ugly

14:40 jkkramer: also, would be nice if you could pass read-edn a non-pushbackreader source

14:40 TimMc: This is all besides the default-bindings question, yes?

14:40 mpenet: verb first feels more natural, and is more common

14:40 amalloy: rhickey: no, i was in favor of the edn-read-string you suggested. just noting why i don't care for read-edn[-string]

14:40 terom: acronym edn is not very obvious but if there isn't a better name, read-edn sounds better to me

14:40 aroemers: read-edn (-string) for me

14:41 rhickey: TimMc: what default bindings question - for read?

14:41 TimMc: For *read-eval*.

14:41 Wrong moment to ask, probably.

14:41 rhickey: TimMc: in this plan nothing about read changes except doc strings

14:41 TimMc: :-/

14:41 rhickey: as above

14:42 TimMc: I can't interpret that emoticon here

14:42 ToBeReplaced: read-edn (-string) for me; verb the noun

14:42 Bronsa: rhickey: reverted safe-default changes to tools.reader, working on an implementation of edn-only reader, should it handle eg deref/quote or not?

14:43 rhickey: Bronsa: I'm pushing edn-only reader momentarily, but no deref/quote in edn

14:43 TimMc: rhickey: "Disappointment" is probably the closest description.

14:44 Bronsa: rhickey: right

14:44 rhickey: TimMc: I'd like to work on getting everyone on board with educating people about being explicit, and/or just not using read if you don't understand or need it

14:44 aroemers: amalloy: you can read read-edn-string as "read edn from string", maybe that makes better sense then?

14:45 TimMc: Education is a losing game.

14:45 rhickey: So the idea here is that libraries will either call read or read-edn/edn-read?

14:45 rhickey: changing default doesn't really make safe

14:45 read-edn is safe

14:46 technomancy: TimMc: well, libraries that aren't targeting 1.4 compatibility

14:47 xeqi: I'd rather see clojure.edn/read and clojure.edn/read-string, but I'll vote for read-edn / read-edn-string given the two choices

14:47 rhickey: TimMc: I imagine most public libs only need read-edn

14:47 craigbro: TimMc: making read safe means disabling the ability to reliable write/read clojure code as data

14:47 TimMc: rhickey: Doesn't this hardcode a decision into a library?

14:48 craigbro: TimMc: which I think is a reasonable default behavior

14:48 rhickey: TimMc: there's no decision there, and if you need the power of read then lib uses read, all prior arguments about binding apply

14:48 craigbro: TimMc: there are more safety issues with read than just #=

14:49 rhickey: TimMc: e.g. libraries that deal with code will use read, libs that deal with the web will use read-edn

14:50 TimMc: I feel like that's a direct contradiction of what you said an hour or two ago about composability and general-purposeness.

14:50 rhickey: but you simply can't have a facility with a safety switch for primary internet interop, no matter which way it defaults

14:50 aroemers: changing the default might mitigate some risk of bad things happening in current bad code. The argument "don't use bad libraries" is not a strong one, one cannot simply write everything oneself or read all the soure code.

14:51 rhickey: TimMc: there we were talking about a switchable thing being configured one way at the bottom

14:51 I'm still opposed to that, and would advocate that libs that do read not bind *read-eval* for all the reasons I said before

14:52 but I think the discussion demonstrated that we can;'t make that safe, and flipping the default doesn't really make the story different - you should use something much less powerful for internet interop

14:52 aroemers: true

14:52 TimMc: Flipping the default is about layered security.

14:53 Risk management.

14:53 rhickey: now we'll have something, unambiguously the right thing with no caveats

14:53 aroemers: it won't make it safe, just mitigates some risk

14:53 hmm, that's true as well...

14:53 TimMc: So now if I want to use a library that uses read, but I know the data is user-provided, there's no way for me to make the library behave differently.

14:53 aroemers: damn this is hard :)

14:53 rhickey: TimMc: switch to read-edn, what's the problem?

14:54 TimMc: ? read is still controllable

14:54 TimMc: Sorry, I meant it the other way around. >_<

14:54 rhickey: which other way? :)

14:54 craigbro: read-edn

14:55 use a library that uses read-edn, but know data is user-provided...

14:55 TimMc: craigbro: *isn't

14:55 rhickey: library uses read-edn with user-provided (user is trusted???)

14:55 TimMc: could you just restate please?

14:55 craigbro: oh, I take "user provided above to mean, "from trusted source" sorry

14:55 TimMc: rhickey: A library that uses read-edn, and I want to point it at non-user-controlled data.

14:56 rhickey: user doesn't convey anything to me - trusted or not?

14:56 TimMc: Trusted data.

14:56 Serialized stuff.

14:57 rhickey: so you'll have a contract with trusted people, if more than edn you need read

14:57 if not more than edn, trusted/not doesn't matter

14:58 craigbro: TimMc: I think it would be best for the library, where feasible, to treat it as two different types

14:58 hiredman: edn still allows for reader literals

14:58 craigbro: application/edn (or whatever you call it) and application/clojure

14:58 ohpauleez: application/edn is the preferred one now

14:58 craigbro: and choose the reader appropriately, and also provide you with ability to tell it which types you accept so a non-trusted source can't just set their mime type header and p0wn you

14:58 rhickey: hiredman: yes, whatever you've installed, but not whatever an attacker submits

14:59 hiredman: right

14:59 I am just saying that there is a lot of flexibility there still as a serialization format

14:59 TimMc: hiredman: As long as some random common dependency doesn't have an unsafe data-readers...

14:59 rhickey: hiredman: absolutely

14:59 TimMc: Still better than #=, mind you.

14:59 rhickey: TimMc: you can't have everything

14:59 glosoli: hey folks, any ideas what can be the problem I installed LA Clojure plugin in Intellij IDEA CE, but it doesn't appear in the new project list

14:59 ?

15:00 craigbro: TimMc: then we're in the same boat, controller a dynamic binding 8)

15:00 hiredman: rhickey: so when do we get a built in data reader for binary data? #base64 or whatever? :)

15:00 rhickey: read-edn* seems to be the winner, how about that doc string for read?

15:00 hiredman: soon I hope

15:00 craigbro: we've solved the ruby YAML problem, we can't solve the rails XML reader problem 8)

15:01 TimMc: rhickey: I'd prefer the docstring for read mentioned "arbitrary code execution". It needs to leave nothing to the imagination.

15:02 craigbro: Gist of read doc string: https://gist.github.com/craigbro/4717176

15:02 TimMc: "Arbitrary Java objects" isn't scary enough.

15:03 Who's going to read that and think "hmm, static initializers..."?

15:05 akhudek: TimMc: I agree with your "arbitrary code execution" suggestion.

15:06 rhickey: Note that read can execute code (controlled by *read-eval*),

15:06 and as such should be used only with trusted sources.

15:07 what's the unicode character for scary monster?

15:07 aroemers: nice

15:08 angerman: Can I make a String in cljs Meta capable?

15:08 redinger: closest I can get: 👾

15:08 rhickey: looks like a box here

15:09 scary box

15:09 craigbro: Unicode hexadecimal: 0x1f627

15:09 bruceadams: i don't suppose a snowman is scary ☃

15:10 angerman: I'd like to do something ^raw "<b>…</b>"

15:10 e.g. tag a string as raw html… but apparently, String does not like meta.

15:10 kalizga: i suppose the shortcut for deref can't be used here; the macro system will think i want to splice: `[1 ~(deref (ref 2)) 3]

15:10 amalloy: angerman: if you can, you probably shouldn't

15:11 xeqi:

15:11 amalloy: kalizga: ~ @(ref 2) probably works fine, though i prefer deref

15:11 kalizga: yep, that works, amalloy

15:12 rhickey: so everyone's still unhappy with this plan, or the happy people are silent?

15:12 angerman: amalloy: so I better create a wrapper type?

15:13 akhudek: read-edn* sounds fine to me

15:13 amalloy: angerman: personally i wouldn't use a type (or at least wouldn't call it by such an exalted name). just a map with some data in it is fine

15:14 rhickey: we also could revisit the arglists for read-edn, Clojure's read following CL but I don't think the other args are often used

15:14 keeping it the same makes a migration easier, but saddles us with those arglists moving forward

15:15 angerman: amalloy: hmm good idea. Maybe I was on the wrong path anyway.

15:15 amalloy: rhickey: fwiw, i never use eof-error? or recursive?. i'd be happiest with ([] [stream] [stream eof-value])

15:15 rhickey: amalloy: same here

15:16 jonasen: What does recursive? do?

15:16 amalloy: honestly i usually rebind *in* rather than using `stream`, too

15:17 pepijndevos: ahoy clojure discussion! What's happening?

15:18 matthavener: seems like the unhappy people are silent

15:19 jweiss: anyone have a problem opening large (3mb) clojure data file in emacs? font-lock-mode goes crazy for me, emacs hangs longer than i'm willing to wait.

15:21 svedubois: Is this java to clojure correct?

15:21 long minSize = randomAccessible.dimension( 0 );

15:21 (def minSize (..dimension randomAccessible 0)

15:21 pepijndevos: svedubois: why the double dot?

15:21 hyPiRion: Is there still a discussion about read-edn-string ?

15:22 Because I seem to be late to the party, and want to propose string->edn

15:22 svedubois: (def minSize (.dimension randomAccessible 0)

15:22 matthavener: svedubois: just one dot, yep

15:22 tcrawley: svedubois: and a closing )

15:23 svedubois: yes

15:23 pepijndevos: hyPiRion: what is it anyway? I'm extra late to the party

15:24 hyPiRion: pepijndevos: 1.5.0-beta7 added safe-read and safe-read-string, which binds *read-eval* to false before calling the "unsafe" versions.

15:24 matthavener: hyPiRion: the function would be called "string->edn" ?

15:24 mattmoss: Some of us silent folks don't understand the issue, btw.

15:24 hyPiRion: matthavener: Well, I think it makes sense.

15:24 mattmoss: Listening to the middle of a conversation has never been a strong point of mine. :)

15:25 nDuff: hyPiRion: Are you actually serious about that?

15:25 pepijndevos: hyPiRion: And then the edn version is the one that reads edn as opposed to safe???

15:25 lazybot: pepijndevos: Yes, 100% for sure.

15:25 * nDuff isn't always good at detecting sarcasm over low-bandwidth connections

15:26 hyPiRion: nDuff: Every record makes ->Recordname and map->Recordname, so it seems consistent to me. I don't know.

15:27 *defrecord

15:27 kokenx5: hello

15:27 a

15:28 nDuff: hyPiRion: yes, but you're not creating an object of type edn here.

15:28 hyPiRion: pepijndevos: safe-read would be renamed to edn-read (or read-edn, whatver), and safe-read-string would be read-edn-string.

15:28 kokenx5: anyone know what the hell clojue ns is?

15:28 hyPiRion: nDuff: I don't consider records to be objects though. I consider them to be data

15:28 nDuff: hyPiRion: What you're doing is much, much more analogous to reader operation.

15:28 s/you're/this is/

15:29 kokenx5: is clojure a middle tier language

15:30 hyPiRion: nDuff: Oh right, *facepalm*

15:31 nDuff: kokenx5: That's quite a vague term. Clojure is general-purpose.

15:31 hyPiRion: edn stands for extensible data notation, you're not really converting anything, you're parsing it based on edn.

15:31 Wild_Cat: what are language tiers?

15:31 * hyPiRion walks away.

15:31 amalloy: jweiss: are you using rainbow parens?

15:32 callenbot: how much sense would it make to use something like Antlr4 from Clojure programmatically?

15:32 is that even a good idea?

15:32 kokenx5: nDuff: i mean is it used in the place of something like jsp or is it seen as a utility library?

15:32 nDuff: kokenx5: Clojure can be used all the way from the far back to the far front.

15:32 jweiss: amalloy: no. i had some extra highlighting on top of the normal clojure-mode but i took that out and still had the same problem. so even vanilla clojure-mode seems to have a problem. maybe because this clojure file was written to disk with the intention of reading it back in (so it's just one long line).

15:33 nDuff: kokenx5: ...you can use ClojureScript to build JavaScript for the client side of the UI, Clojure for the servlet bits, Clojure for any templating you need, etc.

15:33 Wild_Cat: kokenx5: you can run your entire webapp stack on Clojure.

15:33 rhickey: how about a system property for *read-eval* default?

15:33 hyPiRion: callenbot: I've heard of some people using Antlr from Clojure, and they haven't said it's bad at least.

15:33 kokenx5: nDuff: thats insane

15:34 Wild_Cat: insane

15:35 rbxbx: hyPiRion callenbot https://github.com/briancarper/clojure-antlr-example

15:35 nDuff: kokenx5: Flexibility is a strength of LISP-family languages.

15:35 thheller: hey, I have some clj->cljs conversion troubles, I pr-str'd a date in clj and get #inst "2012-01-23T14:03:19.445000000-00:00", cljs complains "Assert failed: timestamp millisecond field must be in range 0..999 Failed: 0<=445000000<=999"

15:35 rbxbx: older, but probably still relevant-ish

15:35 thheller: should I file a bug? :P

15:35 hyPiRion: rbxbx: thanks

15:35 (inc rbxbx)

15:35 lazybot: ⇒ 1

15:35 rbxbx: cheers :)

15:35 kokenx5: nDuff: I am a front end developer, so can you give me a sense to what clojure is most commonly used for?

15:36 nDuff: kokenx5: As I said -- it's a general-purpose language. There's no one niche I can describe to you.

15:36 kokenx5: nDuff: in a wayI might understand

15:36 nDuff: so its a utility library ?

15:36 tomoj: thheller: a Date or a Timestamp?

15:36 hyPiRion: kokenx5: What is Java for you?

15:36 thheller: its coming from sql so probably a timestamp

15:37 tomoj: Timestamps have ns precision, which cljs can't handle.. hmm

15:37 kokenx5: hyPiRion: i dont know squat about java past jsps in a j2ee

15:38 hyPiRion: kokenx5: Okay, tell me some languages you know and what you consider them to be

15:38 tomoj: should cljs just drop the ns?

15:38 or should you have to convert instants to ns-less instants before printing for cljs?

15:38 bruceadams: rhickey: system property for *read-eval* doesn't sound helpful.

15:39 kokenx5: hyPiRion: javascript, a loosly written language for the UI that can have OOP principles applied to it

15:39 thheller: dunno, what does edn say? :)

15:39 dnolen: tomoj: there's a ticket to add support for nanosecond precision

15:40 thheller: I'd be fine with no ns, just blowing up in my face sucks

15:40 rhickey: bruceadams: would let whoever doesn't get their way to swap the default without a major code change

15:41 I guess more transitional than operational

15:41 tomoj: dnolen: what, monkey-patch Date, or have the instant reader return something else?

15:42 bruceadams: rhickey: ah, ok. a way to pretend to have some control over older and/or unknown libraries. sounds like a band-aid.

15:42 kokenx5: is clojure better than groovy

15:42 rhickey: but it's too easy for people to say - everyone can just wrap their code with binding *read-eval* true

15:42 nDuff: *snerk*.

15:42 rhickey: bruceadams: a band-aid is what people are clamoring for IMO

15:42 tomoj: dnolen: nvm, I'll read the ticket :)

15:42 nDuff: kokenx5: You're picking an easy target there -- I'm hard-pressed to think of anything worse than Groovy.

15:42 kokenx5: The bytecode it generates is abominable.

15:42 dnolen: tomoj: instant reader that returns something else that supports all the operations possible of js/Date sounds ok to me.

15:42 bruceadams: rhickey: it does sound that way, yes

15:43 kokenx5: nDuff: LOL

15:43 pepijndevos: dnolen: Is this okay? https://github.com/clojure/core.logic/wiki/Extending-core.logic-(Datomic-example)

15:44 dnolen: pepijndevos: looks good to me

15:44 TimMc: rhickey: To recap, you're worried that defaulting *read-eval* to false will A) lull people into a false sense of security, and/or B) break too much code?

15:45 rhickey: TimMc: both

15:45 pepijndevos: dnolen: cool. I'll see if I can actually replicate it with Redis once I have some time.

15:45 ravster: hello all

15:46 rhickey: TimMc: I'm happier now with he warning on read and the clearly safe read-edn alternative

15:46 dnolen: pepijndevos: thanks for updating the wiki

15:46 rhickey: but people with perfectly fine programs will get broken to protect others

15:46 hyPiRion: rhickey: I think that should be sufficient, as long as the community actively explains the difference between read and read-edn.

15:46 pepijndevos: dnolen: no problem. I'm more than happy to contribute to core.logic in ways that do not break my brain :P

15:46 ravster: I'm trying to learn the 'friend' auth system. I'm finding it difficult to wrap my head around it. Is there a tutorial or some text out there that explains the reasoning behind the 'friend way'?

15:48 rhickey: TimMc: making *read-eval* default to false won't make using read for untrusted data, without explicit binding, ok

15:48 TimMc: Sure.

15:49 I don't think anyone is arguing that it would.

15:49 ivaraasen: dnolen: the Reasoned Schemer is melting my mind, but in a cozy way

15:49 borkdude: wow, discussion still going

15:49 rhickey: TimMc: by arguing for the default to change they are arguing for being able to leave their broken programs alone

15:50 TimMc: That's an unfair characterization.

15:50 kayokenx5: what is the purpose of a jvm based language

15:50 pepijndevos: As someone who doesn't hang out on the mailing list all the time, I find using edn more confusing. Until 5 minutes ago, I did not know or remember what edn was.

15:50 rhickey: maybe others broken prgrams, let's say

15:50 kayokenx5: is it to limit the amount of java written?

15:51 ravster: kayokenx5: :)

15:51 matthavener: I think its more people are afraid the community will be "shamed" ... the YAML::load of clojure

15:51 rhickey: TimMc: certainly some of the programs are broken due to ignorance due to lack of documentation

15:51 Wild_Cat: kayokenx5: the purpose is to benefit from the JVM's omnipresence, the Java ecosystem (tons of existing libs) and to not have to write a complicated optimizing compiler

15:52 rhickey: matthavener: what's more shaming - we fooled ourselves into complacency by changing a default or mobilized everyone to become actually safe?

15:52 TimMc: rhickey: A default binding of false will not prevent the community from doing education as well.

15:52 kayokenx5: wild_cat: so the flip side would be to use vanilla java which would be inefficient in some cases?

15:52 matthavener: i agree rhickey, i think the long term is to avoid complacency and the expense of a short term patch

15:53 rhickey: TimMc: that's the first iota of that spirit I've seen today

15:53 Wild_Cat: kayokenx5: ah, you meant the benefits of using a JVM-based language compared to using Java itself directly?

15:54 kayokenx5: well, at that point it's mostly a matter of *programmer* efficiency. Languages like Clojure and Scala are far more expressive than plain old Java, and allow you to write your programs more quickly, in much less code and very often with less bugs.

15:54 TimMc: rhickey: You see people warning each other about unsafe stuff *all the time* in here.

15:54 kayokenx5: Wild_Cat: there is no other alternative right, either Java itself or a JVM lang

15:54 ?

15:54 bruceadams: TimMc: changing the default binding has the potential to confuse people into thinking it fixed everything

15:54 Wild_Cat: kayokenx5: of course there is. You can use a non-JVM language :p

15:54 TimMc: bruceadams: Which people?

15:54 aroemers: rhickey: how about a dynamic var that warns us that an unsafe read takes place? Like *warn-on-reflection*? That way it might be easier to pinpoint whether a lib is bad, without reading it's entire soure code?

15:54 matthavener: bruceadams: i think worse it just breaks legacy functionality

15:55 TimMc: bruceadams: THe people who already know it exists at all won't be confused, and the other people, as I said, won't know about it.

15:55 bruceadams: TimMc: i was thinking of people on the edges of the clojure community, who aren't in the constant conversations.

15:55 (which, at times, includes me)

15:55 pepijndevos: dnolen: it seems the actual to-stream for datomic is somwehat more comples than the example. What is going on there? Are you walking vars to pre-select on grounded values or something like that?

15:55 aroemers: rhickey: not that such a thing should be in 1.5, but more, do you like such an idea?

15:56 TimMc: There's such a big fuss right now that anyone who hears about the change will also almost certainly hear about the discussion *not* to change it.

15:56 bruceadams: "did you hear that clojure has a problem like that rubygems.org thing? yeah! but I hear they fixed it..."

15:56 something like that.

15:56 Sgeo: What change?

15:56 rhickey: bruceadams: but changing the default doesn't fix it

15:56 TimMc: bruceadams: I suppose nuance could be lost along the way.

15:56 dnolen: pepijndevos: there some half-baked smarts to access the right index based on ground vars

15:56 clojurebot: vars are a linking construct

15:57 rhickey: dnolen: that reminds me, I owe you fully-baked smarts

15:57 Sgeo: Oh, default of *read-eval*?

15:57 bruceadams: rhickey: exactly. which is why "fixing it" scares me. people will hear "fix" without the subtleties.

15:58 TimMc: bruceadams: And you think these few people would otherwise have been carefully binding *read-eval* to false?

15:59 bruceadams: TimMc: no, certainly not. i can hope that we can get the message out that "read" is dangerous and to use read-edn (or others) instead.

15:59 TimMc: bruceadams: Then at worst they're still writing broken programs, just like they would have.

16:00 Education about read is a separate thing, and only helps, but is not sufficient.

16:00 It's not like this would make *those people's* programs more unsafe.

16:01 xeqi: is read-edn planned as a change for clojure 1.5 or a seperate contrib?

16:01 seangrov`: Where the hell do Infinity/-Infinity live in js?

16:02 bpr: Float/POSITIVE_INFINITY

16:02 similar for Double

16:02 TimMc: seangrov`: In JS, just Infinity

16:03 You mean hwo to get it from CLJS?

16:03 seangrov`: Must be on the window object then

16:03 pimeys: is it really so bad to break things for good sometimes?

16:03 seangrov`: Yeah

16:03 pimeys: I know, it's annoying

16:03 but coming from the ruby world...

16:03 this is peanut

16:03 s

16:03 TimMc: pimeys: And it wouldn't be the first breaking change...

16:03 nDuff: pimeys: You realize that that's a major drawback to Ruby?

16:03 TimMc: God knows the RTE-wrapping stuff screwed up a bunch of programs.

16:04 amalloy: quick, someone caption a "This is Sparta!" image with "This is peanut!"

16:04 pimeys: just a good way to do it is to spam with deprecation warnings

16:04 but I don't know...

16:04 Foxboron: amalloy: i think "This is data!" is better.

16:04 pimeys: nDuff: I do

16:04 and more for Rails

16:04 where upgrading is very expensive

16:06 TimMc: "Breaking code" is an *extremely* weak argument.

16:06 cemerick: wow, still churning

16:06 TimMc: People don't expect Clojure to follow semver.

16:07 xeqi: cemerick: it died for awhile

16:07 cemerick: topic "weather" visualizations for irc/twitter/g+/etc would be an interesting service

16:08 akhudek: cemerick: nice idea!

16:08 seangrov`: cemerick: What's churning?

16:08 akhudek: topic forecasts could be a neat CS problem too

16:08 amalloy: cemerick: weather? like, "there's a big arbitrary-code-execution front blowing through #clojure today"?

16:08 ravster: amalloy: lulz

16:09 cemerick: There's very generalized sentiment stuff, but discrete topics aren't bubbled up last I saw.

16:09 ravster: (about 'this is peanut')

16:09 cemerick: amalloy: exactly

16:09 seangrov`: I saw TimMc talk about 'breaking code' arguments, so I inferred...

16:09 TimMc: ANd you were right.

16:09 seangrov`: Ah, I see

16:10 amalloy: certainly would be fascinating if anyone could make it happen

16:10 cemerick: amalloy: really just a wordcloud or something might be enough

16:10 akhudek: cemerick: applying an online LDA process to it and reporting a sample from the top topics might work

16:10 cemerick: yeah, hook opencalais up to it, etc

16:11 hiredman: dnolen: I've been thinking it would be neat to do something with a rest api (maybe github's?) with core.logic in clojurescript, but it doesn't seem core.logic plays well with ajax, I cannot start a core.logic computation, suspend the computation then resume it after I get a result

16:11 TimMc: cemerick: "Temperatures will rise toward mid-day as an HTML-templating front sweeps in from Callenia."

16:12 cemerick: exactly

16:12 xeqi: y u no use laser?

16:12 dnolen: hiredman: suspending + resumption is something I plan on looking at some point. I think Kanren went there at some point ...

16:12 TimMc: "Deaths may be in the tens of thousands."

16:13 hiredman: dnolen: a cps monad transformer or whatever

16:14 dnolen: hiredman: things will probably easier when I make search configurable

16:15 amalloy: cemerick: a word cloud for that would be pretty neat, in fact. just browsing through what yesterday's cloud might look like is entertaining

16:16 hiredman: ok, good to know it is a "thing" and I am not overlooking something else

16:16 cemerick: yeah

16:18 rhickey: if you don't think code breakage isa big deal, it would be easy to set it up so all code blows up that hasn't explicitly set *read-eval*

16:19 I've got that running, here, it's quite interesting

16:19 everyone seems ok with breaking other people's code, this would break almost everyone - fair?

16:20 svedubois: Is there a better/more concise way to write this java example to clojure?

16:20 https://github.com/imagej/imglib/blob/master/imglib2/examples/src/main/java/Example4a.java

16:20 https://www.refheap.com/paste/9495

16:21 amalloy: svedubois: that doesn't look like it works. the loop form always exits immediately, for example

16:21 dnolen: hiredman: even if I add search customization support I can't say how hard or easy it would be to actually implement restartable search. It's definitely interesting to me.

16:21 rhickey: certainly safest, as the semantics surrounding every read would have to be considered

16:22 abedra: rhickey: what interesting things are you seeing?

16:22 rhickey: abedra: just the breakdown of trusted contexts vs not

16:22 hiredman: dnolen: sure

16:26 rhickey: and thinking through, if there was this must-specify rule for everyone, people migrating would almost certainly just search/replace read with read-edn for untrusted contexts, leaving people with trusted contexts to have to do a lot of binding-wrapping they don't need to now

16:26 abedra: rhickey: interesting

16:28 rplaca: 4

16:28 rhickey: OTOH, I keep trying to protect these people and none have shown up :(

16:29 abedra: rhickey: I like the idea of separating contexts

16:30 rhickey: the idea is already there, but since switchable, no default is reliable

16:30 abedra: potentially even a fn/macro 'trusted' that has different bindings/behavior

16:31 reiddraper: apologies if this is a can-of-worms question, but are there that many cases where users of libraries want to change *read-eval*? why have a var at all, and not just make read*/load functions take an extra argument?

16:31 abedra: Personal opinion is to default to the safest thing and allow easy switching

16:31 aroemers: rhickey: I actually like that idea, of an initialy unbound *read-eval*. hiPerion also suggested this. Does it really break so much?

16:31 cemerick: rhickey: who are you protecting?

16:31 Sgeo: Haskell's unsafe functions typically get unsafe in front of their name

16:32 TimMc: Sgeo: RH has already expressed his concern that we'd get a proliferation of unsafe-* aliases.

16:32 Sgeo: oh

16:32 I should learn to pay attention before saying things

16:33 Frozenlock: Sgeo: I suspect the number of people following this conversion is greater than what it appears to be. They are all paying attention :P

16:34 Sgeo: I'm working on a blog post... that sort of complains about Clojure

16:34 >.>

16:35 Frozenlock: What is its theme?

16:35 abp: the parens

16:35 Frozenlock: ,flame abp

16:35 clojurebot: #<CompilerException java.lang.RuntimeException: Unable to resolve symbol: flame in this context, compiling:(NO_SOURCE_PATH:0)>

16:35 rhickey: cemerick: people who are happy with the status quo, wrap their untrusted reads with bindings, leverage the power of the reader otherwise

16:35 Frozenlock: aw cmon bot, you don't have a `flame' option?

16:35 abp: Frozenlock: yes, Sgeo

16:35 technomancy: for the record I fully agree with cemerick, except for the name of dangerous-read; read-unsafely is a better name.

16:35 Sgeo: Typing. I think trampoline is somewhat unclean

16:36 technomancy: (not that I feel like arguing it right now)

16:36 Sgeo: And that there would be a cleaner design if people approached writing trampoline with the mentality of someone used to static typing

16:36 TimMc: Unbound *read-eval* is kind of tempting, in a "it's so crazy it just might work" sort of way.

16:36 cemerick: technomancy: the naming for the bifurcated fns was a very minor proposal

16:36 Sgeo: Which is not to say I'm advocating for static typing -- just the mindset that static typing brings

16:37 TimMc: The downside is... what, an extra line in every app?

16:37 It certainly would educate people.

16:37 abedra: TimMc: potentially an option in Leningen project.clj file?

16:37 cemerick: rhickey: I keep waiting for the killer use case for #= that make it worth maintaining as a default.

16:37 technomancy: abedra: yeah, but it wouldn't help for people who deploy uberjars

16:38 abedra: technomancy: uberjar could respect that and add it

16:38 TimMc: abedra: `lein new app`'s template could include it

16:38 technomancy: abedra: yeah... I've been reluctant to engage in meddling so far, but this could change my mind =\

16:39 cemerick: s/the killer/any

16:39 technomancy: that'd save me from twiddling things in friend ;-P

16:40 xeqi: I'm happy with the read-edn functions. I'd prefer read not to be able to load classes or eval by default, but I can alter-var-root my way around that

16:41 svedubois: I have changed loop for for: https://www.refheap.com/paste/9499

16:41 Are there any more errors ?

16:41 The java example reference: https://github.com/imagej/imglib/blob/master/imglib2/examples/src/main/java/Example4a.java

16:41 TimMc: xeqi: Good point re: alter-var-root; (binding ...) wouldn't help with non-binding-conveyed threads.

16:45 amalloy: certainly both of those `for`s want to be doseqs, at least. whether there are any other errors i don't know; i'm only really willing to glance over it

16:50 Frozenlock: I'm starting to wonder if #clojure is the official documentation.

16:50 TimMc: How would an unbound *read-eval* affect the REPL?

16:53 rhickey: cemerick: just because you don't use it doesn't mean no one deas. e.g. clojure reader page promises:

16:53 Calls to Java class, deftype, and defrecord constructors can be called using their fully qualified class name preceded by # and followed by a vector:

16:53 #my.klass_or_type_or_record[:a :b :c]

16:53 svedubois: I have changed for for doseq, are there any more errors? https://www.refheap.com/paste/9500

16:54 pppaul: i have an issue with responding with generated xml (via clojure.data.xml). sometimes my tags have spaces in the middle of them…. eg <Fa cet Name="something">....

16:54 anyone have a similar issue?

16:54 by responding i mean via ring/compojure

16:54 TimMc: I had no idea you could do #java.lang.Integer[4] -- I thought that was only for defrecords and deftypes.

16:55 cemerick: rhickey: That surely predated IType and IRecord

16:55 rhickey: cemerick: and being cavalier about breakage is simply not being serious about language stewardship

16:55 TimMc: I suppose the machinery doesn't care.

16:56 rhickey: also the reader can convey and embed classes, and function values

16:56 TimMc: Embed classes?

16:56 rhickey: yes

16:56 Frozenlock: Function values?

16:56 :)

16:56 TimMc: Like, (read-string "java.lang.Integer")?

16:56 rhickey: and Java collections nested in Clojure collections etc

16:57 you can't advocate for the language being one way or another just from your own experience, you have to think about other people doing different things

16:57 #=java.lang.String

16:58 There are people who know about and use these things even if you don't

17:00 user=> (binding [*print-dup* true] (pr-str +))

17:00 "#=(clojure.core$_PLUS_. )"

17:00 Frozenlock: ^

17:00 cemerick: rhickey: Classes, function values, etc can all be conveyed without enabling arbitrary evaluation. I'm not being cavalier about breakage, I'm attempting to be practical and principled given likely threats.

17:00 rhickey: cemerick: but people are complaining that class loading is a form of evaluation, and it is, and you can't draw a line around which classes are ok

17:01 then there are the java types you might want to embed that have no ctor, but do have static factory method

17:02 cemerick: either you are open to the java world or not, and it is difficult to circumscribe it

17:03 cemerick: and claiming there are other ways ignore the fact that people are already using the existing ways, thus breakage

17:04 i.e. #=(clojure.core$_PLUS_. ) might already be sitting in files

17:04 cemerick: rhickey: Indeed, class initialization is a sticky issue; perhaps the other arity of Class/forName helps there. At no point did I claim that e.g. my patch was the end-all, be-all. However, it significantly narrows the vulnerability.

17:04 rhickey: cemerick: not buying that

17:06 TimMc: rhickey: THere are class initializers that can do arbitrary code eval, or what?

17:06 cemerick: rhickey: Well, okay. Likewise, I don't think the advice of "you all just need to be better programmers" is a reasonable stance. We'll just have to agree to disagree, at least for now.

17:06 rhickey: TimMc: every clojure AOT class does, for example

17:07 cemerick: don;t put quotes around something I didn't say

17:07 Frozenlock: That's probably where my java ignorance shines... I don't understand what's the advantage of using #= vs ((eval (read-string "+")) 1 2). :(

17:07 cemerick: rhickey: Apologies. That's my paraphrasing.

17:08 rhickey: Frozenlock: the former yields an object in the read data, the latter needs to be evaluated

17:08 TimMc: Frozenlock: It's for (de)serializing data.

17:08 rhickey: cemerick: what I''ve been saying is clear - only explicit is safe

17:10 cemerick: rhickey: I understand; except that there is case after case of that approach being unsustainable and not applicable to the general programming populace, or to really great programmers having a bad day.

17:10 pbostrom_: svedubois: you need to do some reading about some Clojure fundamentals like immutability and lexical scope; for example, your code implies that you can reassign minSize at will, and radiusLargeSphere will be automatically updated with the latest value

17:11 technomancy: yeah, I'd feel more comfortable with education as a solution if we didn't regularly have people coming into the channel asking why contains? doesn't work on lists.

17:12 rhickey: I fail to see how changing the default solves anything

17:12 * Frozenlock just erased his last question, thanks to technomancy

17:13 rhickey: non-explicit code is at permanent risk of having its risk escalated silently

17:13 cemerick: rhickey: there is no Solution™; there are only ways to limit the scope of such vulnerabilities, and then contain the damage that can be done from within them.

17:13 mattmoss: rhickey: Is there a pull request/bug tracker issue/changelog/etc something that succinctly identifies what the issue is? I can't say I've followed the whole discussion, but I'd like to understand it.

17:13 hyPiRion: mattmoss: I'll give you a link

17:14 https://groups.google.com/forum/?fromgroups=#!topic/clojure-dev/zG90eRnbbJQ

17:14 TimMc: rhickey: I sometimes forget to put in safeguards. I'd like to find out about such an oversight via a code quality checking tool rather than an exploit.

17:14 mattmoss: hyPiRion: Thanks.

17:14 rhickey: cemerick: that's simply not true, the force explicitness code I have is categorically different than changing the default

17:15 normanrichards: As long as proper replacements are provided, I don't see why there's a need to change to the current behavior. I'd feel more comfortable calling a new function whose contract and design is safety rather than worrying about possibly breaking changes.

17:15 rhickey: I find it interesting you guys aren't rallying around that as being superior, I guess cause it will break your code

17:15 TimMc: You're not forcing explicitness by leaving read (and its settings) completely unchanged -- broken code stays broken.

17:15 craigbro: does a modern (1.4) pr ever produce output that cannot be read with *read-eval* false by a modern (1.4) read?

17:15 rhickey: TimMc: no, the code here requires all use bind *read-eval*, if you fail to do so you read will fail

17:16 cemerick: rhickey: I'm not aware of the approach, wasn't watching the channel for most of the afternoon.

17:16 rhickey: TimMc: breaking almost everyone, but in the end all code will be explicit

17:18 TimMc: rhickey: Ah, you're talking about the unbound approach? Yes, that does force explicitness, and I like that about it.

17:18 It's a *different* approach, though, and I am unsure what knock-on effects it would have.

17:18 rhickey: not exactly unbound

17:18 TimMc: Oh, :unbound?

17:19 rhickey: init nil vs true/false

17:19 and can init from System property

17:19 so no one need break

17:19 TimMc: Huh. And then read would yell if it's nil?

17:19 rhickey: read thros if nil

17:19 throws

17:20 craigbro: seems like a solomon's compromise to me

17:20 TimMc: :-D

17:20 craigbro: kinda makes clojure broken by default 8)

17:21 rhickey: The goal at the end of the day should be safe programs. Don't mistake 'safe' default for a goal, it is a mechanism, and a weak one since it doesn't yield the goal. forced init does

17:21 TimMc: rhickey: Yes, I would feel much more comfortable about my own code if that were in place. However, I'm not sure what else might suddenly break.

17:21 Safety is a process.

17:21 amalloy: craigbro: persnickety by default, perhaps. the current "arbitrary eval whenever" behavior is broken by default

17:21 craigbro: I disagree

17:22 amollow: that was, I disagree with your last statement

17:22 rhickey: TimMc: so the process can involve setting the property to start, testing without, making your code explicit then doing without the property

17:22 ChongLi: I like it

17:22 craigbro: amalloy: amalloy even, sorry 8^)

17:22 ChongLi: whenever you force someone to be explicit they can't get around thinking about the implications

17:22 amalloy: doesn't your client have tab-completion for nicks, craigbro?

17:22 craigbro: amalloy: yes, and I thickfingered it by instinct

17:23 amalloy: anyways, I don't think current default behavior is broken

17:23 amalloy: I asked earlier if pr produced anything that read cannot read with *read-evval* set false

17:23 TimMc: So the compiler would need to bind it for its own stuff, the REPL would need to make a decision, etc.

17:23 craigbro: aka, is there anything in clojure that relies on the #= now

17:24 amalloy: of course there is. if nothing in clojure relied on #= it either would never have existed, or rich would just delete it without wasting his time asking us what to do

17:24 craigbro: ok, then it's not broken

17:24 and by default, I should be able to pr and read any fucking data

17:24 profanity denotes enthusiasm, not aggro 8^)

17:25 TimMc: craigbro: I think everyone here agrees that *read-eval* has severe security implications?

17:26 craigbro: reading clojure code over the internet has serious security implications

17:26 that the default behavior of read is to accept all pr produced clojure code is not broken

17:26 TimMc: craigbro: Or data, or from the filesystem, or...

17:26 craigbro: it is misuse of read

17:26 sure, internet was used as shorthand for "untrusted"

17:26 TimMc: craigbro: It happens to be the only utility to do so.

17:27 I await your submission of an EDN reader.

17:27 In core, mind you.

17:27 Raynes: TimMc: Didn't rich just write one...?

17:27 He was asking what to name functions for it like an hour ago.

17:27 craigbro: TimMc: not true at all, there is an edn reader now, read-json is what we use for moving data in our web app

17:27 TimMc: JSON... right...

17:28 technomancy: Raynes: no, that was just read-string with *read-eval* disabled as a placeholder for a future compliant implementation

17:28 Raynes: I see.

17:28 craigbro: TimMc: yes, json, because it's a well defined, non-evaluative, and restricted subset. If we needed more we would use EDN

17:29 TimMc: Well, if you don't see the use case for read + *read-eval* false, I can see why you're having trouble following.

17:30 craigbro: TimMc: I understand the use casespeople have put forward, and I also have pointed out that setting it to false is not sufficient

17:31 abedra: all it takes is a (binding [*read-eval* true] ...) inside of some injected code to put it all back together though right?

17:32 craigbro: abedra: if you can "inject code" at that point you wouldn't bother

17:32 rhickey: abedra: no injecting needed, that might be bound accidentally around some code that presumed the 'safe' default

17:32 TimMc: abedra: You're confusing levels.

17:33 craigbro: if the purpose of read is to read clojure code

17:33 rhickey: gun, meet foot

17:33 craigbro: they don't break it because people might read untrusted data with it

17:33 s/they/then

17:33 technomancy: seems like never binding *read-eval* around anything other than a call to the reader itself would prevent that easily

17:33 rhickey: -Dclojure.read.eval=false

17:33 abedra: craigbro: I completely understand

17:33 hiredman: cemerick: speaking of reading things, nrepl seems to blow up if I put a data literal inside an expression, like (type #foo/bar ...) but not if I do something like (def x #foo/bar ...)

17:33 abedra: just throwing out things to consider

17:34 rhickey: Ok, here's the current package:

17:34 new read-edn* functions, have none of the dangerous code of the reader

17:34 default for *read-eval* is 'unset'

17:35 reads with 'unset' *read-eval* will fail

17:35 you can either bind (preferred in the end)

17:35 or, use properties like -Dclojure.read.eval=false/true

17:36 thus no awkward root binding or thread propagation issues

17:36 craigbro: in CL *read-eval* defaults true

17:36 just a data point

17:36 rhickey: also, no defaults - you must make some choice

17:37 TimMc: Sounds great.

17:37 thheller: Is it still required to sign the CA on paper to get a patch accepted into CLJS? (just opened CLJS-466 with patch)

17:37 craigbro: rhickey: the idea of shipping a lisp that can't read it's pwn pr without setting a config var or binding something strikes me as horrid

17:37 Frozenlock: craigbro: Don't you have to use `eval'?

17:37 Bronsa: rhickey: would the whitelist stuff be still there or not?

17:37 rhickey: Bronsa: no

17:37 cemerick: rhickey: If I had had any inkling that you would have ever considered such a thing, I would have proposed it. :-)

17:38 rhickey: same old reader

17:38 cemerick: yeah, sure

17:38 craigbro: I feel like we're being scared into breaking the system

17:38 TimMc: craigbro: ALl that means is that the printer is too expressive.

17:38 craigbro: 5 years of CL web dev, and noone ever suggested that *read-eval* default to false because someone might read bad data with it

17:38 rboyd: keep clojure weird

17:38 abedra: rhickey: +1 I like that idea

17:38 technomancy: rhickey: definitely an improvement; I approve.

17:39 cemerick: rhickey: you could add a ;-) sometime ;-)

17:39 Raynes: rhickey: I give my approval, because I know how important it is to you.

17:39 ChongLi: count me in, though I don't think this is really a vote

17:39 rhickey: cemerick: I'm too exhausted to emoticon

17:39 craigbro: hah

17:39 rhickey: is that a verb?

17:39 TimMc: emote?

17:39 Raynes: emote

17:39 TimMc: iconemote?

17:39 technomancy: we should take special care that the error message points is actionable and makes it clear what to do in the case of either trust or lack thereof

17:39 s/points //

17:40 rhickey: technomancy: do you have a positive suggestion?

17:40 TimMc: New bikeshed topic: Proper error message for read on unbound-*read-eval*.

17:41 Pass it on.

17:41 ChongLi: how many people are going to point their project.clj to clojure 1.5 and not expect some breakage? it seems reasonable

17:41 craigbro: TimMc: if pr needs it... then it's not too expressive

17:41 ChongLi: that alone seems like a big step to me

17:41 craigbro: king solomon option

17:42 so it goes

17:42 TimMc: "read[-string] requires *read-eval* to be explicitly bound to true/false. If you don't know what that means, use false."

17:42 craigbro: cutting the baby in half, breaking code all over and shipping a broken lisp

17:42 but, hey, I'm a fuddy duddy, so I'll just finish up by saying "off my lawn!!!" 8^)

17:42 rhickey: I need to sleep on it, my fear is it saddles us with a permanent property, even after all code is better

17:43 technomancy: rhickey: the exception message itself is probably too short to explain it well, so it'd probably be best to make it point to *read-eval*'s docstring.

17:43 abedra: rhickey: I think it's a good start

17:43 rhickey: technomancy: that's what it does now

17:43 abedra: Perhaps we should take it to the wiki?

17:43 hyPiRion: rhickey: I suppose read-edn and read-edn-string would still be usable without *read-eval* bound?

17:43 technomancy: rhickey: cool; let me take a look at the latest

17:44 rhickey: now means here on my machine :)

17:44 amalloy: hyPiRion: yes

17:44 rhickey: hyPiRion: yes

17:45 It could become a point of pride that an app/library not need the property

17:45 craigbro: hehe

17:45 TimMc: craigbro: https://gist.github.com/fogus/4716440

17:45 An example of some very good programmers forgetting to set read-eval to false.

17:46 dnolen: rhickey: I guess this set of options complicates developing libraries meant to be used by others that leverage read? the library shouldn't not dictate *read-eval* but will need to test by setting it?

17:46 rhickey: TimMc: did they have read-edn?

17:47 dnolen: no, as before, the *read-eval* policy is at the app, not library level

17:47 TimMc: Nope! It's CL or something.

17:47 hyPiRion: okay, then I'm very fine with this. Only concern is that clojure packages for linux distros doesn't has read-eval as a setting you can turn on/off. But that's not fault of the language itself, though.

17:47 /s/has/have/

17:47 rhickey: a lib should be written presuming the caller will set the policy and binding

17:47 Raynes: dnolen: Well, exactly. They can set it in the tests.

17:48 rhickey: TimMc: joking

17:48 TimMc: Well, you never know.

17:48 craigbro: hey, what is Goldsith's first name?

17:48 TimMc: rhickey: I assume you meant "an equivalent to read-edn".

17:48 dnolen: rhickey: yes, I just meant complicated testing - but I guess it's not a big deal, but it does seem like a lot of pointless breakage.

17:48 complicating

17:49 rhickey: dnolen: you seem to be the only one with me on the pointless

17:49 hyPiRion: dnolen: `lein test :read-eval false`? Or some property in project.clj?

17:50 (Sure, Leiningen doesn't cover all Clojure use cases.)

17:50 abp: hyPiRion: It get's worse.. properties, flags, bindings

17:50 craigbro: TimMc: also, no evidence it is *read-eval* there either

17:50 rhickey: I agree it is pointless breakage

17:50 rhickey: a huge number of apps will never use read

17:50 hyPiRion: abp: Not really. Just ship with read-edn instead.

17:50 rhickey: given read-edn

17:51 TimMc: craigbro: "adding one line of code--'setf *read-eval* nil'--fixes the problem"

17:51 craigbro: TimMc: thanks

17:51 TimMc: It's kind of buried, sorry.

17:52 craigbro: there are other ways past a lisp reader

17:52 rhickey: and thus the ones that use read will be doing so because they need the power of read

17:52 augustl: what does a set use to determine the identity of a java object? It's hashCode or something?

17:52 s/It's/Its/

17:52 TimMc: hyPiRion: This would be to set up the default binding in the test.clj files?

17:52 rhickey: this entire thing exists to avoid people having to switch from read to read-edn - seems quite bogus

17:53 hyPiRion: TimMc: That's my idea, but it's just a proposal. :)

17:53 abp: rhickey: You still absolutely can't stand it, right? ;)

17:53 rhickey: right

17:53 mattmoss: Maybe I missed something, but... I don't understand fully the need to have the third, unset state of *read-eval*. I suppose it throws if unset, but if libs/app can bind it anyway, wouldn't I always be best to rebind *read-eval* just before reading?

17:53 TimMc: That kind of statement makes me feel very twitchy about the possibility of writing secure code in Clojure.

17:53 mattmoss: I don't use #= or read-string myself, so I'm not up on typical useage.

17:54 craigbro: TimMc: the clojure reader != the common lisp reader 8)

17:54 abp: I too, on the other hand I understand people wanting something to be done about it.

17:55 But considering the years with true as default. Now it's a problem and we need to fix it in two days somehow..

17:55 technomancy: abp: have you been following the rubygems.org exploit?

17:56 abp: http://www.kalzumeus.com/2013/01/31/what-the-rails-security-issue-means-for-your-startup/

17:56 abp: technomancy, roughly

17:56 craigbro: given read-edn... my preferences now are default stay the same, default is false, and the bound to nil and throw error option is anathema 8^)

17:56 TimMc: abp: 1. Apathy caused the delay, 2. a recent exploit caused the emergency.

17:56 craigbro: TimMc: a recent exploit in clojure code, or are you refering to ruby

17:56 gfredericks: (binding [*data-readers* {'eval #'eval}] (read-string "#eval (+ 1 2)"))

17:56 TimMc: craigbro: Ruby/Rails.

17:57 abp: Rails, for programmer convenience, used YAML to implement JSON deserialization. => Clojure programmers use their compiler read for edn deserialization for convenience?

17:57 technomancy: abp: ring's cookie store was recently found to be vulnerable to the same bug for cases where user data was placed in the session

17:57 abp: technomancy: I know.

17:57 craigbro: technomancy: assuming they got the app secret

17:57 technomancy: craigbro: no

17:57 abp: exactly

17:57 gfredericks: I've only spent the last hour reading the day's conversation so I'm not fully caught up; I haven't been able to figure out why #= is still a needed feature at all given data readers

17:58 craigbro: technomancy: ok, am I misreading that code then?

17:58 technomancy: you can do it without getting the secret if user data is placed in the session

17:58 matthavener: i wonder if python folks are freaking out too. pickle deserialize has similar eval problem

17:58 amalloy: matthavener: less, from what i hear. allegedly they are already pretty paranoid about unpickling untrusted data

17:58 craigbro: if you can modify that cookie without the app secret, that's the actual bug 8)

17:59 technomancy: matthavener: maybe because that's been documented from the outset

17:59 craigbro: sorry, I use in mem session store with ring, I misunderstood that patch

17:59 matthavener: technomancy: yeah, i think thats the biggest difference

17:59 TimMc: technomancy: It probably helps that the name "pickle" is weird.

17:59 "What the hell is 'pickle'? I'd better go read the docs."

17:59 matthavener: TimMc: maybe the solution is to rename "read" to "unpickle" in clojure :P

18:00 dnolen: this is interesting - http://github.com/search?l=clojure&q=read-string&ref=searchresults&type=Code, read-string usage according to GitHub

18:00 craigbro: dnolen: went over that earlier 8)

18:01 rhickey: technomancy: as long as the reader has a mode that's unsafe, you shouldn't be using it for internet data. e.g. there was (and is) a hole around record literals not being governed by *read-eval*. The only safe thing is something that does not have that possibility at all, e.g. read-edn. People will *have to* switch in order to be safe. We need to move people off read ASAP, not assuage them with defaults

18:04 mattmoss: Why not... "People will *have to* switch in otder to be _unsafe_" ? Or is that part of the decision going on? (ima n00b)

18:05 Making things globally safe breaks too many people/

18:05 ?

18:05 TimMc: Yeah.

18:05 hyPiRion: mattmoss: That's been the discussion for the last 5 hours.

18:05 TimMc: People don't see eye-to-eye on that tradeoff.

18:05 hyPiRion: If not more.

18:06 abp: hyPiRion, much more.

18:06 mattmoss: hyPiRion: Yeah, I kinda got that... just a lot of details flying past I wasn't getting the whole pic.

18:06 craigbro: TimMc: check the last part of this out

18:06 http://letoverlambda.com/index.cl/guest/chap4.html

18:06 skip down to Reader Security

18:07 rhickey: changing the default doesn't make it safe. If we changed the default as soon as there was a cry for it, without looking further, everone would happily be using dangerous reader where record format is not governed by flag - i.e. still not safe, but feeling good about ourselves

18:07 craigbro: TimMc: i present that as an example of the various problems with (read) that are not just *read-eval* being true

18:07 rhickey: read-edn is safe - use it for untrusted sources

18:08 that must be the mantra

18:08 TimMc: rhickey: read-edn is unsafe if a dependency brings in an unsafe data_readers.clj

18:09 craigbro: TimMc: we discussed that earlier, binding *data-readers* is the proper way to be safe with it

18:09 TimMc: Mmm, right.

18:10 rhickey: TimMc: a dep could open a port too, c'mon

18:10 the hyperbole is think in here

18:10 craigbro: well, a backdoor is different than a incompletely understood data reader

18:10 akhudek: Very widespread posting/education about the issue seems the best way to address this. Don't think the defaults for read matter given that read is ultimately unsafe no matter the defaults.

18:11 abedra: craigbro: is it"

18:11 ?

18:11 mattmoss: If read is unsafe while new read-edn is safe, how many will switch or even be aware? If read is made safe and "unsafe-read" is unsafe, people will be very aware. They might just switch to the unsafe one, but that's on them and it's visible in the code.

18:11 That breaks people, but I'm of the school that says you have to break someone to make them stronger. ;)

18:11 dnolen: mattmoss: read will never be safe

18:12 jballanc: hmm...if I want to carry out the same modification on all the values in a hash, but leave the k/v associations in tact...what's the most "Clojure-like" way to do that?

18:12 rhickey: for absolute security use your own reader, written in Ada :)

18:12 craigbro: sure, a backdoor implies intent by the author of the dependency. An unsafe data-reader does not, and could simply be that they didn't think fully about how to read an integer and used read 8^)

18:12 abp: mattmoss, https://groups.google.com/d/msg/clojure-dev/zG90eRnbbJQ/dMjhl5b26l8J

18:12 gfredericks: jballanc: (into {} (for [[k v] m] [k (f v)]))

18:12 mattmoss: abp: YEah, I just skimmed that.

18:12 jballanc: hmm...not update-in?

18:13 hyPiRion: Hurray, rhickey is in mood to emote again.

18:13 craigbro: rhickey: it's the balance of security vs. extensibility. edn gets it correct IMO. I don't bring up bdining *data-readers* to poo poo or to score points

18:13 abp: mattmoss: Thats the answer to unsafe-read

18:13 mattmoss: dnolen: Why not? Maybe I'm missing somehting, but rhickey just indicated that read-edn is safe. So read *could* be safe if it was limited to what read-edn does (or similar), and unsafe-read does what read does.

18:13 gfredericks: jballanc: don't see why you would need it

18:13 abp: mattmoss: "How does one make libraries then, that call read for you? Should there be safe and unsafe variants of every library? Will they compose? Dynamic binding exists precisely for this purpose. The bottom can't be something that hardwires a choice. Rich"

18:13 craigbro: I bring it up to support the argument that you had best fully understand just what code surface is exposed when reading data from untrusted sources.

18:14 ivaraasen: Ada isn't secure enough IMO. should be written in MUMPS. security through obscurity.

18:14 mattmoss: My apologies guys... stuff is over my head, I guess, without spending a lot more time on the issue.

18:14 abedra: craigbro: security is a balance of extensibility, connectivity, and complexity

18:14 craigbro: I think read-edn works great as a default for reading from possibly untrusted sources

18:14 abedra: all have problems

18:15 TimMc: craigbro: I don't think you'll get an argument on that one.

18:16 rhickey: beta9 on its way - read-edn*, old reader, no properties for now

18:17 jballanc: gfredericks: that for comprehension just seems somehow more complicated than it needs to be :-/

18:17 abp: rhickey: Probably docs on read* should be really explicit about security in the first line then. If they aren't by now.

18:17 craigbro: as long time security dork, I find the tippy toe'ing hilarious

18:18 cemerick: craigbro: "tippy toe'ing"?

18:18 abedra: craigbro: ^^

18:18 craigbro: oh my god we gotta do something!

18:18 gfredericks: what's the record vulnerability? just that arbitrary constructors can be called?

18:19 amalloy: gfredericks: "just", yes

18:19 craigbro: tho I suppose the pending 1.5 release is responsible for most of the urgency

18:19 * nDuff is in full agreement with craigbro here -- and that's as someone who's been recently doing a lot of audits of 3rd-party code.

18:19 hiredman: gfredericks: well before then you can cause arbitrary classes to load

18:19 gfredericks: right

18:20 amalloy: as i understand it, calling arbitrary constructors is exactly what the rails yaml vulnerability allowed

18:20 hiredman: clojure namespaces, for example, when aot compiled are just a big fat static init that is run when they are loaded

18:20 gfredericks: so for replacing constructors and #= with data readers, the main pain is backwards compat? or am I missing something?

18:20 abedra: craigbro: as someone who's full time gid is security at a company who uses Clojure, I think it is important to follow up and discuss it. I'm not interested in "tippy toe'ing" whatsoever

18:20 s/gid/gig

18:20 craigbro: abedra: where you at?

18:21 rhickey: gotta run, please try beta9 when it shows up, thanks for the feedback

18:21 abedra: craigbro: Groupon

18:21 Bronsa: no regex for read-edn?

18:21 craigbro: chicago?

18:21 abedra: yes

18:21 nDuff: Heh.

18:21 * nDuff is actually likely to be applying there in 9-10 months.

18:21 craigbro: ah, my hometown, in VT right now tho for a few weeks

18:21 I am also at a suecirty company using clojure, ThreatGRID

18:22 TimMc: Does Clojure currently use #= for anything? Or is it only there as an escape valve for user code these days?

18:23 Perhaps some sort of *print-dup* functionality?

18:23 Frozenlock: TimMc: Apparently 'some mysterious individuals' use it.

18:23 craigbro: TimMc: I recall from a previous discussion that it is used in the compiler and some other places, but I do not concretely know

18:23 hiredman: if you bind *print-dup* to true it is used a lot

18:23 TimMc: OK.

18:23 cemerick: The compiler does not need to use it, though it currently does.

18:23 gfredericks: hiredman: but that could be replaced with an #eval reader, right?

18:23 at worst

18:24 cemerick: sorry, except for tagged literals

18:24 hyPiRion: TimMc: I think I may have a need for it in Swearjure at some point. Don't know what I'll use it for yet though.

18:24 TimMc: (binding [*print-dup* true] (pr-str {:a '[b]})) ;;= "#=(clojure.lang.PersistentArrayMap/create {:a [b]})"

18:24 * nDuff will be having some serious discussion about cost-of-revenue and sustainability during that interview, if it happens, though; having been at Tippr, the market trends in daily-deal space weren't encouraging.

18:24 hiredman: gfredericks: uh, are you proposing adding a reader literal to allow for arbitrary code evaluation?

18:24 gfredericks: hiredman: not adding anything -- you can already add it with binding

18:25 TimMc: Just in dynamic scope.

18:25 gfredericks: yeah

18:25 hiredman: actually I asked rich about this at the conj when he announced reader literals

18:25 Frozenlock: TimMc: looks like (read-string "{:a [b]}" :P

18:25 gfredericks: so you can get nearly equivalent functionality already by opting-in

18:25 so if we only use #= 0.001% of the time then maybe that's okay?

18:26 hiredman: the reason print-dup does that (using #=) is print-dup is very literally "print something that when read in will give me the same thing, classes and all"

18:26 so literals can never be that

18:26 gfredericks: #eval (PAM/create {:a [b]})?

18:26 Frozenlock: So print-dup shouldn't be used if you use clojure on different backends? Cljs/clj

18:27 hiredman: because literals are almost the opposite, here is a semantic tag for the following data, you choose the representation

18:27 gfredericks: that's an interesting point

18:27 hiredman: gfredericks: maybe

18:28 gfredericks: okay; it's sad to me that the reader will be forever unsafe :/

18:28 craigbro: you can't have it any other way 8)

18:28 if you want circular data too work too

18:28 TimMc: read-clj, read-clj-string

18:28 craigbro: hehe

18:29 Frozenlock: circular data?

18:29 hyPiRion: re circular data: I've not yet found a way to let a persistent structure contain itself.

18:29 gfredericks: hyPiRion: lazy seq?

18:29 abedra: /quit

18:30 hyPiRion: gfredericks: boo. I think I meant Maps, Sets or Vectors.

18:30 gfredericks: ,(let [v (promise), h {:foo (lazy-seq [@v])}] (deliver v h) h)

18:30 clojurebot: {:foo ({:foo ({:foo ({:foo ({:foo (#)})})})})}

18:31 gfredericks: hyPiRion: I was actually interested in the ability of repeat and cycle to return seqs that point to themselves

18:31 currently repeat creates lots of new objects when it could get away with one

18:32 hyPiRion: gfredericks: Potential JIRA issue?

18:32 gfredericks: I wasn't convinced the core folk would think it's worth the effort

18:32 craigbro: you know, on the circular data thing, I was confusing the CL reader with clojure

18:32 hiredman: http://stackoverflow.com/questions/357956/explanation-of-tying-the-knot

18:33 tomoj: that would make (= (repeat 3) (next (repeat 3))) true, right?

18:33 hyPiRion: craigbro: yeah, I figured.

18:33 craigbro: with immutable data types, it's not gonna happen

18:33 tomoj: er

18:33 (let [r (repeat 3)] (= r (next r)))

18:33 gfredericks: tomoj: that's an interesting implication

18:33 tomoj: not that anyone should rely on that..

18:33 Frozenlock: ,(let [abc {:a 1 :b (:a abc)}] abc)

18:33 clojurebot: #<CompilerException java.lang.RuntimeException: Unable to resolve symbol: abc in this context, compiling:(NO_SOURCE_PATH:0)>

18:33 gfredericks: it hadn't occurred to me that it could change behavior

18:34 hyPiRion: craigbro: that's not entirely true, Oz has immutable datatypes but have structures which may contain themselves.

18:34 gfredericks: you can do it in haskell amirite

18:34 hyPiRion: Let X be a set containing Y, then unify X and Y.

18:35 hiredman: that is promises though

18:35 which I guess is more less how haskell does it too

18:35 sort of wibbly wobbly on the immutable thing

18:35 hyPiRion: hiredman: Perhaps, but it's more transparent than what Clojure does (you have to deref etc.)

18:35 craigbro: yah, and what happens when you print them?

18:36 we force promises and such when we print, right?

18:36 TimMc: ,(promise)

18:36 clojurebot: #<core$promise$reify__3678@6b6251c1: :pending>

18:36 hyPiRion: craigbro: No, that wouldn't make sense. It should block.

18:36 At least if they were transparent.

18:36 TimMc: ,(doto (promise) (deliver "yep!"))

18:36 clojurebot: #<core$promise$reify__3678@40869efe: "yep!">

18:37 TimMc: ,(let [a (atom nil)] (reset! a a) a)

18:37 clojurebot: #<Atom@3f6f3a0: #<Atom@3f6f3a0: #<Atom@3f6f3a0: #<Atom@3f6f3a0: #<Atom@3f6f3a0: #<Atom@3f6f3a0: #<Atom@3f6f3a0: #<Atom@3f6f3a0: #<Atom@3f6f3a0: #<Atom@3f6f3a0: #>>>>>>>>>>

18:37 craigbro: 8)

18:37 you hit print-level limit I guess?

18:38 TimMc: Yep.

18:38 &(let [a (atom nil)] (reset! a a) a)

18:38 lazybot: java.lang.StackOverflowError

18:38 craigbro: fun

18:38 TimMc: ,*print-level* ##(identity *print-level*)

18:38 clojurebot: 10

18:38 lazybot: ⇒ nil

18:42 craigbro: yah, the explicit nature of promises makes printing them different than in haskell

18:42 the inability to modify a cons cell is what stops you from getting looped lists

18:42 hiredman: you could do it in core.logic :)

18:44 pbostrom_: just catching up, what does it mean exactly that "read will never be safe"? I always assumed (binding [*read-eval* false] (read-string str)) would prevent any sot of nastiness

18:44 s/sot/sort/

18:44 technomancy: pbostrom_: that's correct

18:45 craigbro: pbostrom_: you still have data-readers

18:46 pbostrom_: however, but you can bind *data-readers* as well

18:46 * pbostrom_ googles data-readers

18:47 craigbro: i think of it this way, the YAML.load handling !ruby.class deserialization is like #=

18:48 and issues with data-readers is like what happens when someone adds support for reading YAML to the XML parser

18:49 aka, I can add a data reader for calls (read on a string, expecting a number for example, but I could craft something to get by any protecting regexp and slip it a #= form, and get remote code execution

18:49 so that's like the actual remote code execution bug in Rails

18:50 bbiab

18:54 TimMc: (defn read-really-safe-i-mean-it [s] (binding [*data-readers {}] (read-edn-string s)))

18:55 craigbro: TimMc: i've been scouring the reader for other vectors

18:56 TimMc: I guess we could look at clojurebot code 8)

18:56 TimMc: What do you hope to find there?

18:57 You could also look at clojail.

18:59 craigbro: safe-read in clojail just binds read-eval and munges exceptions

19:00 Raynes: clojurebot is not an example of safe anything.

19:00 Not that clojail is completely safe.

19:00 craigbro: reading edn means you lose #() and some other syntactic sugar, and metadata

19:00 Raynes: But it *tries* to be, hard.

19:01 But there are existing holes.

19:01 aaelony: is there something like clojure.java.io for writing compressed files (from a lazy-seq) ?

19:01 Raynes: I haven't had time to fix them all yet.

19:01 craigbro: Raynes: just looking at the read cycle here

19:01 Raynes: But they aren't related to read.

19:01 Bronsa: EdnReader also reads "foo`bar" as a symbol

19:01 Raynes: Just saying, clojurebot relies mostly on restarting and people here not being asshats.

19:02 Bronsa: while LispReader reads "foo`bar" as the symbol foo, and the syntax-quoted symbol bar

19:02 Frozenlock: aaelony: you could try https://github.com/Raynes/fs/blob/master/src/me/raynes/fs/compression.clj

19:03 aaelony: I'm looking at that but likely need make-zip-stream somehow

19:03 thanks

19:03 Raynes: That compression stuff could probably do with a makeover.

19:04 aaelony: something Buffered would be cool...

19:04 this way, i learn java slowly but surely...

19:27 zakwilson: A local meetup group has suggested writing the *nix ls program in a functional language and asked me to talk about Clojure since I already use it actively. This kind of implies I should write ls in Clojure. Of course, JVM startup time is bad. I know there have been a few attempts to run Clojure with a persistent JVM. Is there one I should actually use?

19:28 seangrov`: Well, repl-dev would be good for it

19:29 amalloy: huh. i didn't expect this: ##(* 60 1/30

19:29 hiredman: zakwilson: drip maybe?

19:29 Raynes: evalfail

19:29 amalloy: &(* 60 1/30)

19:29 lazybot: ⇒ 2N

19:29 amalloy: zakwilson: the only one i'd recommend trying is drip. it probably even works

19:29 Raynes: Drip might help, but not significantly I wouldn't think. Especially if you ran it a bunch of times in quick succession.

19:30 zakwilson: One thing I've noticed is that if you don't load a bunch of stuff and you don't use lein to for starting it up or anything, the startup is actually not too bad.

19:31 zakwilson: Another option might be using a different implementation of Clojure.

19:31 amalloy: Raynes: drip actually has a DRIP_POOL option that lets you keep more than one jvm waiting

19:31 i don't think it's well tested, though

19:31 zakwilson: It looks like drip makes the basic startup about ten times faster. That's pretty much what I'm looking for.

19:31 Raynes: Interesting.

19:31 amalloy: How much memory does an idle JVM take?

19:32 zakwilson: It wouldn't be suitable for sticking in the middle of a loop in a script, but actually doing that would be silly. It'll do for a demo.

19:32 amalloy: *shrug* mostly it gets paged out, i imagine

19:33 ToBeReplaced: is there a clojure/clojurescript mustache implementation? or is anyone working on one?

19:33 Frozenlock: Raynes: Even if noir is deprecated, would you consider upping its dependencies? This is scary https://github.com/ring-clojure/ring/commit/7028d12759ababdcd523ed0881b79ecc0b38f334

19:34 Raynes: Frozenlock: I can do security updates that other people contribute.

19:35 qz: is it possible to get that binding *read-eval* false behavior by default?

19:35 * Raynes chuckles

19:35 hyPiRion: qz: Yeah, just do (alter-var-root #'read-eval (constantly false)) at the start of your program.

19:36 qz: cool :)

19:42 craigbro: ok

19:43 well #"" syntax does open up a possibl denial of service

19:43 TimMc: Good point.

19:44 ChongLi: craigbro: as does using the wrong sorting algorithm (eg quicksort)

19:44 craigbro: ok, but on read...

19:44 qz: isnt #"" a regexp?

19:44 craigbro: not on evaluation. I'm not talking about exploits when you do something with the data

19:45 ChongLi: if the user is supplying the data, they can supply pathological anti-sorted data

19:46 craigbro: ChongLi: I'm only worried about what gets passed to the pattern compiler, not after

19:46 ChongLi: also, computation complexity in a regexp is much higher

19:46 Frozenlock: Raynes: I'm far from being a pro with pull request; let me know if it's what you had in mind https://github.com/noir-clojure/noir/pull/139

19:46 craigbro: https://www.owasp.org/index.php/Regular_expression_Denial_of_Service_-_ReDoS

19:46 ChongLi: basically any algorithm that operates on user data which has terrible worst-case complexity is a risk

19:47 craigbro: ChongLi: yup

19:47 Raynes: Frozenlock: Yeah. I the only other thing I'd ask for is that you made sure it still works.

19:47 Frozenlock: Aw I knew there was a trap.

19:47 Raynes: lol

19:48 If you can find any given noir app that is based on 1.3.0...

19:48 Could clone refheap and checkout to pre-compojure.

19:48 * Frozenlock will be bold and try lein test

19:48 Raynes: I don't know how useful the tests ever were.

19:48 It isn't really a big deal. I can test it out later.

19:49 Frozenlock: Well then... I'll just clojar it and try it on my own server.

19:49 Raynes: Since I know how to work refheap.

19:49 But yeah, I'd accept "Noir app x still works" as 'testing' it. :p

19:50 Frozenlock: Wouldn't that be the same as testign with refheap?

19:50 Raynes: Yes.

19:54 toby3: hello

19:54 anybody here?

19:54 nDuff: Usually.

19:54 Frozenlock: toby3: There's always someone here.

19:55 toby3: oh, soory

19:55 sorry.

19:55 nDuff: toby3: that said, we're a high-latency channel; just because we're here doesn't mean we answer right away, or that we want to be pressed to do so.

19:55 toby3: no one is speaking, i thought i had not connected

19:56 Frozenlock: toby3: We are busy doing stuff :P

19:56 Like I, for example, am busy breaking the Noir library.

19:56 clojurebot: library is "Our class libraries have destroyed our brains." - rhickey

19:56 toby3: ok

19:57 not understood exactly

19:57 JanxSpirit: sorry for the newb question but why isn't the say-foo function visible in this code? http://pastebin.com/GFFh9Ln5

19:58 nDuff: JanxSpirit: Could you consider a pastebin without all the ads?

19:58 JanxSpirit: sorry for that too

19:58 nDuff: JanxSpirit: refheap.com (written in Clojure and maintained by one of our own) would be ideal.

19:58 ibdknox: JanxSpirit: order matters in Clojure

19:59 JanxSpirit: you have to declare something before it is used

19:59 JanxSpirit: doh

19:59 toby3: what is refheap used for ?

19:59 JanxSpirit: should have known that

19:59 thanks ibdknox

20:02 qz: is there any good way to avoid mass-require of stuff that's common? for example nearly every my file requires clojure.string or clojure.java.io or such..

20:03 craigbro: ns+

20:03 tomoj: editor snippets :P

20:03 amalloy: qz: no. files must declare what they depend on, or else how could a reader (human or computer) know where some arbitrary function comes from?

20:03 tomoj: ns+ is bad

20:04 qz: amalloy: well, that ns+ approach solves that. too bad its not in core

20:05 tomoj: why ns+ is bad?

20:07 TimMc: qz: slamhound :-)

20:07 scgilardi: you take that back

20:07 :)

20:09 tomoj: also codeq?

20:10 though that leads to "macros are bad"

20:10 eventually you've gotta have the code actually loaded up anyway, so might as well expand ns+ ?

20:18 craigbro: sorry to go back to this topic

20:18 but I saw some mention about a record constructor bug

20:18 right now read will let you call constructors

20:19 on any loaded java class

20:20 nDuff: craigbro: You were there when the entire damned afternoon was spent talking about that, no? :)

20:20 s/there/here/

20:20 craigbro: yah, but I didn't see the resolution of that record constructor issue. was that going to be changed?

20:21 nDuff: craigbro: You could before, and will continue to in the future, be able to turn that off.

20:21 craigbro: moreover, in the future, there'll be an EDN-only read call, which also won't allow that.

20:22 craigbro: nDuff: uhm, binding *read-eval* false doesn't turn off record constructors

20:22 nDuff: Oh. Then use the EDN reader.

20:23 craigbro: nDuff: sure, assuming that only uses data-readers. I'm asking because this makes read-string and read unsafe no matter what binding there is for *read-eval*

20:24 nDuff: Okay. Grok your question now. Correct answer is "I don't know".

20:24 Apologies about being flippant before.

20:25 craigbro: dude, after this afternoon, no apology needed 8)

20:28 ,(read-string "#java.io.FileWriter[\"/tmp/foo\"]")

20:28 clojurebot: #<SecurityException java.lang.SecurityException: denied>

20:28 craigbro: hehe

20:29 I can touch or overwrite arbitrary files that way

20:30 ,(bindings [*read-eval* false] (read-string "#java.io.FileWriter[\"/tmp/foo\"]"))

20:30 clojurebot: #<CompilerException java.lang.RuntimeException: Unable to resolve symbol: bindings in this context, compiling:(NO_SOURCE_PATH:0)>

20:30 craigbro: ,(binding [*read-eval* false] (read-string "#java.io.FileWriter[\"/tmp/foo\"]"))

20:30 clojurebot: #<SecurityException java.lang.SecurityException: denied>

20:30 Frozenlock: Raynes: well I don't know what I did wrong, but it chokes on "hiccup/page_helpers". (Which seems you were using in 1.3.0 anyway... weird)

20:30 Raynes: Frozenlock: Don't worry about it. I'll take a look asap.

20:34 matthavener: craigbro: so basically, *read-eval* false just disables #=, but you can still call abitrary ctors?

20:37 devn: the *read-eval* thread rages on

20:37 yogthos|away: i'd like to talk to you if you have a chance -- priv msg me?

20:41 craigbro1: ok, I was looking at an old 1.5 clojure

20:42 TimMc: matthavener: The idea is that there will be a separate reader for just "EDN", a safe Clojure subset.

20:42 craigbro1: the thing I just posted, is fixed in 5cf6aa3f

20:42 sorry

20:43 durka42: anyone use nakkaya's vision library?

20:43 (the opencv wrapper)

21:03 tomoj: (persistent! (reduce-kv (fn [ret k v] (if (nil? v) (dissoc! ret k) ret)) (transient m) m))

21:03 (into-kv m (r/keep identity m))

21:03 :(

21:04 oh, that won't work :D

21:04 (into-kv {} (r/keep identity m)) I guess

21:13 loganlinn: what does ANN stand for (in the clojure google group)?

21:13 tmciver: Announcement

21:13 loganlinn: ok

21:13 thats what I thought

21:14 amalloy: arbitrary news network

21:14 aggregated noisy nonsense?

21:15 Raynes: Exceptionally clever.

21:28 semisight: hey all

21:28 I found a bug in clojure.java.jdbc

21:28 and I wanna patch it

21:28 but I'm having trouble

21:28 can anyone help?

21:30 jeremyheiler: semisight: there's a lot of work that has been done on java.jdbc lately. is your bug regarding the latest master?

21:31 semisight: as far as I know it exists in the latest master yes

21:31 I know there's being work done in the dsl

21:31 but this is in one of the older functions

21:33 jeremyheiler: what's the issue? (ping: seancorfield )

21:34 semisight: jeremyheiler: the insert-values function doesn't correctly handle column names that have special characters

21:34 I have column names with ampersands and the like, but these should be quoted before an insert statement

21:34 if I'm right this is a one line fix

21:35 xeqi: semisight: have you signed a CA perchance?

21:35 semisight: xeqi: nope

21:36 xeqi: blah

21:36 semisight: xeqi: does that make things harder?

21:37 I can work around this

21:37 but I thought I'd contribute a fix

21:37 xeqi: clojure.java.jdbc is under the contrib umbrella, which means it can't accept patches without a signed and mailed CA, even for a 1 line fix

21:37 jeremyheiler: semisigh: It appears insert-values is deprecated for 0.3.0, anyway

21:38 semisight: xeqi: damn. can anyone contribute it for me?

21:38 jeremyheiler: semisight, Especially since you don't have a CA, your best bet is to get in touch with seancorfield

21:39 semisight: jeremyheiler: From what I understand it's deprecated for a future release. The current release has no replacement.

21:39 xeqi: heh, I'd recommend telling seancorfield what the issue is, he's the maintainer and could handle it

21:39 semisight: xeqi: ok, I'll try that

21:39 thanks

21:40 xeqi: also, feel free to mention you wanted to contribute a fix, but couldn't to the clojure google group :p

21:40 * xeqi tries to start the second war for the week

21:55 TimMc: xeqi: I thought that was usually on Wednesdays.

21:55 xeqi: by the time it makes it through group moderation ....

21:56 TimMc: heh

21:56 Always remember to lead your target.

21:58 craigbro: heh

21:59 nonuby: how can I get from this input to this output with the least cognitive load (i.e. avoiding nesting maps) https://www.refheap.com/paste/9534

21:59 essentially I just adding the key as the id

22:00 yedi: what exactly does C-c C-l (load file) do in nrepl-jack-in?

22:00 hiredman: for

22:00 yedi: i'm really struggling to get this repl workflow right

22:00 amalloy: &(doc for) ; nonuby

22:00 lazybot: ⇒ "Macro ([seq-exprs body-expr]); List comprehension. Takes a vector of one or more binding-form/collection-expr pairs, each followed by zero or more modifiers, and yields a lazy sequence of evaluations of expr. Collections are iterated in a nested fashion, rightm... https://www.refheap.com/paste/9535

22:04 TimMc: This EdnReader, is it just an eval-purge of the regular LispReader?

22:05 nonuby: first attempt: (flatten (for [topkey (keys a) toparray (topkey a)] (map #(merge {:id topkey} %) toparray)))

22:05 xeqi: TimMc: it looks similar

22:05 ~flatten

22:05 clojurebot: flatten is rarely the right answer. Suppose you need to use a list as your "base type", for example. Usually you only want to flatten a single level, and in that case you're better off with concat. Or, better still, use mapcat to produce a sequence that's shaped right to begin with.

22:06 amalloy: (for [[k vs] m, v vs] (assoc v :id k)), nonuby?

22:08 nonuby: amalloy that works, will digest why/how, thanks, much cleaner than what I had

22:11 on (doc for) what does it mean by rightmost fastest?

22:12 craigbro: TimMc: a littel bit more than that

22:13 TimMc: Looking at the code, it appears to be a derivative version. I guess it would not have made sense to do it any other way.

22:14 craigbro: TimMc: way few dispatch macros, no record ctor, no deref macro, no anon fn macro, no quote, no unquote, no splice, no regex reader, no set reader, no comment read

22:15 xeqi: theres a set reader and comment reader

22:16 TimMc: WTF is RT/suppressRead?

22:16 craigbro: xeqi: you are right, there is a #{ set reader macro. there is no #! comment reader

22:16 xeqi: ah, I saw the ; comment reader

22:17 craigbro: it's always false 8)

22:19 TimMc: ,(clojure.lang.RT/suppressRead)

22:19 clojurebot: false

22:20 TimMc: I don't think I would hire someone whose code looked like this.

22:21 xeqi: I wonder how long it will take libraries (like ring) to give up backwards compatibility of clojure versions

22:23 craigbro: TimMc: the tabs?

22:24 TimMc: craigbro: The complete lack of comments in very important code.

22:26 technomancy: heh; lead you target. nice. =)

22:27 gfredericks: rich just complected a close-paren with an emoticon

22:27 craigbro: TimMc: it's pretty straightfoward code

22:27 TimMc: suppressRead aside 8)

22:28 callenbot: TimMc: what code? where?

22:28 oh, clojure.lang

22:28 TimMc: https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/RT.java#L1710

22:28 //todo - look up in suppress-read var

22:29 gfredericks: lazybot: why you no handle TODOs

22:33 Sleepy_neko: hi

22:33 i'm new to clojure

22:33 gfredericks: sweet

22:34 TimMc: callenbot: Quite.

22:37 craigbro: ,(::foo)

22:37 clojurebot: #<IllegalArgumentException java.lang.IllegalArgumentException: Wrong number of args passed to keyword: :sandbox/foo>

22:37 craigbro: ,::foo

22:37 clojurebot: :sandbox/foo

22:37 craigbro: :/foo

22:37 ,:/foo

22:37 clojurebot: #<ExecutionException java.util.concurrent.ExecutionException: java.lang.RuntimeException: Invalid token: :/foo>

22:37 craigbro: sandbox:/foo

22:38 ,sandbox:/foo

22:38 clojurebot: #<ExecutionException java.util.concurrent.ExecutionException: java.lang.RuntimeException: Invalid token: sandbox:/foo>

22:38 craigbro: ,sand/foo:

22:38 clojurebot: #<ExecutionException java.util.concurrent.ExecutionException: java.lang.RuntimeException: Invalid token: sand/foo:>

22:38 mattmoss: hacker

22:39 :)

22:39 warz: ,(+ 1 2 3)

22:39 clojurebot: 6

22:39 warz: woot

22:39 craigbro: ,:::foo

22:39 clojurebot: #<ExecutionException java.util.concurrent.ExecutionException: java.lang.RuntimeException: Invalid token: :::foo>

22:40 mattmoss: ,:x:x

22:40 clojurebot: :x:x

22:40 gfredericks: oh man clojurebot has an eval vulnerability

22:40 mattkruse: ,(+ (+ 1 2) (+ 3 4))

22:40 clojurebot: 10

22:40 gfredericks: if you type code preceded by a comma it will eval it

22:40 craigbro: ,::foo

22:40 clojurebot: :sandbox/foo

22:40 TimMc: "People have been using read for something for which it is not suited."

22:40 Sleepy_neko: ,(+ 1 2)

22:40 clojurebot: 3

22:40 mattkruse: cool beanz

22:40 TimMc: And that's just after "For instance, programs that write programs often need to embed arbitrary unknown objects into code. I get the feeling a lot of people ready to change things are not leveraging many of Clojure's capabilities in this area."

22:40 >_<

22:40 mattmoss: ,(+(*)(+(*)(*)))

22:40 clojurebot: 3

22:41 mattkruse: how is that three?

22:41 mattmoss: ,(*)

22:41 clojurebot: 1

22:41 TimMc: How isn't it?

22:41 Sleepy_neko: ,(@)

22:41 clojurebot: #<ExecutionException java.util.concurrent.ExecutionException: java.lang.RuntimeException: Unmatched delimiter: )>

22:41 craigbro: TimMc: where thsoe quotes from?

22:41 mattkruse: ,(* 2 3)

22:41 clojurebot: 6

22:41 gfredericks: craigbro: ML

22:42 craigbro: the battlefield!

22:42 holdon, lemme get some scotch first

22:42 gfredericks: I wonder what use cases he's thinking of that can't be handled by a macro

22:42 TimMc: I can't link to this goddamn new Google Groups.

22:42 https://groups.google.com/d/msg/clojure-dev/zG90eRnbbJQ/Q2sJTbNAAqYJ there

22:43 craigbro: gfredericks, anyone where the consuming program does not have all the code of the sending program

22:43 Frozenlock: Perhaps there should be an example of the awesome capabilities of #= and alike?

22:43 craigbro: gfredericks: or, anyone that depends on *print-dup* being true when spat out (aka, the same types being read in as were printed)

22:43 gfredericks: craigbro: there could be a built-in macro? clojure.core/construct

22:44 craigbro: gfredericks: then it is the same as #= no?

22:44 gfredericks: no because the compiler does it

22:44 not the reader

22:44 ppppaul: can someone help me with a clojure.data.xml.zip question?

22:44 https://gist.github.com/boxxxie/4720053

22:44 Frozenlock: From the google group:

22:44 gfredericks: may as well just have something called compile-eval to be totally general

22:44 Frozenlock: For the future 1.6 release, we can be much more clean in how we separate things out, e.g.:


22:44 read : Reads data only, no evaluation. Works for most "normal" Clojure code.

22:44 read-edn : Reads edn data, with whatever tagged value handling is chosen

22:44 craigbro: that it goes thru data-reader dispatch would be only difference?

22:44 Frozenlock: read-clojure : Read with support for #=(...), other dangerous stuff that might be in general Clojure code. Power users / the Clojure REPL will use this.

22:44 gfredericks: craigbro: no it'd be a regular macro

22:44 craigbro: hmm, not getting this "compiler doing it" but

22:44 ppppaul: i want to reuse zip locations with xml->

22:44 craigbro: s/but/bit

22:45 gfredericks: craigbro: that means after read-string nothing has happened

22:45 craigbro: data readers are handled by the compiler

22:45 gfredericks: ,(read-string "(compile-eval (MyType. 1 2))")

22:45 clojurebot: (compile-eval (MyType. 1 2))

22:45 craigbro: I mean data readers are handled by the darn reader, doh

22:45 sorry, long day

22:45 TimMc: ,(read-string "١٢٣٤٥")

22:45 clojurebot: #<NumberFormatException java.lang.NumberFormatException: Invalid number: ١٢٣٤٥>

22:45 gfredericks: (defmacro compile-eval [x] (eval x))

22:46 technomancy: (inc TimMc)

22:46 lazybot: ⇒ 33

22:46 TimMc: ,(read-string "123")

22:46 clojurebot: #<NumberFormatException java.lang.NumberFormatException: Invalid number: 123>

22:46 craigbro: hehe

22:46 TimMc: Good ol' Character/isDigit and its well-meaning users.

22:46 technomancy: western imperialism!

22:47 craigbro: gfredericks: then you have to do some partial evaluation yourself, to do it safely, or eval everything

22:48 gfredericks: craigbro: what's "everything"?

22:48 craigbro: the entire thing you read in

22:48 gfredericks: I think if you're compiling code you're assuming it's safe

22:49 the point is it gets eval'd at compile-time so that the constructed object gets embedded in your code, which is apparently an important capability that I don't know any uses for

22:49 craigbro: if you eval everything, then some things need to be quoted, anything not in that macro for example

22:49 gfredericks: you don't eval everything

22:49 just what's in the macro

22:49 which is what you're asking for by using the macro

22:49 craigbro: I think we're talking across purposes here 8)

22:50 TimMc: I wonder if one could confuse the reader by frobbing the PushbackReader while read is happening.

22:50 craigbro: I'm talking about #= eval-read, which is used by *print-dup* for example

22:50 tbaldridge: my emacs just bugged out. It's trying to indent by lining up all args under the first argument. I'd like to swich back to the 2 space indent. Any ideas?

22:50 craigbro: and let's me preserve type information about the stuff I prit out, without squashing it to clojure primitive types

22:50 TimMc: you mean in a dastardly way?

22:50 tomoj: tbaldridge: huh? first arg is the default

22:51 TimMc: Yes.

22:51 craigbro: TimMc: how do you get a ref to the PushBackReader to do dastardly things, but not also already have code exec?

22:51 tbaldridge: http://stackoverflow.com/questions/8662687/emacs-auto-indent-in-clojure-do-it-like-cl I'd like the first example. the second is what I have now

22:51 TimMc: craigbro: I don't envision an exploit.

22:52 gfredericks: craigbro: so you do #=(MyType. 1 2) and I do (compile-eval (MyType. 1 2)) -- in what important respect are these different?

22:52 craigbro: [1 2 3 #=(MyType 4)]

22:52 I read that, and I have my data

22:52 tomoj: tbaldridge: weird, clojure-mode does the first by default. did you add any config changing indentation settings...?

22:52 tbaldridge: not that I know of. I'll try restarting emacs

22:52 gfredericks: craigbro: if you're reading data why not use data readers instead?

22:53 or just eval if you really like evaling

22:53 craigbro: if you use your macro mechanism, I then have to run something over the result, making (compile-evals run

22:53 tomoj: tbaldridge: specifically for "(if ...)" ?

22:53 TimMc: Hmm, EdnReader duplicates ReadException.

22:53 tomoj: not "(foo/if ...)" or "(something-that-isnt-if ...)"?

22:53 TimMc: ReaderException, rather

22:53 gfredericks: craigbro: I'm trying to address rich's use case, which is code compilation; yours is handled by data readers and eval proper

22:54 craigbro: gfredericks: data-readers means that the receiving side has to know how to construct my objects entirely

22:54 meaning, we have to sync our ontologies

22:54 gfredericks: okay so use eval then

22:54 craigbro: or, have a general purpose data-reader, in which case #= already does it

22:54 do you think this is what CL standards committee meetings were like?

22:55 hehe

22:55 amalloy: tbaldridge: for most functions, the second is normal; if and various other functions indent the first way. are you saying that everything is indenting like that now?

22:55 tbaldridge: tomoj: yeah, when I originally type forms, they show up correctly but then when I run CTRL+ALT+\ it switches everything to the 2nd version

22:56 tomoj: don't use that, use paredit-reindent-defun

22:56 I think it's M-q by default

22:56 craigbro: TimMC: pretty sure it's scoped to EdnReader tho

22:56 aka, it's really EdnReader/ReaderException (or however you notate that in java)

22:56 tomoj: well.. I assume you're using paredit :)

22:57 and perhaps indent-region is not the problem

22:57 tbaldridge: tomoj: that did it, thanks

22:57 craigbro: gfredericks: are you expecting me to go read the ML so we can talk about the same thing now???

22:57 lazybot: craigbro: How could that be wrong?

22:57 craigbro: ok, scotch acquired, I can read it

22:57 gfredericks: craigbro: no, just not very motivated

22:58 TimMc: craigbro: Yes, but I wonder if there's Clojure code out there that will assume all the read* fns use the same exception class.

22:59 craigbro: well, since read-edn* don't exist yet, it would be dumb 8^)

23:00 since ReaderException is also private to LispReader...

23:00 err, contained in..

23:00 TimMc: public

23:00 (try ... (catch clojure.lang.LispReader$ReaderException re ...))

23:01 craigbro: there we go

23:01 $

23:03 proger79: A newbie question: Is it possible to get function object MessageBox/Show without using '/' to use it as: (def mb (??-??-?? "Show" "MessageBox" "System.Windows.Forms.dll")) in ClojureCLR?

23:06 craigbro: TimMc: which ML is that one?

23:07 TimMc: craigbro: Which one?

23:07 clojure-dev is probably what you're looking for, if it's about read-eval

23:10 Well, I don't see any code in my checkouts relying on LispReader$ReaderException that should be switched over to EdnReader's version, so I guess that's OK?

23:10 &(name 'foo/bar/baz)

23:10 lazybot: ⇒ "baz"

23:11 craigbro: TimMc github code search 8)

23:17 TimMc: &(identical? 'foo 'foo)

23:17 lazybot: ⇒ false

23:17 Frozenlock: wut

23:17 aren't they the same symbol?

23:18 TimMc: No, but I'm confused now because I see a Symbol.intern() call in the reader.

23:18 xeqi: &(= 'foo 'foo)

23:18 lazybot: ⇒ true

23:19 TimMc: Ah, Symbol.intern(...) interns the components.

23:19 &(identical? (name 'foo) (name 'foo))

23:19 lazybot: ⇒ true

23:20 Frozenlock: TimMc: Stop it! You're confusing me!

23:20 craigbro: &(^"bar" 'foo)

23:20 lazybot: clojure.lang.ArityException: Wrong number of args (0) passed to: Symbol

23:20 TimMc: Frozenlock: The reader calls this: https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/Symbol.java#L59

23:20 craigbro: ,^"bar" 'foo

23:20 clojurebot: foo

23:21 craigbro: TimMc: the two symbols not being identical gets me

23:21 &(= 'foo ^"bar" 'foo)

23:21 lazybot: ⇒ true

23:22 craigbro: but a robot once told me I don't understand how symbols work in clojure

23:22 so it may be correct

23:22 amalloy: that's what keywords are for

23:22 TimMc: Symbols aren't really intended for use in identity comparison, unlike keywords.

23:23 &(identical? :foo (keyword 'foo))

23:23 lazybot: ⇒ true

23:23 craigbro: it's CL brain damage I am trying to recover from

23:23 gfredericks: craigbro: it's cuz they gotsta have metadata

23:24 TimMc: gfredericks: That explanation used to satisfy me, several hours ago.

23:24 gfredericks: o_O

23:24 TimMc: are you having some sort of clojure crisis?

23:24 craigbro: gfredericks: sure, but why not have them work like keywords, as in, once intered, you always get the same one back by name?

23:24 TimMc: More work, is my guess.

23:24 gfredericks: craigbro: what would (with-meta 'foo {:bar :baz}) give you?

23:25 craigbro: explosion!

23:25 TimMc: gfredericks: '[foo foo] could safely give you back two references to the same object. Why doesn't it? Extra work!

23:25 amalloy: yeah, all interning symbols would do for you is take up space and time, while providing the false expectation that they're always interned

23:25 gfredericks: TimMc: well that part I don't object to

23:26 but like amalloy says it doesn't matter all that much

23:26 craigbro: one of those edge cases that only gets you when doing some metaprogramming

23:26 TimMc: = is fine

23:26 gfredericks: okay fellahs I'ma set read-eval to true and go to bed

23:26 craigbro: it makes writing "DSLs" (hate the word) a bit more difficult

23:27 amalloy: i can't imagine why, craigbro

23:27 tpope: what about keywords, are those interned?

23:27 gfredericks: yes

23:27 TimMc: You betcha.

23:27 That's what they're for, really.

23:28 gfredericks: or rather, as TimMc said, "You betcha."

23:28 TimMc: ,(Keyword/find 'foo)

23:28 clojurebot: #<CompilerException java.lang.RuntimeException: No such namespace: Keyword, compiling:(NO_SOURCE_PATH:0)>

23:28 TimMc: ,(clojure.lang.Keyword/find 'foo)

23:28 clojurebot: nil

23:28 TimMc: ,:foo

23:28 clojurebot: :foo

23:28 TimMc: ,(clojure.lang.Keyword/find 'foo)

23:28 clojurebot: :foo

23:28 amalloy: you're quite lucky nobody had tried :foo recently, or that example would be a bit sillier

23:29 TimMc: I actually thought I *had* tried it recently. :-P

23:29 tpope: that matches my assumption

23:29 I had never stopped to ponder whether symbols were interned

23:29 but the fact they aren't still comes as a surprise

23:31 TimMc: I wonder why they intern their ns and name.

23:32 amalloy: TimMc: fast equality comparisons

23:32 i assume so, anyway. i can't imagine any other reason

23:32 possibly also to reduce the amount of space taken up by N different copies of the same symbol: they can't be pointing at different strings

23:33 TimMc: Right.

23:34 So each 'foo/bar allocates at most one new object instead of 3. (Well, + hashcode.)

23:35 I see that the toString caches its return value.

23:35 *at best

23:43 yedi: why is software so absurdly difficult to set up sometimes?

23:44 amalloy: because we're just monkeys

Logging service provided by n01se.net