#clojure log - Nov 29 2009

The Joy of Clojure
Main Clojure site
Google Group
List of all logged dates

1:18 Jomyoot: What's simplest way of loading a webpage from clojure?

1:20 hiredman: loading as in pulling down?

1:21 http://gist.github.com/119375 there maybe something in contrib

1:21 Jomyoot: yes

1:21 loading and saiving it

1:21 _ato: use duck-streams and (slurp* "http://clojars.org/") if you want a string

1:22 hiredman: if you want to drive a website, htmlunit is pretty nice

1:24 Jomyoot: IS there no built in or contrib?

1:24 _ato: (with-open [out (java.io.FileOutputStream. "/tmp/index.html")] (copy (.openStream (java.net.URL. "http://clojars.org/")) out))

1:25 Jomyoot: clojure doesn't replace the java standard library

1:26 (again copy is in clojure.contrib.duck-streams)

5:32 Jomyoot: Is there a data store built using clojure?

5:42 _ato: Jomyoot: there's tons of options for data stores, depending on what kind of data store you need: cupboard (clojure data structures on BDB), contrib.sql/clojureql/clj-record (relational databases), mongocongo (mongodb doc db), clutch/clojure-couchdb (couchdb doc db), redis-clojure (redis key-value store), clojure-rdf/rdfm (RDF tuple store), sdb (amazon simpledb)

5:42 Jomyoot: bdb has that license thing right?

5:43 _ato: license thing?

5:44 http://www.oracle.com/technology/software/products/berkeley-db/htdocs/jeoslicense.html

5:44 ^ that's the license

5:44 Jomyoot: hmm

5:44 is that equivalence of?

5:46 _ato: it's sort of similar to the GPL

5:46 Jomyoot: LGPL?

5:46 _ato: not exactly

5:46 Jomyoot: does it force my app into opensourcing too?

5:46 _ato: yes

5:48 Jomyoot: if you're just storing small amounts of data, simply writing Clojure data structures to a string with "prn" and reading them back with "read" should work fine

5:50 where "small" means "fits in RAM" and your happy to keep it all in RAM

5:52 Jomyoot: too small :(

5:55 _ato: oh and of course you can use lucene as data store if you want full-text search, which I don't think there's any wrapper for but it's easy to use from clojure

9:22 ankou: hi, is there sth. similiar to a zip function? (sth. like (zip [:a :b] [:c :d]) => [[:a :c] [:b :d]]

9:23 nanodust: ,(map vector [1 2] [3 4])

9:23 clojurebot: ([1 3] [2 4])

9:24 ankou: ah okay

9:37 arj: I'm trying to read a list of structs using read-string from a file that I dumped with prn, but I get Unreadable form. Any ideas?

10:02 got it :)

10:02 was because I had a File stuck in the structs

10:20 wlr: ankou: not what you asked for but sometimes handy is (zipmap [:a :b] [1 2]) => {:b 2, :a 1}

12:50 aldebrn: Trying to build VimClojure; I get the standard error when attempting to build: "java.lang.ClassNotFoundException: clojure.contrib.pprint.PrettyWriter (pprint.clj:14)" despite having built the clojure-contrib.jar using "ant -Dclojure.jar=/path/to/clojure-1.0.0.jar"

13:07 chouser: aldebrn: make sure your local.properties file has the right path to your clojure-contrib.jar

13:08 aldebrn: if it does, then look inside your clojure-contrib.jar to make sure it has a PrettyWriter.class file in it.

13:13 rhickey: urk - paste.lisp.org down?

13:14 I've been planning protocol defs in deftype, defprotocol gens interface when AOT

13:15 chouser: hm!

13:16 method sigs in deftype but not implementations

13:16 ?

13:16 rhickey: http://gist.github.com/244998

13:16 defprotocol will define sigs, deftype can have impls

13:16 so impls of interfaces and/or protocols

13:17 chouser: I bet if you wrote on the google group with a list things to be documented, or even the task to produce a list of "major changes" since 1.0, people would jump right in.

13:17 * chouser reads the gist

13:17 rhickey: when AOT-compiled, defprotocol generates coresponding interface

13:18 so we need grouping per protocol/interface

13:18 this is better for interfaces too, because CLR doesn't merge by name the way Java does

13:19 deftype will see if the protocol has an interface and gen different code if it does/doesn't

13:19 chouser: ok, I see it. it'll do that for each protocal/interface group named in the deftype.

13:20 rhickey: right

13:20 still not sure about the this and . stuff

13:20 it is a very clean story without interfaces or interop though

13:21 essentially, if you know you need to support some protocols when you define the type, you can do so right there, for some perf advantage

13:21 AOT compiling of protocols enables the perf advantage

13:21 but perf advantage still there if dynamically re-deftyping

13:22 only hitch is dynamic re-defprotocol

13:23 need some change-detector, if unchanged interface will still work, if changed it won't, and dependent code (impls of the protocol and callers) will need re-eval

13:23 but no restart

13:24 the let in the extend-type is to allow the body to presume implicit fields, for lack of symbol-macros

13:25 but will this unification confuse people?

13:25 will the presume they can reify protocols?

13:25 or extend-type to interfaces?

13:30 chouser: The potential for confusion is definitely there.

13:30 But when the required code to take advantage of these features is so simple and consistent, I'd like to believe it's just a matter of explaining it properly.

13:31 rhickey: the alternative is explicitly created definterfaces

13:31 deftype can only implement interfaces

13:31 defprotocol gets back :on AnInterface

13:31 chouser: Most people don't need to fully understand the exact requirements for re-eval in order get best performance.

13:32 They'll play with it at a REPL to get the functionality working, but if the performance matters on deployment will at least re-eval everything if not just restart the JVM.

13:33 rhickey: I had hoped to 'fix' a few interface things by having the interface under the hood - first, name munging, second, possibly protocol-prefixing names to avoid name merging, third, since deftype is implementing protocol, no isA presumption

13:33 chouser: it ends up all the perf will be there as long as you don't re-defprotocol

13:33 presuming you AOTed the protocol

13:33 so still pretty dynamic

13:34 chouser: the few who really care about maximum perf with minimal re-eval will be willing to dig in and learn the details.

13:34 rhickey: being able to 'reach' a protocol with reify is interesting and useful

13:35 not a good story with the implicit interfaces

13:36 so, I'm on the fence - :on is in there and fast, and deftype does interfaces, could be done right now, just seems like a lot more work to make protocol 'fast', seems second-class to interfaces

13:37 and the code change to move from external extend-type to impl interface in deftype is not nothing

13:38 chouser: yes, as it stands you could never AOT during dev ad just AOT for delivery with better perf

13:39 with the gist code a clojure-only app with no interop requirement need never talk about interfaces

13:39 Chousuke: hmm.

13:39 rhickey: you would just be told implementing a protocol in deftype is easier and faster if you can do it

13:40 extend-type just for types you don't control

13:41 reify might be made to understand the munging/interface game and work with protocols

13:45 mscala: hi all. quick question: I've just found google's android scripting environment, and now have a python intwrpreter running n my droid. is there any hope that there will be a clojure interpreter?

13:45 rhickey: chouser: the re-eval of impls and callers on re-eval of protocol won't be optional for perf, the old code will be used until re-eval

13:51 Chousuke: I think defining the protocols and interfaces together with the type is going to be quite a common case, so it makes sense to make it fast as well.

13:52 the-kenny: mscala: I think someone has managed to run clojure on android.

13:53 mscala: ah, good. google hasnt turned up any info yet.

13:54 the-kenny: mscala: There's an example-project somewhere on github. The author mentioned the slowness of the program ;)

13:57 noidi: how could I unzip a zipper up to the root node, and get a zipper structure?

13:57 I still haven't quite wrapped my head around these zipper things, but apparently zip/root returns the data at the root, and not a zipper

14:00 ah, got it... I'm dealing with XML, so (= d (zip/xml-zip (zip/root d)))

14:00 chouser: rhickey: so if you add a method to a protocol and then extend it, but re-eval only the extend, you'd get an error?

14:01 noidi: I must say, clojure.contrib.zip-filter is the coolest way of dealing with XML I've ever seen

14:01 so thanks chouser :)

14:02 chouser: noidi: zip/root will zip up to the root, but will return a node, not a zipper

14:02 noidi: you're welcome!

14:03 noidi: be aware though that if you use zip-filter to get at multiple nodes, those are separate trees and "changes" to one won't show up in the others.

14:04 this is an unfortunate detail that I didn't realize until well after the design and implementation was complete.

14:04 noidi: good to know... luckily I'm only parsing an XML document and not changing anything

14:05 chouser: that was my primary use case

14:05 rhickey_: chouser: if the protocol wasn't AOTed, redefs of the protocol (e.g. add a method) will leave existing callers (whixh won't be calling the new method) and existing impls (which don't yet implement) still working

14:06 chouser: cgrand's enlive fixes that problem, but it's designed to work on html not xml, so requires some tweaking for xml.

14:06 rhickey_: if protocol was AOTed, not a very different story - impls will have implemented behind-the-scenes interface and callers will be calling the interface. Once a caller is re-evaled, won't work with old impls, and vice versa

14:07 re-eval of everyone will take the interface out of the loop

14:07 i.e. no interface when non-AOT

14:11 hmm, what would reify of non-aot protocol do?

14:12 if reify works with protocols then the story becomes really good, no talk of interfaces until linterop

14:13 chouser: would reify create a map with a gen'ed :type and ... add it to the protocol fns?

14:14 temporarily.

14:14 protocol fns could have a final case of asking the object for specific implementations, which reify could create

14:14 rhickey_: chouser: meta type?

14:14 chouser: that was a bad idea. skip it.

14:15 rhickey_: protocols can't work with meta types since they cache based on class

14:15 I need to clarify that, deftype should mean the end of meta types

14:16 chouser: reify on a non-aot protocol could return a #^{:protocol-fn-map {:SeqableProtocol/seq-me (fn [this] ...)}} {:a 1 :b 2}

14:16 protocol fns could fall back on calling those :protocol-fn-map functions directly

14:16 rhickey_: reify creates a class now, so no reason not to just extend the protocol to that class

14:17 chouser: oh

14:17 rhickey_: I was concerned about on-off classes filling the caches, but it isn't a real issue

14:17 one-off

14:17 it's not as if there is a new class each call, just one per reify site

14:19 transparent access to closed-overs might be a problem

14:19 for non-aot protocol

14:19 with deftype the fields are enumerated declaratively

14:22 actually np, since fns in method map will be closures too

14:22 chouser: closures currently expose closed-overs as public fields. is this a real feature or a implementation detail?

14:23 rhickey_: chouser: I don't think we need to leverage that, the fns will just closed over the same things

14:24 (let [a 1 b 2] (reify [AProtocol (foo [this] (use a b)]))

14:24 when foo is moved out (not a method) the fn object will just close over the same environment

14:24 where a and b will mean the same things

14:26 chouser: ok

14:29 rhickey_: extend being a function is really wild - you can do (let [x (reify)] (extend (class x) ...) x)

14:54 erm, no, while a and b will be same as instance, not same forever for all instances

14:54 will probably have to tap into fields of reify class

15:35 developernotes: trying to understand the rules that cause the following to return the value of :brown as the first element:

15:36 (first #{:the :quick :brown :fox})

15:36 why is :brown the first item?

15:36 somnium: sets are unordered

15:36 chouser: It's a hash-set -- the order is meaningless pretty much by definition

15:37 developernotes: chouser: gotcha.

15:38 chouser: developernotes: there is a sorted-map, if that helps.

15:38 developernotes: chouser: thanks.

16:04 fliebel: What is a good way to learn Clojure for Python users? I did some Java long ago, but it's quite rusty. After watching 10 movies on youtube I more or less get the concept of Clojure, but when opening an interactive shell even concatenating strings takes me minutes to figure out.

16:06 the-kenny: fliebel: Clojure isn't like Java, it's totally different.

16:06 fliebel: I know that… I quite like the syntax, although I have to go back every time to put parentheses before things...

16:06 somnium: many people find solving project euler problems helpful

16:07 fliebel: I had my own little adventures in mind to master the language. I'm not talking about 3D games, okay? :P

16:07 arbscht: fliebel: that will pass

16:08 fliebel: I was thinking about this code as data thing… Should be fairly trivial to represent XML with code using some macros and things.

16:09 the-kenny: fliebel: There's already clojure.contrib.prxml to print xml out of sexp's

16:09 ;)

16:09 ,(prxml (:foo (:bar 42)))

16:09 clojurebot: java.lang.Exception: Unable to resolve symbol: prxml in this context

16:09 chouser: fliebel: but yes, many many people have done exactly that.

16:09 hiredman: I'm kind of thinking of switching 9-( and 0-) in my keymap

16:09 fliebel: Yea, but I guess there are hundreds of scripts solving project euler problems as well…

16:10 the-kenny: ,(clojure.contrib.prxml/prxml (:foo (:bar 42)))

16:10 clojurebot: java.lang.ClassNotFoundException: clojure.contrib.prxml

16:10 the-kenny: hm.. forget it

16:10 somnium: hiredman: I mapped my useless menu key to open-paren

16:11 have that useless windows key too... hmm

16:12 hiredman: I have those both already mapped

16:12 fliebel: the-kenny: So, is there a lib like that? (I find exceptions in Clojure horribly unclear, by the way)

16:13 the-kenny: fliebel: http://richhickey.github.com/clojure-contrib/prxml-api.html

16:13 patrkris: Hi. What is the primary reason, that vectors are used for the function parameter list?

16:14 hiredman: the parameter list is not a function call, macro call, or special form

16:16 patrkris: Ok. So in Clojure-world, it leaves vectors as the most sensible option? My supervisor, an experienced schemer, keeps pointing to all these 'inconsistencies' in the syntax in Clojure compared to Scheme (e.g. using [ and ] for vectors and specifically for the function parameter list). I'm not an experienced Lisp/Scheme/Clojure-programmer, so I don't really know what to respond to that.

16:17 hiredman: patrkris: ignore him

16:17 the-kenny: In my opinion, clojure is *much* easier to read than common lisp or scheme just bevause of this "inconsistencies" in the syntax

16:18 patrkris: hiredman: hehe, yeah

16:18 hiredman: he is belly aching about nothing, so just let him do it in a corner somewhere

16:18 arbscht: patrkris: the best response would be a question: "can you explain why you think it is inconsistent?"

16:18 chouser: patrkris: Rich said somewhere that scheme is an "atomic" language, in that it provides only the minimal features needed.

16:18 patrkris: ...but that he's not interested in writing in an atomic language, he wants one that provides more than that.

16:18 Clojure is not Scheme.

16:19 rhickey_: definitely not

16:19 patrkris: I agree that Clojure programs are really easy to read, even compared to a minimalist language like Scheme (just compare cond and let in the languages)

16:19 fliebel: Is this sensible Clojure code? I cant think of another way…

16:19 (apply str (concat

16:19 ["<html"]

16:19 (map (fn [attr]

16:19 (str " " (first attr) "=\"" (last attr) "\""))

16:19 {"hallo" 3 "a" "hoi"})

16:19 [">" "hallo wereld" "</html>"]))

16:20 arbscht: lisppaste8: url

16:20 lisppaste8: To use the lisppaste bot, visit http://paste.lisp.org/new/clojure and enter your paste.

16:20 pepijn pasted "untitled" at http://paste.lisp.org/display/91244

16:21 fliebel: I was trying to piece together attributes from a map...

16:21 hiredman: I'd use reduce

16:23 fliebel: http://clojure.org/api#reduce This one? (is that page really all that makes up Clojure?)

16:23 arbscht: fliebel: there are a number of html and xml libraries for clojure

16:23 hiredman: ,(reduce #(format "%s%s=\"%s\" " %1 (name (key %2)) (val %3)) "" {:foo 1 :bar 2})

16:23 arbscht: clojurebot: libraries

16:23 clojurebot: java.lang.IllegalArgumentException: Wrong number of args passed to: sandbox$eval--7844$fn

16:23 Gabh mo leithscéal?

16:23 hiredman: urgh

16:23 ,(reduce #(format "%s%s=\"%s\" " %1 (name (key %2)) (val %2)) "" {:foo 1 :bar 2})

16:23 clojurebot: "foo=\"1\" bar=\"2\" "

16:24 arbscht: clojurebot: libraries is http://clojure.org/libraries

16:24 clojurebot: In Ordnung

16:25 fliebel: hiredman: That looks awesome, but I got no clue what is actually happening… Except that those % look like Python things.

16:25 KirinDave: I wish I could get my friends more excited in clojure. Most of them are into (hurl) Python or if they are into an alt jvm language they're into Scala.

16:26 the-kenny: KirinDave: Show them awesome things like scriptjure :)

16:26 fliebel: KirinDave: sound like me...

16:26 Chousuke: scare the scala people a bit

16:26 ~scala

16:26 hmm :(

16:26 chouser: fliebel: % in clojure isn't % in python

16:26 Chousuke: clojurebot: scala

16:26 ignoring me, huh :(

16:26 hiredman: clojurebot: tell us about scala!

16:26 clojurebot: {((x: Any, y: Any) => (f: Function2[Any, Any, Any]) => f(x, y))(1, 2)((x: Any, y: Any) => x)}

16:26 Chousuke: ah, okay.

16:27 chouser: fliebel: "foo" % [a, b] in python is (format "foo" a b) in clojure

16:27 KirinDave: But to lose to Python?

16:27 somnium: ~python

16:27 clojurebot: python is ugly

16:27 KirinDave: I can think of few worse fates.

16:27 the-kenny: ~c++

16:27 clojurebot: Excuse me?

16:27 hiredman: clojurebot: scala {((x: Any, y: Any) => (f: Function2[Any, Any, Any]) => f(x, y))(1,2)((x: Any, y: Any) => x)}

16:27 clojurebot: Any = 1

16:27 KirinDave: Python is even more of a baby-talk language than java.

16:27 chouser: really?

16:27 fliebel: I quite like Python.

16:27 Chousuke: KirinDave: meh. python is fine. :)

16:27 hiredman: fliebel: the % are params for the special #() anonymous function syntax

16:27 chouser: If you don't mind mutable things (and I do, now, I really do) both python and ruby have a lot of merit.

16:28 KirinDave: I am a big fan of Ruby.

16:28 somnium: I would rather do python than java

16:28 Chousuke: In my mind python and ruby are about equivalent.

16:28 fliebel: hiredman: Wher do I learn al this stuff in a way understandable from a non-lisp perspective?

16:28 Chousuke: as far as I can tell, the differences are superficial

16:28 KirinDave: somnium: You don't find python insulting?

16:29 Another language designed around your presumed stupidity.

16:29 hiredman: fliebel: embrace the lisp perspective

16:29 somnium: it has real first class functions

16:29 Chousuke: fliebel: http://www.defmacro.org/ramblings/fp.html ? :)

16:29 hiredman: it's difficult to embrace when you don't know what it is.

16:29 KirinDave: somnium: Uh, pretty sure they're not strictly first class functions. You pass refs to named functions around.

16:30 somnium: It does have anonymous statements, but they're almost macros.

16:30 fliebel: ChousukeL Indeed… Although I did watch some lengthy videos about it.

16:30 technomancy: KirinDave: as Adam Keys puts it, it's amazing how Guido is trying to corner the market on people who have been programming for less than an hour.

16:31 chouser: python has first-class functions, generators, a clean syntax, strong support for dynamic metaprogramming (though not of the macro variety)

16:31 KirinDave: technomancy: While at the same time, in private mailing lists, talking at length about things he claims he cannot understand.

16:31 hiredman: fliebel: dive right in, you can start by pasting that reduce into an editor somewhere, and teasing it apart to see what it does

16:32 the unladen swallow guys are having a hard time because python is so dynamic

16:32 somnium: KirinDave: I mostly did ruby before clojure, Im just saying, I would rather do python than java/C++

16:32 fliebel: I'll try that later, I'm to tired to absorb anymore information today.

16:36 technomancy: hiredman: I get the feeling that has more to do with "we haven't gotten around to reading the papers on smalltalk/self VM design" than "python is hard to optimize"

16:36 hiredman: technomancy: possibly, but the goal of keeping complete backwards compat with cpython and all the extensions in C for it complicates things

16:37 JAS415: ,(number? 1010B)

16:37 clojurebot: Invalid number: 1010B

16:37 JAS415: why doesn't that return false

16:38 hiredman: it doesn't get that far

16:38 chouser: ,1010B

16:38 clojurebot: Invalid number: 1010B

16:38 KirinDave: Yeah

16:38 JAS415: ooh

16:38 hiredman: that is a reader exception

16:38 JAS415: how would i secure my web app against that

16:38 assuming i'm taking arguments as strings

16:39 Chousuke: catch the exception? :P

16:39 JAS415: hmm

16:39 ok

16:39 * hiredman recognizes the string because he coppied all the exception strings out of LispReader by hand

16:39 ordnungswidrig: would the reade be used for that?

16:42 a bind wrapping a defn will not make the binding available when calling the function, right?

16:43 technomancy: so it looks like using clojure.main with the -e arg disables *command-line-args* ?

16:43 chouser: ordnungswidrig: 'let' would, but not 'binding'

16:43 patrkris: I came across ClojureQL recently. It made me, a n00b Lisp programmer, aware of how inferior a language like e.g. C# is (well ok, I'm generalizing). But think of the effort they made in putting in LINQ into the language. You don't have to change Clojure to get something similar - you don't need to. That was a cool experience for me.

16:43 hiredman: ordnungswidrig: fn's capture lexical scope, not dynamic scope

16:44 Chousuke: patrkris: when talking about non-lisps, you can use "Blub" to avoid offending anyone

16:44 patrkris: Chousuke: thanks

16:45 but is such an experience a milestone on the path to enlightenment as Eric Raymond writes about?

16:45 hiredman: :|

16:45 JAS415: you can also use blub to offend everyone

16:45 ordnungswidrig: chouser: clutch uses a config binding wich can be set by using a "with-db" macro. I'd like to do sth. like (with-db db (foo (fn [x] use-db) (fn [y] use-db))), the anonymouse functions will make use of the binding.

16:46 chouser: that fails. But I cannot use let without rebinding the config using with-cb in every function

16:46 Chousuke: JAS415: heh.

16:47 hiredman: ordnungswidrig: this is why it annoys me that everyone that writes some kind of db wrapper in clojure uses binding

16:48 ordnungswidrig: hiredman: what would be a better method? reader monad?

16:48 chouser: ordnungswidrig: I having a hard time thinking of a solution other than a macro to do the with-db for you. :-/

16:48 hiredman: no

16:48 just pass in the db as a method param

16:48 as if we were programming with functions!

16:48 that take parameters!

16:48 and are composable!

16:49 Chousuke: but then you'll need to pass it to almost every function :P

16:49 hiredman: sure

16:50 chouser: providing an *optional* var to rebind might be ok, but having that as the only way is indeed a bit annoying.

16:50 the-kenny: ordnungswidrig: use my fork clojure-couchdb! It has a parameter for the db-server! :p

16:50 hiredman: (fn [db y] (binding [*db* db] (dostuff y)))

16:51 ordnungswidrig: the-kenny: I heared clojure-couchdb was throwing exception event on update/delete-conflict?

16:51 the-kenny: ordnungswidrig: hm.. I think so, but I'm thinking about removing all these exceptions

16:51 ordnungswidrig: hiredman: that could event be reused as a macro, right?

16:51 hiredman: sure

16:52 ordnungswidrig: hiredman: ok, that _is_ basically clutch's with-db

16:52 chouser: hiredman: ah, of course. good.

16:52 hiredman: ordnungswidrig: inside out

16:53 ordnungswidrig: hiredman: yes, that was what I meant.

16:53 hiredman: clojurebot: binding?

16:53 clojurebot: Pardon?

16:54 hiredman: clojurebot: binding is <reply>:|

16:54 clojurebot: Alles klar

16:54 ordnungswidrig: hiredman: I think I'd like the other way round better. Haven the functions take a db-parameter and optionally wraping them with some binding and a macro that pull the default.

16:56 hiredman: passing parameters via function application is the best choice (as usual)

17:00 ordnungswidrig: Am I right that the more idiomatic way is to defmethod on :default and rais an error instead of checking the method arguments in the dispatch function?

17:09 technomancy: it would be really nice if clojure.main let you specify a namespace to have loaded and its -main function run with *command-line-args* without AOTing.

17:13 the-kenny: Is there a way to access the metadata of a fn passed as an argument? Eg. (defn foo [obj] (meta (var obj)))

17:15 Chousuke: the-kenny: fns don't have metadata.

17:15 the-kenny: ,(meta (var +))

17:15 clojurebot: {:foo 1, :ns #<Namespace clojure.core>, :name +, :file "clojure/core.clj", :line 654, :arglists ([] [x] [x y] [x y & more]), :inline-arities #{2}, :inline #<core$fn__4742 clojure.core$fn__4742@161f77c>, :doc "Returns the sum of nums. (+) returns 0."}

17:15 Chousuke: that's not th fn, that's the var :)

17:15 KirinDave: I wish the clojure compiler could give me better line numbers for bad source. :(

17:16 Chousuke: the-kenny: what you want is to 1) resolve a symbol(!) to a var and 2) then look up the metadata for that.

17:16 KirinDave: "java.lang.IllegalArgumentException: Don't know how to create ISeq from: clojure.lang.Symbol" with a stack trace into the compiler doesn't really help me figure out where my error is

17:16 Chousuke: ,(meta (resolve '+))

17:16 clojurebot: {:foo 1, :ns #<Namespace clojure.core>, :name +, :file "clojure/core.clj", :line 654, :arglists ([] [x] [x y] [x y & more]), :inline-arities #{2}, :inline #<core$fn__4742 clojure.core$fn__4742@161f77c>, :doc "Returns the sum of nums. (+) returns 0."}

17:17 Chousuke: (var foo) is of no use if applied on a local because it doesn't evaluate its argument

17:20 ghegde: Hello everyone, has anybody used the enclosure plugin for clojure dev?

17:20 i am a newbie.. am having some problems with compilation

17:22 somnium: KirinDave: usually that means theres a symbol that should be a list/vector being passed to a macro, often in (ns ...

17:23 the-kenny: Chousuke: hm.. Chousuke I'm hacking on swank-clojure and try to specialize the inspector for functions. To display the parameter list etc. When I try to inspect for example =, I get the evaluated value for =, not the symbol... is there a way I can get the metadata for =? Something like reverse-lookup or so :p

17:29 somnium: ,(meta '=)

17:29 clojurebot: nil

17:29 somnium: ,(meta #'=)

17:29 clojurebot: {:ns #<Namespace clojure.core>, :name =, :file "clojure/core.clj", :line 516, :arglists ([x] [x y] [x y & more]), :inline-arities #{2}, :inline #<core$fn__4668 clojure.core$fn__4668@152c0d>, :tag java.lang.Boolean, :doc "Equality. Returns true if x equals y, false if not. Same as\n Java x.equals(y) except it also works for nil, and compares\n numbers and collections in a type-independent manner. Clojure's immutable data\n

17:29 the-kenny: ,(let [foo =] (meta (var foo)))

17:29 clojurebot: java.lang.Exception: Unable to resolve var: foo in this context

17:29 the-kenny: somnium: That's the problem ;)

17:30 somnium: let doesnt create vars

17:30 the-kenny: somnium: Yeah, I'm aware of this. It was just an example.

17:30 I don't have access to the original symbol, I get the evaluated value

17:32 somnium: ,(let [foo '=] (meta (resolve foo)))

17:32 clojurebot: {:ns #<Namespace clojure.core>, :name =, :file "clojure/core.clj", :line 516, :arglists ([x] [x y] [x y & more]), :inline-arities #{2}, :inline #<core$fn__4668 clojure.core$fn__4668@152c0d>, :tag java.lang.Boolean, :doc "Equality. Returns true if x equals y, false if not. Same as\n Java x.equals(y) except it also works for nil, and compares\n numbers and collections in a type-independent manner. Clojure's immutable data\n

17:32 the-kenny: somnium: Yeah, that works. But then the user has to type '= instead of = for M-x slime-inspect

17:33 Chousuke: you can use a macro then

17:33 (defmacro inspect [sym] `(meta (resolve ~sym)))

17:34 the-kenny: Chousuke: I think you don't understand the problem.. sorry

17:35 hiredman: I don't understand

17:35 ,(-> "=" symbol resolve meta)

17:35 clojurebot: {:ns #<Namespace clojure.core>, :name =, :file "clojure/core.clj", :line 516, :arglists ([x] [x y] [x y & more]), :inline-arities #{2}, :inline #<core$fn__4668 clojure.core$fn__4668@152c0d>, :tag java.lang.Boolean, :doc "Equality. Returns true if x equals y, false if not. Same as\n Java x.equals(y) except it also works for nil, and compares\n numbers and collections in a type-independent manner. Clojure's immutable data\n

17:36 Chousuke: the-kenny: only vars have metadata in this case

17:36 the-kenny: so you need a symbol, from which you get to the var by using "resolve", and then you can get the metadata.

17:36 the-kenny: the actual function itself is completely irrelevant

17:37 the-kenny: Chousuke: Yeah, but I only have the actual function. That's the problem

17:37 Chousuke: then you have no way to access the metadata.

17:37 the-kenny: :(

17:37 hiredman: ,(-> = class .getDeclaredMethods ((partial map #(.getName %))))

17:37 clojurebot: ("getRequiredArity" "doInvoke" "invoke" "invoke")

17:37 Chousuke: though if the user types in "=" then you do have the function name

17:37 hiredman: ,(-> = class .getRequiredArity)

17:37 clojurebot: java.lang.IllegalArgumentException: No matching field found: getRequiredArity for class java.lang.Class

17:38 hiredman: ,(-> = .getRequiredArity)

17:38 clojurebot: 2

17:38 hiredman: hmmm

17:38 must be minimal required

17:40 the-kenny: Chousuke: No, M-x slime-inspect evaluates its argument

17:40 Chousuke: ah.

17:40 oh well, you're screwed then.

17:40 function objects don't have metadata.

17:41 the-kenny: :( No beautiful summary page for functions then

17:42 rhickey_: the-kenny: you could build a reverse-lookup map from fn values to the vars for a ns

17:43 the-kenny: rhickey_: hm.. I think this is a bit too much for this.

17:44 Is there a reason whu function objects don't have metadata?

17:44 s/whu/why/

17:44 somnium: M-x slime-inspect #'= works here, is it so bad?

17:45 rhickey_: the-kenny: because fns can't support with-meta generally

17:46 and originally meta and with-meta were in the same interface, now not, so possible

17:53 hiredman: the-kenny: building a reverse map should not be too hard using clojure.org/namespaces

18:03 rhickey_: how painful if protocols must always be AOT compiled?

18:03 would be the same for interfaces

18:16 hiredman: rhickey_: I think it would kind of put the breaks on the race to protocols, not stop it, but slow it down, which seems ok to me

18:16 (not everything needs to be protocols)

18:16 technomancy: there _is_ a ticket for adding metadata to functions

18:17 the-kenny: as much as I like the idea for what you're trying, it would be much easier to wait till that's fixed in clojure

18:17 the-kenny: technomancy: hm.. yeah, I'll wait.

18:18 rhickey_: hiredman: hmm, if it slows it down what will people do instead?

18:19 hiredman: rhickey_: write functions hopefully, use multimethods

18:19 rhickey_: hrm, multimethods that do the same job are inferior in almost every way

18:20 multimethods should be reserved for things protocols can't do

18:20 hiredman: rhickey_: except they work, don't require AOT, and are available in 1.0 and are documented and examples exist everywhere

18:21 rhickey_: hiredman: well, that's really an argument against the AOT requirement

18:21 technomancy: not requiring AOT makes multimethods more likely to be used in smaller projects that don't have a build scripted yet

18:21 rhickey_: protocols work and are documented, btw

18:21 technomancy: in the (as-yet-hypothetical?) situation where protocols require AOT

18:22 hiredman: rhickey_: ok

18:22 rhickey_: technomancy: yeah, I'm loathe to lose the start-fiddling aspect they currently support

18:22 just trying to satisfy all the requirements, reifying protocols would be awesome, since most designs will become protocol-based

18:23 but reify of protocol w/o aot is troublesome

18:23 technomancy: tricky trade-off =\

18:28 chouser: in what way is it troublesome?

18:30 rhickey_: chouser: the issues we discussed before, not knowing what is closed over, intervening once only to bind type to protocol, when should that be revisited (if protocol was modified)

18:31 technomancy: did https://www.assembla.com/spaces/clojure/tickets/193 only get moved off the 1.1 list because the patch was misformatted? if so, would a correctly-formatted patch put it back in the running, or has the 1.1 list already been decided?

18:33 rhickey_: chouser: because reify both creates the class (when compiled) and the object (every time), the semantics are trickier than with an interface, presumed unchanging relative to creation of class

18:34 could do same for reify of protocol

18:34 but reify code has to participate in knowing about protocols

18:34 macro can;t yield anything that will run every time

18:35 vs the deftype case emitting an extend-type

18:38 chouser: did you see my idea for putting the implementation in the object? I guess that doesn't help the closing-voer problem.

18:45 rhickey_: chouser: protocols cache based on class/dynamic type, adding looking for on-object would slow things down, also a semantic problem as caching is based upon class

18:47 if on-object, user would have to ensure all objects of same class had same impl

18:47 chouser: yeah, that sounds worse. I guess I don't understand the context fully yet

19:19 cgordon: I have another functional n00b question. If I have a function like "(defn test [coll] (lazy-seq (cons (first coll) (test (rest coll)))))", the recursive call isn't in the tail position (cons is), but yet this doesn't blow it's stack, and doesn't run out of memory. Is Clojure somehow switching things around so that the recursive call is in the tail position?

19:22 the-kenny: cgordon: I think lazy-seq does the trick

19:23 rlb: cgordon: I believe lazy-seq just returns the remaining (unevaluatedsequence, so there's no stack growth.

19:23 i.e. there's no immediate recursive call

19:23 cgordon: right, I think I just figured that out. I removed "lazy-seq" and it blew up on me

19:23 :)

19:24 rlb: The code isn't executed until you do something like (first coll) or whatever.

19:24 (and then only up to the next lazy-seq)

19:24 cgordon: how does this work in Lisp (which I presume doesn't have "lazy-seq")?

19:24 rlb: cgordon: it doesn't I don't think (at least not in common lisp)

19:24 depending on what you mean by "work"

19:25 cgordon: rlb: so this would just blow your stack in CL?

19:25 rlb: cgordon: yes, iirc.

19:25 cgordon: rlb: is there a way to rewrite it so that the recursive call is in the tail position?

19:26 rlb: cgordon: you would probably use loop, a trampoline, or otherwise manage what would otherwise be on the stack yourself.

19:26 i.e. pass in the list you're building, whatever.

19:26 cgordon: rlb: alright, thanks, that helps

19:26 rlb: (as an arg)

19:26 cgordon: the hardest part of learning functional programming seems to be understanding the cost of various operations

19:27 Chousuke: the thing is that lazy-seq just returns a thunk, so the recursive call returns immediately

19:27 and the next recursive call is made only when the thunk is evaluated.

19:27 somnium: ~haskell

19:27 clojurebot: haskell is Yo dawg, I heard you like Haskell, so I put a lazy thunk inside a lazy thunk so you don't have to compute while you don't compute.

19:27 rlb: cgordon: cl doesn't even promise tail call optimization, so you might blow the stack even if you were tail recursive.

19:27 nice

19:27 Chousuke: which is only after the first call has finished :)

19:28 so there's no stack growth

19:28 rlb: cgordon: whereas scheme does guarantee tco.

19:28 cgordon: Chousuke: so the seq is never kept in memory, it just has the current head and the thunk for the rest?

19:28 rlb: cgordon: right

19:28 though of course you can get in to trouble if you hang on to the head

19:29 cgordon: rlb: that's because of the linked structure, right? It can't GC it until there are no refs to the linked structure anywhere?

19:29 rlb: cgordon: exactly -- if there's *any* way to reach something, it can't be gc'ed.

19:29 Chousuke: cgordon: the results of evaluating the thunks are cached, so if you evaluate the seq further than just the head, those thunks will "disappear" and be replaced with just their values

19:31 there won't be stack growth but you can still blow the heap if you keep too much stuff in memory :)

19:33 cgordon: Chousuke: yes, I'm trying to understand what circumstances cause things to be held in memory

19:33 I think I need to read more about seqs first though

19:35 Chousuke: well, seqs are linked structures so they're like normal linked lists in that regard. Though it's possible to write code that looks like you don't have a reference to anything but in fact still do

19:36 cgordon: ,(type '(1 2 3 4))

19:36 clojurebot: clojure.lang.PersistentList

19:36 cgordon: ,(type (seq '(1 2 3 4)))

19:36 clojurebot: clojure.lang.PersistentList

19:36 Chousuke: for example (fn [aseq] (let [aseq (drop 1000 aseq)] "the original reference is still here preventing GC, just shadowed"))

19:36 cgordon: persistentlists happen to be seqs themselves.

19:37 cgordon: ah, so they implement ISeq?

19:37 Chousuke: but vectors aren't for example

19:37 yeah.

19:37 cgordon: ,(type '[1 2 3 4])

19:37 clojurebot: clojure.lang.PersistentVector

19:37 cgordon: ,(type (seq '[1 2 3 4]))

19:37 clojurebot: clojure.lang.PersistentVector$ChunkedSeq

19:37 cgordon: interesting

19:42 Chousuke: can you elaborate on your example. I can't figure out where the reference is being held

19:42 rzezeski: Chousuke, can you elaborate on your example above? Are you saying that simply the fact that aseq is function argument causes it to have a hidden reference? Or are you saying the let form is causing the head to be held? If the latter, you would just do #(drop 1000 aseq) instead, right?

19:42 cgordon: :)

19:42 rzezeski: and then call (aseq) in your body anytime you needed the sequence

19:43 Chousuke: rzezeski: the function argument aseq is the reference. That let shadows the *name* doesn't change the fact that the reference is there

19:45 rzezeski: Gotcha, so in that case, when you want to avoid holding on to the head, should you just pass around functions returning seqs? So that the function argument would be a function itself, and in the let you would do [aseq #(drop 1000 (aseq))]?

19:47 Chousuke: well, the correct way would be not to have the let at all. locals are cleared on tail-call, so (fn [aseq] (someotherfn (drop 1000 aseq))) would not have the problem.

19:47 Could be optimised away with a smarter compiler, but no-one's motivated to improve the current one especially because the long-term plan is to rewrite it in Clojure anyway

19:48 so it's just some unintuitive behaviour you need to be aware of for now. :)

19:50 cark: so the head of a parameter seq is always retained unless there's a recur ?

19:50 hiredman: no

19:50 Chousuke: it's not specific to seqs though; it matters to all references

19:51 hiredman: a tail call is a tail call, recur has to be in the tail position, but it does not define what a tail position is

19:51 rhickey_: cark: no, as Chousuke just said, locals (including params) are cleared on tail calls

19:52 cark: so if i map an infinite sequence, it only works because there is a tail call inside map

19:52 Chousuke: I guess it could be said that the reference always exists within the form it was created in until an expression in tail position is evaluated

19:52 rhickey_: cark: no Clojure library functions have head-holding problems

19:53 you can look at their sources to see how they do it

19:53 cark: ight i know that, i wasn't sure why

19:53 rhickey_: most often, they need to do nothing

19:53 cark: +r

19:54 ok thanks

19:56 cgordon: Chousuke: I'm still missing something. You said "well, the correct way would be not to have the let at all. locals are cleared on tail-call, so (fn [aseq] (someotherfn (drop 1000 aseq))) would not have the problem", but I don't see any tail call in that function.

19:57 rzezeski: (defn uhoh [big-seq]

19:57 (dorun big-seq)

19:57 (filter odd? big-seq))

19:57 I think the above illustrates the point?

19:58 (uhoh (repeat 1)) ; will cause OutOfMemory

19:58 cgordon: sorry, just realized that hiredman already answered that question

20:00 Chousuke: yeah. a tail call is a tail call if it happens in a tail position. recur is just clojure's special form for making tail calls that don't grow the stack.

20:01 cgordon: Chousuke: just to make sure I understand, the problem with your initial code was that aseq continued to point at the head of the seq throughout the body of the let, despite the fact that the aseq in the let only pointed at the 1001st item. Is that right?

20:02 In the subsequent code, "someotherfn" had a pointer to the 1001st item and presumably returns a lazy seq, which then returns out of the anonymous function, thereby freeing up the first 1000 items for GC?

20:03 or are the first 1000 items freed up *before* someotherfn is called?

20:04 (I realize I'm using the term "pointer" incorrectly here, but I'm not sure what the right term is)

20:04 hiredman: freed implies the gc has run and freed the memory

20:05 cgordon: hrm, good point, I was trying to say, "available to be freed", I think

20:05 hiredman: the term is reference

20:05 rzezeski: cgordorn, yea I felt like the let was confusing things, I offered my example above as a parallel. In my example dorun does not hold onto the head, and my tail call is lazy. Therefore I should not experience OutOfMemory when passing it (repeat 1), but should just burn the CPU. However, I did recieve an OOM, confirming what Chosuke is saying...I believe.

20:06 erm, I called it at the repl though, that could be my problem :(

20:07 Chousuke: cgordon: well, since it's gc you never know when it's freed but in the corrected version, there is no reference while someotherfn is executing, while with the let there is.

20:07 cgordon: alright, I get it now, thanks for your patience

20:07 Chousuke: Well, it's good practice trying to explain this to other people :P

20:08 I'm still not sure if I completely understand it myself either :D

20:08 cgordon: Heh, yes. I just put up a blog post about my experience writing a little Servlet in Clojure. I didn't realize how little I understood what I did until I tried to write it down :)

20:08 rzezeski: Chosuke, would you agree my example is demonstrating the same thing, but without the let?

20:10 cgordon: "Writing is nature's way of letting you know how sloppy your thinking is." - Guindon

20:10 Chousuke: rzezeski: well, it's not good because the dorun would get stuck even if you weren't holding to the head, but... yeah.

20:10 rzezeski: Oh right, didn't even think of that

20:11 Chousuke: in fact, because you're holding on to the head you'll just run out of memory at some point. if you had no head-holding problems it'd be an infinite loop :)

20:12 rzezeski: yea, that's what I was trying to get at

20:12 Chousuke: I wonder which one is worse :P

20:13 rzezeski: Your example just confused me, because I didn't realize that the fn arg caused an implicit ref, and I thought you were demonstrating that the let form caused a ref that will be helf for the remainder of the function (because I've ran into that problem before).

20:15 this is good, because I've been having laziness problems lately, and I think it was this very thing that is causing me grief

20:20 lusory: [5~[5~

21:49 Drakeson: can incanter be compiled headlessly?

21:49 liebke: Drakeson: yes, use the incanter-core ant task

21:50 Drakeson: I don't get processing then, do I?

21:51 liebke: if you want processing, you can just use the normal build. I fixed the bug the caused problems on headless machines

21:56 qed: hey all

21:58 Drakeson: liebke: i see, thanks

21:58 qed: 'lo tomoj

22:00 tomoj: hello

22:03 j3ff86: is there an easy way to turn a seq collection like ((1 2)(3 4)) into a vector [[1 2][3 4]]?

22:06 nm, mapped the vec function

23:51 qed: Are there any active documentation projects ongoing?

23:51 I'm interested in adding examples to the api documentation

23:53 paan: catch23: which graphs do you guys talking about?

23:53 sorry, wrong channel :(

23:57 qed: let's say I take some input from a user and i want to use that as a regex in the function, how do I say #"user-input-goes-here"

23:58 like (defn abc [s] (let [regex #"s"]))

23:59 _mst: ,(java.util.regex.Pattern/compile (str "hello" "world"))

23:59 clojurebot: #"helloworld"

Logging service provided by n01se.net