#clojure log - Dec 09 2008

The Joy of Clojure
Main Clojure site
Google Group
IRC
List of all logged dates

3:20 Lau_of_DK: Morning gents

3:24 AWizzArd: what wiki software does clojure.org use?

3:44 oudeis: AWizzArd: a quick glance at the code suggests it is http://www.wikispaces.com/

3:46 AWizzArd: Ah oki, thx

4:25 danlarkin: ooof, nesting syntax-quote inside syntax-quote gets tricky fast

8:49 AWizzArd: Is there already a clojure lib for swing?

8:50 arbscht: AWizzArd: clojure can use swing directly

8:50 AWizzArd: yes, but I would not want to do it this complicated way. For example when I want to add an event handler for buttons.

8:51 Why should I use (proxy ...) for that?

8:51 (swing-add-handler my-button (fn [..] ...))

8:53 Not that it is difficult to write. I was just curious if someone with swing knowledge did that.

8:56 blackdog: i'm curious if anyone's doing a javafx clone api

8:56 gnuvince: AWizzArd: as far as I know, no. And I am not sure a lot of people consider it a priority. There was an interesting exchange between Rich and Slava Pestov of the Factor language on Reddit a few months ago where Slava deplored the lack of language-specific idioms in Clojure. Rich replied that saying (.close fd) instead of (close fd) was not a big deal and that the real important thing was to be able to Get Things Dones (tm)

9:02 AWizzArd: Yes, I agree with that. What I have in mind would probably be 50-250 LOC of Clojure code. Just a simple helper tool, to make Java (and interop) pain go away. Definitly this is nothing for Rich to do, as it is really not very important for Clojure.

9:14 RSchulz: AWizzArd: Wouldn't your (swing-add-handler ...) be forced to proxy a local type to match the type required of the particular Swing handler?

9:15 I mean, that's probably a good thing to have in a helper, but proxying is unavoidable when interoperating with existing Java APIs, right?

9:15 AWizzArd: yes probably, I didn't look deeply into it yet

9:15 yes

9:15 the helper would do this proxy stuff though and hide it from us

9:15 RSchulz: Right. I think that would be pretty helpful.

9:16 Has anybody ever used MPW? Do you know what its Commando utility was?

9:16 AWizzArd: I'll see if I can get something like that ready, as I will need to do a little bit of Swing anyway.

9:16 RSchulz: It was a simple way to defining parameter-gathering dialogs for invoking CLI tools (within MPW).

9:16 I was thinking my CLI-phobic end users would like to have such dialogs, but writing them all by hand would be very tedious for me without such a DSL.

9:17 Also, it would get me out of my current Unix (Linux) requirement for the CLI tools.

9:24 AWizzArd: what do you prefer? (.method obj arg1 arg2) vs (. obj method arg1 arg2)?

9:25 RSchulz: I'm not sure I have strong preference. Probably the first, since it's more Lisp-like: (functor args...)

9:25 AWizzArd: yes

9:26 gnuvince: The first form is preferred.

9:31 Chouser: RSchulz: are you familiar with the library of ant tasks?

9:32 RSchulz: No. You mean the ones built into Ant?

9:32 Chouser: yes

9:32 RSchulz: I try to minimize my exposure to build.xml files...

9:32 I can never modify one without consulting the Ant documentation.

9:33 I can't usually get it right the first time.

9:33 Chouser: I only know what I've read in Stuart's book, it looks like they have a bunch of OS-indepedent operations that might be good candidates for wrapping in a dialog box.

9:34 The book has a running example of building a DSL in Clojure so you can write build processes for ant without touching any XML.

9:35 RSchulz: Ah. Right. I actually looked at his ... what was it called? ... Ant DSL for ideas about another DSL I need to write.

9:35 Chouser: Lancet

9:35 RSchulz: Right. Lancet. I kept thinking "enchantment."

9:36 Chouser: It's still not clear to me if he made up the specs and name just for the book, or if it's some kind of re-implementation of some other "lancet".

9:37 RSchulz: I think he did it for the book. There were some messages about it on the mailing list.

9:40 clojurebot: svn rev 1148; support :when and :while together in for, patch from Chouser

9:41 AWizzArd: *thumbs up*

9:41 Chouser: handy!

9:41 AWizzArd: very

9:41 RSchulz: You name in lights!

9:41 Chouser: ah, yes, that too. :-) but I meant clojurebot announcing.

9:42 AWizzArd: Chouser: why are the dots in front of add, setSize and so on left out at http://clojure.org/jvm_hosted

9:44 Chouser: AWizzArd: that's old-style 'doto'. Much of the site documents the release (where that syntax is correct) rather than SVN.

9:44 AWizzArd: oh right, for the downloadable clojure.jar, this old one

9:45 RSchulz: Rich mentioned that the API page, http://clojure.org/api, is generated from the source code, so it remains up-to-date, though no more detailed than what you get from (doc ...)

9:54 whidden: What is the process for generating the API page and would it be "easy" to do for a local copy? -- for long tain rides.

10:02 RSchulz: I don't know. I'd start by looking at the Clojure build.xml file. If it's not there, ask Rich.

10:02 I'd like to get the text flowed, for one thing. And I'd like a more printer-friendly format for that page (no sidebar TOC, e.g.)

10:02 whidden: Ok I'll look into the build.xml when I have a free moment..

10:03 RSchulz: I suppose someone will write a ClojureDoc at some point.

10:06 Chouser: whidden: there is this: http://clojure.googlegroups.com/web/manual.pdf

10:06 though that's a bit out of date by now, I suppose.

10:08 rhickey: RSchulz: there is a print stylesheet - I don't get sidebars etc when I say print

10:08 RSchulz: Really?? Huh. How long has that been the case?

10:08 Is Tom around?

10:09 Anyway, that's great. But without allowing the doc text to flow, there's still a lot of wasted space and it takes too many pages.

10:09 Chouser: For nearly a month now: http://clojure-log.n01se.net/date/2008-11-13.html#14:29b

10:10 RSchulz: Ah. Fabulous. I was hoping I hadn't just hallucinated the last time I tried printing (or print previewing).

10:11 Right now, it would be 41 pages for me (according to Firefox's print preview).

10:11 stuarthalloway: Chouser: A lancet fluke controls the mind of an ant. Dan Dennett talks about it at: http://blog.ted.com/2007/07/dan_dennett_on_2.php

10:11 RSchulz: That could probably be cut nearly in half if the doc text flowed.

10:11 Chouser: stuarthalloway: ha! great.

10:12 RSchulz: Yikes. Biology is gruesome.

10:13 rhickey: RSchulz: text not flowing on API page is my bad from last time I generated it. It flows on other pages for me when printing

10:14 stuarthalloway: The other Ant-mind-controlling organism is a Cordyceps fungus. There is stunning time-lapse video of Cordyceps in the BBC's Planet Earth series. It's even cooler than a Lancet fluke but I decided it was too hard to spell.

10:15 RSchulz: OK. Then I'm happy. (Don't shoot me, but I like paper documentation...)

10:15 rhickey: actualy looks very screwy - get through ns are all underlined for me...

10:16 I'll try to regen later

10:17 RSchulz: OK, thanks. No particular hurry.

10:17 I'm not seeing that. It's usually a missed close tag, obviously.

10:18 rhickey: No, it's some wiki markup thing, that's what I generate

10:20 tomhickey: rhickey & RSchulz: I just got caught up on the discussion, let me know if i need to update anything

10:20 Chouser: rhickey: the code that generates the wiki markup isn't available anywhere, right?

10:20 That question came up last night.

10:21 rhickey: Chouser: I should put it somewhere since I keep losing it :) Then finding and using broken old versions

10:21 Chouser: heh

10:23 * Chouser notices clojure/trunk/clojure.markdown for the first time.

10:23 rhickey: That's really old

10:26 lisppaste8: rhickey pasted "wiki doc gen" at http://paste.lisp.org/display/71857

10:27 rhickey: Probably broken, given the lack of wrapping (should remove non-double newlines), plus the underline thing - help welcome

10:28 RSchulz: rhickey: What did you think of the patch proposed to allow doc strings in defmulti? Chouser was the only one to respond, noting the potential breaking nature for some existing defmulti applications.

10:30 stuarthalloway: rhickey: FYI when I wrote the multimethods chapter, the number of (visible open source) multimethods that dispatched on anything other than type was vanishingly small, so now's the time for any breaking changes

10:32 AWizzArd: you mean the dispatch function was class ?

10:32 stuarthalloway: right

10:32 AWizzArd: ic

10:33 rhickey: class-or-tag will become prevalent

10:33 stuarthalloway: and most of the ones that weren't on class were on other things that server similar purpose, e.g. dispatch on identity to match keywords

10:33 rhickey: then people will start figuring out hierarchies and multi-argument dispatch. It's early days and people are doing the common thing

10:34 I don't see why defmulti doc strings can't be done in a non-breaking way

10:36 though changing default from positional to :default foo would be more extensible

10:40 no one saw the sneak peak at streams in this? - http://groups.google.com/group/clojure/msg/53227004728d6c54

10:40 Chouser: I did. I love it.

10:41 rhickey: still working on timeout handling, since I want these to work well over queues

10:41 Chouser: all this agent/stm stuff is nice -- an insurance policy against the future. But it's the immutable collections and seq stuff that mean something to me today.

10:42 rhickey: I like *timeout* dynamic var, and (with-maximum-timeout 1000 ...)

10:42 tomhickey: rhickey: the wiki markup for gensym currently on site doesn't match what's coming back when i run your code (extra double underscores causing unclosed <u>)

10:43 Chouser: does 'stream' build a mutable iterator-type-thing over coll?

10:44 rhickey: Chouser: yes, a stream generator - next! is a one-shot read

10:44 eos is a sentinel

10:45 the key diff with iterators is that iterators have a built-in race condition: hasNext/next

10:45 not so these generators

10:45 means you can multiplex them

10:45 Chouser: ah, so you solved the 'map' problem?

10:46 rhickey: there's still the inversion of control issue, but having banged my head against it a good bit, I'm unconvinced that LFE for resource management is any easier than with-open

10:47 and with-open could easily handle multiple reasource, and yes, map for streams multiple streams

10:47 in any case, I'm shooting for a simple protocol

10:48 The trick is to avoid duplicate effort - every seq fn should have a stream version, so I want the former to be mechanically generated from the latter

10:50 RSchulz: The only thing I can (quickly) find for LFE is Lisp Flavoured Erlang. Is that what you meant?

10:51 lisppaste8: rhickey pasted "more stream teasers" at http://paste.lisp.org/display/71858

10:51 Chouser: Left something Enumerator, I think.

10:52 rhickey: RSchulz: left-fold enumerators - http://okmij.org/ftp/Streams.html#iteratee

10:52 RSchulz: Thanks.

10:52 rhickey: esp: http://okmij.org/ftp/Haskell/Iteratee/DEFUN08-talk-notes.pdf

10:54 outstanding issues are: what to do with a timeout - exception or return sentinel, and naming - should map et al be overloaded for streams or have map-???, filter-??? etc

10:55 RSchulz: If you write individual xyz-??? calls, they can be unified under a multimethod, right? But if you write them as a multimethod, they cannot easily be used directly? (Splitters vs. Lumpers)

10:56 Chouser: is the fn given to 'stream' guaranteed to get an (eos?) => true to handle resource cleanup?

10:57 rhickey: RSchulz: not sure what you mean, I'm just talking about (map f x) doing something special if x is an IStream

10:58 RSchulz: Isn't a type check an indication you should be using a multimethod?

10:58 rhickey: Chouser: the fn given to stream is responsible for producing an eos, if it's not an infinite stream

10:59 RSchulz: how it branches isn't relevant at this point, the overloading vs non, e.g. map-stream, is the issue

11:00 streams are actually very dangerous things to share, so it's nice to have their use called out. But inside a pipleline they can speed things up significantly

11:00 Chouser: Isn't the resource management issue about the flow of "I'm done" messages from stream consumers down into stream producers?

11:00 RSchulz: But that's the point. If you write map-stream it can still be unified through a multimethod and remain open to extension. But if you bury that inside map, it's no longer open.

11:00 Or am I still missing the point?

11:01 Chouser: RSchulz: if we're ever allowed to write code that uses 'map' on a stream, any change to that would be a big breaking change.

11:01 rhickey: right, it's an API thing

11:01 Chouser: regardless of how its implemented

11:02 hm, and putting it that way sounds like an argument for starting with them called out as 'map-stream'

11:02 RSchulz: Which, I think, is my point.

11:02 Chouser: if it becomes obvious later that 'map' should be able to handle streams, that would be a non-breaking change.

11:02 Zephtar: I think a better approach would be a func that lazily turned a stream into a seq of bytes

11:03 RSchulz: We're not assuming byte streams, are we? Isn't this an abstract stream?

11:03 rhickey: Chouser: not really, I'm done helps, but not all scenarios work that way, ultimately all resource management is - you init, you clean up, even if by explicitly assigning the responsibility to someone else

11:04 but context comes into play as well, can't let resources leak, and there is no solution for unbounded usage context for anything other than memory

11:04 so, LFE or with-open

11:04 my lfe has you getting passed a stream inside, guaranteed to be bad outside

11:05 (reduce-lines fn-of-a-stream afile-spec)

11:06 stream-lines a better name

11:07 file closed when done

11:07 this vs line-seq, which must be paired with with-open

11:07 duck1123: I just added Clojure to the Java template on wikipedia. Should drive more people to that page. (and clojure.org in turn)

11:10 rhickey: RSchulz: right stream as in generator as in ephemeral sequence

11:11 RSchulz: Right. I was just clarifying for Zephtar's sake.

11:12 rhickey: it's really just the old file read protocol, seems well worn and understood

11:12 I'm still ponderng unread/pushback

11:13 whidden: for what its worth I like thinking of streams as "ephemeral" sequences. It reminds me a little of the Sun OS device streams.

11:13 In the Sun streams you could push/pop consumers/producers on to the "stream"

11:14 to allow for arbitrary processing, etc.

11:14 RSchulz: You mean the old SVR3 streams model?

11:14 Is that still around?

11:14 It was pretty cool.

11:14 whidden: yes

11:14 RSchulz: It actually came out of Bell Labs.

11:15 whidden: I think the Sun model added a bit to original Bell Labs stuff, but I could be wrong, or mis-remember :)

11:16 RSchulz: That all was near the end of my Unix kernel days, so I don't know what happened after it got into Sun's kernel.

11:17 whidden: I think what sun added was to allow for consumers/producers of streams to get out the kernel space for most of their processing, akin to what the Mach Microkernel wants/wanted to do.

11:44 mibu: is there a mirror for clojure.org?

11:52 danlarkin: there's always the google cache

11:53 RSchulz: Is there any particular need for mirror?

11:58 duck1123: If I'm using gen-class, does the classloader need any funky permissions to load the implementations?

11:59 possibly something that might be turned off in some situations?

11:59 Chousuke: doesn't gen-class just use the regular java classloader? :/

12:00 the classes are AOT compiled so they should be no different from java classes, right?

12:01 duck1123: that's what I was thinking, but I remember reading something about some security situations where AOT gen-class wouldn't work right now.

12:01 IIRC, android had that situation

12:01 I'm still trying to figure out why my servlet works in Jetty, but not in Tomcat

12:02 a guy in #tomcat suggested it may be a classloader issue

12:03 all of my classes are on the classpath, so that's not the issue

12:03 stuarthalloway: duck1123: "all classes on classpath" does not imply "not classloader issue" :-)

12:05 check the classloader delegation of the class that successfully loads, closest to the point of the problem

12:06 one possible problem is that some classes are on *more than one* classpath

12:06 last I checked, Tomcat has 6 classpaths: core, extensions, system, tomcat-core, webapp-shared, and webapp-specific

12:07 with the "system" being the one configured by the CLASSPATH variabile

12:08 duck1123: $CLASSPATH is empty, I'm not sure I follow the rest of your suggestion

12:09 * stuarthalloway ducks out for lunch, but will elaborate in a bit

12:09 duck1123: fair enough. I'll keep hacking at this

12:12 it's not a runtime codegen issue. Hibernate uses RTCG, and since Hibernate works with Tomcat, that can't be the issue

12:36 stuarthalloway: duck1123: do you have a stack trace?

13:03 duck1123: stuarthalloway: no, I'm not getting any exceptions

13:04 stuarthalloway: what's the symptom, then?

13:04 duck1123: All I'm getting is a 404 error when I try to load my servlet

13:04 It works in Jetty, but not in Tomcat

13:05 stuarthalloway: have you tried placing a Java-created servlet or JSP in the web app to see if that works?

13:05 duck1123: yes. Works fine

13:05 stuarthalloway: and nothing in the logs?

13:05 duck1123: nope

13:06 stuarthalloway: if I simply use Clojure to stub the servlet interface will the be close enough to what you have to be a useful comparison?

13:08 duck1123: my code is at http://paste.lisp.org/display/71768#2

13:15 stuarthalloway: I'll give it a try

13:16 duck1123: I can paste the web.xml file too if you need it, but it's quite simple

13:34 Lau_of_DK: Do we have a Clojure-func for dumping a string to file?

13:34 AWizzArd: in contrib, (spit ...)

13:35 Lau_of_DK: Thanks

13:39 Is there a function to download a page like www.google.com ?

13:39 AWizzArd: yes, in contrib

13:40 (duck-stream/reader "https://www.google.de/&quot;)

13:40 it can also read files, http and ftp

13:45 stuarthalloway: duck1123: how are you compiling the servlet, and do you get an error?

13:46 I tried (compile 'net.mycyclopedia.Servlet), which created some classes but also reported an error: java.lang.ClassNotFoundException: net.mycyclopedia.Servlet$_doGet__78

13:46 duck1123: I wasn't getting any errors

13:47 I think you might need to add the tomcat libs to your classpath

13:47 servletapi in particular

13:47 Lau_of_DK: AWizzArd: Do I pull the new jar in with (use 'clojure.something) ?

13:48 AWizzArd: when you have the clojure-contrib.jar in your CP then you can immediately use it next time you start up Clojure.

13:48 use and require are there to allow you to call functions with a shorter name

13:48 not to make it possible to use them... that's what the cp is for. As soon stuff is in there your app can call all functions from the libs in the .jar

13:50 instead of typing always (clojure.contrib.duck-streams/reader ...) you can say (require '[clojure.contrib.duck-streams :as ds]) and then (ds/reader ...)

13:50 Kerris7: I wanna buy rhickey_ a copy of the Qi book, let him flip through it, see what he thinks of it

13:51 Lau_of_DK: Good tip, thanks AWizzArd

13:51 kotarak: 1:6 user=> (clojure.contrib.duck-streams/reader "http://www.google.de")

13:51 java.lang.ClassNotFoundException: clojure.contrib.duck-streams (repl-1:6)

13:51 1:7 user=> (require 'clojure.contrib.duck-streams)

13:51 nil

13:51 1:8 user=> (clojure.contrib.duck-streams/reader "http://www.google.de")

13:51 #<BufferedReader java.io.BufferedReader@a8f100>

13:51 So it's not totally optional.

13:52 stuarthalloway: duck1123: Tomcat is on the classpath, it's something else

13:52 ...and different from the problem you have :-)

13:53 duck1123: ahh! (set! *compile-path* "WEB-INF/classes")

13:54 stuarthalloway: oh, you aren't putting the files there?

13:54 AWizzArd: kotarak: ah okay right, I already forgot that because I added that guy in my (ns ... (:require ...)) anyway

13:55 duck1123: did it create the class files for you?

13:55 stuarthalloway: yes, but perhaps not correctly

13:55 duck1123: I think by default it'll put them in classes/ not WEB-INF/classes

13:56 stuarthalloway: I put them in my working directory and copied them over myself

13:56 duck1123: ok, I had them build right into the correct location

13:57 stuarthalloway: here'

13:57 here is what I am seeing: http://paste.lisp.org/display/71866

14:02 nevermind, reading the docs saves the day once again

14:08 Lau_of_DK: Didnt we used to have a not-nil? ?

14:09 nevermind

14:31 Isnt (org.apache.commons.io FileUtils) standard when you have apache2 installed?

14:33 hiredman: I imagine it depends on who you get your apache from

14:33 stuarthalloway: Lau_of_DK: what do apache2 the webserver and Apache Commons the Java library have in common?

14:34 Lau_of_DK: stuarthalloway: trick question?

14:34 stuarthalloway: other than a brand name...?

14:34 no, seriously

14:34 hiredman: they are part of the apache foundation, whatever that means

14:34 projects of the "

14:34 stuarthalloway: all the Java "Apache" packages are former "Jakarta" packages

14:34 Lau_of_DK: I found a simple example in Java to dump a file, where they import (org.apache.commons.io) so I figured it was some kind of bundle since no special notice was made about the import

14:35 stuarthalloway: just google "Apache Commons" --it's a single jar

14:35 hiredman: uh, really?

14:35 stuarthalloway: well, commons is anyway

14:36 apache the web server isn't written in Java, so I don't see why it would ship with much in the way of Java libs

14:37 Lau_of_DK: thanks stuarthalloway, right you were

14:42 http://www.kodejava.org/examples/55.html

14:42 Can somebody tell me why they dont close the File. object ?

14:42 hiredman: spite?

14:43 java.io.File objects are not opened or closed

14:43 they are not an io stream, they are a file

14:43 Lau_of_DK: As I recall, file objects are closed so guarantee the final flush

14:44 hiredman: and the writestring method must take care of opening an io stream

14:44 Lau_of_DK: I pretty sure you are thinking of iostreams

14:44 Lau_of_DK: k

14:45 hiredman: http://java.sun.com/j2se/1.4.2/docs/api/java/io/File.html <-- doesn't have a close method

14:45 RSchulz: Doesn't have one. Never will...

14:45 Lau_of_DK: Right you are

14:45 RSchulz: What an incredibly useful comment :)

14:45 hiredman: never say never

14:45 RSchulz: I can't be helpful 100% of the time.

14:46 Except, of course, to utter the timeless phrase: "Never say never."

14:46 Lau_of_DK: hehe

14:47 danlarkin: and the phrase explaining it

14:47 * danlarkin is reminded of family guy...

14:48 RSchulz: Which one?

14:48 PTV was just one the other day. So hilarious.

14:49 hiredman: is there anything in contrib for csv munging?

14:50 or does anyone know of a nice java lib for it?

14:50 danlarkin: lisppaste8: url

14:50 lisppaste8: To use the lisppaste bot, visit http://paste.lisp.org/new/clojure and enter your paste.

14:51 danlarkin pasted "OT time wasting" at http://paste.lisp.org/display/71869

14:51 danlarkin: RSchulz: that's the quote I'm thinking of

14:58 RSchulz: Yup. I know that episode.

15:00 Lau_of_DK: lol

15:35 Is there a simple way to download an .html file and have the javascript evaluated? It will generate some tables which I need the data from

15:37 danlarkin: Lau_of_DK: simple way? no

15:40 Lau_of_DK: danlarkin: Alright, I just need to have it done by midnight, 2� hours from now, where do I look ?

15:41 danlarkin: well you'll need to evaluate the javascript

15:41 so you can either embed a javascript engine or try to shell it out to somewhere

15:45 * Chouser-away peeks in, whispers "Rhino", and wanders off.

15:59 RSchulz: Does anybody know of work similar to http://okmij.org/ftp/Streams.html that isn't phrased in terms of Haskell? I don't know the language and find it entirely abstruse.

16:06 hiredman: I am trying to write a macro the defns some functions,

16:06 java.lang.Exception: Second argument to def must be a Symbol (NO_SOURCE_FILE:591)

16:07 it seems like the first argument to def is supposed to be a symbol?

16:08 mfredrickson: RSchulz: which part specifically -- not that I have anything for you, I'm just curious

16:09 (i've been thinking about linear algebra in Clojure for the last few days, so that caught my eye)

16:09 RSchulz: Well, as a language heavy in syntactic sugar, I find I can't get past line one of any example.

16:09 However, I just found a "cheat-sheet" (13-page tutorial) on Haskell, which should prove helpful.

16:10 duck1123: that's a hell of a cheat sheet

16:10 RSchulz: Exactly what someone remarked on the comment page where I found it...

16:11 I see that it is written "quick reference card" format (three-column landscape)

16:11 For those interested: http://blog.codeslower.com/2008/10/The-Haskell-Cheatsheet

16:31 AWizzArd: Is there a java lib for obfuscating a database?

16:31 (as in oracle, postgres, mysql, ...)

16:31 RSchulz: What sort of obfuscation?

16:36 AWizzArd: database obfuscation that will semantically leave all data inside the db, but rename tables, fields, contents, etc.

16:36 So that someone who is not dedicated into finding out the scheme behind this won't be able to read the db

16:37 danlarkin: AWizzArd: you are asking for a maintenance _nightmare_ by doing that

16:37 AWizzArd: Can you tell me more about this?

16:38 RSchulz: I can't see the virtue. Unlike Java code, you don't have to (and should not want to) expose your DBMS to arbitrary client use.

16:41 Chousuke: database obfuscation sounds like something you should never need :/

16:41 but I can't say for sure, people have had need for many weirder things :P

16:59 jawolfe: hi, i'm new to clojure and have a performance question, if anyone can help ...

17:00 danlarkin: don't ask to ask, just ask :)

17:00 jawolfe: (defn mypow [x n]

17:00 (let [x (double x)]

17:00 (loop [ret (double 1.0) n (int n)]

17:00 (if (= n 0) ret (recur (* ret x) (unchecked-dec n))))))

17:00 RSchulz: Woo-Hoo! "[Pragmatic Bookstore] An updated version of Programming Clojure (PDF) is available"

17:01 Changes in this version include:

17:01 Chapter 7, Macros

17:01 jawolfe: is my best attempt at an efficient clone of the pow function for ints

17:01 int exponents i mean

17:01 but its still 10 times slower than the same code in SBCL

17:01 whidden: RSchulz: gee you're behind the times... already 1/2 trough that chapter :)

17:01 RSchulz: I'm probably just further down the email distribution list...

17:01 jawolfe: Is there something I'm doing wrong?

17:01 RSchulz: Now I have to wait for the gerbils.

17:02 jawolfe: (of course, I'm not writing for its own sake, just to get an idea how much performance hit I may take switching to Clojure from SBCL)

17:03 whidden: jawolfe: there looks to be an unchecked "times" in the recur.

17:04 Chouser: (doc unchecked-multiply)

17:04 clojurebot: Returns the product of x and y, both int or long. Note - uses a primitive operator subject to overflow.; arglists ([x y])

17:05 jawolfe: right, thanks. but x is a double ... is there a corresponding version of unchecked-multiply for doubles?

17:05 Chouser: oh. :-P doesn't appear to be.

17:06 jawolfe: you're running mypow multiple times (hundreds or thousands) with -server on your java command line?

17:06 RSchulz: jawolfe: I'm not an expert on numerics, but for a floating-point value, isn't all the same? You'll get an infinity if you overflow, not an exception, right?

17:07 jawolfe: Chouser: I'm running it 3 times, with very large arguments

17:07 Chouser: (dotimes [i 3] (time (print (mypow 1.00000001 100000000))))

17:08 RSchulz: I don't really care about overflow, I'm wondering about speed. I can't get the clojure version to run better than 10 times slower than the corresponding SBCL version.

17:08 RSchulz: My point is that there's not difference. The checking doesn't happen in machine code, it happens in the FPU.

17:09 jawolfe: Chouser: yes, with -server

17:09 RSchulz: (If I understand correctly, which I'm not really sure I do.)

17:09 jawolfe: OK, that sounds right to me ... I'm not sure where the slow-ness is coming from

17:09 Chouser: try (zero? n) instead of (= n 0)

17:09 jawolfe: Just that adding type declarations only sped things up less than 25%, when I was hoping for more like 20x.

17:10 RSchulz: For kicks, did you try it with float instead of double?

17:10 What hardware are you running on?

17:10 jawolfe: Chouser: you win.

17:10 that cuts down by a factor of 10

17:11 RSchulz: Wow.

17:11 Chouser: the problem there was both = and 0

17:11 RSchulz: The 0 is a BigInt?

17:12 jawolfe: 0 is an Integer I guess?

17:12 Chouser: I think = always uses Object, so it boxes its args. You could use == instead.

17:12 RSchulz: Acquired using valueOf(0)?

17:12 Chouser: Right 0 is also a boxed integer, so (== n 0) isn't any faster.

17:12 RSchulz: This is one of the things that Cliff guy was talking about in his presentation at the JVM summit.

17:12 Chouser: But (== n (int 0)) lets everything stay primitive

17:12 jawolfe: (= (int 0) n) is slow too

17:13 Yeah, you're right ... (== (int 0) n) does it.

17:13 Thanks for your help, I probably would never have figured that out.

17:14 * danlarkin makes a note to himself to always use zero?

17:14 Chouser: and of course 'zero?' is prettier

17:14 drewr: More Schemey anyway.

17:15 jawolfe: I didn't know about == either, I don't recall seeing it in any of the docs I've read

17:16 Chouser: But when trying to decide if Clojure is a good risk, don't forget that for tight loops like this that really matter in your real app, you can always drop to Java or even use JNI and native code if you can't find any other way.

17:17 In practice I think it's unlikely you'll ever need to.

17:17 jawolfe: Yeah, I realize that ... but I tend to do a fair amount of probabilistic stuff and so having at least the possibility of efficient math without dropping into Java (shudder) makes me much happier.

17:18 hiredman: lisppaste8: url?

17:18 lisppaste8: To use the lisppaste bot, visit http://paste.lisp.org/new/clojure and enter your paste.

17:18 jawolfe: One more quick question: my SLIME interrupts don't seem to work with Clojure, does anyone know what gives?

17:18 Chouser: hopefully the JVM will get tagged numbers eventually and make staying on the fast path less tricky.

17:19 lisppaste8: hiredman pasted "getters macro" at http://paste.lisp.org/display/71878

17:19 jawolfe: yes that would definitely be nice

17:19 hiredman: anyone up for some macro help?

17:20 RSchulz: What help do you have?

17:20 jawolfe: haha

17:20 hiredman: you are truly hilarious sir

17:20 Chouser: jawolfe: sorry, I don't use SLIME. But plenty of others do, surely they'll speak up any moment now...

17:21 jawolfe: Chouser: no worries, thanks ... what do you use?

17:21 hiredman: somehow I seem to be not evaluating something in my macro instead of accidently evaluating twice

17:21 Chouser: vim and a repl in a terminal.

17:21 RSchulz: I'll see if I can read ahead fast enough in chapter 7 of Stu's book to give you an answer.

17:22 drewr: jawolfe: I use SLIME, but don't much experience with CL. What do you mean by interrupts? You mean restarts?

17:22 RSchulz: Is it the 71878

17:22 ?

17:22 hiredman: RSchulz: yess

17:22 Chouser: clojurebot: macro help?

17:22 clojurebot: macro help is http://clojure-log.n01se.net/macro.html

17:22 Chouser: hiredman: ^^^

17:23 jawolfe: drewr: I mean, how do you interrupt what's running at the REPL

17:23 RSchulz: Doesn't a def have to appear at the top level? You've got it in a let body.

17:23 jawolfe: drewr: like, if you type (repeat 0)

17:24 hiredman: Chouser: the macro works fine if I just use it, but I try to use it in a map and I get something not right

17:24 RSchulz: defn is itself a macro...

17:24 drewr: jawolfe: I think someone was working on that, but I'm not sure the JVM is going to play nicely.

17:25 Chouser: jawolfe: setting *print-length* helps

17:25 hiredman: RSchulz: the macro forks fine if I call it directly (not using map)

17:25 er

17:25 works

17:25 RSchulz: I think it has something to do with when the macro is applied. It's happening once, not once per iteration of the map.

17:26 lisppaste8: hiredman annotated #71878 with "no map" at http://paste.lisp.org/display/71878#1

17:26 RSchulz: So you get the same result repeatedly.

17:26 jawolfe: Chouser: thanks

17:26 drewr: When it's unresponsive, I kill the java process and restart. :-/ You can write a function to do it automatically.

17:26 jawolfe: drewr: hmmm, so you have to killall lisp and start over?

17:26 hiredman: RSchulz: there is no :x key to that map

17:26 jawolfe: er i mean killall java

17:27 hiredman: the X must be the x paramter to the (fn [x] ...) in the map

17:27 RSchulz: It's suspiciously similar to (up-first-letter "x")...

17:27 hiredman: that is the only place x is anywhere in the code

17:27 drewr: jawolfe: Yeah, but I tend to keep my code in buffers where I can do a quick C-c C-k and keep going.

17:27 RSchulz: Change it to y and see if the result under map changes similarly...

17:28 jawolfe: drewr: makes sense, just a bit annoying ... interrupting can be especially nice for debugging

17:28 hiredman: RSchulz: uh, there is no (up-first-letter "x")

17:29 RSchulz: You know what I mean.

17:29 Chouser: try: `(name ~k)

17:29 jawolfe: drewr: but i'll take what i can get

17:29 hiredman: no I don't

17:29 RSchulz: Just try it. change that x to something else and see what happens under map.

17:29 drewr: jawolfe: Definitely. That's really my only complaint with Clojure compared with CL. Debugging feels more cumbersome. But you learn tricks that at least make it scientific. :-)

17:29 hiredman: Chouser: Unable to resolve symbol: k in this context

17:30 Chouser: oh. uh.

17:30 RSchulz: You don't think the X in the result, the up-first-letter and the x in your macro definition are suspicious?

17:30 Suggestive of some relation between them?

17:30 Chouser: `(name ~ky)

17:30 RSchulz: I think you've got your finger on it.

17:30 jawolfe: drewr: Good to know, thanks!

17:31 lisppaste8: hiredman annotated #71878 with "x to y" at http://paste.lisp.org/display/71878#2

17:31 jawolfe: OK, thanks for your help all. Time to try to write my first real clojure code ...

17:31 hiredman: oh

17:31 woa

17:31 bug

17:32 drewr: jawolfe: No problem. Don't be afraid to ask for help. We all usually learn something.

17:32 hiredman: still doesn't work, but what I pasted as k where ky should be in the macro

17:36 Chouser: ky is the object passed in. ':id' in one case, 'x' or 'y' in the others.

17:37 all of that has nothing to do with a value that may or may not be stored in a local or global var by that name.

17:37 gnuvince_: I thought ky was what was used to pass objects in ;-)

17:37 hiredman: ?

17:37 gnuvince_: </bad joke>

17:37 Chouser: ouhc

17:37 ouch

17:37 or not, I suppose.

17:38 hiredman: the only place in my code the getter macro could be getting the

17:39 x or y is from the fn surrounding the call to getter

17:39 Chouser: hiredman: think about it this way. The whole 'map' expression gets compiled once, so you macro gets run once.

17:40 hiredman: Chouser: if the macro gets compiled once, how come I get a whole list of the result of (defn) stuff?

17:41 Chouser: the expression your macro returns gets evaluated for each item in the seq

17:41 RSchulz: The code it produces gets mapped across the sequence supplied.

17:41 hiredman: ok

17:41 I guess

17:41 thanks

17:42 Chouser: to see what that expression is, use: (macroexpand '(getter f "family" x))

17:42 RSchulz: But keep in mind that macroexpand only expands until the top level of an expansion is no longer a macro application.

17:43 If there are nested macros, they won't be expanded by macroexpand.

17:43 Chouser: right, which is why you want to run it directly on the macro you wrote

17:43 RSchulz: Someone posted a pevasive macro-expander.

17:43 AWizzArd: you need to manually expand all macros if you want to write a code analyzer tool

17:44 RSchulz: I'd rather wait for someone else to write a code analyzer tool.

17:44 AWizzArd: RSchulz: eval must be doing it

17:44 RSchulz: I have code to write...

19:24 clojurebot: svn rev 1149; added :exposes-methods to gen-class, patch from Matt Revelle

20:25 pjb3: Has anyone tried building an Android app with Clojure yet?

20:26 Chouser: As far as I know, if Android worked applets would as well.

20:27 The last time I tried an applet it did not work, as a dynamic classloader is still created.

20:28 RSchulz: Do the Android tools operate at the JVM bytecode level, or at the Java source-code level?

20:28 Wait, I guess it must be bytecode, 'cause I heard of people getting Scala code working on Android.

20:29 So with AOT it should work, in principle, right?

20:38 Chousuke: RSchulz: bytecode AFAIK.

20:40 Chouser: RSchulz: yes, I think once the dynamic classloader is fully optional, applets and android ought to work

20:40 Chousuke: Google's being pretty clever. apparently they have their own bytecode to get around some restriction imposed by sun; so they're not officially Java-compatible, just de facto so :P

20:41 Chouser: or be able to be made to work.

20:41 RSchulz: I don't think I see the significance of dyamic classloaders.

20:41 Chouser: for android?

20:41 RSchulz: Actually, don't they _translate_ JVM to an entirely separate set of opcodes?

20:41 In general. What's the issue with dynamic classloading that's tripping up Clojure-generated bytecodes?

20:42 Chousuke: RSchulz: as far as I know, yes

20:42 mattrepl: it's register-based

20:42 Chouser: in non-AOT code, each Clojure expression is compiled into Java bytecode and then has to be loaded to run

20:43 RSchulz: Right.

20:43 Chouser: that's a problem for applets because they don't trust your custom dynamic classloader

20:43 RSchulz: Ah.

20:44 Chouser: it's a problem for android because the runtime platform has no ability to translate Java bytecode to what it needs to run -- that's a development and compile-time function.

20:44 RSchulz: How do the other dynamic languages, Groovy, Scala, handle this problem? I run my Grails app under Tomcat without problem.

20:44 Chouser: As far as I know, tomcat runs Clojure code fine.

20:44 RSchulz: But that's where AOT comes in, right? You get all the bytecodes up front for Android's tools to translate.

20:45 I think duck1123 would disagree with you...

20:45 Chousuke: that would disallow redefining stuff at runtime though. :/

20:45 Chouser: yes, AOT when fully realized should solve both the applet and android problems, and it's getting very close, but it's not quite there yet.

20:46 RSchulz: Right... So only those programs that, in essence, require the Clojure compiler at run-time (after AOT compilation) would be left out.

20:47 Chouser: I saw webjure hello world work in tomcat months ago. I'm not sure what's causing problems for duck1123, but I don't think it's anything inherent in Clojure.

20:47 RSchulz: right. like a REPL, for example.

20:47 RSchulz: I foresee the day that Android's device-resident run-time includes a JVM bytecode -> Android opcode translator.

20:47 Well, if I can't run a Clojure REPL on an Android phone, it's no-sale for me.

20:48 Chouser: you could write a clojure interpreter (in Clojure, if you wanted) and then you'd be all set.

20:48 RSchulz: Of course, I've yet to buy my first cell phone.

20:48 ... Yeah. I'm a real retro-grouch.

20:49 I wonder... Does anybody know of Android in a non-cell-phone device?

20:49 I guess that would be a PDA, basically.

20:49 I don't suppose there's much of a market for such a thing.

20:55 duck1123: Chouser: Webjure uses actual java, not pure clojure

21:16 jawolfe: Hi again. I'm wondering how to re-export vars from a namespace. Suppose I have namespaces a, b, and c, and i'd like to export some subset of all of their functions under a single new namespace d. Is this possible, and/or is there a more ideomatic way to produce "interfaces" to large collections of code? Thanks...

21:19 hiredman: jawolfe: http://clojure.org/api#toc358 should get you started

21:19 duck1123: so you want to be able to require d and have select fns from a b and c available?

21:21 jawolfe: duck1123: yes, exactly

21:22 hiredman: thanks, have already read the docs. all i was able to find were cryptic references to "public vars" without any details of what they are or how to make a var "public"

21:23 duck1123: jawolfe: in that case, I don't know of anything to really do that

21:24 jawolfe: duck1123: thanks

21:24 duck1123: but you could probablt have a macro to take a seq of symbols to require

21:24 def the macro and the var in d, then call it from e

21:25 you might not even need the macro

21:25 jawolfe: hmmm, that could work

21:26 although i'd rather not have to do anything extra in e

21:27 i'm not necessarily tied to this type of organization

21:27 but just trying to start a big project

21:27 and wondering how best to organize and compartmentalize things

21:30 RSchulz: I'm not sure it's documented or if I just saw it in reading Clojure code, but (def- ...) and (defn- ...) define "private" (non-exported) Vars.

21:33 Actually, the only private def form in core is (defn- ...)

21:34 Others, defmacro-, defonce-, defstruct-, defunbound- and defvar- are defined in clojure.contrib.def

21:35 jawolfe: Thanks, that helps

21:38 It still seems useful to me to have a way to declare refer'd symbols as public in a namespace

21:39 i guess i can simulate by defining new vars in the namespace with values copied from other namespaces ...

21:39 ah well, gotta go, thanks for your help.

23:01 danlarkin: anyone have a remove-whitespace function that won't remove whitespace from inside quoted strings?

23:06 zakwilson: danlarkin: I'm not sure I understand what you mean.

23:07 danlarkin: zakwilson: for unit testing my JSON parser I'm slurping in text files of JSON and encoding them to json and then decoding again and checking against the original

23:07 but part of the test is if it handles wacky unconventional whitespace in the original JSON file

23:08 so when I do (= original-json decoded-json) the original-json has wacky whitespace while the decoded-json doesn't, so they aren't equal

23:09 zakwilson: Ahh... I think I understand. I don't have a function like that, but I do have something for parsing CSV that you could probably adapt without much effort. Want it?

23:09 danlarkin: yes if you would be so kind, that would probably help

23:11 zakwilson: One moment - haven't used a pastebin before, and I need to copy a utility function or two out of another file.

23:17 lisppaste8: zakwilson pasted "function for danlarkin" at http://paste.lisp.org/display/71897

23:19 danlarkin: zakwilson: thanks! :) One thing I notice is that this function can overflow the stack if it recurses too deeply

23:21 zakwilson: danlarkin: How? I was under the impression that recur couldn't overflow the stack.

23:22 danlarkin: scan-for-delimeter calls itself

23:24 hiredman: clojurebot: mk-bean is http://gist.github.com/34229

23:24 clojurebot: Roger.

23:24 zakwilson: So it does. I thought I had changed it to recur, and I think it could be so changed.

23:25 I was using the self-calls to make a stack overflow possible after an early attempt resulted in an infinite loop.

23:25 Chouser: (apply str (re-seq #"\"(?:[^\\]|\\.)*?\"|\S" text))

23:25 that's probably buggy. :-)

23:26 you don't need to handle chunks quoted with 'single quotes' do you?

23:27 gotta scoot. later, all.

23:28 zakwilson: Just changed it to recur and it works.

23:28 danlarkin: ah yes zakwilson, I'm all too familiar with infinite looping :)

23:30 zakwilson: I've gotten in to the habit of using self-calls instead of recur when I'm not sure about the end-test in a function.

23:59 danlarkin: Chouser-away: I think that regex works, thanks

Logging service provided by n01se.net