#clojure log - Aug 23 2010

The Joy of Clojure
Main Clojure site
Google Group
List of all logged dates

0:01 qbg: You could do (merge-with + m {keyword-var 1})

0:01 hiredman: amalloy: nothing silly about it

0:01 qbg: I don't think that would be idiomatic though

0:03 amalloy: hired-man: it just feels like there ought to be an update function for symmetry with assoc(-in)

0:05 hiredman: amalloy: uh, thats what update-in is

0:06 mefesto: i think he means a non-nested version like how there is get/get-in assoc/assoc-in

0:06 update/update-in

0:06 wwmorgan: I've run across that actually. I expected an update that expects a single associative layer

0:07 amalloy: mefesto: yes, exactly

0:10 rhudson: amalloy: from your example form, I wonder if 'frequencies might be suitable for what you're doing

0:10 amalloy: hm. that's an interesting thought. as soon as you said it i started explianing why that doesn't make sense for me, but maybe it does

0:12 i'm tracking something that changes over time, though, and i need up-to-date counts at an arbitrary time, so frequencies would involve a lot of unnecessary recounts unless it can do some remarkable caching/optimizing

0:13 technomancy: polypus: lein-clojars is ... well, if it's not deprecated it's at least un-recommended

0:16 so... this post about Leiningen for Windows ... I have no idea what it means.

0:17 "Wow! I touched it and it reminded of REBOL interpreter. I downloaded leningen-1.3.0, and lein batch. I put leningen-1.3.0.jar on by path. Hmmm, I got electrified. Now, I can make boast by saying this 'is the real one'."

0:17 is that good?

0:17 defn: heh

0:18 I think it's good.

0:18 "I can make boast by saying this 'is the real one.'"

0:18 that sounds good.

0:18 technomancy: isn't rebol that forthy language with the terrible license?

0:20 defn: i wouldn't know

0:20 naming your language with a /bol/ at the end fell out of favor a long time ago though

0:20 so it wouldn't surprise me

0:55 * technomancy wishes shutdown-agents were reversible

0:55 amalloy: how robust is clojurebot? for example, if we give him a computation that never completes, is he smart enough to give up?

0:58 technomancy: amalloy: short answer: yes. longer answer: maybe...

1:08 danlarkin: granted it's only two characters longer

1:08 technomancy: technicalities...

1:11 danlarkin: a world of depth lies in those two characters

1:15 amalloy: the ellipsis makes it a healthy five longer

1:19 hiredman: clojurebot is semi-robust, Arkham has the possibility of making it much more robust but I am on the fence about using it

1:24 tomoj: clojurebot seems to have bad holes to me, but I guess the people here are generally nice to bots

1:27 hiredman: tomoj: the jvm sandbox disallows io

1:28 tomoj: right, so the box is safe, but is the bot?

1:29 hiredman: semi-robust

1:30 fixing those holes either ends up with lots of regexs or instrumenting an interpreter

1:33 http://github.com/hiredman/Arkham

1:34 if I do use arkham as clojurebot's evaluator I think I would like to run it out of process and have clojurebot communicate with it over a message bus

1:34 (more and more reasons to just leave things as they are)

1:36 slyrus: :require :only is supposed to make it so I can use unqualified names, right?

1:36 hiredman: :require doesn't have :only

1:36 slyrus: d'oh... :use :only

1:37 got it. thanks.

2:53 LauJensen: Good morning gentlemen

2:56 * slyrus knows he's made that same mistake before

3:20 sunkencityryleh: How can I convert a sequence of chars to a string? (str '(\F \O \O)) doesn't work and (reduce #(.concat %1 (.toString %2)) "" '(\F \O \O)) seems a bit unweildy

3:21 wwmorgan: ,(apply str [\F \O \O])

3:21 clojurebot: "FOO"

3:21 sunkencityryleh: ok tnx

3:30 slyrus: arggh... error: java.lang.IllegalArgumentException: No single method: make_atom of interface: chemiclj.core.PAtomGenerator found for function: make-atom of protocol: PAtomGenerator

3:34 I know I've been through this before, but no & args with protocol methods right?

4:10 amalloy: i think that's correct. they map directly to java methods, right?

4:28 LauJensen: AWizzArd: Man Im suffering today, thanks to you (neo)

4:56 Nikelandjelo: Web application runs on jboss server. One class contains static field, which must be initialized with value, which I put in context.xml in tomcat folder.

4:58 Now field is set in index.jsp. But I don't think it's good way. How can I set it on initializing time?

4:58 Or sorry, it's for #java channel :(

6:10 fliebel: hey

6:21 Would Clojure be a nice language to write an easy and fast network server, like Twisted and Node.js? Not necessarily evented or threading. Just thinking about the idea…

6:22 raek: I've heard that Aleph is a Node.js-like server

6:22 to make a simple network server is certrainly easy

6:23 check out http://clojure.github.com/clojure-contrib/server-socket-api.html

6:24 aleph: http://github.com/ztellman/aleph

6:24 this could be relevant too: http://dosync.posterous.com/22397098

6:25 fliebel: raek: Reading...

6:28 lenw: hi all, trying to implement some swing in clojure and just stuck with implementing an interface - e.g. for a button need to add an instance of an ActionListener - how do i do that in clojure please ?

6:29 fliebel: raek: beautiful!

6:33 *torn between EMCA and lisp* *wants Clojure in JavaScrip*

6:34 raek: lenw: proxy

6:34 lenw: raek: thanks

6:34 LauJensen: (defmacro onClick

6:34 [obj & body]

6:34 `(.addActionListener ~obj

6:34 (proxy [ActionListener] []

6:34 (~'actionPerformed [evt#]

6:34 ~@body))))


6:35 (onclick (JButton. "OK") (println "clicked"))

6:36 lenw: LauJensen: thats pretty nest thanks - it was the proxy bit that i am seeinf for the 1st time

6:37 raek: I was just going to point you to slide 31 of http://www.slideshare.net/skillsmatter/clojure-and-swing

6:38 but LauJensen's code makes that redundant

6:38 lenw: raek: thanks looks like i need the banna slide show anyways :)

6:39 Chousuke: doesn't that create a new proxy for every ActionListener? Or did proxy do "caching" of classes?

6:43 raek: each (onclick ...) form will result in the creation of a new instance (and a anonymous subclass) of ActionListener

6:43 Chousuke: It might be useful to do something like (defn action-listener [f] (proxy [ActionListener] [] (actionPerformed [evt] (f evt)))) and then (defmacro on-click [obj & body] `(.addActionListener ~obj (action-listener (fn [evt#] ~@body))

6:43 raek: ah, I see

6:44 this allowes one to reuse the same action listener for multiple buttons

6:44 lenw: yes thats neat

6:47 Chousuke: in general, you don't want to pack too much functionality into one macro or function :)

6:48 (or even if you do, do it for convenience and by composing the smaller functions)

8:10 AWizzArd: LauJensen: oh oh oh, yes, the first days are the worst.

8:11 LauJensen: I sent myself emails were I reported to myself my typing speed. I think after just a few days 250 kpm are possible.

8:13 LauJensen: :(

8:14 AWizzArd: LauJensen: about the swing example code above: I see it uses proxy. Can reify be used instead?

8:14 LauJensen: Yes

8:14 It was an almost 3 year old code snippet I had lying around :)

8:19 AWizzArd: rhickey: http://clojure.org/jvm_hosted could maybe also be updated to use reify vs proxy.

8:21 chouser: proxy reuses classes based on what they extend

8:21 leafw: ,(def a (reify java.awt.event.ActionListener (actionPerformed [evt] (println "buh!"))))

8:21 clojurebot: DENIED

8:22 leafw: that code fails. I have lots to catch up with 1.2

8:22 fliebel: What is the difference between reify and proxy?

8:23 AWizzArd: reify works for interfaces only, but produces very efficient/fast code.

8:23 leafw: AWizzArd: mind to correct my code above? What is the proper usage of reify for, say, ActionListener ?

8:23 AWizzArd: proxy is more general and can subclass, but it needs more lookups and is thus a bit slower.

8:23 fliebel: Ah, so it's good for implementing interfaces, not for inheritance.

8:24 That is one not-so-good thing about Clojure in my opinion: there are many ways to do something, unlike in Python(import this).

8:24 gerryxiao: how does pmap work?

8:25 chouser: thus proxy is meant for interop. reify encourages Clojure's ideas of good design

8:25 gerryxiao: ,(pmap inc (range 1 100))

8:25 clojurebot: (2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100)

8:25 gerryxiao: it means pmap fork 100 threads to do inc ?

8:26 or something?

8:26 chouser: gerryxiao: no

8:26 LauJensen: gerryxiao: pmap chunks the collection and passes it to n+2 threads, where n is the core count

8:26 fliebel: gerryxiao: I think it;s a pool

8:26 leafw: gerryxiao: it's a pool of Ncores + 2 or so

8:26 gerryxiao: ok

8:27 fliebel: I thought pmap wasn't in-order, but the above example seems sorted.

8:27 leafw: fliebel: pmap is like map--so ordered.

8:27 LauJensen: it works on a sliding window, so its sorted so to speak

8:27 Which is also something you need to consider against the performance profile of the task, in that threads will wait for each other

8:27 leafw: pmap is a drop-in replacement for map, when the fn to apply to each element is heavier than the threading overheads.

8:28 gerryxiao: but pmap works weird sometimes

8:28 chouser: pmap returns a lazy seq. as the consumer of the seq approaches the end of what has been generated so far, pmap kicks off another cores+2 simulateous tasks and collects the results in order at the end of the lazy seq

8:29 AWizzArd: leafw: you need to list 'this' explicitly in your reify example above.

8:29 leafw: AWizzArd: thanks!

8:29 back to python "self" then

8:30 chouser: the overhead of tasking threads from the pool and collecting the results is substantial -- the fn you pass to pmap needs to take a fair about of processor time in order to end up being faster than plain map

8:30 fliebel: leafw: But I assume also he advantages of self ;)

8:30 leafw: makes sense though; otherwise there isn't a handle for the object fields, and a magic keyword "this" could lead to errors.

8:30 gerryxiao: i used pmap to update drawing pictures in swing application, it work fine in linux, but werid in windows

8:30 LauJensen: chouser: or if the op doesnt take time, partition the input accordingly

8:31 AWizzArd: Try (time (count (map (fn [_] (Thread/sleep 1)) (range 1000)))) vs pmap

8:32 fliebel: AWizzArd: Ugh, my Clojure is rusting… What is the underscore for? It's basically a blank hole argument, right?

8:32 chouser: LauJensen: sure, though usually that partitioning and recombinding adds even more overhead

8:33 fogus: fliebel: "I don't plan to use this"

8:33 chouser: I wonder if it'd be worth making pmap checked-seq-aware.

8:34 AWizzArd: leafw: btw, your "this" can be named as you wish. You don't need to call it 'this'.

8:34 fliebel: AWizzArd: Just like self, but it's confusing to name it different.

8:34 AWizzArd: fliebel: the underscore is just the name for a variable. In Clojure it is idiomatic to use that name to signal to the reader of the sources that he can ignore that parameter.

8:36 LauJensen: chouser: are calls to partition that expensive?

8:38 fliebel: *sigh* Clojure is such a nice language, but PHP, Python and Javascript are hard to avoid, and sometimes easier for a certain job.

8:39 gerryxiao: (doall (pmap draw-pics-fn data-lists)) works in linux, but in windows, some pictures werenot drawed when updating paint

8:39 i have to change pmap to map

8:40 _fogus_: fliebel: Use the best language for the job. You can't go wrong that way.

8:40 gerryxiao: map works both in linux and windows

8:40 Raynes: Clojure isn't trying to replace other languages. Just trying to provide a new tool for your box.

8:41 AWizzArd: The problem is that it infects your mind and makes you feel bad when you must read/type code in other langs (:

8:41 fliebel: all very true :)

8:41 AWizzArd: fliebel: best way is to convince your boss to rewrite the complete code base in Clojure, whatever you have.

8:42 fliebel: I don't have a boss, so that's easy :)

8:42 Raynes: I'll be your boss.

8:42 Fetch me some tea.

8:42 fliebel: Raynes: Try sudo...

8:43 Raynes: sudo fetch me some tea.

8:43 fliebel: *gives tea to Raynes*

8:44 There are currently 2 projects I have to do in non-clojure. One is because it is in the browser, the other because Twisted preovides imap and smtp while aleph does not.

8:45 *waits for Clojure in Clojure, and the for Clojure in JS and everywhere else*

8:47 bobo_: i work in php, beat that

8:48 fliebel: bobo: Hey, you could run quercus and bring your PHP to Clojure :)

8:49 bobo_: i wouldnt be allowed to use it anyway

8:50 AWizzArd: fliebel: depending on your customers need *maybe* a swing applet would be okay, as the client. Don't know.

8:54 fliebel: AWizzArd: Nope

8:54 Bjering: I have finished my chat-server, I am sure it is full of very suboptimal clojure. As it is it works very nicely with ~5k clients (using my old C++ stressbot to simulate them). However it lags and eventually fails with 20k clients and my C++ server used like 2% CPU at 50k clients. Any good suggestions on how to profile a clojure chat server?

8:56 * defn needs a de-stress bot

8:56 * Raynes needs a hugbot.

8:56 Raynes: Oh, wait.

8:57 $huggle Raynes

8:57 * sexpbot Hugglez Raynes. I lubs yous.

8:57 dnolen: Bjering: VisualVM ? https://visualvm.dev.java.net/

8:57 Raynes: !

8:58 Bjering: googling also let me know about TPTP for Eclipse, as I use counterclockwise perhaps I should try this one? Anyone succeeded profling clojure code with it?

8:59 dnolen: Bjering: It's pretty popular. YourKit also seems to be used by many.

8:59 Bjering: Alright, Ill test TPTP first, and then visualvm

9:06 LauJensen: Bjering: Is the source code online somewhere?

9:07 Bjering: No, but I dont mind putting it somewhere

9:08 LauJensen: Try gisting it

9:08 Raynes: http://gist.github.com

9:08 Bjering: It is messy though, I thought I found a good way to organize my message passing, but I think I overengineered it and created a little mess. I want to sort that out if I proceed ofc. But the primary reason for this excercise was to stresstest it. But perhaps I should start with just making it clean first...

9:08 LauJensen: Raynes: you mean M-x gist-region

9:08 Bjering: it is 7 files, make 7 gist posts?

9:09 Raynes: LauJensen: Assuming he has gist-mode.

9:09 LauJensen: done

9:14 Bjering: alright

9:14 main.clj https://gist.github.com/fd1b5b2b1bbe08ff8c30

9:15 fliebel: Bjering: You can attache mutiple files to a gist

9:15 Bjering: server.clj https://gist.github.com/5b4f5a7d299068c7b978

9:15 session.clj https://gist.github.com/bd04ff4594a3f5413bf2

9:15 fliebel: Ah, too late ;)

9:15 user.clj https://gist.github.com/e81bb48793d311df1eea

9:15 channel.clj: https://gist.github.com/fd6b3bf4d2426b95aed0

9:15 common.clj: https://gist.github.com/9af1d5dc17539d94cc77

9:16 fliebel: Bjering: You use Netty? Have you had a look at Aleph?

9:16 Bjering: If anyone looks at it, be ruthless, I suspect it is bad in many ways and my purpose is to learn!

9:16 LauJensen: First off, using Netty for something this I/O heavy is likely a mistake

9:17 fliebel: LauJensen: What would you use?

9:17 LauJensen: Something synchronous

9:17 fliebel: LauJensen: With threads and stuff?

9:17 LauJensen: Yes

9:17 chouser: Bjering: did you know you can put multiple files in one gist?

9:17 LauJensen: When a requests exceeds a certain threshhold, the async performance will crater

9:17 fliebel: chouser: I told him.. to late

9:18 LauJensen: Compare with something like Jetty you dont get a fraction of the scaling ability

9:18 Bjering: LauJensen, it is a "port" of a C++ boost::asio C++ server that is async and does 50k klients without sweat.

9:18 chouser: fliebel: ah, sorry, missed that among the other posts.

9:18 Bjering: chouser: I know now, will do next time :)

9:18 LauJensen: Bjering: Likely its spending less time on each request then

9:19 fliebel: LauJensen: What dimension of a request determines if async or sync is better? I assume it has something to do with the duration and the amount of waiting that is done?

9:20 Bjering: LauJensen: I am curios, all my research and all my own attempts in C++ lead me to think asynch io was _the_ way to reach anything near 50k clients. But I take it its because in C++ that would mean sync would mean 50k native threads and that is NOT a good idea I am sure.

9:20 LauJensen: fliebel: Duration. You probably saw the comparison where Netty blew Node.js out of the water. Thats the sweet spot for async apps. Once you add a little I/O, you die on performance

9:21 fliebel: LauJensen: I don't think I did...

9:21 LauJensen: But aren't Jetty and Node both async?

9:21 *Netty

9:22 Bjering: My stressbots are sending a "Hello I am here" message every 1-10 (uniformly random) seconds to its channel, I have 10 clients in each channel, each mesage is then sent to all members of that channel.

9:22 LauJensen: Not Jetty

9:22 http://groups.google.com/group/clojure/browse_thread/thread/f67b156b39f8e2ed/fad2f3ddc4c96ec7?lnk=gst&q=aleph+timeslice#fad2f3ddc4c96ec7

9:22 I'll let Peter Schuller explain

9:23 AWizzArd: Netty with 100k clients: http://www.jboss.org/netty/performance/20090607-asalihefendic.html

9:25 Bjering: LauJensen: I am happy to rewrite using another IO-lib if it is the reason. Now I really dont think it is. I think it must be something stupid I do in my clojure code (beeing clojure (and basically any jvm-language) newbie).

9:25 LauJensen: Bjering: the stressbot is just sending messages right? So its not logging in and out of channels?

9:26 Bjering: Right, all channel-membership is hard-coded (in main.clj)

9:26 LauJensen: (Bjering, before you rewrite, check out Peter Schullers explanation, its sound)

9:28 Bjering: I think my first step would be to run a profiler and find the bottleneck (jvisualvm will do fine), and then perhaps check back here with the details

9:31 Bjering: Well, I have very high concurrency (I think, ~25k IO actions per second) and each call is very fast (no DB-access just a little doseq over 10 items, queuing up async-sends). And those are the cases where Peter Shuller sais an asyncrounous model ges the most advantage. If I read that post correctly.

9:31 ofc I am probalbly getting the worst of both worlds as I mix in agents in it. Paying for threads anyway.

9:32 Learning to profile something non-trivial like this will be well worth learning anyway, I'll do that.

9:33 LauJensen: I would really be disappointed if you cant get that app to handle 50k connections

9:34 Bjering: LauJensen: When you did your chat-bot on the course you held, did you ever stresstest it?

9:34 LauJensen: No - It was mostly an interop exercise

9:37 Bjering: LauJensen: Any comment on my use of partial to return a function I can use an object and pass messages to it? I feel huge guilt of overenginnering that one, but it did give me very nice message passing. But I feel guilty trying to force my OO thinking on this problem.

9:39 LauJensen: Bjering: Where is dispatcher defined?

9:39 Bjering: same file/namespace earlier

9:40 LauJensen: Bjering: Every call to partial creates a new fn, so I think that might be hogging some resources

9:41 Bjering: A function is expensive?

9:42 LauJensen: If you're creating thousands of functions every second, instead of just calling them, then yes I think so

9:43 Bjering: I am so used to template metaprogramming in C++, creating a function there isnt quite the same thing, the cost is deducted at compile time.

9:45 LauJensen: ,(time (dotimes [i 1e6] (partial + 1 2)))

9:45 clojurebot: "Elapsed time: 273.291 msecs"

9:45 LauJensen: ,(time (dotimes [i 1e6] (+ 1 2)))

9:45 clojurebot: "Elapsed time: 130.757 msecs"

9:45 LauJensen: ,(time (dotimes [i 1e6] (+ 1 2)))

9:45 clojurebot: "Elapsed time: 186.13 msecs"

9:45 LauJensen: Should be about 80% - 90% difference I think

9:46 Bjering: ok, thats ok, I am still at the stage as I need to learn how to avoid the x100 slowdowns, the x2 slowdowns I'll fix later :)

9:47 I would pick clojure for this project if it was withing a x5 performance of my C++ without hesitation.

9:48 LauJensen: Then I think you should grab the profiler

9:48 Bjering: yep, dipping my toes into visualvm now. I'll let you know what I learn.

9:52 wow, all tools in the Java worls are so nice :) I get good stuff from visuaVm without even doing a tutorial first, it found my running process!

9:52 LauJensen: Yea its almost too easy

9:52 Bjering: clojure.lang.Reflector.invokeMatchingMethod 82%

9:52 LauJensen: Then what you want to do is hook it into the process. Get th clients connected, then start profiling, stop after a couple of minutes

9:53 That will narrow it down for you quite a bit

9:53 Bjering: its what I did

9:53 LauJensen: Bjering: Ok, compile your code using (set! *warn-on-reflection* true), see where its reflecting

9:53 Then sort that out with type-hints. Any reflection will grab 500x performance or so. (ball park)

9:54 Bjering: second place goes to org.jboss.netty.socket.nio.SelectorUtil.select, but thos 7.7% is probably ahrd to avoid.

9:54 will do

9:54 bobo_: Bjering: are you doing this for work?

9:54 Bjering: bobo: Yes

9:55 bobo_: it just sounds like so much fun. hard to belive its work =)

9:55 Bjering: Its my own company ;)

9:55 bobo_: ah!

9:57 bartj: Bjering, company link ?

9:59 Bjering: bartj: None, yet, I have link to my old company, but thats not really relevant. Setting up a new one now, todo a _fun_ web-mmo-rpg/strategy game with real syncronous interaction (not that async bs that 99% of webstrategy games today use, especially the facebook ones).

9:59 _fogus_: dnolen: Are you planning on a Chambers/Chen impl?

10:00 Bjering: hmm, attaching visualvm to my process made it unresponsive to ctrl-c ?

10:00 bartj: Bjering, may the force be with you

10:00 Bjering: thanks

10:01 alright plenty of warnings on reflection

10:02 LauJensen: Great - If any of those are in repetitively called code, you'll get a huge boost

10:02 Bjering: such as getBytes called on every string every time I send :)

10:02 LauJensen: haha

10:02 Yea, something like that

10:03 Bjering: ok, so this is where I use type hints?

10:03 line 24 in this one: https://gist.github.com/bd04ff4594a3f5413bf2

10:04 LauJensen: If its guessing what str is, then prefix it with ^String str

10:06 Bjering: ok, then next one is wrappedBuffer on the same line, that is a static method, didnt thought that needed type hint?

10:07 eh not same line, but almost, line 22

10:07 ah its the overload on arguement causing trouble I guess

10:11 How should I type hint that?

10:18 this is the java signature of the method I reflection warns me about: public static ChannelBuffer wrappedBuffer(byte[]... arrays)

10:19 can I? if so how do I? type hint so that the warning goes away.

10:21 fixed it

10:22 #^byte[][]

10:26 chouser: really? that worked?

10:26 Bjering: yep

10:26 chouser: huh, cool.

10:27 Bjering: well, warning went away, if it crash I dont knwo yet ;)

10:32 LauJensen: Bjering: Hows performance now?

10:33 Bjering: Still working my way through the warnings, I have an odd case now where the type I "know" it should be doesn seem to HAVE the method I am calling on it... I wonder how that works...

10:35 chouser: Bjering: perhaps the args of the method are of the wrong type(s)?

10:36 Bjering: Yes, I have used a confusing argument name and tricked myself...

10:36 LauJensen: well played

10:40 npoektop: hi! i've just set up lein with clojure 1.2. When i do lein swank, it just starts server. How to make it run the code and start server? I just did (swank/start-repl) in my main.clj before. Is there a new way?

10:46 Bjering: #^byte[][] crash my program :(

10:46 gives me #<ClassCastException java.lang.ClassCastException: clojure.lang.PersistentVector

10:46 cannot be cast to [B>

10:47 chouser: a vector is not a byte array

10:47 Bjering: but it DID resolve the static method though

10:47 or I dont know, atleast the warning went away

10:47 so now I am back

10:48 chouser: that suggests that your hint is right and that the method you're trying to call is expecting that kind of array

10:48 but that the actual object you're passing in is a vector not an array.

10:48 Bjering: The problem is on line 22 here: https://gist.github.com/bd04ff4594a3f5413bf2

10:49 If (into-array) dont give me the byte[][], how do I get it?

10:50 ,(into-array [(byte-array (byte 0)) (byte-array (byte 0))])

10:50 clojurebot: #<byte[][] [[B@41785a>

10:50 Bjering: its the right type...

10:51 * Raynes is surprised no one has created as Clojure lint yet.

10:51 LauJensen: Raynes: Christophe is working on it IIRC

10:52 Raynes: Oh. Of course he is.

10:53 mefesto: http://docs.jboss.org/netty/3.2/api/org/jboss/netty/buffer/ChannelBuffers.html#wrappedBuffer(byte[])

10:53 Bjering: does the method expend just an array of bytes and not an array-of-array?

10:53 *expect

10:54 looks like you are passing it a byte[][] when it wants byte[] only

10:54 Bjering: This is how it looks: public static ChannelBuffer wrappedBuffer(byte[]... arrays)

10:55 there are many of it

10:55 mefesto: Bjering: that is equiv to: wrappedBuffer(byte[], byte[], ...)

10:55 Bjering: I want the one with var-args

10:55 ok

10:56 mefesto: So how do I type hint that?

10:57 mefesto: Bjering: this is just for your 'encode fn right?

10:57 Bjering: yes

10:57 mefesto: Bjering: do you need to type hint on it?

10:58 Bjering: Ah I thinkg I got it working, getting rid of the into array

10:58 mefesto: http://gist.github.com/545640

10:59 Bjering: well, your version is missing the ending 0 that flash XMLSocket need as delimiter

10:59 npoektop: i do not get it. How to use lein swank? If i do lein swank, it just starts a server. What's next? I should do M-x slime-eval-buffer in my core.clj?

10:59 mefesto: Bjering: sorry, http://gist.github.com/545640

11:01 Bjering: that doesnt resolöve, but this does: https://gist.github.com/4ea661b655ee113bb086

11:01 mefesto: Bjering: ah i see

11:01 Bjering: but dont work...

11:05 mefesto: Bjering: hmmm i guess it's having probs with variable arg java methods

11:05 Bjering: yes, its all failing all the time :(

11:06 mefesto: Bjering: would it be good enough to append the byte 0 at the end of the (.getBytes str ...) call and pass just the single byte[]?

11:06 Bjering: perhaps as last resot make a little own static Java wrapper of my own?

11:06 sure, it is probably good enough, and prettyer thatn getting Java in there.

11:08 dnolen_: _fogus_: I saw you messaged me on the backlog.

11:11 _fogus_: dnolen: I saw that you were chatting about predicate dispatch and was curious if you plan to go down that path?

11:12 dnolen_: _fogus_: it is interesting to me, also seems related to your Trammel work.

11:12 mefesto: Bjering: maybe this would work? http://gist.github.com/545675

11:12 _fogus_: dnolen_: I think so too. The reason that I ask is because I've started down that road and if you plan to do so also perhaps we could collaborate

11:13 mefesto: Bjering: might need to add your type hint again ^byte[]

11:13 Bjering: mefesto: This also worked: https://gist.github.com/b5422d6b599e91835eed

11:13 dnolen_: _fogus_: I'd be up for that. do you have anything in a repo yet?

11:13 mefesto: Bjering: cool

11:14 _fogus_: dnolen_: I do not, but I hope to have a unification lib pushed out soon

11:14 mefesto: Bjering: i like your version much better :)

11:14 Bjering: new test run, 15k connections ok response time but still 75% CPU, time to profile again

11:15 20k and it still works

11:15 much beter than before

11:15 I'll let it run and see if it get all the way to 50k this time

11:15 LauJensen: Look forward to hearing the results

11:18 Bjering: Now it dies at 35k connections

11:19 durka42: is anyone using leiningen on OS X?

11:20 dnolen_: _fogus_: nice! is there some good reading you followed for your unification design?

11:21 npoektop: durka42: i do

11:21 durka42: npoektop: did you upgrade it recently? i just did and now when i run "lein help" my CPU usage spikes over 100% and nothing happens

11:22 npoektop: i've just downloaded it from it's site, and it works fine

11:22 * durka42 reinstalls it

11:23 _fogus_: dnolen_: I started with the description in *Problem-solving Methods in Artificial Intelligence* and then looked at the impl in PAIP. Then Clojure-ified it

11:29 durka42: technomancy: have you had any other reports of this issue?

11:36 qbg: Yes! One of the textbooks for a university class that I'm taking is Programming Clojure

11:36 tomoj: awesome

11:36 qbg: I've accomplished some of my goals

11:37 mrBliss: qbg: I envy you

11:37 pdk: wowzers

11:37 qbg: I could probably make by without the book, but I bought it anyways

11:37 Financial aid can help cover it :)

11:40 dnolen_: _fogus_: thx, I'll look into those.

11:41 _fogus_: dnolen_: I'll ping you when I finally finish up this lib

11:41 Bjering: I think the profiler interefere alot though, with it it can start failing at only 1k users.

11:41 but

11:41 Before that it tells me 86% of CPU is in in org.jboss.netty.socket.nio.SelectorUtil.select

11:42 time to read that source code

11:42 Bahman: Hi all!

11:42 qbg: Hi Bahman

11:44 Bahman: Hi qbg!

11:44 npoektop: is it normal that lein-run needs clojure and clojure-contrib 1.1 jars while i'm using clojure-1.2?

11:46 Vinzent: npoektop, v1.1 is in deps of this plugin

11:47 npoektop: Vinzent: so it won't crush 1.2?

11:52 durka42: ninjudd: i'm trying to install cake on OS X. it fails because it can't find bake-0.4.4.jar in cake/lib/dev - do you know how to fix this?

11:57 Bjering: Anyone know if Java Hotspot Client VM for Windows 7 actually use iocp for its NIO.Selectors, all I read is that it use select, which on windows is basically broken (well for >64 sockets atleast). Perhaps this is what explains why by boost::asio solution works so much better on my windows machine.

12:04 technomancy: durka42: if you run something like "lein help" in ~ where the src/ dir is huge, there's nothing leiningen can do; the JVM will just freak out when you try to load anything

12:04 that's just a bug in Java

12:05 durka42: technomancy: same with "lein version" in an empty directory

12:06 technomancy: must be something else then; maybe a mac-specific quirk

12:20 slyrus: durka42: wfm

12:31 LauJensen: Bjering: 1) Yes the profiler inters enormously, so everything looks weird in the numbers, 2) Time to try something other than Netty, if you have to dig into the source of it?

12:32 Bjering: The source was a dead end anyway, only contained one line and that was into a Java Nio class

12:36 dnolen_: Netty seems pretty well suited to this task. Bjering have you done any memory profiling?

12:36 npoektop: hi! I've just ran (println (clojure-version)) using lein run. It says 1.2. I ran lein-1.3 with lein-run plugin that needs 1.1. Won't those 1.1 jars needed for plugin interfere with 1.2 jars for code?

12:36 Bjering: just rerun my C++ server... 50k connections, 2% cpu, its like a x100 difference :(

12:37 LauJensen: Bjering: Well, you're not done optimizing yet :)

12:37 Bjering: dnolen_: Nope, I am new to trying to make high-perf jvm programs. So there might be another story then the 85% CPU in org.jboss.netty.socket.nio.SelectorUtil.select the profiler told me about?

12:38 ninjudd: durka42: about to leave for work, but I can help you when I get in. for now, try removing lib and rerunning

12:38 dnolen_: Bjering: you probably need to startup with JVM process with a lot of RAM

12:39 LauJensen: dnolen_: So you think its just dying on memory ?

12:39 dnolen_: Earlier he was at 75% CPU with 20k connections

12:40 Bjering: I wonder if the -server switch to java does me any good

12:40 dnolen_: Bjering: I would start with a gig and go from there.

12:40 Bjering: you should always user -server

12:41 LauJensen: there could be a lot of things at play, the I doubt Netty is the problem.

12:41 LauJensen: Bjering: How much mem does your system have?

12:41 Bjering: 6 GB

12:42 but I run 32 bit java

12:42 LauJensen: java -server -Xms3G -Xmx3G then.

12:42 (at least I think G is the notation for Gigabyte)

12:42 dnolen_: Bjering: at the same time you need some realistic expectations. For a such a simple app, I think the CPU and memory profile of the C++ version will always trump Java even if they can serve the same # of connections.

12:43 Bjering: dnolen: I am hoping for 10% CPU at 50k, that is x5, then Clojure with STM will be my pick for game logic rathern then spending the next 6 months fiddling with locking strategies in C++ for my game logic.

12:44 LauJensen: Bjering: I didn't see you using the STM anywhere in your code, did I miss something?

12:45 Bjering: LauJensen, no not in this app, but this is just a benchmark, I can't write my entire game twice ;)

12:46 no locks to speak off in the C++ version either...

12:46 LauJensen: Bjering: I understand, but the different ref types have different performance profiles. The STM is much slower than something like atoms for instance - different use-cases entirely

12:46 Bjering: yes, I'll try to pick wisely

12:46 I dont expect many collisions though

12:47 game objects will be far apart, rarely interfering

12:48 LauJensen: If you're picking between atoms and refs, the question is simple "Do you need to coordinate change?", if you do, refs are your only option

12:54 Bjering: it lags less, my ping is now 0-1 ms just like the C++ (I am at 15k uses now), but the CPU is maxing and making it hard to type in irc window even!

12:54 ping starts rising at 20k connections though.

12:56 AWizzArd: Bjering: I did not follow exactly what you are talking about. I saw “netty” before. Do you work on some kind of web server?

12:56 Bjering: AWizzArd, no a chat server

12:56 connections are not closed

12:57 AWizzArd: irc or your own protocol?

12:57 dnolen_: Bjering: heh. what other calls were taking up yr CPU time besides select?

12:57 LauJensen: AWizzArd: read the log from 15:00 up to now, its mostly been about this client :)

12:57 Bjering: Well, its a benchmark/learning experiment. If it works well it will become a gamesever for a flash-strategy-rpg-mmo....

12:58 slyrus: ok, so if I split my (library) code up into a bunch of files and a bunch of namespaces, how do I go about providing a simple interface to the library? A core file with a bunch of defaliases?

12:58 make people remember which ns things belong to?

12:59 chouser: you can split into multiple files without splitting to multiple namespaces.

13:00 slyrus: true... I was under the impression the "idiomatic" way to do things in clojure was one ns per file, however.

13:01 and how is the order of loading of multiple files with the same ns controlled?

13:02 and, in general, how does clojure know which files I want loaded when I :use or :require an ns?

13:02 (:use and :require operate on ns'es, not files, right?)

13:03 once again, I'm used to the CL way of telling asdf which files are supposed to get loaded and having the package thing somewhat decoupled from the loading thing

13:03 Bjering_: hmm... does java -server means "I give you permission to totally hang my user interface if you think you need to" ?

13:04 Chousuke: no :P

13:04 LauJensen: it means, use a compiler which optimizes more heavily

13:04 Bjering_: had to hard reset my win 7 machine...

13:04 Chousuke: if your user interface hangs, the OS is not doing the scheduling right.

13:04 LauJensen: Bjering_: Yea, developing on a Windows system.. perhaps not the best idea :)

13:04 Chousuke: or you're heavily oversubscribing your RAM capacity.

13:06 slyrus: so to spill my ns across multiple files on file gets a (ns ...) and the others get (in-ns ...)? is there a particular order in which those are loaded? does require load all of the files with the in-ns form?

13:06 Bjering_: I am somewhat wondering if perhaps its windos that scews my performance measurements also. As I know boost::asio is very well built on top of IOCP, windows preferred high performance network api. Perhaps Java.NIO/netty is less well tuned on Windows systems but very competetive if I ahd a system with epoll or such. ?

13:07 qbg: require loads a file at the expected path; that file would have to load the others

13:07 dnolen_: Bjering: it would be interesting to test on Linux yes. But looking around briefly Java NIO on Windows seems to use IOCP as well.

13:07 chouser: slyrus: yeah, I think that's frequently recommended. you have to load those other files explicitly at the end of your main (ns ...) file, so you get to define the order.

13:07 slyrus: ah, ok! thanks chouser. I was missing the explicit load step.

13:08 chouser: clojure/core.clj does this -- looke for "helper files" near the end.

13:09 slyrus: (I think) it would be nice if ns forms had some way of re-exporting symbols

13:11 and I'm I correct in surmising that defalias yields a new var with the current value of the old var s.t. if I, say, reevaluate a defn for original val, the alias points to the old function?

13:12 Chousuke: (doc alias)

13:12 clojurebot: "([alias namespace-sym]); Add an alias in the current namespace to another namespace. Arguments are two symbols: the alias to be used, and the symbolic name of the target namespace. Use :as in the ns macro in preference to calling this directly."

13:12 Chousuke: I think defalias uses that.

13:13 slyrus: not the way I read it

13:13 ICBW

13:13 Chousuke: hmmh

13:13 * technomancy grumbles at namespaces split among multiple files

13:13 slyrus: grumbles at having to :use/require 6 namespaces for 1 "logical" set of interfaces

13:14 and at technomancy :)

13:14 qbg: If your namespace is as big as clojure.core, you have done something wrong

13:15 technomancy: right; clojure.core is not generally a good example of how to structure user code

13:16 anyway, if it takes six files to implement what you want, you should still be able to gather all the external interface functions into one or two nses

13:16 just don't use immigrate.

13:16 LauJensen: technomancy: Im trying to learn this and thought of you :) http://neo-layout.org/

13:16 slyrus: technomancy: yes, that's what i'm trying to. trying to figure out the best way to do that

13:16 and i've never heard of immigrate

13:18 the other wrinkle is that I've got protocols, which can-get/need-to-be loaded early, I've got the implementations which I'd like to keep as discrete, disconnected units, and then I've got the external interface functions.

13:19 Bjering: Perhaps I should rewrite the server in Java, just to really isolate the issue and have a good way to approach people like those on Nettys mailing list.

13:20 dnolen_: Bjering: not a bad idea, it would be cool if you wrote up your experience somewhere. I'm interested in how this all turns out for you.

13:21 Bjering: also if you put your code up, I'd be interested in checking it out / running it.

13:22 slyrus: technomancy: the problem is that i'd like both the protocol methods (is that the write term?) and the other external functions in the same ns, but I need to define the concrete classes in between those two steps. Easy enough if everything's in one file, harder if you split it up.

13:27 does putting all of the protocols in their own ns and expecting folks to :use/:require that make sense?

13:27 oh, wait, I can do the load trick here. nvm...

13:27 * slyrus needs more coffee

13:28 Bjering: dnolen_: https://gist.github.com/25aa623f0df970b7282d

13:28 I included main.cpp for my stressbot too

13:30 and a little readme for the protocol

13:31 dnolen_: Bjering: cool, will give a looksee later when I got a bit more time.

13:32 Bjering: very cool, Iäll do that java rewrite. If nothing else I should write 500 LOC java per year to keep Java on my resume without lying too much....

13:36 LauJensen: Bjering: Wouldnt it save you a ton of time, just to test it with Jetty or some already written server first?

13:36 Bjering: LauJensen, sure, how do I do that?

13:36 webserver isnt what I need

13:37 LauJensen: I haven't looked into your options - I just occured to me, that you could save some time by taking that route first

13:37 Though, that route may start on Google :)

13:37 Bjering: I need long-living connections, and I do not want to do Comet style hacks...

13:38 ninjudd: durka42: should be fixed now. looks like i uploaded a bad jar for cake when i was playing with the release task this weekend

13:38 LauJensen: Bjering: Hang on, memory chip activated

13:39 ninjudd: durka42: you may have to clean out lib. let me know if it works. and thanks for catching the problem!

13:39 LauJensen: Bjering: check out James Reeves reply: http://groups.google.com/group/compojure/browse_thread/thread/1f0df9d6656fa069/56fba3d74b15c38a?lnk=gst&q=long+lived#56fba3d74b15c38a

13:39 That was actually for a chat I did as well

13:40 durka42: ninjudd: where should i download from? the version from git clone does the same thing

13:41 Bjering: LauJensen: i have written a long-poll solution myself in C#, in my opinion it was a brittle thing, and wastful of bandwith etc. Why not use proper TCP-sockets when that is what you really want?

13:41 LauJensen: In this case because I just need a simple web based cht

13:41 chat - took almost no time to implement

13:43 Bjering: But did you stresstest it? The C# thing I wrote died horribly early arguable becuuse the way I hooked it into IIS, but I have a very time thinking that all that extra communication that comes with a http/long poll is going to help performance.

13:43 very hard time

13:47 LauJensen: I didn't no

13:47 I think I saw a socket interface in contrib earlier today though

13:47 Or maybe my memory chip is failing

13:48 Bjering: Well well, the way this is going before it is over I shall have made a Jetty version, and a node.js version and a Python Twisted version before I can even start thinking about delivering business value ;)

13:50 ninjudd: durka42: forgot to bump the version... pull and try again

13:52 durka42: ninjudd: now it just says "unable to find cake version on clojars" :(

13:55 ninjudd: hmm. 0.4.6 is there. I'll fix it when I get to the office. that's what I get for trying to fix code fro

13:55 m my phone

13:55 durka42: heh, no problem

13:57 ninjudd: FWIW, using the gem-installed version, it tries to fetch deps clojure and clojure-contrib, and can't find them, even though they are there

13:59 slyrus: hmm... it feels a little dirty, but some explicit (use ...) sprinkled throughout my chemiclj/core.clj allows me to define the protocols and helper functions in the core ns, and the defrecord forms and what not in more specific ns'es (bond, atom, etc...)

13:59 LauJensen: hehe, sounds a little dirty as well

14:00 slyrus: I'm open to alternatives, but this is the best I've come up with so far.

14:04 it's in here: http://github.com/slyrus/chemiclj/tree/refactor2 if anyone's curious

14:05 LauJensen: curious, but a little pressed for time tonight

14:11 ninjudd: durka42: seems to work from a fresh git checkout for me. if you want i can help you debug in #cake.clj

14:21 slyrus: ok LauJensen, let me know if you get a chance to take a look and have any questions

14:22 LauJensen: I will be sure to do that

14:22 slyrus: awesome

15:06 arkh: clojurebot: ping

15:06 clojurebot: PONG!

15:15 tomoj: grr.. clojure.core.Vec isn't print-dup'able

15:24 hamza: guys, I need to read a file containing clojure code but some variables defined in the file are defined in the namespace I am reading it in something like the following (let [content "asdasd"] (eval (read-string "(println content)"))) file contains print content but contant is defined in the file reading the script is it posssible to make this work?

15:25 mebaran151: any recommendations for a good mail client in emacs? I realized that instead of cutting and pasting buffers into my mail client, it would probably be more efficient for me to integrate mail handling directly into emacs

15:26 LauJensen: mebaran151: I think currently wanderlust is your best bet - I demoed it in my last screencast

15:27 bobo_: is there anywhere i can find a clj-cassandra that actually have al its dependencies? tried 0.1.1-0.1.3 and master. all filing on deps

15:27 *failing

15:29 bmh: Any Penumbra users here?

15:29 technomancy: mebaran151: gnus is great, but it's very different from traditional MUAs.

15:29 mebaran151: can I set it up to work with imap properly?

15:29 * Raynes still uses POP3.

15:29 Raynes: And Evolution. I'm too frightened of gnus.

15:30 mebaran151: I'm on Evolution right now

15:30 LauJensen: mebaran151: If by properly you mean get notified every time a mail rolls in no, thats an outstanding bug in Wanderlust, so I wouldnt use it without some kind of tray icon notification handy

15:30 hamza: mebaran151: gnus works fine with imap but it is dog slow.

15:30 mebaran151: I actually use my phone for that

15:30 technomancy: mebaran151: no, since Emacs is single-threaded all imap calls are blocking

15:30 so you have to use offlineimap or fetchmail or something like that.

15:30 LauJensen: technomancy: It has an async option

15:31 bmh: oh, technomancy, perhaps I should hassle you. When I run `lein native-deps` I get an IllegalAccessError: "default-repos does not exist (native_deps.clj:1)"

15:38 ninjudd: bmh: i think dnolen_ wrote the lein native-deps plugin

15:43 Raynes: http://gist.github.com/546158 Why is the output different in these situations?

15:44 I need the former output in the latter example.

15:45 chouser: Raynes: exceptions get wrapped as they bubble up through exception handlers

15:46 ninjudd: *e is a clojure.lang.Compiler$CompilerException in that case

15:46 chouser: your try/catch is catching it at a lower level before the REPL's eval wraps it

15:46 Raynes: It appears that print-throwable seems to print it properly.

15:47 _fogus_: Our own Brian Carper gets #4 geekiest HN post! http://www.swimwithoutgettingwet.com/blog/most_technical_hn/

15:49 Raynes: Geeky posts on HN? No way.

15:53 dnolen_: bmh: feel free to open an issue.

16:14 LauJensen: I have this annoying little NS issue. when I compile my namespace I get a few of these WARNING: something already refers to something else. Which is fine except for 2 things: 1) It doesnt say which import is triggering the warning, so debugging is tedious, and 2) I cannot reeval that ns declaration without killing the repl first. whats up ?

16:17 mebaran151: I sort of wish emacs package management were more like Lein and Maven, where I could just supply and edit a text file of the elisps I want to try, call build, and have them all magically available. Then if I didn't want them, I could just remove them and rerun the build function or whatnot.

16:26 chillitom: anyone having success with leiningen and cygwin? (work laptop.. i know i know)

16:27 LauJensen: haha

16:28 chillitom: You've been grilled a little in the past I see :) I've only heard of people have problems with lein on Windows, but that may be because people arent good enough at coming in here and sharing their success stories :)

16:28 chillitom: LauJensen: i sense a little Nelson in your 'haha'

16:28 well i'll give it a try

16:28 failing that i guess virtualbox will be the next step

16:28 LauJensen: No no, I just thought your comment was funny :)

16:29 chillitom: Do you know that you can boot a physical partition with virtual box?

16:29 chillitom: LauJensen: i did not, interesting indeed

16:29 LauJensen: Yea - I can boot Windows 7 from /dev/sda2 right within Arch

16:29 chillitom: (but for another occasion, can't touch this MBR)

16:29 LauJensen: A real timesaver in case you need to debug

16:30 ninjudd: chillitom: you could also give cake a try. i know several people who use it successfully on windows

16:30 LauJensen: Yea that would be my first stop

16:30 chillitom: ninjudd: thanks for the suggestion, i'll take a look at that

16:33 technomancy: mebaran151: you can basically do that with package..el

16:33 just not many lib authors make their stuff available through that

16:34 you can pester them about it though. =)

16:39 * chillitom downloads VirtualBox

16:40 chillitom: ninjudd: Cake looks nice, does it have a package repo side to it like lein/Maven?

16:42 ninjudd: yeah, it uses maven like lein

16:50 raek: hrm, why is my slime evaluation broken? it always eval everything in the repl's current namespace

16:50 instead of the namespace of the file

16:51 previously, everything was fine if I just evaled the (ns) form first

17:02 Scriptor: hi, I've got a question about clojure's implementation of multimethods, is the specific function that's needed determined at run-time?

17:03 or does it try to figure out as much as possible at compile-time?

17:03 arohner: Scriptor: the dispatch function is called each time the multimethod is called. the matching multimethod is picked based on the return value of the dispatch function

17:03 so it all happens at run time

17:03 Scriptor: arohner: ah, that makes sense, thanks

17:05 chouser: but some of that is cached the first time

17:09 duncanm: is it okay if i just place some jars in the lib directory of a lein project without listing them in project.clj?

17:09 arohner: duncanm: as a quick hack, yes

17:09 duncanm: arohner: cool, that's all i'm looking for

17:10 fliebel: duncanm: How do you run Clojure with all those on the classpath? or do you just compile a super jar?

17:12 I remember having a special clj set up somewhere to use that dir, but maybe I missed some magic that way.

17:19 duncanm: fliebel: i dunno

17:19 i have my own maven server, but i was hoping not have to write a pom for each of these jars

17:25 Bjering: Alright, dont know if anyone is awake from when I did my experiments earlier. But if you are, I have now rewritten server in Java and the performance is exactly the same as for the Clojure one.

17:31 LauJensen: Bjering: Thats nice :)

17:31 Bjering: Well, Nice for Clojure, bad for jvm.

17:32 LauJensen: Exactly

17:32 fliebel: Bjering: Didn't catch the whole story, but parts of it.

17:32 LauJensen: Bjering: In most of the benchmarks Ive seen, the JVM can keep up with C++ pretty well, not worse than 2x slower in the worst cases, and faster in the best

17:33 Bjering: jvm/netty/windows 7, perhaps more accuratly. Now I should post to the netty mail list and hear if this is as good as it gets or if I am doing something very wrong.

17:35 That, and/or make so I can run Linux on this machine, to see if that is it.

17:35 blackrain: Hi guys, which book should I choose to learn clojure? Which is the most up to date?

17:37 fliebel: blackrain: The one that;s not yet released… forgot the name… joy of clojrue, or something like that? ask… *digs memory* fogus and chouser?

17:37 bmh: blackrain: Are you sure you need a book? The online resources are wonderful. (That said, I did almost buy Practical Clojure yesterday)

17:37 LauJensen: fliebel: http://joyofclojure.com/buy I think

17:38 blackrain: fliebel: yeah, I've seen that

17:39 bmh: I believe I do not, but in book everything is in one place, no need to browse net

17:39 bmh: blackrain: totally reasonable.

17:51 dnolen_: Bjering: Interesting. That is good for Clojure :) On a side note, I've been using AWS EC2 to run quick Clojure perf tests on fast servers. They're easy to setup and tear down quickly. It does take some time to get a proper image setup, but once you have that it's pennies to keep an image around ready to boot.

17:53 slyrus: good for clojure not running on the JVM anyway...

17:54 Bjering: How fast is such a system? I mean, my workstation is a Dell T7500, that is a 3.2ghz quad-core Xeon W5580. Does it blow that out of the water?

17:55 dsantiago: Bjering, sorry I missed the start of the conversation; what did you write?

17:56 Bjering: dsantiago, a very basic chat server, to test/learn clojure and netty.

17:56 dnolen_: Bjering: I've run tests on 8-core Xeon servers with 23gbs of RAM. Mostly to compare the performance of Aleph with various DBs as well as benching comparisons w/ Node.js

17:56 dsantiago: Oh, cool.

17:57 How does that relate to clojure not on the JVM?

17:57 slyrus: dsantiago: this sounds like a JVM-specific problem

17:58 dnolen_: slyrus: possibly only on Windows

17:58 slyrus: ah, ok, nvm

17:58 yes, that is good then :)

17:58 * slyrus imagines that there are likely to more non-windows specific JVM problems than windows-specific, unfortunately

18:00 Bjering: dnolen: Now the "interesting" part for me is what I see in my taskmanager right now, the C++ server, now running only one single thread, actually for some reason takes alittle more time with 1 thread than with 4, proably causes some queuing at times and that costs. It now takes about 5% CPU on average with 50k clients. If I cannot use Clojure with its nice concurrency mechanisms best contender is actually single threaded C++, then the locking issues I fea

18:00 r goes away...

18:02 dnolen_: Bjering: that might be the case up until the point your start having significant computations on the server right?

18:02 Bjering: Yes, and I will keep pretty much all to O(1) operations to stuff in RAM so server load should be light.

18:03 light per connection atleast.

18:06 Well, I'll test on Linux, I should get this machine dual-boot anyway and then we will see. And perhaps ask around some more. And then finally before I give up I will try a silly solution with a connection-multiplexer/demultiplexer-only C++ server infront of a clojure server and have them speak only over small number of sockets.

18:07 now its time to go home.

18:19 AWizzArd: dnolen_: how did that 8 core perform with Aleph?

18:22 msf: hello everyone

18:23 just started playing with clojure 1.2.0, when I load clojure.contrib.duck-streams into my namespace I get warnings printed to the console, is there anything I can do to suppress this ?

18:25 tomoj: (:use [clojure.contrib.duck-streams :exclude (spit)]) :/

18:27 raek: much of duck-streams has been moved to clojure.java.io

18:30 slyrus: it would be nice if the warning would tell you who the offending ns was

18:30 dnolen_: AWizaArd: on par with Node.js for the hello world benchmarks, faster of course for talking to a DB, and way better if clients are accessing shared memory

18:33 bmh: dnolen_: Can you link me to the tracker for lein native dependencies? I want to submit the `native deps explodes` issue. I don't know if it's a penumbra issue or a native deps issue.

18:36 dnolen_: bmh: http://github.com/swannodette/native-deps/issues

18:36 duncanm: boo, i have this jar that i want to use, but some of the classes don't live in a package

18:36 i dunno how to import them

18:37 wooby: technomancy: i noticed lein uberjar succeeds even if compilation fails, is that desired for some reason + would a patch be welcome?

18:38 mebaran151: I'm thinking of wrapping MessagePack for Clojure RPC: are there any better clojure RPC centric solutiosn out there these days?

18:40 technomancy: wooby: that's definitely a bug; patch would be great.

18:40 gfrlog: does lazy seq => stackoverflowexception blatantly suggest a particular kind of error?

18:41 wooby: technomancy: cool i'm on the case, thanks

18:44 ihodes: question: i forked a git repo (lein's) and use it to patch stuff etc. how can i pull upstream changes? should i add a remote of the main branch and fetch from there? or is there a better/different way to do that?

18:46 technomancy: ihodes: yes, you want to add a remote, but if you've committed to your master branch it gets a bit confusing since pulls result in unnecessary merges. that's why I recommend using a topic branch for contributions

18:47 ihodes: ah alright

18:47 technomancy: ihodes: details here: http://technomancy.us/135

18:47 ihodes: if i fetch and merge it should be ok for now, though… i'll deal

18:47 ah thanks

18:47 i'm going to look at issue #98

18:48 technomancy: ihodes: actually hold off on that one; I have half a fix ready

18:48 thanks though. =)

18:48 ihodes: haha alright

18:49 you work too fast :P i definitely want a sticker, but i've got to do at least a few more fixed/features before i ask :P

18:49 technomancy: not at all; every little bit helps and is sticker-worthy. =)

18:53 ihodes: I need a *bit* more than, what, 50 characters? i refuse a sticker, for now ;) any particular issues/low-hanging fruit i might be able to hack on?

18:57 mebaran151: gfrlog: could the function that produces the lazy-seq have an infinite loop?

18:57 it sounds like one of the steps (maybe the first one?) is resulting in a computation that will never end

18:57 gfrlog: no, I don't think so. It's produced with iterate, using an iterate function that's pretty simple I think

18:58 technomancy: ihodes: maybe a :min-leiningen-version in project.clj where it would warn if your current version was too old?

18:58 either that or issue 92

18:59 gfrlog: mebaran151: regardless, the stack trace clearly involves unwinding a lazy seq

19:00 ,(first [])

19:00 clojurebot: nil

19:02 ihodes: i'll check both out!

19:07 chillitom: leiningen isn't putting itself on my path, is that normal?

19:07 if i install the script myself where should I put it and with what perms?

19:11 ihodes: chillitom: are you on a *nix?

19:12 if so, 755 /usr/bin/lein is mine. i'd use the self-install if possible.

19:13 chillitom: ihodes: yeah linux, i made a ~/bin and added it to path for now

19:13 does the clojure repl make use of jline if it's available?

19:14 ihodes: the one you get from runing java -cp pathwhatever clojure.main?

19:15 gfrlog: ihodes: I don't know what chillitom meant, but I'm curious about the answer to that version of the question

19:16 ihodes: gfrlog: chilliton: if you want to use jline, it needs to be on your classpath, and you need to run it like this java -cp jlinpath:cljpath jline.ConsoleRunner clojure.main

19:16 clojure's getting started page covers this i think, if you're having trouble setting it up

19:16 http://clojure.org/getting_started

19:16 chillitom: ihodes: cool

19:16 ihodes: but i'd recommend using lein repl or cljr repl

19:17 so you don't have to worry about your classpath

19:17 chillitom: so can i just add it to a lein project to get it on the lein repl then?

19:17 gfrlog: I think lein repl includes it automatically

19:17 duncanm: is reify the new proxy? shouldn't i use reify instead proxy if i'm on 1.2?

19:18 chillitom: gfrlog: will investigate, i'm getting escape codes when pressing arrow keys

19:18 duncanm: (reify java.io.FilenameFilter (accept [^java.io.File dir ^String name] (-> name (.startsWith "foo"))))

19:18 gfrlog: maybe the version of lein is old?

19:18 ihodes: chilliton: gfrlog is right. well, it used rlwrap first, then jline if rlwrap isn't around

19:18 uses*

19:20 chillitom: thanks guys, i'll investigate further tomorrow after another delightful day in the procedural day job

19:21 ihodes: chillitom: enjoy :)

19:21 duncanm: oh, the reify docs are out-of-date - it doesn't say that you need to explicit list the 'this' argument to a method

19:23 ihodes: duncanm: I think it is there: "Note that the first parameter must be supplied to

19:23 correspond to the target object ('this' in Java parlance)."

19:23 duncanm: ihodes: ah, but it's not in the example (the toString)

19:24 ihodes: it is in the one i'm looking at! haha here's what i see: http://cl.ly/25pG

19:25 duncanm: oh, i'm looking at http://richhickey.github.com/clojure/clojure.core-api.html#clojure.core/reify

19:25 maybe that's not the right site?

19:25 ihodes: ah no, that's old

19:26 http://clojure.github.com/clojure/clojure.core

19:26 clojure is all under the clojure organization now

19:31 duncanm: also, while i'm not sure reify is totally a replacement for proxy, it looks like it is. there's some different in syntax, reify seems to be able to do all that proxy can and more.

19:31 i'm curious why it's still there (proxy)

19:31 i guess it's nice it is, otherwise i'd have to update some of my code… but no big deal…

19:32 hiredman: you cannnot reify classes

19:33 tomoj: isn't it just the opposite? proxy can do all that reify can and more? but reify is faster?

19:34 hiredman: you cannnot reify classes

19:34 tomoj: ...thus proxy can do all that reify can and more?

19:35 ihodes: tomoj: ah yeah, that might be it. though can you proxy protocols? i think you can only reify those.

19:51 duncanm: hmm, doseq isn't quite like for-each in Scheme

19:53 ihodes: why not use map?

19:56 duncanm: ihodes: i could, but i'm only using it for side-effect

19:56 i like the symmetry between MAP and FOR-EACH in Scheme

19:58 ihodes: (doseq (map (fn sideeffects-and-wahetever coll)) should do you then :)

19:58 dorun*

20:00 gfrlog: (println ob) => StackOverflowError

20:00 (println ob) ; again, immediately after. Works fine

20:01 pdk: needs related code for context gfr

20:02 gfrlog: yeah

20:02 it's probably not worth explaining till I've done more testing

20:02 for now I'm just crying and cursing lazy seqs

20:11 duncanm: hmm, how do i load clojure.java.io in slime/swank?

20:11 (require 'clojure.java.io) doesn't seem to work

20:17 weird

20:17 gfrlog: okay

20:17 I must be crazy now.

20:17 http://gist.github.com/546627

20:18 with that in my repl, I call (def ob (process-from-ob)), and when it returns, I call (println ob) twice. The first is stackoverflow, the second suceeds

20:29 duncanm: hmm, do i get destructuring in doseq?

20:31 gfrlog: ,(doseq [[a b] [[1 2] [3 4]]] (println b))

20:31 clojurebot: 2 4

20:31 gfrlog: looks like it

21:03 duncanm: somehow i can't require clojure.java.io in my project

21:03 jstirrell`: So I'm attempting to learn about neural networks without much success. By any chance can anyone explain to me what's going on in the "apply-activation-function" function on this page:

21:03 http://github.com/fffej/ClojureProjects/blob/master/neural-networks/src/uk/co/fatvat/mlperceptron.clj

21:03 duncanm: if i write 'clojure.java.io/file', it works, but 'file' alone doesn't

21:04 gfrlog: how could I concat eagerly?

21:05 wwmorgan: (doc doall) ; <- gfrlog

21:05 clojurebot: "([coll] [n coll]); When lazy sequences are produced via functions that have side effects, any effects other than those needed to produce the first element in the seq do not occur until the seq is consumed. doall can be used to force any effects. Walks through the successive nexts of the seq, retains the head and returns it, thus causing the entire seq to reside in memory at one time."

21:06 gfrlog: does that have an affect on the original object? like is it easier to compute the second time?

21:07 msf: what is an easy way to coerce a vector into a set ?

21:07 gfrlog: I'm feeling a need to use this and I don't even have side affects

21:07 slyrus: hmm... so much rope. this is perhaps the best I've come up with so far... a separate protocol ns used by the implementation ns'es, with each method defaliased in the core file s.t. folks don't need to (and shouldn't) pull in protocol, but only core and any other ns'es as needed.

21:07 gfrlog: ,(into #{} [:this :is :my :vector :my :my]) ; <- msf

21:07 clojurebot: #{:my :is :vector :this}

21:07 slyrus: still, would be nice to get rid of the defaliases

21:07 msf: thanks

21:08 duncanm: ihodes: ping?

21:08 wwmorgan: ,(set [1 2 3])

21:08 clojurebot: #{1 2 3}

21:08 gfrlog: aw, pwned

21:10 ihodes: duncanm: i'm here :0

21:10 duncanm: ihodes: have you used clojure.java.io?

21:10 ihodes: a bit yes

21:10 gfrlog: ,(loop [n 100000 v []] (if (zero? n) v (recur (dec n) (concat v []))))

21:10 clojurebot: Eval-in-box threw an exception:java.lang.StackOverflowError

21:10 gfrlog: does anybody know what's happening there?

21:11 duncanm: ihodes: i can't seem to 'require' it correctly

21:11 ihodes: ,(require 'clojure.java.io)

21:11 clojurebot: nil

21:11 ihodes: seems to work

21:12 ,(require '[clojure.java.io :as jio])

21:12 clojurebot: nil

21:12 ihodes: ,jio.resource

21:12 clojurebot: java.lang.ClassNotFoundException: jio.resource

21:12 ihodes: ,jio/resource

21:12 clojurebot: #<io$resource clojure.java.io$resource@857163>

21:12 ihodes: just do that and you should be fine

21:13 wwmorgan: gfrlog: my guess is that since concat is lazy, it stacks up all the calls to find the first element. You could solve this by calling seq on v inside the loop

21:13 gfrlog: I figured it was something like that, but it seems like a bug...shouldn't you be able to use lazy seqs without worrying about the call stack?

21:14 ihodes: gfrlog: something like what your doing should probably be a lazy-seq

21:14 gfrlog: ,(loop [n 100000 v []] (if (zero? n) v (recur (dec n) (concat (seq v) []))))

21:14 clojurebot: ()

21:15 gfrlog: I guess I don't understand the difference between a lazy seq and the result of concat

21:15 the docs say concat "Returns a lazy seq"

21:16 ihodes: concat does, but the seq you're building is a seq (non-lazy) OF lazy seqs

21:16 wwmorgan: gfrlog: this may shed some light on the subject: (if (concat []) 1 2) returns 1, but (if (seq (concat [])) 1 2) returns 2

21:17 gfrlog: ,(concat [])

21:17 clojurebot: ()

21:17 gfrlog: ,(seq (concat []))

21:17 clojurebot: nil

21:17 gfrlog: wwmorgan: I think that just made it weirder

21:17 ,(seq '())

21:17 clojurebot: nil

21:18 gfrlog: ,(seq [])

21:18 clojurebot: nil

21:18 gfrlog: now I don't know what seq does

21:18 (doc seq)

21:18 clojurebot: "([coll]); Returns a seq on the collection. If the collection is empty, returns nil. (seq nil) returns nil. seq also works on Strings, native Java arrays (of reference types) and any objects that implement Iterable."

21:18 gfrlog: huh.

21:18 ihodes: gfrlog look at this - but try it in your own REPl please ;)

21:18 gfrlog: man I been using clojure for over a year now and apparently I don't understand seqs

21:18 ihodes: http://gist.github.com/546704

21:19 wwmorgan: seq doesn't force lazy collections, but it will force the first element, because it needs to know whether there is "something" (a sequence) or "nothing" (nil)

21:19 gfrlog: ihodes: did you intend the recur error?

21:20 ihodes: haha no i didn't - i just whipped it together in textedit. one second

21:21 ah dumb me, can't use recur there. not tail position. is that the error? instead, use a function

21:21 and call the function instead of recur

21:21 gfrlog: yeah, it was bad recur position

21:22 so we just wanna cons a bunch?

21:22 ihodes: http://gist.github.com/546704

21:22 gfrlog: gee wiz, the (seq) thing totally fixed the error I'd been staring at all day

21:23 ihodes: gfrlog: maybe look at the top two responses here: http://stackoverflow.com/questions/3247045/how-are-lazy-sequences-implemented-in-clojure/3247417#3247417

21:24 gfrlog: ihodes: thanks, I will

21:25 or am rather

21:27 duncanm: ihodes: i can only get it to work if i use refer instead of require

21:29 ihodes: can you send me a gist or something of the errors you're getting/what you're doing?

21:38 duncanm: ihodes: http://gist.github.com/546728

21:44 ihodes: duncanm: right, because you're not qualifying file. try doing (:require [clojure.java.io :as jio]) instead, and then use jio/file in lieu of file

22:02 bmh: clojurebot: ping

22:02 clojurebot: PONG!

22:18 msf: is there an alternative for read-lines from duck-streams in core ?

22:18 defn: you could slurp and then split-with \n

22:19 (i think)

22:19 markgunnels: We are playing with this concept at work called Dynamic Object Model where you basically read in an Ontology and dynamically generate your object model. It's easy to do in Ruby and I'm trying to pursue it in Clojure with defrecord using a macro.

22:19 defn: markgunnels: cool

22:19 markgunnels: When I run my macro -

22:19 (defmacro make-record

22:19 [name attr1 attr2]

22:19 '(defrecord ~name [~attr1 ~attr2]))

22:20 (This is just a play macro but you get the idea)

22:20 I run into an error.

22:20 Duplicate key: null

22:20 [Thrown class java.lang.IllegalArgumentException]

22:21 Am I doing something spectacularly stupid or are records not meant to be used in this fashion?

22:22 (dom.core/make-record "FullName" "first-name" "last-name") is how I am calling the above prototype macro.

22:24 phaer: lisp

22:24 ah, wrong window

22:24 tomoj: markgunnels: try (macroexpand-1 '(make-record ...

22:24 defn: markgunnels: i couldnt answer you with any degree of certainty, im trying to find out why that's the case

22:25 msf: defn: is slurp lazy ?

22:25 I'm working with a relatively large file

22:25 defn: msf: i believe so yes

22:25 tomoj: markgunnels: using ` instead of ' in your macro will help a bit, but you've still got another problem

22:25 ihodes: msf: it is not lazy. use line-seq instead

22:26 defn: ihodes: thanks for that

22:26 sorry msf

22:26 markgunnels: tomoj: Thanks. I switched it to a `.

22:26 It gives a new error which I think you were alluding too.

22:26 ihodes: defn: no problem, was recently writing stuff around line-seq and stuff

22:26 tomoj: ok, now if you macroexpand-1 you should see the remaining problem clearly

22:26 defn: is the other problem the vector?

22:27 tomoj: though.. your macro with ` could work just fine, you're just calling it wrong :)

22:27 defn: tomoj: would you mind pasting macroexpand-1 of his macro

22:27 markgunnels: tomoj: It works if I don't pass in strings.

22:27 :-p

22:27 tomoj: yep

22:28 markgunnels: tomoj: Thanks.

22:28 tomoj: defn: with ` you get (clojure.core/defrecord "FullName" ["first-name" "last-name"])

22:28 markgunnels: which dev env do you use for clojure?

22:28 markgunnels: tomoj: Emacs, swank, and leiningen.

22:29 tomoj: I feel pretty stupid for not spotting the backtick.

22:29 tomoj: What dev env do you use?

22:29 tomoj: cool, then try putting point right on the first paren in (make-record FullName first-name last-name) and hit C-c RET or C-c C-m

22:29 defn: the same :)

22:29 tomoj: same

22:30 defn: emacs brotherhood

22:30 markgunnels: Heh.

22:30 tomoj: easier than using macroexpand-1, and it tidies things up too

22:31 defn: tomoj: you're my new hero

22:31 great trick

22:31 thanks

22:31 markgunnels: tomoj: mine too.

22:31 I'll dedicate tomorrow's RDF and Clojure work to you.

22:31 :-)

22:31 defn: markgunnels: another fun one with SLIME is C-.

22:32 errr no

22:32 M-.

22:32 if | if your point, (defn|

22:32 is your point*

22:32 M-, takes you back to your file

22:32 It's a handy way to view core source

22:33 markgunnels: Very nice.

22:33 Wow. That is VERY NICE.

22:33 defn: :)

22:34 markgunnels: best part? It even works with third party libs

22:34 poorly documented libs become much more manageable with that little trick

22:34 tomoj: and it's a stack, so you can dive n levels deep with M-. and then find your way back to whatever you were working on with M-,

22:35 defn: I didn't even consider that! Even cooler!

22:36 markgunnels: defn: I'm bouncing through clojure.contrib now. :-)

22:36 tomoj: someone around here figured out how to get it to jump into clojure's java source too

22:36 defn: tomoj: that I'd be interested to know

22:36 For exceptions there's nothing better

22:37 tomoj: I think it was lau

22:37 (well.. lau demoed it in that video, someone else did the figuring out I believe)

22:41 defn: C-c I

22:41 brings up the inspector

22:42 tomoj: after all this time I still use repl-utils/show for some reason..

22:42 defn: noob...

22:42 kidding of course heh

22:43 i wonder if fuzzy completions work better now

22:43 tomoj: and I still switch to the repl for (doc ..) instead of just C-c C-d C-d'ing

22:43 defn: i disabled it because it only seemed to work for core

22:44 yeah i use (doc ...) a lot too

22:44 when you're writing code just doing (foo ... starts to become a habit

22:44 i tried to switch from clojure to Ruby the other day for awhile, and it was like.. (def foo^H^H^H^H^H^H^H^Hdef foo

22:45 tomoj: :)

22:45 defn: Actually I even did it while writing some regular plain text.. "(hey how's it going ..."

22:45 heh

22:46 tomoj: wait.. what??!!

22:46 looks like slime-who-calls works now!

22:47 defn: what's that

22:47 tomoj: C-c C-w C-c map RET

22:47 then scroll through the list and it shows you each call site... beautiful

22:47 defn: i get evaluation aborted

22:47 maybe im doing it wrong

22:48 hrm

22:48 tomoj: maybe it's only in swank-clojure 1.3.0-SNAPSHOT

22:48 defn: tomoj: are you connected to a swank session, or are you running slime

22:48 ah yes, i bet that's why

22:49 i was just M-x sliming

22:49 yep it works

22:49 that's awesome

22:51 tomoj: looks like it was tcrayford's doing

22:54 msf: hrm

22:54 why would spit be throwing NPE's all of a sudden

23:08 arohner: when I try C-c C-w C-c map RET, I get Invalid token: ::xml/dtd [Thrown class java.lang.Exception]

23:10 looks like it also doesn't work on names with dashes in them

23:13 ihodes: technomancy: you should have a pull request in your inbox re: :min-lein-version. I'll check out more issues/features tomorrow!

23:43 technomancy: tomoj: yeah, tcrayford is th eman

23:48 tomoj: arohner: hmm, works fine with names with dashes for me

23:49 swank-clojure 1.3.0-SNAPSHOT?

23:49 (or at least, it works fine as far as I can tell...)

23:49 arohner: tomoj: yeah. 1.3.0-snapshot.

23:50 tomoj: huh, this time it worked. Last time I tried, it didn't

23:50 tomoj: uh oh

23:52 redalastor: I'm learning D and it's pretty neat. It got immutability, pure functions, lazyness and lambdas. I wonder why I never heard people wishing for D as a target of Clojure-in-Clojure.

23:53 rickmode: I would think JVM alternatives for Clojure-in-Clojure would be other VM's as opposed to other languages

23:54 for example CLR (getting Clojure to work well on .Net again)

23:54 technomancy: redalastor: D isn't really a runtime, is it?

23:54 redalastor: Nope

23:56 They are thinking about AST macros. It's not likely going to be anywhere near lisp macros but it definitely look like one of the most interesting C descendent language that came out since a long while.

23:56 technomancy: sure, but that's not saying much. =)

23:59 redalastor: I believe the contrary. Java was meant to bring people "half-way to lisp" but the language itself wasn't a big success in that regard.

23:59 slyrus: tomoj: that is nice. thanks for pointing that out!

Logging service provided by n01se.net