9:52 Chouser: rhickey: if you don't mind sharing, is that corporate or academic work?
9:53 rhickey: I work for myself
10:05 Chouser: you've found customers for your work, or you have sufficient capital you don't need customers yet?
10:06 I don't mean to pry, I'm just very curious.
10:06 rhickey: that's something I can't discuss
10:06 Chouser: ok
10:07 well, I'm sure glad you've taken the time to work on Clojure. :-)
10:07 cgrand: Maybe it's old news to you but I just noticed this morning that the Eclipse debugger is able to step into clojure code (highlighting current line etc.).
10:08 rhickey: me too - I finally get to develop software the way that I want to, and have huge libraries at hand
10:08 cgrand: can you set breakpoints? last time I tried I couldn't
10:09 Clojure emits the right stuff for debugging, local variable names etc
10:10 Chouser: rhickey: that's how the stack traces from exceptions work, right?
10:11 rhickey: that's how the line numbers get in there, but there's more to it, you need a SourceDebugExtension to map to the source files, which Clojure also emits, and locals entries in the bytecode, also there
10:12 cgrand: Yeah I saw that you emit smap strings. With a rudimentary plugin (never played with eclipse dbugger api) I can set breakpoints
10:12 rhickey: cool
10:13 cgrand: which source files? Clojure emits java source files?
10:14 rhickey: no the smap is a mapping from Clojure to non-existent Java files, which is the bridge
10:14 Full Clojure plugin for Netbeans is coming, if you haven't seen this: http://
10:14 cgrand: I saw
10:15 rhickey: they had to do some work to get Netbeans to debug Clojure
10:17 I'd love to see Eclipse support too
10:19 cgrand: The editor framework is a bit complex...
10:19 rhickey: I guess Cusp would provide good editing support, but native debugging is preferable to custom protocol
10:20 does Cusp do debugging anyway?
10:22 cgrand: Not sur but I think so
12:02 Chouser: someone recommended it to me when I was looking for a JVM lisp. Of course I ended up using Clojure instead, so I know nothing useful.
12:04 sanity: Chouser: i only ask because a friend of mine created it
12:05 Chouser: although i think, unlikely Clojure, it is interpreted, rather than compiled
12:05 Chouser: interesting.
14:27 wabash: If I have a multicore cpu, like 8 cores, how would I parallelize a clojure app? Would I start a separate JVM on each core? And, would threads on separate JVMs be able to communicate via some sore of message passing?
14:27 like Erlang?
14:28 rhickey: A single JVM can address multiple cores
14:28 Clojure agents will get distributed among them
14:28 wabash: rhickey: wow, sweet!
14:29 I'm sorry to ask such basic questions. I just found out about clojure like, last week. The more I find out, the more interested I am.
14:29 rhickey: Clojure fns are Callable and Runnable, and will work with the java.util.concurrent.Executors framework as well
14:30 wabash: What is the clojure way of thread communication? And, are threads expensive like Java or are they cheap like Erlang? is msg passing expensive like sockets or cheap like shared mem?
14:30 rhickey: Are all Java libs and classes basically available?
14:30 rhickey: Erlang processes are not threads
14:31 they may or may not map to threads depending on how you have Erlang SMP set up
14:31 wabash: rhickey: Sorry, I got semantically lazy.
14:31 rhickey: Erlang w/o SMP mode runs on one core
14:31 but does its own scheduling
14:32 wabash: Right, I just misused a term.
14:32 rhickey: Clojure agents use a fixed thread pool on message sends
14:32 the pool size is related to the number of cores
14:32 wabash: linearly related?
14:32 rhickey: Agent state can be read directly w/o messages
14:33 agent state gets changes via actions, functions of their state that get executed asynchronously
14:33 it's a different model than Erlang's
14:33 wabash: How do you read stat directly? Is it some sort of polling? is an agent like a process?
14:33 Oh, I see.
14:33 It's a very different way of thinking about hings.
14:33 is the state reading quick between cores?
14:34 rhickey: Erlang's can be transparently distributed, but getting at state requires send/ack & blocking
14:34 wabash: Can clojure be transparently distributed?
14:34 rhickey: Clojure is a single memory space, reading agent state is like any other object access
14:35 No, I don't believe in transparent distribution, but may add distributed actors at some point
14:35 wabash: Is the single mem space a problem for concurrency then? How is that abstracted? Erlang seems so elegant......
14:36 rhickey: Clojure agent states are immutable
14:36 wabash: I get it now.
14:36 Thank you.
14:36 rhickey: immutable objects are no problem for multicore
14:37 wabash: rhickey: I appreciate your time. Learning something new, I'm always faced with the paradox of what to ask when I don't know enough to formulate questions.
14:37 rhickey: I don't think send/ack message pairs to read state is elegant at all, just a necessary evil of distribution
14:37 vs RPC
14:38 wabash: RPC is more elegant than message pairs?
14:38 rhickey: no, message pairs is less evil
14:38 distribution is simply very hard
14:39 why incur the complexity unless you really are distributed?
14:39 wabash: premature scalability, I suppose.
14:40 rhickey: but the constant factors are different - you can make it look the same, but the same payloads and chatter rate that worked locally can totally fall down when distributed
14:40 wabash: Yes, I understand and agree.
14:41 That's kind of what I was getting at with spreading over cores; will the constants be high for communicating between (threads/processes) on different cores.
14:41 wabash: In Clojure, then, you think in terms of "agents"?
14:41 thank you for the link. I will read it after lunch.
14:42 rhickey: agents are one of several tools for MT programming. There are also refs and STM
14:43 wabash: I see. If I may ask an over-general question, if I run Clojure on an 8 core machine, will the communications overhead be small enough that the distribution will be even? Will I have to worry about which core is closer to which?
14:43 Or if I have a small cluster of separate machines, will I have to adjust my algorithms to match the architechture?
14:44 rhickey: that totally depends on the granularity of your work units. Overhead/work is what matters
14:44 wabash: I see.
14:44 rhickey: separate machines is a totally different architecture in my book, and where Erlang's model makes the most sense
14:44 wabash: Right.
14:45 wabash: You seem to have a lot of experience with different languages/models. Is your work research oriented? Or are you lucky enough to have a corporate job doing all these things?
14:46 rhickey: I'm just old
14:46 But lots of MT in my experience
14:46 wabash: MT?
14:47 rhickey: multithreading
14:47 wabash: Got it.
14:47 Thanks for the discussion. It is useful, significant to me.
14:47 rhickey: sure
14:48 wabash: I went to an average school, got wrapped up in some programming jobs, got ejected, dejected, discouraged, but now have discovered higher level languages, and FP, and think that programming is fun again, a bit.
14:48 well, must have lunch. thanks for sharing your knowledge and experience.
16:14 jgracin: hi all! this place is getting more crowded with each day. :-)
16:14 rhickey: :)
16:15 * mbishop shoves jgracin out the door
17:24 Chouser: I'm not sure this is the right question, but if I wanted to make something that acts exactly like a hash-map, but also generated some Actor dispatches, how would I do that? "implements"? multi-methods?
17:26 ok, that's not quite right. I want something that acts like ref. You can call set-ref on it, and it does that but also it dispatches and Actor function.
17:26 If this were some other language, I'd sub-class Ref.
17:28 rhickey: why not just use a map as the state of an agent?
17:28 or of a ref?
17:29 agent actions anf ref alter/commute functions can be multimethods
17:35 Chouser: I'm right at the edge of my understand here, so bear with me... I've got a function that calls (ref-set db (assoc ...)). It currently expects db be to a ref to a map, but I'd like to also allow db to be one of these new things I'm making. Are you suggesting that I change my set-ref to be (alter db foo) and then make foo a multi-method that can handle either standard ref-maps, or my new thnigs?
17:36 rhickey: right, in your first case, (ref-set db (assoc... you could have used (alter db assoc...
17:36 now swap foo in for assoc...
17:36 make foo a multimethod that can distinguish the things you might out in db
17:37 are your new things going in refs?
17:37 Chouser: ok, I think I follow.
17:38 I don't *think* so. I think they will act like refs.
17:38 rhickey: hmm...
17:38 Chouser: Maybe I didn't understand the question.
17:38 rhickey: why can't they be refs?
17:39 Chouser: I'm thinking I'll make a thing that acts like a ref, but whenever it gets set it will ask an actor to write its new value out to disk.
17:40 rhickey: I see, I've thought about connecting on-commit and validate fns to refs as a general solution for that kind of thing
17:41 but refs are special, don't try to derive...
17:41 you can have your mutating fns send an agent action, but not automatically... could use a macro to define the mutating fns
17:43 Chouser: but the solution you outlined above should still work, right? I'll use (alter db db-assoc ...), and db-assoc will either just assoc, or assoc and dispatch.
17:44 This will be my first agent.
17:44 And my first multi-method. ;-)
17:44 rhickey: oh boy
17:45 yes, if db-assoc sends an action, that will only happen if the transaction commits
17:47 Chouser: whew, good point. I hadn't thought about the assoc getting rolled back. Ok, I'll try to do this.
17:48 rhickey: that's the beauty of the agent/transaction coordination, there's no other way to get run-once-only side-effects in a transaction
22:14 Chouser: Ok, I don't think I really need a multimethod.
22:14 Is there any way to refer to an agent action, or do I have to wrap the call to it in a function.
22:15 rhickey: ?
22:15 agent action are functions, or are you talking about the agent's state?
22:17 Chouser: hm, no I was misunderstanding how they work. Let me read that docs page again...
22:18 rhickey: you can make one-off agents just to execute side-effects:
22:19 (send (agent nil) (fn [_] (side-effect)))
22:20 Chouser: send is the same as ! ?
22:20 huh. ok, I think that's exactly what I want.
22:21 rhickey: yes, send is the new !, there is also send-off for potentially blocking actions
22:21 Chouser: oh, is ! going away?
22:21 rhickey: yes ! is going away
22:22 try it
22:22 it's gone
22:22 Chouser: heh, ok.
22:45 should I avoid multiple deref in a transaction?
22:46 I've got @foo all over the place, but I could put it in a local symbol instead.
23:16 rhickey: multiple deref of refs is no problem, should be same result throughout unless transaction itself changes ref