#clojure log - Aug 26 2008

The Joy of Clojure
Main Clojure site
Google Group
List of all logged dates

8:17 dabd: hi anyone knows if the video from the clojure and the semantic web meeting will be available?

9:42 rhickey_: dabd: I've been on vacation and haven't had a chance to look at the semantic web video yet

9:44 dabd: rhickey: ok thanks

14:36 mbeau: I'd like to simulate keyword parameters in a function; it seems that this is not directly supported in clojure so it seems that I should be able to easily do this myself

14:37 one idea I had was to take the 'rest' parameter list and turn that into a hashmap - that gives me pretty much what I'm looking for

14:37 how can I take a list and convert that into a map?

14:37 seems like it should be obvious, but I can't figure it out

14:38 erochester: try this: (apply hash-map [:a 1 :b 2])

14:39 mbeau: ok, thanks

14:40 erochester: no problem.

15:33 cemerick: ...it's too bad that stuff like map, reduce, etc., aren't functions on ASeq. That'd make Java a whole lot more sane.

15:35 rhickey: cemerick: if Java had closures

15:35 that implemented IFn :)

15:36 Var map = RT.var("clojure","map");

15:36 map.invoke(...);

15:36 Clojure is definitely callable from Java

15:37 cemerick: yeah, I know I could do that -- I don't think this particular chunk of Java will be sticking around for long though, so I don't want to prod at it *too* much.

15:37 I'd be happy enough to drop in anonymous AFn subclasses, really.

15:37 Don't take the suggestion seriously, though -- just a side-effect of having a sizable Java codebase that has clojure stuff starting to filter in.

15:44 lisppaste8: Chouser pasted "starved transaction?" at http://paste.lisp.org/display/65898

15:45 Chouser: when the transaction in slow completes, you see ":show-returned" and then the function dead completes.

15:45 rhickey: ok

15:46 Chouser: However, as long as the fast transactions keep committing, slow keeps resetting and never seems to finish.

15:46 rhickey: right

15:46 what is your expectation?

15:46 Chouser: I was expecting some Magic to how up and fix my problem. :-)

15:46 sorry, brb...

15:46 rhickey: I see

15:48 blackdog: has anyone been thinking of a ruby gem like package system?

15:51 cemerick: rhickey: why is Delay no longer a fn? (sorry if I missed some prior discussion)

15:52 rhickey: cemerick: there is now a proper force, which is a no op if the arg isn't a delay

15:53 Chouser: I'm back. I was under the impression that at some point new transactions would be held up to allow a very old transaction to complete.

15:55 cemerick: rhickey: yeah, I saw that, and that's fine, but there are definite advantages to being able to treat delays as fn's

15:56 rhickey: Chouser: no, only if there is a live conflict will an older transaction 'win', your slow doesn't acquire anything until after its delay - try putting (ensure c) before sleep

15:58 cemerick: I'm on the fence about how rich to make delays

15:58 when would you use them as fns?

15:59 cemerick: anytime I have a map containing fns for values; some functions may be very involved, so caching their results transparently is super-handy

16:00 (handy enough that I'd use my own fn-delay impl to get the same results, FWIW)

16:01 rhickey: cemerick: why wouldn't you just delay the invocation?

16:02 then force = call

16:03 delays as fns was always just sugar

16:03 cemerick: because I want to wrap up fns into a map and toss it over the wall to a caller -- I don't have control over when those functions are pulled out and invoked

16:04 rhickey: but the caller doesn't pass args

16:04 so they could call force instead of invoke

16:04 or I'm confused about the use case

16:05 cemerick: sure, but then they'd have to know about delays as well as fns -- I'd end up littering (if (delay? fn) (force fn) (fn)) all over the place.

16:05 rhickey: no - force does that for you

16:06 cemerick: so you're suggesting making all of the values in such maps delays?

16:06 rhickey: no, force on non-delay just works

16:06 user=> (force 42)

16:06 42

16:07 From Java, Delay.force(42)

16:07 cemerick: oh, I see; my initial reaction to that is "yuck"

16:09 I guess I don't see the advantage of having a split between fns and delays to begin with; using (force fn) all the time instead of simply (fn) isn't obviously better

16:09 rhickey: yes it is - because you can't say (42)

16:10 now you can have mixed delayed and non-delayed values in a mixed environment and handle uniformly

16:14 but (d) was just sugar for (get d), no difference in behavior

16:17 cemerick: the problem with that is that forcing a fn returns the fn, not the value of invoking the fn; a mixed environment of delays and fns doesn't yield the results I'd expect in that case (not that those results are incorrect, just not what I'd expect). I view delays as essentially transparent wrappers around fns, so treating them as values themselves trips me up.

16:17 I suppose my perspective could be fundamentally faulty *shrug*

16:19 rhickey: delays delay expressions, the fact that they are implemented using closures is an implementation detail - you could always have said (delay 42) - where's the fn in that? :)

16:20 cemerick: touch� :-) Of course, any (delay x), where x is already a strict value, doesn't really have any application (that I can think of).

16:22 rhickey: even when not a constant, it's still an expression that's delayed, not a fn: (delay (+ 21 21))

16:23 cemerick: sure

16:23 rhickey: but without force that detects delays, people did/do have to say (delay 42) sometimes when creating a data structure with mixed delayed/non-delayed values. Now they don't have to

16:25 cemerick: I'd *much* rather put the onus on the creator of the data structure, and not the caller.

16:26 rhickey: there is always some onus on the caller, since there isn't a value without force/get/call

16:27 cemerick: surely fn invocation is more fundamental than force/get, certainly in the overwhelmingly common cases?

16:27 rhickey: pretending all the values are fns doesn't change that

16:28 and it precludes uniform handling

16:28 that's the key point

16:29 * cemerick curses his irc client

16:31 cemerick: I guess we have different common use-cases in mind. Doing (force x) and then conditionally invoking the result seems pretty backwards to me, but then I can't think of the last time I put delays and numbers (for example) into the same map or list.

16:32 My final point would be: why not permit the old behaviour? Delays-as-fns is sugar, for sure, but makes things a lot simpler (in my use cases, anyway).

16:32 rhickey: > Doing (force x) and then conditionally invoking the result - I don't see when that happens ever. force is the exact same operation as call was, except it works on non-delays too

16:34 invoke called get(), force calls get()

16:35 iff it's a delay

16:35 cemerick: the problem is this; consider a vector: (def v [(fn [] 5) (delay 5)])

16:36 (map force v) => (user.fn__2127@1c2812 5)

16:37 if delays are fn's, you can do (map #(%) v) => (5 5)

16:38 that's what I mean by conditionally invoking the result -- that fn is its own value which then needs to be invoked after being forced in order to get to the same place

16:40 rhickey: your example has no purpose to using delay, so it's hard to see the point

16:41 if the second 5 is some expensive expression, then (def v [5 (delay 5)])

16:42 cemerick: yes, it's watered down to make an example; consider {:root root-node :size (delay (fn [] ....))}, a tree structure with a :size attribute that will walk the tree and return and cache the size of the tree rooted at root-node. You want the value of :size to be a fn, so it can be optionally delayed depending upon the specific costs associated with the size calculation.

16:43 rhickey: then it's not (delay (fn [] ...) but (delay ...)

16:44 cemerick: right, sorry -- the point's the same, though

16:44 rhickey: no it's not, once you have an expression, then fns have nothing to do with it

16:46 all delays-as-fns do is superficial, and if you've been wrapping values in fns to make them match delays you'll no onger have to

16:46 longer

16:47 check the diff on Delay.java - it's not functionally different

16:47 cemerick: yeah, I'm looking at it right now; no longer being an AFn is a significant difference

16:48 rhickey: not in what happens during invoke/force or the nature of delayed values

16:49 cemerick: you can't invoke a delay any more -- that's quite different, implementation detail or not

16:50 yeah, I mistyped the (delay (fn []...)) bit -- I meant (delay ...)

16:51 rhickey: the reason I took out delays as fns and delays as IRef is that they do not have the same behavior when the contained _value_ is itself a fn, e.g. invoking a var casts its value to fn and invokes it, but a delay didn't do that

16:56 cemerick: I don't currently use refs much (if at all); I wouldn't think that their requirements are tied to delays, though.

16:56 rhickey: cemerick: delays really aren't functions of anything, not taking args makes that plain

16:57 I understand it's a breaking change for you, and I'm sorry about that, but I made delay public to help you along and have subsequently refined its design - I've got to be able to do that between releases

16:58 uniform force is plainly a better interface than invocation

16:58 cemerick: oh, no need to apologize, please :-)

17:00 rhickey: but there is a transformation between what you've been doing and the new way, that is easier on the producer side and, let's say different on the consumer side, but strictly more powerful even if you don't currently leverage that

17:06 Chouser: I guess what bothers me about my starved transaction is that I could have a large system that's functioning just fine, but by adding some new code that innocently pokes away too quickly at a ref, I could cause some existing slow transaction to start hanging for long stretches at a time.

17:07 I suppose you'd just have to test for that and fix your slow transactions -- split them up, add ensure, or whatever?

17:09 rhickey: Chouser: that's not a problem I can solve for you. There will always be issues with contention that are part of a concurrent design. STM and agents handle the correctness part, but you will still have to deal with transaction granularity, degree of overlap, transaction duration etc.

17:10 one general rule is not to have long-running transactions compete with short running transactions for the same refs

17:11 (writing)

17:15 Chouser: did you try the ensure before sleep?

17:16 Chouser: yep, wokred like a charm

17:16 rhickey: so you do have some tools to work with

17:16 Chouser: yes

17:17 cemerick: rhickey: thanks for clarifying things earlier; I hope I didn't come off as irritated or whatever...

17:18 rhickey: np, I understand it sucks to have something changed from underneath you

17:18 cemerick: nah, it's to be expected when you're running off of HEAD :-D

17:19 besides, nothing will change much; I expect I'll create my own DelayFn class (or whatever) to retain the same behaviour for myself. Oh, the joys of interfaces...

17:21 also besides, such things are a very, very small price to pay to be able to use something as truly phenomenal as clojure

17:21 rhickey: Chouser: I meant 'can solve for you automatically' before, of course I'm always willing to help you

17:22 Chouser: rhickey: heh, I understood.

21:12 jamii: how does clojure find namespaces when refer is used? do I have to load the file first?

21:13 Chouser: yes, refer only works with names that have already been loaded

21:27 (apply concat (partition 1 2 [1 2 3 4 5])) => (1 3 5)

21:27 but there's got to be a better way, right?

22:22 arohner: I'm finding myself doing

22:23 (let [foo (assoc foo key val)]...)

22:23 rhickey: user=> (take-nth 2 [1 2 3 4 5])

22:23 (1 3 5)

22:23 arohner: that doesn't feel idiomatic

22:23 is there a better way to accomplish that?

22:25 rhickey: arohner: what feels wrong - reusing foo?

22:25 arohner: yeah

22:25 rhickey: you could call it bar :)

22:25 arohner: hah

22:25 I'm immediately recurring on foo

22:25 so for more context

22:25 (let [foo (assoc foo key val)] (recur foo))

22:26 but it still feels slightly weird

22:26 rhickey: the first foo is in a loop?

22:26 arohner: yes

22:27 rhickey: I'd just (recur (assoc foo key val))

22:27 arohner: ok, I guess that makes it idiomatic :-)

22:28 hmm, maybe that's not the source of my hesitation

22:28 rhickey: but consider reduce

22:28 arohner: ah

22:28 maybe that's what I'm looking for

22:29 I didn't think of treating the problem that way, but that might be a better fit

22:30 thanks

22:32 rhickey: sure

22:36 Chouser: rhickey: take-nth. I feel dumb now.

22:37 rhickey: Chouser: don't worry - I had to look it up myself :)

22:37 Chouser: arohner: and if you're using reduce to build a map, you might like into

22:38 rhickey: yes, if you are not doing any other logic

22:38 arohner: what does conjoined mean in this context?

22:39 rhickey: user=> (conj {} [:a 1])

22:39 {:a 1}

22:39 user=> (conj {} [:a 1] [:b 2])

22:39 {:b 2, :a 1}

22:40 arohner: ah, that conjoined :-)

22:40 rhickey: maps are collections of entries, so you can conjoin an entry

22:41 or another map:

22:41 user=> (conj {:a 1} {:b 2 :c 3})

22:41 {:c 3, :b 2, :a 1}

22:42 Clojure is like Lego

22:42 arohner: hah

22:42 that wouldn't be a bad irc topic

23:00 are there any functions for working with ratios?

23:01 i.e. get the numerator, get the denominator, increment the fraction?

23:03 rhickey: (.numerator 22/7)

23:04 and .denominator

23:04 arohner: ah, cool

23:05 Chouser: huh. Why don't I see those with (doseq x (.getMethods (class 3/4)) (prn x))

23:06 rhickey: because they are fields

23:06 Chouser: ah

23:06 rhickey: public final

23:06 arohner: Chouser: that is a cool trick

23:07 and yet again clojure is a better java than java

23:07 not to say that's only what it's good for!

23:08 Chouser: arohner: one of the tricks I use to give the impression I know things when I don't.

Logging service provided by n01se.net