#clojure log - Dec 26 2010

The Joy of Clojure
Main Clojure site
Google Group
List of all logged dates

6:36 maacl: LauJensen: Glaedelig jul!

6:37 LauJensen: maacl: Tak for tanken, men jeg fejrer ikke jul :)

6:37 maacl: Ok - religioest eller anti-religioest betinget?

6:38 LauJensen: ups - religioest eller anti-religioest betinget?

6:40 LauJensen: You got pinged in private )

6:40 :)

6:55 zvrba: ok, installing clojure in ANY os is a PITA

6:59 Raynes: Define 'install'.

6:59 You don't really 'install' Clojure. All Clojure is is a jar file. The closest thing you get to 'installing' Clojure is just downloading cake or leiningen, which isn't all that difficult at all.

6:59 zvrba: i write BigInteger/LONG_MASK in REPL, and clojure replies with not being able to find static field.

7:00 Raynes: install = making it work with emacs. and understanding how to do it in proper order. persuading windows to get paths right. figuring out whether I need ant+maven (didn't install them, still unsure whether I need them - netbeans plugin recommends maven strongly)

7:00 etc.

7:01 Raynes: i.e. install = anything more than running a simple REPL in shell prompt.

7:02 Raynes: Getting it to work with Emacs isn't really part of installing Clojure. You don't really *need* Emacs to use Clojure. There is a plugin for IntelliJ and Eclipse. As for ant and maven, you don't really need them unless you need them to build any particular project. Most Clojure projects use Leiningen or Cake as their build tools.

7:03 zvrba: i know that I don't *need* emacs.

7:03 given that clojure uses a lot of java, i might really be better off using IntelliJ

7:04 last time I tested plugin for IntelliJ, it didn't work. it was a year ago. i DO NOT want to use eclipse. which leaves me with NetBEans vs. IntelliJ.

7:04 any thoughts on that one?

7:04 Raynes: The act of getting Clojure to work with Emacs is actually a lot easier than it used to be though. It used to be that you'd need to put stuff in .emacs, which was error prone and never really worked for everybody. Now, you can just grab ELPA and use it to install clojure-mode and slime-repl, and then grab cake or leiningen and use them to get a swank server running on your project which you can then connect to via M-x slime-connect

7:04 The IntelliJ plugin was pretty young a year ago. I bet you'll have much better luck now.

7:04 zvrba: yeah, i figured that out. the toughest part was realizing that I needed (package-init) to get packages installef with ELPA to load :-)

7:05 Raynes: any thoughts on IntelliJ vs. NetBeans in general?

7:05 (related to clojure programming)

7:06 Raynes: I don't use either, so anything I say is based on what I've heard. It looks like Enclojure's development is much slower than the other plugins development, and a lot of people simply cannot get Enclojure to work.

7:06 I've heard good things about the IntelliJ plugin.

7:06 zvrba: Also, it doesn't look like BigInteger has a field called LONG_MASK

7:07 http://download.oracle.com/javase/1.4.2/docs/api/java/math/BigInteger.html

7:07 zvrba: Raynes: oh, ok. I get it listed with tab-completion in slime.

7:07 hence, confusion.

7:07 Raynes: Hrm.

7:08 zvrba: BigInteger/<TAB> lists that field.

7:08 ah. maybe it's protected, or something.

7:08 Raynes: Bizarre.

7:08 Yeah, it could be.

7:08 zvrba: swank maybe is not aware of access levels.

7:09 ok, i'll give IntelliJ a try then.

8:01 there doesn't seem to be a way to nicely interact with REPL from intellij

8:01 like, write something in the source file, send it to REPL and set focus to REPL to test it

9:31 Raynes: That was quick.

9:32 cemerick: Raynes: you rang? :-P

9:32 Raynes: Merry Christmas cemerick. <3

9:32 cemerick: same to you, man :-)

9:32 Raynes: What did you get? Looks like Land of Lisp was a popular gift item this year.

9:33 cemerick: LoL isn't really my style :-)

9:33 Raynes: I spent all of yesterday morning finishing that initial draft of your book.

9:34 cemerick: let's see: a bluray player (which I'll probably trade in for a ps3), a very nice watch, and some light reading I'll tweet about later

9:34 Raynes: not too much of a disaster, I hope?

9:34 Raynes: I was on the interop chapter, which I had plenty of motivation to read, given that I'm actually just getting to that chapter in my own book.

9:35 The book wasn't bad at all. The CouchDB chapter was also relevant to my interests, given that I'm working on a project that will call for it.

9:35 cemerick: I'm leaving a fair bit of that undone, given the prim/numerics changes, etc.

9:35 Raynes: Kind of disappointed at how awkward talking to CouchDB is compared to MongoDB.

9:35 cemerick: that's interesting

9:36 Raynes: do you have a favorite mongodb/clojure interaction blog post?

9:36 Raynes: Well, given the nature of a predominately electronic unpublished book, I can change and add things as they become relevant and release new "editions" easily after the book has been finalized.

9:36 I don't think I've read any. Just a look at the congomongo README should be sufficient though.

9:37 Note that just because it seems awkward to me doesn't really say a lot about it.

9:37 It's obvious that views are a much more powerful concept.

9:37 I just wish they provided for MongoDB-like lookups in simpler cases.

9:38 cemerick: oh, you mean ad-hoc query interaction?

9:38 Raynes: Right.

9:38 cemerick: yeah, tha'ts a whole 'nuther ballgame

9:39 Raynes: :p

9:39 cemerick: the theory there being that it's just not the right tool for that particular job

9:40 There's actually very slick replication endpoints for pushing couchdb data into mongo, where you would presumably do your ad-hoc querying in the latter.

9:40 Raynes: I was considering switching to Couch in sexpbot, but I definitely need to do more research on that. I need to know precisely what I would gain by doing so, because it certainly wouldn't be more concise code.

9:42 cemerick: I've nearly beat you as far as pages go, by the way. ;)

9:42 cemerick: Raynes: sexpbot uses mongo on the backend?

9:42 Raynes: 76 in my latest draft. No stub chapters or anything like that.

9:42 Right.

9:43 A lot of plugins call for a database.

9:43 cemerick: Raynes: tortoise and the hare and all that comes to mind :-)

9:43 I think we're probably aiming at different demographics anyways ;-)

9:43 Raynes: Well, I don't have a job or much of a life either, so I naturally have more time to write.

9:43 cemerick: Raynes: I hope you're doing whatever is required to make mongodb do sane things re: durability?

9:44 I assume you're not running a "web scale" cluster over there ;-)

9:45 Raynes: Not really. I just store stuff in it. I'm not sure how much sexpbot could really bring out the bad in MongoDB at this point. I really just picked a DB and started using it, sans actual research. I don't know much about the general pros and cons regarding the various databases.

9:46 If I had to store data that was actually important, I might care more.

9:46 chouser: I think running a single mongodb server without a replication set may very well lead to heartache

9:46 I think mysql promises more about durability than an unreplicated mongodb

9:47 Raynes: chouser: I've been looking all over for you.

9:47 cemerick: Raynes: insofar as sexpbot may provide logging capability in the future, that may change.

9:47 Raynes: cemerick: Define 'logging'.

9:47 cemerick: chouser: Well said, sir!

9:47 Raynes: channel logging, etc

9:47 chouser: Merry belated Christmas :-)

9:48 Raynes: chouser: There is an example in The Joy of Clojure that defines a print-method method for queues: may I steal it for usage in my book?

9:48 Giving credit where credit is due, of course.

9:48 cemerick: sexpbot already does channel logging.

9:48 cemerick: But that has nothing to do with the database.

9:48 http://raynes.me/logs/

9:49 cemerick: Raynes: it does if the logs are in an unreplicated mongo instance.

9:49 Raynes: Why would they be?

9:49 I just write to log files for consumption by a web browser or text editor.

9:49 No database storage necessary.

9:50 cemerick: nevermind, then :-)

9:50 I figured they'd be in the database to support potentially interesting query capabilities, etc.

9:50 chouser: heh, the queue fish? Sure, I think quoting text at that scale is clearly within fair use.

9:50 Raynes: Anyways, I have been planning to discuss database options with amalloy_ very soon, before we *do* have to store important data. You know, get while the getting is still good and all that.

9:51 chouser: The queue fish is adorable.

9:51 chouser: cemerick: I once worked very hard to get sane irc log search working in a mysql database (for a private IRC channel). My best was horrible little joke compared to google indexing the static pages for #clojure.

9:52 Raynes: My IRC logs are extremely important. I mean, I just couldn't imagine what I would do if I needed to view the last 15 minutes of logs before chouser's updates. ;)

9:55 cemerick: chouser: I can imagine that :-)

9:56 Raynes: CouchDB is gigantic compared to MongoDB.

9:57 I'll probably have to go with something more minimal.

10:03 cemerick: Raynes: gigantic?

10:04 like, the size of the rpm/deb or something?

10:04 Raynes: cemerick: Including dependencies, of course.

10:05 cemerick: The all-in-one installer is over 20 megs, while the binary distribution of MongoDB is less than 2, and on my barebones Ubuntu VPS, I didn't have to install any outside dependencies.

10:06 I couldn't imagine convincing anybody to install a 20MB database in order to run a ~5mb IRC bot.

10:07 cemerick: Raynes: I'm pretty sure no one gives a moment's thought to what apt or yum installs. *shrug*

10:08 Raynes: cemerick: I guess not, given that there are few people left on this planet that have a connection quite as slow as mine.

10:08 * Raynes looks at his cellphone, longing to be closer to a 3G cell tower.

10:09 Raynes: I used the Ubuntu installer from couch.io, because I have this phobia of older versions of things.

10:14 cemerick: Raynes: that's a wise thing to do, especially given ubuntu/debian's tendency to screw around with packaging, often to the detriment of proper functioning of things.

10:18 neilmock: what is the proper way to use a protocol defined in another namespace? haven't had any luck with use/require/import...

10:21 Raynes: Why hasn't clojure-clr moved to the Clojure organization?\

11:11 shortlord: how do compare vim and emacs for clojure development? I haven't learned either one yet, but so far vim looks quite nice to me. I have read that clojure/lisp development is better in emacs though (because of better slime/swank-clojure integration) how big are the differences and when would you choose one over the other?

11:12 krumholt: if i have a function. can i check how many arguments it takes?

11:13 Raynes: &(:arglists (meta #'clojure.core/range))

11:13 sexpbot: ⟹ ([] [end] [start end] [start end step])

11:14 Raynes: Clojure functions can take different numbers of arguments (arities) and do different things depending on the number passed.

11:14 The arglists are stored in metadata, and that's how I got it.

11:14 krumholt: thanks. i can work with that

11:15 Raynes: shortlord: The majority of Clojure programmers use Emacs for Clojure development. SLIME is mostly unparalleled as far as a Lisp development environment goes. Emacs is also configured in a Lisp, and that is a bit of a plus for Lispers.

11:15 However, VimClojure's support isn't bad.

11:16 I'd typically say "go with whichever you're familiar with", but since you aren't familiar with either of them, you might just want to investigate a bit before making your decision.

11:16 I'd focus on what the editors themselves bring to the table rather than just how good their Clojure support is.

11:17 You'll almost certainly fall in love with whichever editor you choose, and you'll want to use it for more than just Clojure.

11:17 In any case, Emacs supports SLIME, and it doesn't get much better than that for a Clojure development environment.

11:21 shortlord: Raynes: ok, thx for the advice. What are the distinguishing features of slime that are not available in other editors? I've read vim supports slime through slimv as well, how far superior is slime in emacs?

11:23 Raynes: http://common-lisp.net/project/slime/ Has some feature highlights and links to a couple of screencasts.

11:24 And I haven't heard of slimv before.

11:31 shortlord: Raynes: ok, thx a lot. I'll look into both editors then

13:26 ajazz: Does 1M mean new BigDecimal(1) ?

13:57 LauJensen: ,(class 1M)

13:57 clojurebot: java.math.BigDecimal

14:40 powr-toc: has anyone here used gloss?

15:05 tomoj: powr-toc: yes

15:36 powr-toc: tomoj: great... I've been getting on pretty good with it, decoding a binary file format... but I'm wondering how to coerce 3 bytes into an integer

15:39 i.e. 20bits

16:05 looks like BigInteger can do it

19:12 ossareh: 'lo all

19:30 tomoj: powr-toc: sorry, forgot you asked that

19:30 dunno how to do that yet

19:30 I need half-byte numbers soon :/

19:30 are you using any finite-blocks?

20:03 powr-toc: tomoj: it's pretty easy

20:08 tomoj: not using finite-blocks yet no... but if you need nibble numbers, you can throw the byte (as a byte-array) into BigInteger and then shift left/right on it as you need

20:08 BigInteger should have almost everything you need

20:41 tomoj: powr-toc: thanks..

20:41 sucks to be using bigints for 4 bit numbers though

20:48 krl: any clojure emacs mode that properly indents 'extend-type' ?

21:02 technomancy: krl: will need a patch for that

21:02 krl: technomancy is there one already?

21:08 technomancy: not yet

21:11 krl: i could try it if noone else is doing it

21:12 technomancy: that'd be great. I don't use protocols myself.

21:18 powr-toc: tomoj, yeah I know

21:48 livingston: are there reader macros like the pipe character in lisp so that I can control the characters in a symbol name? or can I only do that with a call to symbol?

21:55 krl: technomancy: turns out it was pretty simple: https://github.com/krl/clojure-mode/commit/6cf4f64de92bf2da18d9c8adb7c05cb088d60954

22:00 technomancy: krl: hm; that seems to be branched from jochu's, which is out of date

22:00 it looks like my fork actually has (put 'extend-type 'clojure-backtracking-indent '(4 (2)))

22:00 what does the additional 4 signify?

22:00 krl: i'm not sure, it's not documented

22:01 i used the one from proxy

22:01 but which are the newest forks?

22:02 lghtng: clorqle!

22:03 krl: ok looking at the github fork tree

22:07 technomancy: krl: jochu disappeared over a year ago, so I took over clojure-mode and swank

22:07 I am not very familiar with how indentation works though.

22:09 I guess I should add some message to jochu's that makes it clearer =\

22:11 krl: technomancy: ok yeah your fork worked directly too :) jochus version still has better google pagerank

22:12 joshua__: How would you get the result of a post request with enlive or clojure in general?

22:12 * technomancy growls at google

22:12 technomancy: pagerank is always throwing people off with outdated material

22:12 krl: guess it's because it's still being linked to?

22:12 technomancy: krl: if you install via package.el you'll be sure to get the latest though

22:13 krl: ok, never got around to using elpa

22:13 technomancy: I don't think any new links are being created for it; google is just not very well-calibrated for high-churn domains like Clojure

22:22 joshua__: clojure-contrib has http-agent apparently

22:22 it looks like it will work

22:30 Does clojure http-agent actually work?

22:33 dnolen: joshua__: I use https://github.com/rnewman/clj-apache-http

22:33 joshua__: there's also https://github.com/clj-sys/clj-http, haven't used that myself tho.

22:34 joshua__: dnolen: Copying the examples given in the http-agent documentation crashes slime for me.

22:34 dnolen: so thanks for the alternatives =)

22:50 tomoj: ,(->> (fn [] (future (let [t (Thread/currentThread)] [(.getId t) (.getName t)]))) (repeatedly 10000) (map deref) set)

22:50 clojurebot: #{[359 "pool-2-thread-4"] [360 "pool-2-thread-5"]}

22:50 tomoj: how exactly are futures executed?

22:50 I see they use soloExecutor from Agent, guess I need to read those javadocs

22:56 ,(time (->> (fn [] (future (let [t (Thread/currentThread)] (Thread/sleep 10) [(.getId t) (.getName t)]))) (repeatedly 100) (map deref) set))

22:56 clojurebot: "Elapsed time: 1100.567 msecs"

22:56 #{[362 "pool-2-thread-6"]}

22:56 tomoj: why?

22:56 clojurebot: http://clojure.org/rationale

22:57 tomoj: oh

22:57 of course

23:03 Guest32627: If you have a big pile o' calculations to do, there is no advantage to dividing them up between multiple agents vs having one agent perform them, right? Won't you still end up with an unbounded pool of threads working on (send-off) or a fixed pool working on (send)?

23:11 qbg: Guest32627: If they go to one agent, they will be done in a serial order

23:12 Guest32627: qbg: not true. multiple threads will be working the queue concurrently

23:13 qbg: Function #'send to an agent are queued up

23:13 *Functions

23:13 Guest32627: qbg: yes they are queued up to be handled by the thread pool assigned to the agent

23:13 qbg: if you use "send-off" the thread pool is unbounded

23:13 qbg: They will still happen serially because the result of the previous function is feed to the next one

23:14 Guest32627: qbg: that is absolutely not true at all

23:14 qbg: the result of the function merely sets the current agent state

23:14 qbg: Which will be feed into the next function

23:14 Guest32627: qbg: that is false

23:15 qbg: i'm sorry, i mean to say it definitely does NOT go to the very next item on the queue, because they will very likely go out of order if you put a lot of i/o messages in the queue, for example

23:16 the state just becomes whatever the return is from the current thread that finished running your function

23:16 the items in the queue can be processed out of order, and will be, with slow i/o handling on an unbounded thread pool

23:20 technomancy: Guest32627: if you have a pile of calculations that aren't meant to be a series of computations against a single identity, you shouldn't be using agents at all

23:20 Guest32627: technomancy: that's a very good point and i agree

23:21 technomancy: so let's assume they are not against a single entity

23:21 technomancy: if it's possible to split your work among multiple agents, then it's likely your work would be better done using futures

23:22 Guest32627: technomancy: so my suspicion is right that there is no point to doing such work with multiple agents? for example, i can see using an agent for cpu-bound calcs and another for some sort of i/o like logging, but not multiple agents for the same thing

23:22 is that the right idea?

23:23 technomancy: Guest32627: the point of agents is that they are a single reference type that can be used to model a given entity that has many values over a period of time

23:23 qbg: Only one function will be ran on an agent at a time

23:23 Guest32627: technomancy: that is true but insufficient. the asynchronous nature of agents is also a key feature. what you describe holds equally well for atoms and refs

23:24 qbg: And they will happen in order

23:24 http://clojure.org/agents

23:24 Guest32627: qbg, that is false

23:24 qbg: read the part about thread pools

23:24 technomancy: Guest32627: sure, but if you're just concerned with asynchronicity, just use a thread or thread pool on its own

23:24 qbg: "At any point in time, at most one action for each Agent is being executed."

23:24 "Actions dispatched to an agent from another single agent or thread will occur in the order they were sent, potentially interleaved with actions dispatched to the same agent from other sources."

23:26 Guest32627: qbg: then what *exactly* is that unbounded thread pool doing that is handling (send-off)?

23:26 qbg: All sends are sent to a bounded thread pool

23:26 So a send that is io-bound will consume resources from other agents

23:26 Guest32627: qbg: ergo more than 1 thread (# cpu + 1) but send-OFF is unbounded

23:26 qbg: send-off uses an unbounded thread pool so other agents will not be starved

23:28 Guest32627: qbg: so you mean to say you need to create as many agents as you would like to have potentially handling messages?

23:28 qbg: if i want to fan out to, say, 100 threads, i need to create 100 agents?

23:28 qbg: Yes

23:29 Guest32627: qbg: ahh... my sincere apologies then! this was not clear from joy of clojure

23:29 qbg: Agents are likely the wrong tool for what you want to do though

23:30 Guest32627: qbg: what if i want to asynchronously handle responses in a server app? is that a good use for them?

23:31 tomoj: agents appear nowhere in aleph/lamina, interesting

23:31 qbg: If it is simpler to just use a thread, using an agent is likely the wrong thing to do.

23:34 The only thing that I have written that was a good fit for agents was a parking lot simulator

23:34 Guest32627: qbg: it would be pretty simple to cycle through some max number of agents though in order to bound the upper size on your thread pool

23:34 qbg: rather than managing some thread pool yourself. why bother when you get it for free?

23:35 qbg: agents carry state; if you don't need that state, using agents is probably overkill

23:35 Guest32627: qbg: the example i'm looking at now in Joy of Clojure uses that state as merely a counter for logged messages

23:35 qbg: java.util.concurrent gives you easy to use thread pools also

23:36 Guest32627: qbg: sure but why drop down to that if you don't need to. i get the same functionality by staying entirely within clojure itself

23:36 qbg: oops, that was a question :)

23:37 qbg: Tastes vary. I don't like to abuse agents/futures in my code for their side effects.

23:39 Guest32627: qbg: i don't see how leveraging asynchronous processing in an agent is an abuse if it's a fundamental requirement. you mean just because the inherent state of an agent is not terribly important? is that what you mean by abuse?

23:40 qbg: If the functions you send to an agent ignore their first argument, you are probably abusing the agent system

23:41 Likewise, if you don't care about the return value of a future, why not use a thread instead?

23:41 Guest32627: qbg: ok, fair enough. joy of clojure is definitely abusing agents then in their example :)

23:42 qbg: A counter would be fair use

23:42 Guest32627: qbg: for log messages? he doesn't even use it in the example

23:42 qbg: A fair use of agents that ignore their first argument is for logging because they integrate with the STM

23:46 Guest32627: qbg: that's actually kind of interesting when you think about it. because if the transaction fails, you get no log either :)

23:47 qbg: And if it retries but eventually succeeds, you get only one

23:48 Guest32627: but only if it succeeds. not if there is an uncaught exception, for example

23:48 well i guess you could have an error handler do some magic at that opint

23:49 maybe this is why it's so hard to find clear, convincing uses of agents.

23:50 i've only heard how they are NOT good for various uses. not what they are good for (only a parking lot simulator)

23:50 sounds like a worthless feature tehn

23:51 qbg: The integration with the STM is a good thing

23:52 They are also good for autonomous identities

23:54 Actually, agents are rather close to smalltalk style OO

23:55 Guest32627: qbg: but if they aren't doing some work on your behalf then their autonomy and integration with the STM isn't of much value

23:56 qbg: i was thinking of them as more like MDB, where the asynchronous processing is the key feature. you seem to think that's not too important compared to the state that the agent holds

23:56 qbg: i can see your point but i'm struggling to think of any good use for them at all. ok, so i can log with them. i'm sort of underwhelmed

23:57 qbg: I think you are underwhelmed because there aren't many good uses for object either

23:58 *objects

23:58 Guest32627: qbg: when you put it in those terms, sure. i agree! however there are very many good uses indeed for asynchronous processing

23:58 qbg: but for that you're saying "well yes, but just use regular java threads and don't bother with agents"

23:58 qbg: If you need async + state, then agents work. If you only need one of them there are probably better choices

23:59 Guest32627: you don't need async + state for logging either

23:59 qbg: Agents are useful there as a side effect of their interaction with the STM

Logging service provided by n01se.net