0:10 technomancy: wolfjb: you probably just want to use clojure.java.io/copy between a file and a zipoutputstream
0:16 wolfjb: technomancy: thanks! amalloy and ataggart have been very helpful in leading me toward that solution as well
0:33 I'm not understanding apply very well tonight, I thought I could call (apply addentry files zip-outputstream) which would turn into (addentry file zip-outputstream) for each file in files. however, that isn't working right.
0:33 technomancy: yeah; apparently I didn't read the backlog
0:33 ataggart: ,(apply + [1 2 3])
0:33 clojurebot: 6
0:34 ataggart: wolfjb: you probably want doseq
0:34 wolfjb: ah
0:34 ataggart: e.g., (doseq [f files] ...)
0:35 amalloy: wolfjb: apply "expands" the last argument (which must be a seq) in-place to result in a function call with more arguments
0:35 &(apply (partial list '+) 1 2 [3 4 5 6])
0:35 sexpbot: ⟹ (+ 1 2 3 4 5 6)
0:35 wolfjb: ah
0:35 (I feel like I'm saying that a lot)
0:35 heh
0:35 amalloy: then you're doing it right
0:35 wolfjb: thanks for the help
0:36 you guys are awesome!
0:37 woot! my zip stuff works now!
3:41 amalloy: do agents have a reset!-like functionality, or do i have to send them (constantly x)?
6:24 Vinzent: I'm using appengine-magic and have created a protocol and extended it to the appengine entity, but when I'm calling protocol fn on the entity queried from database, implementation for Object is chosen
6:29 example: http://
6:30 What could be the problem?
7:09 Cozey: Sorry for an OT, but I'd like to query Lisp programmers -- if you use/have to use Python or Ruby, which one do you choose and why?
7:14 manutter: Lisp programmers aren't awake this early ;)
7:15 Cozey: :-)))
7:16 vijaykiran: Hi all a quick question - I want to create a simple REST based app using compojure - which JSON library should I use ?
7:20 Vinzent: vijaykiran, take a look at clojure.contrib.json
7:22 raek: Vinzent: clojure.contrib.json has now been lifted to its own project: https://
7:22 [org.clojure/data.json "0.1.0"]
7:22 (works with both clojure 1.2 and 1.3)
7:22 vijaykiran: ^
7:22 Vinzent: oh, ok, i thought it requires 1.3
7:22 vijaykiran: Vinzent & raek: thanks!
7:23 timvisher: hey all
7:23 i find myself pulling out google whenever i want to look for a reference to a namespace
7:24 `doc` is great if i know the function i'm looking up
7:24 `find-doc` is great if i don't, but it doesn't seem to work for namespace prefixes, i.e. `(find-doc "clojure.pprint/*")` doesn't return all the functions in that namespace
7:24 raek: Cozey: kinda OT answer: I like Python 3 because it has a good string model (i.e. strings are sequences of characters rather than bytes, so you don't work with the data in its encoded form)
7:24 timvisher: is there any way to get the references that i find at clojure.org at the repl?
7:25 raek: ...but I've heard that they have fixed the string model in Ruby too
7:25 manutter: timvisher: dunno about the repl, but I use clojuredocs.org for a lot of that kind of stuff
7:26 timvisher: manutter: thanks for the pointer! hadn't seen that before
7:26 raek: timvisher: print-namespace-doc
7:26 timvisher: i'm primarily interested in it because i want access to these docs offline, and i want them to be specific to what the version i'm working with
7:26 Vinzent: timvisher, not exactly what you want, but i use auto-complete which shows all public vars and doc-strings in the ns
7:27 Cozey: raek: will Ruby has similar migrating difficulties as python2->python3 ?
7:27 timvisher: hmm
7:28 raek: that looks like what i want, but calling it like `(print-namespace-doc 'clojure.pprint)` returns clojure.pprint \n nil
7:28 raek: Cozey: from what I have heard, they added the support in a non-major version
7:28 Cozey: mhm
7:28 Vinzent: timvisher, it prints docstring of the ns
7:28 raek: and that sounds scary to me
7:29 timvisher: vinzent: ah
7:29 cemerick: timvisher: there's also http://
7:30 timvisher: cemerick: that looks cool to
7:30 i still want the offline thing.
7:30 raek: I'm so happy that Clojure got the string model thing right...
7:30 timvisher: maybe it just doesn't exist at the moment
7:31 i thought there may be a simple function that they call to generate the online docs
7:31 raek: well, it's mostly Java's effort...
7:31 cemerick: timvisher: you'll be able to download a static copy of your atlas very shortly :-)
7:31 timvisher: but it's probably akin to javadoc
7:31 raek: timvisher: you can clone the gh-pages branch of clojure
7:31 Vinzent: I'm using appengine-magic and have created a protocol and extended it to the appengine entity, but when I'm calling protocol fn on the entity queried from database, implementation for Object is chosen
7:31 raek: that way you get the same html files that are on the site
7:31 Vinzent: example: http://
7:32 timvisher: raek: that's an interesting option
7:32 Cozey: yeah. i need to code something in python and it is a pain in a way now. but from github i see many clojure people useing ruby as welll
7:32 timvisher: hadn't considered doing that yet
7:35 Vinzent: No ideas? Am I missing something obvious?
7:35 manutter: timvisher: btw, if you poke around clojuredocs.org, there's a downloadable standalone archive. I got it running on my laptop fairly easily (though iirc it's a Rails webapp!)
7:36 raek: Vinzent: maybe you extend the protocol to the wrong type or something...
7:37 also, I'm not very sure how ambiguities are handled (your type is both of the entity type and of the Object type)
7:37 timvisher: manutter: thanks. that's a litle like Apple using those windows mobile devices for awhile in there stores for their mobile point of sale devices in their stores. :)
7:38 manutter: Heh
7:38 timvisher: is there an easier way than including multiple Exceptions in a catch clause than multiple (catch...) forms?
7:38 Vinzent: raek, no, when i'm creating object explicitly it works well, so type can't be wrong...
7:38 timvisher: i.e. a seq of Exception types or something along those lines?
7:38 Vinzent: raek, but every type is an Object!
7:38 timvisher: (catch [FooException IndexException BlahException]...)?
7:38 raek: Vinzent: http://
7:39 manutter: timvisher: I guess you could just catch RuntimeException and then check for which subclass the actual exception is?
7:39 timvisher: manutter: mmm. trying to avoid that
7:40 i could always extract the catch function out and then call that from each catch statement
7:40 raek: timvisher: you can with a macro
7:40 manutter: true
7:43 Vinzent: raek, but my entity type is an descendant of the Object; i believe such cases are handled well
7:43 anyway i'll check this
7:55 raek, thanks, you are right: i've removed default implementations and all works fine now; i'm completely confused with that
8:00 raek: Vinzent: with 'extend', you can "mix" the implementation from multiple sources. it expects a map, so you use ordinary functions (assoc etc) to compose the new implementation
8:01 Vinzent: an example https://
8:04 Vinzent: raek, yes, i know that, just thought protocols "know" about inheritance
8:05 so user don't have to extend protocol on his type unless he needs special behaviour
8:06 raek: I know that they were not designed to provide inhertitance, but I cannot present any of the "inheritance is bad" arguments
8:06 well, except for reuse with maps in extend
8:07 all I know is that protocols were never meant to be a class system like that of java
8:13 dnolen: Vinzent: raek: a longish but excellent explication around the issues around inheritance. tl;dr, implementations have many assumptions backed in, inheritance locks you into those assumptions. Better to program against protocols. In Clojure the benefits are obvious, trivial to create things from scratch which interoperate cleanly w/ the entire system - i.e. your own IPersistentSet.
8:13 http://
8:15 inheritance only gets you convenience, protocols get you much further - scalability and sustainability.
8:35 Vinzent: dnolen, thanks, I'll definitely read it tonight!
8:39 dnolen: Vinzent: it's a good read. Objective-C also takes the inheritance is bad stance + explicit support for protocols. Java interfaces were basically lifted from Objective-C protocols as far as I understand. http://
8:40 old ideas.
8:57 Vinzent: dnolen, interesting point. I don't know much about obj-c (just heard that it's "right" cpp), but it would be good to learn (at least briefly) other, non-lispy languages to understand clojure better
9:07 btw, can I somehow re-extend protocol dinamically?
9:10 dnolen: Vinzent: if you mean add methods to a protocol, then no.
9:12 Vinzent: dnolen, no, I mean redefining from the repl/recompiling, but looks like the problem is in appengine
9:12 dnolen: Vinzent: oh yeah, definitely.
10:17 amac_: hmm, anyone know how I can use contrib.sql/insert-records and find the auto-inc primary key the record was assigned... querying on the provided columns might bring back multiple rows
10:47 cemerick: OK, continuing on my stream of numerics-related ignorance: bigints are contagious, so (+ 1 XXXN) => XXYN. Why aren't BigDecimals contagious as well? As things stand, (+ 1.0 1e400M) => Infinity.
10:48 It seems like the loss of precision there is undesirable…?
11:08 joegallo: cemerick: 1.0 as a double isn't very precise, though, is it? wouldn't making the result be a bigdecimal mean a magical gain in precision? double + bigdecimal ==> double because you can't manufacture precision out of nowhere. the same doesn't apply for an integer because they are already precise. (caveat: i have no clue what i'm talking about.)
11:16 cemerick: joegallo: fair point, that may be it.
11:18 TimMc: or... maybe it's a bug :-)
11:18 cemerick: Perhaps the "right" thing to do might be to promote as necessary, and then flatten the precision of the result down to the least-precise input you have.
11:19 TimMc: I don't think so; there's never been any discussion of BigDecimal contagion, so I have to assume that that's intended. I figure I'll ask some stupid questions to understand why, though. :-)
11:20 TimMc: joegallo's explanation is pretty convincing from a semantics perspective, though
11:21 Does BigDecimal has a larger exponent range than doubles?
11:21 cemerick: arbitrarily so
11:22 TimMc: I suppose it has to.
11:22 Also, I can't English today. >_<
11:23 pjstadig: yeah i think it is basically what joegallo says
11:23 there's like essentially a subset relationship between the various ints
11:23 every long can also be represented by a big int
11:23 but i guess it's not the same way with floating point, or so i've been lead to understand
11:30 cemerick: pjstadig: NaN and infinity are unrepresentable in BigDecimal (AFAICT), so perhaps that's it.
11:32 pjstadig: there is also a difference between 1.1 and 1.10 in terms of precision (or accuracy i forget?) so upcasting from a double to a BigDecimal may essentially add zeroes onto the end of the base or something
11:33 TimMc: cemerick: Ooof, good point.
11:33 pjstadig: (caveat: i also have no clue what i'm talking about)
11:34 cemerick: pjstadig: Same here, but negatory on that last one:
11:34 ,(java.math.BigDecimal/valueOf 1.1)
11:34 clojurebot: 1.1M
11:34 cemerick: *shrug*
11:35 pjstadig: cemerick: well it prints that way, but how is in represented in memory?
11:35 meh
11:35 it's a problem i don't have :) (having to worry about all this precision stuff)
11:36 cemerick: ,(.precision (java.math.BigDecimal/valueOf 1.1))
11:36 TimMc: ,(java.math.BigDecimal. Double.POSITIVE_INFINITY)
11:36 clojurebot: 2
11:36 java.lang.ClassNotFoundException: Double.POSITIVE_INFINITY
11:36 TimMc: ,(java.math.BigDecimal. Double/POSITIVE_INFINITY)
11:36 cemerick: so, it's not inflating precision or anything
11:36 clojurebot: java.lang.NumberFormatException: Infinite or NaN
11:36 cemerick: yeah, there's no infinity in BD
11:36 TimMc: however
11:36 ,(.doubleValue 1e9999999999999999999999)
11:36 clojurebot: Infinity
11:37 pjstadig: i have on my reading list some paper (that was footnoted in JoC) about how floating points are evil...guess i should read it
11:44 dnolen: ,(/ (BigDecimal. 1.05) 3)
11:44 clojurebot: java.lang.ArithmeticException: Non-terminating decimal expansion; no exact representable decimal result.
11:45 dnolen: ^ a really good reason for no BigDecimal contagion
11:47 cemerick: dnolen: Good point.
11:48 I guess * and / involving BigDecimals could rationalize them first…
11:48 TimMc: dnolen: Not sure I see it...
11:48 * cemerick is kidding
11:48 TimMc: could pass in a rounding mode.
11:48 / could, that is
11:50 dnolen: TimMc: if it's rounding what's the point?
11:50 TimMc: hm
13:38 amalloy: technomancy: looks like http://
13:41 technomancy: amalloy: sweet; thanks for following up on that
13:41 amalloy: technomancy: i couldn't bear to have the bug blamed on me, see :)
13:42 technomancy: heh. do you know how long the bug's been around?
13:42 wondering if I should drop the check or if I should assume for a while that all 24 users are affected
13:42 amalloy: technomancy: uh. well, i reported it last week. if you mean, how long it's been broken, i don't know
13:42 but git blame ought to reveal it pretty quickly
13:43 edw: Yay, amalloy and technomancy! Thanks for looking into this.
14:33 coopernurse: trying to get my clojure/emacs/slime setup going on Aquaemacs. Installed clojure-mode via marmelade, but when I restart emacs the mode isn't available. do I need to add anything to my .emacs file?
14:33 I'm a little confused where the .el files were installed to
14:35 technomancy: it should be in ~/.emacs.d/elpa, but in general forks of GNU Emacs are not supported.
14:35 * technomancy recommends http://
14:35 coopernurse: ah, ok thanks
14:35 I see stuff here: /Users/james/Library/Application Support/Aquamacs Emacs/elpa
14:36 but it looks like they're not being loaded when Aquamacs starts
14:36 I'll try the other emacs distro
14:37 desertman: sorry i seem to have a connection problem
14:41 i don't know if i already posted this or not because of a connection problem but i am trying to undertstand contrib.sql http://
14:42 ordnungswidrig: *grr* is there any sensible imap mail client on osx. thunderbird eats my cpu
14:43 drewr: people like sparrowapp.com
14:43 raek: desertman: clojure.lang.ArraySeq *is* a seq. are you sure you're not using (a potentially implicit) str instead of some of the printing functions? (pr prn pr-str prn-str)
14:43 drewr: I find it doesn't support gmail's imap well
14:44 ordnungswidrig: drewr: I have a *real* imap server :)
14:44 raek: ,(pr-str (seq (to-array [1 2 3])))
14:44 clojurebot: "(1 2 3)"
14:44 raek: ,(str (seq (to-array [1 2 3])))
14:44 clojurebot: "(1 2 3)"
14:44 raek: hrm
14:44 ,(str (lazy-seq [1 2 3]))
14:45 clojurebot: "clojure.lang.LazySeq@7861"
14:45 drewr: ordnungswidrig: you could try it then
14:45 ordnungswidrig: drewr: I will. thanks for the point
14:45 coopernurse: I was missing: (package-initialize)
14:45 ordnungswidrig: I tried of measure the number of retries of a dosync block by incrementing a atom
14:45 coopernurse: in .emacs.. I'm a ELPA n00b
14:46 ordnungswidrig: But I do not really trust the numbers
14:46 manutter: ,(apply str (lazy-seq [1 2 3]))
14:46 clojurebot: "123"
14:46 manutter: hrm, what was that other command now...
14:46 raek: ,(apply str (lazy-seq [(lazy-seq [1]) (lazy-seq [2]) (lazy-seq [3])]))
14:46 clojurebot: "clojure.lang.LazySeq@20clojure.lang.LazySeq@21clojure.lang.LazySeq@22"
14:46 desertman: (type (do-commands "show databases")) returns clojure.lang.ArraySeq$ArraySeq_int , which is an array of integers
14:47 raek: desertman: not actually an array, but an array seq
14:47 ordnungswidrig: using pcalls to start 100 parallel executions on 2 refs on a 2 core machine should give some retries, shouldn't it?
14:47 raek: ,(seq (to-array [1 2 3]))
14:47 clojurebot: (1 2 3)
14:47 raek: ,(type (seq (to-array [1 2 3])))
14:47 clojurebot: clojure.lang.ArraySeq
14:48 raek: desertman: it works and prints like an ordinary sequence
14:48 manutter: desertman: ah, yes, you're getting back the results of executing a sql command
14:48 desertman: do-commands is intended for non-select sql commands like drops and inserts
14:49 desertman: You're probably getting back a 1 element array containing the number of rows returned by the "show databases" command
14:50 desertman: yes
14:51 manutter: If you want to see the actual database names, you need (with-query-results results ["show databases"] (doseq [row results] (println row)))
14:52 or something like that
14:53 desertman: ok
14:55 <manutter> - yes, that works, thank you
14:57 manutter: :)
15:00 desertman: <manutter> although the syntax is a bit strange to me, , because it seems that '(doseq [row results] (println row))' is being passed as an argument to with-query-results results ?
15:01 manutter: It's the body of the with-query-results macro -- wraps your code in some setup/cleanup stuff
15:03 ,(doc clojure-contrib.sql/with-query-results)
15:03 clojurebot: Cool story bro.
15:03 desertman: ok, i wouldn't have seen that in the docs, but i guess you need to look at the source to know what these things do, although i'm not there yet
15:04 coopernurse: Ok, got my emacs/slime setup working. thanks for the tip on the other emacs distro
15:04 I never cared for all the extra junk in aquamacs anyhow
15:05 technomancy: "training wheels on a humvee" is one description I've heard =)
15:05 manutter: desertman: there's a few other macros that do similar things, like with-out-str etc
15:06 coopernurse: technomancy: heh.. after years of Java development, this clojure/repl/slime setup is pretty interesting. love reloading code in the repl without restarting the vm
15:07 I'm going through some project euler problems and realizing I know _nothing_ about FP :-)
15:07 my solutions are 10x too long
15:10 * technomancy mumbles something cryptic about "mind like water"
15:11 manutter: ,(doc clojure-contrib.sql/with-query-results)
15:11 clojurebot: Cool story bro.
15:12 manutter: no, doh, that's what I typed last time
15:12 ,(doc clojure.contrib.sql/with-query-results)
15:12 clojurebot: "([results sql-params & body]); Executes a query, then evaluates body with results bound to a seq of the results. sql-params is a vector containing a string providing the (optionally parameterized) SQL query followed by values for any parameters."
15:12 manutter: there we go
16:03 markoman: hi. is there some standard way to formalize strings to ascii form? I need this for searching string from datastore
16:04 im thinking something like: Caté1 -> cate1
16:07 ampleyfly: you want cate1 and Caté1 to collide, then?
16:07 coopernurse: markoman: stackoverflow thread that is relevant to what you want to do
16:07 http://
16:07 not clojure specific..
16:08 but it seems like your first challenge is to make a mapping, since ascii can't represent all unicode characters (obviously), so it's not clear what heuristic you want to use
16:08 TimMc: markoman: What should بينظير ڀٽو become?
16:08 coopernurse: TimMc: right
16:09 markoman: good question, i have no idea...
16:09 hiredman: you render each string as set of paths and then find best fits
16:09 markoman: and is there a upper and lower case version of بينظير ڀٽو?
16:09 TimMc: I work at a company that makes transliteration software, but I don't happen to know the open source answer to this.
16:09 hiredman: good luck
16:16 markoman: there is this example in java: String unicode = "Unicode: \u30e6\u30eb\u30b3\u30fc\u30c9"; byte[] bytes = unicode.getBytes("US-ASCII"); how could I test it on clojure repl?
16:17 TimMc: markoman: That will just get you munged data.
16:18 raek: ,(String. (.getBytes "Unicode: \u30e6\u30eb\u30b3\u30fc\u30c9" "US-ASCII") "US-ASCII")
16:18 clojurebot: "Unicode: ?????"
16:19 amalloy: better than Unrecognizable Rectangles
16:19 raek: the default behaviour seems to be to replace unrepresentable characters with \?
16:20 markoman: yeah, it replace nontransable chars to question marks, not good, but some start
16:21 amalloy: markoman: it is a start down the path of evil. unicode is a much larger set than ascii; trying to downconvert will bring only pain
16:21 raek: amalloy: tell that to "Mr. Yurukôdo"
16:21 ahem. "Yurikôdo", i.e. "Unicode"
16:21 * amalloy doesn't really get it
16:23 raek: I think that's it not very useful for the Japanese (the string contains Katakana) if their text is replaced with questionmarks
16:23 markoman: thats true
16:23 amalloy: that's what i said, right? don't convert unicode to ascii?
16:23 markoman: well then is string to upper and lower case smart enough, it works with unicode?
16:24 raek: amalloy: oh, sorry. I accidentally read "better than Unrecognizable Rectangles" as a comment on "Unicode: ?????"
16:24 amalloy: markoman: insofar as such a thing is possible, yes, it works
16:24 raek: yeah, sorry. better, but not actually good
16:25 "Unicode: ?????" neatly sums up the developer's feelings: "I clearly don't understand Unicode"
16:25 raek: :)
16:25 markoman: I could also try to make a map, as suggested, but for special punctuations only, partial solution
16:25 TimMc: markoman: http://
16:26 raek: markoman: if you only consider the latin alphabet, maybe you could normalize the string to NFKD form (Normalization Form Compatibility Decomposition) and strip all the combining characters
16:26 amalloy: TimMc: that too. but java's toLowercase gets that right, doesn't it?
16:27 TimMc: amalloy: What does "right" mean?
16:27 You have to know the language of origin.
16:27 amalloy: converts uppercase dotted-i to lowercase dotted-i, and likewise for dotless?
16:27 raek: markoman: this can problably do something useful: http://
16:28 amalloy: i guess not in all cases
16:28 i see what you mean about language of origin
16:28 markoman: basically, if I use lower case to this: بينظير ڀٽو does the meaning change totally different?
16:28 raek: it basically knows the stuff about the letters that human know
16:28 markoman: it would be a problem for getting right results with searches...
16:29 raek: markoman: it won't change it, since the Arabic script does not have the lower/upper case distinction
16:29 ,(.toLowerCase "بينظير ڀٽو")
16:29 clojurebot: "بينظير ڀٽو"
16:29 TimMc: markoman: Are you beginning to see why there are are so few search engines? :-)
16:29 amalloy: TimMc: b/c for turkish lowercase of I is dotless, but for english the lowercase is dotted? that's the point you're making, right?
16:30 TimMc: amalloy: Indeed.
16:30 * amalloy is glad not to deal with this stuff
16:30 raek: ,(let [s "بينظير ڀٽو"] (= s (.toLowerCase s) (.toUpperCase s)))
16:30 clojurebot: true
16:30 pdk: turkish has two separate letters for I
16:30 coopernurse: different subject - anyone out there running clojure on a VPS with limited ram?
16:30 pdk: one has a dot on both lower/upper cases, one has a dot on neither
16:31 amalloy: coopernurse: for given values of "limited"
16:31 coopernurse: I'm interested in how small you were able to get the JVM - if you tried..
16:31 I was playing with compojure on a rackspace cloud 512mb instance and it worked fine
16:31 jvm rss was about 125mb
16:31 with hello world loaded
16:32 I was curious if anyone has tried to get that smaller.. I had start/max heap set to 32mb
16:32 hiredman: on osx I can't seem to get the rss of the jvm below 80mb no matter what settings I use
16:32 coopernurse: hiredman: yeah, that's about what I've seen too
16:32 hiredman: just fiddling around with it
16:32 markoman: simple things come easily complicated in this digital world
16:33 TimMc: markoman: You're trying to make a program that can deal with any language. :-P
16:33 markoman: but hey thanks, I got something to chew
16:33 TimMc: Rather... it turns out that search engines are not language-agnostic.
16:34 markoman: nah, Im finding limits and making compromises then :)
16:35 TimMc: I wish I could share some of the crazy stuff we're dealing with in our own software, but I'm not really sure what's confidential and what isn't.
16:36 markoman: *promises not to copy and paste everything on twitter*
16:37 I get gray hair even with basic character encodings
16:37 amalloy: haha i'd like to see someone try to distribute their whole codebase in tweet-sized chunks
16:37 TimMc: public static void main(String[] args) { #loloutofspace
16:41 raek: markoman: ##(apply str (remove #(= (Character/getType %) Character/NON_SPACING_MARK) (java.text.Normalizer/normalize "åäö" java.text.Normalizer$Form/NFKD)))
16:41 sexpbot: ⟹ "aao"
16:44 markoman: hmh... on intelliJ repl it gives questionmarks, need to test lein repl
16:45 TimMc: markoman: Make sure to have rlwrap installed -- it handles Unicode printing.
16:46 raek: markoman: install rlwrap if you don't have it. jline, which is the default, is broken and does not have support for UTF-8
16:46 markoman: thats clojure lib, or lein extension or java lib?
16:46 raek: markoman: OS package
16:47 then the test to confirm everything works: ##(seq "åäö") ;should return *three* characters
16:47 sexpbot: ⟹ (\å \ä \ö)
16:49 markoman: ok, found from mac port
16:52 cemerick: hiredman, coopernurse: I'll bet that mapped byte buffers are being used for your bootclasspath, so a very large chunk of that 80MB is rt.jar and friends.
16:52 technomancy: so... how hard would it be to create a lazy seq from which items could be dropped without calculating them?
16:53 coopernurse: cemerick: interesting. I'll google "mapped byte buffers" and learn something
16:53 hiredman: technomancy: very
16:53 technomancy: I'm thinking if drop rebound some kind of var, the lazy seq construction could check it and simply skip the work where possible if it knew the results could be discarded.
16:53 hiredman: (defprotocol internal-drop ...)
16:54 technomancy: seems easier to just have a lazy-seq of delays
16:54 technomancy: hiredman: did we have this conversation already? I have a vague idea I once determined it'd be harder than it sounds, but I don't recall why.
16:55 certainly easier, but less elegant.
16:55 cemerick: coopernurse: Looking at URLClassLoader, it's loading class byte[]s from ByteBuffers, and there's a comment that makes me think they're direct 99% of the time (would certainly make sense if the jars are on disk).
16:55 hiredman: technomancy: yeah, I vaguely recall that, but don't recall specifics
16:57 markoman: hmh, i still get questionmarks after installing rlwrap and restarting apps
16:57 hiredman: ~google java file encoding
16:57 clojurebot: First, out of 323000 results is:
16:57 utf 8 - Setting the default Java character encoding? - Stack Overflow
16:57 http://
16:57 coopernurse: cemerick: ah. so does that mean I'm seeing disk blocks that are mmap'd in the RSS figure?
16:57 markoman: is there something else I need to include when repl starts up?
16:57 coopernurse: I have a tenuous grasp on how mmap works
16:59 cemerick: coopernurse: Perhaps. And, if those jars are being mmap'ed from Java, then the corresponding memory allocation happens outside of the heap.
16:59 Thus, the disconnect between -Xmx and such and rss.
16:59 coopernurse: cemerick: ah, cool. I always wondered why that difference was so huge
16:59 looks like -XX:MaxDirectMemorySize may be a way to tune that?
17:01 cemerick: coopernurse: I'll bet that's a hard limit that will cause an exception to be thrown when the classloader attempts to mmap the files, not prompt it to fall back to a non-mmap code path.
17:01 coopernurse: which option reference are you looking at? I don't see that one on http://
17:01 coopernurse: cemerick: interesting. and it appears the default is 64MB
17:02 cemerick: http://
17:02 cemerick: oops, that's jrockit..
17:04 cermerick: normal java -X doesn't show it.. oh well. but I suspect you've explained why you can't really get a JVM down below 100MB RSS in practice.
17:05 cemerick: coopernurse: there's a *lot* of options that aren't listed on the page I linked. There's a more comprehensive reference, I just can't find it at the moment.
17:06 markoman: when repl starts on intelliJ it says "/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home/bin/java -Dfile.encoding=UTF-8 " but still I get question marks when printing scandic letters
17:07 offby1: markoman: might have to do with your terminal program.
17:07 Or your choice of font.
17:09 coopernurse: cemerick: I see. my goal is to see how many JVMs you could reasonably fit in a given amount of RAM for small-ish webapps. My preference would be to use a separate JVM per app so I can restart them separately. I realize that's not Java's preference, but it would make the Compojure web apps more similar to Ruby/Python apps from an ops standpoint
17:10 cemerick: hot swapping the WAR files is another option, but my experience with that has been less than stellar (run out of PermGen due to classloader leaks, etc)
17:12 cemerick: coopernurse: undeploy/redeploy is the good middle ground; you could probably run a dozen or more small webapps in 128MB of heap, and deploy them independently.
17:12 coopernurse: cemerick: undeploy/redploy - is that a jetty feature?
17:13 cemerick: coopernurse: any (worthwhile) container supports undeploying an app while leaving the container and other apps undisturbed
17:13 jetty, tomcat, glassfish are all suitable
17:13 S11001001: except for the permgen thing
17:14 cemerick: turn on class GC and you should be fine
17:14 or, use a VM that doesn't have permgen (jrockit is now free)
17:16 coopernurse: I'm not saying you can have heroku-in-a-box on this route, but it certainly works for handfuls of apps. 1:1 app:container setups are never going to be efficient in any way (unless there's a magic config parameter that we've not discovered that will eliminate the rss issue).
17:17 coopernurse: cemerick: agreed. the JVM seems to favor long running processes. redeploying seems like a decent route -- I guess I've just been bit by that in the past. perhaps I should take a look at jrockit
17:18 cemerick: coopernurse: some containers (weblogic, IIRC?) have previously offered a "hot redeploy" option that, yes, have never worked as advertised. Doing discrete undeploy + deploy operations are a whole different (simpler) approach, though, and "just work".
17:19 markoman: yeah, on terminal i needed to do: export JAVA_OPTS="-Dfile.encoding=utf-8"
17:19 cemerick: Also, the shorter your uptime is, the less you'll get JITed code.
17:19 coopernurse: cemerick: interesting, I would have figured hot deploy does the same thing as undeploy/redeploy - or is the latter just not atomic, and therefore more reliable?
17:20 markoman: but intelliJ is harder to get functioning on repl, need to test it on clj files tomorrow, sometimes they still work, just repl not
17:21 thanks guys, more next time
17:22 cemerick: coopernurse: exactly, those are two operations that you run / script as part of your "redeploy" process. Things marketed as "hot redeploy" try to be tricky, probably attempting to do things like what JavaRebel et al. do.
17:23 (and, JavaRebel works well when it can — but there are always changes that cause it to fall over and force a full rebuild / deploy cycle)
18:34 ssideris: hello, has anyone seen the "make-dependency does not exist" error with leiningen native-deps 1.0.5?
18:35 it seems that native-deps is a bit sensitive to the version of leiningen or something...
18:36 technomancy: ssideris: what lein version?
18:36 ssideris: 1.5.0
18:36 (on windows if that's relevant)
18:36 technomancy: odd; that should work.
18:36 if you use lein 1.6.0-SNAPSHOT you don't need a plugin for native-deps
18:37 ssideris: oh, ok let me give it a go
18:37 thanks
18:37 and thanks for leiningen btw :-)
18:37 technomancy: sure =)
18:39 having make-dependency move was a long time ago though; 1.0.5 shouldn't be affected by that
18:43 ssideris: sorry just a clarification: does that mean that lein 1.6.0-SNAPSHOT does not differentiate between native and non-native deps?
18:43 do I put everything under :dependencies?
18:44 technomancy: ssideris: not yet; so far it works the same as the plugin
18:45 except you don't need a separate "lein native-deps" step
18:45 but you do need them in :native-deps in project.clj
18:45 hopefully by the time it's released you can put them in :dependencies though.
18:46 ssideris: hm, that's odd, there must be something really weird with my setup because it didn't seem to pick up :native-dependencies\
18:46 it's being ignored...
18:47 technomancy: hmm... it's very new code; not well-tested yet. could you open an issue?
18:47 ssideris: ok
18:49 technomancy: what's the dependency?
18:50 ssideris: [penumbra/lwjgl "2.4.2"]
18:54 technomancy: https://
18:54 opened!
18:54 thanks
19:47 technomancy: ssideris: if you can take a look it would help to get it fixed faster; I don't have any experience with native deps and have a few search features I need to finish first
19:47 but I'll get to it before the release if not
19:50 ssideris: technomancy: erm, wouldn't know where to start, but if you can direct me a bit I would be happy to give it a go
19:51 technomancy: here's the function in question: https://
19:51 pretty much taken verbatim from the plugin
19:53 ssideris: ok, I'll see if I find anything, but don't put too much faith in my clojure abilities :-)
21:16 mec: Is there anywhere that has the syntax for literal defrecords in alpha 7?
21:17 amalloy: mec: it's just #user.Foo[1 2] or #user.Foo{:a 1 :b 2}
21:18 ataggart: http://
21:20 and it's not just for records, but a generic constructor literal.
21:31 mec: ataggart: so its good for any java class?
21:38 ataggart: yes, e.g. #java.util.Date[12345]
21:38 bear in mind that this is really intended for print-dup, not something you should be typing out.
22:12 WuHoUnited: hello clojurebot
22:14 (doc clojurebot)
22:14 clojurebot: It's greek to me.
22:14 WuHoUnited: (source clojurebot)
22:15 amalloy: clojurebot: source?
22:15 clojurebot: source is http://
22:15 amalloy: WuHoUnited: ^
22:15 WuHoUnited: ^_^
22:15 amalloy: $whatis source
22:15 sexpbot: source is http://
22:15 amalloy: (also that if you're interested)