[10:56] <poolie> awesome
[10:56] <poolie> markup
[12:37] <rick_h_> sweet poolie
[12:37] <poolie> hey
[12:37] <poolie> thanks
[12:37] <poolie> jfdi ftw :)
[12:38] <rick_h_> I like the markdown since there was the JS renderer for it
[12:38] <rick_h_> client side instant preview ftw
[12:38] <rick_h_> lol, often the only way to go jfdi
[12:43] <poolie> i think i will stop for tonight soon
[12:43] <poolie> but it does render some stuff
[12:43] <poolie> recognizing our existing patterns will be a little bit of work
[12:43] <poolie> but not too hard
[12:43] <poolie> just a 40-line regexp
[19:08] <lifeless> erm, argh:
[19:08] <lifeless> The following packages have unmet dependencies:
[19:08] <lifeless>   launchpad-dependencies: Depends: python-apt (>= 0.7.94.2ubuntu6.4) but 0.7.94.2ubuntu6 is to be installed
[19:10] <lifeless> E: Version '0.7.94.2ubuntu6.4' for 'python-apt' was not found
[19:21] <mtaylor> lifeless: fail
[20:24] <lifeless> mtaylor: not exactly success, no
[21:19] <lifeless> mwhudson: thanks :P
[21:31] <mtaylor> lifeless: I try to be helpful
[21:35] <mwhudson> lifeless: ?
[21:37] <lifeless> mwhudson: recordSuccess
[21:37] <mwhudson> lifeless: ah right
[21:38] <mwhudson> lifeless: it occurred to me that in some ways codehosting has some of the properties of a microservice already
[21:38] <lifeless> mwhudson: it does
[21:38] <lifeless> mwhudson: its ripe for split out - see for instance the thing thomi and thumper did
[21:38] <lifeless> which includes a split out, but not a migration to the split out code
[21:39] <thumper> also a simplification as we didn't care about sftp
[21:39] <mwhudson> lifeless: well, in as much as "it does not talk to the db directly"
[21:39] <mwhudson> lifeless: yeah, split should be relatively straightforward
[21:40] <lifeless> mwhudson: yes, though that is merely one component of the microservice criteria :)
[22:17] <wgrant> Wasn't it originally more of a microservice?
[22:17] <wgrant> Talking to XML-RPC rather than the DB?
[22:21] <lifeless> wgrant: codehosting still talks xmlrpc, and yes it started that way
[22:21] <lifeless> wgrant: you may be thinking f the codebrowsing url mapper
[22:21] <lifeless> which was an API but was moved to DB
[22:24] <wgrant> I thought codehosting talked to the DB for some things now.
[22:24] <wgrant> I forget.
[22:25] <wgrant> lifeless: rmadison is your friend
[22:27] <lifeless> codehosting itself? no
[22:27] <lifeless> we have db access on the machine for oops prune
[22:32] <wgrant> lifeless: Speaking of oops-prune...
[22:42] <lifeless> ggrrr
[22:42] <lifeless> dpkg: error processing /var/cache/apt/archives/mountall_2.15.3_i386.deb (--unpack):
[22:42] <lifeless>  unable to make backup link of `./lib/init/fstab' before installing new version: Invalid cross-device link
[22:42]  * lifeless embugginates on lxc
[22:43] <wgrant> Yes :/
[22:43] <wgrant> I can't remember how I fixed that.
[22:43] <wgrant> Google might know.
[22:43] <wgrant> LXC is extremely fragile.
[22:43] <wgrant> And buggy as hell.
[22:44] <wgrant> It takes over my TTYs every couple of weeks :(
[22:44] <StevenK> And we want to run our testsuite, which is extremly fragile, and buggy as hell on top of it ...
[22:44] <StevenK> Sounds like win.
[22:44] <lifeless> fortunately the bugs don't intersect much
[22:44] <wgrant> Yeah
[22:44] <wgrant> This is probably the sort of thing that LXC is suitable for.
[22:44] <wgrant> ie. no security requirements
[22:45] <lifeless> ubuntu-cloud folk want to run openstack test runs with openstack deployed on lxc
[23:05] <lifeless> poolie: on markdown, yes, we do need the nofollow rule.
[23:06] <poolie> k
[23:07] <poolie> that should be easy enough to add
[23:07] <poolie> do you need person descriptions treated differently to all other text?
[23:07] <lifeless> per curtis' rules?
[23:07] <lifeless> uhm
[23:07] <mwhudson> wgrant: 'codehosting' in general talks to the db a bit, eg for jobs
[23:08] <lifeless> Arguably no, but perhaps we should treat -all- comments by low karma folk as not-capable-of-making-links.
[23:08] <mwhudson> wgrant: but the ssh server daemons & bzr lp-serve processes do not
[23:08] <wgrant> mwhudson: Ah, right.
[23:08] <lifeless> put another way, I htink the no-links-for-newbies is a good thing, and the limitation of only doing that on the persons homepage is what should change
[23:09] <lifeless> however, I'd suggest getting a definite from curtis - I don't have a strong enough opinion any which way to override his preference, which appeared to be 'do not link low karma person homepages'
[23:10] <poolie> it would be interesting to run them all through eg http://www.surbl.org/
[23:10] <poolie> i think 'not enough reputation to make links' generically could be good
[23:10] <poolie> mm
[23:10] <poolie> more so if you only earn reputation by doing useful things
[23:19] <huwshimi> Again, I'm guessing this test failure is nothing to do with me? http://paste.ubuntu.com/744818/
[23:21] <mwhudson> huwshimi: allenap fixed a bunch of those bugs recently i think?
[23:21] <mwhudson> as in, over the weekend
[23:22] <huwshimi> Hmm.. guess I should just land it then
[23:22] <wgrant> huwshimi: I believe that was fixed on Friday.
[23:22] <wgrant> So yes, land away.
[23:23] <huwshimi> mwhudson, wgrant: thans
[23:23] <huwshimi> *thanks
[23:23] <poolie> huwshimi, hi
[23:23] <huwshimi> poolie: Morning
[23:23] <poolie> so, markdown! :)
[23:24] <huwshimi> poolie: Markdown, indeed
[23:24] <wgrant> + Alternative fix from #postgresql is to change the sort to 'datecreated +
[23:24] <wgrant> + interval '0'' which runs this query otherwise unmodified in 300ms on
[23:24] <wgrant> lifeless: wtf?
[23:24] <wgrant> + qastaging.
[23:25] <lifeless> wgrant: there are 12K bugs found by the bug supervisor rules
[23:25] <lifeless> wgrant: grabbing those and sorting separate -> fast
[23:26] <lifeless> wgrant: walking 150K tasks one by one consulting to see if the task is in that 12K -> slow, as there are only 4 bugs matching the full filters
[23:26] <wgrant> Ah
[23:26] <poolie> can i send https://code.launchpad.net/~mbp/launchpad/ec2-region/+merge/82487
[23:27] <lifeless> +interval '0' makes the sort computed and not satisfiable by index. Today anyhow.
[23:27] <poolie> i have used it locally for several tests and it seems pretty solid
[23:27] <poolie> or actually i suppose https://code.launchpad.net/~mbp/launchpad/ec2-kill/+merge/82631 which incorporates it
[23:38] <mwhudson> poolie: i can't see why you shouldn't land the ec2 changes
[23:38] <poolie> yup
[23:38] <mwhudson> poolie: monday am is probably a good as time as any to break it, and i can't think of anything more you can do that would make landing safer really
[23:38] <poolie> exactly
[23:39] <mwhudson> now someone finish off parallelization and let's have a high cpu quadruple extra large instance or whatever it's called at the problem :-)
[23:40] <poolie> yeah
[23:41] <poolie> you're right the times are extremely variable so it's hard to say if tmpfs is faster
[23:41] <poolie> just a funny coincidence i got plausibly stable times
[23:41] <wgrant> I haven't tested LP on tmpfs on the cloud before.
[23:41] <poolie> perhaps 3h30m is the "if nothing goes wrong" time
[23:41] <wgrant> Only locally.
[23:41] <poolie> maybe if there's no contention
[23:42] <mwhudson> poolie: yeah, i guess it depends how much disk io other vms on the physical box are doing blah blah blah
[23:42] <wgrant> It's critical when doing it parallel on a physical machine.
[23:42] <poolie> yes
[23:42] <mwhudson> i wonder if there is any way we could have ec2 test report run time to some central location, trends and averages should still be meaningful
[23:42] <poolie> i have heard ec2 io can bog down
[23:42] <poolie> mm
[23:42] <poolie> would be good
[23:42] <poolie> even just mail to a mailbox
[23:43] <poolie> since it already knows how to send mail
[23:43] <mwhudson> poolie: apparently EBS is pretty slow
[23:43] <mwhudson> (if you have a database on EBS SOP is to raid 0 stripe it, apparently)
[23:43] <mwhudson> but can believe local io is slow too
[23:43] <wgrant> A DB on EBS?
[23:43] <wgrant> Sounds somewhat insane.
[23:44] <mwhudson> wgrant: it's what simpledb probably does, i think?
[23:44] <wgrant> No clue.
[23:44] <mwhudson> my ec2 neurons are fading
[23:44] <lifeless> well
[23:44] <lifeless> simpledb is a distributed hash right ?
[23:45] <mwhudson> maybe i mean the other thign :)
[23:45] <lifeless> I am pretty sure simpledb is dynamo
[23:45] <mwhudson> rds?
[23:45] <mwhudson> yes, rds
[23:45] <lifeless> yes, rds is sql dbs
[23:46] <poolie> wgrant, uh, we run a db on ebs :)
[23:47] <poolie> or, we did prior to my branch landing
[23:47] <wgrant> poolie: Do we?
[23:47] <mwhudson> poolie: no we didn't
[23:47] <wgrant> I thought we ran a test DB on instance storage.
[23:47] <wgrant> Not a real DB at all.
[23:47] <poolie> oh, true, sorry
[23:47] <wgrant> And certainly not a real DB on EBS.
[23:48] <poolie> well, my point is, what is "real"
[23:49] <poolie> you could have large but fairly disposable dbs
[23:50] <huwshimi> I have changed the way tags are ordered in the portlet. This breaks this test: http://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel/view/head:/lib/lp/bugs/stories/bug-tags/xx-official-bug-tags.txt in this way: http://paste.ubuntu.com/744836/
[23:50] <huwshimi> Any suggestions about how to fix that doctest in a way which isn't just re-ordering those tags
[23:50] <poolie> i'm really keen to see that land
[23:51] <huwshimi> I don't want the output in the doctest to just be a jumbled mess, or at least that seems like a bad idea to me
[23:51] <poolie> "then why are you using doctests?" :)
[23:52] <mwhudson> delete the test?
[23:52] <huwshimi> poolie: Thanks, very helpful :P
[23:52] <poolie> rewrite it as python
[23:52] <poolie> is the test meant to be asserting the official tags come first or something?
[23:53] <huwshimi> poolie: I don't know!
[23:53] <mwhudson> huwshimi: semi seriously, the question you have to answer here is "what is the intent of this test"
[23:53] <mwhudson> well, entirely seriously now
[23:53] <mwhudson> i was being semi serious before
[23:54] <mwhudson> it seems to be testing that all the official tags are displayed, but not all of the unofficial ones
[23:54] <StevenK> -        self.assertThat(recorder, HasQueryCount(LessThan(100)))
[23:54] <StevenK> +        self.assertThat(recorder, HasQueryCount(LessThan(101)))
[23:54] <StevenK> :-(
[23:54] <huwshimi> mwhudson: I can't even make sense of the original comments/output
[23:54] <mwhudson> if that's still true, then maybe printing out the number of tags that .startswith('official') and .startswith('unofficial') is a good fix?
[23:55] <poolie> if the test is not adding value i think you should bias towards deleting it
[23:56] <mwhudson> huwshimi: it doesn't seem /that/ cryptic to me :)
[23:56] <mwhudson> but +1 poolie
[23:56] <huwshimi> mwhudson: "The 10 unofficial tags will be the 10 most popular tags"
[23:56] <mwhudson> huwshimi: ah yeah, that bit is harder i guess
[23:57] <poolie> write a python test that clearly says what you want the behaviour to be
[23:57] <huwshimi> mwhudson: Oh I see, they are more popular, so 10 and 11 are pushed out
[23:57] <lifeless> StevenK: what is increasing the query count?
[23:57] <mwhudson> huwshimi: right
[23:57] <mwhudson> huwshimi: it's a pretty terrible test :-)