| wgrant | http://highscalability.com/blog/2012/3/26/7-years-of-youtube-scalability-lessons-in-30-minutes.html | 00:28 |
|---|---|---|
| wgrant | "They wrote their own BSON implementation which is 10-15 time faster than the one you can download." | 00:29 |
| StevenK | Nice of them to release it. | 00:29 |
| lifeless | wgrant: nice post | 00:47 |
| wgrant | Indeed. | 00:49 |
| wgrant | wallyworld_: Your rev breaks 3 JS tests | 03:04 |
| wgrant | They time out. | 03:04 |
| wallyworld_ | not locally, i checked | 03:04 |
| wallyworld_ | and the ones which do do not use any of my code | 03:04 |
| wallyworld_ | well, they might use my code indirectly | 03:05 |
| wgrant | wallyworld_: I can reproduce the timeout new on 15018 locally. | 03:05 |
| wallyworld_ | so i wonder why they pass locally | 03:05 |
| wallyworld_ | for me | 03:06 |
| wallyworld_ | let me check | 03:06 |
| wgrant | Different html5browser version? | 03:06 |
| wgrant | Anyway, reverting. | 03:06 |
| wallyworld_ | could be, though i'm fairly up to date | 03:06 |
| wgrant | Exactly | 03:06 |
| wgrant | That's the problem. | 03:06 |
| wgrant | You're running precise's. | 03:06 |
| StevenK | Ahhh, I love fallout between lucid and precise. | 03:09 |
| wallyworld_ | i don't :-( | 03:10 |
| wallyworld_ | hopefully this will go away soon | 03:10 |
| wgrant | wallyworld: http://paste.ubuntu.com/901543/ fixes the three broken tests | 03:20 |
| wgrant | But probably breaks others. | 03:21 |
| lifeless | well, there is our space usage: | 03:21 |
| lifeless | * 54474 Exceptions | 03:21 |
| wgrant | Wrong | 03:21 |
| lifeless | FDT | 03:21 |
| wgrant | That's just normal fastdowntime | 03:21 |
| wgrant | The space usage is the 25000 OOPSes per day from qastaging and staging | 03:21 |
| wgrant | because gandwana doesn't have a memcached | 03:21 |
| lifeless | ah | 03:22 |
| wallyworld_ | wgrant: hmm. i don't see why .prototype is needed if i am using yui inheritance | 03:22 |
| lifeless | thanks for tracking that down, are ops on it? | 03:22 |
| wgrant | lifeless: The immediate problem will go away once I unbreak the build. | 03:22 |
| wgrant | lifeless: Since the garbo job that's whinging about memcached is about to be deleted. | 03:22 |
| wgrant | But did ping webops about memcached. | 03:22 |
| spm | oops. sorry, one sec | 03:23 |
| lifeless | interesting, adsl modem isn't dropping link anymore, but apparently having routing issues nonetheles | 04:09 |
| lifeless | s | 04:09 |
| hloeung | tried moving to the motherland Australia? ;-) | 04:11 |
| lifeless | been there done that | 04:12 |
| StevenK | lifeless might have even got the t-shirt. | 04:16 |
| lifeless | 'last ride on the big dipper' | 04:17 |
| * StevenK missed out on the big dipper | 04:17 | |
| === almaisan-away is now known as al-maisan | ||
| === al-maisan is now known as almaisan-away | ||
| lifeless | wgrant: whats up with deleting the decoratedresultset doctest thunk | 06:50 |
| wgrant | lifeless: It was already being run by test_doc | 06:53 |
| wgrant | The registration got duplicated during the apocalypse. | 06:53 |
| lifeless | kk | 06:53 |
| lifeless | http://attractivechaos.github.com/HN-prog-lang-poll.png | 07:19 |
| lifeless | flacoste: http://ceklog.kindel.com/2011/08/06/90-of-the-decisions-you-make-dont-matter/ | 07:27 |
| adeuring | good morning | 07:50 |
| === almaisan-away is now known as al-maisan | ||
| === al-maisan is now known as almaisan-away | ||
| === almaisan-away is now known as al-maisan | ||
| lifeless | stub: lib/lp/services/mail/sendmail.py line 9. Possibly the oldest TODO in the code base. | 09:03 |
| stub | \o/ | 09:04 |
| stub | From before we had a working bug tracker to track that sort of thing :) | 09:04 |
| cjwatson | So I have a trivial patch for bug 966092 that doesn't increase LoC in itself (and is arguably a slight simplification). Before the new maintenance policy I'd have added a 20-line test for it without worrying. What should I do now? | 10:40 |
| cjwatson | I guess I could try to find some bits of those tests to compactify or something | 10:42 |
| cjwatson | or I could not add a test on the (dubious) grounds that it's currently untested anyway | 10:42 |
| cjwatson | I don't know what people would prefer | 10:43 |
| lifeless | you could reduce duplication/waste in the tests, you could reduce it elsewhere to make room for your test | 10:44 |
| cjwatson | that general approach then? ok | 10:47 |
| cjwatson | but in particular I guess you'd prefer not to see a branch where I hadn't written a test in order to comply with the maintenance policy? | 10:47 |
| lifeless | the working theory is that we've had little/no pressure for compact code, and we now have one. | 10:48 |
| lifeless | Right, that would be pathological, and reviewers would reject it anyway : if it wasn't ok to do that before, its not now. | 10:48 |
| cjwatson | ok | 10:48 |
| lifeless | An alternate strategy is to ask someone that has been deleting lots of code for some credit; we don't really track per-person or anything, but something of a barter system has arisen | 10:49 |
| cjwatson | yeah, I should have saved my big deletions for after this policy was instituted ;-) | 10:50 |
| lifeless | Push comes to shove, we'd rather have simpler-and-correct code in than not. | 10:50 |
| cjwatson | I'll figure something out. Thanks | 10:50 |
| lifeless | the LoC thing is documented to be a proxy metric. If you do decide you need a waiver for more LoC, flacoste or I can provide one. | 10:50 |
| wgrant | cjwatson: Ah. I've been considering fixing that for a while, but thought there must be some catch that had kept that horrid step in place for 6 years. | 11:06 |
| cjwatson | wgrant: yes, it's called "inertia" | 11:13 |
| wgrant | Heh | 11:13 |
| === matsubara-afk is now known as matsubara | ||
| salgado | benji, hi there. so, I've done the changes that lifeless suggested to that branch (https://code.launchpad.net/~linaro-infrastructure/launchpad/team-engineering-view/+merge/99342). would you have some time for another look? :) | 12:42 |
| benji | salgado: sure, I can look at it now | 12:42 |
| benji | salgado: looks great | 12:50 |
| salgado | benji, cool, thanks! would you mind ec2-landing it again? :) | 12:51 |
| benji | salgado: sure | 12:52 |
| nigelb | benji: Great post about tests! I didn't realize LP tests could be run in 45 minutes now. That is cool. | 12:57 |
| benji | salgado: the ec2 wheels are turning | 12:58 |
| salgado | benji, great, thanks! | 12:58 |
| benji | nigelb: hopefully we can pull together a few more loose ends and get it deployed so buildbot will be a bit more responsive than it is now | 12:59 |
| nigelb | \o/ | 12:59 |
| deryck | Morning, all. | 13:07 |
| deryck | rick_h, ping for standup | 13:31 |
| deryck | rick_h, so we need a bug describing the issue that led to: https://code.launchpad.net/~rharding/convoy/add_listmod_header/+merge/94437 | 14:13 |
| deryck | rick_h, it's implied in the MP, but a proper bug describing the problem would be nice. ;) | 14:13 |
| rick_h | deryck: ok | 14:13 |
| deryck | rick_h, and to clarify, the reason U1 and Landscape didn't get hit by bug 966220 is because they run convoy in tree, rather than as a separate service? | 14:15 |
| _mup_ | Bug #966220: using the path info to route breaks in various deployment cases <Convoy:New> < https://launchpad.net/bugs/966220 > | 14:15 |
| rick_h | deryck: well it's not clear they're running the updated convoy at all. | 14:16 |
| deryck | rick_h, ah. | 14:16 |
| rick_h | deryck: but it's that their not doing any routing, so as it works now, we couldn't get it to work either | 14:16 |
| deryck | rick_h, what do you mean by routing? | 14:16 |
| rick_h | deryck: so in the way it broke works toward how U1 and landscape work | 14:16 |
| deryck | ah | 14:16 |
| rick_h | so /combo/12345 would get eaten in apache to /12345 which convoy then ate the first part of and so no path hint to do extra path finding on disk | 14:17 |
| rick_h | deryck: so what we need is /combo/12345 to make it to convoy as /12345 which tells convoy to look in /12345/build/js/... | 14:17 |
| deryck | rick_h, so they're not using the new feature of revnos being significant? right. Their use only cares about the constructed query? | 14:18 |
| rick_h | deryck: right, they only care about the ?yui-min... | 14:18 |
| deryck | rick_h, and the current version of convoy is backwards compatible to work either way, i.e. even if they did upgrade, they wouldn't notice because of using convoy the way they do. is that right? | 14:19 |
| rick_h | rigght | 14:20 |
| rick_h | deryck: correct, so in order to really make things work 'as they should' would require being backwards incompatible | 14:20 |
| deryck | rick_h, so why don't we get hit by this in local dev? | 14:22 |
| rick_h | deryck: because we don't attempt to use the revno locally | 14:22 |
| rick_h | so path is just /combo? | 14:22 |
| rick_h | which gets cut to just / | 14:22 |
| rick_h | and that doesn't effect our routing to the build location on disk at all | 14:22 |
| deryck | rick_h, and by routing, you mean a script alias directive? We're using that, not a proxy pass directive, right? | 14:30 |
| rick_h | so currently it looks at what the server puts into thw wsgi PATH_INFO env variable | 14:31 |
| rick_h | deryck: right, this is all seperate from any proxy pass/etc | 14:31 |
| deryck | rick_h, i.e. it's the script alias directive that consumes the +combo fragment and hands off the rest to mod_wsgi? | 14:31 |
| deryck | rick_h, sorry if I'm being overly specific here. just trying to think it through. | 14:32 |
| rick_h | deryck: think so, but not 100% sure. let me pull up the apache config | 14:32 |
| rick_h | right, the proxy pass says NOT to proxy /combo urls, but then it hits the alias, which passes the request to convoy and the url is different in there | 14:33 |
| deryck | rick_h, ok, I see now. | 14:38 |
| * deryck is playing locally | 14:38 | |
| rick_h | deryck: yea, I'm reading the pep docs to see just what PATH_INFO needs/should be | 14:38 |
| deryck | rick_h, this feels to me like there ought to be some apache magic we can do to make sure the script gets passed everything it needs. | 14:39 |
| === al-maisan is now known as almaisan-away | ||
| rick_h | deryck: right so in testing out a different server there's a RAW_URI environment variable I could use, but that's not available in pure wsgiref | 14:41 |
| rick_h | so it blows up in tests/etc | 14:41 |
| rick_h | but really, I think the magic should be on the other end. If you want to ignore the url and not deal with extra paths, you should rewrite your urls to get rid of it | 14:41 |
| rick_h | so if you want a magic ignored /12345 it should be on you to rewrite that to / | 14:41 |
| rick_h | the problem is we're trying it do it backwards and it's causing things to break down | 14:42 |
| deryck | rick_h, does convoy log anywhere locally? | 14:42 |
| rick_h | deryck: no, it doesn't | 14:43 |
| deryck | rick_h, how can I tell what URL it's getting handed? | 14:43 |
| abentley | adeuring: Could you please review https://code.launchpad.net/~abentley/launchpad/run-via-celery/+merge/99099 ? | 14:43 |
| adeuring | abentley: sure | 14:44 |
| rick_h | :) playing with the source to print/log/pdb | 14:44 |
| rick_h | deryck: ^ | 14:44 |
| deryck | ouch :) | 14:44 |
| rick_h | deryck: yea, it's not the best thing ever writte, but that's why I think it's been wanting to move on | 14:44 |
| rick_h | deryck: so in looking, if we wanted to keep doing the magic I've been doing, I'd need to combine the SCRIPT_NAME and PATH_INFO to keep the full url | 14:52 |
| rick_h | and then keep doign the regex/cleaning I do now | 14:52 |
| deryck | rick_h, yeah, exactly. I was trying to see if mod_wsgi had some directive to include script_name in path_info. | 14:53 |
| rick_h | but I think that would then still break things for the other users. | 14:53 |
| rick_h | deryck: I don't think so, I don't think it's meant to be done. They have clear goals for each part of that | 14:54 |
| deryck | rick_h, right. so at the least, if convoy.wsgi gets something like environ['PATH_INFO'] = environ['SCRIPT_NAME'] + environ['PATH_INFO'] that should work, right? | 14:57 |
| rick_h | deryck: right, so if I used that, it should work I think, but when I look at that, I think it'd still end up backwards incompat with U1/landscape. | 14:58 |
| deryck | rick_h, how so? if they're dropping the first part of path_info anyway? | 14:59 |
| rick_h | deryck: looking, sec | 15:03 |
| deryck | rick_h, np. | 15:04 |
| rick_h | deryck: can we jump on a call? | 15:06 |
| deryck | rick_h, was just about to ask you the same. spin up a hang out and give me 2 minutes to warm coffee. | 15:06 |
| rick_h | https://plus.google.com/hangouts/extras/talk.google.com/orange-one-on-one | 15:06 |
| rick_h | deryck: ^^ | 15:06 |
| jcsackett | can someone point me to docs about landing lazr stuff? particularly lazr.restfulclient? | 15:14 |
| adeuring | abentley: I'm wondering, if line 34 of the diff might result in a race condition: You call CeleryRunJob.delay() for an "absolutely fresh" job: What happens if celeryd tries to execute the job before txn.commit() is called in the transaction that creates the job? | 15:19 |
| abentley | adeuring: I think you're right. Is there a way to hook in so the next txn.commit executes CRD.delay()? | 15:21 |
| adeuring | abentley: I have no idea. But if there no such mechanism, we could also let celeryd wait for a few seconds, I assume | 15:22 |
| abentley | adeuring: We could, but we would lose the responsiveness we gain by using Celery. | 15:23 |
| adeuring | abentley: right... And a hook for "after txn.commit() work" would anyway be much cleanr | 15:24 |
| abentley | adeuring: transaction appears to support current.addAfterCommitHook | 15:26 |
| adeuring | abentley: sounds good | 15:26 |
| deryck | jcsackett, what kind of docs do you need? I've always just branched, coded, put up for mp,done. or do you mean how to update versions in lp after that? | 15:26 |
| jcsackett | deryck: nothing happens post MP? just mark approved and it lands? | 15:27 |
| deryck | rick_h, also, I'm going to drag that card in progress for you on the board. | 15:27 |
| rick_h | deryck: thanks | 15:27 |
| * jcsackett hasn't landed lazr.restful stuff before | 15:27 | |
| deryck | jcsackett, same story as us. submit to pqm like lp. | 15:28 |
| jcsackett | deryck: ah. | 15:28 |
| deryck | jcsackett, then you need to make a tarball and added to download-cache and update versions.cfg to use the new version. for us. not sure about packaging releases if you need to do that. | 15:28 |
| adeuring | abentley: in class UniversalJobSource, you override the dbuser. That sounds fine for branch management job, but it will not be useful for example in translation imports. | 15:32 |
| jcsackett | deryck: thanks. | 15:32 |
| deryck | jcsackett, np, man! | 15:33 |
| abentley | adeuring: Indeed. I'm not worried yet, since BranchScanJob is the only supported job type in that branch. | 15:34 |
| adeuring | abentley: well, ok, let's defer that to another branch | 15:34 |
| abentley | adeuring: I want to talk to lifeless about using launchpad_main for all Jobs, because the alternative is pretty messy. | 15:34 |
| adeuring | ok | 15:35 |
| abentley | adeuring: i.e. you have to switch DB user after you've loaded the job, and then you have to re-load the job so it uses the new db user. | 15:35 |
| adeuring | right | 15:36 |
| abentley | adeuring: skip_celery should be True only in tests. | 15:52 |
| adeuring | abentley: yes, that's how I understand it. Just worth a short explanation, Ithink ;) | 15:52 |
| abentley | adeuring: Ah. Your review said "...skip_celery should be False only in tests" | 15:53 |
| deryck | abentley, adeuring, rick_h -- forgot to remind in standup, self reviews and peers selected due tomorrow. | 15:53 |
| adeuring | abentley: gah, I like to mix up reft and light | 15:54 |
| deryck | rick_h, you and I probably need to chat about this later, since you haven't done it yet. | 15:54 |
| deryck | but for now, it's lunch and work out time ;) | 15:54 |
| === deryck is now known as deryck[lunch] | ||
| === salgado is now known as salgado-lunch | ||
| === salgado-lunch is now known as salgado | ||
| === matsubara is now known as matsubara-lunch | ||
| === deryck[lunch] is now known as deryck | ||
| rick_h | deryck: convoy MP approved and pushed so hopefully this helps the deploy/test crew. | 17:26 |
| rick_h | deryck: let me know when you want to chat re review stuff. I filled out the one, but looks like we need to chat re goals | 17:26 |
| deryck | rick_h, yeah. let's hangout once more. | 17:31 |
| deryck | rick_h, https://plus.google.com/hangouts/extras/talk.google.com/orange-dudes | 17:32 |
| salgado | benji, did you get any failure/success from that ec2 land you did for me? it should be done by now, right? | 19:27 |
| benji | salgado: I got a failure. I thought that a message would be sent to you too. If not, I will forward it. | 19:27 |
| salgado | benji, no, I didn't get it. probably because is not owned by myself but by a team | 19:28 |
| benji | ah | 19:28 |
| salgado | the branch, I mean | 19:28 |
| benji | salgado: email forwarded | 19:29 |
| salgado | thanks benji | 19:29 |
| lifeless | morning | 19:29 |
| benji | np | 19:29 |
| salgado | benji, looks like I broke it when I replaced the Join+Coalesce on Person. the filter I wrote didn't have the exact same semantics of coalesce | 19:47 |
| salgado | benji, I've fixed it and pushed the change: http://paste.ubuntu.com/902724/ | 19:47 |
| lifeless | salgado: sounds like fallout from my suggestion... sorry@! | 19:48 |
| benji | salgado: cool, I'll warm up the ec2 machinery | 19:49 |
| salgado | lifeless, yeah, I shouldn't trust you when it comes to coalesce! (everything else I think I can, though ;) | 19:49 |
| salgado | thanks again, benji | 19:49 |
| lifeless | salgado: did it bring an unintended workitem,.. from a spec that the spec assignee was a participant | 19:49 |
| salgado | lifeless, yep. luckily my tests caught it. if only I hadn't forgotten to run my tests after that one final change, I'd have found this out earlier | 19:50 |
| lifeless | cool | 19:57 |
| lifeless | did you update to use the public bugtaskset.search API? | 19:57 |
| salgado | yes | 19:57 |
| lifeless | great | 19:57 |
| lifeless | sorry that its not clear that its private | 19:57 |
| lifeless | I'll nag wgrant (who extracted it for code hygiene only) to make that really clear | 19:58 |
| lifeless | (that the other function is private I mean) | 19:58 |
| StevenK | sinzui, wallyworld_, wgrant: http://pastebin.ubuntu.com/902944/ | 22:06 |
| sinzui | wallyworld_, wgrant, StevenK: http://people.canonical.com/~curtis/audit/ | 22:57 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!
