/srv/irclogs.ubuntu.com/2012/10/30/#launchpad-dev.txt

StevenKPurple buildbot00:20
wgrantException?00:21
wgrantYeah00:21
wgrantsubunit corruption00:21
StevenKwgrant, wallyworld_: http://instagram.com/p/RYnio1s6gf/00:25
wallyworld_yeah, saw that. i don't like instagram00:26
wallyworld_i can't see what all the fuss is about00:26
wallyworld_why ruin perfectly good photos with crappy filters00:26
StevenKwallyworld_: Yes, but you don't like anything that's social. And I'm getting off your lawn now.00:27
wallyworld_how is ruining a good photo being social?00:27
wgrantWe may have a dogfood DB in about 20 minutes, finally...00:27
StevenKFacebook bought out Instagram, which I thought was your issue.00:27
StevenKHowever, I was talking about the content, not the platform hosting the photo. :-)00:27
wgrantThe issue with Instagram is that it's the worst idea ever00:28
wgrant*and* it's "social"00:28
StevenKNote to self: Do not share hurricane photos with workmates.00:28
wallyworld_StevenK: sorry, i saw instagram and my radar went off00:29
wallyworld_StevenK: did you see the one of the crane?00:29
StevenKwallyworld_: Nope00:29
wallyworld_StevenK: it was on this morning's news - a huge apartment building, still under construction - the crane is folded in half and dangling in the wind00:30
StevenKhttp://www.smh.com.au/photogallery/environment/weather/hurricane-sandy-strikes-us-coast-20121030-28g5z.html?selectedImage=300:30
StevenKThat one?00:30
wallyworld_yeah00:30
StevenK... holy ...00:31
wallyworld_the latest pictures have it swinging in the "breeze"00:31
wallyworld_those apartments are selling for $90M00:31
enginespotHi everyone , I get this channel from here https://help.launchpad.net/01:50
enginespotcurrently I have a question , I want to get people list of launchpadlib01:51
enginespotbut I only get 50 persons01:51
enginespotso I do not know how to to01:51
wgrantStevenK: DF is back, and timing out merrily02:07
wallyworldwgrant: what's the difference between spr.upload_archive and spph.archive? are these related?02:44
StevenKLoosely02:45
StevenKspph.archive == Archive it is published in, spr.upload_archive == Archive it was first published in, so they can be different.02:45
wallyworldright, ok. thanks02:46
wgrants/first published in/first uploaded to or seen in/02:46
wgrantIt wasn't necessarily ever published there.02:46
wallyworldroo bad spph.archive is not denormalised ontop spr02:46
wallyworldtoo02:46
wgrantwallyworld: That doesn't make sense02:46
StevenKIt can't be.02:46
wgrantSince an SPR can be published in an unbounded number of different archives02:46
StevenKOr even different series in the same archive02:46
wallyworldok02:46
enginespotwhen I use the method PersonSet.find()02:54
enginespotit told me AttributeError: type object 'PersonSet' has no attribute 'find'02:54
StevenKwallyworld: http://www.smh.com.au/photogallery/environment/weather/hurricane-sandy-strikes-us-coast-20121030-28g5z.html?selectedImage=102:58
wallyworldwgrant: these 2 queries should produce same result, but 2nd one faster. can you try on df for me when you have a moment? https://pastebin.canonical.com/77405/02:59
wallyworldStevenK:  that's a lot of water02:59
wallyworldenginespot: find() is exported, so perhaps you can paste your code for us to look at03:00
StevenKwallyworld: http://www.smh.com.au/photogallery/environment/weather/hurricane-sandy-strikes-us-coast-20121030-28g5z.html?selectedImage=10 is pretty impressive too03:00
wallyworldyes indeed03:00
enginespotmy code like the follows:03:05
enginespotpeople=PersonSet.find()03:05
enginespot    #    for project in pad.projects:03:05
enginespot    #        print project.name03:05
enginespot    people = pad.people03:05
enginespot    i = 003:05
enginespot    j=003:05
enginespot    for person in people:03:05
enginespot        i+=103:05
enginespot        ppas = person.ppas03:05
enginespot#        r.set(person.name, 1)03:05
enginespot        for ppa in ppas:03:06
enginespot            j += 103:06
enginespot            r.set(j, ppa)03:06
enginespotI want to get a person list03:06
enginespotor launchpadlib can supply to me a pointer , I can read a person one by one03:07
StevenKenginespot: You've been told by two different people that what you want isn't possible. Asking different people isn't going to change the answer.03:09
enginespotyes , I find a different interface from launchpadlib03:09
wallyworldStevenK: what was the strategy going to be to stop the mixed visibility error oopses? what we do now to show <hidden> seems reasonable? i can't recall what the end game was to be03:27
StevenKI've can't recall off the top of my head, maybe wgrant can.03:28
wgrantIt's complicated™03:31
wgrantIgnore for now03:31
wallyworldwgrant: i made a mistake on the sql above, here's a fixed version https://pastebin.canonical.com/77410/03:51
wgrantwallyworld: Ah, sorry, distracted by secure boot destroying the world03:54
wgrantWill run in a sec03:54
wallyworldno hurry03:54
wallyworldi'm not sure if we need the filter on the spph subquery03:54
wgrantwallyworld: So removing the persistent cache fixed the ec2 issue?04:00
wallyworldwgrant: yeah, thanks. my mind was telling me it wasn't persistent when clearly it was04:01
wgrantGreat04:01
wallyworldif you want to look at my soyuz mp, it's up.04:01
wgrantI've glanced at it and opted to try to finish my current stuff before diving into it, unless you're in a hurry04:02
wgrantwallyworld: Is this the +ppa-packages query?04:03
wallyworldoh, no not at all. just mentioning it in case you hadn't noticed04:03
wallyworldyes04:03
wgrantRight, so you've changed the semantics slightly04:03
wgrantBut probably not in a way that anyone cares about04:03
wallyworldi can't see why it was joining to spph when it didn't need to04:03
wgrantAnd it's now much faster04:03
wgrantIt did technically need to, for the old definition of the package04:03
wallyworldthe semantics should be the same04:03
wgrants/package/page/04:03
wgrant... oh, I see04:04
wgrantYou're right04:04
wallyworldsince there's a clause that says spr.uploaded_archive = archive.id04:04
wgrantSo it joined Archive against SPR.upload_archive anyway?04:04
wallyworldyes, so it seems04:04
wgrantRight, so that SPPH join is completely unused?04:04
wallyworldas far as i can tell04:04
wallyworldbut it's there to eliminate spr that haven't been published?04:04
wgrantAh, yes, of course04:05
wallyworldso i did it as a subquery04:05
wallyworldso the semantics should be the same04:05
wgrantRight, I see the subquery now04:05
wgrantMissed it because of the indentation04:05
wgrantUm, well04:05
wgrantIt's slightly different04:05
wallyworldit was quicker?04:05
wgrantHeh04:05
wgrantheh04:05
wgrantheh04:05
wgrantThe subquery one is still running...04:05
wallyworldoh :-(04:06
wallyworldi would have thought postgres would have optimised that04:06
wgrantI'm not sure why it didn't here.04:07
wgrantBut rewording it as an EXISTS is a bit cleaner and works fine04:07
wallyworldi did it as an exists originally, but thought it too verbose04:08
wgrantIt's about 30% faster for me04:08
wallyworldok, will change it04:08
wgrantusing       AND EXISTS (SELECT 1 FROM sourcepackagepublishinghistory spph WHERE spph.sourcepackagerelease = sourcepackagerelease.id AND spph.archive = sourcepackagerelease.upload_archive)04:08
wallyworldbut is it faster than the original join?04:08
wgrantAlthough the spph.archive constraint is new; it's not in the original SPPH-joining query04:08
wgrantRight04:08
wgrant30% faster than the original query for me04:08
wallyworldhow much faster?04:08
wgrantTrying with pathological cases now04:08
wallyworldok04:08
wgrant35ms vs 24ms04:08
wallyworldthe oops should it taking a lot longer04:09
wallyworldshowed04:09
wgrantWell, I'm sort of the optimal case for the query04:09
wallyworldtry with fta04:09
wgrant14s -> 8s with fta04:12
lifelessto slow :)04:13
wallyworldwell, it's still almost 50% improvement :-)04:14
wallyworldand thats on DF04:14
wgrantprod won't be significantly faster for this04:15
wallyworldmight be ok on prod04:15
wallyworldi'm not sure hoe else to make it faster04:15
wgrantWell04:16
lifelesswallyworld: yes its a lot better, but prods cpu isn't /that/ much better than dogfoods; once you're in cache...04:16
wgrantThe general approach for this sort of problem is to not be stupid in the first place04:16
wgrantSadly we have history04:16
wgrantSo we can't really decide to not be stupid in the first place04:16
wgrantBut, in general, trying to calculate this sort of stuff out of the live tables is a terrible idea.04:17
wallyworldagreed04:17
wgrantWe could perhaps redesign the page04:17
wgrantSuch that it returns recent PPA publications created by the user04:17
StevenKWhy do you need to chain through SPR anyway?04:18
wallyworldi'm not sure of the requirements or history of what's required04:18
wallyworldStevenK: not sure, that's how it was04:18
wgrantIt's not chaining through SPR04:18
wgrantIt's entirely based on SPR04:18
StevenKEw04:18
wgrantIt doesn't care about publications, beyond the "has this been published?" check to get around rejections04:19
wallyworldwith the CPU issue, that assumes the query will be in cache, but i would have thought this page would nearly always be loaded cold04:21
wgrantThe involved tables total only 10GB or so04:21
wgrantAnd some of that is probably (hopefully) TOASTed04:21
StevenKI wonder if SRF and batching will help04:21
wgrantNo04:22
wgrantSRF helps only if we can calculate an ordered batch substantially more quickly than we can calculate the entire set04:23
wgrantWhich requires sensible schema design and indexing04:23
wallyworldi may as well land this small change till we figure out what to do next04:23
wallyworldsince it helps as is04:24
wgrantNo04:25
wgrantIt probably doesn't help significantly, and it may regress other cases04:25
wgrantIt's liable to turn the subquery into a 4-6s seqscan if the wind blows the wrong way04:25
wallyworldwhat would cause that to happen?04:26
wgrantIf it decides that there will be too many SPRs to efficiently use an index lookup04:26
wallyworldbut don't we batch so that we only ask for 75 or so at once?04:27
wgrantWe only ask for 75 at once, sure04:28
wallyworldi guess it still needs to know the number04:28
wgrantBut it's an ordered set04:28
wgrantAnd the order is sufficiently complex and layered that it can't be indexed04:28
wallyworldyeah, sadly04:28
wgrantSo it has to calculate the entire set04:28
wgrantSort it04:28
wgrantThen take the first 7504:28
wallyworldare you sure it could become a seqscan?04:29
wgrantYes, I've seen it happen here with fta once04:29
wallyworldand the join avoids that?04:29
wgrantSomehow, yeah04:30
wgrantThe join has to be applied after the conditions04:30
wgrantSo for the join it doesn't have to optimally determine the condition order to work out numbers04:30
wallyworldabove you said for fta it went from 14->804:31
wallyworldand he is a bit of a corner case04:31
wgrantAnd then it went to 16 or so a couple of times, as it chose a different plan for reasons which are unclear04:31
wallyworldah, bollocks, ok04:32
wallyworldi could use a CTE perhaps04:32
wallyworldto narrow down the spr records04:32
wallyworldbefore checking for publiscation04:32
wallyworldwgrant: is this any faster? https://pastebin.canonical.com/77411/   (when you have a moment)04:45
wgrantI suspect not, but let's see04:46
wgrantstill going...04:47
wallyworld:-(04:47
StevenKwallyworld, wgrant: https://code.launchpad.net/~stevenk/launchpad/sensible-superseded-by/+merge/13200904:49
wallyworldmaybe i need to add an array column to spr - "archives_published_in"04:49
wallyworldthat would eliminate the join04:49
wgrantThat wouldn't help significantly.04:50
wgrantIt would eliminate the seqscan, but leave the big performance issue04:50
wallyworldany other ideas?04:51
wgrantI'm trying to use window functions to do it04:51
wallyworldthat don't involve a schema redesign04:51
wallyworldStevenK: typing a blueprint name seems unfortunate.04:52
wallyworldmaybe a picker?04:52
StevenKwallyworld: So you missed that part of the call when Curtis said don't write a picker?04:52
wallyworldmust have04:53
wallyworldsometimes words cut out04:53
wallyworldor whole sentences with curtis speaks04:53
wgrantHm04:53
wgrantCurtis did say that :)04:53
wallyworldi would hate to type a blueprint name04:53
StevenKCopy and paste, hit continue04:54
StevenKMove on04:54
wallyworldguess so. seems very primative04:54
StevenKwallyworld: Better or worse than a dropdown with 6200 items?04:56
wallyworldthat's why a search based solution like a picker is best04:56
wallyworldnot sure why it was rejected04:57
StevenKOh good god https://twitter.com/raywert/status/263102070989680640/photo/1/large04:58
wallyworldStevenK: in validate, fetch the spec and put it in the data map. then do not fetch it again in the submit04:58
StevenKwallyworld: I wasn't sure if I could do that.04:59
wallyworldStevenK: yeah, the data dict is just passed arounf during the submit04:59
wallyworldrun the tests just to be sure your implementation is all good05:00
StevenKI've not run all the blueprint tests, but xx-superseding{,-within-projects}.txt pass05:00
wallyworldthat should be enough to send to ec2 with05:01
StevenKPfft, I was going to play buildbot bingo05:01
StevenKI already lost once today, though05:02
wallyworldStevenK: also05:02
wallyworld192+ if result is None:05:02
wallyworld193+ return result05:02
wallyworldrs.one() should be enough05:02
wallyworldi think?05:02
StevenKI thought that would die horribly?05:03
wallyworldlet me check the code05:03
wallyworldyes, rs.one() works05:03
wallyworldit returns None if rs is empty05:04
wgrantIt returns None, the sole value, or raises an exception05:04
wgrantFor 0, 1 and >1 results respectively05:04
StevenKYeah, I've changed it and checked it in iharness05:05
wallyworldStevenK: r=me, gotta pick up kid from school05:06
wgrantblah05:11
wgrantPARTITION BY seems to always want to traverse the whole set05:11
wgrantI guess it's not smart enough to realise that the inner and outer sorts match05:16
wgranteg. SELECT id, purpose, row_number() OVER (PARTITION BY purpose ORDER BY id) FROM archive ORDER BY id LIMIT 5; should just be able to walk down the id index until the fifth row05:17
wgrantBut it actually grabs the full table due to the PARTITION :/05:17
wgrantAh, it wants them sorted by purpose, id05:19
wgrantPerhaps it doesn't want to have to remember the latest row_number for each purpose05:19
wgrantSo there's no efficient way to do the DISTINCT ON on the server05:20
wgrantThe quickest solution is probably to ask for a reasonable number and do the distinct in Python :/05:21
wallyworldwgrant: so removing the distinct on from the query will make it fast?05:39
wgrantwallyworld: Well, with an index on (creator, dateuploaded DESC, id)05:39
wgrantYou can then do a direct indexed query05:39
wgrantIf you ignore the DISTINCT ON bit05:40
wgrantThe page tries to only show the latest SPR for each (distroseries, archive, sourcepackagename)05:40
wallyworldi guess i could iterate the result set till i have the requisite number of records05:40
wgrantAnd that is the bit that is slow to implement in SQL05:40
StevenKBleh, sinzui didn't use rollback05:40
wallyworlds/in SQL/in Postgres05:41
wgrantWe can do that using DISTINCT ON or a rank() with PARTITION BY, but both of those require that it be sorted by the partition/distinct first05:41
wgrants/in SQL/in SQL in postgres/, but yes05:41
wallyworldso i assume without checking yet that we don't have the index05:41
wgrantWe don't yet, indeed05:42
wallyworldwould adding it make the distinct faster?05:42
wgrantNo05:42
wallyworldok, we can add it live too then i think05:42
wgrantIndeed05:42
* StevenK tries to work out how to QA r1620705:43
wallyworldoh well, thanks for experimenting with it05:43
wallyworldtoo bad it doesn't want to play nice05:43
wgrantStevenK: librarian, codehosting, regret that you didn't wait until we had builders05:43
wgrantbasically05:43
StevenKwgrant: So push and pull a branch, but I'm not sure how to fiddle with the librarian05:44
wgrantclick link to librarian file05:44
wgrantsee if link to librarian file works05:44
wgrantupload file to l[ibrarian05:45
wgrantsee if librarian file works05:45
StevenKBleh, can't push to lazr.restful on qas due to bzr: ERROR: Server sent an unexpected error: ('error', 'NotBranchError', 'Not a branch: "chroot-67793680:///+branch-id/43364/".')  which I guess is stacking05:55
wgrant+junk is your friend05:55
StevenKExcellent, all four look good05:57
enginespotHi everyone06:19
enginespothow to get ppa from a project06:19
nigelbenginespot: Projects don't have PPAs. People do.06:28
wallyworldwgrant: do you think that without the distincts, it's best to do the subquery for the spph check or stick to a big join?06:49
wgrantwallyworld: You'll need to do the subquery06:51
wallyworldcool, thanks. that's what i've done06:51
wallyworldjust wanted to check06:51
wallyworldi also need an index on maintainer06:52
wallyworldsince another query filters on that06:52
=== yofel_ is now known as yofel
=== Ursinha-afk is now known as Ursinha
deryckrick_h_, you actually on irc?08:23
rick_h_deryck: rgr08:23
deryckrick_h_, there's a hacking room all the way at the back of the hall.  right side of the hall as you go back.08:24
deryckrick_h_, in case you need a place between sessions.08:24
rick_h_deryck: cool thanks. hacking out in the main area from yesterday atm08:24
deryckrick_h_, ok, might come say hello when I get coffee and chat through a couple things.08:24
adeuringgood morning08:46
rick_h_morning adeuring08:49
deryckmorning, adeuring08:49
adeuringhi ric, deryck08:49
=== almaisan-away is now known as al-maisan
=== benji is now known as Guest93766
StevenKdpm: O HAI10:53
StevenKdpm: Can you look at production in terms of raring translations?10:53
dpmStevenK, hey people at UDS say hi, you were on the projector10:53
StevenKHaha10:53
dpmajmitch says hi, and go to sleep :)10:53
StevenKPft, it's only 10pm10:53
StevenKHe can go to sleep.10:53
nigelblol10:54
dpmStevenK, raring translations look good to me in production - only the contributors column has gone down in terms of number of contributors for each language comparing q to r - any ideas why that could have happened?11:00
StevenKNone, I'm afraid11:02
ajmitchStevenK: I'm hardly going to go & sleep at noon11:02
StevenKajmitch: Pft. You fail at jetlag, then.11:02
dpmStevenK, so the contributor numbers is the only issue I see, otherwise translations look good. Could you investigate what caused the decrease in number of contributors?11:09
=== benji___ is now known as benji
=== Ursinha is now known as Ursinha-afk
=== al-maisan is now known as almaisan-away
=== almaisan-away is now known as al-maisan
=== Ursinha-afk is now known as Ursinha
=== Ursinha is now known as Ursinha-afk
=== Ursinha-afk is now known as Ursinha
=== matsubara is now known as matsubara-lunch
jtvsinzui, are you here?14:58
sinzuiI am14:58
jtvHi!  Long time no see.15:01
jtvsinzui: Laura mentioned that you'd been busy opening translations, and were having some trouble with Blender's uploads.15:01
jtv(As two separate issues)15:01
sinzuiblender was broken before we started opening15:01
jtvThe two things I wanted to ask are:15:02
jtv(1) Need any urgent help with the blender issue?  I may even be able to meet up physically with them in the coming days or weeks.15:02
jtv(2) Maybe we should try skipping the whole translations-copying step for S.15:02
sinzuiplease help with blender, the users do not understand lp -- they are not configuring translations as I suggest and they do not believe that open source communities will hate them if they setup a separate bug tracker15:04
sinzuitranslations for raring are in qa now. We might enable them in 12 hours15:04
sinzuijtv, this is the first opening where there there no errors in the db or scripts for translations...15:05
sinzuia non-event that we don't know how to document15:05
jtvsinzui: that is fantastic — documentation should just show date done, steps taken & time spent.  The way we always hoped it would become.15:14
jtvCan you give me a quick summary of what's wrong with the blender setup?15:14
sinzui1 they did not setup the right series.15:15
sinzui2. they did not setup the branch form the series15:15
sinzui3. yesterday, they had still not set the sync to import templates15:15
* jtv looks at blender's series15:15
sinzuiyes, we both can do it, but I think the suer should do it show that he knows ho to change it15:16
jtvI agree.15:16
sinzuiand he has set pots and pos I see15:16
jtvOuch — a separate “translations” series15:16
=== al-maisan is now known as almaisan-away
sinzuibut still note mplates15:16
sinzuistill no templates15:17
sinzuibut I don't know when this change was made15:17
jtvsinzui: this is https://launchpad.net/blender ?  I see a template.15:18
sinzuijtv https://translations.launchpad.net/blender/2.6x/+templates has no templates, which is what he was trying to do15:20
jtvAh!15:20
sinzuioh, the tree is scons an deep15:20
sinzuiis this intl tools?15:20
jtvBut there's nothing in the upload queue waiting for review.15:20
sinzuiI don't see any pots or pos15:21
jtvI'm not finding it either.15:24
jtvMaybe they want to get Launchpad to extract the pots.  But are we still running that?15:25
jtvAnd it doesn't look as if strings are even marked for translation…15:26
sinzuijtv: http://wiki.blender.org/index.php/Dev:Doc/How_to/Translate_Blender15:35
jtvAh, I was getting the branch15:36
jtvsinzui: not having much luck running their tools…  but it doesn't sound as if there's very much I can do to help.  :/15:42
sinzuiyuck15:45
sinzuiI think this is a case where we advise them to stick with the import of translation branch, and work closer with the upstream community15:46
jtvWait... these aren't the upstream people?15:50
=== matsubara-lunch is now known as matsubara
=== Ursinha is now known as Ursinha-afk
=== Ursinha-afk is now known as Ursinha
sinzuijcsackett, would you have time to review https://code.launchpad.net/~sinzui/launchpad/hide-question-comment/+merge/13217717:33
jcsackettsinzui: sure.18:04
jcsackettsinzui: r=me. thanks for the fix. :-)18:11
sinzuijcsackett, how goes bug 16453019:45
_mup_Bug #164530: User translations showing broken links <404> <lp-translations> <oops> <Launchpad itself:In Progress by jcsackett> < https://launchpad.net/bugs/164530 >19:45
jcsackettsinzui: it died in ec2 when i sent it out to land on thursday. i sent it out to ec2 again earlier today when i caught up with email and realized it had never landed and i had no ec2 results.19:46
sinzui:(19:46
jcsackettsinzui: it's been out for 3h45m. should get a (hopefully) ok result soon, so barring pqm issues it will be qa-able before i EoD.19:47
* jcsackett knocks on all available wood surfaces19:47
sinzuijcsackett, baring that, I would run all lp.translations tests and submit if they all pass19:49
sinzuijcsackett, what bugs are you looking at now?19:49
jcsackettsinzui: bug 79895419:50
_mup_Bug #798954: InvalidProductName: Invalid name for product: bookmark:galapagos.  <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/798954 >19:50
jcsackettwe discussed it the other day, i'm trying to replicate the error now.19:50
sinzuihmm19:51
sinzuijcsackett, Do we have any modern oopses19:52
sinzuijcsackett, https://oops.canonical.com/oops/?oopsid=OOPS-1df2ed046d05000215e8d42933e4493419:52
jcsackettsinzui: what method are you using to search the OOPS DB? you seem *much* faster at it than me.19:53
sinzuijcsackett, I was already 2F logged in. I went to the lp production page, opened the latest report and search for InvalidProductName, got nothing, then url hacked to the day earlier, and repeated the search19:54
jcsackettsinzui: ah.19:55
sinzuichromium also remembers the search so I only type it once19:55
sinzuiI think I get these often when I forget how to push to lp://qastaging/ , but I don't see the oopses19:56
jcsackettit's easy to reproduce manually on dev; i just had to spend some time getting codehosting working in my lxc.20:00
sinzuiMaybe I should never switch to lxc20:04
sinzuithis oops might be more interesting. https://oops.canonical.com/oops/?oopsid=OOPS-a3ef3a2868f310958e03998c18758fdb I don't think the user ever figured our how to push a branch20:05
jcsackettsinzui: does look like they were having problems.20:17
=== Ursinha is now known as Ursinha-afk
sinzuiwgrant, https://bugs.launchpad.net/launchpad/+bug/40858522:24
_mup_Bug #408585: choosing blueprint for branch is broken <lp-code> <Launchpad itself:Triaged> < https://launchpad.net/bugs/408585 >22:24
=== StevenK changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: ~200
StevenKwgrant: http://pastebin.ubuntu.com/1319322/22:37
StevenKwallyworld, sinzui: You two coming back?23:03
sinzuiI am not. got eat23:04
wallyworldback where?23:04
wallyworldsorry, i thought cal lhad finished23:04
StevenKThe server kicked everyone but wgrant off23:04
=== Ursinha-afk is now known as Ursinha
=== Ursinha is now known as Ursinha-afk

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!