/srv/irclogs.ubuntu.com/2010/12/03/#launchpad-dev.txt

mwhudsoni predict this new will make jml unhappy00:02
wgrantWhich new?00:03
wgrantThe restructure?00:03
pooliespm it turns out, just fyi, that qastaging has no mx record00:07
pooliei will send a mail to the list00:07
spmah. k00:07
wgrantflacoste: Thanks for forwarding that to the list.00:25
lifelesswgrant: hey00:30
lifelesswgrant: so, I don't know if you noticed, but the archive:+index timeouts are going up daily. I don't think this is random.00:31
wgrantlifeless: That's odd.00:32
wgrant(sorry, I've been distracted with other stuff the last couple of days)00:32
lifelessdoesn't deeply worry me00:32
lifelessits a little concerning, and I want to be worry free on my holiday00:32
lifeless21st:      387 / 6695  Archive:+index00:33
wgrantHmmm.00:33
lifelessbah00:33
lifeless1st:      387 / 6695  Archive:+index00:33
lifeless30th:      324 / 7217  Archive:+index00:33
lifeless29th:00:33
lifeless     287 / 6525  Archive:+index00:34
wgrantThe other pageids aren't showing a similar trend?00:34
wgrantThat's a pretty massive escalation.00:34
lifeless29th:00:34
lifeless     287 / 6525  Archive:+index00:34
lifeless     170 /  390  BugTask:+index00:34
lifeless      43 /  260  Distribution:+bugs00:34
wgrantIf it's the only pageid doing it, that's more 'deeply concerning' than 'a little concerning'.00:34
lifeless28th:00:34
lifeless     260 / 6568  Archive:+index00:34
lifeless     142 /  301  BugTask:+index00:34
lifeless      26 /  290  Distribution:+bugs00:34
lifeless27th:00:34
lifeless     115 / 5419  Archive:+index00:35
lifeless      92 /  233  BugTask:+index00:35
lifeless      23 /  244  Distribution:+bugs00:35
wgrantUh.00:35
wgrantI think we have a general problem :)00:35
lifelesspossibly00:35
lifeless30th:00:35
lifeless     324 / 7217  Archive:+index00:35
lifeless     148 /  451  BugTask:+index00:35
lifeless      36 /  150  ProjectGroupSet:CollectionResource:#project_groups00:35
lifeless     19 /  296  Distribution:+bugs00:36
lifeless  1st:00:36
lifeless     387 / 6695  Archive:+index00:36
lifeless     155 /  482  BugTask:+index00:36
lifeless     126 /  226  Question:+index00:36
lifeless      49 /  104  RootObject:+login00:36
lifeless      39 /  247  Distribution:+bugs00:36
lifelessThese are all genuinely problematic pages00:36
lifelessso I'm not inclined to shout fire yet00:36
lifelessrather fix the pages.00:36
lifelessand the worst.. well you know the worst.00:37
lifelessit is, on its own, 25% of the timeouts on the site.00:37
lifelessbesides which00:38
lifelessI know flacoste will be looking at these reports and biting his teeth in my absence ;)00:38
wgrantHeh.00:38
wgrantAh, I remember why I didn't get to it last time. download-cache took forever to upgrade.00:39
wgrantIt's done now, so the appserver might finally start...00:39
lifelesspillarname ||  963021.70 tuples/sec00:39
lifeless                                  branch ||  443538.40 tuples/sec00:39
wgrantWTF?00:39
lifelessarchive ||   41660.33 tuples/sec00:39
lifeless28th00:40
lifeless                              pillarname ||  825961.90 tuples/sec00:40
lifeless                                  branch ||  463744.27 tuples/sec00:40
lifeless                              sl_confirm ||  133113.62 tuples/sec00:40
lifeless          binarypackagepublishinghistory ||   49345.86 tuples/sec00:40
lifeless                                 archive ||   33620.98 tuples/sec00:40
lifeless                         shippingrequest ||   24934.80 tuples/sec00:40
lifeless1st:00:40
lifeless                           pillarname ||  827065.04 tuples/sec00:40
lifeless                                  branch ||  469628.31 tuples/sec00:40
lifeless                              sl_confirm ||  130679.92 tuples/sec00:40
lifeless          binarypackagepublishinghistory ||   53159.91 tuples/sec00:40
lifeless                                 archive ||   40228.91 tuples/sec00:40
lifeless                       sourcepackagename ||   24256.23 tuples/sec00:40
lifelessnot hugely different on a day to day basis00:41
lifelessit may be that traffic spike that I called out on the list causing trouble00:41
wgrant'the list' being the internal one, or have I missed something?00:41
lifelesswgrant: uhm00:48
lifelesschecking00:48
wgrantlifeless: Aw, is that really my first? :(00:54
lifelesswgrant: no :)00:54
mwhudsonsomething must be sequentially scanning branch to get numbers like that01:09
mwhudsoni'd love to know what01:09
mwhudsonwell not 'must' i guess01:10
mwhudsonseems very likely though01:10
wgrantAnd Julian doesn't think I'm crazy :D01:13
abentleypoolie: Any progress with QA for r12008?01:34
poolieabentley: hi, is that the dkim and gpg bugfixes?01:34
abentleypoolie: yes.01:34
pooliei presume yes, as that's the only one i landed01:34
poolieit turns out qastaging doesn't receive mail01:35
poolieit doesn't even have an mx01:35
pooliethat's where i'm up to01:35
pooliemy next thing is to send mail about perhaps turning it on01:35
poolieit may take too long to do it01:35
pooliei mean, it may take a while to turn it on and get it working, and that could cause rollouts to stall for too long01:35
lifelesspoolie: does your patch make anything worse, do you think?01:36
poolieperhaps there's a better approach?01:36
abentleypoolie: normally, I'd suggest using staging instead, then.  But unfortunately, we are doing a full db restore in order to ensure we don't run overtime.01:36
lifelesspoolie: and if so, can it be disabled easily?01:36
pooliei thought that qastaging would be like staging in this regard (ie reads mail, doesn't send it)01:36
lifelessabentley: Will my subunit db patch be in the rollout, do you think?01:36
pooliebut it's not01:36
poolielifeless: it changes existing code01:36
abentleylifeless: Yes, that's the plan.01:36
poolieso, it's possible it will cause a regression01:36
lifelessabentley: \o/01:36
poolieand it's not isolated by a flag01:36
abentleypoolie: I think qastaging is still being configured.01:37
poolieso if it does turn out to break anything that was previously working, i think we would have to roll back or reverse-cherrypick or i guess live with it until a new fix is landed :/01:38
wgrantpoolie: It's not on staging yet either?01:38
lifelesspoolie: db deploy - we can't roll those back.01:38
abentleypoolie: like it doesn't have import slaves or a build farm at the moment.01:38
abentleywgrant: It would be, but staging is doing a full restore ATM.01:39
wgrantabentley: Yay :(01:39
pooliei'll send mail first at least01:40
poolieshould we/do we have a bug asking for real-time log feeds?01:41
abentleypoolie: do you mind testing it in ~24h?01:42
pooliedo you mean <24h or >24h?01:42
pooliei mean, do you mean "will you please test it on saturday au/sydney?"01:43
abentleypoolie: that is what I mean.  If that's okay with you, it's probably the simplest solution.01:43
poolieand you mean test it on staging?01:44
abentleypoolie: yes.01:44
pooliethat's ok with me01:44
abentleypoolie: Thanks.01:44
poolieis there any more specific estimate when it'll be live there?01:45
pooliejust trying to work out if i can do it first thing saturday or something (like 20h from now)01:45
abentleyChex: Do you have an estimate when the staging update will complete?01:47
poolieabentley: i might also work out/document/improve testing it locally01:50
pooliemanually testing it locally, i mean01:50
pooliebut i think checking it in the dc would be really good too01:50
pooliethere are enough different moving parts01:51
abentleypoolie: Yes.  We don't normally consider it QAed until it's tested in the datacentre.  And I know our mail handling can be a bit odd, so I think testing it on staging is really worthwhile.01:52
poolieme too01:53
pooliethanks for following up on it01:53
abentleypoolie: np01:53
lifelesswgrant: so, please look at that branch :)01:56
lifelessciao01:56
pooliehey, can someone help me work out how to land max's loggerhead changes02:24
poolieis it enough to just merge them to the relevant branch?02:24
pooliespm? wallyworld?02:24
spmpoolie: I'm not sure at all tbh02:54
maxbpoolie: Don't you just PQM them onto lp:~launchpad-pqm/loggerhead/devel, and then update utilities/sourcedeps.conf?03:04
pooliei think so; i'll try it later03:05
poolieare one-letter project names banned?03:15
wgrantHmm, are the appservers overloaded?03:17
wgrantI'm getting truncated responses again.03:17
wgrantLess frequently than last time, though.03:17
spma couple are, yeah03:21
spmhttps://lpstats.canonical.com/graphs/AppServerRequestLpnet/ wheeee03:23
spmwgrant: with any luck, things should be better. looks like a buggy script was being rude.03:47
wgrantspm: Thanks.03:54
spmyeah the # requests graph is dropping off due to our heavy hand of stabbing application03:55
cody-somervillewgrant, Do you know much about SQL indexes?04:24
wgrantcody-somerville: Some. What's the issue?04:24
cody-somervillewgrant, I'm wondering if an index could be used to make finding the latest of something that relates to something else faster - ie. lets say finding the latest build for a project in a CI.04:26
wgrantcody-somerville: Of course.04:27
cody-somervilleWould just creating an index on say the finished_at field of the buildresult table be sufficient?04:28
cody-somervilleor would an index on the foreign key to the project and the finished_at field be better?04:28
wgrantProbably the latter.04:29
spmbe aware that adding indexes can have major negative performance impacts. if you're looking to modify LP stuff, I'd strongly urge you to check with stub, and at least get some explain's happening to show if an index *may* help.04:30
wgrantlaunchpad.dev seems to be just about unusable for me.06:00
wgrantIt has to grab several hundred YUI files on every page load...06:00
spmO_O06:02
wgrantYeah, more than 400 YUI JS files are included in each page's <head>. This seems questionable.06:02
StevenKwgrant: Whereas, usually, launchpad.dev is *fast*06:04
wgrantStevenK: I think it would be almost acceptably fast if the JS was all in one file like it is on prod.06:04
StevenKI thought the Makefile did combine-css?06:04
wgrantThe main issue is that only a single request can be processed at a time.06:04
StevenKOh, wait06:04
StevenKI'm a bozo06:04
StevenKDoesn't lazr-js do that for us?06:05
wgrantThe dev appserver doesn't use the combined launchpad.js, even if it has been built.06:05
wgrantI wonder which config key it is.06:06
wgrantAh, devmode.06:06
wgrantOMG so fast06:07
wgrantThere's also a disturbing number of 404 OOPSes that have just shown up in the last few minutes.06:11
wgrantLike 3000 of them.06:11
wgrantI think something might be wrong with the lazr-js upgrade :/06:12
wgrantStevenK: Could you pastebin the query log from OOPS-1780G1265 for me?06:17
StevenKNo, you can bloody well wait 10 days06:17
wgrantHeh.06:17
wgrantBut IS probably won't get around to creating my account for years :P06:17
StevenKI doubt it06:18
StevenKwgrant: http://pastebin.ubuntu.com/539296/06:18
StevenKOh sigh, that's the wrong bit, isn't it06:18
StevenKwgrant: http://pastebin.ubuntu.com/539297/06:19
wgrantStevenK: Thanks.06:21
=== almaisan-away is now known as al-maisan
lifeless=== Top 10 Time Out Counts by Page ID ===07:03
lifeless    Hard / Soft  Page ID07:03
lifeless      80 / 5210  Archive:+index07:03
lifeless      62 /  271  BugTask:+index07:03
lifeless      21 /  248  Distribution:+bugs07:03
lifeless      18 /    0  BinaryPackageBuild:+retry07:03
wgrantlifeless: Yes, I'm working on it right now :P07:03
lifeless      15 /    3  ProjectGroup:+milestones07:03
lifeless      12 /  276  Distribution:+bugtarget-portlet-bugfilters-stats07:03
wgrantBut the factory is being shitty.07:03
lifeless      12 /   19  Archive:+copy-packages07:03
lifeless      10 /  123  ProjectGroupSet:CollectionResource:#project_groups07:03
lifeless       9 /  146  POFile:+translate07:03
lifeless       8 /    2  Person:+bugs07:03
lifeless*interesting*07:03
wgrantThe 300 or so missing ones?07:03
lifelesscool07:03
lifelesswgrant: factory fixin time?07:13
wgrantlifeless: I think it's OK now.07:14
wgrantI was confused for a while after I fixed it, since the baseline assertion had started failing without me noticing.07:15
wgrantBut I think it's sorted now.07:15
lifelesskk07:31
adeuringgood morning08:48
mrevellMorning Launchpad devvers09:20
gmbSo, StevenK... about that failure in db-devel. Is anyone looking at why it happened?10:00
=== al-maisan is now known as almaisan-away
=== jkakar_ is now known as jkakar
gmbjml: Given the nature of the current db-devel problem (i.e. ongoing) is there any reason not to force the build so that we don't block on it? I'll add a comment to the bug, obviously.11:01
wgrantHudson's happy. Let's use Hudson.11:01
* gmb can't stop thinking people are referring to mwhudson when they say "Hudson."11:04
wgrantMy statement may still stand, even in that case.11:04
gmbHudson, old boy, merge this branch would you? There's a good chap.11:04
gmbTrue.11:04
gmbjml: Ah, ignore me; it's obviously not all that intermittent; 3 out of the last 4 builds failed because of this, if I'm reading buildbot right.11:15
wgrantHow easy is it to find a timeout by its pageid?11:56
deryckMorning, all.12:02
StevenKgmb: I utterly forgot to look and got distracted by work12:11
StevenKgmb: But if I'm reading your last statement rightly, it's not really not my fault, it's buildbot being crap?12:11
wgrantWhen did it start?12:11
gmbStevenK: Yeah, it looks like it's an annoying but ongoing problem. I haven't had time to dig further.12:11
wgrantAround the time of the databasefixture landing?12:12
gmbwgrant: The bug was filed on the 29th, FWIW: https://bugs.edge.launchpad.net/launchpad-foundations/+bug/68286112:12
_mup_Bug #682861: db-devel librarian hung with '[Errno socket error] timed out' <build-infrastructure> <Launchpad Foundations:Triaged> <https://launchpad.net/bugs/682861>12:12
wgrantThat's, er, slightly suspicious timing.12:13
wgrantThe databasefixture stuff landed on the 28th.12:13
* StevenK is going to bang the Hudson drum in the new year12:14
StevenKI might actually *gasp* do some work on it while I'm off next week12:14
wgrantDamn, then who am I going to harass during my last week without ~launchpad superpowers?12:15
StevenKwgrant: Go play in the traffic12:15
wgrant:(12:15
bigjoolswgrant: well done on passing12:19
wgrantbigjools: Thanks!12:19
bigjoolsdid you get graded?12:20
wgrantOverall for the degree? No.12:22
bigjoolswhen do you know?  or don't you?12:22
wgrantI'm not sure if they do it... but let's see what it would be.12:23
wgrantOh good, H1.12:23
* bigjools has no idea what that means12:24
wgrantFirst-Class Honours, ie >=80%.12:24
bigjoolsI did get a first class honours in hindsight from the university of life12:24
wgrantHaha.12:24
bigjoolswell done, anyway!12:25
wgrantThanks.12:25
wgrantIt is great to be done.12:25
leonardrjml, i sent email about this, but what do you mean when you refer to the set-based design we discussed 3 years ago?12:30
=== mrevell is now known as mrevell-lunch
wgrantderyck: Hi.12:55
deryckhi wgrant12:56
wgrantderyck: I'm seeing around 450 YUI JS files included in <head> when it devmode. It makes page loads sort of slow. Is this likely to be related to your recent lazr-js upgrade?12:57
deryckwgrant, this is definitely related.  After I do qa today on the production changes, then I'll turn my attention to making that better.12:58
wgrantderyck: Thanks.12:58
deryckI should have sent mail about that already, sorry.12:58
wgrantTurning of devmode makes it fast again, since it just uses launchpad.js.12:58
wgrantI cannot type tonight, apparently.12:58
deryckheh12:59
deryckyeah, that is definitely a work around if you're not hacking on js itself.12:59
gmbflacoste: with db-devel being broken because of this ongoing but not-all-the-time timeout problem, does it make sense to force the build so that people can land things in the interim (and who knows, maybe it'll work this time)13:09
gmb?13:09
* gmb JFDoesIt.13:12
=== vednis is now known as mars
=== mrevell-lunch is now known as mrevell
marsflacoste, ping13:41
=== almaisan-away is now known as al-maisan
marsgmb, +1 on forcing the build, +1000 on finding a possible clue as to its cause13:56
gmbmars: Yeah. I'll take a look later if I get the chance.13:56
wgrantmars: It's probably the librarian fixture changes.13:59
wgrantIn db-devel r1001313:59
bigjoolswgrant: awake enough for a very quick chat about removing non-PPA pockets from their indexes?14:02
wgrantbigjools: 'course.14:02
wgrantbigjools: They shouldn't be there, though...14:02
bigjoolswgrant: I think it's an easy case of fixing the loops in C_writeIndexes and  D_writeReleaseFiles14:03
bigjoolsI was going to abstract the pockets to a call on IArchive14:03
wgrantbigjools: They're not actually ever written at the moment.14:03
wgrantBecause they're never dirty.14:03
bigjoolsright14:03
bigjoolsbut it loops them nonetheless14:03
wgrantWe should skip them in A and B as well.14:03
bigjoolsah yes14:03
wgrantbigjools: So, I'd just replace all the 'for pocket in PackagePublishingPocket.items' calls with 'for pocket in self.archive.getPocketsForSeries(distroseries)'14:05
=== matsubara is now known as matsubara-lunch
bigjoolswgrant: why by series?14:06
wgrantbigjools: Because you never know what Ubuntu will want to do next. But I guess for now we can skip that bit.14:07
wgrantSince it'll only be used in a couple of places.14:07
wgrantAnyway, it is getting late.14:16
flacostehi mars14:29
bigjoolsaww jml you painted over my green bikeshed!14:45
bigjoolsfoundations guys: I think I need to add lazr.enum._enum.EnumItems in lp_sitecustomize.py so that it has no security proxy, any thoughts?14:53
=== flacoste changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 2 of 10.12 | PQM is open | firefighting: db-devel broken | https:/​/​dev.launchpad.net/ | Get the code: https:/​/​dev.launchpad.net/​Getting
marsflacoste, pong15:12
=== matsubara-lunch is now known as matsubara
flacostemars: what can I do for you?15:17
marsbigjools, I think Gary would be best positioned to answer that question, but unfortunately he is out today15:18
bigjoolsmars: yeah, he's blowing out a lot of candles15:18
mars:)15:18
bigjoolsI can't talk, I've got the same number in 3 months...15:19
jcsacketthow often is the data on qastaging restored?15:25
* bigjools shakes fist at zope testrunner15:34
bigjoolsjcsackett: one a week15:35
bigjoolsonce*15:35
jcsackettthanks.15:35
bigjoolsso, if I use an argument to bin/test like "-t soyuz -t !windmill" it ends up running every single test except windmill ones rather than all the soyuz  tests except windmill.15:36
jelmer-t is unfortunately || rather than &&15:41
* jelmer often just uses a file with a list of tests to run and calls ./bin/test --load-list=FILE15:42
bigjoolsjelmer: yeah, && would be more useful15:55
=== beuno is now known as beuno-lunch
bacjtv: are you disappearing soon?16:31
jtvbac: from the channel, yes.16:31
bacjtv: rats.  i'd like to work with you on a problem but i realize it is really late for you.16:31
jtvbac: oh, you're back on the West Pole already?16:32
bacyep, for a week16:32
jtvA week.16:32
jtvbac: but hey, see if you can grab my attention.  What is it you wanted to work on?16:33
=== deryck is now known as deryck[lunch]
=== abentley changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 2 of 10.12 | PQM in RC mode | firefighting: - | Get the code: https:/​/​dev.launchpad.net/​Getting
=== abentley changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 3 of 10.12 | PQM in RC mode | firefighting: - | Get the code: https:/​/​dev.launchpad.net/​Getting
=== deryck[lunch] is now known as deryck
=== benji is now known as benji-lunch
=== beuno-lunch is now known as beuno
cody-somervillelibmozjs2s from LP-PPA-Launchpad is trying to uninstall my xulrunner :(17:44
=== benji-lunch is now known as benji
derycksinzui, ping18:55
mkanatIs there an icon that would mean something like "give me the raw content of this" in Launchpad that I should re-use for loggerhead?18:58
maxbNot that I have seen19:09
sinzuihi deryck19:09
maxbAlso you'd have to jump through lots of licensing hoops if you wanted to take an icon from LP into Loggerhead19:10
deryckhi sinzui.  Is the calendar widget used on registry pages in js or python?  I can't find tests or a js file for it.19:11
sinzuideryck, the calendar is yui2. I think it was moved to lp.app19:11
sinzuideryck, we want the widget to be used every time there is a date or datetime field, but we do not seem to know how to ensure the js loads because the field is present19:12
derycksinzui, ok, cool.  That helps then.  The lazr-js upgrade has borked the style of this widget, but I suspect it's an easy fix via class names.19:13
mkanatmaxb: Why, are the LP icons not GPL?19:13
maxbmkanat: No, they are copyright Canonical, restricted rights for use in development only19:14
mkanatmaxb: Oh, okay.19:14
sinzuideryck, I thought there was a windmill test for the calendar, but I do not see it19:15
derycksinzui, yeah, it doesn't appear to have any test coverage, as far as I can see.19:16
deryckbut no biggie.  I'll fix it all up next week.19:17
sinzuiI there a yui3 calendar widget yet?19:17
deryckhmmm, not sure.19:17
deryckI can look into that a do a replacement.19:17
=== al-maisan is now known as almaisan-away
benjiIs there a design document (or something similar) that describes the librarian?19:31
lifelessyes19:31
lifelesslibrarian.txt, docstrings.19:32
benjik, thanks19:32
lifelessjkakar:19:51
lifelessjkakar: hey19:51
jkakarHi! :)19:52
lifelessjkakar: wondered what you think of my object graph to model query proposal.19:52
lifelessI'm sure its not new, but none of the python ORM's I've seen do it ...19:52
jkakarlifeless: It's still marked as "read" in my mail... I haven't made the time to read through it properly. :(19:53
jkakarlifeless: I want to, just been pressed with lots of other things.19:53
jkakarlifeless: Will try to get to it this weekend.19:53
=== matsubara is now known as matsubara-afk
wgrantflacoste: Do you really think that Soyuz can be simplified such that domain expertise is no longer required?20:54
flacostehi william!20:54
jcsackettthe whole pqm/release thing now largely has to do with stuff in db-devel, right? things done in devel will still roll out on a fairly regular interval, so RCs are less important, aren't they? or have i completely misunderstood something?20:54
wgrantHeh, morning.20:54
flacostewgrant: i do believe that we can make it so that many changes can be done without requiring vetting by somebody who "understands the whole system" and without the risk of breaking everything20:56
flacostewgrant: but people will have to understand the basics of package building and archive management20:56
jelmerjcsackett: No, that's right.20:56
flacostebut i don't think it requires a year of dedicated Soyuz hacking to get that20:56
jcsackettjelmer: thanks.20:56
* jcsackett relaxes a touch.20:57
flacostejcsackett: you should still make sure that all your revisions before 12019 are qa-okl20:58
flacosteor get a rc to fix any qa-bad in that range20:58
flacostejcsackett: but yeah, we'll continue nodowntime deployment afterwards from devel20:58
jcsackettflacoste: i may need to do that. it's not a qa-bad, per se, but a qa-ok that's an incomplete solution to an inprogress bug. but i think i'm about to have the full solution, which would be nice to release all together.20:58
flacostejcsackett: right, you can negotiate that with abentley20:59
jcsackettflacoste: right. first though i have to know if have anything to negotiate. was just checking on time tables. :-P20:59
abentleyjcsackett: are any of your changes going to introduce regressions?21:03
jcsackettin the thing i'm working on?21:03
jcsackettabentley: in the thing i'm working on, no. in the branch that's qa-oked but we've discovered is incomplete, it wouldn't appear so. nothing is worse than it was, or we'd qa-bad it.21:04
lifelesswgrant: required? no. God Like Knowledge Of Every Line Of Code - that can be fixed.21:05
abentleyjcsackett: Okay, so it won't need to block deployment.21:05
lifelesswgrant: for instance, concurrent uploads to the uploader, what unique keys are :)21:05
jcsackettabentley: yeah, the branch already landed isn't a blocker.21:05
abentleyjcsackett: We can give it an RC if you think it's necessary to deploy your changes at the first available opportunity.21:06
wgrantlifeless: It's not quite that simple to fix, I'm afraid.21:06
abentleyjcsackett: That would be Wednesday next week at 10:00 UTC.21:06
jcsackettabentley: dig.21:06
lifelesswgrant: is it worth fixing?21:06
lifelesswgrant: (rhetorical)21:07
abentleyjcsackett: Otherwise, land it normally, and it can be part of the first nodowntime deployment of the next cycle.21:07
jcsackettabentley: okay. thanks.21:07
lifelesswgrant: I think its worth fixing because the current set of folk that can work on soyuz is the intersect of 'knows zope well', 'knows twisted well', 'knows debian packaging well', 'knows debian archive mgmt well'21:07
lifelesswgrant: ... very small set.21:07
wgrantlifeless: Sure, that sucks.21:08
lifelessevery one of those that we can relax by better layering, clearer code, safer system, the larger the set of folk that can contribute sensible.21:08
wgrantBut the new model means that the last two sets will vanish.21:08
lifelessuhm21:09
wgrantAnd after that it's just a matter of time until someone mistakenly qa-ok's something and tens of millions of machines notices.21:09
lifelesswgrant: I think you're mistaken.21:09
abentleyrockstar: chat?21:09
lifelesswgrant: right now, the review policy does not require domain reviews of changes.21:10
lifelesswgrant: the reorg asks folk to take on the challenge of a wider view of the system. It doesn't expect them to suddenly know things they didn't.21:10
lifelesswgrant: learning will be involved.21:10
wgrantlifeless: No, but few outside the team make Soyuz changes. And I know that at least I review everything Soyuzy.21:10
lifelesswgrant: what will stop you doing that?21:10
wgrantlifeless: Trusting everyone to make changes to such a fragile, critical piece of Launchpad. Probably starting with myself.21:11
lifelesswgrant: do you mean 'there will be so many soyuz patches wgrant cannot look at them all' ?21:12
wgrantlifeless: No. I mean what I said. When I can be reasonably confident that people aren't able to accidentally break things, I will have a reason to stop watching.21:14
wgrantSoyuz is fragile. Some of this fragility is implied by the domain. And the consequences of a mistake can be immense.21:15
rockstarabentley, sure.  Mumble?21:16
abentleyrockstar: sure.21:16
lifelesswgrant: you're talking about breaking the primary archive in some fashion, right ?21:18
wgrantlifeless: That's the worst case, but sure.21:18
lifelessso, some of the fragility is due to apt disk layout, sure.21:18
flacostewgrant: i really don't get 'fragility implied by the domain'21:18
lifelessbut a lot of it is due to our code.21:18
flacostei think that's my main point of disagreement21:18
flacostewhat is so fragile in archive management21:18
wgrantlifeless: A lot of the code is atrocious, sure.21:19
lifelesswgrant: I'll buy 'expected cost of a qa failure is huge'21:19
wgrantI'll also throw in "QAing completely is just about impossible"21:19
lifelesswgrant: but I don't see a reason for a qa failure to be more or less likely simply because its archive mgmt21:19
flacostethe fact that we relied on poor perl code for stuff isn't something implied by the domain21:19
lifelesswgrant: builds are already more-than-soyuz-team today.21:19
wgrantlifeless: Um, not really, but OK.21:20
lifelesswgrant: buildd manager isn't, but it breaking isn't catastrophic (compared to the main web UI going down, APIs going down, or codehosting going down)21:20
lifelesswgrant: its not less catastrophic either.21:21
wgrantlifeless: buildd-manager problems are always handled by Soyuz.21:21
wgrantStill.21:21
lifelessyes, I know21:21
wgrantDespite it in theory being shared for 11 months now.21:22
lifelesswell21:22
lifelessbuildd manager wasn't ever shared yet AFAIK21:22
lifelessslaves are21:22
wgrantHm?21:22
lifelessand they get worked on21:22
* jelmer waves21:23
jelmerflacoste: I don't think the fragility is implied by the domain. Even if it is, it's certainly not why things are fragile at the moment.21:24
wgrantSome of it is. A lot of it isn't.21:24
flacostejelmer: thanks! i was starting to think i was blind to something21:27
wgrantYes, Soyuz is far worse than it needs to be.21:30
jelmerflacoste: I do agree with Julian's concerns about Soyuz' fragility in the shorter term though.21:33
jelmerperhaps I should follow up on the ml21:37
cody-somervilleSounds like there is an interesting discussion going on somewhere that I haven't seen yet.21:37
wgrantOn canonical-launchpad@l.c.c21:38
flacostecody-somerville: 'Announcing Launchpad Squads' on canonical-launchpad mailing list21:38
lifelessflacoste: perhaps its time to take it to lp-dev ?21:39
flacosteperhaps21:39
flacosteor start a different thread there21:39
lifelessthe latter is what I meant21:40
elmo"Grave mistakes caused by soyuz rookies setting off booby traps is a future problem that might not ever actually happen.", i.e. "this risk may not happen, therefore it does not exist"21:41
elmothis thread is really beginnning to depress me21:41
cody-somerville:-(21:43
wgrantlifeless: On a lighter note, my Archive:+index and Archive:+packages fix is reviewed and tested.21:48
lifelesswgrant: sweet.21:49
lifelesswgrant: land it!21:49
wgrantlifeless: Aren't we still r-c?21:49
* lifeless twitches21:50
wgrantThe topic suggests it, although db-stable was merged recently.21:50
lifelessyes21:50
lifelesswe need to qa *past* the db merge21:50
lifelessand then we can unlock21:50
wgrantAh.21:50
lifelessthats the critical section21:50
lifelesscurrently waiting for buildbot21:51
flacosteelmo: i think i understand your concerns, iow, fuck ups in Soyuz can have a high-impact, we shouldn't thread this lightly22:21
flacosteelmo: it does mean that we should be careful when reviewing changes and QA there, but not sure that keeping the people who can make changes there is a solution22:23
flacosteelmo: it didn't really got us far now22:23
EdwinGrubbslifeless: if a page explicitely gets the master store, doesn't it mean that every *Set class will load objects with the wrong store since they usually use IStore to get it? It seems like passing a store object into methods on *Set classes is a an inefficient way to handle this.23:01
lifelessEdwinGrubbs: there may be many cases where its a problem.23:01
lifelessEdwinGrubbs: my primary point was you're adding a new instance of the problem :)23:01
lifelessEdwinGrubbs: we had timeout/performance issues on some pages due to this with LibraryFileAlias, for instance.23:02
lifelessEdwinGrubbs: I don't mind if you want to search for a cool elegant way to handle it, but supplying the store is a fairly obvious and will-work approach.23:02
lifelessEdwinGrubbs: essentially, IStore should be used as a matter of last resort, because its a global lookup rather than related-state lookup.23:03
wgrantThe Sets traditionally use I(Master|Slave)?Store.23:04
wgrantSince they don't have an object on which they can call Store.of.23:04
lifelessright23:10
lifelessits a bug23:10
lifelessthe whole set structure is a bit buggy23:12
lifelessthey are exposed as singletones23:12
lifelesss/es/s/23:12
wgrantRight.23:12
lifelessbut correct behaviour for them depends on non-global state23:12
wgrantI'm hoping that your next piece of work will make this a bit more fixable.23:12
wgrantcody-somerville: What do I have to do to convince you not to use PPAs for obsolete series?23:32
lifelesswgrant: customers which we have contracts with need them.23:33
wgrantWell that's stupid.23:33
wgrants/stupid/insecure/, if you must.23:34
lifelesss/.*/reality/23:34
wgrantShh.23:34
lifelessinsecure?23:34
wgrantOr does OEM maintain security updates for obsolete series?23:35
lifelessa per device thing, I believe23:35
lifeless(so yes)23:35
wgrant:(23:35
jpdsEw.23:36
lifelessjpds: well, its essentially a LTS for a specific customer, AIUI. cody-somerville will know all the gory details.23:57
lifelesspoint is, *some* PPA's today need to keep publishing non-regularsupported suites.23:58

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!