/srv/irclogs.ubuntu.com/2011/05/05/#launchpad-dev.txt

wgrantlifeless: No.00:03
wgrantlifeless: I'll do it.00:03
wgrantIt is rather intimidating.00:03
LPCIBotProject windmill-db-devel build #234: STILL FAILING in 1 hr 5 min: https://lpci.wedontsleep.org/job/windmill-db-devel/234/00:17
* wgrant cries at r1297000:32
lifeless\o/ 1297600:35
wgrant!!!00:35
LPCIBotProject windmill-devel build #26: STILL FAILING in 1 hr 10 min: https://lpci.wedontsleep.org/job/windmill-devel/26/00:37
wgrantlifeless: Just by looking at the diff I can see it is bad.00:55
lifelesswgrant: 1297000:55
lifeless?00:55
wgrantYes.00:55
wgrantIt affects the API.00:55
wgrantWhen this was to most definitely not affect the API yet.00:55
wgrantIt will break security unembargoing.00:56
wgrant(since the security team doesn't have upload privileges)00:56
lifelessthey don't ?00:57
wgrantHee hee hee.00:58
wgrantlifeless: I invite you to look at canonical.launchpad.security.AppendArchive00:59
lifelesswgrant: blink01:00
lifelessso how does anyone upload to Ubuntu ?01:00
wgrantlifeless: AppendArchive isn't used for uploads.01:01
lifelessclass AppendArchive(AuthorizationBase):01:01
lifeless    """Restrict appending (upload and copy) operations on archives.01:01
wgrantIt's used for protecting syncSource, basically.01:01
lifelessthat seems misleading01:01
wgrantYes.01:01
lifelessso01:01
wgrantIt's also used for PPAs, I guess.01:01
wgrantBut not primaries.01:01
wgrantAnd:01:01
wgrantwgrant@lamuella:/tmp/ubuntu-archive-tools$ ./edit_acl.py query --person ubuntu-security01:02
wgrant== All rights for ubuntu-security ==01:02
wgrantArchive Upload Rights for canonical-partner-dev: archive 'partner', component 'partner'01:02
wgrantThere is no ArchivePermission granting them access.01:02
lifelessis this a defect or oversight01:03
wgrantMerely one of the stack of hacks that is security-in-soyuz.01:03
lifelessI mean, it seems to me that only folk that can upload should be able to unembargo01:03
lifelessotherwise there is a backdoor-to-uploading01:03
wgrantSort of. ubuntu-security is only meant to be able to upload to -security.01:03
wgrantBut that can't be done yet.01:03
wgrantAnd syncSource has to be extremely restricted.01:04
wgrantBecause it is a hack.01:04
wgrantIt is full of hacks.01:04
wgrantAnd it cannot be widely used yet.01:04
lifelessso, is this unembargoing untested?01:04
wgrant(it doesn't respect the queue, announcements, etc.)01:04
wgrantlifeless: Unembargoing is normal package copying by this special set of users that probably aren't represented in sampledata.01:04
wgrantBut let's see.01:05
wgrantHeaddesk.01:07
wgrantIt only checks that launchpad.Append is sufficient to call syncSource (with somebody who already has upload privileges), and then checks that adding someone else to ubuntu-security gives them launchpad.Append.01:08
lifelessso the test is invalid01:08
lifelessbecause the control is incomplete01:09
lifeless?01:09
wgrantYes, and because security-in-soyuz is mostly P3As + a stack of several horrible special cases.01:09
wgrant(which will hopefully be eliminated as part of the second stage of DDs)01:10
wgrant(but not yet)01:10
wgrantAnyway, let me make myself a security person on dogfood and actually try it.01:12
lifelesshttp://www.lifehacker.com.au/2011/05/the-cognitive-cost-of-doing-things/01:19
lifelessflacoste: http://www.google.com/support/analyticshelp/bin/answer.py?hl=en&answer=1205784&topic=112071801:27
wgrantvirtinst 0.500.1-2ubuntu6.1 in lucid (Signer is not permitted to upload to the component 'main'.)01:30
* wgrant rolls it back.01:30
magciusjam, did you take a look at the new CSS stuff I added?01:32
wgrantStevenK: Deleted delayed copies yet?02:15
wgrantAll of the 70ish syncSource timeouts are from the unoptimised delayed copy path :(02:15
spmlifeless: wrt the page speed via GA, last I used that in anger it gave accurate percentages; but lousy actual numbers of X's doing Y. ie treat as strong guideline vs any sort of "X many Users are..."02:19
lifelessspm: thanks02:20
StevenKwgrant: Ah02:21
wgrantlifeless: Time to whitelist Distribution:+bugs and Distribution:EntryResource:searchTasks?02:22
lifelesswgrant: the poor queries are > 20 seconds02:22
wgrantlifeless: We've only had frequent complaints about these over the last month or so.02:22
lifelessyes02:22
wgrantlifeless: I suspect they used to work eventually.02:23
wgrantAfter the cache got reasonably warm.02:23
wgrantAs it stands, common single-tag searches fail consistently.02:23
lifelesssince we added the optimised index which is causing pathological tag lookups02:23
lifelessits not coldcache02:23
wgrantAh. Wonderful.02:23
lifelessall of bugtags,bugs,bugtasks sit in hot cache02:23
lifelesseither the vm cache or the pg shared region02:24
lifelessfun 1 /  121  RootObject:+login02:25
lifelessStevenK:  271 OperationalError: FATAL: Ident authentication failed for user "sync_packages" FATAL: Ident authentication failed for user "sync_packages"02:26
lifelessStevenK: do you need bugs filed about this? [there are two issues - the auth, and the OOPS prefix not being configured.02:26
StevenKlifeless: Sounds like a plan -- please assign them to Gavin, since he worked on it and it should be fresh in his mind.02:27
wgrantThe first one needs a LOSA nudge, not a bug.02:27
lifelessboth are config02:27
wgrantThe second needs a trivial prod-config branch, which is just as fast as filing a bug..02:27
lifelessright02:27
lifelessbeen two days though02:27
lifelessallenap: ^02:27
lifelesswell, > 2, but as top oops02:28
StevenKSigh. TALES, please DIAF.02:28
spmit's problematic that ident faileds are getting into prod. these should be being added to the "funky rollout notes" at least. :-(02:29
wgrantlifeless: Hm, that RootObject:+login is a bit bd. The OpenID assoc takes 20s :/02:29
StevenKIt raises LocationError, but it doesn't tell me which template causes it.02:30
* StevenK works it out using the test02:31
wgrantThat is definitely the biggest problem I have with TALES most of the time.02:32
StevenKlifeless: So, Gavin's work is based on much earlier work by jelmer that was written, but never used. I guess he assumed it was already in the prod configs, but I can't say for sure.02:32
StevenKspm: ^ you too02:33
StevenKRargh, so the test and ZCML all point me to a view and template that don't make use of parent_series02:49
wgrantStevenK: Which view is erroring?02:50
StevenK(distroseries, '+index')02:50
wgrantIt's not using SourcePackage.packaging at all?02:50
mwhudsoncoming from nowhere in particular, but why does launchpadlib use oauth signing if all requests are over https?02:51
StevenKwgrant: I don't think that's relevant, given the LocationError is (None, 'displayname')02:53
mwhudsonoh hang on, it doesn't02:53
wgrantmwhudson: Doesn't it?02:53
mwhudson        oauth_request.sign_request(02:54
mwhudson            OAuthSignatureMethod_PLAINTEXT(),02:54
mwhudsonmy lplib might be hilariously out of date, mind02:55
mwhudsonok, it was, but it still seems OAuthSignatureMethod_PLAINTEXT is used03:03
wgrantYeah.03:03
* mwhudson tries to remember what oauthnonce is for then03:03
wgrantPLAINTEXT doesn't mean unsigned.03:03
wgrantI don't understand how it works, though.03:04
mwhudsonhttp://tools.ietf.org/html/rfc5849#section-3.4.4 says It does not03:04
mwhudson   utilize the signature base string or the "oauth_timestamp" and03:04
mwhudson   "oauth_nonce" parameters.03:04
wgrantThat indeed makes sense.03:04
mwhudsoni'm not sure it's what lib/contrib/oauth.py does though03:05
mwhudsonwhee03:05
wgrantYeah :/03:05
lifelessnoone is03:05
mwhudsonhooray03:09
StevenKwgrant: distroseries-index.pt was a red herring, the ZCML points out the page includes distroseries-portlet-derivation03:09
wgrantStevenK: Ah.03:09
LPCIBotProject windmill-devel build #27: STILL FAILING in 1 hr 5 min: https://lpci.wedontsleep.org/job/windmill-devel/27/03:19
lifelessstub: morning03:24
stubIndeed! It really is!03:24
lifelessstub: I've put https://bugs.launchpad.net/launchpad/+bug/534203 in the dba bugqueue03:24
_mup_Bug #534203: Timeouts on POFile:+filter (filter by person) <lp-translations> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/534203 >03:24
lifelessstub: yeah, congrats!03:24
stubOh yay. Translations. The *other* place we have large amounts of asymmetric data yielding crappy plans.03:26
lifelessif translationmessage.potemplate was usable03:26
lifelessI think it would be fine03:26
stubSo a lot of these queries have already been optimized at some point by jtv & danilo, trying to get certain cases to work. I don't know which queries they are though.03:28
stubBut we should jtv in on translations queries when he is available.03:28
lifelessthis bug was filed by jtv in fact03:28
stubc/should/should bring03:28
lifelessI wanted to ask him about the potemplate oclumn03:28
lifelessyes, I agree03:28
stublifeless: About it being inconsistently populated?03:32
* stub wonders if the data model changes have been completed, and all the unwanted columns now dropped.03:33
lifelessstub: yeah, and/or repopulating it and using it to make the query fast03:34
stublifeless: We need a modern OOPS code in that bug btw. I think the linked ones are all from last year still.03:34
lifelessstub: hmm, I refreshed it yesterday :)03:35
stubok... not obvious in the web ui that the description is fresh.03:35
stuboic03:35
stubJust looking at one of the example queries in the desc, but TranslationMessage.variant no longer exists.03:45
lifelessstub: yeah, I kept most of jtv's analysis because I wasn't sure how much was still relevant or not03:50
lifelessspeak of the devil03:50
jtvuh-oh03:51
StevenKHaha03:51
jtvwhat is this about?03:51
jtvam I liable?03:51
stublifeless: how long before we make decisions on moving the translations to Cassandra or not? Is that a 6 month or multiyear change do you think?03:51
stubjtv: Analyzing https://bugs.launchpad.net/launchpad/+bug/53420303:51
_mup_Bug #534203: Timeouts on POFile:+filter (filter by person) <lp-translations> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/534203 >03:51
lifelessstub: making a decision we could do now; scheduling the work ...03:51
jtvAh03:52
lifelessstub: I have some thoughts on restructuring the app stack so that we can carve off parts of the system03:52
lifelessstub: I want to finish a current analysis I'm on, but then if you're up for the weekly call I can run it past you03:52
stubSo, if it stays in PG, partitioning the data on language makes sense. But that is laborious and time consuming to set up, especially if we are avoiding large outages.03:53
stubAnd will make all-language queries slower, but I don't think we have any of them.03:54
stubAnother alternative is language specific partial indexes.03:55
stubProbably tm__english__sumitter__idx ON TranslationMessage(submitter) WHERE language=1; for all hundred odd languages. We could combine too for lesser languages, but I suspect better just to create one index per language and put up with the noise.03:56
jtvWould it make sense to cluster on language?  That wasn't an option in the past, but is now.03:57
stubjtv: The queries I'm looking at, yes. But I don't have a comprehensive overview on the queries.03:58
lifelessjtv: what is translationmessage.potemplate03:58
jtvlifeless: that means "this message is not shared, but rather diverged for this specific template."03:58
lifelessjtv: so if its null many different templates may use the message?03:59
jtvYes.03:59
lifelessjtv: and if its not null the list of templates is 103:59
jtvIt should be mostly null.03:59
lifelessjtv: is it going to be deleted?03:59
jtvNo.03:59
lifelessok03:59
stubThe schema migration is all done now btw?04:00
lifelessso this page - is it showing /who/ contributed to the translation, or /what translations/ the person did?04:00
stubI think it was completed.04:00
jtvstub: you mean for translations message sharing?  We still have a few obsolete database columns but I don't think we use them now.04:00
stubjtv: Do you know if there are bugs open on dropping them?04:00
jtvlifeless: pofile:+filter shows a particular person's contributions.04:00
jtvstub: some, I think04:01
LPCIBotProject windmill-devel build #28: STILL FAILING in 41 min: https://lpci.wedontsleep.org/job/windmill-devel/28/04:01
lifelessjtv: the tti.sequence component in the sort seems unuseful to me04:01
lifelessjtv: or do I misunderstand the model04:01
jtvlifeless: sequence expresses the order in which the translatable strings occur in the template.  Even to someone who doesn't know or care about the text of the template, it conveys context that may be useful.04:02
jtvI'm loading up the bug so I can be more specific.04:02
stubbwahaha... there is a blessed SQL 'hack' that makes that final pastbin work in 1% of the time04:03
lifelessjtv: so we're looking at a users contributions to the one project, is the sequence an ordinal in the file?04:04
lifelessjtv: if it is, the the date sort is useless04:04
lifelessjtv: if it isn't, then wouldn't most-recent-first be simpler?04:04
jtvlifeless: both sort criteria are needed.04:04
lifelesswhy?04:04
jtvA single user may have submitted multiple translations for the same message.04:05
lifelessand we show them all? even rejected/old/stale ones?04:05
jtvYes.  With indication of status.04:05
stubcompare https://pastebin.canonical.com/47090/ with https://pastebin.canonical.com/47136/ . The latter is identical except for the OFFSET 0 in the subquery, which is an unofficial query hint that lots of people use. Now I know why.04:05
lifelessgrah04:05
jtvThis is useful in evaluating a translator's work.04:05
lifelessjtv: it slowly degrades over time though04:06
stubc/query hint/planner hint04:06
jtvIt's also useful for a translator to get insight into review status.04:06
lifelessstub: !!04:06
lifelessstub: that may be hot04:07
stublifeless: it is. I get 20 s with the original query.04:07
lifelessstub: can you do that explain on the standalone replica ?04:07
lifelesswhich should be cold04:07
stubcold, 5ms04:08
stubThis is silly04:08
* stub double checks his query04:08
lifelessthe 84756 lookups in the related table are the issue in the first place04:08
stublifeless: standalone is being build. The queries are strangely fast when the tables exist and have 0 rows.04:09
lifelesshhhaaaa04:09
lifelessok04:09
jtvI suspect the difference is mainly due to the planner selecting by language first and by user after, which the optimization barrier inverts.04:10
jtvI'd expect more selectivity from selecting by translator first and then by language.04:10
lifelessdoing subselect vs join also did that04:10
lifelessplans are in the bug IIRC04:11
lifelessstub: on qastaging04:12
lifeless explain select count(*) from bug where latest_patch_uploaded is not null;04:12
lifeless                            QUERY PLAN04:12
lifeless------------------------------------------------------------------04:12
lifeless Aggregate  (cost=644851.46..644851.47 rows=1 width=0)04:12
lifeless   ->  Seq Scan on bug  (cost=0.00..644777.55 rows=29562 width=0)04:12
lifeless         Filter: (latest_patch_uploaded IS NOT NULL)04:12
lifeless    "bug__latest_patch_uploaded__idx" btree (latest_patch_uploaded)04:12
lifelessstub: its indexed, and a small % of rows have that set:04:13
lifeless select count(*) from bug where latest_patch_uploaded is not null;04:13
lifeless count04:13
lifeless-------04:13
lifeless 2956204:13
lifelessstub: so why would it do a table scan ?04:13
stubsmall = 5-10%04:14
lifelessyeah, we have 800K bugs, 30K is < 5%04:15
lifelesswith seqscan off:04:16
lifeless Aggregate  (cost=10000644851.46..10000644851.47 rows=1 width=0)04:16
lifeless   ->  Seq Scan on bug  (cost=10000000000.00..10000644777.55 rows=29562 width=0)04:16
lifeless         Filter: (latest_patch_uploaded IS NOT NULL)04:16
lifelessit doesn't think the index is valid?04:16
lifelessstub: can you reproduce, and/or see if prod has the same issue?04:16
stublifeless: possibly because the bug table is pretty much pinned in RAM04:16
stubbut yeah, on prod seq scan for pulling 4% of the bugs04:17
stublifeless: probably fixable by making it partial - WHERE latest_patch_uploaded IS NOT NULL04:18
lifelessexplain select id from bug where latest_patch_uploaded is not null order by latest_patch_uploaded ; grabs the index04:19
lifelessstub: could you add a test index like that to qastaging ? I'm working on the +patches perf bug04:19
stublifeless: And does explain analyze show it takes longer real time?04:19
lifelesschecking04:19
stubBuilding the index atm, so your checks will suck.04:21
lifelesshttp://pastebin.com/FBRt1HPr04:21
lifelessits using your new index :>04:22
lifelesscan you run those two to compare on prod ?04:22
stubSo that is quite interesting. See that PG correctly picked the plan that starts returning rows soonest? The 'order by' query runs slightly faster to completion, bug doesn't start returning rows for 240ms04:23
stubwhich is rather stupid when this is a COUNT()04:24
stubBut the estimated time for the second query is higher than the first, so understandable. And the differences probably statistically insignificant.04:24
lifelessstub: right, I couldn't compare counts to counts, so changed it to get th eid04:25
lifeless               ->  BitmapAnd  (cost=43791.26..43791.26 rows=23966 width=0) (actual time=16801.315..16801.315 rows=0 loops=1)04:25
lifeless                     ->  Bitmap Index Scan on bug__latest_patch_uploaded__idx2  (cost=0.00..397.06 rows=29562 width=0) (actual time=15.446..15.446 rows=29562 loops=1)04:25
lifeless                     ->  Bitmap Index Scan on bug_duplicateof_idx  (cost=0.00..43382.43 rows=577826 width=0) (actual time=16773.721..16773.721 rows=578095 loops=1)04:26
stubpartial index built and table analyzed btw. Look for '_idx2' in your plans04:26
lifelessyeah04:26
lifelessstub: could you add a similar one for duplicateof ?04:30
lifeless(where duplicateof is null)04:30
lifelessI don't know that that will be sufficient04:30
lifelessor...04:30
lifelessperhaps index id where duplicateof is null and latest_patch_uploaded is not null04:31
lifelessI wonder if we need to denorm the openness or not of a bug into the bug04:32
lifelessthis is the bug I'm looking at https://bugs.launchpad.net/launchpad/+bug/61839204:32
_mup_Bug #618392: Distribution:+patches slow 10% of requests timing out <lp-bugs> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/618392 >04:32
stubWe have some indexes elsewhere on id WHERE [partial clause] to speed up joins and ordering - may well work here too.04:35
stubduplicateof index built and table analyzed04:35
lifelessnope, unused04:37
stuband the id one04:37
lifelessah, with just duplicates by id04:38
lifelessthis is what its doing:04:38
lifeless               ->  BitmapAnd  (cost=43791.26..43791.26 rows=23966 width=0) (actual time=20175.537..20175.537 rows=0 loops=1)04:38
lifeless                     ->  Bitmap Index Scan on bug__latest_patch_uploaded__idx2  (cost=0.00..397.06 rows=29562 width=0) (actual time=16.256..16.256 rows=29562 loops=1)04:38
lifeless                     ->  Bitmap Index Scan on bug_duplicateof_idx  (cost=0.00..43382.43 rows=577826 width=0) (actual time=20146.004..20146.004 rows=578095 loops=1)04:39
lifeless                           Index Cond: (duplicateof IS NULL)04:39
lifelessthe expensive  bit is pulling all the bits out from the nondupes04:39
lifelesscould you drop all these indices04:39
lifelessand try04:39
lifeless(id) where duplicateof is null and latest_patch_uploaded is not null04:39
lifelessOTOH this may be index bloat I guess04:40
stubwhoops... I'd created duplicateof index IS NOT NULL, not IS NULL04:41
lifeless:>04:41
lifelessok, so lets try fixing that first04:41
lifelessgeneral purpose things being better, all other stuff equal04:41
lifelessbooyah04:43
lifelessthats fast04:43
stubYou now have two indexes - on id - WHERE duplicateof IS NULL , and WHERE duplicateof IS NULL AND latest_patch_uploaded IS NOT NULL04:43
stubwhich one gets used?04:43
lifeless         ->  Index Scan using bug__new_patches__idx on bug  (cost=0.00..1013500.47 rows=23045 width=4) (actual time=0.086..93.803 rows=28607 loops=1)04:44
lifeless500ms for the batch, 300ms for the count04:44
lifelessdo you think this is reasonable to add live ?04:45
stubsure04:45
lifeless\o/04:45
lifeless-65-1.sql ?04:45
stubfirst, I'll drop the more complex index. Lets see if it is nearly as fast with just the id WHERE duplicateof IS NULL04:45
lifelessok04:45
stubThat index will be useful for other stuff too04:46
lifelessyeah04:46
stublifeless: ok. try now.04:46
stubyeah - 65-104:46
lifelessTime: 3089.849 ms04:46
lifelessfaster than it was without it I think,04:47
lifeless Aggregate  (cost=7085778.93..7085778.94 rows=1 width=0) (actual time=2905.470..2905.471 rows=1 loops=1)04:47
stubok. crappy time. Wonder if I should create both indexes or just the fast one for this case?04:47
stubor is the time close?04:47
lifeless2.5 second difference04:47
stubright. crappy time :)04:48
lifeless500ms vs 3000ms04:48
lifelessI suggest just the tuned one04:48
lifelessless chance of destablising some other query04:48
stubShould be impossible to destabalize - if it is possible to use the partial index, it will be faster because it is smaller. Assuming no memory pressure on the indexes because index size is insignificant compared to total RAM.04:49
stubBut from change management pov, yeah.04:49
LPCIBotProject windmill-db-devel build #235: STILL FAILING in 1 hr 5 min: https://lpci.wedontsleep.org/job/windmill-db-devel/235/04:49
lifelessstub: https://bugs.launchpad.net/launchpad/+bug/735977 appears to be a destablisation result to me04:50
_mup_Bug #735977: MaloneApplication:+bugs timeouts searching for tags across all bugs <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/735977 >04:50
lifelessstub: so my paranoia is heightened04:50
stubI'll do that now and repack bug_fti while I'm at it.04:50
lifeless\o/04:50
lifelesspaste me the sql04:50
lifelessand I'll make it into a branch04:50
stubI'll do the -1 patch if I may - I do the patch first then translate it to live statements to run on the slaves.04:51
stubless chance of screwups04:52
lifelessI'm always happy for someone else to do work04:53
lifelessonce you've done this lets do the catchup call04:53
LPCIBotProject devel build #689: FAILURE in 5 hr 7 min: https://lpci.wedontsleep.org/job/devel/689/04:53
stubI should flag this patch as fixing Bug #618392 yes?04:58
_mup_Bug #618392: Distribution:+patches slow 10% of requests timing out <lp-bugs> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/618392 >04:58
lifelessyes04:58
stubprod_1 done05:07
lifeless361 queries/external actions issued in 2.65 seconds05:08
lifeless\o/05:08
stubprod_2 complete05:08
lifelesswiiin05:09
stubprod_3 taking its time05:09
* stub checks for long running transactions - probably the backups at this time05:09
stubyer... prod_3 blocked until the backups stop or I kill 'em05:11
lifelessthats master right ?05:11
stubyes05:11
lifelessup to you05:12
lifelessare the backups offsite?05:12
spms/u/a/ <== stub05:12
=== Ursinha is now known as Jorjao
=== Jorjao is now known as Mariazinha
stubspm: eh?05:19
stubThe reapers are killing archivepublisher jobs for about a day now.05:20
spmstub "... or I kill 'em" ergo stub ==> stab. s/u/a/ :-)05:20
stublifeless: I think we can wait another 3 hours until they complete.05:21
lifelesssure05:22
lifelessso, call?05:22
stublifeless: sure05:22
* stub kills his u1 sync05:22
stubSo I installed wonderwall as a workaround for some throttling issues, and it seems to be helping my system a lot. Lets see if this call is stable - Skypeout was crappy yesterday before that install.05:24
lifelessI'm here05:27
lifelessstub: I'm here when your network comes back :>05:34
lifelessstub: I can hear you mostly fine05:34
stubSre. I'm watching the network monitor. I'm seeing a sine wave05:34
stubLike there is some sort of really crappy traffic shaping going on, or I'm only getting05:34
lifelessof traffic or signal ?05:34
=== cinerama_ is now known as cinerama
lifelessstub: brewaking up?05:47
lifelessstub: hows the graph?05:47
stubstill broken05:47
stubthink it is isp conjestion05:47
LPCIBotProject windmill-devel build #29: STILL FAILING in 1 hr 4 min: https://lpci.wedontsleep.org/job/windmill-devel/29/05:54
lifelesssine wave?06:08
lifelessstub: another 'fun' one for you - https://bugs.launchpad.net/launchpad/+bug/77760107:12
_mup_Bug #777601: BugComment:EntryResource timeouts <dba> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/777601 >07:12
stublifeless: so that was qa staging yesterday. Was something happening there at 11:18:48 UTC like a database update?07:24
lifelessstub: huh, that was prod07:44
stubreally?07:45
lifelessOOPS-1950DQ23107:46
lifelessDQ07:46
stuboic. 'last seen' was qastaging07:46
lifelessso (IIRC) wampee07:46
stubI saw QASTAGING in big type :)07:46
lifeless:>07:47
stubI'm not aware of any activity that would explain that. I'll need to dive into the logs.07:48
stubBut first a shower. I'm all sticky.07:49
StevenKTMI, stub07:49
allenaplifeless: Okay, I'll follow that up (the sync_packages thing). Nothing is creating jobs that use the sync_packages user yet so it's good someone saw this now before we do start creating jobs. Thanks.09:02
bigjoolsmorning dudes09:03
mrevellhello!09:05
StevenKallenap: O hai! Can you look at http://pastebin.ubuntu.com/603553/ ? It's the last bit for the mega-branch you've been reviewing.09:07
StevenKallenap: This cleans up the UI tests to work09:07
allenapStevenK: Sure.09:09
bigjoolswgrant: did you see the MP I assigned you?09:20
allenapStevenK: _create_child_and_parent() is defined several times. If they're all the same perhaps you could factor them out?09:24
allenapStevenK: Otherwise it looks good, r=me.09:24
StevenKallenap: Thanks!09:25
StevenKallenap: I was not amused at all of the duplication of derived_series = ...09:25
allenapNo :)09:25
=== almaisan-away is now known as al-maisan
LPCIBotProject windmill-devel build #30: STILL FAILING in 1 hr 4 min: https://lpci.wedontsleep.org/job/windmill-devel/30/09:32
wgrantbigjools: Ah, sorry missed it in the middle of lots of other review emails that I didn't care about.09:45
wgrantLooking now.09:45
bigjoolswgrant: heh, I unsubscribed from the review list a while ago09:45
wgrantbigjools: Hmm, that simplifies the code a little bit, doesn't it...09:46
bigjoolsa tad09:46
bigjoolsdunno what that guadalinux stuff was in there for09:47
wgrantFor DDs with binary copies, without considering at all how it would work.09:47
wgrantStevenK/anyone: https://code.launchpad.net/~julian-edwards/launchpad/slow-build-hunt-bug-777183/+merge/59968 would like to be mentored.09:48
lifelessallenap: cool thanks09:48
adeuringgood morning09:50
* stub wonders why launchpadlib is pulling credentials from kwallet when he is using Unity10:04
bigjoolsthe irony, I am using KDE and it pulls them from gnome-keyring10:04
LPCIBotProject devel build #690: STILL FAILING in 5 hr 11 min: https://lpci.wedontsleep.org/job/devel/690/10:04
bigjoolslifeless: around?10:05
stubI wonder if it remembers somewhere where you first used it?10:05
bigjoolsIIRC it depends on the ordering of importing dependent modules, whichever works first it uses10:05
bigjoolsyou can make a .keyringrc as well10:06
stubI'm sure there must be an environment variable to sniff for a heuristic.10:06
bigjoolsjust the config I think10:06
bigjoolsStevenK: still there?10:07
bigjoolsok, allenap?10:08
lifelessbigjools: vaguely10:09
* stub wonders what a .keyringrc file looks like10:09
bigjoolslifeless: if you have time, can you mentor wgrant's review of my branch please?10:10
bigjoolshttps://code.launchpad.net/~julian-edwards/launchpad/slow-build-hunt-bug-777183/+merge/5996810:10
bigjools81 lines, mostly removals10:10
bigjoolsstub: the file's actually called keyringrc.cfg10:16
bigjoolsyou can use it to override the backend10:17
bigjoolsI think it's supposed to live in ~ which is somewhat annoying10:17
=== allenap changed the topic of #launchpad-dev to: Merge to devel are open again | https://dev.launchpad.net/ | On call reviewer: allenap | https://code.launchpad.net/launchpad-project/+activereviews
bigjoolsallenap: ooo I had no idea PG had a notification mechanism11:03
bigjoolsstub: apparently you didn't like it though11:03
stubbigjools: Unity? I worked out how to turn off the scrollbars that were driving me batty when they popped up and am now starting to appreciate it. KDE was nice, particularly plasmoids built in and working and some decent ones shipped, but still a little busy for me or something.11:05
bigjoolsstub: I was talking about http://www.postgresql.org/docs/8.4/interactive/sql-notify.html11:06
bigjools:)11:06
stuboh... notification. Its fine. The only reason I don't like it is we really shouldn't be spending time reimplementing rabbitmq badly.11:06
bigjoolsmaybe11:06
bigjoolsbut considering we already did that anyway ...11:07
bigjoolsJob *cough*11:07
stubbigjools: I can't quite recall if it works outside transaction boundaries - if LISTEN is stuck inside a transaction, we could be shooting ourselves in the foot with long running transactions.11:07
bigjoolsI was thinking more of doing it in a twistd process, we have a bit more control over transactions11:08
stubBecause if we do something wrong once, we should keep making the same mistake until everyone believes it is the right thing to do.11:08
bigjoolsheh11:08
stubOr am I thinking of politics?11:08
lifelessif you want queues11:21
lifelessland my rabbitmq branch11:21
lifeless:)11:21
bigjoolslifeless: well we keep adding polling jobs everywhere, it's about time ...11:26
lifelessI have a branch, tests all pass locally, something in ec2 hates it and I haven't had time to debug11:27
rvbawgrant: bigjools: Just pushed the modifications to the branch William marked as bad earlier today. Could you have a look? (http://bazaar.launchpad.net/~rvb/launchpad/change-perm-sync/revision/12950)11:30
rvbaI changed the MP's status back to WIP but the partial diff does not get updated.11:31
bigjoolsrvba: you need a new branch + MP11:32
rvbaok11:33
bigjoolsrvba: I would add a comment explaining why lp.append is needed for the security updates11:33
LPCIBotProject windmill-db-devel build #236: STILL FAILING in 1 hr 5 min: https://lpci.wedontsleep.org/job/windmill-db-devel/236/11:33
bigjoolsit's a bit of a hack unfortunately11:34
wgrantbigjools, rvba: I think we will be able to remove the hack this year, but until then it needs to stay (and probably have a test)11:34
bigjoolsyeah he has a test11:35
bigjoolsrvba: I would move that test somewhere else though, let me think11:35
wgrants-i-s seems to be basically P3As + a stack of hacks that all need to be genericised for DDs to work properly.11:35
lifeless'special cases'11:36
bigjoolsrvba: lib/lp/soyuz/scripts/tests/test_copypackage.py11:37
wgrantlifeless: Have you *seen* them?11:37
lifelesssome11:37
rvbabigjools: ok11:37
lifelessjust noting that they may me lacking the 'elegant solution to a non trivial problem' aspect of 'hack'11:37
bigjoolsrvba: I'd be tempted to also move the syncing tests out of test_distroseries11:38
bigjoolsthat's not the first place I'd look for them11:38
bigjoolsbut the waters are already muddied by this feature being half in lp.registry and half in lp.soyuz11:38
wgrantIt should be entirely in lp.soyuz, or the trees should be merged :/11:39
wgrantWhy was it in lp.registry to start with?11:39
bigjoolswgrant: noodles put the DSD stuff there11:39
wgrantAh, right.11:39
wgrantThat seems so long ago now :/11:39
lifelesswgrant: are you around tomorrow?11:39
bigjoolsabout a year :)11:39
wgrantlifeless: Unless they've instantiated a new public holiday while I wasn't looking.11:39
bigjoolsthat happened here!11:40
stubWe only get that on military coups.11:40
bigjoolsThe Royal Wedding could be considered a military coup11:40
lifelesswgrant: I'd like to run some concepts I'm tossing round - starting to socialise - past you; I'd use bigjools but ETIMEZONE11:41
rvbabigjools: right, I'll leave one to check for UI notifications and see if some of the other tests can be moved to test_copypackages.11:41
lifelesswgrant: so skype/mumble at some convenient point of th day for you11:41
bigjoolsrvba: thanks11:41
bigjoolslifeless: I can be around later today my time if you want11:41
lifelessbigjools: I'm doing big picture sketching and looking to understand more of the bits of soyuz that glue together11:42
bigjoolslifeless: you definitely want me then :)11:43
wgrantlifeless: Sure, whenever.11:43
lifelessbigjools: I'd be delighted, but I plan to sleep till i wake tomorrow, after the 5:45 start this morning11:43
lifelessbigjools: I don't want to interrupt your evening unnecessarily11:43
wgrantSleep till you wake.11:43
wgrantWhat a novel concept.11:43
rvbabigjools: btw, I'm not sure I know why exactly "lp.append is needed for the security updates" ... it's a hack all right ... but why is it there?11:44
bigjoolslifeless: nae prob11:44
lifelesswgrant: theres a book on it11:44
bigjoolsrvba: it was the only reasonable way for the security team to get permission to copy to -security at the time11:44
lifelessbigjools: if you are around in my morning and want to chat, I'd be delighted; from experience (this will be the 4th/5th time I've run it up the flagpole) its about a 45-60 minute topic11:44
bigjoolsheh11:44
lifelessbigjools: failing that, we can make a time next week if you like11:45
bigjoolsI'll pop in to irc tonight11:45
wgrantrvba: Basically, the security team needs to be able to copy any package into the Security pocket, even if they lack upload privileges. And *only* the security team can be permitted to use syncSource into the primary archive, since it bypasses the queue and announcements.11:46
lifelessbigjools: to give you context, with performance a Solved Problem, I'm starting to look longer term at our architecture11:46
lifelessbigjools: at how we evolve the system, what platform we want to be on, and how we can break out of the monolithic situation we're in safely11:46
rvbawgrant: bigjools ok. Thanks, I'll had a comment then.11:46
bigjoolslifeless: excellent, I have a lot of ideas and semi-plans about that11:46
wgrantlifeless: I am relieved to hear that you are looking at this.11:46
rvbas/had/add/11:46
wgrantrvba: Thanks!11:47
bigjoolsa lot of our architecture was designed around more machines being in the pipeline11:47
bigjoolsrvba: thank you11:47
lifelessbigjools: its nearly 11pm and I've worked straight through except for dinner with Lynne; I guarantee 0% retention of anything you say now :>11:47
wgrantrvba: We'll want to redo what you did in a few months, I suspect. But we can't get ahead of ourselves :(11:48
bigjoolslifeless: I hear you11:48
bigjoolswgrant: baby steps11:48
lifelessanyhow, I'll be active here again in 9ish hours.11:48
lifelessnight all11:48
bigjoolsnn!11:48
wgrantNight lifeless.11:48
wgrantlifeless: I'd like to be around for this specialised deploy, so please don't try it before I'm around.11:49
=== henninge is now known as henninge-lunch
bigjoolswgrant: just sent you email so you can sanity check the steps I think are necessary for syncs via the queue12:01
wgrantbigjools: Looks sane.12:04
wgrantbigjools: We'll need to shuffle the upload policy stuff around a bit... but it should be pretty much OK.12:04
wgrantI think +queue and the queue tool should mostly work, as long as PackageUpload.displayname works.12:04
bigjoolsyeah, I want to do the policy stuff in such a way that it's easy to convert for generic suites12:04
stubSpeaking of holidays, apparently it is coronation day and I'm not supposed to be here. I guess I'll be slacker than usual tomorrow.12:04
wgrantAnd acceptFromQueue does the right thing,12:05
wgrantBut if not, easy enough to fix.12:05
bigjoolsstub: how will we notice?  /me hides12:05
bigjoolswgrant: right, the change will be in acceptFromQueue12:06
bigjoolsalthough I expect we'll need to amend both for display purposes so it says it's a sync12:06
wgrantYeah.12:07
wgrantbut it's not going to be as hard as I have long feared.12:07
wgrantMostly because of that convenient job.12:07
wgrantStevenK: Did you get anywhere with the overrides?12:14
bigjools20 bugs away from #77777712:15
wgrantbigjools: Have you had any more thoughts about how to arrange the generic copier underpinnings?12:19
bigjoolsno12:20
bigjoolstoo busy thinking about this project :)12:20
wgrantYeah.12:20
bigjoolsnow, where's the damn table for copy jobs12:20
wgrantIs it DistributionJob or ArchiveJob or similar?12:21
wgrantlib/lp/soyuz/model/packagecopyjob.py:class PackageCopyJob(DistributionJobDerived):12:21
bigjoolsBug #77628312:22
_mup_Bug #776283: PackageCopyJob needs separate archive and series columns for query purposes <derivation> <Launchpad itself:Triaged> < https://launchpad.net/bugs/776283 >12:22
bigjoolsit uses DistributionJob12:22
wgrantgmb: Well done!12:45
gmbwgrant: Yeah. Trigger happy, premature filing.12:45
gmbThe failure state of "clever" is "arsehole"12:46
wgrantYeah.12:46
wgrantStill, 4 bugs to go!12:46
gmbI'll leave it to mpt.12:46
gmbAnd stick to being doggedly successful instead of smart.12:46
wgrantThat sounds like a plan.12:49
wgrantAny Yellow people want to handle the bug in #launchpad?12:49
wgrantYUI doesn't seem to like + in an id selector.12:49
wgrantbigjools: Are you going to separate the two different facets of upload checking?12:55
wgrantSome of the checks are uploader policy (eg. signature checks), while others are distroseries acceptance policy.12:56
wgrantThey are currently conflated, but that doesn't really work.12:56
=== henninge-lunch is now known as henninge
wgrantIs unmuting a bug meant to give me the option to unsubscribe the team?13:21
wgrantThat seems sort of the opposite action...13:21
bigjoolswgrant: yes13:33
bigjoolsuploader policy is fixed for the most part13:34
wgrantbigjools: Right. Or it's controlled by an option on process-upload.13:40
wgrantNot series-specific.13:40
bigjoolsexactly13:40
wgrantSo it must be split.13:40
bigjoolsthat's the plan13:40
wgrantExcellent.13:41
bigjoolsanyone know if you can change the billing date for AWS?13:41
wgrantYou mean so it appears more than a day before expenses are due?13:41
wgrantThat would be lovely.13:41
wgrantBut I don't think so.13:41
bigjoolsthey bill just at the exact wrong time for me13:41
henningedanilos: could you comment on bug 196679, though?13:46
_mup_Bug #196679: ValueError on +language-packs <lp-translations> <oops> <Launchpad itself:In Progress by henninge> < https://launchpad.net/bugs/196679 >13:46
henningedanilos: I mean, do you have a hint for me if this still needs to be done?13:47
daniloshenninge, I thought jtv and you got the hang of it (I remember seeing his comment about how it's still there somewhere)13:47
henningedanilos: in a mail but it sounded much like guess work on his part.13:47
henningeJust wondering if you had more insight.13:47
daniloshenninge, let me take a look13:48
wgrantbigjools: Have you seen the repeated archivepublisher transaction killings over the past couple of days?13:49
LPCIBotProject windmill-db-devel build #237: STILL FAILING in 1 hr 6 min: https://lpci.wedontsleep.org/job/windmill-db-devel/237/13:49
bigjoolswgrant: no, I'm not paying any attention to oopses13:49
bigjoolsand, BTW, ****ing keyboard drivers13:50
wgrantbigjools: Not OOPSes, but pgkillidle spam. The first was on the 3rd, and there have been ~8 since.13:50
wgrantI can't find anything in the logs.13:50
wgrantDon't even know which host it's from.13:50
bigjoolswgrant: I suspect it's to do with the timeout getting lowered13:50
bigjoolsif that happened yet13:51
wgrantExtremely unlikely.13:51
wgrantThese transactions had been idle for half an hour.13:51
bigjoolsthat it happened?13:51
wgrantIt's been 9s for a week now.13:51
bigjoolsnot that timeout13:51
bigjoolsthere's a script timeout IIRC13:51
wgrantHmm.13:51
daniloshenninge, it seems it should be able to reproduce the bug with more than 15 language packs (shortlist default), and the base pack not being in the list that shortlist returns, as jtv noted as well13:51
bigjoolsI vaguely remember stub saying something about lowering it and panicking13:51
bigjoolsand then forgot about it13:51
wgrantHeh13:52
henningedanilos: I see, I'll try that. thanks13:52
bigjoolswgrant: so I'd check with stub to see if he changed anything13:53
stubeh?13:54
stubwgrant: I'll grab the hosts for you - they should be in the pg logs13:54
wgrantstub: Oh, that would be great, thanks.13:54
daniloshenninge, I think the right fix would be to use a larger shortlist value there (find what's the most we had for a distro release)13:54
wgrantstub: Then we can hopefully identify the process from the ps logs.13:55
=== jcsackett changed the topic of #launchpad-dev to: Merge to devel are open again |
jcsackett          https://dev.launchpad.net/ | On call reviewer: allenap |13:58
allenapOops.13:58
allenap:)13:58
jcsackettcrap, i didn't even hit enter.13:58
=== jcsackett changed the topic of #launchpad-dev to: Merge to devel are open again | https://dev.launchpad.net/ | On call reviewer: allenap, jcsackett | https://code.launchpad.net/launchpad-project/+activereviews
deryckMorning, all.14:00
deryckhenninge, ping for standup14:01
bigjoolswgrant, I am going to start the mawson DB upgrade later today, you don't need it do you?14:03
LPCIBotProject windmill-devel build #31: STILL FAILING in 1 hr 5 min: https://lpci.wedontsleep.org/job/windmill-devel/31/14:04
=== matsubara-afk is now known as matsubara
wgrantbigjools: I need to do some minor QA on it tomorrow.14:05
wgrantbigjools: So probably best to run it over the weekend...14:05
wgrantOr I can start it when I'm done tomorrow.14:05
wgrantThat's probably best.14:05
bigjoolswgrant: what QA?14:06
wgrantr1297914:06
bigjoolsI need DF *done* before end of Sunday so I can have it ready for monday14:06
wgrantbuildd-manager path change.14:06
wgrantAh, true.14:06
bigjoolsyou can QA that on staging14:06
wgrantI guess we can do this on staging.14:06
wgrantYeah.14:06
wgrantOK, blow it away!14:06
bigjoolscheers :)14:06
wgrantI guess we also need to clean the librarian out, but can leave most of the archive.14:07
bigjoolslibrarian purging is part of my monkey-following script14:07
wgrantGreat.14:07
=== Ursinha-afk is now known as Ursinha
bigjoolswgrant: for this policy checker I am thinking to sort of copy the uploadpolicy one but makeit  a) use DB objects, b) only have distro policy checks.  Then add a PackageUpload.checkPolicy(policy) or something.14:11
wgrantbigjools: Can't it basically be encapsulated in DistroSeries or Archive?14:12
wgrantThere's only one or two methods that need to be exposed.14:12
bigjoolsno14:12
bigjoolswell14:12
bigjoolsI'd rather it was a totally separate module14:12
bigjoolsthen it can be re-used in more ways14:12
wgrantDo we need to replace anything other than autoApprove and autoApproveNew?14:13
bigjoolsat the moment, no, but I expect it to grow14:14
wgrantRight.14:14
bigjoolsrejectPPAUploads, for example14:14
bigjoolsmaybe announcelist14:14
wgrantMost of that is archiveuploader-specific.14:14
wgrantExcept for announcelist, which is just wrong.14:14
bigjoolsit should be distro-specific14:15
bigjoolss/it/they/14:15
bigjoolsnow,  the hardest part has just arrived.  What do I call it and where should it live :)14:20
wgrantIt is similar to archivedependencies, so it might belong alongside it in adapters/14:20
bigjoolsadapters is a horrible name but there's nowhere else better that I can see14:21
bigjoolsI mean, packagecopier is in scripts/  ffs14:22
wgrantHmm, I argued with you about packagecopier a while ago and you said it belonged in scripts/ :)14:24
bigjoolsdid I? :)14:26
deryckhenninge, we don't hear you any more14:26
henningederyck: I still hear you14:26
bigjoolswgrant: bugger, test failure from the getBuildByArch change14:53
wgrantbigjools: Ew, which?14:54
bigjoolspublishing.txt and testDuplicatedBinaryUploadGetsRejected14:59
wgrantHmm. The second one is a braindead test.15:03
wgrantCopying to the same distro and archive and expecting a new build to be created.15:03
wgrantNonsensical.15:03
bigjoolsnot look at it yet, just going in now15:04
wgrantpublishing.txt seems to be the same.15:04
benjiabentley: will you be able to QA bug 747844 today?  I have a revision behind it that I'd like to get released.15:05
_mup_Bug #747844: process-mail oopses with TypeError due to non US-ASCII chars in the email address <oops> <process-mail> <qa-needstesting> <Launchpad itself:Fix Committed by abentley> < https://launchpad.net/bugs/747844 >15:05
wgrantIt copies from one series in a PPA to another.15:05
wgrantANd expects builds to be created.15:05
bigjoolswgrant: huh, the description is insane15:05
abentleybenji: yes.  I am QAing it right now.15:05
bigjools"source gets copied to another suite in the same archive..... The binary rebuild"15:05
bigjools!15:05
benjiabentley: great! thanks15:05
bigjoolswgrant: that's completely bong, wtf15:06
wgrantbigjools: Yeah, so it asserts that the duplicated build will fail to upload.15:06
wgrantbigjools: It's a good check.15:06
wgrantBut it relies on a bug: that the build is created at all.15:06
wgrantNeed to create the build manually.15:06
benjirvba: will you be able to QA bug 775529 today?  I have a revision behind it that I'd like to get released.15:06
_mup_Bug #775529: Synching to a released series should put packages in -updates. <derivation> <qa-needstesting> <Launchpad itself:Fix Committed by rvb> < https://launchpad.net/bugs/775529 >15:06
wgrantAnd publishing.txt needs to be reworded to use a different archive.15:06
wgrantbigjools: DF is still usable?15:06
wgrantI'll QA that thing now.15:06
bigjoolswgrant: yes, still waiting for the dump15:07
rvbabenji: I would need DF to be up for that ... bigjools?15:07
rvbabigjools: can you update DF?15:07
bigjoolsit's up, but we can defer QA since it won't block rollout15:08
bigjoolsmark it untestable for now15:08
rvbabenji: bigjools will do.15:08
benjiawesome, thanks15:08
wgrantbenji: You need to deploy at least 12980 due to a bad rev, and 12979 has an unusual rollout requirement on LPS -- could you make sure that's done if you request one before I'm around again?15:08
benjiwgrant: I'm doing a lazy release.  I'm just lining things up so my revision will be ready when you do your deploy.15:10
wgrantbenji: Ah, OK. That makes things easier.15:10
benjifor the both of us ;)15:11
LPCIBotYippie, build fixed!15:13
LPCIBotProject devel build #691: FIXED in 5 hr 8 min: https://lpci.wedontsleep.org/job/devel/691/15:13
abentleylosa: please run process-mail for qastaging15:16
hloeungabentley: hi, give me a sec...15:17
abentleyhloeung: thanks.15:17
wgrantAh, I was wondering why DF was so slow.15:19
wgrantDriven into iowait by a network file copy. Yay.15:20
hloeungabentley: done... see https://pastebin.canonical.com/47159/15:21
wgrantbigjools: I'm giving up for now, will do it on staging tomorrow. Do with mawson what you will.15:22
abentleyhloeung: Hmm.  That doesn't look right, and it hasn't generated the outgoing mail I expected, either.15:23
abentleyhloeung: that was definitely qastaging, not staging?15:23
hloeungabentley: yep, qastaging15:23
bigjoolswgrant: ok15:25
abentleyhloeung: could you try running it on staging, please?15:26
hloeungabentley: process-mail runs every 10 minutes on both staging and qastaging15:27
abentleyhloeung: Oh.  I was under the impression that no scripts run on qastaging.15:27
abentleyhloeung: I sent mail to 5977@bugs.staging.launchpad.net more than 20 minutes ago, and it still hasn't appeared in the staging mail folder.15:30
bigjoolswgrant: publishing.txt is also on crack15:38
bigjoolsit's copying intra-archive and expecting rebuilds15:38
jtvhenninge, mrevell: one very small note about the sharing announcement: where it says "That sharing also works between different versions of a project or package:" it may be helpful to say "already" instead of "also" (to set the right expectation, i.e. that this is not the part that has changed most recently).15:43
mrevelljtv, Thanks, that's a good suggestion.15:44
jtvAlso, "series" is one of those things that are obvious to us devs but I'm worried about how clear they are to others.  I'd prefer to say "release series" a lot of the time to avoid leaving those "not in the know" behind.15:45
jtvmrevell, in case you were already off typing: ^  :)15:46
mrevellah, thanks jtv, I did wonder about that but then I kinda thought that we don't have much of a better alternative.15:47
jtvRelease series.15:47
jtvWhen we talk internally, that's needlessly verbose.  But every time we omit it, a potential user may go "what are these people talking about?  All I want is to host my project!"15:48
jtvAnd consequently, strangle a kitten.15:49
jtvmrevell: also, the the dreaded repeated word across line line breaks: "opt"15:50
mrevellah15:51
jtv(It is a dark day for travelers worldwide.  All the sudokus I picked up on this trip were so easy that they might as well have said "each little square should have a number in it, but some are empty.  Can you see where they are?")15:53
jcsackettallenap: can i get you to take a look at https://code.launchpad.net/~jcsackett/launchpad/spam-button-ui/+merge/6007215:54
allenapjcsackett: Sure.15:55
jtvmrevell: moving on, the paragraph starting with "Previously, Ubuntu took translation templates" introduces two consecutive notions with the word "this" (which makes it a bit unnatural to read) and uses the word "allowed" to describe an undesirable effect (which may confuse readers who lack confidence with the subject matter).  Especially because it also uses the word "diverge" in a way that's hard to avoid, but doesn't sit quite right with the 15:58
jcsackettallenap: i believe the diff appears pretty big, but a goodly junk of that is boilerplate for javascript tests and the fact that i renamed a test file.15:59
allenapjcsackett: No worries. It's such a cool feature I'd read it twice.15:59
jcsackettallenap: :-)15:59
mrevellthanks jtv, this is very helpful.16:00
jtvmrevell: how about moving the cause ahead of the consequence there?  "New upstream translations made after that initial import had no way [...].  Thus Ubuntu often ended up with different translations as translation work upstream and in Ubuntu continued independently over Ubuntu's 6-month release cycle."16:00
mrevellthat's great, much clearer jtv16:01
jtvmrevell: explaining things is important, and any improvement is multiplied by the number of readers.  That's why I always appreciated your documentation work so much.16:02
mrevellAh, thanks man. Looks like I'll have to buy you a beer in Dublin :) Although, I seem to remember that buying beer in Dublin requires a hefty bank loan first.16:03
jtvWell let's pool our mortgages and try the Guinness.  :)16:03
sinzuijcsackett: do you have time to mumble?16:04
* bigjools will be putting some beverages in the car16:04
jcsackettsinzui: i do. one moment.16:04
* jcsackett fires up mumble and prays16:05
jtvbigjools: bringing your own lukewarm English beer to Dublin?  How does that make things any better?16:05
bigjoolsjtv: you, as a Dutch man, have no right at all to talk about beer16:06
jtv(Apart from the slight reduction in lukewarm English beer the United Kingdom will experience)16:06
jtvOhhh yes.  Bring it on!16:06
jcsackettsinzui: i am here, but it seems that you cannot hear me.16:06
mrevellhaha16:06
bigjoolsif you want to drink beer served at sub-zero temperatures, be my guest!  But I prefer mine to actually taste of something ;)16:06
jtvbigjools: I appreciate that English cooking gave you a taste for cat's urine, but proper beer is best served at 4 degrees centigrade.16:07
bigjoolsjtv: you lie.  like a cheap carpet.16:07
bigjoolsthe only reason it's served that cold is so you can tell it from urine my friend16:07
jtvbigjools: if you wanted to hit me where it hurt, you would have said "is someone trying to teach me about beer from the United States of America?"16:08
jtvWouldn't have hurt _me_, but at least it would have hurt _someone_ who deserved it.16:08
bigjoolsjtv: anyone who drinks beer at 4C deserves it :)16:09
sinzuiwhat's to know? American is like making love in canoe.16:09
sinzuiWe do have real breweries now, but that does not mean you will find drinkable that beer in an established16:09
bigjoolsI did find some great beer in the US once16:10
jtvbigjools: thank you, I'm glad you don't begrudge me my delicious cool beer.16:10
bigjoolsit had been imported from England :D16:10
jtvExcept the Americans probably knew to refrigerate it, and it was a whole new experience of pain-free beer-drinking.16:10
huwshimiActually the states is where a lot of the micro brewers say modern beer is at16:11
bigjoolsjtv: get back in your box, troll!16:11
huwshimiThey influence a lot of beer these days16:11
huwshimiparticularly a lot of pale ales16:11
jtvFace it, mainland Europe figured out how to make lager taste good while the Brits mumble out ale and "proper beer" into their warm, smelly pilsner ripoffs.16:11
jtvAhhh this feels good.16:12
jtvEven American coffee can make you irritable if you drink enough of it.16:12
huwshimiSomething like the Sierra Nevada Pale Ale is amazing and has influenced a bunch of amazing beers in Aus16:13
huwshimiI used to hate on American beer, but you can find crap beer anyway16:13
jtvI'll see if I can get some.  :)16:13
huwshimijtv: You should16:13
jtvmrevell: I can't escape the impression that the blog post is trying to go in multiple directions at once.  Is it highlighting the new sharing, or more the whole new data flow that emerges from it?  The bit about branch imports is at the same time useful and a bit of a side track.16:14
jtvmrevell: it may help to separate the two aspects of the story.  "What has just changed?  How does it compare to existing sharing features?  What do these and other recent improvements add up to in terms of data flow?"16:18
abentleybenji: qa-ok16:19
benjiabentley: thanks!16:19
jtvmrevell: Not very different from what you do now, except I think it will make it clearer what each individual part of the text is driving at.16:20
mrevelljtv sorry otp16:20
jtvOK16:20
rvbaallenap: could you take a look at https://code.launchpad.net/~rvb/launchpad/change-perm-sync2/+merge/60077 ? (I think you are a good candidate to review this, because you've heard the whole story about this branch ;))16:22
rvbabigjools: ^^16:23
allenaprvba: Sure.16:28
allenaprvba: I'm doing another review right now, but I can do your's once I've finished.16:29
rvbaallenap: great, thanks.16:29
=== al-maisan is now known as almaisan-away
jtvbigjools: when we discussed creating distroseries indexes from the regular publisher cron, you mentioned that the manual verification steps were a bit of a holdover from when this was all a much bigger deal.  But since this goes into the new python-based publish-ftpmaster (another big change), shall I just leave the notification of release managers in place for the time being?  I can make it say something like "please verify manually."16:44
bigjoolsTDD = Tedious Driven Development with our test layers16:44
* jtv adds TLA16:44
bigjoolswhich is turning into RDD, Rage Driven Development16:44
jtvbigjools: hang on, I was already uploading TDD16:45
bigjoolsjtv: you could ask them if they would like to keep it!16:45
bigjoolsI honestly don't know16:45
jtvbigjools: what I mean is, it'd be good to keep their existing procedure for at least one more cycle, but more to ensure that my changes didn't screw things up.16:45
bigjoolsjtv: well sending them an email is not existing procedure, so I'd check about that since it might not be necessary.16:49
jtvThe email is a relatively superficial change, I hope--from "run this and then verify" to "wait until the system tells you this has been run and then verify."16:50
jtvBut as long as there's a manual step triggered by completion of an automated step, I think the notification makes sense.16:51
jtvI'll ask cjwatson though.16:51
bigjoolsok cool16:52
jtvcjwatson, are you here?16:52
cjwatsonjtv: we'll likely do the manual checks whether you want us to or not :-)16:54
jtvGood to hear.  :-)16:55
jtvThe question is:16:55
jtvnow that we're automating the manual "publish-distro.py" runs to create indexes for a new series,16:55
jtvdo you want Launchpad to notify the release managers when that has been done?16:55
cjwatsonoh, we don't need e-mail about that, no16:56
cjwatsonwe are already thoroughly used to monitoring publisher activity16:56
jtvWell, that certainly makes things simpler!16:56
cjwatsone-mail> or any other notification16:56
cjwatsonthere are plenty of existing Ubuntu processes that involve waiting for a publisher cycle to complete16:57
jtvOkay, then I'll rip out the email and you'll just have to assume that the regular publish-ftpmaster run will, the first time around, just initialize the indexes and quit.16:57
jtv(By the way, those publish-ftpmaster runs will be a bit faster now because of changes in process architecture and cache management)16:57
cjwatsonIOW we no longer need to stop the publisher and run publish-distro by hand?16:57
jtvDefinitely no longer the latter; let me just verify the former.16:58
cjwatson(the first publish-distro run already created indexes, it's just that it had to be given careful flags to do so)16:58
cjwatsonwell, and that we had to run initialise-from-parent beforehand, which I understand is going away16:58
jtvThe first publish-distro run already created indexes, but that's currently a manual run.  We're talking about moving that into the regular cron job.16:59
jtvYes, initialise-from-parent will be going away AIUI but I'm still fuzzy on the details there so can't make promises on whether you'll still need to stop the cron jobs.17:01
jtvBut I'm at least taking steps 12-14 out of that window.17:01
cjwatsonyou're not taking out step 13, because that's not in Launchpad's gift to remove. :-)17:02
cjwatsonbut ack on 12 and 1417:02
cjwatsonthat's what I meant by careful flags (-A)17:02
jtvI am taking step 13 out of the disabled-cron window, but not out of the process.  Steps 12 & 14 will become automated.17:02
cjwatsonhaving the publisher notice that the suite hasn't been published yet and Just Doing It is all that needs to happen to remove 12 and 14.17:02
jtvExactly; that's what I'm working on.17:03
bigjoolseverything's frozen anyway so no harm by JFDI17:03
cjwatsoni.e. bug 5521117:04
_mup_Bug #55211: shouldn't require careful publisher run when cloning new released distrorelease <lp-soyuz> <new-release-cycle-process> <soyuz-publish> <Launchpad itself:Triaged> < https://launchpad.net/bugs/55211 >17:04
bigjools5 digit bug!17:04
mrevellHey jtv, I'll run up another draft of the announcement17:04
* cjwatson closed two five-digit bugs just in the last week or so17:04
cjwatsonthey're less rare than people seem to think ... :-)17:05
jtvack mrevell17:05
bigjoolswe're probably unwittingly closing a bunch with derived distros17:05
jtvI'll mark that one as in progress then.17:05
jtvbigjools: if I make the initial-run check check for Frozen status, then the change won't affect previously released distroseries.17:09
bigjoolssounds very sane17:10
jtvbigjools: it probably doesn't make much sense to have a feature flag for this then, either.  The work will simply come with the new publish-ftpmaster.17:14
bigjoolsyup17:14
jtvyuuuup17:16
bigjoolsphrasing!17:16
allenaprvba: I'm not going to get to your branch until later this evening. Is that okay?17:17
jtvIt's not a competition, bigjools!17:17
rvbaallenap: of course, no worries.17:19
allenaprvba: Thanks, sorry about that.17:20
bigjoolsI am executing a remarkably satisfying sql statement on mawson17:22
bigjoolsDROP DATABASE launchpad_dogfood17:23
cjwatsonjtv: as long as it's clear that the Frozen status doesn't apply only to just-opened series17:24
jtvcjwatson: otp, brb17:24
=== m4n1sh_ is now known as m4n1sh
pooliebigjools, :)17:36
poolieshould i have a bug for even cleanup type landings, like https://code.launchpad.net/~mbp/launchpad/mail-cleanup/+merge/58867 ?17:37
=== matsubara is now known as matsubara-lunch
bigjoolspoolie: it's getting the last laugh though, the production restore is failing!17:37
=== salgado is now known as salgado-lunch
LPCIBotProject windmill-devel build #32: STILL FAILING in 1 hr 6 min: https://lpci.wedontsleep.org/job/windmill-devel/32/17:49
jcsackettallenap, rvba: i can take rvba's MP.17:55
jcsackettunless there is a particular reason you should do it, allenap.17:56
LPCIBotProject windmill-devel build #33: STILL FAILING in 41 min: https://lpci.wedontsleep.org/job/windmill-devel/33/18:31
=== matsubara-lunch is now known as matsubara
jcsackettallenap: saw your notes on my MP; i concur on the on and failure success handlers; meant to connect them up and clearly forgot.18:41
jcsackettregarding the permission checks tho, i was under the impression that check_permission did cache permission stuff on it's own. i take it i'm laboring under false info?18:42
LPCIBotProject windmill-db-devel build #238: STILL FAILING in 1 hr 5 min: https://lpci.wedontsleep.org/job/windmill-db-devel/238/18:51
benjijcsackett or allenap: hi guys, I have a very small branch for you to review: https://code.edge.launchpad.net/~benji/launchpad/bug-777766/+merge/6010018:52
jcsackettbenji: i can look at it in a bit, but i'm currently looking over a 1400+ diff. :-P18:52
benjicool, thanks18:53
jcsackettnp. just wanted you to know of the wait. :-)18:53
=== salgado-lunch is now known as salgado
=== deryck is now known as deryck[lunch]
sinzuiWhy do we keep seeing edge URL in reviews. That env is deprecated, soon to be obsolete?19:20
jcsackettbenji: i am looking at your branch now.19:53
benjisinzui: I used to have bookmarklets with edge in them, and prompted by you I just found "edge" in one of my MP-related scripts which was the source of the "edge" above19:53
sinzuiah19:54
=== deryck[lunch] is now known as deryck
allenapjcsackett: Ah, I didn't see your comment about taking rvba's branch. I'm back now so I can do it, but thanks for the offer.19:59
jcsackettallenap: it's done. :-)19:59
allenapjcsackett: Oh, awesome :)19:59
jcsackettbenji: only one question on your branch. does the crazy character mash that is base64 show up in a url now? or is this just in ids? from the test, i'm assuming the latter.20:01
benjijcsackett: yeah, it's just in IDs; user's wont see it20:01
jcsackettbenji: good stuff. :-) r=me, then.20:01
benjiI just used the URL base 64 because instead of + and / it uses - and _ (so it's ID-safe (except for the stuping paddin characters (which I remove after the fact))).20:02
* lifeless stretches20:16
lifelesswgrant: if its specialised, is it on unusual deployment notes ?20:16
LPCIBotProject windmill-devel build #34: STILL FAILING in 1 hr 4 min: https://lpci.wedontsleep.org/job/windmill-devel/34/20:20
abentleylifeless: is it true that testtools doesn't provide a matcher for regular expressions?20:22
lifelessI'm not aware of such a matcher; it would be great to have one20:28
lifelessmatchers were an experiment, one I think that has panned out well, but we haven't gone and migrated all the custom asserts etc into matchers20:29
lifelessI was thinking yesterday that its about time we did that20:30
abentleylifeless: Ah, I see.  It seemed like an odd oversight.20:31
lifelessabentley: we started with just a couple - equals and doctestmatches20:35
lifelessand have grown from there as people found them useful and nice to work with20:35
lifelessflacoste: what are these wiki edits all about20:35
lifelesse.g. ... +  * r9242  [r=jtv][ui=none][bug=297458] For wgrant: export bug nominations.20:36
flacostelifeless: testplans -> Trash20:36
lifelessflacoste: but they are being edited not deleted20:36
flacostelifeless: nah, i'm using Delete Page20:36
flacostenot sure how the notification looks liek20:36
lifeless-weird-20:36
lifelessI can see its gone now I try the page url itself20:36
lifelesslet me forward you the notification so you can go WTF20:37
flacostelifeless: i don't nee dmore WTF moments :-)20:37
lifelesssent :>20:37
flacostelifeless: Moin crack :-)20:40
lifelessyeah20:41
lifelessyou can understand my confusion20:43
flacosteit sure is confusing20:47
LPCIBotProject db-devel build #518: FAILURE in 5 hr 8 min: https://lpci.wedontsleep.org/job/db-devel/518/20:49
LPCIBotProject windmill-devel build #35: STILL FAILING in 40 min: https://lpci.wedontsleep.org/job/windmill-devel/35/21:01
lifelesssinzui: can you spare 45m to an hour sometime today to talk with me ?21:02
sinzuiI can21:03
lifelesssinzui: I've started looking beyond the immediate exigencies of performance... great21:03
lifelessbigjools is apparently popping back into irc at some point his evening; given tz constraints I want to chat with him when he does that; we could either wait for that to have happened, or have a call that might be interrupted21:04
lifelessI need a keyboard with catguards21:13
lifelessI'm thinking raised 2cm ridges at each end21:13
deryckjcsackett, you might like to review my current js fix.  It's stuff you and I chatted about before.21:20
deryckjcsackett, if you have the time, of course.21:20
lifelessderyck: I'd like to chat with you similarly21:20
lifelessderyck: but I imagine you are right on EOD as you run to UK times for some bizarre reason?21:20
derycklifeless, about js?21:20
lifelessderyck: about our architecture21:21
deryckah21:21
derycklifeless, I've got about 1/2 hour left.  but I have some CHR-like duties I've yet to do.21:21
benjijcsackett: I have another small review if you have another few minutes: https://code.launchpad.net/~benji/launchpad/bug-771239/+merge/6011321:21
derycklifeless, but I don't mind chatting.  Or we can chat first thing for you tomorrow?21:21
lifelessI would (even though tomorrow is sat) but we have to go pick up a kitten21:21
jtvallenap: you're not really still here, are you?21:22
lifeless1/2 hour can probably touch the high notes21:22
lifelessderyck: so, lets do it!21:22
derycklifeless, ok, let's do.  firing up mumble.21:23
lifelessderyck: d you have skype or voip?21:23
lifelessmumble is crappy from here21:23
deryckah21:23
deryckI don't know if I ever installed skype.  let me see.21:23
lifelessthe canonical voip is pretty nice if you have that21:23
lifelessif you haven't got skype we can try mumble, we might get lucky21:24
derycklifeless, I haven't setup voip yet.  doing skype install now.  should be quick.21:24
lifelesskk21:24
deryckI need to get voip up.21:24
deryckI'll warn you its loud here.  Working from my wife's shop and it's after school time. :-)21:25
lifelessthats ok21:25
deryckyou say that now ;)21:25
* deryck is not bothered by noise, but late day calls are fun21:25
benjiallenap: I have a small review if you have a few minutes: https://code.launchpad.net/~benji/launchpad/bug-771239/+merge/6011321:27
maxblifeless: If you have a moment between all the calls, could you revisit the question of giving me access to the lpreview-body packaging branch as broached in bug 766149? Thanks.21:28
_mup_Bug #766149: bzr-lpreview-body from launchpad ppa blocks python 2.6 install on natty by using site-packages <packaging> <Launchpad review body plugin:Confirmed> <bzr (Ubuntu):Invalid> < https://launchpad.net/bugs/766149 >21:28
lifelessmaxb: I have no issue with that.21:28
lifelessflacoste: ^21:28
flacostemaxb: me neither21:29
jtvjcsackett: are you available for a review?  It's large, but more than half the overall diff is deleted lines.  https://code.launchpad.net/~jtv/launchpad/bug-777941/+merge/6011621:29
maxbGreat, in that case, could you migrate ownership of https://code.launchpad.net/~launchpad/+recipe/lpreview-body-daily and https://code.launchpad.net/~launchpad/lpreview-body/packaging to ~launchpad-committers ?21:39
LPCIBotProject windmill-devel build #36: STILL FAILING in 44 min: https://lpci.wedontsleep.org/job/windmill-devel/36/21:45
lifelesssinzui: so with that in mind, what time suits you for a call21:48
=== jtv is now known as jtv-afk
lifelessbah21:54
lifelessI have to get that battery sorted21:54
=== lifeless changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: allenap, jcsackett | https://code.launchpad.net/launchpad-project/+activereviews
lifelessmaxb: what were those urls21:55
maxb<maxb> Great, in that case, could you migrate ownership of https://code.launchpad.net/~launchpad/+recipe/lpreview-body-daily and https://code.launchpad.net/~launchpad/lpreview-body/packaging to ~launchpad-committers ?21:55
maxb(thanks)21:55
lifelessapparently not21:56
pooliehello lifeless21:56
lifelessI get 'undefined' on the picker widget21:56
lifelesshi poolie21:56
lifelesssinzui: if you replied I missed it21:57
pooliei'm trying to ec2 test some branches21:57
pooliethey seem to be sinking without trace21:57
pooliei suppose while trying to send mail or something21:57
lifelessif the failure is too big it can hit gmails size limits21:58
sinzuilifeless: sorry. I was distracted. we can talk now or in 4 hours21:58
lifelesssinzui: now sounds good to me21:59
poolieis there any way to ask it to just leave the instance running serving the file?22:00
sinzuilifeless: skype?22:00
lifelesspoolie: I don't think so22:00
poolie:/22:04
flacostepoolie: patch welcome :-)22:05
flacostepoolie: there is an option not to detach from the instance22:05
flacosteyou could tail the output22:05
poolietail the output of what?22:06
pooliethe hotel wireless drops out occasionally22:07
pooliei guess i could start an ec2 machine, ssh there and run screen, then run things from that22:07
salgadopoolie, you can use ec2 demo to start the instance; that way you have everything ready to run the test suite when you ssh into the instance22:11
poolieam i supposed to normally be able to ssh in to the test versions?22:11
pooliethey seem to have a server but it doesn't accept my key22:11
salgadopoolie, I think your key is associated to the ubuntu user there22:13
poolieoh ok22:13
poolieno, apparently not?22:14
salgadoI'm pretty sure your key is there, just need to find out under which user22:15
poolieoh ec2test@22:16
jcsackettderyck and benji: sorry for not replying, i seem to be having IRC connection issues today. i have looked at your branches, and they are a-ok. :-)22:17
jcsackettjtv: i am looking at yours now.22:17
poolieok, so ssh'd in, the log shows22:23
poolieRunning command: /usr/bin/xvfb-run --error-file=/var/tmp/xvfb-errors.log --server-args='-screen 0 1024x768x24' /var/launchpad/test/bin/test -vv --subunit -vvv22:23
poolieand nothing more22:23
poolieit does seem to be using cpu22:23
LPCIBotProject windmill-devel build #37: STILL FAILING in 41 min: https://lpci.wedontsleep.org/job/windmill-devel/37/22:27
mwhudsonhmm, i seem to be getting launchpad bug mail again22:45
jcsackettjtv: r=me. i have two suggestions in the MP, but they don't block landing--just things to consider.22:46
mwhudsoni guess this is to do with the better subscription story stuff being enabled for ~launchpad?22:46
mwhudsonah no, it's ~launchpad loosing its contact address i think22:46
=== jcsackett changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews
mwhudsonluckily the better subscription story lets me turn off the firehose22:48
jelmerooh, the new bug subscriptions are nice :)22:52
LPCIBotProject windmill-db-devel build #239: STILL FAILING in 1 hr 5 min: https://lpci.wedontsleep.org/job/windmill-db-devel/239/23:04
=== matsubara is now known as matsubara-afk
=== mwhudson_ is now known as mwhudson

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!