/srv/irclogs.ubuntu.com/2011/12/19/#launchpad-dev.txt

lifelesswallyworld_: WHATS UP?01:12
lifelessbah, caps lock01:12
wallyworld_lifeless: it's ok, i already sorted it. the deployment report had a rev that should have been tagged rollback=xxxx and i was wondering how to fix it01:13
wallyworld_and no-one else was shown as logged into irc :-)01:13
StevenKMissing rollback tags are annoying01:14
StevenKSince you can't add them after the fact01:14
wallyworld_yeah, so i found out01:17
lifelessqatagger is fixable :)01:27
wgrantlifeless: Yay for fixing bug watches.01:33
* StevenK doesn't get it.01:40
StevenKReplicating the conditions of bug 892025, and I don't see the link. But I do on qas/prod01:40
_mup_Bug #892025: Forbidden following link to configure code hosting <403> <qa-ok> <trivial> <ui> <Launchpad itself:In Progress by stevenk> < https://launchpad.net/bugs/892025 >01:40
wgrantStevenK: You probably have permission to set it, but can't see the dev focus branch.01:41
wgrantUnauthorized: (<Branch u'~hexr-dev/hexr/trunk' (406169)>, 'unique_name', 'launchpad.View')<br />01:41
StevenKOh01:41
StevenKSo configure_codehosting should learn that too?01:42
wgrantNo.01:42
wgrantWell, maybe.01:42
wgrantIt's difficult.01:42
wgrantLeave it for now.01:42
wgrantWe'll revisit it if it's still a problem once we've fixed privacy.01:42
StevenKBah, Gavin hinted at the issue in the description too01:43
wgrantI just looked at the traceback :)01:43
StevenKI saw the error page and assumed it was "You can't see this page, go away"01:44
* StevenK stares at BugContextMenu.nominate()02:22
StevenKYou can Target if you're a driver, nominate if you're a bug supervisor and everyone else can bugger off?02:22
lifelesswgrant: context? my email I presume?02:23
wgrantStevenK: Yes02:25
wgrantlifeless: Yes02:25
StevenKwgrant: How is that related to bug 297528, then?02:32
_mup_Bug #297528: Permissions not checked properly when deciding whether to present "Target" or "Nominate" link <lp-bugs> <trivial> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/297528 >02:32
wgrantStevenK: Well, drivers or uploaders can target.02:34
wgrantBut whatever decides to show Target/Nominate doesn't take into account uploaders.02:35
wgrantSo for uploaders it says it will nominate, when it will in fact target.02:35
StevenKwgrant: Right, so BugContextMenu.nominate() needs a elsif I can upload, which says target.02:36
wgrantYes, but that needs to be faster than it probably currently would be.02:36
wgrantStevenK: There's more MixedVisibilityErrors, if you want them.02:38
StevenKOh?02:38
wgrantProductSet:+all02:38
StevenKCan I have an OOPS too?02:39
wgrantShould be pretty obvious :)02:39
wgrantGiven what the page does02:39
wgrantBut OOPS-01faea627b2e0a135da5d8e3640e22ef02:39
StevenKOh, it returns every signle product?02:40
StevenKwgrant: So I guess we want to hide the product if the user can't view the registering team?02:51
wgrantPossibly.02:52
wgrantI didn't think private teams were allowed to own projects :/02:52
wgrantWe may be best to leave this.02:53
wgrantAs the new rules should mean that the team is to be disclosed.02:53
StevenKChewie02:53
StevenKChewie exports system settings through a menu service such as dbusmenu and/or gmenu02:53
StevenKRegistered by <hidden> on 2011-12-0202:53
wgrantThat's not the one that's the main problem here.02:53
wgrantBut it's a similar case.02:54
wgrantdpkg-coverity is the one here, though I can see the team due to some kind of subscription02:54
StevenKActually, disclosing teams in a public role should fix this, you're right02:54
wgrantRight.02:54
wgrantBut that requires discussion in Budapest.02:54
StevenKAnd beating lifeless with a stick02:54
StevenKI think02:54
StevenKSo I have 1.5 days left. And Curtis has forbidden me to have any branches in progress over the break.02:55
wgrantHeh02:56
lifelessStevenK: hmm ?02:59
wgrantWe have things to discuss in Budapest.03:00
StevenKwgrant: So I guess I should find a critical03:00
StevenKHm, I think bug 739070 could be fixed with two new columns, but I doubt it's completly doable in the time I have left.03:14
_mup_Bug #739070: Archive:+repository-size timeout retrieving many hundreds of package sizes <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/739070 >03:14
StevenKAnd wgrant may shoot me for wanting to add two new columns to Archive.03:15
wgrantCorrect.03:17
StevenK:-(03:18
StevenKlifeless: Due to your work with OOPSes, I guess bug 788518 can be closed?03:27
_mup_Bug #788518: staging services share OOPS prefix <critical-analysis> <Launchpad itself:Triaged> < https://launchpad.net/bugs/788518 >03:27
StevenKAnd bug 805546, I guess03:28
_mup_Bug #805546: persontransferjob does not have a unique oops prefix <Launchpad itself:Triaged> < https://launchpad.net/bugs/805546 >03:28
wgrantStevenK: Indeed.03:32
wgrantStevenK: Are you on launchpad-error-reports?03:33
wgrantStevenK: There's an issue with that OOPS work of lifeless' which somebody need to look at.03:33
StevenKwgrant: I am not03:34
wgrant2011-12-18 09:52:03 ERROR   Unhandled exception -> http://launchpadlibrarian.net/87826942/iDGKk0EWB2rD9ZfBovt3JrR3nLJ.txt (unsupported operand type(s) for +: 'NoneType' and 'str')03:34
wgrantFrom checkwatches on loganberry.03:34
StevenKI guess either config[self._default_config_section].oops_prefix is None?03:37
wgrantIt looks that way, but I can't see how.03:37
StevenK_default_config_section is error_reports03:38
wgrantIndeed.03:38
StevenKYou were fiddling with the configs last week ... :-P03:39
wgrantYes, but lifeless fiddled with this bit :)03:39
StevenKhuwshimi: AH ha! You have QA to do.03:40
huwshimiStevenK: I do!03:40
huwshimiStevenK: Telstra have been out trying to fix my internet all morning03:40
StevenKThey may have succedded?03:41
huwshimiStevenK: It appears that way03:42
huwshimiStevenK: At least it is working, if  a little slow03:42
StevenKwgrant: I don't get it either.03:42
StevenKwgrant: Right, production error_reports has no oops prefix, and the code is expecting one if reporter is None.03:47
huwshimihmm.. I thought I had 4 branches to qa03:48
wgrantStevenK: Lies.03:49
wgrantStevenK: production/launchpad-lazr.conf's oops_prefix is PRODUCTION.03:49
StevenKwgrant: Oh?03:49
StevenKsteven@liquified:~/lp-production-configs% grep -c oops_prefix production/launchpad-lazr.conf03:50
StevenK003:50
wgrantbzr pull03:50
StevenKThat's with rev 24703:50
wgrantsrsly?03:51
wgrantr567 is the latest.03:51
StevenKbzr pull gives "No revisions to pull."03:52
wgrantYou be pulling from the wrong place.03:52
StevenKbzr+ssh://bazaar.launchpad.net/%2Bbranch/lp-production-configs/ is the wrong place?03:52
wgrantThat's the right place.03:53
wgrantAh03:53
StevenKsteven@liquified:~/lp-production-configs% bzr revno bzr+ssh://bazaar.launchpad.net/%2Bbranch/lp-production-configs/03:53
StevenK24703:53
wgrantThat was the crontabs revno I was looking at.03:53
wgrant247 is indeed configs03:54
StevenKNow, grep yourself and take back your lies comment :-)03:54
wgrantlpnet-lazr.conf is where PRODUCTION is specified.03:54
wgrantAnd production/launchpad-lazr.conf inherits that.03:54
wgrant$ LPCONFIG=production bin/py03:54
wgrantPython 2.6.5 (r265:79063, Apr 16 2010, 13:09:56)03:54
wgrant[GCC 4.4.3] on linux203:54
wgrantType "help", "copyright", "credits" or "license" for more information.03:54
wgrant>>> from canonical.config import config03:54
wgrant>>> config.error_reports.oops_prefix03:54
wgrant'PRODUCTION'03:54
StevenKThen what crack is checkwatches on?03:55
wgrantThat's why I asked you to investigate :)03:55
StevenKOh, hah.03:56
StevenKclass CheckWatchesErrorUtility(ErrorReportingUtility):03:56
StevenK_default_config_section = 'checkwatches'03:56
wgrantThat would do it.03:56
wgrantmurder03:56
StevenKKill it with a large amount of fire?03:56
wgrantIdeally.03:56
wgrantlib/lp/services/mailman/monkeypatches/xmlrpcrunner.py:    _default_config_section = 'mailman'03:57
wgrant:D03:57
* StevenK feels ill.03:57
huwshimihmmm... why might I be getting this pqm failure? http://paste.ubuntu.com/774941/04:25
wgranthuwshimi: Check the attachments.04:27
wgrantWe're in testfix.04:27
StevenKdb-devel went bang with 4 failures and 31 errors.04:28
wgrantThey are real, too.04:28
StevenKProgrammingError: permission denied for relation milestonetag looks like the root cause04:28
wgrantlifeless is more than adequately opinionated on this matter.04:29
StevenKlifeless said bad idea and it landed anyway?04:29
StevenKwgrant: https://code.launchpad.net/~stevenk/launchpad/force-checkwatches-production/+merge/8618704:29
wgrantStevenK: disapprove04:30
StevenKAwww04:30
StevenKWhy?04:30
wgrantThat will leave the reporter as just PRODUCTION.04:31
wgrantRather than PRODUCTION-checkwatches04:31
wgrantI suspect.04:31
wgrantYou probably want to call .configure(section_name='checkwatches')04:31
StevenKYeah, I was just reading that.04:32
huwshimiwgrant: The pqm failure complained about the commit message being in the wrong format. I assume that's because the branch is attached to a bug that already has a branch that has landed. Do I need to do something to get it to play nice for the second branch?04:33
StevenKhuwshimi: So, someone needs to fix db-devel.04:33
StevenKI suspect the easiest thing to do is to rollback the milestonetag rev04:33
huwshimiStevenK: I think that was intended for wgran04:34
huwshimi*wgrant04:34
StevenKhuwshimi: The PQM failure complained about the commit message not containing [testfix] since db-devel failed on buildbot.04:35
StevenKThat's what "We're in testfix" means.04:35
StevenKI'll fix it after lunch.04:35
huwshimiStevenK: Oh right04:37
huwshimiStevenK: But it also complained on Friday when other branches of mine landed04:37
huwshimiStevenK: I thought it might have been because the commit message was missing [bug=894442]04:39
StevenKhuwshimi: PQM doesn't care about the content of the tag, it would happily deal with [bug=999999]04:43
StevenKwgrant: Diff updated, if you're happy with it, I'll self-review and toss it at ec2.04:43
huwshimiStevenK: I think it does. It's failed because of the commit message: http://paste.ubuntu.com/774951/04:44
StevenKNote the lack of [bug=04:45
wgranthuwshimi: You need either [bug=foo] or [no-qa]. That is why that one failed.04:45
StevenKOr [no-qa]04:45
wgrantBut the current failure is because of testfix.04:45
huwshimiStevenK, wgrant: Yes, that's what I was saying :)04:45
StevenKwgrant: No fair approving. But thanks. :-)04:46
huwshimiI was also wondering why it might not have appended that, and thought it might have had something to do with the bug already having a landed branch04:46
lifelesswgrant: StevenK: It landed before I commented, which is fine; its not something I'm going to veto anyhow, because we do need to solve the users problems; I'm hopeful we can solve the deeper cause of friction.04:56
StevenKlifeless: I was about to revert it anyway05:20
StevenKlifeless: Did you see the two bugs I linked?05:26
StevenKwgrant: Did we disable the beta bug listing on qas/prod?05:31
lifelessStevenK: I'm on leave; not following much of anything. Do I need to see them ?05:33
wgrantStevenK: It's disabled on prod.05:33
wgrantAnd qas is a separate team.05:33
StevenKlifeless: I'd like to close them, I was wondering if you agree.05:34
StevenKwgrant: Which team on qas, I'd like to QA rick_h__'s change.05:34
wgrantcustom-buglisting-demo05:35
StevenKwgrant: Thanks, qa-ok05:38
lifelessStevenK: links ?05:55
StevenKlifeless: bug 788518 and bug 80554605:56
_mup_Bug #788518: staging services share OOPS prefix <critical-analysis> <Launchpad itself:Triaged> < https://launchpad.net/bugs/788518 >05:56
_mup_Bug #805546: persontransferjob does not have a unique oops prefix <Launchpad itself:Triaged> < https://launchpad.net/bugs/805546 >05:56
lifelessso, uhm05:57
lifelesswhat report will ptj be using now ?05:58
lifelesswill someone be able to tell its ptj that an error occurs in05:58
lifelessfor the staging one, definitely downgrade as we are no longer losing oopses05:59
StevenKDowngrade, or close?05:59
lifelessif we have reasonable reporters for ~everything on staging, then its closable. I'm not sure if we do yet - because not everything uses configure(section=)05:59
StevenKlifeless: I just fixed checkwatches05:59
wgrantWe should just use script_name and be done with it.05:59
lifelessthings that don't reconfigure with a specific section will be getting logged as 'STAGING' whether or not they are an appserver.05:59
lifelessindeed, doing a systematic fix across the system is a great idea. Its a little more than script-Name though06:00
wgrantSlightly.06:00
StevenKlifeless: Happy to slam it to High?06:00
lifelessStevenK: high makes sense to me, along with a comment about the status, of course06:01
lifelesswgrant: jobs need consideration06:01
StevenKlifeless: Which you made in October06:01
wgrantYes.06:01
wgrantThat's the only real issue I can see.06:01
lifelessStevenK: other code changes have happened since then, so its worth noting that they *haven't* corrected everything.06:02
lifelessStevenK: its almost always worth a short comment when changing things, helps keep everyone on the same page06:02
lifelesswgrant: I'd like to move all lp to a single reporter, and use topic to split out scripts06:02
lifelesswgrant: be a good idea to do at the same time06:03
wgrantOnce topics exist, sure :)06:03
lifelesswgrant: they already exist06:03
lifelesswgrant: its where pageids are reported06:03
wgrantAh, true.06:03
StevenKhuwshimi: You can reland your branch06:04
huwshimiStevenK: :D06:04
huwshimithanks06:04
StevenKhuwshimi: Sorry, I could have told you 15 minutes ago, but I got distracted.06:04
huwshimiStevenK: I can't believe you would do that to me06:05
StevenKHah06:06
lifelessis https://bugs.launchpad.net/launchpad/+bug/39630 a dup of the other domination bugs ?07:14
_mup_Bug #39630: Older packages are removed when newer versions FTBFS <boobytrap> <lp-soyuz> <soyuz-publish> <Launchpad itself:Triaged> < https://launchpad.net/bugs/39630 >07:14
lifelessalso, I'm amazed https://bugs.launchpad.net/launchpad/+bug/40096 hasn't had a look-in07:16
_mup_Bug #40096: do not show architecture 'all' builds as i386 <confusing-ui> <lp-soyuz> <oem-services> <trivial> <Launchpad itself:Triaged> < https://launchpad.net/bugs/40096 >07:16
nigelbBug listing is disabled for beta testers temporarily or something?07:16
wgrantlifeless: Howso?07:17
wgrantnigelb: Yes.07:17
nigelbwgrant: Ah! Thanks.07:17
lifelesswgrant: howso to which ?07:17
wgrantlifeless: The latter.07:17
lifelesswgrant: because it looks like a shallow UI tweak07:17
wgrantlifeless: ROFLOL07:17
wgrantHow do you propose to determine that? :)07:18
wgrantWe'd need to set a flag on the build at upload-time.07:18
lifelesswgrant: that its only arch-any ?07:18
wgrantarch-all, but yes.07:19
lifelesswgrant: bah, ENOPACKAGINGENOUGH07:19
wgrantAnd then what if we now confuse people because normal i386 builds still build arch-all, so you have to wait for them anyway?07:19
lifelessyou mean 'we dispatch arch-all builds to the i386 queue' ?07:20
wgrantIn the case of an arch-any package, the i386 build builds the arch-all bits07:20
wgrantSo it should perhaps be presented as "i386 + all"07:20
wgrantThe UI is not trivial.07:21
wgrantAnd it requires model changes too.07:21
wgrantAnd it matters to nobody after their first 5 minutes with LP.07:21
lifelessdon't all the builders build -all, and we discard them on other builders? or do we actually call the specific targets..07:21
wgrantSo it's unsurprising it hasn't been tackled :)07:21
wgrantNo.07:21
wgrantOnly nominatedarchindep builds arch-indep.07:21
lifelessk07:21
lifelessdo you happen to remember the bug about only i386 being allowed to build -all ?07:22
wgrantBug #15800407:25
_mup_Bug #158004: Arch independent packages are only built on i386 <lp-soyuz> <ppa> <soyuz-build> <Launchpad itself:Triaged> < https://launchpad.net/bugs/158004 >07:25
wgrantlifeless: Bug #217427 is also slightly relevant07:25
_mup_Bug #217427: Please support arbitrary arch/buildd affinity for arch:all builds <feature> <lp-soyuz> <motu> <soyuz-build> <Launchpad itself:Triaged> < https://launchpad.net/bugs/217427 >07:25
wgrantlifeless: ENOTTRIVIAL07:28
wgrantAh07:29
wgrantYou've already fixed it.07:29
wgrantThanks.07:29
lifelesshttps://bugs.launchpad.net/launchpad/+bug/40096/comments/307:29
_mup_Bug #40096: architecture 'all' builds are shown as i386 <lp-soyuz> <oem-services> <Launchpad itself:Triaged> < https://launchpad.net/bugs/40096 >07:29
lifelessalso overhauled the summary07:29
lifelesswgrant: so what do you think of bug https://bugs.launchpad.net/launchpad/+bug/3963007:44
_mup_Bug #39630: Older packages are removed when newer versions FTBFS <boobytrap> <lp-soyuz> <soyuz-publish> <Launchpad itself:Triaged> < https://launchpad.net/bugs/39630 >07:44
wgrantlifeless: Half fixed.07:44
lifelesswgrant: do me a favour and update its metadata?07:44
stublifeless: So since you are obviously now taking your holiday that seriously, I worked out an edge case with moving the 'add new tables to replication' code outside of the downtime window that would seize up replication.08:01
stubA db patch that adds a new table, and adds a constraint to an existing table that references the new table. If data gets added to the existing table referencing the new table before the new table is replicated, we are screwed.08:03
=== danilo_ is now known as danilos
stubWe could say we are fine though, provided we promise that no attempts to access the new table are made before the deploy has fully completed (so no code 'if this table exists, fill in this data')08:04
lifelessstub: cannot add references outside downtime anyway08:05
lifelessstub: because they require exclusive locks to update the table mgmt triggers08:06
stublifeless: We create the new table and the reference during downtime, we exit the outage, we add the new table to replication (proposed workflow)08:07
lifelessstub: oh right, different case.08:07
lifelessstub: uhm, yes, so we know that no code can possibly be using the new column or table08:07
stubyer, I can't think of how this could bite launchpad.08:07
stubUnless triggers are involved :-)08:08
lifelessstub: a default value that causes trouble could be made, but that is something we can pick up in review08:08
stubA default value will not be referencing a non-existing row in the new table08:08
stubWe could only shoot outselves in the foot with a trigger. Like BugSummary.08:09
stubSo if we go ahead with the change, we would need to ensure that creating tables and any code using the tables (standard or triggers) must land separately and be deployed separately08:10
lifelesswe could define an empty trigger so that the definition is there08:12
lifelesscan update the procedure in NDT, but not the existence/absence of a trigger08:13
stubSo it is an obscure case to catch during review, and very rare. So it is unlikely to happen, but we are likely to miss it if it does happen.08:15
stubRecovery would not be amusing on the live system.08:16
lifelesswe can shoot ourselves in the foot many different ways08:16
lifelessuhm08:17
lifelessI'm ok with the risk, if we document it well and prominently08:17
lifelessare you?08:17
stubI'll sleep on it :)08:19
lifelessI can think of other ways we can mess ourselves up and not have qa catch it :)08:19
lifeless(like e.g. bad indices or queries)08:19
wgrantstub: We're in huge trouble if we're touching a not-yet-replicated table anyway, aren't we?08:21
stublifeless: That doesn't involve taking the system down, manually copying data to slaves and massaging replication events08:21
wgrantBecause the new data won't get replicated...08:22
stubwgrant: Only if it is referenced by a replicated table, which is this case08:22
wgrant(unless you manually clone it across first, which seems silly given it never needs to happen)08:22
wgrantstub: Does adding the table to replication also replicate the existing data?08:22
wgrantI assumed it would not.08:22
stubIt will08:22
wgrantAh08:23
wgrantThat seems excessive.08:23
wgrantBut perhaps.08:23
stubThe trick is that adding a table to replication won't happen until pending events have been processed, and in this case pending events won't be processed because the slave is attempting to insert data its constraints won't let it.08:23
wgrantRight.08:23
lifelessstub: I'm not sure the constraints are checked on the slave08:24
lifelessstub: because slony disables all the triggers on the slaves, and constraints are triggers08:24
stubHmm... easiest recovery would actually be to disable the foreign key constraints on the slave, add tables and wait for catch up, and reenable foreign key constraints.08:24
lifelessstub: I suggest trying it locally and seeing if it actually borks08:24
wgrantI was thinking I'd seen constraint violations on a slave locally.08:25
stubforeign key constraints are not disabled IIRC.08:25
wgrantBut that's only from a unique index.08:25
wgrantSo it's possible they're disabled.08:25
lifelessunique indices will stay live, thats index structure08:25
wgrantExactly.08:25
wgrantI was just thinking I had evidence that they were left enabled, then realised that it wasnt.08:25
stubAre continual WARNINGs that we need to fix HIGH or CRITICAL?08:28
stubBug #90619308:28
_mup_Bug #906193: BugHeatUpdater never completes <Launchpad itself:New> < https://launchpad.net/bugs/906193 >08:28
lifelessarguably critical - we have no way to handle/detect 'garbo job X is not getting to run' and thats an operational issue08:29
lifelessbeing able to detect that probably involves fixing the 'blah was interrupted' warnings08:29
lifelesspoolie: http://pad.lv/OOPS-78b4ff3d44483f940b04d6a2ef5c321e 404's08:30
stubOnly emailing WARNING and above, we could easily enough tie that into alerts08:30
lifelesssorry, not enough context to grok that08:31
lifelesswgrant: just got an ajax oops link to work \o/08:31
lifelesswgrant: OTOH https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-78b4ff3d44483f940b04d6a2ef5c321e08:31
stubWe (should be) running scripts logging everything to file, but only emitting WARNINGS and above to stdout.  We could tie that into nagios.08:32
stubOr just grep the log files for WARNING|ERROR|CRITICAL08:32
pooliehi lifeless, ok08:33
lifelesswe recently overhauled all the cronscripts in prod to log everything to a file08:33
stubIf todays log and yesterdays log contain warnings or above, flag the service as failed.08:33
lifelesshigh logging levels generate OOPSes already08:33
lifelessstub: so being interrupted isn't a problem per se; its being starved for days that matters, we need something with a longer view of things, or something.08:35
stubYes, so a day or two running with warnings should be enough to prompt someone to take a look and either file a bug or get the alert disabled for a few days until transient issues have been cleared.08:36
stubWe could also happily enough log each garbo job to ScriptActivity08:37
stubeach task I mean08:37
lifelessstub: indeed, though after the rewrite please :)08:37
lifelessstub: does a WARNING log item cause an OOPS or is it ERROR and above ?08:38
stubSure. Log file grepping -> nagios could be done right now if we want08:38
stubIt should be WARNING and above.08:38
stubWARNING should generate an informative OOPS, ERROR and above a standard OOPS.08:38
lifelessok, so we're getting warnings nows. I agree with your analysis FWIW - just leave the reporting as is and fix the issues that show up.08:39
lifelessstub: 'informative' doesn't exist anymore as a flag, got pruned in the redo.08:39
lifelesswgrant: could you do me a favour and expand on the rather brief description in https://bugs.launchpad.net/launchpad/+bug/78466 ?08:40
_mup_Bug #78466: FTPArchiveHandler doesn't support allowed_suites <lp-soyuz> <soyuz-publish> <Launchpad itself:Triaged> < https://launchpad.net/bugs/78466 >08:40
wgrantlifeless: It's not clear?08:44
lifelesswgrant: its opaque as a frog sitting in ink08:44
lifelesscjwatson: are you perhaps around ? could use your input on bug 8701208:47
_mup_Bug #87012: Cannot start developing next ubuntu release before the prior one is released <lp-soyuz> <soyuz-publish> <Launchpad itself:Triaged> < https://launchpad.net/bugs/87012 >08:47
poolielifeless, fixed, i think, except i'm now having network trouble08:47
lifelesspoolie: thanks!08:47
wgrantlifeless: I think it's probably just about obsolete.08:49
wgrantNot much uses allowed_suites directly any more.08:49
wgrantThey just check dirtiness, which a-f already does.08:49
lifelesswgrant: so, invalid?08:53
wgrantI think so.08:54
wgrantIn a disaster-recovery situation we're thoroughly screwed anyway, so it doesn't really matter.08:54
lifelesswgrant: doit :)08:54
wgrantIs done.08:55
lifelessstub: is https://bugs.launchpad.net/launchpad/+bug/90809 in progress ?08:55
_mup_Bug #90809: Launchpad should run with standard_conforming_strings=on in postgresql.conf <lp-foundations> <Launchpad itself:Triaged by stub> < https://launchpad.net/bugs/90809 >08:55
lifelessstub: actually, what I mean is08:56
lifelessstub: did we do this, or are we in deep dodo with respect to it ?08:56
adeuringgood morning08:57
wgrantlifeless: "This is a broken-by-design issue: its not safe to run two different09:01
wgrantLPCONFIG's from the same source instance, ever."09:01
wgrantWe do that all the time.09:01
wgrantMostly on loganberry/ackee09:01
lifelessthen we're freaking lucky that its not breaking all over the shop09:01
wgrantBut also cocoplum/germanium/wildcherry/sourcherry09:01
wgrantHow?09:01
lifelessand we have to stop doing that09:01
wgrantWhat state are they going to write out?09:02
lifelessthe overrides zcml09:02
StevenKbigjools: O hai!09:02
lifelesswgrant: its presumably identical for us now, but wasn't in the past; if its not written atomically, we must be freakishly lucky09:02
StevenKbigjools: So, we have two issues this weekend and today, both of them due to work your squad has done. buildd-manager broke badly on the weekend due to bugs 905853, 905855 and 906079; and germanium was reverted since the poppy-sftp breakage caused germanium to hit a load of 1700.09:04
wgrantlifeless: Hm, that is remarkably, remarkably evil.09:04
wgrantAnd has very little justification.09:04
lifelesswgrant: I went blind when I found it ~ a year ago09:04
lifelesswgrant: working on test parallelisation w/out lxc09:04
wgrantStevenK, bigjools: Note that poppy-sftp survived until just a few hours ago.09:04
poolielifeless, confirmed that oops url does work09:05
wgrantlifeless: I've actually managed never to run into that before.09:05
pooliei started turning ec2test into a juju charm09:05
poolie...09:05
pooliekind of ok09:05
lifelesspoolie: charm or charms ?09:05
wgrantpoolie: I considered that a couple of weeks back.09:05
wgrantBut decided that juju scared me.09:06
poolielifeless, one charm at first09:06
poolieso it's kinda monolithic and a bit of a degenerate case09:06
StevenKfrankban: Hi. I reverted your milestonetag revision on db-devel since it broke buildbot.09:13
lifelessbigjools: hey, so bug https://bugs.launchpad.net/launchpad/+bug/117557 - doesn't that cause an OOPS ?09:15
_mup_Bug #117557: Nascent Upload code doesn't check properly for bad distroseries <lp-soyuz> <soyuz-upload> <tech-debt> <Launchpad itself:Triaged> < https://launchpad.net/bugs/117557 >09:15
frankbanStevenK: ok, thanks09:15
lifelessbigjools: you say it won't, but I don't understand how it wouldn't.09:15
bigjoolslifeless: the exception is caught and translated to an error returned to the user09:17
stublifeless: We are not in deep doodoo - I'll be dealing with standard conforming strings settings with the PG upgrade. IIRC the things that were quoting oldskool were psycopg2 and maybe a few items in comments.sql. Doesn't really matter as the old format is still supported.09:18
lifelesswgrant: https://bugs.launchpad.net/launchpad/+bug/118870 - you comment that its a strange view, but it looks ok to me.09:19
_mup_Bug #118870: $sourcepackage/+changelog only shows one entry per suite (distroseries-pocket) <lp-soyuz> <Launchpad itself:Triaged> < https://launchpad.net/bugs/118870 >09:19
pooliewgrant, so as an initial wrapper around starting an instance it seems ok09:20
poolieusing something like this to start ppa builders might be more interesting09:20
wgrantlifeless: The strange mangled view I speak of was later filed as bug #14462009:21
_mup_Bug #144620: Some displayed sourcepackagerelease changes files don't have attribution <lp-soyuz> <Launchpad itself:Triaged> < https://launchpad.net/bugs/144620 >09:21
lifelessstub: what about https://bugs.launchpad.net/launchpad/+bug/120404 ? - do we still need to do this audit ?09:21
_mup_Bug #120404: Database permission audit <lp-foundations> <Launchpad itself:Triaged> < https://launchpad.net/bugs/120404 >09:21
wgrantlifeless: It's not relevant to the bug beyond confirming it wasn't a regression, anyway.09:22
stublifeless: I'd like to, yes09:23
stublifeless: Unless we go ahead and drop permissions entirely ;)09:23
lifelesswgrant: I'm not sure that the bug is actionable then, because the changelog looks ok to me09:25
lifelessstub: indeed, and thats a curly discussion :)09:25
wgrantlifeless: It's indeed no longer as it was described in the bug.09:26
wgrantlifeless: It's not ideal, but it's not quite so bad.09:27
wgrantIt seems to show the changelog_entry from every publication in the series.09:27
wgrantRather than just one per pocket.09:27
lifelesswgrant: can you update the bug?09:27
wgrantBlah, OK.09:28
frankbanStevenK: can you confirm the problem is here? https://lpbuildbot.canonical.com/builders/lucid_db_lp/builds/1596/steps/shell_6/logs/stdio09:30
StevenKfrankban: The summary is much better for that sort of thing09:32
StevenKfrankban: But the first error is "ProgrammingError: permission denied for relation milestonetag" which puts the smoking gun firmly in your hand.09:32
frankbanStevenK: ok, seen09:34
wgrantlifeless: Hah09:34
lifeless?09:35
wgrantI think archiveremovalredesign fixed it.09:39
wgrantBut annotate keeps misleading me.09:39
wgrantAh, no, was accidentally fixed in r561309:42
wgrant[r=barry] Fixing bug #179028 (+files does not work for removed09:42
wgrantsources). Now SourcePackage class is able to traverse to source09:42
wgrantpackage releases removed from disk.09:42
_mup_Bug #179028: +files doesn't work for removed SPRs <lp-soyuz> <Launchpad itself:Fix Released by cprov> < https://launchpad.net/bugs/179028 >09:42
lifelesswgrant: https://bugs.launchpad.net/launchpad/+bug/144620 is stale too09:46
_mup_Bug #144620: Some displayed sourcepackagerelease changes files don't have attribution <lp-soyuz> <Launchpad itself:Triaged> < https://launchpad.net/bugs/144620 >09:46
wgrantlifeless: Fixed09:47
bigjoolswgrant: only poppy on germanium went bong?09:55
wgrantbigjools: Yes.09:59
wgrantSame as last time.09:59
wgrantI assume it's load-based.09:59
wgrantWe should watch cocoplum's carefully, but I don't want to revert it unless we have to, because of the publisher optimisations.09:59
bigjoolsyeah10:00
bigjoolsgah it's that weird logging thing again10:00
wgrantReally?10:00
wgrantThe top of the latest log has that.10:00
wgrantBut only because rotation isn't happening.10:00
bigjoolsbWTF10:01
bigjoolswhy10:01
wgrantBecause logrotate isn't set up, I assume :)10:01
bigjoolsoh it has 2 log files, one is old style one is new10:01
bigjoolsnothing to do with that10:01
bigjoolsthere's some recursive logging going on10:02
bigjoolslifeless: need your halp10:02
bigjoolshttps://bugs.launchpad.net/launchpad/+bug/90621110:02
_mup_Bug #906211: poppy died in a pulp of its own swappiness <Launchpad itself:Triaged> < https://launchpad.net/bugs/906211 >10:02
bigjoolsthe recent changes made no difference10:02
wgrantbigjools: Huh, didn't notice that recent bit.10:03
wgrantOh.10:03
wgrantbigjools: That's *after* the revert.10:03
wgrantOf course your reverted changes didn't help :)10:03
bigjoolsah10:04
bigjoolswhat time was it reverted?10:04
wgrantAn hour after the >1000 load was reported.10:05
wgrantFor that hour it was running the new code, but restarted.10:05
wgrant07:42Z the rollback completed10:05
wgrantThis should be on LPS, hopefully.10:05
bigjoolsgeez, log looks perfectly normal10:06
wgrantLook towards the end.10:06
wgrantDuring the deathspiral.10:06
bigjoolshmmm10:06
bigjoolswhat bit?10:07
wgrantThere's very frequent DTPFactory logs or whatever it is.10:07
* wgrant relooks.10:07
wgrantOh, blah, has it rotated the logs away again.10:07
bigjoolsit's there10:07
bigjoolspoppy-sftp.log.100610:07
wgrantThere's also one from around the outage in ~wgrant/poppy-sftp.log10:08
wgrantOops10:08
wgrantjust ran ls in lp_upload10:08
bigjoolsbad idea10:08
lifelessbigjools: hi10:08
bigjoolslifeless: hi - sorry was looking at wrong log, but still could do with your help!10:09
wgrantbigjools: Look at the last couple of seconds before the world ended.10:09
wgranteg. 06:18:4810:09
bigjoolsyeah10:09
wgrantDozens of lines of DTPFactory crap.10:09
bigjoolshmmm10:10
bigjoolsI see a lot of entries from hyperair10:10
wgrantThat's true, but from a different IP.10:11
wgrantThe initial symptom was that SSH auth failed.10:11
wgrantBut that took a day or so to appear.10:11
bigjoolsthat has been happening on branches too10:11
wgrantHmmmmm, that's true.10:11
wgrantVery occasionally.10:11
wgrantBut entirely unexplained.10:11
bigjoolswell some guy filed a question about it, was consistentely failing for him10:12
* bigjools has cold fingers and can't type today10:12
wgrantYou mean the question on like Friday about it?10:12
wgrantI thought that was poppy.10:12
lifelessbigjools: do you think https://bugs.launchpad.net/launchpad/+bug/39630 is high still?10:13
bigjoolsah yes it was poppy10:13
bigjoolslifeless: yes10:14
wgrantHowever, I checked when I saw that and it was working at that point.10:14
wgrantSo it broke then fixed itself :)10:14
bigjoolsthis looks like a DoS to me10:15
wgrantIt does.10:15
wgrantBut I'm more likely to blame poppy.10:15
bigjoolsright - we need a connection limit10:15
bigjoolsin the meantime, let's have a word with mr hyperair10:16
lifelessdoing the RT for haproxy on the uploaders will get you that for free10:16
lifeless(just saying)10:16
wgrantIs that 1962 a port number?10:16
wgrantI'm not sure how that logging works.10:16
lifelessbigjools: so, you're set, you don't need me ?10:16
bigjoolslifeless: yes, thanks and sorry to bother10:16
lifelesshey no worries10:16
lifelessI chose to be on IRC :P10:16
lifeless(and to re-triage 75 high bugs...)10:16
bigjoolswgrant: so I think we can revert the revert10:16
bigjoolsgiven that the old one has known issues10:17
wgrantbigjools: There's still the SFTP issue.10:17
bigjoolswas that happening before the new code rollout?10:17
wgrantNo.10:17
bigjoolsthe sftp code has not been touched10:17
bigjoolshard to see how it was broken10:17
wgrantIt's clearly broken it somehow.10:17
wgrantNFI how, though :/10:18
lifelessbigjools: SSHServer was touched wasn't it ?10:18
wgrantIt's happened twice now.10:18
lifelessbigjools: which is the backing for sftp10:18
bigjoolslifeless: yes - but only logging10:18
lifelessagreed; just being pedantic10:18
bigjoolslifeless: can you see how that might cause auth fails?10:18
bigjoolspedanticism is good here10:18
lifelessbigjools: only thing I can think of is something logging triggering a barf which then oops, or something.10:19
wgrantRemember that auth goes via XMLRPC10:19
lifelessdid we see oopses for the auth errors?10:19
wgrantSo it's async.10:19
bigjoolsthere's things like "unauthorized login: unable to get avatar id"10:19
bigjoolswgrant: ?!10:19
bigjoolslifeless: AHA10:19
bigjoolsthere's a timeout failure10:20
bigjoolsright at the same time as the auth starts10:20
wgranttimestamp?10:20
* bigjools pastes10:20
bigjoolshttps://pastebin.canonical.com/57356/10:20
wgrantAh, yes.10:21
wgrantSo there is.10:21
bigjoolsseems to happen on all attempts10:21
wgrantI was looking at the wrong hour.10:21
wgrantYeah10:21
wgrantSo it sits there for 30s.10:21
wgrantAnd then returns a TimeoutError10:21
bigjoolsand as usual Twisted is unhelpful with no traceback10:21
lifelessfirewall issue?10:21
wgrantNo.10:22
wgrantSince a restart fixes it.10:22
bigjoolsI think someone has got frustrated and set up a loop to keep retrying10:22
wgrantInteresting.10:22
wgrantbigjools: 2011-12-16 10:19:4010:22
lifelessnn10:22
wgrant4 seconds before the first occurrence of the TimeoutError10:22
wgrantThere's lots of DTPFactory again.10:22
bigjoolsnn lifeless10:22
wgrantFrom that same IP.10:23
wgrantHmm10:23
wgrantInteresting.10:23
wgrantMy tests were 3 minutes before that.10:24
wgrantHowever, I did those same tests on cocoplum and it survived.10:25
bigjoolswgrant: I can see others connecting to SFTP and validating ok10:31
wgrantWhere?10:31
bigjools02:22:36+000010:32
bigjoolsINFO:poppy-sftp.Hooks:Post-processing finished10:33
bigjoolsoh it's ftp10:33
bigjoolsignore me10:33
wgrantheh10:33
=== almaisan-away is now known as al-maisan
cjwatsonlifeless: 87012> commented10:37
bigjoolswgrant: so I uploaded to cocoplum over SFTP ok10:37
wgrantbigjools: Yes, cocoplum continues to work.10:38
wgrantIt's not had to be restarted once.10:38
bigjoolsgermanium is not happy10:38
wgrantIsn't it fine now it's reverted?10:38
bigjoolsand coupled with the other issue where it needs restarting every 6 days ...10:38
bigjoolsignoring the revert10:38
wgrantThe new code needs restarting every 6 *hours*.10:38
wgrantOn germanium.10:38
StevenKPerhaps germanium is just getting hit so much harder than cocobanana.10:39
bigjoolsnice,p nearly an hour to do a soyuz logsync10:39
wgrantThat's been my assumption for a long time.10:39
wgrantbigjools: It's because germanium is in its logdeathspiral.10:39
wgrantSo it's redownloading 30GB of logs every time.10:40
StevenKgermanium is hellishly overloaded. News at 11.10:40
bigjoolswgrant: so, right, the first failure was just after your tests10:40
wgrantIt's not overloaded.10:40
wgrantOur code is just shit.10:40
wgrant:)10:40
wgrantbigjools: Yep10:40
wgrant3 minutes later or so.10:40
StevenKSadly, that isn't news.10:40
wgrantbigjools: Very odd.10:40
wgrantbigjools: But I did very similar things to cocoplum several times over the weekend, AFAICR.10:40
bigjoolsI'd prefer not to judge until we know everything10:40
wgrantAnd it continues to live...10:40
* gmb wishes pushing a branch up to a local codehosting instance didn't take so flaming long.10:58
* gmb finds something else to do in the meantime10:58
wgrantgmb: What sort of branch?11:13
wgrantIt's normally lightning fast for me, unless you get hit with big 2a fetch slowness.11:13
gmbwgrant, A bigun. I'm working on bug 808930.11:21
gmbMy complaint should have read "_this_ branch"11:21
wgrantgmb: If you want to cheat, you can just copy it to the right place under /var/tmp/bazaar.launchpad.dev/mirrors/11:22
gmbwgrant, Wouldn't I then need to manually create the job(s) for the branch scanner?11:22
gmbOr is there a cheat for that, too?11:23
wgrantgmb: You can then do a no-op push to create the job.11:23
gmbAh!11:24
gmbYes, of course.11:24
gmbwgrant, You wise, wise monkey you. Thanks.11:24
wgrantA no-op push may not work -- but if not, just push -r-2 or so to ensure the head changes.11:24
gmbUnderstood.11:24
wgrantgmb: You may want to talk to jelmer about this.11:26
wgrantSince he has a branch to rework bits of the scanner.11:26
wgrantRelevant bits, I assume -- it made it several times slower.11:26
gmbwgrant, We talked last week about it. This bug looks very much like death by SQL.11:26
wgrantAh11:26
wgrantDidn't know that.11:26
gmbLots of SELECTS and INSERTS in RevisionSet.new()11:27
wgrantAh11:27
wgrantIn that case an initial scan might not trigger it.11:27
wgrantThe initial scan is a special-case.11:27
wgrantWhich is much more efficient in the general case.11:27
gmbAh.11:27
wgrantBut it depends on the branch.11:28
gmbHmm.11:29
gmbWell, I'll get this one into codehosting .dev and see what happens. Having to re-do a bunch of stuff since my Oneiric VM decided to go wonky this morning; I had to restore from a snapshot.11:30
wgrantgmb: You may be able to use the example in comment #5 to reproduce the partial scan slowness.11:31
wgrant(it's even one of yours)11:31
wgrantIt's still broken on prod.11:31
gmbAh yes. I hadn't scanned that far down the bug :). I was working with info from benji, since I've taken this over from him.11:32
=== al-maisan is now known as almaisan-away
wgrantThere may well be two reasonably separate bugs here.11:37
gmbwgrant, You mean death by SQL + the bug for which jelmer has been making optimisations?11:40
gmbOr something else11:40
gmb?11:40
wgrantThe initial and partial scans may be separate code bugs, but both are covered in that single report.11:41
gmbAh, ISWYM.11:41
gmbGood point.11:41
wgrantBecause we know that scanning a Launchpad branch from scratch works.11:42
wgrantBut bringing up to date a Launchpad branch that is a year old appears not to.11:42
gmbIndeed.11:43
wgrantThen there's another separate issue, where inserting a whole lot of new Revisions (not BranchRevisions) takes too long.11:43
wgrantMostly occuring with new massive svn/git imports.11:43
wgrantSo there's probably at least three bugs here, conflated in that one report :)11:43
gmbOh, joy.11:43
wgrantI aim to please.11:44
gmb:)11:44
gmbI'll try and explore the first two, at least. Not sure I'll have time to tackle the third problem.11:44
wgrantYou may have little choice.11:45
gmbSssh11:45
wgrantScanning a massive branch into a dev instance may well fail with issue #3, because all the Revisions are new.11:45
gmbDon't want to think about that yet...11:45
gmbActually, that would fit well with what Benji's observed so far, AFAICT.11:45
gmb(I've got lots of how-to-follow-his-tracks info but no actual data to analyse)11:46
wgrantYeah11:46
* gmb lunches11:57
=== almaisan-away is now known as al-maisan
=== Beret- is now known as Beret
=== nigelb_ is now known as nigelb
stubWe should fix that. No need to create all the revisions in one transaction.12:54
=== benji changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: benji | Firefighting: - | Critical bugtasks: 3*10^2
=== micahg_ is now known as micahg
=== matsubara is now known as matsubara-lunch
rick_h__benji: have a sec to review? https://code.launchpad.net/~rharding/launchpad/batch_nav_900900/+merge/8593514:27
rick_h__or sorry, I'll bug jcsackett since he just arrived and originally peeked at it14:27
rick_h__so benji nvm and jcsackett can you look at https://code.launchpad.net/~rharding/launchpad/batch_nav_900900/+merge/85935 again when you get settled?14:27
jcsackettrick_h__: happy to look again. :-)14:28
rick_h__jcsackett: ty much14:28
jcsackettsame branch i looked at the other day?14:28
benjik14:28
rick_h__jcsackett: yep, just modified the test for the lower batch14:28
rick_h__jcsackett: missed the test over in lp/app when the change was in canonical/launchpad so thanks for catching that and saving me a test run14:29
jcsackettrick_h__: your welcome. with the change mod, this is r=me. :-)14:29
jcsacketts/change mod/test change/14:30
rick_h__jcsackett: awesome, thanks14:30
* jcsackett goes to drink his coffee so he typoes less.14:30
rick_h__has anyone run into this ec2 land error lately? https://pastebin.canonical.com/57375/14:52
rick_h__it worked over the weekend, so wondering if my package updates from last night have caused me pain now14:52
jmlis it possible to get the binary package urls from a BinaryPackagePublishingHistory object?15:13
jmllooking in publishing.py, the answer is no.15:14
bigjoolsjml: urls to what exactly?15:14
jmlbigjools: the debs15:15
bigjoolsjml: ah, no, but it's trivial to contruct a path to the archive15:15
jmlbigjools: what SourcePackagePublishingHistory.binaryFileUrls gets you the URLs for, except from one BPPH rather than the BPPH's associated with an SPPH15:15
bigjoolsthat's returning librarian URLs15:16
bigjoolsI think it was done for the security guys15:16
jmlso it's not actually recommended to download files from the URLs that Launchpad provides?15:17
bigjoolseither is fine15:19
bigjoolsbut I tell people to use the archive url, because it's less likely to be unavailable15:19
jmlbigjools: and the way that's trivially constructed is to inspect the distro, component and binary package name to make a URL to the deb in the pool area, right?15:20
bigjoolsthat sort of thing15:21
bigjoolsbut if you want to get it from the API, file a bug :)15:21
jmlwell, I might submit a patch.15:21
bigjoolseven betterer15:21
cjwatsonbenji: could you have a look at https://code.launchpad.net/~cjwatson/launchpad/soyuz-test-mail-ordering/+merge/85979 ?  Just a test fix15:23
benjicjwatson: sure15:23
bigjoolsjml: actually you need to return a proxied librarian file like the other bits of code, in case it's a private archive15:24
jmlbigjools: yeah. I was planning on factoring out what's in SPPH, and have SPPH delegate to BPPH to figure out binary urls.15:24
bigjoolsjml: ok, careful it doesn't end up potato programming15:25
jmlbigjools: will do.15:25
jcsackettsinzui: do you know if wallyworld is actually working on bug 885524?15:28
* sinzui looks15:28
sinzuijcsackett, not directly15:29
jcsackettsinzui: yeah, it looks like he's working on a related bug. so i'll leave that alone to avoid stomping on toes.15:29
sinzuijcsackett, wallyworld__is working on bug 904295 which is a pre-req and may actually fix the issue15:30
sinzuijcsackett, lets mumble about the remaining cards. I have to pick one too15:30
jcsackettsinzui: dig. be on mumble in just a moment.15:31
benjicjwatson: the branch looks good15:34
rick_h__adeuring: deryck  https://pastebin.canonical.com/57375/15:39
deryckrick_h__, we lost you.15:40
=== jpds_ is now known as jpds
adeuringderyck: https://code.launchpad.net/~adeuring/launchpad/bug-903359/+merge/8624415:43
cjwatsonbenji: lovely, thanks.  I have no PQM access; do you think you could send it to EC2 for me?15:44
benjicjwatson: sure15:44
cjwatsonta15:44
benjicjwatson: EC2 has been engaged: http://ec2-184-72-167-164.compute-1.amazonaws.com/15:54
deryckrick_h__, did your bazaar.conf change by any chance?  i.e. does it still exist in $HOME/.bazaar ?15:55
rick_h__deryck: no, it's still there and seems normal15:56
rick_h__deryck: I did have to make some changes to my locations.conf to get a manual land in pqm over the weekend15:56
rick_h__but that was just adding some email settings. Removing them again doesn't help me15:56
deryckrick_h__, ok, it seems to be choking over email and signing the submission some how.15:57
rick_h__deryck: right15:57
deryckgary_poster, are you around?15:58
gary_posterderyck, hi15:58
deryckgary_poster, hi.  Trying to help rick_h__ fix this error: https://pastebin.canonical.com/57375/  and I'm really not sure what's up. Have you seen that before?15:58
gary_posterderyck, no.  that's a curious one.  The bzr-2.5 makes me suspicious though16:00
gary_posterI wonder if 2.4 will be better16:00
deryckah yeah, missed that.16:00
gary_posterthough I guess that means we are all using 2.5 though?16:00
deryckand it's using the egg version not his local bzr.16:00
gary_posterright16:00
deryckI didn't realize we did that.16:00
gary_posterme either deryck :-P  or at least didn't remember, I dunno :-)16:01
deryckheh16:01
rick_h__my local bzr is  2.5.0~bzr6383.6136~ppa4002~oneiric116:01
deryckseems kind of wrong to me, but I'm sure someone had a good reason to.16:01
deryckbut I guess if we're all on dailies we'd be on that.16:02
gary_posterrick_h__, I see this on my oneiric: $ bzr --version16:02
gary_posterBazaar (bzr) 2.4.216:02
rick_h__gary_poster: yea, I did an upgrade this morning and that seems to have blown me up16:02
gary_posterah ha16:02
rick_h__gary_poster: I saw bzrlib and launchpadlib updates16:02
rick_h__gary_poster: yea, just call me the canary in the coal mine I suppose16:02
gary_posterrick_h__, heh :-)  so we have two options I guess16:03
deryckyeah, I'm still on 2.4.2 as well, though I thought I ran the daily.  Guess I killed it at some point.16:03
gary_poster1) try going back to 2.4.216:03
gary_poster2) dig in and fix it.16:03
gary_poster:-P16:03
gary_posterI vote for option 1 but somebody has to pay the price16:04
gary_posterit would be nice if that person had some bzr experience though16:04
gary_posterrick_h__, can you try option 1?16:05
rick_h__gary_poster: yea looking into it. Set a pin and upgrade again?16:05
gary_posterrick_h__, yeah.  (I'd do it in aptitude but that's because that's what I'm comfortable with)16:06
rick_h__gary_poster: ok, looking. Been years since I've pinned a package16:06
gary_posterThe thing that will need to be fixed will be either pqm or bzr (or both conceivably)16:07
gary_posterrick_h__, do it in aptitude then. It's easy :-)16:07
gary_posterMaybe synaptic too16:07
rick_h__gary_poster: installed16:07
gary_postercool16:07
rick_h__gary_poster: so the 'F' command?16:09
gary_posterrick_h__, this is what I do, for a temporary test change:16:10
gary_posterfind the package in aptitude16:10
gary_posterPress enter16:10
gary_posterscroll to the bottom16:10
gary_posteryou should see the different options to install16:10
gary_posterwith an i on 2.5 presumably16:10
rick_h__gary_poster: yea, I see the p    2.4.216:10
gary_posterright16:11
gary_posterpress "+" on that16:11
gary_posterthat will be a temporary install I think.  If not,16:11
gary_posteryeah, no that should do it.  If you want to keep it, use "=" on the package in the future16:11
rick_h__ugh, looking, wants to remove 134 package16:12
gary_poster(or remove the bzr daily ppa :-) )16:12
gary_poster:-(16:12
deryckrick_h__, also, if you're on the daily, you can remove that from sources and go back to the 2.4.2 in O.16:12
gary_posterjust do downgrade?16:12
gary_posterto16:12
derycksorry gary_poster just said what I said :)16:12
rick_h__gary_poster: well I hit + on it and it says "Suggest 134 removals"16:13
rick_h__gary_poster: not sure if it's going to carry out that suggestion though I suppose16:13
gary_posterderyck, I don't think that alone will downgrade (because of doing something similar recently), you have to explicitly downgrade or else the packages hang around16:13
rick_h__yea, if I remove the ppa I'd still have to remove/reinstall or downgrade16:13
gary_posterrick_h__, ok, if it is not working easily then this is a blindman leading...you take a list at the suggestions and see what you think, or we can find someone else to talk to :-)16:14
gary_posterI mean, I am a blind man leading :-)16:14
rick_h__gary_poster: going to see if I can remove the ppa and see what it wants to do to remove bzr16:14
rick_h__then reinstall16:14
gary_posterok16:14
gary_posterhopefully this is actually the problem :-P16:15
gary_posterit seems like it though16:15
gary_posterI need to head out soon16:15
rick_h__gary_poster: ok thanks16:15
gary_postermy wife and baby are calling me to an early lunch :-)16:15
deryckadeuring, r=me on that change.  looks great!  Thanks.16:15
adeuringderyck: thanks!16:15
deryckrick_h__, gary_poster -- yeah, I'm sorry, I meant drop the daily from sources and uninstall/reinstall.  That's how I usually do it. Maybe that's the clunky way.16:16
* deryck has weak apt foo16:17
rick_h__deryck: yea, that's worked for removing/adding16:18
rick_h__deryck: working on updating devel/make clean/etc to test out16:19
gary_posterrick_h__, I suggest shooting a note to launchpad-dev about what you encountered so that others know the problem and can investigate16:22
gary_posterassuming this fixes it16:22
rick_h__gary_poster: will do16:22
gary_posterthx16:23
rick_h__deryck: gary_poster landing is processing thanks for the notice of the 2.5 and help with that16:25
bigjoolsderyck: apt-get package=<release name>16:33
=== matsubara-lunch is now known as matsubara
=== al-maisan is now known as almaisan-away
deryckbigjools, ah, thank you.16:35
bigjoolsde nada16:35
deryckadeuring, I reproduced the double count locally, saw it doing: https://bugs.launchpad.dev/ubuntu/++profile++show/+bugs16:37
deryckadeuring, and this fixes it:  http://pastebin.ubuntu.com/775460/16:37
* adeuring is looking16:37
deryckI had to build up my local data to get a couple batches as we discussed this morning.16:37
deryckwithout the extra batch we don't even do a single count because we don't need lastBatch.16:37
adeuringderyck: very nice fix!16:40
deryckadeuring, thanks!  seems simple enough too.16:40
adeuringyeah16:40
deryckI'll fix my branch to use this and update the MP.16:40
deryckadeuring, I guess we can drop all the stuff to avoid lastBatch now.  I did the pastebin patch in devel, where the older use of lastBatch still exists.16:41
adeuringderyck: right, the old patch should be obsolete16:41
deryckadeuring, ok, cool, that was my thinking too.16:42
deryckadeuring, do you care to weigh in or review the updated https://code.launchpad.net/~deryck/launchpad/avoid-extra-buglist-count-901124/+merge/85940 ?17:39
adeuringderyck: sure17:40
adeuringi'll look17:40
deryckadeuring, thanks17:40
jmlis there an API for determining a source package name given a binary package name?17:43
adeuringderyck: r=me17:43
deryckadeuring, thanks much!17:43
sinzuijml: no. guessPublishedSourcePackageName() is closed to what you want, but I was thinking of removing it once we update all the callsites to use the new DSP vocab18:45
=== matsubara is now known as matsubara-afk
* sinzui ponders if the vocab is public in a way to used it18:45
lifelessprobably not18:53
derycklifeless, hi.  I believe I have finally *really* fixed the double count issue now.18:56
derycklifeless, I'd appreciate you looking again at the MP if you don't mind.18:56
lifelessderyck: I have18:56
lifelessderyck: :)18:56
deryckah ok :)18:56
lifelessderyck: it looks good to me, the new-batch object is a common cause of this18:57
deryckthanks!18:57
lifelessderyck: I didn't look at the larger context; a lot of other views use a cached property rather than an explicit instance variable18:57
lifelessbut 6/1 1/2 dozen of the other18:57
deryckyeah, I'm not picky either if you prefer the other way.18:57
* lifeless shrugs18:57
=== aldeka_ is now known as aldeka
lifelessderyck: I *had* replied, but lp hasn't seen it.19:25
lifelessderyck: I suspect incoming mail processing is naffed19:25
deryckah, I wondered if that's what you meant.19:26
lifelessand now I can't find it in my outbox.19:28
lifelessENOIDEA.19:28
lifelesssomething odd is happening though, I'm trying to add a comment on https://code.launchpad.net/~deryck/launchpad/avoid-extra-buglist-count-901124/+merge/85940 and its stuck on 'Saving' for way more than the 30s ha proxy timeout :(19:30
deryckhmmm, weird.19:30
derycklet me try....19:30
derycklifeless, I just added a comment no problem.19:31
deryckless than a second XHR call.19:31
lifelessweird19:32
lifelessoh19:33
lifelessmerge proposal19:33
lifeless-> long poll19:33
lifelessI wonder if I'm hitting my browser host connection limit19:34
lifelessyes, I was19:34
lifelessJULIAN!19:34
lifeless:)19:34
deryckah ha! :)19:37
lifelessbug 90648219:37
lifelessbug 906482 ?19:38
lifelessbah https://bugs.launchpad.net/launchpad/+bug/90648219:38
lifelessderyck: this explains why I hadn't commented previously, it was stuck in a browser tab19:39
lifelessderyck: for an hour or so :)19:39
deryckheh, ouch19:39
derycklifeless, but I've seen your browser.  You really do have too many tabs open. ;)19:40
lifelesssee, thats what 16GB of ram is *for*19:41
deryckheh.19:44
deryckI stand corrected then. :)19:44
lifelessthat and skyrim19:45
deryckno, that's what a PS3 is for. ;)19:45
lifelessheh, Lynne plays it on the PS319:45
deryckthat and dcuo.19:45
lifelessit is -much- shinier on a PC19:45
deryckyeah, most games are actually.19:45
deryckbut it's so comfortable gaming from my bean bag on the floor in front of my 47 inch TV.19:46
lifelessthere was a period where the consoles were better, but then moores law kicked in :)19:46
deryckheh, yeah19:46
lifelessbean bags++19:46
lifelessanyhow, as I'm on leave, I think I'm going to go exercise some of that there gaming stuff ;)19:46
deryckheh. 3 more days for me.19:49
lifelesssinzui: nice curly bug there20:25
sinzui?20:25
lifeless61161720:25
sinzuilifeless, yes. I really am not sure what to do to close it20:26
lifelessI don't think the person-linking aspect matters, because the uploader will always be a person (they have to gpg sign the package, which they cannot do as the team)20:26
lifelessthe maintainer field can be a team - its free form text; there are two issues:20:27
lifeless a) only allow specifying a private team as maintainer when the uploader can see that team/ is a member of the team20:27
lifeless b) disclosure of the team because of this20:27
lifeless-> I think20:28
sinzuiI do not think the team should be disclosed in the case of a private team's archive. Copying to a public archive makes the relationship public20:29
lifelessright, I'm saying that there is a probing attack20:29
sinzuiah20:29
lifelessi could upload to my public ppa with the maintainer set to 'private-canonical-XXX'@lists.l.n20:29
lifelessand if it links, voila, I can see the team.20:30
sinzuilifeless, I was thinking of migrating the logintoken code to lp.services tonight. logintoken is not a good name. I was considering naming the module authentication, then I consider the value it offers is a verification workflow. Maybe lp.services.verification20:34
lifelessthats the email verification stuff ?20:36
lifelessthere is a bug about that using bounces@c.c., and a suggestion to use a new envelope-sender address. Might want to fix that in passing20:37
lifelessit should really be a separate service I think though, let anyone at the company trigger a 'is this a real email' probe.20:37
lifelesssinzui: I've replied on stakeholders, I hope I make sense20:39
sinzuilifeless, agreed. Thanks for the stakeholder reply too20:39
=== jcsackett_ is now known as jcsackett
pooliehi all20:59
poolierick_h__, gary_poster, hi, what is the bzr problem you're talking about?20:59
gary_posterpoolie, hi.  jelmer replied on the list about it21:01
gary_posterpoolie, see launchpad-dev thread "upgrade today got me bzr 2.5 and broken ec2 land"21:02
rick_h__poolie: yep, did an upgrade this morning and got a bzr 2.5 with the bunch21:06
rick_h__poolie: which pqm plugin wasn't ready for but it's coming it appears21:06
poolieok21:09
pooliea bit out of sync21:10
pooliejelmer, we might need some glue to let you check flags from inside the ssh server?21:17
pooliewhich would be worth having anyhow21:17
jelmerpoolie: that would be nice21:18
jelmerpoolie: I was actually wondering about the forking lp service21:18
jelmerpoolie: it seems we have enough data to let people switch to the forking server by specifying a different bzr_remote_path21:18
jelmerso that we could have the forking lp server available without immediately forcing it on all codehosting users21:21
pooliethat would probably work21:31
pooliewe could possibly do something like this in a client controlled way21:39
lifelesswe need, if we don't have it already, and xmlrpc flag checking api21:50
=== benji changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugtasks: 3*10^2
sinzuiwallyworld__, see https://bugs.launchpad.net/launchpad/+bug/85567022:34
mwhudsonlifeless: i add a xmlrpc flag checking api22:35
mwhudsoni keep meaning to revisit anonymous ssh connections using it22:35
wgrantlifeless: "and if the package is22:43
wgrantcopied to a new context, inform the copier it will lead to disclosure"22:43
wgrantUh, how exactly?22:43
wgrantWe can't change *every single API in Launchpad* to prompt if there's a private team involved.22:43
wgrantThat's just stupid.22:43
sinzuiStevenK, this is the bug I was thinking of bug 56708822:48
StevenKhttps://bugs.launchpad.net/launchpad/+bug/90615122:50
poolielifeless, http://www.stathat.com/22:53
jmlsinzui: that means there's no functional way to get binary package urls from bpph objects over the API. that's ok. will fix.22:54
poolieo/ jml22:54
jmlpoolie: hi22:54
pooliehi there22:54
wgrantjml: There's not always a corresponding file in the archive.22:57
wgrantjml: You may be better off exposing the webapp URL of the file, on BPPH or BPB.22:57
jmlwgrant: and yet spph has binaryFileUrls22:57
wgrantRight, those are webapp URLs.22:58
wgrantNot archive ones.22:58
jmlwgrant: I'd be happy w/ webapp urls. and if I were to expose that'd be the natural path.22:58
jmlwould just refactor binaryFileUrls to delegate properly22:58
wgrantThe source name is not relevant to those, so you can probably even calculate them now.22:58
jmlwgrant: I haven't even looked to see what they are23:02
jmlblack box ftw.23:03
wgrantHeh23:03
wgrant/ubuntu/+archive/primary/+files/FOO_VERSION_ARCH.deb23:03
huwshimiIf you add a second branch to a bug that is already fix-released will that branch get deployed to qa staging?23:38
huwshimi(this bug: https://bugs.launchpad.net/launchpad/+bug/894442)23:39
StevenKhuwshimi: Yes23:39
huwshimihmmm...23:39
huwshimiStevenK: Any reason why this one hasn't then?23:40
StevenKhuwshimi: Is the branch linked to the bug?23:40
huwshimiStevenK: Yeah23:40
huwshimiStevenK: Wait, it doesn't look like it merged23:40
StevenKI was about to say :-P23:40
huwshimiStevenK: shush23:42
lifelesspoolie: yeah, basically statsd23:44
lifelesspoolie: we're outside the free zone anyhow, ignoring disclosure considerations23:45
poolielifeless, yeah, basically statsd+better displays, i think23:51
pooliefree zone?23:51
pooliei'm not saying we should actually use it23:52

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!