[00:20] <Logan> doko: eek
[11:50] <coreycb> hello, can someone on the release team please take a look at the following FFEs:  bug 1488094 and bug 1489096?  this will help unblock most of the openstack packages.
[11:50] <ubot93> bug 1488094 in glance (Ubuntu) "[FFE] glance liberty-2 release" [Undecided,New] https://launchpad.net/bugs/1488094
[11:50] <ubot93> bug 1489096 in python-troveclient (Ubuntu) "[FFE] Please sync python-troveclient (1:1.2.0-4) from Debian (experimental)" [Undecided,New] https://launchpad.net/bugs/1489096
=== thierry-ibm is now known as thf
[17:08] <teward> can an archive admin take a look at and handle this removal request? https://bugs.launchpad.net/ubuntu/+source/electrum/+bug/1481033
[17:08] <ubot93> Launchpad bug 1481033 in electrum (Ubuntu) "Please remove electrum from the archive" [Undecided,New]
[17:08] <teward> been almost a month since filed
[17:08] <infinity> teward: Looking.
[17:08] <teward> infinity: thank you
[17:09] <infinity> Oh look, yet another bitcoin thing.
[17:09] <teward> infinity: yeppers.
[17:09] <teward> i hate having to say things should be removed, but...
[17:10] <teward> (it should be treated as an independent entity from bitcoin source package though)
[17:10] <infinity> teward: Removing it from wily won't do much for <= vivid, do we care?
[17:10] <infinity> And should we care?
[17:10] <teward> infinity: i've had it on my radar to "null" the package
[17:10] <teward> should we care?  I think so.
[17:10]  * infinity nods.
[17:10] <teward> is it on my radar to propose an SRU package? yes.
[17:11] <teward> i'm a little overloaded with other things on my to do list of higher priority
[17:11] <teward> (for private development projects, or nginx ppas)
[17:11] <infinity> Alright, cool.  There should be prior art there in the form of other packages we've replaced with null packages with NEWS.Debian and debconf notes.
[17:11] <infinity> Other bitcoinish stuff, I suspect. :/
[17:11] <teward> i'm going to look at the nulled bitcoin source and look there
[17:11] <teward> and perhaps use that as a base.
[17:11] <teward> infinity: in the interim can we get it removed in wily?
[17:11] <infinity> teward: Yup.  On it.
[17:12] <teward> i know Debian's bitcoin people were poking @ the latest code but no new package updates're there
[17:12] <teward> so... :P
[17:12] <infinity> teward: I'll get it done after this call.  Multitasking is hard.
[17:12] <teward> OK thanks
[17:39] <Logan> infinity: I feel like there should be an archive queue, a la sponsoring queue
[17:39] <Logan> there's a lot of stuff that hasn't been handled
[17:47] <teward> infinity: FYI: I updated the bug with the details that older versions should probably be nulled out like bitcoin was but i haven't prepped an SRU debdiff for that yet
[17:47] <teward> but that it's on my radar.
[17:47] <teward> no rush :)
[17:48] <infinity> Logan: There's a queue, it's called stuff we're subscribed to, but we don't check it nearly as religiously as we should.
[17:48] <infinity> Logan: I'm working on getting a couple more people on the team, then we might get to go back to have "AA days" where we each eat 2h/wk on just doing AA stuff.
[17:49] <Logan> I'd love to be on the team, but I'm guessing I'd have to apply to core dev first
[17:49] <infinity> teward: Cool, thanks.
[17:49] <infinity> Logan: Yes, I'm also going to put a proposal to the TB that AA, release, and SRU have official policies around core-devness, thanks for the reminder. :P
[17:49] <Logan> ha, no problem
[17:50] <infinity> Logan: (Well, the technical wording would be that no one should be given queue permissions for a series/set they don't have upload permissions to, but for AA/SRU/release, that would be core-dev)
[17:50] <infinity> Since queue is a backdoor to upload.
[17:50] <Logan> but you could trust people not to use the backdoor maliciously ;P
[17:51] <infinity> Logan: Up to now, yes, we've trusted, but trust is a poor replacement for security.
[17:51] <Logan> fair enough
[17:51] <infinity> (See the US cheque clearing system)
[17:51] <Logan> so there are archive admins who are not core devs?
[17:51] <infinity> Logan: No.  There's one member of -release that isn't, but she's stepping down, allowing me to make this proposal without it looking like I'm firing anyone.
[17:51] <Logan> oh I see
[17:52] <ScottK> infinity: Does that mean you'll fix the back door through the CI train too?
[17:53] <infinity> ScottK: Is that still not enforced by the train properly?  I thought sil2100 and robru were meant to sort that. :/
[17:53] <infinity> ScottK: If it's still an issue, it really shouldn't be, so yes, we need to fix that.
[17:53] <robru> infinity: which backdoor?
[17:53] <ScottK> Last I heard (and I haven't since I exited the DMB) it was still an on your honor check with a dev if there are packaging changes thing.
[17:54] <robru> ScottK: devs don't publish silos though, only core devs & me.
[17:55] <ScottK> robru: Do they have the technical capability to?
[17:55] <robru> ScottK: no, the publish job has an ACL saying only core devs, plus "trainguards" (me & sil)
[17:55] <infinity> robru: The part where you publish silos without having upload permissions is still a problem, mind you.  Unless the people uploading to the silos had upload permissions for the bits.
[17:56] <robru> infinity: yes, that's on the list of things to fix. there's a lot to do
[17:56] <infinity> robru: We need to figure out a way to make sure the system is enforcing "someone with upload rights was involved in this".
[17:56] <robru> infinity: yes I'll be getting to that soon.
[17:56] <infinity> robru: Kay, cool.
[17:56] <robru> infinity: slangasek already even wrote some example code, I just need to get to it.
[17:57] <robru> ScottK: so to clarify, publishing is on *my* honor, not "any random dev's honor"
[17:57] <infinity> Logan: Anyhow, get yourself core-devy, you're a good candidate.  Then if you don't manage to burn yourself out over the next 6-12 months, talk to me about teams with elevated permissions. :)
[17:58] <ScottK> robru: OK.  That's at least better, but unless you're an Ubuntu dev with upload rights for the affected packages, it still breaks the archive trust model (don't take that personally).
[17:59] <Laney> Is it https://launchpad.net/~ci-train-ppa-service/+members#active this team?
[17:59] <robru> ScottK: I understand, it is on the list of things to fix. just swamepd.
[17:59] <robru> Laney: no it's the poorly named ubuntu-unity team which has nothing to do with unity development. it's exclusively used by the train
[18:04] <Logan> infinity: cheers
[18:19] <infinity> robru: For the record, while I'm fine with you working toward core-dev yourself, the solution to the train problem shouldn't be "make robru core-dev so he has permissions to upload any junk", since every package you upload from the train is sponsored by you, and you're taking responsibility for.
[18:19] <infinity> robru: The solution needs to be that we can identify someone else with upload permissions has signed off on the upload and is taking responsibility for it.
[18:20] <robru> infinity: right, was that not clear? we have a bug and some example code for how to make the train respect who has correct upload rights. it's just a matter of doing it.
[18:20] <infinity> robru: Bonus points if that person then becomes the "uploader" in .changes and the changelog sig, so we can stop the madness of attributing thousands of uploads to a bot I can't yell at. :P
[18:20] <infinity> robru: Okay, cool.  Wasn't sure (so, I guess not clear), but glad we're on the same page.
[18:22] <infinity> robru: This is as much about responsibility as it is about permissions, it's been amazingly frustrating to try to track down people who "own" these uploads and can be held accountable.
[18:22] <robru> infinity: even though the changelog entry ends with " -- CI Train Bot", the names in square brackets above that should correspond to the people to yell at.
[18:23] <slangasek> robru: the names in square brackets are the people who prepared the merge, not the people who signed off on it
[18:23] <robru> hm
[18:23] <infinity> ^
[18:23] <slangasek> making people the uploader would require some (probably otherwise useful) changes to the process, since it means signoff would need to happen before build
[18:23] <slangasek> that could be good, or it could be a problematic bottleneck
[18:23] <infinity> In a VCS sense, view this as the difference between 20 commits and a signed tag.
[18:24] <robru> slangasek: that sounds like a bottleneck to me. people build & rebuild constantly, you can't get a sponsor for every single build.
[18:24] <infinity> I care about the committers, but the tagged version is the responsibility of the person who signed off on the whole thing.
[18:24] <slangasek> robru: it would at most be needed for each build that changes the packaging and thus requires sign-off
[18:24] <infinity> And, indeed, if we get to "build from git", that's exactly how it would look.  A bunch of committers in the changelog, but the "uploader" is the one who signed the packaging tag.
[18:25] <robru> slangasek: but that's chicken and egg then. the train can't know whether there are packaging changes until after it makes the build and sees the diff.
[18:25] <slangasek> and there could be an optional bypass of the sign-off if you know you're only doing a test build
[18:25] <infinity> For what it's worth, I oppose completely the premise that only packaging changes need a sponsor.
[18:25] <infinity> Upload permissions are for the package, not the debian directory.
[18:26] <robru> that makes sense
[18:26] <slangasek> robru: I don't see how that's chicken and egg.  It tries to build the source package; it sees that there are packaging changes; it kicks it into "needs signoff" state; someone signs off and then it does it again and actually submits it for building
[18:26] <infinity> If an upload of upstream-only changes breaks the world, I still need accountability.
[18:26] <slangasek> infinity: this was the compromise that was approved by the DMB when the train went live
[18:26] <robru> slangasek: the logic for diffing and finding packaging changes happens after the build is uploaded & completed in the PPA.
[18:26] <infinity> slangasek: I'm inclined to say the DMB was wrong.  Just sayin'.  Rules aren't set in stone. :P
[18:27] <robru> infinity: I'm pretty sure debian rules are set in stone ;-)
[18:27] <slangasek> robru: sure.  that could be changed though if we thought it was better to do it the other way
[18:27] <infinity> Is it so onerous to ask people to get upload rights?
[18:27] <slangasek> no, debian/rules are set in make
[18:27] <infinity> robru: Debian has a living constitution and a living policy, so no.
[18:27] <slangasek> infinity: how many people has the DMB approved upload rights for this year?
[18:28] <infinity> slangasek: Not sure, but that's more interesting compared to people who have applied.
[18:28] <slangasek> and what should the uploader approval process look like for an upstream dev who is a domain expert and doesn't want to take responsibility for the packaging anyway?
[18:28] <infinity> But, really, if upstream changes aren't part of the package, could I have punted that glibc bug that caused endless debate back and said "talk to upstream about it, it wasn't my debian directory that broke it"?
[18:28] <infinity> Accountability is important.
[18:28] <doko> could somebody hint gdal mia vtk6 gtkglextmm ipe grass cgal qscintilla2 bullet graphicsmagick  (or tell me what is still missing)?
[18:29] <slangasek> doko: seen the autohinter output?  there are a bunch of other packages that gdal wants to go in at the same time as, including octave
[18:30] <robru> infinity: rest assured that within a couple weeks the train will be honoring proper upload permissions. I'm just finishing up some other stuff before I start on that.
[18:31] <infinity> robru: Sure, but we seem unclear again on what that means. :P
[18:31] <infinity> robru: If it's only honoring them for packaging changes, I still think it's hilariously broken, DMB discussion notwithstanding.
[18:31] <slangasek> robru: "honoring proper upload permissions" hopefully means "recording in bileto who signed off on packaging" :)
[18:31] <robru> infinity: what part isn't clear? the train will only allow uploads from proper uploaders.
[18:32] <robru> slangasek: that's within the realm of possibility.
[18:32] <infinity> robru: Err, but the conversation we just had implied that's not what you're doing.  So, I'm confused.
[18:32] <slangasek> robru: infinity is trying to argue that the agreed rules should be overturned.  but this isn't really the proper forum for that, and not something you should worry about as implementor of the existing rules
[18:33] <infinity> slangasek: Honestly, I'm trying to clear up what he MEANS by "honoring upload permissions".
[18:33] <slangasek> infinity: ah
[18:33] <infinity> Because if there's a mechanical check on publishing that the upload was approved by someone with upload rights, that's correct (IMO), but not what the current rules state (only for packaging diffs).
[18:33] <infinity> So...
[18:33] <doko> slangasek, yes, and qscintilla2 too
[18:33] <infinity> Which is it? :P
[18:33] <robru> infinity: https://bugs.launchpad.net/cupstream2distro/+bug/1459186 if you're interested
[18:34] <ubot93> Launchpad bug 1459186 in CI Train [cu2d] "Implementing proper permission checks during silo publish" [High,In progress]
[18:34] <slangasek> infinity: packages that are Canonical upstream, exist in the archive, and have no packaging changes don't need an ubuntu-dev to sign off.  Packages that don't have Canonical upstream, or that have packaging changes, must be signed off by an ubuntu-dev with upload rights for the package in question
[18:34] <robru> infinity: feel free to comment on that bug with a detailed explanation of The Right Way to do things so i won't forget it when I get around to it (actually it looks like sil started on that already, hmmm)
[18:35] <infinity> slangasek: So, there's a whitelist somewhere for these Canonical upstream packages?
[18:35] <infinity> Or is that all honor system too?
[18:36] <infinity> Cause they should probably just be in a packageset that trainguards have upload rights to, to codify the DMB position.
[18:36] <infinity> And then this can all be proper archive permissions checks.
[18:36] <robru> infinity: there isn't an explicit whitelist, but the train only supports MPs for canonical upstream packages, so the difference would be "if there's an MP, we're upstream, if it's a manual source in the PPA, we're not"
[18:37] <slangasek> infinity: in practice it's possible the whitelist is of the form "anything that has branches in the form that the train is capable of figuring out an MP for"
[18:37] <robru> yeah, that
[18:39]  * infinity -> lunch.
[18:40] <robru> mmmhm, yes
[18:40] <infinity> robru: I'll try to summarize my views later.  No, I'm not implying one grumpy old man can overturn the current state of affairs, and the current state is probably "fine" if enforced in a more mechanically sane and verifiable way.
[18:41] <infinity> (Though, I'd prefer the exception not exist at all, but whatever)
[18:41] <robru> infinity: yes, I agree, the current system is suboptimal and needs improvements. it's on the list of things I'll fix soon
[18:59] <Logan> ruby-rspec needs ruby-threadorder to be MIRed
[18:59] <Logan> the new version of ruby-rspec will fix a bunch of Ruby FTBFSes
[19:02] <Logan> ruby-thread-order*
[20:31] <cjwatson> robru: oh FWIW putting a real person in the changelog trailer line rather than "CI Train Bot" has been on my list to push for for a while - IMO that should be the silo owner or something like that, because doing that would mean that LP can notify a real person of failures
[20:31] <cjwatson> robru: and hence fix the "cjwatson has to grep logs" problem
[20:32] <cjwatson> (hope I'm not barging in on an existing agreement that implies something contrary, but if so, I wouldn't mind checking over it to ensure that it implies the correct thing for LP mail notifications)
[20:32] <robru> cjwatson: oh yeah, silo owner would be a more reasonable thing to put there. Silo sponsor would be weird and cause problems implementing. Can you file a bug against lp:cupstream2distro for that?
[20:32] <cjwatson> robru: sure
[20:32] <robru> Thanks
[20:34] <cjwatson> "silo sponsor" certainly wouldn't need to be in a changelog - that would be more analogous to syncpackage -s
[20:34] <cjwatson> but IIRC you already pass a sponsor to the copy request
[20:35] <cjwatson> maybe, sometimes
[20:38] <cjwatson> robru: I discussed this with Didier something like a year and a half ago and then forgot to file it anywhere, apparently - https://bugs.launchpad.net/cupstream2distro/+bug/1490729
[20:38] <ubot93> Launchpad bug 1490729 in CI Train [cu2d] "changelog trailer should name the silo owner rather than "CI Train Bot"" [Undecided,New]
[20:39] <robru> cjwatson: yeah it makes sense. just never occurred to me.
[20:40] <cjwatson> infinity: fwiw this doesn't conflict with what you were saying - the changelog trailer line clearly doesn't have to be somebody with upload permissions, because Debian
[20:40] <cjwatson> (or anything else we might copy from)
[20:41] <robru> cjwatson: when launchpad is notifying about upload failures, does it go by the changelog entry? or by the signer? because it'll still be signed by the train key even if I change the name in the changelog.
[20:42] <cjwatson> robru: both
[20:42] <cjwatson> it should be signed by the train key, that's correct
[20:42] <robru> cjwatson: ah ok
[20:43] <infinity> cjwatson: No, the changelog line doesn't have to correspond, it's just also pointless if it's a bot.
[20:44] <cjwatson> robru: in fact, the rules are a bit complicated, but this proposed change isn't going to make it harder to find somebody to notify, at least - and if it's more conceptually correct, it's easier for us to find a consistent way to improve things if it is still broken
[20:44] <robru> cjwatson: yeah good point.
[20:44] <robru> cjwatson: infinity: I can probably get to both of those this week once I finish up some stuff I have going right now
[20:46] <cjwatson> (I believe we may not additionally notify the changed-by person about failures when the target archive is a PPA, but we do when the target is the primary archive, whether it's an upload or a copy)
[20:47] <cjwatson> but failures in copies to the primary archive are the main case that's currently opaque
[20:47] <cjwatson> oh, we only add changed-by if it's a person with component upload permissions, hm
[20:48] <cjwatson> anyway, as I say, we could at least *conceptually* improve it given that change :)  more information good
[20:48] <infinity> cjwatson: Yeah, the mail rules are intentionally restrictive to avoid backscatter to DDs (and others) who aren't also ubuntu-dev.
[20:49] <cjwatson> right, and also to avoid "person copies my Ubuntu package to a PPA, I shouldn't get spammed about it"
[20:49] <infinity> cjwatson: But I want the information purely for better tracking/blame, what we do with it at the infrastructure level is a whole different story.
[20:49] <cjwatson> the fact that we only mail component uploaders is curious - I wonder if I'm the first person to notice that
[20:50] <infinity> (And I also contend that "this package is owned by a bot" is completely useless information to anyone)
[20:50] <ScottK> infinity: It does always seem a little odd to me to get LP mail about stuff that's autosynced from Debian.
[20:50] <infinity> The only place the bot should be evident is the signing key for the sponsored upload.
[20:50] <cjwatson> ScottK: I think this is because the changed-by in that case is attached to your LP account and you have component upload privileges to Ubuntu
[20:51] <cjwatson> so it's evident to people who work in both distributions, but at least not to Debian developers in general
[20:52] <ScottK> That's good.  If it were general, then I'm sure there'd be lots of complaining.
[20:52] <cjwatson> hm, I just rewrote a chunk of this code, I wonder if that means I'm now on the hook for all its deficiencies :-)
[20:52] <infinity> cjwatson: I believe that's how it works, yes.
[20:53] <infinity> cjwatson: I'm not sure at what percentage of rewrite you get to stop cursing cprov and blame yourself entirely, mind you.
[20:55] <cjwatson> infinity: for once, Celso is just about the only Soyuz hacker *not* implicated in this code
[20:55] <cjwatson> ah, but I think that's because of a rearrangement by Steve in 2011 - anyway, ancient history
[20:59] <infinity> cjwatson: Blaming him is fun regardless. ;)
[21:29] <teward> infinity: care to do a very-brief glance-over on the SRU for Trusty to 'nullify' the electrum package?
[21:32] <infinity> teward: I'm a bit busy at the moment, but perhaps if you give me a nudge tomorrow.  Or ask someone else. ;)
[21:34] <infinity> teward: Hrm.  Does the part where Debian has 2.4 in unstable change this conversation at all?
[21:34] <infinity> teward: I expected that if it was universally considered a Bad Idea to distribute it, they would have removed it.
[21:36] <teward> infinity: mmm
[21:36] <teward> 2.4.4 is updated but iirc they didn't close the ticket yet.  At the time of the bug here for removal, the issue was they were giving pushback
[21:37] <teward> they also need to make sure there's no MITM vector, which is why sbeattie is subbed to the removal bug (I poked -hardened first for general ida of whether there's a security risk with the older versions)
[21:37] <teward> I'd be OK with a sync into Wily, and the removal/blacklist bug going away
[21:38] <teward> on the provision that the older versions go away for being insecure and not reverse compatible or even usable with the latest forks in the chains
[21:38] <infinity> teward: Didn't close which "ticket"?
[21:38] <infinity> teward: The Debian bug was closed with the upload.
[21:38] <teward> infinity: i meant the Debian bug
[21:38] <teward> infinity: it didn't ping me and i'm subbed
[21:38] <teward> oversight by email probably
[21:38] <teward> infinity: as i said, i'm OK with it getting into something after Wily, because FF is currently in place, unless someone wants to override that.
[21:39] <teward> but the current version in Wily, and older, are all 'bad' and actually will cause problems when being used
[21:39] <teward> (because of the issues detailed in the bug on LP)
[21:39] <infinity> teward: Anyhow, if this is going to be the same as other bitcoin stuff, where distributing it in stable releases is just wrong because people *need* the latest version, we should remove.
[21:39] <infinity> teward: But I don't know enough about this software to say that.
[21:39] <teward> infinity: ack, i'll have to poke at it more, but I *do* know that  items detailed in point 3 on LP are still valid:
[21:40] <infinity> teward: IOW, don't evaluate wily on "is 2.4 good enough today" but "will 2.4 be good enough in 9 months (or 5 years, if that's what's in 16.04).
[21:40] <teward> 1: bitcoin soft-fork on July 4, 2015, and only newer software versions don't know about it.
[21:40] <teward> 2: wallet recovery seeds don't work on the older versions (when existing on newer)
[21:40] <teward> infinity: i think it's a similar concern
[21:40] <teward> infinity: because those 'forks' in the blockchain affect everything
[21:41] <teward> but we may want to sit on it a week while i poke the electrum people
[21:42] <infinity> teward: If this sort of thing is standard operating procedure and breaking old versions of clients is expected, I think remove/blacklist is probably still the right answer.
[21:42] <infinity> teward: "The bitcoin network keeps becoming incompatible with old clients" is effectively why we removed all the rest.
[21:42] <infinity> teward: So, yeah, if this is tightly coupled to the same issues, let's remove.
[21:43] <teward> infinity: i'm double checking that understanding, but if it's a similar issue, then i still say removal is better.  I know in #electrum i've seen reqs for help with old versions and people say "Use the newer one because it works" (but with more technical reasons)
[21:46] <teward> infinity: at best guess, that applies, the "Bitcoin network and Electrum software continue to change, making older versions incompatible." argument is still valid especially with points already in place on the bug I filed
[21:46] <teward> I still support removal...
[21:47] <infinity> teward: Yeah.  Let's just remove, then.  I was just curious, since your bug specifically mentioned "updating to 2.x is hard", but 2.x is indeed in Debian.
[21:47] <teward> infinity: their argument at the time was 'hard'
[21:48] <infinity> teward: If the real answer is "updating doesn't help because the updated one will also break in a year", then what version we carry today doesn't matter, and we should just remove.
[21:48] <teward> I agree with the second one there more
[21:48] <teward> brb power needed
[21:55] <teward> and back
[22:01] <teward> infinity: saw the remove, thanks.  I'll poke the debdiff up shortly for review, no rush.  Wily+ was my current concern