[00:00] <cjwatson> One bot with queue admin on Ubuntu is quite enough thankyouverymuch and I'd like to try to keep privileges at least slightly least
[00:00] <wgrant> It all becomes a lot simpler when scalingstack is everywhere, but we still need to make private archives non-hideous.
[00:00] <wgrant> If possible.
[00:00] <cjwatson> So maybe we can still have somewhat different rules for public and private archives
[00:02] <cjwatson> But trying to have a common class of solution for private and devirt means either something that looks for any devirt archive, or adding an ArchivePermission to Ubuntu primary, or a special-case hack of some kind
[00:03] <cjwatson> I'd managed to avoid adding a celebrity so far ...
[00:08] <wgrant> https://docs.google.com/a/canonical.com/document/d/1F1wh8MxaxC-pSx5yMsFNpKFm5Mytsvn0Ugw2AIgQXzU/edit#
[00:10] <cjwatson> I was just writing up something too, only in vim :P
[00:10] <wgrant> vim sadly isn't easily multi-user.
[00:10] <wgrant> As much as I'd prefer it :)
[00:11] <cjwatson> Right, just lots more pleasant to use when my mirror sync and backups are both running
[00:11] <cjwatson> But let's see
[00:11] <wgrant> Heh
[00:11] <cjwatson> Any LiveFS can be built against a public archive.
[00:11] <cjwatson> To build a LiveFS against a private archive, the owners must match exactly.
[00:11] <cjwatson>  => registrant is in common owner => registrant can see archive
[00:11] <cjwatson> was what I had so far
[00:11] <cjwatson> LiveFS gains a require_virtualized column, set by admins as for PPAs.  (This is a bit more cumbersome, but lets us vet owners, and LiveFSBuild : LiveFS :: PPA builds : Archive, after a fashion.)
[00:12] <wgrant> Right, the require_virtualized thing is hideous, but hopefully ~temporary.
[00:12] <wgrant> The private archive restriction is hopefully not terribly onerous.
[00:12] <wgrant> And can always be relaxed later, I suppose, if we run into real problems with PES.
[00:13] <cjwatson> Even though that means the answers to the two problems are quite different rather than paralleling each other, I think that's actually sufficient given the existing LiveFS.requestBuild security
[00:14] <wgrant> Having such a security-sensitive flag duplicated on another table is awful, but hopefully of limited life due to scalingstack taking over the world.
[00:14] <wgrant> So I'm not as far against it as I was late last year, when everyone was "omg we can't do scalingstack for Ubuntu the world will be on fire"
[00:14] <cjwatson> It's sort of duplication but not entirely
[00:15] <wgrant> It's another class of objects that we have to check for terrible security holes.
[00:15] <wgrant> In terms of nagios checks for owners and such.
[00:15] <cjwatson> Yes, that's true, I should dig those up for comparison.  Are they in puppet?
[00:15] <wgrant> But I think those two solutions are workable for now.
[00:16] <wgrant> I'm not sure if they actually exist in any particularly current fashion. There are RTs which suggest they might not actually work.
[00:16] <cjwatson> Yay.
[00:19] <wgrant> Anyway, sounds like this should be relatively easy to implement for you?
[00:19] <wgrant> Just need to ensure that the permission checks occur at dispatch time (as well?)
[00:20] <cjwatson> Trying to rationalise this: a write permission check on the archive helps for privacy (buildd secret), but is wrong for virtness because really we're only reading from the archive and might well need to do a livefs build on devirt hardware for make-it-work reasons but with a virt PPA as a dependency.
[00:20] <wgrant> Though I guess the lack of retries means that's not such a huge issue, still.
[00:21] <wgrant> Right, that sounds reasonable.
[00:21] <cjwatson> Yes, I can do this tomorrow.  I have indeed got the message that it needs to be done at dispatch time. :-)  Worth doing at least lightweight checks (and probably all of this is sufficiently lightweight) in the model on requests as well.
[00:22] <wgrant> Definitely.
[00:22] <wgrant> It's all pretty lightweight now you're not doing a hideous query over every ArchivePermission evar.
[00:22] <cjwatson> SSD DBs baby
[00:22] <cjwatson> or maybe not
[00:22] <wgrant> Maybe before the heat death of the universe.
[00:24] <cjwatson> I've done the rest of your review, so will just need to go round again and make sure I haven't broken the browser code, and make sure it still works end-to-end
[00:25] <wgrant> cjwatson: I'm just wondering how likely it is that people will shoot themselves in the foot by building some random PPA on a non-virt LiveFS.
[00:26] <wgrant> s/themselvesk in the foot/us in the face/
[00:26] <cjwatson> Well, the most important use case for building a LiveFS against a PPA is the CI engine stuff
[00:27] <cjwatson> Secondarily, letting flavours run short-term experiments
[00:27] <cjwatson> The first is already all devirt, and perhaps we can just say that for the second you get to copy the LiveFS to a require_virtualized=True flavour
[00:27] <wgrant> Yeah, exactly.
[00:28] <wgrant> The only cases in which it really makes sense to do a nonvirt livefs on a virt PPA are narrow
[00:28] <cjwatson> And then say that if LiveFS.require_virtualised is False then so must Archive.require_virtualised be.
[00:28] <wgrant> Arch-indep only changes, and old Xen kernels
[00:28] <wgrant> And the latter is going to go away in a couple of weeks i hope.
[00:28] <wgrant> So I think that restriction would be sensible.
[00:28] <cjwatson> Certainly don't think it makes sense to design this around the Xen constraints
[00:29] <cjwatson> Kubuntu want to do PPA-based livefs experiments in the not too distant future
[00:29] <wgrant> Yes, mostly documenting that so I can review IRC logs when in 18 months I wonder why we made stupid decisions.
[00:29] <cjwatson> But I think we can hold that off for a while
[00:29] <cjwatson> The CI engine stuff can't really wait
[00:30] <wgrant> CI is all non-virt
[00:30] <wgrant> Presumably Kubuntu would have to be too.
[00:30] <cjwatson> Exactly
[00:30] <cjwatson> Well
[00:30] <wgrant> Or they'll be missing powerpc packages
[00:30] <wgrant> In which case they wouldn't want powerpc ISOs anyway
[00:30] <cjwatson> I'm not sure they care about powerpc for the experiments in question
[00:30] <cjwatson> I haven't really analysed it but I suspect they could go all virt
[00:31] <cjwatson> Which would save us from having to deal with the devirt => Canonical restriction
[00:31] <wgrant> Right, but the only interesting case is a mixed one.
[00:31] <wgrant> And Kubuntu doesn't seem to require that.
[00:31] <wgrant> Nor does CI
[00:31] <wgrant> And I can't think of any that do.
[00:32] <cjwatson> The ones I can think of are quick experiments - "what happens if I build an image based on this change", outside the CI system
[00:32] <cjwatson> But we could have people copy the livefs for that
[00:33] <wgrant> Right, and they already have to copy if they don't participate in the livefs owner.
[00:33] <wgrant> So copies have to work well anyway.
[00:33] <cjwatson> Or even just say that if you try to build a LiveFS against a virt archive then the build ends up virtualised too.
[00:33] <wgrant> Ah, that would work, indeed.
[00:33] <wgrant> A LiveFS build is non-virt iff its LiveFS and Archive both are.
[00:34] <cjwatson> It's require_virtualized not require_devirtualized, so it can be implicit in that direction.
[00:38] <wgrant> I think those were the only thorny issues in the review, weren't they?
[00:40] <cjwatson> There were a few things I had to slightly guess at how to implement correctly, but nothing else was fundamentally hard, no.
[00:42]  * cjwatson sleeps, thanks for the help
[00:43] <wgrant> Night, thanks for working this out
[00:43] <wgrant> I'll hopefully approve your UI branch today, now that we know model changes aren't required.
[13:13] <wgrant> stub: https://code.launchpad.net/~wgrant/launchpad/ppa-reset-2.0-db/+merge/223395 could use a review some time tomorrow, if you've time.
[13:14] <stub> wgrant: k
[13:14] <wgrant> Oh, you're still alive.
[13:18] <stub> wgrant: what does a null vm_reset_protocol mean?
[13:19] <wgrant> stub: Same as null vm_host -- incomplete setup if the virtualized flag is set
[13:19] <wgrant> We'll refuse to dispatch in that case, as we do with vm_host
[13:19] <wgrant> I could add a CHECK constraint to that effect, but then I'd have to fix all the tests that violate that constraint with vm_host already.
[13:19] <wgrant> And given this will hopefully all go away within 12 months...
[13:20] <stub> Yup. and unlikely worth adding the constraints for that, if we can.
[13:21] <wgrant> stub: Thanks.
[16:38] <cjwatson> wgrant: I believe I've implemented all the livefs stuff from last night (including a db-livefs change) and fixed up livefs-browser to match.  Just running an end-to-end build now.
[16:38] <cjwatson> wgrant: But should be ready for re-review of the changes.
[17:33] <cjwatson> wgrant: End-to-end build test still works.
[23:46] <wgrant> cjwatson: Lovely, let me see.