[00:00] k [00:31] abentley: so the final gpg one: staging is oopsing too much to test it [00:32] i don't seem to have broken anything at least :/ [00:35] poolie: did you create that merge proposal, or did it already exist? [00:36] poolie: I think you're hitting the bug where the staging librarian oopses instead of falling back to the production one. [00:36] poolie: I bet you won't have that issue if you create a fresh merge proposal. [00:46] ah, it already existed [00:46] ok [01:06] abentley: everything is qa-ok now, i believe [01:07] poolie: great. [01:08] can you tell me the url where i can check the unreleased changes? [01:23] what is the name of the single signon project? [01:23] canonical-identity-provider [01:25] ta [03:03] poolie: you might like the buffer bloat stuff thats getting airtime right now [03:03] poolie: vis-a-vis bzr network perf issues [03:03] where? [03:04] http://gettys.wordpress.com/category/bufferbloat/ [03:15] interesting [03:18] === Top 10 Time Out Counts by Page ID === [03:18] Hard / Soft Page ID [03:18] 286 / 6529 Archive:+index [03:18] 110 / 328 BugTask:+index [03:18] 36 / 268 Distribution:+bugs [03:18] 21 / 143 ProjectGroupSet:CollectionResource:#project_groups [03:18] 15 / 32 DistroSeries:+queue [03:18] 14 / 0 BinaryPackageBuild:+retry [03:18] 13 / 18 Archive:+copy-packages [03:18] 10 / 264 Distribution:+bugtarget-portlet-bugfilters-stats [03:18] 10 / 39 MailingListApplication:MailingListAPIView [03:18] 9 / 7 Person:+bugs [03:25] mwhudson: hey, how is vostok [03:26] lifeless: resting [03:26] ah [03:28] there are so many intranet hosts called portal.*.com, and so few of them actually let you play the game :-( [03:49] poolie: haha [03:50] :) [03:50] how are you? [03:50] i'm doing one of max's reviews [03:51] i was looking into splunk earlier and it looks very good [03:56] Good, about to flood the review queue slightly :) [03:57] :) [03:57] * spiv shifts to #bzr [07:14] thumper: don't suppose you're still here? [08:22] poolie: not really [08:36] good morning === almaisan-away is now known as al-maisan [09:10] Morning [09:13] morning people [09:18] Morning. [09:31] hi mrevell, wgrant [09:31] Morning jelmer. === _mup__ is now known as _mup_ [11:30] wgrant: yo [11:31] bigjools: Hi. [11:32] wgrant: so the zombie sources are all these I think: http://pastebin.ubuntu.com/540590/ [11:32] does that query look sane to you? [11:32] It does. [11:32] it probably needs to ignore some more [11:32] stuff that went superseded in the minutes before the query runs will get picked up [11:33] maybe I can ignore stuff newer than an hour old [11:33] Yeah. Filter for older datesuperseded. [11:33] Or datecreated. [11:33] I'm going to convert that to an UPDATE to set spph.datepublished [11:33] Er. [11:33] Why? [11:33] fail [11:33] spph.scheduleddeletiondate [11:33] paste pebkac [11:34] The "Why?" still stands. [11:34] Mark the binaries Deleted, and the sources will be condemned automatically. [11:34] yes [11:35] I guess so, was gonna do binaries separately [11:35] The minimal hacker is to set the status and datesuperseded of the binaries. [11:35] Everything else will follow. [11:35] And I like minimal production DB hackery :) [11:36] yes ;) [11:37] * bigjools cooks query to find associated binaries [11:58] wgrant: sane? http://pastebin.ubuntu.com/540597/ [11:58] 5744 binaries! (I forgot the time constraint) [12:00] http://pastebin.ubuntu.com/540605/ is better [12:07] bigjools: You need to constrain bpph.archive, but otherwise it looks OK. [12:07] yeah was just making that change [12:07] trying to think whether I need to constrain on distroseries too [12:08] Also constrain the status. [12:08] We only care about published ones. [12:08] yes [12:08] Thinking about the distroseries thing... it probably won't make a difference, but try it both with and without. [12:09] hmmm, zero rows [12:10] removing the archive constraint makes it 1684 [12:10] 1584 [12:11] Which archive constraint? bpph.archive = spph.archive? [12:13] yeah [12:13] That has to be there. [12:13] So sounds like we've got something wrong. [12:27] wgrant: I think I know [12:27] Oh? [12:27] they are probably all copied [12:27] but that doesn't explain the lack of binaries causing domination to not set scheduleddeletiondate [12:28] * bigjools looks at the code [12:28] Right, they have to be in the same archive. [12:29] so they should get comdemned :/ [12:29] Well, no. Our query is wrong. [12:29] that is the other obvious problem :) [12:30] http://pastebin.ubuntu.com/540608/ [12:30] that's the current one === Ursinha-afk is now known as Ursinha [12:30] bpph.id = binarypackagerelease.id and [12:31] Wut? [12:31] sigh [12:32] 2237 rows. hurray [12:32] I love typesafety :) [12:32] now to restrict on series [12:32] And status. [12:32] Status is far more pressing. [12:32] heh [12:32] down to 141 [12:33] which is odd, that's less than the number of source rows [12:33] Not terribly odd. [12:33] More relevant is how the number of archives in those rows compares to the number processed each time. [12:35] 141 again restricting the series [12:35] so that's pleasing [12:35] Excellent. [12:35] time to sweep up some cruft then [12:35] I'd compare the list of archives first... [12:36] mmm [12:37] 14 of them [12:56] wgrant: so there's actually 23 PPAs that come up on the query. However 4 of them don't appear in the publisher log. [12:57] bigjools: Are those archives disabled or unpublishable? [12:57] ah let me check [12:59] yup that's it [12:59] all deleted [13:00] no harm in fixing those publications anyway [13:00] How many of the archives in the log does that leave unfixed? [13:00] We need to do a mass-fixup of deleted archives' publications anyway. [13:00] I don't know, it's very hard to see what's genuine and what's not [13:00] the only way to tell is to see it repeated in the log each run [13:01] Right. [13:02] wgrant: http://pastebin.ubuntu.com/540616/ [13:03] bigjools: Do not touch SPPH. Set BPPH.status = Deleted and BPPH.datesuperseded = NOW() [13:03] wgrant: gah [13:04] right [13:04] I clearly need more coffee [13:05] if we set status=Deleted, I need to set the person who deleted it [13:05] or the UI will go bong [13:06] wgrant: http://pastebin.ubuntu.com/540622/ [13:08] bigjools: I don't think the UI will go bang. But I guess it might. [13:08] it's bad data either way [13:10] spph.datepublished is null and [13:10] I'm pretty sure that is crack, but it will only restrict the set to something less than it should be. [13:10] Otherwise it looks OK. === Ursinha is now known as Ursinha-lunch [14:25] gmb, I meant to ask you to look at bug 380362 if you could. [14:25] <_mup_> Bug #380362: Launchpad couldn't import several bugs from Debian Bug tracker. [14:25] may be an old one that is fixed, but the linked bug watches still have "can't find bug" or some such error. [14:26] Ah, the neverending saga of debbugs watches. [14:26] deryck: It's on my list for today. I'll try to poke at it when the LOSAs are available (they're currently tied up with another issue). [14:27] gmb, ok, thanks. === matsubara is now known as matsubara-lunch [14:49] deryck, abentley: the revert is missing qa tags and isn't recognised by qa-tagger, you should probably manually tag the bug [14:50] flacoste, abentley did. That was my fault sorry. [14:50] * deryck should never take theraflu during work days [14:51] flacoste: Because deryck did not use --rollback, I have tried to emulate it from the documentation, which says "The bug will be tagged as bad-commit-*, all the qa-* tags will be remove..." [14:52] flacoste: I think there's some difference between documentation and implementation, but I'm not sure what's supposed to be done in this situation. Maybe roll it back to "in progress"? === Ursinha-lunch is now known as Ursinha === matsubara-lunch is now known as matsubara === flacoste changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 4 of 10.12 | Performance Tuesday | PQM in RC mode | firefighting: - | Get the code: https://dev.launchpad.net/Getting [16:10] sinzui: A few questions about https://bugs.launchpad.net/launchpad-foundations/+bug/684668 : May I toss it to registry, since we have no knowledge of these bits? If not, my impression is that this is not of high importantance: am I wrong? [16:10] <_mup_> Bug #684668: Unable to delete messages from list archive [16:11] gary_poster, yes [16:11] thank you [16:11] gary_poster, I am sure it is an ml-archive-sucks bug. We know the message was deleted, but the page generator/squid cache wont show it gone [16:12] ah, gotcha === beuno is now known as beuno-lunch === benji is now known as benji-lunch === deryck is now known as deryck[lunch] === beuno-lunch is now known as beuno === benji-lunch is now known as benji === al-maisan is now known as almaisan-away [18:12] hello [18:12] hello [18:13] benji, gary_poster: how are you? jml said you are the right persons to talk about this new keyring thing? [18:13] pitti: hi, yes. did you see benji's reply on the pertinent bug [18:13] yes, I just replied [18:13] oh, lemme go look [18:13] but I wondered why we deliberately broke existing API, instead of creating a new login_keyring() thing for keyrings [18:14] that's not .. nice [18:14] it suddenly breaks each and every script and cron job we've written [18:14] and isn't noninteractive [18:15] your cron jobs use login_with? [18:15] (and using a keyring by default is total overkill -- after all, you can just snatch the cookie from firefox) [18:15] benji: sure [18:15] that's unfortunate [18:15] that was _the_ primary function to get a Launchpad lib so far [18:16] and it's also what the documentation said [18:16] (e. g. https://help.launchpad.net/API/Examples) [18:19] even without the changes in the latest version, cron scripts using login_with are in a precarious sitution; if the credentials cache is ever removed or if the token is revoked on the server-side they will attempt to re-authenticate with a web browser [18:21] not to say that the situation isn't worth discussing, but one of the outcomes apparently needs to be some form of better advertising when the different APIs should be used [18:22] benji: not really [18:22] benji: first, it had a credentials_file argument, so that you can store them in a well-known placed [18:22] and second, if you run the cron job manually the first time, it showed the URL to you, you could open it locally, and then you were fine [18:23] benji, pitti, I'm a big +1 on getting pitti's scripts happy, both as an important user and as a representative. I'm happy to weigh in if needed, but mostly intend to get out of the way. A few notes: [18:23] - the security implications of this have been looked at by many (including Kees, importantly), and I'd really prefer not to reopen them. [18:23] - as you might expect, we believed that we were doing the right thing; what benji described was our understanding of the usage, and our expected usage. The examples you cite were part of the general difficulty we find ourselves in: we want to make the right thing happen by developers who don't want to worry about it, and who have not worried about it in the past. [18:23] - we changed the behavior of login_with because it meant users that we were interacting with would not have to make a change to get the uniform behavior. uniformity on the desktop is very important for this effort. [18:23] - our docs obviously needed and need improvement in this regard in particular. [18:23] - leonardr, the person most connected with these changes, is not around this week, though benji and I should be able do a reasonable job. [18:23] benji: I opened https://bugs.launchpad.net/launchpadlib/+bug/686690 about this [18:23] <_mup_> Bug #686690: 1.8.0 completely breaks login_with() API (and forces keyrings) [18:23] I think we should probably discuss it there, to have a better paper trail [18:23] pitti, your "and second" is exactly the kind of thing benji was describing [18:24] pitti, the paper trail is the upside. The decreased speed is the downside. [18:25] in this particular case, with leonardr not around, maybe the paper trail wins anyway [18:25] I understand that you might want a different API on "desktopish" programs [18:26] and we are dealing with an installed base [18:26] that needs to be uniform [18:26] but as it happens, most of our use cases (QA scripts, apport, ubuntu-archive-scripts, sru report, and what not) are not desktop apps [18:26] the only non-"server"ish use case I know of is Ubuntu One [18:27] nit: non-desktop apps are not a particular problem for this; cronscrpts are. [18:27] ...a subclass... [18:27] Ground Control [18:27] others, I believe [18:27] or in general, taking a well-established existing API and turning it into something entirely different [18:28] even for desktop apps [18:28] it now means that suddenly every desktop user needs to reauthenticate [18:28] as their existing credentials files are ignored [18:31] pitti, I think you are stretching your description ("entirely different"). from our perspective, the API presented a black box. the credentials files are generally not intended to be the domain of a launchpadlib-using script. But I also think that this part of the conversation is counter-productive. [18:31] We have a use case and approach that has been vetted by more people than us. There is a reason that it is the way it is, and it has been discussed. I believe you have a valid, concrete concern for cronscripts that we did not foresee. We need to determine a way around this. [18:32] One obvious approach is to change the scripts to use our expected approach. That is admittedly unfriendly, so even though I think that's the longer-term desire, we should do better now. [18:32] couldn't we get a new API for using keyrings, and make keyrings optional? [18:32] at least for a transition period of a year or so [18:32] so that we have the new keyring API available for some time [18:33] and everyone has time to port their stuff [18:36] we could; part of the motivation for the keyring integration is to support better desktop integration without the need for clients to change their API use. It would be great if we can fulfill that goal while also addressing the cron script problem. [18:36] pitti, keyrings are "optional" right now in that, if there is not one available, the keyring library will do something else. You know that already. I *think* what is more important to you is that, if legacy credentials files exist, they are used. Is that in fact sufficient, or am I missing a detail? [18:36] gary_poster: how can I make them optional? [18:36] (and no, I don't know that already) [18:36] oh [18:36] if I run a script now, it creates a new keyring [18:36] asks me for a passworrd [18:36] (which can't be empty) [18:36] and then subsequently asks me for that password again [18:37] Waiting to hear from Launchpad about your decision... [18:37] Please input your password for the keyring [18:37] they are optional in the sense that, if there is not one around (e.g., not Gnome or KDE), it is not used. [18:37] Password: [18:37] that's what I get [18:38] I'm fairly certain that's the prompt for the encrypted file storage used when no keyring is available. [18:38] well, I have GNOME running [18:39] I didn't test it on a server with ssh yet [18:39] right, the keyring library does not require a keyring implementation. It provides a backup. [18:39] apparently python-keyring can't communicate with the keyring for some reason [18:40] (my comment is pertinent to "I didn't test it on a server with ssh yet") [18:41] * gary_poster is going to be quiet about details because benji knows more than he about them [18:43] benji: note that we currently have a very old version of p-keyring in the archive; did you test this with 0.5? it might work better [18:43] so far it sounds to me like (1) python-keyring is not happy on pitti's machine, which is worrisome; and (2) I'm not sure yet whether, if the first problem didn't exist, my proposal ("if legacy credentials files exist, they are used") is a sufficient compromise for this backwards compatibility problem. [18:43] gary_poster: if the credentials_file parameter and existing cred files could continue to work, that'd pretty much solve the API break, I think [18:44] as for (1), I have no idea, I'm afraid [18:44] as I pointed out in the MIR, we should update to the current 0.5 upstream version first [18:44] pitti, do you always use the credentials_file parameter in your scripts? [18:45] it drops its own gnome-keyring module and uses libgnome-keyring, which is the right thing [18:45] pitti: yep, the latest launchpadlib is meant to be used with python-keyring 0.5, but I'm not sure if using an older version would work more pooly [18:45] gary_poster: only for apport; for most scripts we just use the default in ~/.launchpadlib/ [18:45] ah [18:45] benji: ah, that could be it then [18:45] (see above) [18:46] sorry, need to leave for today [18:47] thanks for the discussion so far! [18:47] pitti, ack. I was hoping to only accept existing cred files if credentials_file were passed, and then issue a deprecation warning [18:47] but that still sounds not good enough [18:47] ok thanks [18:47] benji or I will summarize in the bug [18:47] ah, thanks [18:49] I also sent a summary [18:49] thanks, and good night! [18:49] excellent, thanks [18:49] night === matsubara is now known as matsubara-afk === abentley changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 4 of 10.12 | Performance Tuesday | PQM open for business | firefighting: - | Get the code: https://dev.launchpad.net/Getting === salgado is now known as salgado-afk [21:57] How come the apidoc lists getSpecification but it doesn't actually appear with a launchpadlib client? [22:00] bdmurray: does the version of the docs you're looking at and verion of the API match? [22:00] benji: yes I'm using devel for both [22:03] hrm, might be a bug [22:06] benji, ping [22:06] sinzui: hi [22:07] benji, I want to subclass a Choice field to guard conflicting transitions between values. I am not sure if I should redefine _constraint() or _validate() [22:12] benji, The crux of the issue is that while the value is valid, it cannot be accepted because it violates another relationship. (something like a invariant, but between a hierarchy of objects) [22:14] salgado-afk: I'm having some troubles using ubuntu.getSpecifications() that should work on devel right? [22:15] sinzui: I suspect using constraint() (no leading underscore) will be slightly easier. [22:15] benji, yes, sorry for the underscores [22:16] sinzui: note the interface difference between the two: constraint() should return a boolean, validate() should raise an exception (or not) [22:17] ah [22:17] bdmurray, it should yes [22:18] benji, so I think I may want to define both for their respecting uses. [22:18] makes sense [22:18] bdmurray, do other specifications things work? natty.valid_specifications worked for me on qastaging last week [22:18] validate() could call constraint() and convert False to TeamSubscriptionPolicyError [22:19] james_w: no valid_specifications doesn't show up either but I'm production which seems to have the right revision [22:20] bdmurray, you get "AttributeError"? [22:21] sinzui: the default implmentation (from Field) does something similar, validate() is responsible for calling constraint() and raising a ConstraintNotSatisfied error if a false value is returned [22:21] james_w: yeah [22:21] bdmurray, are you using it anonymously? [22:22] james_w: no, I've seen that bug report too [22:22] sinzui: (ConstraintNotSatisfied is a subclass of ValidationError) [22:22] bdmurray, it's working for me using "edge" [22:23] benji, thanks. I will follow that pattern. I was thinking or returning a very specific class of error with a meaningful reason. [22:23] yep; the default "Constraint not satisfied" isn't real helpful. [22:30] bdmurray, works for me against production too. My guess would be cached wadl or it not actually using devel for some reason. [22:31] james_w: okay, thanks I'll poke at it some more [22:31] np === matsubara-afk is now known as matsubara === matsubara is now known as matsubara-afk