[00:00] i guess i should file a bug [00:00] please [00:00] That test probably also deserves a bug. [00:00] Although it will be less important with the new zope.testing, I guess. [00:01] wgrant: yeah, i guess so [00:01] i'm surprised and a bit scared if "browser.open('http://launchpad/dev')" commits to the db though [00:02] mwhudson: bonus points for making a glob function that provides a launchpadlib object [00:02] mwhudson: and if there is no user passed in, make one === Ursinha-busy is now known as Ursinha [00:03] thumper: that sounds a little bit difficult because you have to create an oauth token [00:03] mwhudson: there is a function for that [00:03] * thumper looks for it [00:04] mwhudson: It will at least commit to update the session time. [00:04] Although I guess that's on a different DB. [00:04] And maybe it doesn't go through the real session machinery anyway. [00:04] wgrant: if it's that sort of thing then i'm going to be even more angry i guess [00:06] mwhudson: webservice_for_person has most of what you need [00:07] * thumper remembers writing that [00:07] ah [00:07] a not-surprising coincidence [00:07] mwhudson: What was the diff you used to get the pagetests to fail? [00:09] wgrant: i think i pasted the diff didn't it? [00:09] i'd have to read my logs to find it now... [00:09] Oh, just a browser.open? [00:10] yep [00:11] oh FFS [00:11] make fails [00:11] ImportError: No module named collection [00:11] I have updated everything and did a make clean [00:11] any ideas? [00:11] collection isn't in 2.5! [00:12] wgrant: yes it is? [00:12] Who's trying to import it? [00:12] /lib/lp/bugs/configure.zcml", line 935.4 [00:12] Oh, right, collections. [00:13] oh [00:13] "collection" sounds wrong [00:13] mwhudson: So, yeah, that browser.open() commits a couple of times across each database. [00:13] wgrant: AWESOME [00:13] So that is indeed the problem. [00:13] If that test commits at all, the second testTearDown attempts to destroy the non-existent DB. [00:14] thumper: ? [00:14] i'm looking [00:14] wgrant: do you want to follow up to my mail saying as much [00:14] can't find the import yet [00:15] mwhudson: Will do. [00:15] hahah [00:15] pebkac [00:18] 'twas me [00:18] from collection import defaultdict [00:18] ah good [00:22] (I think I was probably thinking of collections.defaultdict, which wasn't in *2.4*.) [00:26] wgrant: correct [00:45] mwhudson: want to review a branch that renames thumper -> dumper? [00:46] mwhudson: I'm sick of not having make-lp-user work for me [00:46] thumper: :-) [00:46] thumper: ok [00:48] * thumper waits for the scanner === matsubara is now known as matsubara-afk [00:58] * mwhudson lunches [01:19] ok, there is now https://code.edge.launchpad.net/~james-w/launchpad/new-code-import-method-on-branchtarget/+merge/22190 [01:19] and https://code.edge.launchpad.net/~james-w/launchpad/code-imports-for-source-packages/+merge/22191 [02:14] mwhudson: can you go 'make run_codehosting' and push a branch to lp://dev/ using devel? [02:14] mwhudson: I can't [02:14] bzr: ERROR: Connection error: while sending POST /bazaar/: [Errno 111] Connection refused [02:15] thumper: err, usually yes [02:15] i usually use the key in the sample data, so i may not have done it in a while [02:15] thumper: is the app server actually running? [02:18] mwhudson: as far as I can tell [02:18] ah, I bet I need to make install and apache isn't working [02:18] * thumper looks a bit [02:19] I've not tried it in lucid yet [02:20] that might do it, i guess [02:20] How's the EC2 run looking? [02:21] yep, sudo make install fixed it [02:25] wgrant: lib/lp/registry/tests/../stories/product/xx-product-add.txt failed [02:25] that's it so far though [02:25] OK. [02:31] ah fuck [02:31] I thought I had broken something [02:32] but it is just the make-dummy-branches [02:32] if you have pushed code, and restart codehosting, it blows away your pushed branches [02:32] but of a fubar there [02:38] thumper: Oh, I thought that was a feature. [02:38] wgrant: nope [02:38] It pisses me off immensely when testing recipe builds. [02:38] argh. ok. who broke pygpgme. <== grumpy losa venting and hence ignorable [02:39] spm: Lucid? [02:39] looks like a branch upgrade actually [02:39] spm: or jelmer [02:39] * spm hunts for the bug we raised on jelmer hisself ;-) [02:42] mwhudson: do you remember what to import in scripts to get around the circurlar import madness? [02:43] thumper: uh, i think there's more than once source of the madness [02:43] it is mildly insane, but import canonical.launchpad.interfaces before anything else makes it run :( [02:45] is there a way I can just reapply database/schema/security.cfg without make schema? [02:46] database/schema/security.py [02:46] Will apply to launchpad_dev by default. [02:48] ta [02:52] * mwhudson gives some consideration to warsaw's law [02:52] :-) [02:54] The second law? [02:55] the one about fridays [02:55] Yeah, the second. [02:56] yes, then [02:58] * thumper is having a WTF moment [03:06] * thumper is confused [03:06] * thumper uses pdb [03:07] huh?!?!? [03:07] * thumper can only guess at stale pyc files [03:07] doesn't make clean clear pyc files? [03:09] ARGH!!!! [03:09] PEBKAC again! [03:09] it must be friday [03:09] * sidnei hands some band-aid over for thumper's forehead [03:10] must be time for a cocktail [03:10] why isn't the email being delivered I wonder... [03:15] ah crap [03:15] thumper: time for a cocktail? it's after 9am over there eh? [03:15] somewhere there is a setting saying that scripts don't send email [03:15] spm: it has been one of those days (or weeks) where things take longer than expected, and you don't get as much done as you expect [03:16] thumper: ahhh! so you've become a losa eh? sweet! [03:17] thumper: Zopeless emails do not go anywhere in devmode. config.zopeless.send_email [03:17] wgrant: so how am I supposed to test this damn script eh? [03:17] arse [03:17] also, why don't they go anywhere? [03:18] thumper: Because there's no mail utility, so it uses raw SMTP. [03:18] they should go the same place that the app server emails go [03:18] Which is very dangerous, since it can send email to ANYWHERE IN THE WORLD. [03:18] wgrant: there is a mail utility in the scripts [03:18] has been for ages [03:18] Although it could just override the destination address to root@localhost anyway, i guess. [03:18] Not the full Zope queued delivery mail utility. [03:18] wgrant: yes, the same as the app server [03:19] well, that is a problem for another day [03:22] * thumper tries something [03:27] wgrant: http://pastebin.ubuntu.com/401551/ works [03:27] wgrant: sends email to root@localhost [03:27] wgrant: please excuse the commenting out :) [03:28] wgrant: actually this is easier to read http://pastebin.ubuntu.com/401552/ [03:29] thumper: So raw_sendmail obeys the destination override? [03:30] yes [03:30] * thumper commits that into his branch [03:30] Convenient. [03:30] bzr commit -m "Zopeless is no longer really zopeless, and to test the local email sending, we want this. Scripts now send email in the dev environment to root@localhost just like the app server" lib/lp/services/mail/sendmail.py [03:30] that's what it says [03:46] wgrant: yay that was the only failure [03:47] mwhudson: Excellent! [03:49] mwhudson, wgrant: how long does canonical.testing.layers.ZopelessAppServerLayer take to start up for you? [03:49] it seems to be taking ages for me [03:49] perhaps blocking completely [03:49] bin/test.py -vvt TestCodeHandler [03:50] Took 12 seconds here. [03:51] hmm. not starting here [03:51] ~same here [03:51] I noticed that my make run didn't kie [03:51] die [03:51] killed it [03:51] trying again [03:52] ha ha 11 seconds [03:52] going now [03:52] thanks anyway [04:06] thumper: still here? [04:06] mwhudson: just [04:07] whazzup? [04:07] thumper: https://code.edge.launchpad.net/~mwhudson/launchpad/aiee-everything-is-broken/+merge/22185 will update in a couple of ticks, can you take a look? [04:07] yep [04:10] * thumper waits for the diff [04:10] grr still not updated [04:10] perhaps its broken [04:10] :P [04:10] thumper: http://pastebin.ubuntu.com/401568/ [04:11] or someone's prosed merge mysql 5 into mysql 6 or something [04:13] mwhudson: done [04:13] mwhudson: I'm heading out now [04:13] * thumper EODs [04:13] thumper: thanks [04:16] * mwhudson shoves the branch in the direction of ec2 [04:53] WHat's the correct way to get the db username and password when working with Storm to access LP's database? [04:53] * NCommander wants to start work on the converter [04:56] NCommander: You don't do that manually. You should probably use the normal script infrastructure which sets Zope and Storm up for you. [04:56] wgrant: oh, which script is good for an example? [04:57] NCommander: utilities/soyuz-sampledata-setup.py is one I know well. [04:57] execute_zcml_for_scripts() [04:57] txn = initZopeless(dbuser='launchpad') [04:57] Those are the critical lines. [04:58] You can then use the IStoreSelector utility to get your store. [04:58] wgrant: cool, thanks. I'm probably going to have a ton more questions in the future [04:58] s/questions/sleep/ [07:29] eww. A javascript alert box on ajax-changing ubuntu bug status?! [07:29] Yes :( [07:29] For anybody who is not a bug supervisor or owner. [07:29] It is crack. [07:29] Both because of the rules and because it's AN ALERT()! [07:30] eww [07:30] That needs an opt-out [07:31] Or I shall have to learn greasemonkey [07:31] It does. [07:31] * maxb wonders who signed off a UI review on an alert() :-) [07:32] The excuse was that there wasn't time. [07:32] The excuse for that was also that there wasn't enough time. [08:48] noodles775: Is there any timeline for the work to extract application information from PPA builds? [08:50] wgrant: I'm not sure what you mean by "extract application information"? [08:51] noodles775: soyuz is meant to eventually extract .desktop files and icons, and publish them for Software Centre to use. [08:51] It seems like it would make the PPA binary listing a lot less problematic. [08:52] wgrant: yeah, totally (long-term goal). No, I'm not certain what the timeline is for doing that, but I expect we'll find out at UDS. [09:04] gmb, allenap: if you have some time, i think it would be a good exercise rewriting TestBugWatchScheduler not do do any commits. also, why is LaunchpadFunctionalLayer used? [09:08] wgrant, NCommander: IStore(Account), IMasterStore(Account) or ISlaveStore(Account) is the preferred spelling now over using IStoreSelector directly. This will give you the store where the Account objects are stored, and will keep working if things get moved around (probably YAGNI, but the spelling is nicer anyway). [09:09] stub: I keep forgetting about that, since it's used in so few places. [09:09] But it is much nicer. [09:10] BjornT: Sure, we'll do that if we get time; LPFL is a copy-paste error, so we'll fix that too. [09:11] So launchpad-dependencies no longer installs for me. It seems to want python2.5-apt and I can't find that. Any clues? [09:11] gmb: cool. another question. in what way does this fit into the "Database garbage collector"? it doesn't seem to collect any garbage? [09:11] stub: the PPA needs updating. [09:12] stub: Force the installation of python-apt IFORGETTHISBITubuntu2launchpad1 [09:12] BjornT: It doesn't. However, basic advice in the past has been to include things like this (and BugHeatUpdater, which doesn't garbage collect either) as part of garbo because we get the benefits of some built in testing (with test_garbo.py). Also, managing DB permissions is slightly less troublesome. [09:14] gmb: it sounds a bit dangerous, unless we redefine what garbo is supposed to do. in the past garbo has been broken, and not been running for a while. this is acceptable for garbage collection, but is it acceptable for bug watch updating? [09:14] BjornT: Hmm. Good point. No, it isn't; if it breaks then bug watches stop getting updated. [09:14] BjornT: I'll prepare a branch to put the scheduler in a separate cronscript. [09:14] I thought I fixed garbo so if one task fails it keeps going? [09:15] Oh. [09:15] Even has timeouts so if one task takes too long it is aborted and the others allowed to continue. [09:16] Morning [09:16] stub, BjornT: Ah. So in that case I'd say it *is* an acceptible place to put it, though it doesn't fit under the garbage-collecting umbrella. [09:16] stub: could be, i don't know. it still seems a bit dangerous to do a bunch of important things in one script. especially from a scalability point of view. [09:17] Its more than garbage. [09:17] The goal with garbo was to stop the proliferation of all those little cronscripts. [09:20] I personally would rather see scalability type fixes made if/when they become a problem than worry about the deployment issues [09:20] stub: i wonder, could this somehow be combined with the job system? they seem quite similar. [09:21] (more cron emails to monitor, more crontab entries, more script monitoring, more database users + permissions) [09:21] BjornT: You could replace garbo-daily.py and garbo-hourly.py with garbo.py which runs regularly, pulling scheduled jobs. [09:22] stub, BjornT: But then we'd need a garbo job to put jobs into the job system to be run. [09:22] At the moment, that is probably overkill. We also don't have to deal with long running nightly jobs blocking hourly jobs. [09:23] allenap: And some turtles for it to stand on [09:23] :) [09:26] stub: ok. we'll see what we'll do. you've just volunteered to document what garbo does and is supposed to be used for, since you might be the only one that knows enough about it :) (more on that later) [09:26] The other gain we get with garbo is jobs are serialized. We end up with unnecessary load spikes whenever hourly or nightly jobs fire off at the same time - with -hourly and -daily we get a very primative scheduler, which so far has been 'good enough' [09:26] BjornT: Stick it on my kanban board if you want :) [09:51] bigjools: Did that firewall fix end up happening? [09:57] wgrant: yes, gonna test it RSN [09:57] bigjools: Excellent. [09:57] We might finally get it done... 2 months after the plan. [09:58] well the UI work is still in progress [09:58] that's the last piece of the puzzle [09:58] Does anyone know where mwh got to with his branch to combat the test-suite horrorshow? [09:58] gmb: The one where 7000 tests dropped off the face of the earth? [09:59] It was shoved into EC2 around 5 hours ago. [09:59] With the known failures fixed. [09:59] Er, nearly 6 hours ago, now. [09:59] wgrant: Ah. Just long enough for me to start to worry that it's failed... [09:59] Yes... [10:00] Although it had just one failure when it was run a couple of hours earlier, and that failure is fixed. [10:00] Hrm. [10:00] So launchpad-dependencies no longer installs for me. It seems to want python2.5-apt and I can't find that. Any clues? [10:01] I uploaded the new source 10 hours ago, the amd64 build is still pending, "Start in 8 hours" [10:01] A buildd admin could make it jump the queue O:-) [10:01] maxb: Better than the 30 hour queue yesterday.. [10:03] I wonder if it's worth someone liasing with the owners of the hugest daily build PPAs to get them to skip a few days when the buildfarm is borrowed [10:04] The problem should go away once builder pools appear. [10:04] Which is probably RSN. [10:10] how is that going to help the current problem, which is just a shortage of builders across all architectures? [10:10] maxb: Not architecture pooling Daily builds will be restricted to a set of dedicated builders. [10:11] ah [10:18] I'm working on designing builder pools right now :) [10:18] will put it out for comment in due course [10:35] wgrant: 2010-03-26 10:34:38 DEBUG hello: (source) NEW [10:35] \o/ [10:36] bigjools: Nice! [10:36] wgrant: one bug is that the builds were created with a score of 0 [10:36] bigjools: 'Builds' being the SPRBs? [10:37] buildqueue to be more specific [10:37] but yes [10:37] Right, scoring is not yet implemented for SPRBs [10:37] https://dogfood.launchpad.net/builders/ferraz [10:37] Also, the version on that recipe is screwed. I should really fix that. [10:37] it's even building the source now [10:37] Excellent. [10:38] So it does actually work. [10:38] * bigjools does a little dance [10:39] If you fix the recipe, the binary builds should work, too. [10:40] (I threw a $ in front of the variable in the version, where it didn't want one) [10:41] bigjools: Wouldn't it be nice if we had complete integration tests for all build types? [10:43] wgrant: there's a shedload of nice things that could be done :/ [10:43] Indeed, indeed. [11:55] How would I go about switching DB user when I'm in the DatabaseFunctionalLayer? switchDbUser is on LaunchpadZopelessLayer and refers speficially to Zopeless stuff. === matsubara-afk is now known as matsubara [12:11] Ignore previous question. I have since hit myself with the clue-bat. [12:26] bigjools: hey, are you still getting emails? [12:27] Ursinha: no [12:27] does that make it your fault? :) [12:31] oh ffs [12:32] bigjools: yeah, it seems :/ weird thing is that the script seems to be doing what it should, I'll have to dig deeper [12:35] Ursinha: also, how can we make that script not do anything to the bug if I so desire? [12:35] (as per my email to you earlier) [12:35] bigjools: I haven't seen it yet [12:35] a moment [12:37] bigjools: did you send it directly to me or through some list? [12:37] to you [12:37] Ursinha: subject "Fwd: [Bug 392887] Re: Cannot delete a PPA" [12:37] Bug #392887: Cannot delete a PPA

[12:37] sod off mup [12:37] hm [12:37] lol [12:38] bigjools: checking [12:39] slow... internet......zzzzzzzz...... [12:42] bigjools: what do you mean? what exactly you don't want to happen with the bug? to be targetted? assigned? marked fix commit? [12:43] Ursinha: no action at all [12:43] in this case it marked it fixed, but there's still loads of outstanding branches [12:43] bigjools: so the problem would be the mark as fix committed/tagging [12:43] ? [12:44] Ursinha: yes, but preferably I want it to do nothing at all [12:44] bigjools: we need to keep track of branches being landed to fix that [12:44] targeting and assigning are potentially wrong as well [12:44] so at least a comment pointing that would be nice to have [12:45] yeah you could do that [12:45] bigjools: problem is that fits most use cases [12:45] but the branch linked on the page shows that too, LP could do that itself [12:45] people forget do assign/target their bugs [12:45] and that ruins how we use the tagging schema [12:45] *forget to [12:46] those people should be slapped :) [12:49] bigjools: how hard is to you to unmark it as fix committed? :) [12:49] Ursinha: that's not the point - on a popular bug it generates a lot of email spam [12:50] bigjools: what do you suggest? [12:51] Ursinha: first thing I guess is to not do anything to the bug if it has multiple branches listed [12:51] anything I don't agree :) [12:51] at least a bug comment [12:51] Ursinha: that should not be done by your bot [12:52] it should be done by LP [12:52] bigjools: is there a bug for that? [12:52] Ursinha: since we're only just talking about it, I guess not :) [12:53] bigjools: I don't know if all projects would like to have comments added to their bugs, but that's not up to me to decide :) [12:53] bigjools: could you open a bug for that, please? [12:54] bigjools: script works now basically with how lp works today, if lp adds new features maybe we could add changes to the current qa tagging process [12:56] Ursinha: what triggers the script? [12:57] bigjools: cron [12:57] :P [12:57] bigjools: it checks all the new commits since last run, and if they're on edge or staging [12:57] Ursinha: what triggers the changes to bugs I mean [12:57] http://pastebin.ubuntu.com/401769/ - anyone know why that's happening? [12:57] Ursinha: brb [13:00] bigjools: if the commit is already on edge or staging, then the script does his dance with the bug [13:00] *its [13:04] Ursinha: how does it know which bug? does it use the bug= in the commit msg? [13:04] or does it inspect the --fixes bzr stuff? [13:07] bigjools: not just that, if the bug# is in the commit message [13:07] bigjools: again, not all people use to remember adding the bug= tag :) [13:08] Ursinha: ok [13:08] with ec2 that changed because ec2test/12 [13:08] argh [13:08] Ursinha: so I can avoid it by not mentioning the bug number [13:08] wait [13:09] Ursinha: however, the codehosting code can do something like I suggested earlier and add a comment to the bug if it sees a linked branch merged [13:13] argh === danilo_ is now known as danilos === mrevell is now known as mrevell-lunch [13:39] EdwinGrubbs: bac: stand up in two minutes [13:39] adeuring, if you have time, could you look at the OOPS for Bug #548575? more ShortListTooBigErrors. [13:39] deryck: sure [13:39] adeuring, thanks! [13:40] Bug #548575: Not able to unsubscribe from a bug [13:43] deryck: did we really CP the fix? [13:44] sinzui: standup? [13:44] adeuring, I thought we did. Maybe we didn't. Did you submit it to production-devel? [13:44] deryck: can't remember... [13:44] let me check... [13:51] EdwinGrubbs: https://code.launchpad.net/~bac/launchpad/bug-524302/+merge/22180 [14:15] adeuring, I assume we didn't CP that fix then? [14:16] deryck: no, we didn't cherrypick that branch. Sorry, needed some time to figure out which branch fixed the bug... [14:18] adeuring, no worries. Can you mark the bug a dupe of the bug that does fix this, please? [14:18] sure === mrevell-lunch is now known as mrevell [15:27] gary_poster: here's what I was talking about yesterday: https://code.edge.launchpad.net/~james-w/lazr.restful/object-unmarshal/+merge/22182 [15:28] * gary_poster looks [15:29] james_w: do you want a review? [15:29] rephrase: [15:29] do you want me to review [15:29] yes please [15:30] ok [15:30] I'd like leonard to look as well [15:30] I'm not entirely convinced by the approach as it isn't a great example of elegance, and prevents you doing some odd but legal things. [15:30] but I can't find a better way [15:32] james_w: leonard will not be able to before this release (not around today). I agree that he should review. Because he is not around today, this won't make it into the cycle unless we push it as release-critical. My gut feeling so far is that this is more important for developing (that is, future work) that for production (that is, this release). Do you agree? [15:33] (IOW, the fact that this will not make the release does not seem like a release-critical problem; but we should get it into the devel branch asap) [15:33] gary_poster: yes, sinzui landed the branch that fixes the problems yesterday, this is to stop any more from creeping in. [15:34] james_w: cool. I'll look now to see if I have anything to add, and make sure that reviewing this branch is in leonardr's to-do list. [15:34] thanks [15:35] there's a long line in the doctest that I don't know how to wrap [15:37] james_w: you can use #doctest: +ELLIPSIS and elide the text with an ellipsis. #doctest: +NORMALIZE_WHITESPACE might let you wrap the line, but IME sometimes tracebacks are "exceptions" to the rule. A-ha-ha. But you could try that first. [15:37] james_w: >>> reference_marshaller.unmarshall(None, cookbook) # doctest: +NORMALIZE_WHITESPACE [15:37] that goes on the test line, or on the response line? [15:38] ah, thanks [15:38] or [15:38] >>> reference_marshaller.unmarshall(None, cookbook) [15:38] ... # doctest: +NORMALIZE_WHITESPACE [15:40] fixed, thanks [15:40] cool, np === matsubara is now known as matsubara-lunch === salgado is now known as salgado-lunch [15:57] james_w: yeah, that [15:57] 's not fun. Can you tell me where the adapter is that you mention in the cover notes? [15:58] That uses an Object marshaller for a Choice? [15:58] (with the certain kind of vocab) [15:58] some webservice zcml [15:58] lib/canonical/launchpad/zcml/webservice.zcml [15:58] ack thanks [15:59] I thought the most elegant way to solve it would be for tales.py to just add _link if it would just an ObjectMarshaller for that field, but there's already a comment there saying it can't do that [16:00] james_w: My thought would have been that using an ObjectMarshaller is an abuse. We should write our own marshaller for this case that does the right thing. [16:01] Does the tales comment (which I have not yet looked up) contradict that? Going to hunt down tales comment... [16:01] search for Marshaller in src/lazr/restful/tales.py [16:02] I'm not sure what "the right thing" for the new marshaller to do would be [16:04] so, the root cause is that there are two different things, using two different criteria, deciding what the field name should be in the API. When they disagree you get problems. [16:05] james_w: a minimal "right thing" would be to puke, as you are doing here. Puking where you have it now (in lazr.restful) doesn't make sense because, looking at the code, why would an ObjectMarshaller ever be called with anything other than an IObject? It shouldn't. [16:05] A secondary concern is that we want exported objects to be serialised to links, and when that is done for the field name to have _link appended. [16:05] root cause: that sounds valid, but it's been too long since I looked at the code for me to have an informed opinion without staring for awhile more. [16:06] gary_poster: right, it never is. The issue is when that Marshaller isn't bound to an "object" field type, the name is already wrong in the wadl. [16:06] so I'm fixing the wrong end in effect, as the problem is already past, but this barfs at you and tells you something is wrong at least, rather than silently doing the wrong thing. [16:07] I couldn't see how to fix it at the "other end", which would be the field declaration and tales.py, as we need the developer to indicate whether it is a field that references another exported object or not. [16:07] which they do by using ReferenceChoice instead of Choice [16:09] perhaps the right thing to do is change the adapter declaration so that it only affects IReferenceChoices? [16:09] james_w: right, which is why I called it an abuse. We can't protect ourselves from all abuses in lazr.restful. Here's my proposal for your current motivations. Since I have this alternative, I would not approve your approach. Leonard may have an even better answer. [16:10]