[01:47] rockstar: ping? [01:47] wallyworld_, hi. [01:48] hey, with merge queues, i had started to do personproduct listings as a next step for person listings. is that something you still need me to do? [01:48] and i guess product listings [01:50] btw, i missed getting the person listings landed before the cut off. the branch passed ec2 ages ago but i was never lucky enough to find a non-testfix window to get the thing landed [01:51] wallyworld_, I'm not sure why we'd need personproduct listings. [01:51] Oh, product listings. [01:51] Um, I'm not sure we do, but thumper might have an opinion. [01:51] We have this idea that we need listings of everything, and I don't necessarily agree with that. [01:52] ok. wrt not getting the branch landed - it was done in devel to pick up the db changes but after the rollout, thise db changes will be in prod [01:52] wallyworld_, so, one thing that would be helpful with merge queues would be to take the JSON constraint out of the config. You were right, we shouldn't force JSON on users. [01:53] should i merge the stuff i did in db-devel into devel and land that after rollout? to get the person listing stuff in? [01:53] ok. you want me to remove the json constraint? [01:58] wallyworld_, yeah, if you could. [01:59] ack [01:59] wallyworld_, it's still (currently) only editable through the API anyway. [01:59] wallyworld_, I think landing on devel once the devel rev has been tagged is okay, no need to merge (provided there are no conflicts) [02:00] ok. i'll do that, so that you can pick up what i've done [02:01] rockstar: is there a bug filed already for the json issue? [02:02] wallyworld_, no, I don't think so. [02:02] ok [02:02] wallyworld_, the duplication of bug and kanban task prevented me from ever being organized enough. [02:02] :-) [02:03] wallyworld_, the only other backend thing that really needs to get done immediately is queuing a merge proposal, which I'm working on right now. [02:03] ok [02:03] Launchpad still has to be in a chroot though, because you can't develop ubuntuone and launchpad together on the same system. [02:04] that's too bad [02:05] buildout mess? [02:05] Not fixable with virtualenv? [02:06] wgrant, well, first, Launchpad eats postgres. All of it. [02:06] It only does that once. [02:06] After that it leaves it alone. [02:13] lifeless: Can you recommend a commandline testrunner compatible with testrepository? [02:17] abentley: I'd guess trial --reporter=subunit [02:18] spiv: I don't have a dependency on Twisted. Doesn't trial depend on it? [02:19] yes [02:20] (but your tests don't have to depend on Twisted or Trial, if that makes a difference) [02:22] There are probably subunit plugins for other tools like nose, but that seems no different to me than using trial. [02:22] abentley: python -m subunit.run [02:23] abentley: or as spiv says trial --reporter=subunit [02:23] lifeless: oh, subunit has a runner? Cute! [02:23] lifeless: cool. [02:24] there is a nose subunit plugin floating around but its not really all that polished AIUI [02:25] rockstar: u1 have a very good system of spinning up a non-system postgres for their test suite, you should be able to mix and match [02:27] elmo, yeah, and at some point, I'll sort that out. [02:27] (Probably not on my first day with the u1 team) [02:27] abentley: oh, and bzr selftest --subunit [02:27] === Top 10 Time Out Counts by Page ID === [02:27] Hard / Soft Page ID [02:27] 180 / 53 Person:+commentedbugs [02:27] 78 / 268 BugTask:+index [02:27] 46 / 2908 Archive:+index [02:27] 19 / 331 Distribution:+bugtarget-portlet-bugfilters-stats [02:27] 14 / 312 Distribution:+bugs [02:27] 10 / 12 Person:+bugs [02:28] 8 / 100 MaloneApplication:+bugs [02:28] 8 / 5 DistroSeries:+queue [02:28] 5 / 15 DistroSeriesLanguage:+index [02:28] 5 / 6 ProjectGroup:+milestones [02:28] lifeless: sure. And the Launchpad test runner too. But this is a separate project. [02:28] sinzui: ^ Person:+commentedbugs has a fix in lp from stub [02:28] Hmm... Archive:+index is still broken? [02:28] abentley: righto, was just enumerating as things occured to me ;) [02:28] wgrant: broken - not. Not all it can be - yes. [02:30] lifeless: Hard timeouts == broken. [02:31] wgrant: sure [02:31] 46 is tolerable while we're recovering from 8.4 [02:31] wgrant: compared to hundreds per day [02:31] wgrant: also note - no XMLRPC ones ;) [02:31] Indeed. [02:31] A bit disturbing that that was all just an overloaded appserver. [02:33] I'm a bit surprised that +branches pages don't show up on that list, it suggests to me that branch listings are rarely used, which is concerning in itself [02:33] Is +branches linked from anywhere? [02:33] sorry, I was being overly precise [02:33] +code-index? [02:33] whatever you get when you go to the "Code" tab [02:34] particularly person and person+pillar listings give me timeouts [02:35] * james_w causes a few OOPS for those pages to try and get lifeless to fix them :-) [02:37] james_w: https://devpad.canonical.com/~lpqateam/ppr/lpnet/latest-daily-pageids.html [02:43] thumper: https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1781G129 [03:04] lifeless: what would test_id_option be for subunit.run ? [03:18] abentley: subunit.run doesn't support filtering at the moment [03:18] cat .testr.conf [03:18] [DEFAULT] [03:18] test_command=PYTHONPATH=. python -m subunit.run foo.tests.test_suite [03:18] abentley: ^ something like that is all you'd need [03:19] lifeless: but I wouldn't be able to use testr run --failing? [03:19] it will work, but it would't filter [03:20] I really should add filtering to subunit.run [03:21] lifeless: Yes, I think that would be very helpful. [03:22] Trial can filter :P [03:23] spiv: indeed, and I guess that makes it the best tool for the job, at the moment. [03:24] *nod* [03:24] spiv: err, how do I make it filter? [03:25] abentley: you can pass it test names [03:25] (or test class names, or test module names) [03:26] spiv: So what do I specify for test_id_option ? [03:27] e.g. if there's a test called package.tests.module.Class.method, trial will run it with any of "trial package", "trial package.tests", "trial.tests.module", "trial.tests.module.Class", "trial.tests.module.Class.method" (although obviously all but the latter are likely to find more tests than just that one) [03:27] heh, I typoed some of those badly [03:28] Basically "trial $python_dotted_name" will run all the tests contained in that dotted name [03:28] It'll probably fail to work nicely if you have test ids that don't correspond to python dotted names (e.g. if you have test multiplication via testscenarios or something) [03:29] spiv: I can't use something like that. I need something that allows me to specify a file with test ids. [03:29] trial `cat $file` ? [03:30] spiv: testrepository does not appear to support that. [03:30] sh -c "trial `cat $file`" ? ;) [03:30] Hmm, what a shame. [03:30] So close and yet so far! [03:31] spiv: testrepository expects that there is an option you can supply to the elsewhere-specified command that will make it read a file listing the test ids to run. [03:31] spiv: So I can't change the command, only append an argument. [03:31] abentley: sounds like a job for a shell script until one of these tools gets better :( [03:32] spiv: Yeah, seems like. [06:44] spm: Around? [06:45] I think germanium is pretty upset. [06:46] No uploads accepted or packages published for nearly half an hour. [07:03] losa ping: ^^ [07:32] jml: up yet? [08:04] Could someone prod ISD about the issue in #launchpad? [08:06] SSO isn't LP per se is it? [08:06] Exactly. [08:06] Needs ISD. [08:06] And they like to hide. [08:08] wgrant: :/ [08:08] I remember seeing a sort of rant mail about it in launchpad-dev [08:08] It's not an awfully wonderful situation. [08:10] i will try [08:10] Thanks poolie. [08:10] what is the issue, in one sentence? [08:10] and how critical? [08:10] Another email address that SSO won't send mail to. [08:10] Two support cases have been opened about it. [08:10] But both have been ignored. [08:10] :/ [08:10] ixokai in #launchpad [08:11] and the thing with germanium is separate? [08:11] Yes. [08:11] That doesn't need ISD, fortunately. [08:11] It just needs a LOSA, hopefully. [08:11] ah, spm is away today [08:12] Ahh. [08:12] o/ [08:13] mthaddon, meet wgrant :-) [08:13] yeah, just catching up on backscroll in #launchpad [08:14] mthaddon: The main issue is that germanium's publisher is somehow unhappy. [08:15] wgrant: any clues as to where to look? [08:16] mthaddon: The logs for cron.publish-ppa might have something interesting. [08:16] I don't yet know where they live. [08:16] yeah, looking at that now [08:17] can't see anything untoward in there [08:17] Is there anything not untoward there? [08:18] looks like there was issues with claiming the lockfile for a while, but seemed to clear itself up a few mins ago [08:19] definitely seeing stuff processing now - you sure it's still an issue? [08:19] Ah, so it is. [08:19] It's happening now. [08:19] Not quite done, but getting there. [08:19] I wonder why it was broken. [08:19] k [08:19] Thanks. [08:40] good morning [09:01] Morning bigjools. [09:02] morning [09:02] The PPA publisher was MIA for ~2 hours earlier, but mysteriously recovered once mthaddon started poking around. [09:03] :/ [09:03] that happened the other day too [09:03] the librarian went awol [09:03] Ah, interesting. [09:04] no librarian = sad soyuz [09:05] Are we running Librarian with tokens now? [09:05] no idea [09:05] Not sure what has changed with the Librarian apart from that the last few years. Or Apache stuff could be wonky I guess. [09:05] We're not. [09:05] Or Squid [09:05] It's still webapp proxying. [09:10] Bonjour === Guest86531 is now known as jelmer [09:34] poolie: am now [09:36] File "/srv/launchpad.net/production/launchpad-rev-11926/lib/lp/registry/interfaces/person.py", line 172, in validate_person_common obj, getattr(obj, 'name', None))) PrivatePersonLinkageError: Cannot link person (name=canonical, visibility=PRIVATE) to (name=None) [09:36] I get this when I try to make ~canonical a member of ~canonical-data-science. [09:36] What's the deal? [09:38] http://docs.fabfile.org/0.9.3/tutorial.html [10:44] jml: did you have any comments about the buildd-manager changes? === almaisan-away is now known as al-maisan === al-maisan is now known as almaisan-away [12:04] bigjools: none from my skim. glad that BuilderSlave has a multi-file fetch method. [12:04] * jml skims again [12:05] jml: and shockingly easy to implement [12:05] bigjools: yeah [12:05] bigjools: I found that once I got the hang of Twisted, a lot of stuff like that just falls into place. [12:05] jml: it's starting to click for me now [12:06] I need to read up on all the protocol stuff though [12:08] *nod* [12:08] bigjools: I don't think you need files_downloaded (in packagebuild.py) [12:08] you could also do: [12:08] d = slave.getFiles(filenames_to_download) [12:09] d.addCallback(lambda x: self.storeBuildInfo(self, librarian, slave_status)) [12:09] d.addCallback(build_info_stored) [12:09] return d [12:09] possibly, I'll take a look. I went through many iterations writing that code and I expect there's some cruft [12:10] spot the lie: http://paste.ubuntu.com/532951/ [12:10] no lie :/ [12:10] well, maybe a half-truth [12:11] when was the "next Paris summit"? [12:11] :) [12:11] yeah :) [12:12] * jml is looking forward to finishing the damn testtools-experiment branch. [12:13] thank you again for your tutelage jml [12:13] np. [12:38] danilos, henninge: I can't find any remaining call sites for migrate_kde_potemplates.py… am I missing one? Or can I just drop it? [12:39] jtv: no idea but that sounds dropable ;) [12:41] jtv, I think we had it killed already [12:41] danilos: yes, I think we did—and this is just a remaining piece. [12:42] Then there's message-sharing-populate.py. [12:42] jtv, do away with it then... that too [12:42] jtv, there was another one, remove-obsolete-translations I think [12:42] Oh! And here I just updated that one. :) [12:43] That's in something called KNOWN_BROKEN. [12:44] I'm offline to get some more writing, diagramming and strategizing done. (so leet). Available on mobile, back in time for standup. === mrevell is now known as mrevell-lunch [13:01] gmb: hey [13:01] gmb: so I wanted to chat with you & deryck about bug subscriptions [13:02] gmb: we seem to be a little confused about them. Perhaps a LEP - not about the 'subscribe-to-search' bit - but about subscriptions /per se/ would clarify things? [13:02] gmb: I mean, the 'we email dup subscribers' is deeply embedded in the code - we've unpleasant queries that exist just to answer that question. [13:09] lifeless: I think a LEP to clarify what we actually want to do with subscriptions themselves is a very good idea. === Ursinha-afk is now known as Ursinha === mrevell-lunch is now known as mrevell [13:47] danilos: hi [13:47] danilos: I'm replying now, but would love more data about the use case! [13:48] lifeless, which one? (I am kinda mentioning both :) [13:48] danilos: these guys recommend model by 'what queries do you want to do' [13:48] gimme 2 minutes, I'll get this reply to you. [13:48] lifeless, sure thing [13:48] lifeless, I have a vague feeling that it wouldn't really work for us at all, but it's worth asking the questions while you are there :) [14:05] danilos: hi [14:05] danilos: sorry about that, had to move to the training room :) [14:05] lifeless: training is good for your health :) [14:06] :) [14:27] danilos: anyhow, does my reply help? Should we discuss it some more? [14:30] lifeless: sorry, got distracted myself === Ursinha is now known as Ursinha-lunch [14:31] sinzui: hi - from your bug comment are you volunteering to fix bug 654476? [14:33] lifeless: paraphrasing for the second use case seems about right [14:34] bigjools, I thought I had volunteered a few weeks ago [14:35] sinzui: really? I didn't know we'd narrowed it down to the issue I last mentioned on the bug [14:36] bigjools, well, we scheduled the team membership policy fix for this release. I think add a check for PPA as well as other teams takes but a few minutes of work [14:36] sinzui: indeed, ok thanks. I'll re-assign the bug to your team. [14:36] okay === matsubara is now known as matsubara-lunch === jam1 is now known as jam [15:00] bigjools: I'm really happy that the builddmanager is coming along so nicely [15:00] lifeless: you are by no means the only one! [15:00] lifeless: I have one more scalability issue to talk to you about though [15:02] when you've got some free time it would be good to chat in more deptch [15:03] bigjools: shoot [15:03] oh, free now? [15:04] We're doing some simple stuff right now [15:04] I can track IRC well enough to get bootstrapped on your issue [15:04] lifeless: ok, so, the main issue is that every time we add a new builder, it makes the b-m issue "the" query is issued at regular intervals. That does not scale. [15:05] partly because the b-m is using the master store [15:05] what does this query do (in english) [15:05] figures out what to dispatch next [15:06] *cough* a bit like a queueing system one might say *cough* [15:06] do we model the queue directly, or do we model imputs and assemble the queue every time? [15:07] there is no queue as such, it issues the query every 15 seconds (was 5 but I slowed it) to see if there's a candidate [15:07] so more like the latter [15:07] so long term I think we need to overhaul the entire queueing model for buildds [15:07] the buildqueue used to model a "queue" but things got quite complicated in the last year ;) [15:07] as I think I've said before [15:08] to reduce starvation and so forth [15:08] it's what everyone has said before I think :) [15:08] in the short term [15:08] it's not a short-term problem [15:08] but when we get those 200 extra builders ... [15:08] what about leaving the builder idle until the next 15 second poll [15:08] I don't understand what you mean by that [15:09] ok [15:09] so lets say we had the 200 builders all attached [15:09] and nothing to build [15:09] they are presumably in some state where we can dispatch a build whenever we want [15:09] yes [15:10] and then if I upload a build, after processsing etc, your 15 second frequency queue sampling operation will find a build and dispatch it. [15:10] bigjools: I'm assuming your query is something like this - psuedo code: [15:10] needed_build_requests = len(builder for builder in builders if builder.is_idle) [15:11] build_requests = issue_THE_query(needed_build_requests) [15:11] no [15:11] so that if there are no idle builders you don't issue the query at all [15:11] there is a separate query for each builder [15:11] bigjools: is it grouped by equivalent builders? [15:11] no [15:11] like, surely all virtualised i386 builders could be queried at aonce [15:12] yes that would be eminently sensible [15:12] anyhow, what I meant was [15:12] but since forever it has always issued one query per builder [15:12] put the builder in whatever state a builder waiting for a build is [15:12] sinzui: hi [15:12] hi jml [15:12] and let the next lookup after 15 seconds sort it out [15:12] sinzui: thanks for deduping my bug [15:12] sinzui: I was wondering if you know of a work around? [15:13] bigjools: but if you're going to do one-per-builder anyway... [15:13] jml: maybe make canonical the owner of the team? === Ursinha-lunch is now known as Ursinha [15:13] sinzui: even though I can't add canonical to the team? [15:13] bigjools: then I think making the modelling be able to answer the query instantly is probably the key thing. [15:13] lifeless: so we could mitigate things a lot by remodelling to issue one query per (arch,virt) [15:13] the query is already fairly quick to be fair [15:14] it's just issued *a lot* :) [15:14] bigjools: and add a limit for the number of available builders ? [15:14] sinzui: "The person/team named 'canonical' is not a valid owner for Canonical Data Science CoP." [15:14] bigjools: (such that you can short circuit when limit=0) [15:14] yes, we'd need to track state carefully - but this is a major, major change to the b-m operation [15:14] bigjools: the idea being that that would let you drop it down to checking every 1-2 seconds, for instance. [15:15] lifeless: I think that this is a good direction to head. It's an almost total rewrite of the b-m to do it though :( [15:16] jml: yes that is correct, it will not be valid because it will cause a membership change. I think OEM/ISD has employed scripts to add members from one team to another [15:17] sinzui: so, effectively there is no practical work-around to the problem beyond manually adding every member of ~canonical to the new team? [15:17] jml: I really do not think there is an acceptable work around. This bug has been a super pain for 18 months [15:17] or scriptedly [15:17] sinzui: ok. [15:18] sinzui: we are going to fix this with the new privacy model, right? [15:19] Our pleas for help were largely ignored last year. So while I want to fix this. I cannot say it will make everyone happy. [15:19] restating the problem as a membership policy issue, not visibility issue was a break-through for us two months ago [15:20] sinzui: making everyone happy is not a requirement. covering this use case (make a team that's visible to members of another team) is. [15:21] multi-tasking is melting my brain. [15:21] The need for team hierarchy will be lessened when users and teams can be permitted to observe without being members. And explaining that members need to guard their messages on mailing lists and contact-this-team emails may suffice [15:22] jml: This is a super hard problem. Lots of people have worked on this and it has befuddled everyone. [15:22] bigjools: the long term one would be 'be able to answer a single nodes needs very very efficient'y [15:23] bigjools: though I do think querying for a set of buuilders needs at once is sensible regardless. [15:23] bac checked the TP leaks a few months ago and reported that they were quite bad. [15:23] lifeless: totally. In fact when we start using a real queueing solution we just have one queue per (arch,virt) and be done with it. [15:24] lifeless: one of the complications right now is that stuff can be in the queue but we have rules that stop it getting dispatched [15:24] oh? [15:24] is there lots of that? or just a few jobs at a time? [15:24] lifeless: there's two: 1., private sources need to be published, 2. we don't allow PPAs to issue too many concurrent builds [15:25] jam: cassandra is basically bzr's store ;) [15:27] gmb, hi. In the continuing saga of lazr-js upgrade, I have a clean test run! hurrahs heard round the world! [15:28] deryck: Let joy be unconfined. [15:29] gmb: so you'll do a LEP then? [15:31] lifeless: well, pack files, but certainly different indexing at the top level, and it isn't multi-writer safe :) [15:32] lifeless: Yes. I'll start drafting one on Thursday (I'm away tomorrow) [15:33] gmb: awesome, thanks! [15:33] jam: the indexing is a little different, but not all that much ;) [15:33] lifeless: the in-memory-only hash map is pretty different [15:33] jam: its (rowkey, columnname) tuples for the indices AFAICT. === salgado is now known as salgado-physio === You're now known as ubuntulog === henninge changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 4 of 10.11 | PQM open for 10.12 | firefighting: - | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting === You're now known as ubuntulog_ === You're now known as ubuntulog === matsubara-lunch is now known as matsubara [16:49] so, back from vacation and i'm trying to get the branch-lander working again per matsubara's email, but all i get when trying to authorize is a redirect circle between the login page and "confirm sharing these details" page. anyone run into this when the change landed last week? [16:50] jcsackett, what's the URL for that page? [16:52] matsubara: login.launchpad.net [16:52] auto-taken there when i run land. [16:53] jcsackett, sounds like you're having problem with sso authentication. do other LP pages work for you? === benji is now known as benji-lunch [16:56] matsubura: i had thought they were, but you're right, i can't login period right now on this machine (oddly on another one i'm having no problems). i'll fidget with it a bit more. [16:56] thanks. [17:06] ah, botched my ssl configuration. [17:10] jml: was there some sort of bug in Trial to do with leaving a dirty reactor sometimes? [17:11] bigjools: it's not a bug in trial, it's a bug in the tests that cause it. [17:11] (or the code that the tests call) [17:11] jml: I know that, but I was wondering if there was something in Trial too. I have a test that's randomly leaving a dirty reactor :/ [17:12] so far only happens on a full test run [17:12] bigjools: it's almost certainly not a bug in trial. [17:12] ok, good to confirm if nothing else [17:12] bigjools: a very common class of test that leaves junk in the reactor does so intermittently [17:12] bigjools: point me at the test and the error. I might be able to spot the problem. [17:13] jml: test_binarypackagebuild/test_date_finished_set [17:14] bigjools: in which branch? [17:14] jml: the one you reviewed [17:15] jml: lp:~julian-edwards/launchpad/async-file-uploads-bug-662631 [17:16] bigjools: ta [17:16] * bigjools wishes the dev wiki would redirect back to the page you were on after logging in [17:17] mars, ping. [17:17] Hi deryck, what's up? [17:17] Hi mars. I got my lazr-js upgrade branch passing ec2 now, but I still have horribly long run times.... [17:18] mars, can I send you a log and some notes this morning and see if you have any idea what's going on? [17:18] deryck, do you still see thread trash lying around? [17:18] bigjools: which subclass? (specifically, what is self.build set to?) [17:18] mars, yes [17:18] jml: TestHandleStatusForBinaryPackageBuild [17:19] it seems to work for the recipe one [17:19] deryck, I can look, certainly. Fire the mail over. I can reply with a HTTP on-the-wire technique I successfully piloted the last time we did this, it should sniff out the off-site accesses, if any are around. [17:19] bigjools: ta [17:19] mars, ah, ok. That would be useful to try. === salgado-physio is now known as salgado [17:21] bigjools: what's the left-over? [17:22] jml: http://pastebin.ubuntu.com/533122/ [17:22] wow. that's a lot. [17:22] I would guess at a timeout [17:23] mars, thanks for the help! Two emails fwd'ed to you. [17:23] jml: it's probably the getFiles() - it sets up a lot of callbacks [17:23] bigjools: self.buildqueue_record.builder.cleanSlave() returns a Deferred but the code doesn't handle it. [17:23] * bigjools looks [17:23] bigjools: maybe not the bug here, but it's a start. [17:24] bigjools: in _handleStatus_OK [17:24] I didn't change any of that [17:24] so it was there before [17:25] bigjools: but you've moved more tests to run under trial with the stricter checking [17:25] jml: and that same calls is repeated in a few places [17:25] and that code is in production right now [17:26] bigjools: perhaps it doesn't matter when the slave gets cleaned, as long as it's eventually [17:26] I think it's luck that it's not breaking [17:27] the polling loop will wait until it's clean [17:27] this is probably the test problem though [17:27] bigjools: I'm inclined to support the luck theory [17:27] but why is it intermittent [17:27] bigjools: it's the most obvious part. [17:27] bigjools: because sometimes it finishes before the test does? [17:28] ah [17:28] and sometimes it finishes after [17:28] mebbe [17:28] fun [17:28] yeah [17:28] can't guarantee order of execution without deferreds. === benji-lunch is now known as benji [18:12] mars, I *just* found more combo loading attempts going on. [18:12] ugh. [18:12] deryck, where? [18:13] deryck, outside your search-and-replace sweep? [18:13] mars, vocab picker uses YUI().use [18:13] 10 personal emails to answer. My friends, family and contributors to my free software projects must hate me :( [18:13] mars, yeah, missed it somehow. [18:13] no one hates jml :-) [18:13] :) [18:15] mars, found a bunch with a more liberal grep now. Oh, well. Easy sed fix and maybe this will be finished. [19:04]