/srv/irclogs.ubuntu.com/2010/09/15/#launchpad-dev.txt

marsmbarnett, any word on the buildbot stuff?  I'm way past EOD here, but I don't want to leave the whole system offline like this00:16
lifelessperhaps spm has the torch now00:18
=== lifeless changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 1 of 10.10 | PQM is open for business | firefighting: - | https:/​/​dev.launchpad.net/​ | Get the code: https:/​/​dev.launchpad.net/​Getting
* spm hates at buildbot, reiterates plea from the heart and declares success in getting the PoS back00:19
marsthanks spm00:19
spmmars: np; it will need any 'was happening' builds to be stabbed tho. requerued? whatever it's called. forced?00:20
marsspm, could you please restart the lucid_lp and lucid_db_lp build slaves too?  I don't want to have to return to this tonight00:21
marsafter that I will force everything to rebuild00:21
spmmars: you can do all of that yourself00:22
spmthe lucid slaves have to be killed as part of the recovery00:22
marsoh, good00:22
spmbasically you *must* shutdown everything BB related. kill/stab/burn with fire. then bring up the lucid slaves; then the master. repeat on restarting the master till the *(%(%^*&%^&*$%&*$%ing thing works.00:23
marsspm, so your procedures now include the steps to preven process trash from half-done builds lying around?00:23
marsok00:23
marsso 'yes'00:23
spmmars: that's the burn with fire part :-)00:23
mars\o/00:23
lifelesssinzui: still around?00:23
lifelesssinzui: how does class="listing sortable" interact with batching?00:24
marsall builds forced00:24
mwhudsonlifeless: poorly, because sortable just sorts the current batch00:25
marsspm, lifeless, if the prod_lp slave dies again, we'll have to do a full restart and force all the builders again.  It's the only way.00:25
lifelessmwhudson: is it tolerable? (for +assignments)00:25
lifelessmars: thanks00:25
marsaw FL*(#00:26
spmmars: orsum00:26
mwhudsonlifeless: no idea really, it's not great00:26
marslifeless, spm, so, about that "if prod_lp" dies thing - surprise!  Guess what just happened...00:27
lifelessmars: raarh00:27
mwhudsonsubstantiate failed!00:27
mwhudsonquickly too00:27
marsugh, all the EC2 builders died00:27
spmmars: I'll need a few more clues? ;-)00:27
lifelesshammer00:27
lifelesstongs00:27
mwhudsonmaybe aws is having a bad day00:28
lifelessapply00:28
marsspm https://lpbuildbot.canonical.com/waterfall00:28
spmmars try a second force - I did terminate the (old) ec2 instances that were running. so the master may have belatedly got poor signals?00:28
marsneed to go afk, crying babes in arms, back as soon as possible00:30
mbarnettmars: sorry for dropping you there00:33
* spm forces a build on prod_lp00:33
wgrantAfter the last rollout, we have a few users who need their multiple accounts merged.00:42
wgrantExcept they now only have access to the new account -- the one they don't want to keep.00:42
wgrantShould I ask them to file a question so an admin can merge them the right way?00:42
pooliejam: 'from twisted.internet import defer; defer.setDebugging(True)'00:43
lifelessmwhudson: are there tests of specification template rendering ?00:46
mwhudsonlifeless: mostly only in the page tests00:49
mwhudsonakaik00:49
lifelessmwhudson: where would they be? the .txt files i found had no browser_ stuff00:49
lifelessmwhudson: skip that, time I learnt something new00:50
lifelessmwhudson: whats the preferred way to render a template00:50
lifelessI want to check that it gets batched in unittest style code.00:50
mwhudsonlifeless: using testbrowser is probably easiest00:50
lifelessself.getUserBrowser() ?00:51
mwhudsonotherwise, create_initialized_view() and call .render() i think?00:51
mwhudsonyeah00:51
lifelessah, .render is probably cleaner00:51
lifelessless layers for what I'm interested in.00:51
mwhudsonyeah, probably00:54
lifelessahhh stories/standalone is where stuff was.00:54
lifelessnvm I'm deep into doing it right :)00:54
wgrantspm: Do you have a moment to do a quick account merge? https://answers.edge.launchpad.net/launchpad/+question/125444 -- he's been unable to log in since the rollout last week.,01:35
spmnot really, but will do.01:35
wgrantHeh, thanks.01:35
spmahh. thanks! you've done the research. much appreciated!01:36
spmis done01:37
wgrantYeah, that's why it was really quick.01:37
wgrantThanks.01:37
wgrantabentley: Why do we not permit arbitrary code execution in the tree build step?01:49
wgrantSome seem to think that it is because of security.01:50
wgrantBut that cannot be the case.01:50
poolierockstar: hi?01:58
lifelesswgrant: why can't it be the case?02:09
wgrantlifeless: Because the chroot doesn't provide security.02:10
wgrantSo doing something outside the chroot can't be a security vulnerability.02:10
lifeless Hard / Soft  Page ID02:17
lifeless     229 / 1575  ScopedCollection:CollectionResource:#message-page-resource02:17
lifeless      91 /  255  BugTask:+index02:17
lifeless      67 /   34  DistributionSourcePackage:+filebug02:17
lifeless      52 /    7  Distribution:+search02:17
lifeless      48 /  231  CodeImportSchedulerApplication:CodeImportSchedulerAPI02:17
wgrantWhat is message-page-resource?02:19
wgrantI haven't seen that before.02:19
lifelessI'm looking ATM02:20
lifelesshttps://api.launchpad.net/1.0/bugs/136469/messages02:20
_mup_Bug #136469: toshiba p100 series dsdt acpi error no sound, works with acpi turned off. <linux (Ubuntu):Fix Released by ubuntu-audio> <linux-source-2.6.22 (Ubuntu):Won't Fix by ubuntu-audio> <https://launchpad.net/bugs/136469>02:20
lifelessws.start=75&ws.size=7502:21
lifelessSQL time: 4344 ms02:21
lifelessNon-sql time: 10882 ms02:21
lifelessTotal time: 15226 ms02:21
lifelessStatement Count: 28802:21
wgrantSomehow I doubt it.02:21
lifeless?02:21
rockstarpoolie, hi.02:22
wgrantThat much non-SQL time for so few objects?02:22
pooliehi rockstar, i was just going to ask if you could pair with jam on getting his bzr-lp forking server finished02:22
lifelesswgrant: welcome to bad-O, or thread-thrashing, or a few things02:23
poolieand on reviewing what he's done so far02:23
wgrantlifeless: I meant that it was probably not bad code, but thread-thrashing, yes.02:23
lifelesswgrant: its -so- frequent its not going to be a passive victim of worse pages02:23
pooliei think you may have been there for some discussion of it at the epic02:23
rockstarpoolie, I might not be the best guy to pair with him.  I'm mostly a frontend guy recently.02:23
rockstarpoolie, in fact, I don't know much about the forking stuff at all.02:23
jamrockstar: the big advantage is that you are in my timezone :)02:23
rockstarpoolie, but if you think I might be of help to him, I'm happy to do it.02:23
rockstarjam, :)02:23
rockstarjam, abentley might be better.  He's paired with jml on parts of that code at least.02:24
jamrockstar: I've got a lot of small questions02:24
poolieabentley could also be great02:24
rockstarjam, okay.  Shoot.  I might be able to answer a few of them at least.02:24
lifelesswho knows about how deferred mail works (for bugs, mp's etc etc)02:26
lifelessquestions need this done to them pretty soon02:26
lifelessbut I don't want to do it arbitrarily differently.02:26
wgrantlifeless: Haven't we established that we really don't want to follow the Bugs method?02:27
lifelesswgrant: policy of naming, no. Method of deferring - I haven't considered.02:27
lifelesswgrant: tell me about it.02:27
wgrantBugs and Code do it differently.02:27
lifelessgreat. Please be informing your humble TA02:27
* lifeless grovels and drools a little02:27
wgrantBugs changes create rows in BugNotification, with a BugNotificationRecipient for each (direct or indirect) subscriber.02:28
wgrantsend-bug-notifications.py then processes those every 5 minutes or so, I believe.02:28
wgrantMPs only started deferred sending recently... and I think they use BranchMergeProposalJob for that, but I haven't looked closely yet.02:29
jamrockstar: I'm mostly watching a movie with my son right now, but I'll chat with you a bit tomorrow if you're around02:34
rockstarjam, okay.  Maybe abentley and I can combine our powers.02:34
lifelessmtaylor: oh hai03:08
lifelessspm: I can has profile?03:11
spmdamn. just 5 secs too slow on heading too lunch ;-)03:11
lifelessrotfl03:11
lifelesshar03:11
lifelessyes? no? maybe?03:12
spmoh, the yes was iomplied, just logging in03:12
lifelessif you like, once its enabled, I'll keep poking at it while you go to lunch03:13
spmsure03:13
spmis restarting....03:13
lifelesswgrant: https://api.staging.launchpad.net/1.0/bugs/1/messages?ws.start=150&ws.size=75 - da boom -03:13
_mup_Bug #1: Microsoft has a majority market share <iso-testing> <ubuntu> <Clubdistro:Confirmed> <Computer Science Ubuntu:Invalid by compscibuntu-bugs> <EasyPeasy Overview:Invalid by ramvi> <GNOME Screensaver:Won't Fix> <Ichthux:Invalid by raphink> <JAK LINUX:Invalid> <The Linux OS Project:In Progress> <OpenOffice:In Progress by lh-maviya> <Tabuntu:Invalid by tinarussell> <Tivion:Invalid by shakaran> <Tv-Player:New> <Ubuntu:In Progress by sabdfl> <03:14
lifelessspm: whats staging load like ?03:15
lifeless(can you kick it low before going)03:15
spmlifeless: 4ish, kickin'03:16
spmlifeless: and dropping. should be good to go03:17
lifelessthank thee kindly03:17
lifelessgrah03:18
lifelessno03:18
lifeless++profile doesn't work on the API03:18
lifelessbug filing time03:18
lifelessspm: turn it back off thanks, and shoo fo rlunch03:18
lifelessspm: (when you get back is fine, no rush from my POV)03:18
spmlifeless: is done03:20
lifelesswgrant: https://bugs.edge.launchpad.net/malone/+bug/497386 for your skepticsm03:23
_mup_Bug #497386: ScopedCollection:CollectionResource:#message-page-resource timeouts for bugs with many messages <api> <timeout> <Launchpad Bugs:Triaged> <https://launchpad.net/bugs/497386>03:23
wgrantlifeless: Am I missing something, or can (1) be done trivially with a DRS?03:25
lifelesswgrant: with a rank() function probably, as long as someone unwinds the login in indexed_messages appropriately03:26
lifelesswgrant: (thats yes)03:26
lifelesspossibly better to just store in the table though03:27
lifelesswgrant: more data in it now03:34
lifelesshah03:37
lifelessindexed_messages is whats exported03:37
lifelessI bet its calculating one of the links in it that is the *actual* bottleneck today03:37
lifelesshmm, I wonder how to trigger jsonification of an object for a test03:44
lifelesswgrant: have you perchance poked at that ?03:44
wgrantlifeless: No, sorry.03:58
james_wlifeless: jsonified by the lazr.restful machinery?04:05
james_wThere are methods to make webservice requests, but I've never seen a test that jsonifies something in Launchpad04:05
james_wlooking at the lazr.restful doctests might give a hint04:05
lifelessjames_w: yeah04:07
lifelesswondering how close to the issue I can get my test was all04:07
james_wlifeless: representation = simplejson.dumps(self, cls=ResourceJSONEncoder)04:14
james_wsomething like that might be close, but isn't everything the lazr.restful does04:14
lifelessyeah04:14
lifelessI'm just going to test the larger case - the key one - adding messages shouldn't increase queries.04:15
lifelessonce I get out of this ubuntu-bugs quagmire with our latest enthusiastic spammer04:15
james_wotherwise you need to get an EntryResource for the object, and call _representation on it apparently04:15
james_wand you need a request to do that04:16
james_wyeah, the simplejson.dumps should be close, but bypasses all the lazr.restful machinery, so it depends where you want to test04:17
james_wwell, not all, but lots04:17
lifelessneed the machinery :P04:21
james_wlifeless: you need the cache and things, not just the serialisation?04:22
lifelessI need to reproduce what the appserver does04:24
lifelessotherwise the test isn't complete.04:24
lifelessanyhow, I'm good.04:24
james_wwell, you don't run the test via apache :-P04:29
StevenKI've added a new model to soyuz in db-devel, and now that I'm getting around to using it, the security proxy is hiding all of it. How can I check/debug/fix that?04:30
lifelessjames_w: yes, but thats outside the scope that can trigger more db calls04:32
lifelessStevenK: interfaces04:32
StevenKlifeless: I added that too04:32
lifelessand enabled it via zcml ?04:33
StevenKlifeless: I certainly added it to the zcml, I'm not sure if it was right.04:33
StevenKlifeless: I can dig up the MP if you want to take a squiz04:33
lifelessStevenK: perhaps you should describe the symptoms more ;)04:36
lifelesslike 'in a test xxytyzz'04:36
StevenKlifeless: I've added a new method to lp.registry.interfaces.IDistroSeries called deriveDistroSeries(), and in the model code it's calling job = getUtility(IInitialiseDistroSeriesJobSource).create(child), and whenever I try and access any attribute on job, I get denied.04:38
lifelessis job the right type?04:39
StevenKIf I remove the security proxy to check, yes04:40
lifelesscheck the security you've permitted for the attributes is appropriate04:41
lifelessalso getUtility(IInitialiseDistroSeriesJobSource).create(child) seems awfully ugly04:41
StevenKlifeless: How do I check the security? My zcml-foo is fairly non-existant04:44
poolieoh lifeless i meant to say: what a nice tuesday mail05:19
pooliehere, have a peanut :)05:19
poolie(in the sense of gallery, not monkey)05:19
StevenKBwaha05:19
StevenKIs there any easy way to introspect the name of a variable?05:23
poolielook at it05:27
pooliewhere are you starting from?05:27
poolieStevenK: if you just have an object reference in python you can't directly tell what variable holds it05:27
pooliehowever, you can look at locals() and globals() and walk through the stack05:27
lifelesspoolie: thanks05:29
StevenKpoolie: I'm looping over a list of variables and croaking when they're not set, I was wondering if I could get at the variable name easily for a nice error message05:29
pooliepastebin the code?05:29
lifelessStevenK: paste the code05:29
lifelesslol05:29
poolie:)05:29
pooliepics or gtfo05:29
lifelesspoolie: was there anything particular about the tuesday may that made it nice?05:30
pooliei meant actually the mail about it05:30
poolieand only indirectly the content05:31
pooliebut it sounds like it was fun05:31
pooliei liked the "today i learned about $x"05:31
poolieit suggests other people could possibly learn about things too :)05:31
StevenKhttp://paste.ubuntu.com/493988/05:32
poolieoh it's going to say "None can't be unset"? :)05:32
poolieso you could do 'if not locals().get(param_name)' and list the things as quoted strings05:33
pooliei don't know if that's exactly idiomatic05:33
pooliealternatively just 'assert displayname;assert summary;...'05:33
StevenKpoolie: Exactly05:33
pooliewhich will give a clear but less pretty error05:34
poolie(and it wouldn't be allowed in bzr code, which bans the 'assert' statement because it can be turned off)05:34
lifelessso05:34
lifelessI'd define a tested helper in lp.services05:34
pooliealso i would do raise DerivationError(....05:34
poolienot the old syntax05:34
lifelessand do05:34
lifelesssomething useful with it :)05:35
lifelessthe args are available on the stack too ;)05:35
poolieone could have a decorator05:36
lifelessI think we may already have one05:36
poolie@require_nonnull_parameters(['breakfast', 'lunch'])05:36
lifelessthe apis stuff has some declaritive foo here05:36
StevenKYes, but the arguments are only required in one circumstance05:38
lifelesswhich circumstance is that?05:41
spivargvalues = inspect.getargvalues(inspect.currentframe()); for arg in argvalues.args: if argvalues.locals.get(arg) is None: raise ...  # ;)05:42
lifelessjtv: hey, so hows the template page in edge ?05:43
jtvlifeless: works fine05:44
lifelesstime to render?05:44
jtvfound it surprisingly fast on staging earlier05:44
StevenKspiv: Hold on, I'm dabbing the blood from my eyes :-)05:44
jtvlifeless: pretty decent; still slow but probably from a part of the process that we don't watch as closely such as transfer to the client05:45
jtvHaven't seen any timeouts at all on edge; I just managed to trigger one on staging though.05:46
jtvI'm guessing on staging it depends a lot on load.05:46
spivStevenK: oh, if you want blood then I'm sure I can find uglier ways to do that... sys._getframe().f_code.co_* springs to mind.05:47
jtvUnfortunately chromium's "view source" breaks a lot on this page05:47
jtvspiv: doesn't fit as neatly into the meter as "you got it" though05:47
StevenKspiv: Bah :-)05:48
jtvIf You Want Blood… sys._getgrame().f_code.co_*05:48
jtvlifeless: just got a render time of 7 seconds.  There's other things the new code will let us do if it's too much.05:48
jtvlifeless: in its current basic structure we could bring the critical loop body down to a single "%" operation.05:51
lifelessjtv: jtv check the render comments to evaluate time, for now :)05:52
lifelessonce we're down to a second of render time we can worry about transfer time05:52
lifelessjtv: oh, I see chrome fail05:52
jtvlifeless: just got one rendered in 2.33 seconds, so I think rendering time is close to losing its position as the dominant time sink.05:55
jtvFunny: looks as if firefox is actually faster than chromium for this page05:56
=== almaisan-away is now known as al-maisan
lifelessjtv: thats great06:03
jtvA bit disappointing to see so much variability.  It may be because we've become entirely bottlenecked on appserver CPU.06:06
jtvAt least, I think we have; I'll know more when my oops shows up.06:07
jtvBut as this becomes the norm for "performance-interesting" pages, we may have to look more critically at the GIL.06:07
lifelessI've filed a bug and RT to experiment on this06:08
jtvAh, there's my informational oops for +templates: 3885ms, of which 206ms are spent blocked.06:09
lifelesshow many rows?06:09
jtvJust over 2K.06:10
jtvAll the queries are at the head and the bottom, with a big chunk of rendering time for the table in the middle.06:11
jtvThe Big Query takes 86ms.06:12
lifelessso 2K - probably 666ms in storm06:36
lifelesson an idle machine.06:36
lifelessmore or less depending on row width06:36
StevenKlifeless: O hai, did you miss my question?06:36
lifelessquestion?06:57
StevenK< StevenK> lifeless: How do I check the security? My zcml-foo is fairly non-existant06:57
lifelessabout the same as mine I suspect06:58
StevenKlifeless: Bugger :-)06:59
lifelessstub: how long till 8.4 ?07:12
stubThe packages are on sourcherry so I want to do staging today.07:12
lifelessawesome07:13
lifelessI want to use rank() to get message indices07:13
lifelessshould shave a tonne of time off of bug 1.07:13
_mup_Bug #1: Microsoft has a majority market share <iso-testing> <ubuntu> <Clubdistro:Confirmed> <Computer Science Ubuntu:Invalid by compscibuntu-bugs> <EasyPeasy Overview:Invalid by ramvi> <GNOME Screensaver:Won't Fix> <Ichthux:Invalid by raphink> <JAK LINUX:Invalid> <The Linux OS Project:In Progress> <OpenOffice:In Progress by lh-maviya> <Tabuntu:Invalid by tinarussell> <Tivion:Invalid by shakaran> <Tv-Player:New> <Ubuntu:In Progress by sabdfl> <07:13
stublifeless: tsearch2 rank()? Should be in 8.3.07:13
lifelessstub: window function rank()07:15
lifelessrank() OVER (..)07:15
lifelessspm: hi07:15
spmyo07:15
lifelessspm: production rollout - buildbot borkage.07:15
lifelesshttps://lpbuildbot.canonical.com/builders/prod_lp/builds/9807:15
lifelessit grabbed rev 976207:16
lifelessI see 9765 as tip of production-devel.07:16
lifelesswe want ot remove the cowboy and get stuff out there. Dunno whats going on.07:16
lifelessHowever, I has to run - family time.07:16
spmbeyond bb being a pos as in?07:16
lifelessyes exactly07:17
lifelesswhy isn't it getting the tip of production-devel07:17
lifelesshttps://lpbuildbot.canonical.com/builders/prod_lp/builds/98/steps/bzr/logs/stdio07:17
lifeless argv: ['/usr/bin/bzr', 'checkout', '--revision', '9762', 'bzr+ssh://bazaar.launchpad.net/~launchpad-pqm/launchpad/production-devel', 'build']07:17
lifelessso its explicitly asking for an old rev07:17
spmthat... is seriously weird07:17
lifelesswhich is the nuts.07:17
lifelessspm: can you take it on, and hand it off to someone else when you EOD?07:18
lifelesswe need a build of 9765 to do all three CPs07:18
spmprobably not. Tom's not around. I'll do what I can, but no promises.07:18
lifelessand remove the thing, cowboy07:18
lifelessI'll check in in ~ 4 hours I guess.07:18
spmOh I think I may know what did that.07:18
stubSo does everything explode if people push --overwrite a branch that is linked to builds etc.?07:19
spmI used the 'restart with last failed build' juju in an attempt to encourge the pos to work. that'd do it.07:19
spmbuilds forced, see if that grabs the latest07:20
lifelessbbiab07:27
adeuringgood morning08:47
mrevellHey up everyone.08:49
jmlhello all10:34
lifelesshello10:35
jmllifeless, you broke prod_lp again :(10:37
lifelessjml: no, buildbot failed10:38
lifelessand failed again10:38
lifelessand failed afte that10:38
jml:(10:38
lifelessand after that10:38
lifelessso the changes, in production-devel, are still in production-devel and not in production-stable10:38
jameshyou'd get less buildbot failures if you had less tests10:41
lifelessjamesh: thanks for that.10:42
lifelessnight all11:11
bigjoolslifeless: wait!11:17
bigjoolsdarn too late probably.  jml, prod_devel is failing still11:17
bigjoolscan you help please?11:17
jmlbigjools, I can try. On the phone.11:17
bigjoolsme too :)11:18
lifeless/usr/bin/bzr checkout --revision 9762 bzr+ssh://bazaar.launchpad.net/~launchpad-pqm/launchpad/production-de11:18
lifelessits grabbing the wrong rev11:18
lifeless(you're lucky, I am looking up the phone number for tomorows meeting)11:18
lifelessthe branch is fine11:19
lifelessBB is confused11:19
lifelessdunno why11:19
lifelessI see rev 9765 in prod-devel11:19
lifelessok, water, phone, phone # all set.11:19
lifeless_> flip side11:19
jmlI'v queued another build11:23
bigjoolsjml: what do you mean?11:51
jmlbigjools, I went to "Force build" and forced a build.11:51
jmlbuilding11:52
jml1 pending11:52
bigjoolsjml: ok - do you know if that's any different to what happened at 07:43?11:52
bigjoolsthe overnight build failed too, and then someone kicked it off at that time11:52
bigjoolsI don't have much hope that it'll work this time :/11:52
jmlbigjools, no, I don't. but I'm going to try it in lieu of having a better idea.11:52
bigjoolsok11:53
bigjoolsI've got a rather urgent CP to get out :(11:53
jmlbigjools, ok. I'm off the phone. You have my full attention.12:04
bigjoolsjml: not much we can do until the current run finishes12:04
bigjoolswe can see if the next one checks out the right revno12:04
bigjoolsif not, then I'm getting on the batphone12:05
jmlI wonder where it gets the revno from12:05
bigjoolswhy does it even need to know12:06
jmlbigjools, I'm guessing something about race conditions.12:07
jmlbigjools, or perhaps some secondary approval step from the losas?12:07
jmlgood grief. every thing is failing for different reasons.12:09
bigjoolsew, it takes 45 minutes from buildbot instantiation until it actually starts running tests12:09
jmlyes.12:10
deryckMorning, all.12:15
bigjoolshi deryck12:15
=== mrevell is now known as mrevell-lunch
bigjoolsjml: it got the right revno this time12:28
bigjoolsonly 4 hours to wait now... :/12:28
=== al-maisan is now known as almaisan-away
=== matsubara-afk is now known as matsubara
jmlbigjools, yeah I see it :)13:20
wgrantLaunchpad's misrepresentation of partner makes me sad :(13:23
wgrantAnd makes RMS FUD harder to debate.13:23
=== mrevell-lunch is now known as mrevell
jmlwgrant, which aspect in particular13:25
wgrantjml: It says that Skype and Flash are in Ubuntu.13:25
wgrantWhich makes it difficult to tell RMS that he's wrong when he says they are :P13:25
beunooh, everything is difficult with RMS anyway...   :)13:26
wgrantIt is...13:26
jmlwgrant, I can see why that would be frustrating.13:28
jmlwgrant, otoh, I'm increasingly of the opinion that we should focus our efforts toward helping the people who are using Launchpad to try to get things done.13:29
wgrantProbably, yes.13:29
=== almaisan-away is now known as al-maisan
* gmb -> breaking to escape brain-stall. 14:25
deryckgmb, when you're back, could you look at bug 638284?14:36
_mup_Bug #638284: Launchpad's Remote Tracker does not honor Debian Bug Tracker status changes <Launchpad Bugs:New> <https://launchpad.net/bugs/638284>14:36
deryckI believe this is the debbugs sync disabled due to disk space issue, but would like you to confirm.14:37
gmbderyck, You're right.14:57
gmbderyck, Perhaps we should use that bug as a way of keeping users up-to-date with the problem. Thoughts?14:58
deryckgmb, yes, definitely.  If we don't already have a bug about it, then let's use that one.14:58
gmbderyck, I just looked and don't see one. I think that's mostly because we've been fielding Questions, IRC pings and emails about it but no bugs until now.14:58
gmbAnd I know that the LOSAs have an RT for the problem.14:59
deryckgmb, ok, cool.  Do you mind updating the bug and doing this?15:00
gmbderyck, Done.15:01
bacabentley, adeuring, bac, danilo, sinzui, deryck, EdwinGrubbs, flacoste, gary, gmb, henninge, jelmer, jtv, bigjools, leonard, mars, noodles775: reviewer meeting starting15:01
deryckawesome!15:01
jmlsinzui, where's the list of bugs remaining for the bridging-the-gap work that registry is finishing off?15:06
sinzuihttps://bugs.edge.launchpad.net/launchpad-registry/+bugs?field.tag=bridging-the-gap15:07
jmlsinzui, cunning. thanks.15:10
jmlsinzui, is there a bug filed for the bad tooltip text for the branch configuration link?15:11
sinzuiyes, your incomplete one15:11
sinzuijml, I cannot reproduce it. The description of what is in play (rendering engine + os) make it unlikely to be an issue with our code15:12
jmlsinzui, I don't mean the corruption thing, I mean the fact that the text says "Set branch for this series"15:12
sinzuioh, sorry15:13
sinzuilet me look.15:13
* jml is sorely tempted to move LEP mgmt to blueprints.15:14
bigjoolsI think that's a good idea15:15
bigjoolsit would give a needed thrust to improving blueprints15:16
sinzuijml, I tried that earlier this year, there is still a lot of blueprint update by hand because the statuses are more than any project needs and notifications are broken15:16
sinzuiYou cannot use feedback requests for example...you still need to send the feedback request yourself15:16
deryckjml, and then we should merge blueprints and bugs. ;)15:17
jmlderyck, *yes*15:17
sinzuijml: I do not see a bug for the tooltip. The bug is in an Involvement Menu subclass and is trivial15:17
sinzuideryck, and answers.15:17
jmlsinzui, ok. I'll file it anyway, if you don't mind.15:17
derycksinzui, woah there.  Not sure I'm feeling that one. ;) :)15:18
sinzuiplease do, my team are asking for some easy work since the app UI is very hard15:18
sinzuideryck, sure you do, less timeouts converting a bug to question.15:18
deryckheh15:18
deryckwell that is persuasive15:19
sinzuiNo more bug-question links and more portlet to show it15:19
deryckI just think we want to work at moving the support requests out of bugs, not making it easier to misunderstand the two.15:19
deryckgmb or allenap, bug 5786 is no longer relevant, correct?  NEEDSINFO is not referring to upstream status, but to an old bugtask state I take it?15:20
_mup_Bug #5786: NeedsInfo should be moved into a separate bug or bugtask flag <Launchpad Bugs:Triaged> <https://launchpad.net/bugs/5786>15:20
gmbderyck, I think so, yes. In which case it was superseded by Incomplete aeons ago.15:21
deryckright15:22
jmlsinzui, we talked about https://dev.launchpad.net/Registry/RegistryKarma a while back. What did we decide?15:28
sinzuiWe will do nothing but continue to hope for a replacement for karma15:28
jmlsinzui, excellent. I'll make a note.15:29
* sinzui has talked to 2 users this month to explain karma is not a measure of value or work15:30
jmlsinzui, what about this one? https://dev.launchpad.net/Registry/EssentialSourcePackageInformation15:32
sinzuiThat is done!15:33
jmlsinzui, ooh. I'll review then.15:35
jmlsinzui, the LEP says that https://edge.launchpad.net/ubuntu/+source/wsjt should have a link to copyright information. I can't see a link like that.15:36
sinzuijml, that page should not, the series source package does...they change by series15:37
sinzuijml: the SP descriptions will be used in the derivative distros diff page. the copyright is being used in register an upstream project15:38
jmlsinzui, sweet!15:40
deryckadeuring, hi.  I'm linking your current kanban board card against bug 633698.  This bug is good for this issue you're seeing?15:41
_mup_Bug #633698: missing API oops reports? <Launchpad Bugs:New> <https://launchpad.net/bugs/633698>15:41
* sinzui wonders if it is appropriate to quote his mistypes: "The involvement15:42
sinzuiportlet is controlled by the pillar owners and its links are enabled to15:42
sinzuiattack contributors to the applications the pillar uses."15:42
adeuringderyck: sure15:42
sinzuiI think I meant to type attract, but maybe subconsciously i did mean attack15:42
jmlheh heh15:43
deryckA little more clearly described now:  bug 63369815:43
_mup_Bug #633698: Librarian errors missing API oops reports <librarian> <Launchpad Bugs:In Progress by adeuring> <https://launchpad.net/bugs/633698>15:43
deryckIt seems people are either ignoring results suggested by dupe finder or it's become less useful with the recent preformance work.15:45
jmlbigjools, are package dependencies available over the API?15:54
bigjoolsjml: do you mean as properties of source packages?15:55
jmlbigjools, Yes.15:55
bigjoolsjml: no, we don't export those15:55
jmlbigjools, I mean, if they're available in another way that would be interesting too.15:55
jmlbigjools, any reason or just round tuits?15:55
bigjoolsjml: we don't export source packages at all, just their publications.15:56
jmlderyck, I think it has become less useful.15:56
bigjoolsderyck: I hardly ever see people paying attention to it15:56
jmlderyck, but a third hypothesis is more bugs filed via email.15:57
jmlbigjools, is there a reason for that?15:57
bigjoolsjml: yes, what is a source package's URL?15:57
jmlbigjools, https://launchpad.net/ubuntu/+source/packagename15:57
bigjoolsjml: wrong! that's a distrosourcepackage15:57
deryckyeah, I can tell the difference in bugs filed via email and against the project group.  I'm seeing it more commonly filed straight against malone, too.15:58
jmlbigjools, ubuntu/lucid/+source/packagename15:58
sinzuibigjools, jml, I want to toss a small grenade into your conversation. I do not think the DSP/SP pages and API is very good. Our pages have poor rank in Google. packages.ubuntu.com always wins. When a user is searching the Google package information, they are going to the other site, then following that link to Lp :(15:58
jmlsinzui, interesting.15:58
bigjoolsjml: unfortunately wrong again.  We need a way to put a sourcepackagerelease in its own URL.  But I don't think that's useful at all though.15:58
bigjoolsbear in mind that dependencies can change between versions15:59
jmlbigjools, something is using the word incorrectly here.15:59
jmlbigjools, https://edge.launchpad.net/ubuntu/maverick/+source/wsjt perhaps15:59
bigjoolsthis is why I added a bunch of source-related properties to publications15:59
jmlbigjools, since it does say "source package"15:59
bigjoolsyes, but it's not a source package release16:00
bigjoolsit's a source package in ubuntu maverick16:00
jmloh, you didn't ask for a URL for that :)16:00
* bigjools rolls eyes16:00
sinzuijml: bigjoolsI do not have any immediate ideas, I think this conversation is registry related too. SPR is a specific detail that users often do not care about, they want the current info for their distros or series. I think we need to re-ask who uses this information and what pages do we provide for them16:00
bigjoolsok ok :)16:00
jmlbigjools, my use case here is wanting to get build dependencies so I can start working on a branch.16:01
bigjoolssinzui: I think probably both are valid, which is another reason why I prefer adding more properties to publishing records16:01
bigjoolsbecause traversing to those from the SPR or the DSP is trivial16:01
jmlbigjools, so I need to be able to get from the branch's project to some kind of package record, and from there to the dependencies16:02
jmlbigjools, is this currently possible over the API?16:02
bigjoolsjml: it's not :(16:02
sinzuiI think users arrive at an SPR too early. Searching a series takes me to the SPR. If I bookmark that page, It could be wrong in 24 hours16:02
bigjoolssinzui: we don't have a page for an SPR16:03
jmlbigjools, what would we need to do to make it possible?16:03
=== deryck is now known as deryck[lunch]
sinzuibigjools, True16:03
bigjoolsjml: are you looking for a package in a distroseries in this project?16:03
jml(I don't really care about the web ui for this, fwiw, although I do acknowledge that it's interesting)16:03
bigjoolsor a particular version? or something else?16:03
jmlbigjools, I guess I don't care & am looking for the latest in Ubuntu, since that's likeliest to be close to trunk16:03
bigjoolsok16:04
bigjoolsjml: are you looking at build dependencies or installation dependencies?16:04
jmlbigjools, both. but for this use case, let's focus on build.16:04
bigjoolsthen I'd do this:16:04
bigjoolsadd the build deps as a method on sourcepackagepublishinghistory16:04
bigjoolsthen use distrosourcepackage.currentrelease or similar to get the publication16:05
bigjoolsbut you need to export all that on the API16:05
jmlbigjools, are DSP and SPPH already exported?16:06
bigjoolsactually, scratch that latter, use distribution.getCurrentSourceReleases()16:06
bigjoolswhich takes a string package name16:06
bigjoolsSPPH is exported16:06
bigjoolsdistro is exported16:06
jmlOK, so it's just a simple matter of programming then.16:07
bigjoolsyep, hopefully16:07
jmlbigjools, thanks16:09
bigjoolsjml: oh actually, feh, getCurrentSourceReleases() returns DSPs which are not exported.  You should use IArchive.getPublishedSources16:09
bigjoolswhich returns SPPHs16:09
jmlbigjools, ah. fair enough.16:09
bigjoolsgetting the archive is trivial16:10
bigjoolsdistro.main_archive for example16:10
jmlbigjools, from time to time the thought occurs to me, perhaps soyuz's apis could be simpler.16:10
bigjoolsjml: hellyes.  however, we're constrained by the decision to expose the internal model16:10
bigjoolsbut if you have any ideas on how to make it better, I am all ears16:11
jmlby "api" I meant internal model, actually.16:11
jmlno ideas right now :)16:11
jmlI'll let you know if any come up :)16:11
bigjoolsjml: I get frustrated with the model too.16:11
bigjoolsdo you remember my presentation at the original epic?16:11
bigjoolsand the db diagram?16:11
jmlbigjools, yeah, I do.16:13
bigjoolsjml: recipe builds made that twice as bad :)16:14
henningedanilos: ping16:22
daniloshenninge, hi16:29
=== benji is now known as benji-lunch
daniloshenninge, sorry, was otp16:29
henningenp16:29
henningedanilos:  what is the meaning of the "from_upstream" attribute of a queue entry nowadays?16:29
henningeused to be "is_published"16:30
daniloshenninge, ah, I think it needs to change (I know we changed it already)... it's not really from_upstream, I think it should be "from_package"16:30
daniloshenninge, though, then again, we can keep it as from_upstream to indicate what side  it's coming on16:31
henningedanilos: to determine if an import is from upstream, the potemplate can be queried for productseries/distroseries.16:32
henningethat would make from_upstream a computed attribute16:32
daniloshenninge, except in sourcepackage uploads, which are from_upstream, basically16:32
henninge???16:32
daniloshenninge, well, translations we import on ubuntu from source packages are upstream translations16:32
daniloshenninge, they are just from tarballs instead of from a VCS16:33
henningedanilos: I am not sure I understand16:33
=== benji-lunch is now known as benji
henningedanilos: will they be imported into upstream templates or ubuntu templates?16:33
daniloshenninge, uhm, with the new model, into both16:34
daniloshenninge, but they will be considered is_current_upstream16:34
henningedanilos: ah!16:34
henningedanilos: so this is another parameter to the "share with other side" policy16:35
daniloshenninge, well, kind of... the good end result will depend on that policy being right :)16:35
henningedanilos: "If the import is for a sourcepacakge but is marked as 'from_upstream', share it with upstream"16:35
daniloshenninge, no, "if import is for a sourcepackage but marked as 'from_upstream', consider it an 'upstream' translation"16:36
henningebut that is more difficult for the importer16:36
daniloshenninge, then, a policy of "if there's an upstream translation and it's better than ubuntu one, use that one for ubuntu as well" will give us the same result we get today16:36
daniloshenninge, how come? it just means go through the same code path as if it's an import for a productseries16:37
henningeyes but it needs to determine the productseries first.16:37
henningefrom the sourcepackage16:37
henningecode duplication16:37
daniloshenninge, it should be possible to avoid it by calling setCurrentTranslation(upstream=True) directly16:39
henningedanilos: My take: when sharing translations with the other side it does not matter on which side the translations are imported16:39
henningehm, maybe this is about template creation.16:40
daniloshenninge, well, I think we just need to call setCurrentTranslation with appropriate parameters, we don't need to know the exact template16:40
daniloshenninge, because we've got potmsgset already16:40
henningeon template import? not necessarily16:41
daniloshenninge, I mean, we've got it on "ubuntu" side, we don't need it on the upstream side as well16:41
daniloshenninge, by "got", I mean "just created or existing"16:42
daniloshenninge, we should not import templates from sourcepackages as "upstream"16:42
jmlbigjools, the stakeholder board says "generalized build histories" is done. is that accurate?16:43
daniloshenninge, basically, what will happen is that we'll internally have potmsgsets with upstream translations, but there may not even be any upstream templates in LP16:43
bigjoolsjml: yup16:43
henningeyes, I am beginning to see that16:43
jmlbigjools, cool! I'll review the LEP for sign-off & get back to you.16:43
bigjoolsjml: we're just waiting on jtv to finish off his work on actually using it, and then we can go on a made delete-fest of all the old code16:44
bigjoolss/made/mad/16:44
bigjoolsrecipes already use it16:44
jmlbigjools, yay deleting code16:45
bigjoolsit's a good feeling :)16:45
danilosbigjools, most of what we could do so far was in ec2 this morning ;)16:47
bigjools\o/16:48
jml"could do"?16:48
danilosjml, if I understood it correctly, some bits need to be moved to using the new model fully (we were blocking that), and we didn't do any of that16:49
jmldanilos, ahh ok.16:50
jmlI have a mild mistrust of anything that claims to be generalized and is only used for one type of thing.16:51
abentleyjml, +1.16:53
=== benji is now known as benji-lunch
=== deryck[lunch] is now known as deryck
abentleyDoes anyone know how to configure zope to treat the return value of itertools.groupby() the same way it treats generators?17:07
jmlabentley, I don't follow. Doesn't itertools.groupby() return an iterator?17:10
abentleyjml, yes it does, but the security proxy doesn't treat iterators the way it treats generators.17:11
jmlabentley, I'm surprised, and also desire to be crystal clear on terminology.17:11
jmlabentley, generator here means... ?17:12
abentleyjml, so if I return itertools.groupby(), I get a security proxy violation on __iter__.  If I return (x for x in itertools.groupby), everything is good.17:12
jmlbuggeration.17:12
* jml checks a thing17:13
abentleyjml, generators are iterators produced by functions with yield statements or generator expressions.17:13
abentleyiterators is a superset including generators as well as hand-crafted iterator classes.  itertools.groupby appears to return the latter.17:14
jmlabentley, ok. I think I see what's going on.17:14
jmlabentley, groupby returns an object that implements the iterator protocol17:14
=== al-maisan is now known as almaisan-away
abentleyjml, correct.17:15
jmlabentley, I'm guessing that Zope's handling for __iter__ and what-not is done in a type-based fashion.17:15
jmlabentley, and that the highly specific type returned by itertools.groupby() is not included in the list of types for which zope makes this exception.17:15
abentleyjml, me too.  I suppose they may have associated interfaces with the built-in types, though.17:15
jmlabentley, that's my guess17:16
jmlexcept they've missed the built-in type called itertools.groupby17:16
jmlI guess now it's time to explore zope.security17:16
abentleyjml, well, I was hoping someone would know off the top of their head.  But I don't see gary here today.17:17
=== salgado is now known as salgado-lunch
jmlabentley, something in zope.security.checkers._defaultCheckers17:25
jmlworking up from there17:26
jmlabentley, z.s.checkers.defineChecker(itertools.groupby, z.s.checkers.NamesChecker(['next', '__iter__'])) might do the trick17:27
jmlnot sure how to do that in zcml17:28
abentleyjml, thanks, I'll give that a shot.17:28
=== Ursinha is now known as Ursinha-lunch
=== matsubara is now known as matsubara-lunch
jmlabentley, I've just done a pass over the SPRB LEP and kicked it into "In progress"17:50
abentleyjml, does that reflect drafting or implementation?17:51
jmlabentley, implementation17:51
jmlabentley, I'm happy with the LEP if you are.17:52
jmlI haven't looked over the UI stuff.17:52
abentleyjml, I think it's okay.  What do you mean by Allow users to use James Westby's imports to provide the debian directory *without the user leaving the web UI*17:53
jmlabentley, you noticed. I mean that if the way to use the lp:ubuntu/foo imports involves switching to a terminal and making a new branch or something similar, then that's not really acceptable17:54
abentleyjml, I am surprised by the idea that using the lp:ubuntu/foo imports might involve switching to a terminal and making a new branch or something similar.17:55
abentleyjml, Wouldn't that defeat the purpose of using recipes?17:55
jmlabentley, pretty much :) perhaps the qualifier is redundant.17:56
abentleyjml, to me it suggests perhaps you want it to automatically recommend a packaging branch or something.17:56
abentleyjml, so that the user doesn't have to leave the recipe creation page to search for the correct branch.17:57
jmloh, I see.17:57
jmlwell that would be nice but certainly not critical.17:57
abentleyjml, what makes mark a stakeholder?17:59
jmlabentley, this is something he cares about a great deal. it's largely his vision.18:00
abentleyjml, does "Promptly provide full information on failed builds to someone capable of debugging the failure" mean it must spam me?  Or the packaging author?  Or the build requester?18:01
=== benji-lunch is now known as benji
abentleyjml, "Builds are performed in a timely manner" is something that the build farm does not guarantee for any builds.  Why do we have this special burden?18:02
jmlabentley, re timely18:02
jmlabentley, it's something that is necessary for me to consider the feature complete. it doesn't mean at all that you will be obliged to do the work to make it so.18:03
jmlabentley, indeed, that work is already been done by team soyuz.18:04
abentleyjml, it seems like a big scope creep to me.18:04
jmlabentley, not to me.18:04
jmlabentley, it's always been _daily_ builds18:04
rockstarjames_w, how do you usually run the bzr-builder tests?18:05
james_wrockstar: bzr selftest -s bp.builder18:05
abentleyjml, It has not been done.  In fact, I was supposed to work on a new build farm, and AIUI that task is now on hold.18:05
jmlabentley, sorry, I meant "being"18:05
rockstarjames_w, ah, didn't know about -s18:05
abentleyjml, I'm not aware of any Soyuz plans to change the basic scheduling algorithms, but until that's done, we'll have the potential for build starvation.18:07
bigjoolsit's an extremely hard problem to solve, but I've not forgotten it18:07
jmlabentley, so we'll have to solve it then.18:07
* jml seizes this opportunity to order a book on queuing theory18:08
abentleybigjools, but until someone solves it, jml won't consider the daily builds LEP complete.18:08
bigjoolsthat's crazy18:08
jmlbigjools, what can I say, I'm loco.18:09
bigjoolsrecipe builds are now scored the same as PPA jobs, they won't starve18:09
abentleybigjools, if we lose some of our loaded builders and a bunch of higher-scored builds get created, they can.18:10
jml(£80 is not the price of a book!)18:10
abentleys/loaded/loaned18:10
bigjoolsabentley: in theory yes, but there's only one thing that generates higher-scored builds and they are not frequent, so in practice it does not happen18:11
abentleybigjools, aren't there a bunch of modifiers that can push up the score?  Even a bunch of builds with a 2010 score can starve the daily builds.18:12
abentleyOr was that 2510?18:13
jmlabentley, wrt "provide information on failed builds", out of those, I mean the build requester18:15
jmlabentley, errors in the system are already (I hope!) handle by OOPSes18:16
jmlhandle*d*18:16
jmlabentley, or the recipe author, I guess...18:17
abentleyjml, I believe the build requester is required to be a member of the recipe author.18:17
jmlabentley, that's good. even so, maybe the broader group is more appropriate. I don't know right now.18:18
abentleyjml, we don't distinguish between errors in the build that are system errors vs user errors.18:18
abentleyjml, rather like code imports.18:19
jmlabentley, by design?18:19
abentleyjml, kinda.  Our builds are just like other builds in this regard.18:20
jmlit seems that, for example, any exception that is raised outside of slave is guaranteed to not be a user error.18:20
abentleyjml, sure.18:20
* jml is having a bad day with grammar.18:20
abentleyjml, that's why I said errors in the build.18:20
=== Ursinha-lunch is now known as Ursinha
jmlabentley, what happens with errors in the build now?18:21
abentleyOnce the build is completed and the slave is no longer responsible, errors can be handled as user errors or oopses.18:21
abentleyjml, we send an email to the user who requested the build.18:22
jmlabentley, so how do we learn of failures in the system?18:22
abentleyjml, like with code imports, we rely on users to escalate system errors as bugs.18:23
jmlI could have sworn we had oopses for code imports.18:23
abentleyjml, maybe we do.18:24
=== salgado-lunch is now known as salgado
jmlabentley, ok.18:25
jmlabentley, in any case, I'll leave banging on about architectural transparency to lifeless. I mostly care that users who have a build that screws up find out about it and are given enough data to try to fix it.18:26
abentleyjml, anyhow, how can we in principle know whether an error is due to user error or system error?18:26
jmlabentley, with the puller we have some heuristics. but that approach is probably not relevant here.18:26
abentleyjml, can I substitute "the person who requested the build" for "someone capable of debugging the failure"?18:29
jmlabentley, sure.18:31
abentleyjml, I'm also adding "Renaming branches does not break recipes" under "nice to have".18:34
jmlabentley, good call.18:34
jmlabentley, Launchpad more broadly needs to get its act together wrt renaming stuff.18:34
abentleyjml, (we kind of assumed that was important, so branches are stored as DB ids)18:35
abentleyjml, the relevant iterator from my zope security question is itertools._grouper, and it comes from a C module and is not exposed.18:36
jmlyou mean type(itertools.groupby([1, 2, 3]).__iter__()) == itertools._grouper?18:37
jmlabentley, any more questions about the LEP?18:39
abentleyjam, I mean type(list(itertools.groupby([1, 2, 3]).__iter__())[0][1]) == itertools._grouper18:39
jmlahhh.18:39
jamabentley: I think you mean jml :)18:40
abentleyjml, should there be any requirements about bzr branch format support?18:40
jmlPython sucks.18:40
abentleyjam, sorry.18:40
jamnp, I just showed up18:40
jmlabentley, I don't care about anything that's not 2a.18:40
jmlabentley, is that what you mean?18:40
abentleyjml, many of our supported distros don't support 2a.18:40
jmlabentley, why does that matter?18:41
jmlabentley, because the people using those distros won't be able to push up compliant branches? or because we run the bzr-builder recipe in an image based on the distro it's targeted for?18:42
abentleyjml, do we want to be able to build bzr recipes for old distros if said distros don't support 2a but the relevant branches are in 2a?18:42
jmlabentley, yes, I think so.18:42
abentleyjml, our initial implementation did not support that, so it's easy to imagine that an implementation would not support that unless we make it a requirement.18:43
jmlabentley, for sure.18:44
jml"branch formats usable within recipes is independent of branch formats supported by target distro" or something.18:44
abentleyjml, didn't see you writing.  I put "Allow recipes to be built from bzr 2a-format branches for any supported distorseries (whether or not the distroseries itself supports 2a). "18:46
=== matsubara-lunch is now known as matsubara
jmlfair enough18:46
jmlabentley, if bzr gets another format between now and the LEP being finished we can adjust accordingly :)18:47
abentleyjml, if you keep the timeliness requirement, it probably will :-P18:47
jmlhah18:47
abentleyjml, anyhow, aside from the timeliness requirement, I'm happy with the LEP as it stands.18:49
jmlabentley, cool.18:50
lifelessmoin18:51
abentleyjml, do we have a graph of how long it takes SRPBs to get dispatched?  I know we have one for build queue size.18:52
abentleyah, lifeless is around.  Must be lunchtime.18:52
jmlabentley, I don't know.  I don't think so.18:52
abentleyjml, could you set that up, or should I ask thumperfrancis?18:53
jmlabentley, ask flumper.18:53
jmlabentley, I just recalled a use case that isn't very well documented18:53
jmlabentley, one big use case for recipes is having a daily build of trunk without debian/ubuntu patches.18:54
jmlor with as few as possible.18:54
jmlabentley, I'm not sure how this translates into requirements.18:55
abentleyjml, I don't know, either, because some patches may be necessary to build at all.18:55
abentleyOne could perhaps fork the packaging branch, remove unwanted changes.18:56
jmlabentley, or make a completely new packaging branch18:57
abentleyThen either use that in the recipe, or use the recipe to merge the packaging branch.18:57
jmlabentley, and express the ubuntu one as a recipe on top of that.18:57
abentleyjml, you can also use nest-part to exclude changes outside the debian directory.18:57
rockstarmars, ping18:58
jmlabentley, the difference between the "pure" packaging and the ubuntu packaging is primarily of interest for testing upstream code.18:58
abentleyjml, but of course, some patches will be provided within the debian directory.18:58
abentleyjml, it also provides a vector for upstreams who feel that ubuntu's changes are unwelcome to get a pristine version to their users.18:59
jmlabentley, hello.19:00
abentleyjml, hi.19:00
jmlabentley, sorry, mischan19:00
jmlabentley, yes. that's also a good case.19:01
jmlabentley, perhaps this means that we should have some kind of special marking for "pure" packaging...19:02
jmlsomething to chew on.19:02
* jml otp19:02
abentleyIf you look at bzr, the ubuntu version removes configobj from the tree and depends on its package instead.19:02
abentleya "pristine" bzr would have configobj in the tree, and would therefore have an unnecessary dependency.19:03
abentleyUnless you change the bzr packaging manually.19:03
lifelessderyck: can we have a brief call after this ?19:09
derycklifeless, I have a call with thumper after this, but depending on how long it lasts, after that would be fine.19:10
jkakarDoes the LP schema allow a project to be in more than one project group... I know the UI doesn't allow it... but is it possible?19:11
lifelessderyck: great19:12
lifelessjkakar: iirc no there is a unique constraint19:12
jkakarlifeless: Cool, thanks.  It wonder if enough people would benefit from projects being able to be in more than one group to make it worth changing.19:13
lifelessjkakar: I want to eliminate pgs as a special case19:13
lifelessuse multiple tagging/categorising systems19:14
lifelesse.g. tags to to bottom up associations19:14
jkakarlifeless: That sounds cool.19:14
lifelessand 'defined groups' to do top down defined structures19:14
jkakarlifeless: What I want is a way to group projects such that I can have a +activereviews page that shows me reviews for the group.19:14
lifelessand permit both to exist19:14
jkakarlifeless: Also, to see bugs in the group.19:14
jkakarlifeless: Also, to create milestones for the group.19:15
jkakarSo I guess I want the group to be identical to a project, but with less arbitrary limits on how I can form them. :)19:15
lifelessyes19:15
lifelessI would like the same too, in the fullness of time.19:15
jkakarlifeless: Cool!19:15
* jml gone19:42
marssinzui, do you think a wiki page would be a better way than the ML to track Maverick upgrade issues?19:43
lifelesssinzui: hi https://bugs.edge.launchpad.net/launchpad-registry/+bug/63892419:43
_mup_Bug #638924: Milestone:+index timeouts with many announcements <timeout> <Launchpad Registry:Triaged> <https://launchpad.net/bugs/638924>19:43
lifelesssinzui: I've just dug a bit deeper into that timeout19:44
lifelesssinzui: I drew some different conclusions about the issue, and thought perhaps you'd like to spend a little time later discussing it19:44
sinzuilifeless, I would like to discuss it. I have two more meetings today, but I am free this evening and tomorrow19:47
lifelesssinzui: ping me anytime19:47
sinzuifab19:47
lifelesssinzui: I'd like to get some hints out there for doing the analysis stuff19:47
lifelessUrsinha: https://devpad.canonical.com/~lpqateam/qa_reports/launchpad-stable-deployment.txt seems to have not updated ?19:48
lifelessUrsinha: also, as that rev *is* deployed, I think you're pointing it at the wrong branch ?19:49
Ursinhalifeless, wait, I'm checking19:50
lifeless(its meant to start at the deployed rev, not randomly far back)19:50
Ursinhait seems it was updated few minutes ago19:50
marslifeless, Ursinha, the ctime on those reports says they are only 3 minutes old19:50
lifelessso, there is a bug then19:50
lifelessbug 1151819:50
_mup_Bug #11518: Linux kernel-image-2.6.9-1-686 synaptic pass through problem hoary <Ubuntu:Invalid by fabbione> <https://launchpad.net/bugs/11518>19:50
lifelessbah19:50
lifelessrev 11518 has bug 62774119:50
_mup_Bug #627741: oops-prune is aborting with a regex mismatch(?) <canonical-losa-lp> <qa-untestable> <Launchpad Foundations:Fix Committed by stub> <https://launchpad.net/bugs/627741>19:50
lifelessthats qa-untestable19:50
lifelessthats one issue19:51
lifeless the second issue is that that rev is *already deployed*19:51
lifelessit should be starting much higher up19:51
lifelesswhat branches are you giving it ?19:51
Ursinhalifeless, it seems last deployed revision on lpnet was 1151519:57
Ursinhascript starts processing at 1151619:58
lifelesshang on though19:58
Ursinhawhich is correct19:58
lifeless'stable' is deployed to edge19:58
Ursinhanow for the other problem19:58
lifelessdb-stable is deployed to lpnet19:58
lifelessactually thats a convenient lie but its good enough for now.19:58
lifeless(because we need something to measure which db commits can be deployed)19:59
lifelessUrsinha: what branches is it running with please ?19:59
Ursinhalifeless, stable and db-stable20:02
lifelessUrsinha: it takes branch pairs IIRC20:02
lifelessUrsinha: source and target20:02
UrsinhaMergeDeployment is lpnet20:02
Ursinhadevel, stable and db-devel, db-stable20:03
Ursinhalifeless, merge environment for both stable and db-stable is lpnet20:03
Ursinhaso it looks lpnet for the last merged branch20:03
lifelessok, that will make sense once we remove edge20:03
lifelessfor now though, the devel/stable one needs to check against 'stable'20:04
lifelessUrsinha: ah yes, I see20:05
Ursinhalifeless, let me see if I got the picture: people land code on devel/db-devel, and it goes do edge/staging, they QA it,  and it supposedly should go live on lpnet20:05
Ursinharight?20:05
lifelessand I didn't provide a knob for this.20:05
lifelessignore db- for this discussion20:05
lifelessUrsinha: actually, what I'll do is ask for a CP of everything.20:06
lifelessthat will sort this out.20:06
Ursinhanot sure what you mean20:06
lifelessbut it would be nice to know that everything currently in stable has been QA'd, to pick a rev to CP20:07
lifelessUrsinha: well a CP is a merge to production-devel20:07
Ursinhalifeless, I thought this was exactly how the script is working now20:07
lifelessahh,, and I'll be able to do that if we figure out this other isue.20:08
Ursinhawhat's on stable is what should be QAed20:08
lifelessUrsinha: ok, yes, I'm being noddy.20:08
lifelessUrsinha: so lets talk about the other thing :) [sorry!]20:08
Ursinhahehe20:08
UrsinhaI'm investigating the bug thing20:08
lifelessok20:08
=== almaisan-away is now known as al-maisan
marsrockstar, I have a reply on the way to you, but I need to answer a question about the JS build system first20:17
rockstarmars, great, thanks.20:21
rockstarmars, I'd like to deal with this today, as it means that I have a kanban card that's just sitting.20:21
Ursinhalifeless, ok, so that was a stupid thing: qa-needstesting wasn't considered a deployable status, only qa-ok20:21
* Ursinha goes add tests20:21
lifelessUrsinha: qa-untestable do you mean ?20:22
Ursinhalifeless, argh, yes, sorry20:22
lifelessqa-needstesting isn't deployable ;)20:22
Ursinhalifeless, qa-untestable20:22
Ursinhalifeless, https://devpad.canonical.com/~lpqateam/qa_reports/launchpad-stable-deployment.txt20:22
Ursinhait didn't go too far though20:22
lifelessthats ok20:23
lifelesswe can look into that in a minute20:23
rockstarARGH!  Stabby stabby moin!20:31
rockstarthumper, when you're around, I'd like to chat about merge queues.20:47
=== al-maisan is now known as almaisan-away
deryckLater on, everyone....21:04
lifelessrockstar: we hsould just use em.21:04
lifelessif bugs turn up, we can fix.21:05
rockstarlifeless, :)21:05
rockstarlifeless, there are some things about the current model that I think complicated things greatly.21:05
marsrockstar, understook21:05
marsunderstood even21:05
lifelessrockstar: not disagreeing21:05
lifelessrockstar: but its inventory today.21:05
rockstarmars, wtf keyboard layout are you using that had d and k right to them?  :)21:05
rockstarlifeless, yeah, inventory that I want to throw out on the streets!  :)21:06
rockstarlifeless, you will see, though, that we have a whole lane in kanban for finishing up merge queues now.21:11
jamrockstar: ping about bug #61275421:13
_mup_Bug #612754: Submit Request Failure: Signature couldn't be verified: (7, 8, u'Bad signature') - with email signed and sent from sup-mail <Launchpad Bazaar Integration:Incomplete> <https://launchpad.net/bugs/612754>21:13
jamIt is blocking me pretty much on every submission21:13
lifelessrockstar: a lane? wouldn't it just fit under features?21:13
lifelessrockstar: anyhow, details aside...21:13
lifelessrockstar: I saw your update; why one queue per project? isn't one queue per series easier conceptually?21:14
* benji goes afk for a bit.21:22
rockstarlifeless, one queue per branch, but many branches can be in one queue.21:29
rockstarlifeless, we can't just draw the line at series or anything.  Think about all the branches currently managed by ~launchpad-pqm21:29
rockstarjam, EVERY submission?21:29
jamrockstar: any time I send a "merge: approve"21:31
jamand a lot of bug reports, too, I believe21:31
rockstarjam, yeah, it would use the same mechanism.  I think might fall under "foundations" domain, but not sure.21:32
rockstarjam, could you try with a different mail client?21:32
jamI can obviously go to the website, but the email interface is broken for me21:32
thumperjam: which email client?21:32
rockstarjam, mathiaz was having intermittent problems, so I was suspecting that it was a mail client issue.21:32
jamrockstar: I don't have another mail client integrated with gpg. But even so, I can check the raw texts pass, and I had Vincent do the same thing for me21:32
thumperrockstar: I'm nomming21:33
thumperrockstar: and I'd like to grab abentley first21:33
jamthumper: thunderbird 3 w/ enigmail21:33
rockstarjam, is Vincent using the same client?21:33
jamrockstar: vincent uses gnus, or whatever the emacs client is21:33
rockstarthumper, we have our 1-1 today, right?21:33
rockstarjam, *sigh*21:33
abentleythumper, certainly.21:33
thumperrockstar: I guess21:33
jamrockstar: you should be able to just try "gpg --verify submit_request_failure.txt" that I attached to the email21:33
rockstarjam, okay.21:34
jamrockstar: I just sent another email (which I expect to be blocked). I suppose I can CC/BCC you one of them?21:36
rockstarjam, absolutely.21:36
jam(I just BCC'd you on an artificial one, though I think that request is actually bogus, as you can't set WIP from email)21:37
jambut it should still gpg verify21:37
rockstarjam, so I've verified that your message was good.21:38
lifelessrockstar: sure, s/series/target branch/21:41
lifelessrockstar: what I mean is that we wouldn't want db-devel and devel to both use the same queue - they would starve each other21:41
jamrockstar: well, at least you and I agree. Now if we can just get LP to feel the same :)21:42
jamrockstar: I haven't see the messages show up on 'review' but I also haven't gotten the "Submit Request Failure" yet.21:42
rockstarlifeless, I disagree that they'd starve eachother.  We're essentially doing the same thing now with pqm.21:48
lifelessrockstar: pqm isn't running the tests21:49
lifelessrockstar: and pqm did starve each other21:49
lifelessrockstar: huge issues with that21:49
rockstarlifeless, tarmac, in this case, wouldn't run the tests either.21:49
lifelessrockstar: we want to make the lander run the tests again21:50
lifelessrockstar: thats gary's experiment, landing in batches of 521:50
rockstarlifeless, and we can do something like that.  But at that point, we'd have to separate out what branches are on what queue.21:50
lifelessis it just me22:07
lifelessor is22:07
lifelessFAILURE: lp.services.job.tests.test_runner.TestJobRunner.test_oops_messages_used_when_handling (subunit.RemotedTestCase)22:07
lifelessfailing for every branch ?22:07
lifelesson ec222:08
jelmerI had an error like that as well22:10
jelmerI figured since the error in the oops was soyuz related it was somehow my fault22:10
wallyworld_lifeless: i got the same error too when trying to land a branch22:11
lifelessok22:12
lifelessits probably not unrelated to my oops work22:13
lifelessrunning just that test works22:14
wallyworld_don't you hate it when that happens :-(22:14
lifelessso either oops-writing is naffed22:15
lifelessor get last oops is22:15
lifelessmars: still here?22:17
wallyworld_thumper: rockstar: abentley: skype?22:18
rockstarwallyworld_, sure.22:18
thumperrockstar, wallyworld_: abentley and I are chatting22:18
thumpergive us a few minutes to finish what we are talking about plz22:18
wallyworld_thumper: np22:19
mwhudsonmorning22:19
* jelmer waves22:19
lifelessI'm looking in uniquefileallocator if folk are interested22:21
marslifeless, yep22:22
marslifeless, what's up?22:22
lifelessmars: how clean is the oops dir in an ec2 test run22:22
lifelessmars: see the list for the context22:22
marslifeless, no idea. someone could run with ec2 test with --postmortem and find out though.22:23
lifelesswe really should change the filenames on disk to be less magical22:28
lifelessand for fun22:39
lifelessrunning the two relevant tests together, here, works.22:39
lifeless:!bin/test -vvt test_uploadprocessor.TestUploadProcessor.testOopsCreation -t lp.services.job.tests.test_runner.TestJobRunner.test_oops_messages_used_when_handling22:40
lifelessmars: a premortem for ec2 would be nice22:41
lifelessmars: e.g. set it all up, but run nothing22:41
james_wyou can do it with "-d" and some stepping22:43
james_wI thought jml had implemented --dont-test or something22:43
thumperjelmer: any idea why https://code.edge.launchpad.net/~vcs-imports/mesa/master is failing?22:43
=== matsubara is now known as matsubara-afk
lifelesshah, I just noticed my lp vm is in sydney time.. that explains some confusion I had22:48
jelmerthumper: looking22:49
marslifeless, jml was implementing that option.  Otherwise you can do it with "ec2 test -p -o '-t asdf'"22:49
lifelessthis really has me flummoxed22:50
jelmerthumper: Looking at the history, it seems the branch was reimported from scratch recently?22:50
thumperjelmer: yes, yesterday22:50
lifelessi have tried deleting some oopses22:50
lifelessand it correctly ignores the holes and writes to the end and finds them again22:50
jelmerthumper: So I guess this can't really be a bzr-git bug that caused incorrect repo data.. I'll try locally & file a bzr-git bug.22:51
jelmerthumper: From just looking at the logs I don't really have an idea what could be going wrong.22:51
=== Ursinha is now known as Ursinha-afk
lifelessI've put a branch up that may help23:17
poolielifeless: so, about sending output from process-mail.py to a log file23:42
lifelesstheres an option to do that already apparently23:43
lifeless--log-file or something23:43
pooliemm, apparently it's not used23:43
lifelessshould just need an RT (and for someone to be monitoring the log)23:43
pooliethey just manually redirect it in the cron script23:43
lifelessyeah23:43
pooliei did look at it a few weeks ago and ISTR that --log-file may be broken or something23:44
poolieimbw23:44
pooliebut i think i said "how does it even work?" and spm said they don't use it :/23:44
lifelessits a thing that was built and not deployed23:44
lifelesswe should use it23:44
pooliealso, as i recall, someone said "we can't just send it to a file because we need to know about errors"23:45
pooliemaybe this was you?23:45
lifelesswe do need a story around that23:45
pooliebut i don't see how sending it to a file is much worse23:45
lifelessideally --log-file would not hide ERROR severity messages23:45
lifelessif you wanted to pull on this yak I think that would be great.23:45
poolie'i am in yaks stepped in so far that should i wade no more, returning were as tedious as to go over'23:46
poolieso i propose23:47
poolie- for now, just sending an rt asking them to send it to a file23:47
poolieif they object, we'll deal with that, but i don't see how it should make things any worse23:47
poolie- filing a bug about splitting >=error to stderr and everything to a file23:48
poolie(which will necessitate actually using --log-file not just >&file)23:48
lifelessseems fine to me if you add in23:48
poolieand i may work on that one23:48
lifeless - email lp-dev letting folk know that errors from it may not show in the -errors-report anymore23:48
pooliethen using the results of #1 to work out what's up with dkim23:48
poolieso i can say the rt has your approval?23:49
lifelesssure; cc me?23:49
pooliefor the sake of parallelizing, is the errors mailbox anywhere i could see it?23:51
lifelesslp-error-reports list should have it, yes23:52
poolieso i'd need to join that list, disable delivery, then look in the archives?23:53
lifelessyeah, I think so.23:53
pooliesome devops people would say that sending mail even in the case of errors is not the best idea23:54
poolieif things are falling over, mailservers may be swamped or unreliabel, etc23:54
lifelessmmm23:55
lifelessthat applies to all automation23:55
lifelessemail is one of most robust systems we have : more robust that being able to ssh into the problem box.23:55
poolieit's a cheap but inefficient way to broadcast things23:56
lifelesssure23:57
lifelesswe don't have better yet.23:57
lifelessparticularly for errors.23:57
poolieso is it reasonable to send everything, including errors, to a file for now?23:59
lifelesscan we do voice? I'm having deja vu and perhaps more bandwidth will address your questions quicker.23:59

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!