lifelesswgrant: blacklist +100:00
wgrantlifeless: But that means having the list of exclusions in a separate tree.00:00
lifelesswgrant: we've needed one exclusion in 6 years00:00
lifelesswgrant: I think we can deal00:01
wgrantAh, allenap beat me to a bulk loading helper.00:03
* lifeless isn't sure its the right way00:03
wgrantI'm not sure it is either.00:04
lifelesswe want, for instance, to eager load visibility rules etc00:04
lifelessso I think it needs per-type handling rather than a separate helper00:05
lifelesshowever, allenap has written something that will be usable for somethings00:05
wgrantRight, that's what I was thinking. It is a pattern that we already use in several places.00:05
wgrantAnd this helper can do nice stuff like not issuing a query if everything is already preloaded.00:06
lifelessit introspects the storm cache?00:06
wgrantSince that is going to become a common case, I think.00:06
wgrant*can do*00:06
wgrantNot does now.00:06
thumperleonardr: I don't suppose you are still around?00:07
thumperleonardr: I have a lazr-restful problem00:07
leonardrthumper: i'm here00:07
thumperah, awesome00:07
thumperleonardr: let me pastebin something...00:07
wgrantlifeless: https://code.launchpad.net/~wgrant/launchpad/exclude-lpnet-template/+merge/5249700:08
wgrantlifeless: You're landing lamont's thing and the staging fix?00:10
lifelessthats how I found this, yes.00:11
lifelessI'm not sure what branch the check checks out00:11
thumperleonardr: http://pastebin.ubuntu.com/577252/00:11
wgrant./launchpad                             bzr+ssh://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable00:11
thumperleonardr: getting TypeError: Only a read-only field can have a mutator method.00:11
wgrantlifeless: But you can fix that in the branch :P00:12
lifelesswgrant: meh, will wait for bb00:12
thumperleonardr: wanting the accessor and mutator only in "devel"00:12
lifelessstable is not unreasonable00:12
thumperleonardr: any ideas?00:13
leonardrthumper: have you tried putting @operation_for_version first?00:17
thumperleonardr: first as in at the top, or bottom?00:17
leonardrat the bottom00:17
* thumper tries00:17
thumpernope, that doesn't fix it00:18
leonardrright now the mutator is exported in beta nad the field is not exported at all in beta00:18
leonardrsame error?00:18
leonardrthumper: the field itself needs to be read-only. it can't just be exported as read-only00:19
leonardrsee the mutator_for method in lazr.restful declarations.py00:19
leonardrthat's what it says00:19
leonardr        if not self.field.readonly:00:19
leonardr            raise TypeError("Only a read-only field can have a mutator "00:19
leonardr                            "method.")00:19
thumperwe should fix that00:19
wgrantWhy can't you make the field read-only? It only affects forms and the webservice.00:20
lifelesssinzui: are you going to land the new lazr restful egg?00:20
thumperwgrant: because the forms edit it00:20
thumperwgrant: I'd have to override the field...00:21
thumperwgrant: and it changes the permissions00:21
thumperwgrant: if the field is readonly, the permissions don't allow setting00:21
wgrantIt doesn't affect security proxies.00:21
leonardrlifeless: i think it would be okay to let mutator_for succeed if the field is never *published* as read-write in any version for which it has a mutator00:21
wgrantHmm, unless maybe it affects set_schema, but not much uses that.00:22
* leonardr has to go00:22
thumperwgrant: I think it does...00:22
thumperleonardr: ok00:22
wgrantthumper: Anyway, shouldn't the form be using the mutator anyway?00:22
thumperwgrant: it means more hoop jumping to extract the field from the data dictionary, getting the delta object first, raising the object modified event manually00:23
thumperman I wish our form infrastructure used the API :)00:23
* thumper has a plan00:25
thumperohh... tentative success00:27
thumper:( now getting  You tried to modify a read-only attribute. in the 400 response00:35
wgrantcheckwatches OOPSes and bye-bye-nullbugtask today, I think.00:41
lifelesswgrant: but its perf tuesday!00:43
wgrantlifeless: Killing NullBugTask gets rid of some timeouts!00:43
wgrantBut OK.00:43
wgrantMaybe getBuildRecords.00:44
wgrantI also need to attack +copy-packages again.. the copying is now fast, but the rest of the page is not :/00:44
lifelesswgrant: its a never ending story00:45
leonardrthumper: it sounds like the lookup of the mutator operation didn't succeed. you could try putting a breakpoint in generate_entry_adapters() (lazr.restful declarations.py)01:00
leonardrit should be creating a PropertyWithMutator for the field in the appropriate version01:00
thumperi'll try that01:01
* thumper sighs01:02
thumperevery freaking entry01:02
thumperlots of continues, and found the right interface01:04
lifelesshuwshimi: Bug 73053501:16
_mup_Bug #730535: Enter key ignored the first time in bug tag editing field <bugtag> <javascript> <Launchpad itself:Triaged> < https://launchpad.net/bugs/730535 >01:17
lifelesshuwshimi: is that a regression from your changes?01:17
huwshimilifeless: it looks like the same issue. I believe my changes won't be rolled out until lazr-js is updated in LP.01:18
lifelesshuwshimi: ah! then your bug should not be closed01:19
lifelesshuwshimi: and this new bug should be a dupe.01:20
huwshimilifeless: yeah01:20
huwshimilifeless: I think it was marked as closed during a deployment or something01:20
* thumper sighs01:22
lifelesshuwshimi: do you know how to move forward on this?01:22
huwshimilifeless: Nope :)01:22
lifelessfirst things first, lets get the bugs sorted out01:23
lifelessfind your old one01:23
lifelessset to in progress01:23
lifelessdupe the new onto the old01:23
lifelessthe old one should have two tasks, one on lazr-js and one on launchpad,t he lazr-js one should be fix released if your change is merged into it.01:23
lifelessI'm moderately sure deryck finished upgrading us to the latest lazr-js recently so your fix should be easy to get to.01:24
wgrantAh, sorry, that was probably my fault. I thought I set it back, though :/01:25
huwshimilifeless: The old one (bug #580404) doe not have a lazr-js task, I assume I can just add that?01:32
_mup_Bug #580404: pressing enter in tags field doesn't save them <ajax> <javascript> <lp-bugs> <qa-needstesting> <Launchpad itself:In Progress by huwshimi> < https://launchpad.net/bugs/580404 >01:32
lifelesshuwshimi: yes01:33
lifelesshuwshimi: you made the change in lazr-js right ?01:33
huwshimilifeless: ugh, sorry I'm getting random lockups with my keyboard and mouse (but I can see that the OS is still working)01:37
huwshimilifeless: Yes the changes were made to lazr-js01:37
LPCIBotYippie, build fixed!01:38
LPCIBotProject db-devel build #427: FIXED in 5 hr 14 min: https://hudson.wedontsleep.org/job/db-devel/427/01:38
huwshimilifeless: ok, I've done all that. What next?01:42
lifelessnext, you need to figure out whats new in lazr-js leading up to the commit where your patch landed, so you can decide how much qa you want to do01:42
lifelessand/or whether you want to land everything or just up to your patch01:43
lifelessuse bzr and or bazaar.launchpad.net to look at the history and decide01:43
lifelesspick the rev of lazr-js you want to update to01:43
lifelessand then we go to the actually-doing-it step01:43
huwshimilifeless: ok thanks will take a look01:44
huwshimilifeless: How do I know what hasn't been landed?01:46
huwshimilifeless: Or rather released01:47
lifelesslook at versions.cfg, see the version there, look at the branch for lazr-js, walk back until the release01:47
huwshimilifeless: I'm not entirely sure what you mean by that, there are a lot of version numbers in that file and none seem to refer to lazr-js? I may be completely mis-understanding you01:51
lifelesslazr-js = 1.6DEV-r20201:52
wgrantOOPS reports during release week are depressing.01:53
lifelesswgrant: because deploys stop ?01:53
lifeless14:51 < huwshimi> lifeless: I'm not entirely sure what you mean by that, there are a lot of version numbers in that file and none seem to refer to lazr-js? I may be completely mis-understanding you01:54
lifeless14:51 < lifeless> lazr-js = 1.6DEV-r20201:54
huwshimilifeless: Sorry crashed again01:54
wgrantlifeless: Yes.01:54
huwshimilifeless: Ah right, you're referring to the versings.cfg in Launchpad, not the one it lazr-js :)01:55
huwshimilifeless: ok, so there is one other commit since the last release01:56
wgrant * 1954 Exceptions01:57
huwshimilifeless: The other commit does not have an attached bug, is it still possible for me to be able to qa it somehow?02:00
lifelesshuwshimi: of course02:01
lifelessbugs don't permit qa, they are just about recording stuff02:01
lifelesshuwshimi: make an assessment of the other change02:02
huwshimilifeless: It's a module that's been updated (looks like a file has been replaced with a whole new version).02:05
huwshimiok next time this crashes I'm giving up and going back to my old laptop02:17
lifelesshuwshimi: what is it?02:17
huwshimilifeless: the laptop?02:17
lifelesshuwshimi: so you need to decide if its safe to include in lp as-is02:17
lifelessyes the laptop02:17
huwshimiit's a 2011 Macbook Pro.02:17
Ursinhalifeless, and now?02:18
UrsinhaI forgot to change a link02:18
lifelessUrsinha: brilliant02:18
huwshimiok that time it was Natty crashing. Laptop, I'm giving you one more chance02:20
huwshimilifeless: from a quick grep I can't find any references to that lazr-js module.02:29
lifelesswhere ? ;)02:29
huwshimilifeless: in the launchpad codebase02:30
huwshimilifeless: I could be wrong though :)02:30
lifelesshuwshimi: so if we're not using galery-form, its a no brainer02:32
lifelesshuwshimi: I think its on the wiki somewhere, but what you need to do next is make a new sdist, put it in the lp download cache (copy it in, bzr add, bzr commit), update versions.cfg in lp and propose that as a branch to land in devel02:32
huwshimilifeless: Where do we define which javascript files we include?02:33
* thumper leaps through a few more hoops02:37
lifelesshuwshimi: I don't know02:38
huwshimilifeless: So I'm wrong about us not including that file02:59
lifelesshuwshimi: in which case, check that lp will still work with the new version :)03:02
huwshimilifeless: And how do I do that? :)03:04
lifelessI'm sure you'll think of something03:05
lifelesse.g. how did you make sure the bug tag editing worked03:05
huwshimilifeless: Well I'm trying to figure out if we actually use this module at all03:06
lifelesswgrant: do you remember where we set allowed attributes for built in types?03:17
lifelessthumper: ^03:18
thumperfor built in types?03:19
thumperlike what03:19
lifelessfor instance03:19
thumperlifeless: hmm...03:19
lifelessin this case, defaultdict03:19
lifelessdoesn't have __getitem__ permitted03:19
thumpergrep is your friend :)03:20
lifelessand making an interface would be batshit03:20
thumperit'll be in a configure.zcml03:20
huwshimiDoes anyone here know how we handle which javascript files are included in Launchpad? I've found this, but it doesn't seem to be correct: https://dev.launchpad.net/JavaScriptBuildSystem03:22
thumperhuwshimi: what do you want to know?03:23
thumperI've learned some things03:23
huwshimithumper: There is a javascript file (as part of lazr-js) that is included in our launchpad.js, but I can't see where we actually include that to be built.03:24
thumperjsbuild is full of magic03:25
thumperand a PITA03:25
thumpermake jsbuild calls out to a couple of utility scripts to determine dependancies03:25
huwshimithumper: do you remember which scripts they are?03:27
thumperhuwshimi: look in the Makefile03:28
thumperfor the jsbuild: target03:28
thumperthey are in backquotes03:28
wgrantThe file list is hardcoded now, isn't it?03:28
wgrantIn utilities/yui-deps.py03:30
thumperwgrant: does that include the lazr-js bits?03:32
thumperhuwshimi: are you doing something in lazr-js?03:34
thumperhuwshimi: I have a request03:34
wgrantthumper: Ah, no, just all of YUI :(03:35
huwshimithumper: I'm trying to build lazr-js for Launchpad and trying to determine if we are using a module that was updated to a new version by someone else.03:36
huwshimithumper: What's the request03:36
thumperhuwshimi: somewhere in the picker code, it is adding a &thinsp;03:36
thumperhuwshimi: it needs to die, and CSS used instead03:36
huwshimithumper: I don't think I want to change anything else right now or else my brain will melt, but feel free to file a bug or something and I'll try and get to it :P03:39
huwshimithumper: if you landed a branch quickly I could probably include this in the build.03:45
thumperhuwshimi: ETOOMUCHTODO03:45
huwshimithumper: Fair enough :)03:45
thumperhuwshimi: I've already been on a diversion03:46
thumperanyone: https://code.launchpad.net/~thumper/launchpad/choice-widget/+merge/5235703:49
huwshimithumper: Those scripts don't seem to mention lazr-js stuff03:52
huwshimithumper: or am I missing something?03:53
thumperhuwshimi: that is the last line of the jsbuild command in the Makefile03:54
thumperhuwshimi: I'm guessing that you'll find that is a symlink to the location where lazr-js is built03:54
wgrantWhat's the setuptools sdist option to add a version suffix?03:55
thumperusing the jsbuild_lazr make target03:55
thumperwgrant: I don't remember but I need it too03:55
thumperwgrant: what are you packaging?03:55
wgrantthumper: Updating pydkim.03:55
* wgrant greps logs.03:55
thumpernot in my history sorry03:56
wgrantBecause it's not in --help :(03:56
huwshimithumper: Ah right, so we do just include the whole of lazr-js and don't control which files we include.03:57
thumperhuwshimi: maybe...03:57
huwshimithumper: Thanks for that. Now I can get on with my investigating.03:58
wgrantExcept that the egg_info command doesn't work here :/03:58
wgrantIt's like setuptools isn't installed...04:00
huwshimi lifeless: so I think after all that we include that file but don't use it, but I can't be sure04:22
StevenKDoes anyone want to fix the failed stable into db-devel merge?04:23
wgrantNot again :(04:23
StevenKwgrant: Thanks04:23
wgrantThis is changing one that I already resolved on Saturday :(04:23
=== MTecknology is now known as MT-TakinBreak
StevenKwgrant: Can you have another look at https://code.launchpad.net/~stevenk/launchpad/populate-spr-changelogs/+merge/52363 ? I've addressed all of your concerns, bar IMasterStore, since everything else done by garbo uses it.04:36
wgrantStevenK: You renamed getCandidateSPRIDs, but it still returns SPR IDs...04:37
wgrantAlso, why are we still restricting to Debian?04:38
StevenK1. Our sampledata sucks. 2. I don't think there is any point doing it over more.04:39
wgrantApart from that it looks good.04:39
wgrantWhat about all the Ubuntu publications from more than a year ago?04:39
StevenKwgrant: Er? It returns a ResultSet of SPRs ordered by id?04:39
wgrantNo, it returns a ResultSet of SPR.id ordered by id.04:40
StevenKBah, fixed04:40
StevenKwgrant: I'm happy to say upload_archiveID IN (ubuntu.main_archive.id, debian.main_archive.id)04:41
wgrantStevenK: Why say that at all?04:41
StevenKRunning it over sampledata which the tests is horrid04:41
StevenKEr. Which the tests do04:41
wgrantIt's easy to clean the sampledata up.04:42
StevenKI'd rather just delete it :-)04:42
wgrantRight, but deleting it in the tests is hard.04:42
huwshimilifeless: What should I do with this then? I don't want to release these changes if they'll break things...04:42
lifelesshuwshimi: will they break things?04:43
huwshimilifeless: Well that's the problem, I don't know. I can't see where we're using this module, but that doesn't mean we're not using it. I could just be looking for the wrong things.04:44
StevenKwgrant: Personally, I'd prefer to leave it for Debian. For the derived case, we aren't going to care about the older Ubuntus04:45
wgrantStevenK: There is stuff in Natty that has no changelogs...04:45
lifelesshuwshimi: check the wiki for info, failing that ask for help: mail the dev list asking for 'how to find out where a lazy-js module is used by lp'04:45
huwshimilifeless: OK thanks, will do.04:45
wgrantStevenK: Anything that was uploaded before we started importing changelogs not-very-long-ago.04:46
StevenKwgrant: The DB on DF for maverick said zero had a null changelog ...04:46
wgrantStevenK: With upload_distroseries=natty? Sure.04:47
wgrantBut not much stuff in natty was uploaded to natty.04:47
StevenKThe natty DB on DF is very little04:47
wgrantEven moreso there.04:47
lifelessStevenK: the point is the things uploaded to maverick that haven't been changed in natty04:47
StevenKwgrant: So, how do you propose I have the tests ignore the sampledata?04:48
wgrantStevenK: Expire all LFAs at the start of the test.04:48
wgrantThe brutal approach.04:48
StevenKAllow me to say "Ew"04:48
wgrantIt is better than sampledata.04:48
* StevenK tries to work out how to do so04:49
wgrantWe may eventually want two template DBs, one without crap.04:49
wgrantSince we won't be able to drop the crapful one for a long time :(04:50
StevenKwgrant: I can't think of an elegant way to expire all the LFAs04:54
wgrantStevenK: Store.find(LibraryFileAlias).set(content=None)?04:55
StevenKwgrant: Having it return SPRs directly complains about the GROUP BY04:59
wgrantStevenK: Group by SourcePackageRelease instead.04:59
wgrantThat might be why I was grabbing IDs... grouping by all the fields may have proved slower, although that seems odd.05:00
wgrantSince it should be able to optimise to a pkey grouping.05:00
lifelessshoulda coulda woulda05:00
lifelessthe way to do that is to use the id based query as an inner query05:00
lifelesslike e.g. _getExternalMessages does05:01
jtvStevenK: can you think of any reason why gina would initialize the references in SPPHs it creates using the ids for distroseries, SPR, etc. instead of the actual model objects?05:17
StevenKgina is .... special05:18
* StevenK grumbles05:21
lifelessjtv: code fragment ?05:21
StevenKNow it dies due to a variable used before assignment :-(05:21
StevenKAnd if I change its indent level, it loops until killed05:22
jtv        entry = SourcePackagePublishingHistory(05:22
jtv            distroseries=self.distroseries.id,05:22
jtv            sourcepackagerelease=sourcepackagerelease.id,05:22
jtv            status=PackagePublishingStatus.PENDING,05:22
jtv            component=component.id,05:22
jtv            section=section.id,05:22
jtv            datecreated=UTC_NOW,05:22
jtv            datepublished=UTC_NOW,05:22
jtv            pocket=self.pocket,05:22
jtv            archive=archive05:22
jtv            )05:22
jtvStevenK: is gina special enough to warrant that?05:22
wgrantI have a branch which fixes that.05:23
lifelessjtv: I would change it to distroseriesID=self.distroseries.id etc05:23
StevenKjtv: Er, things like that are the REASON why gina is special05:23
lifelesswe shouldn't overuse Reference objects, they are fragile05:23
jtvOkay, but is there any justification?05:23
wgrantFor manually instantiating it, rather than using PublishingSet.newSourcePublication?05:23
jtvNo, for using the ids instead of the model objects.05:24
StevenKI suspect SPPH support was bolted on.05:24
jtvBecause I'm trying to make it use newSourcePublication.05:24
wgrantOne remaining issue: it doesn't set datepublished.05:24
StevenKWhich is fine05:25
StevenKWe don't want it to05:25
jtvIt shouldn't even do that in the first place.05:25
wgrantPerhaps so, but inconsistent.05:25
jtvShame that branch sat around for half a year—it's a change I need for my current work.05:25
wgrantjtv: I didn't want to get into the gina argument at that point.05:26
jtvBut all in all we don't really know whether there's any particular reason to use ids there?05:27
wgrantThere isn't one.05:27
wgrantIf you see something odd in gina, it's because gina is crap, not because it needs to be that way.05:28
jtvThat's nice to hear.05:28
jtvBrings me to the next question: newSourcePackagingPublication creates a new DSP if one does not exist—which is breaking gina tests.05:29
wgrantI added a permission in that branch.05:29
wgrantI think it's the right thing to do.05:30
jtvI was wondering if maybe it was just laziness of the test, and there should always be a DSP05:30
wgrantgina predates DSP.05:30
wgrantThere's no problem with creating one.05:30
StevenKwgrant: Why didn't you want to get into a gina argument?05:31
wgrantStevenK: Because nobody knows whether we want to stop setting datepublished or start creating them Published or burn gina to death.05:31
lifelesswhats gina used for?05:32
StevenKJulian said yesterday we want to stop setting datepublished05:32
wgrantOK, how does _'s string interpolation work?05:32
StevenKlifeless: It imports Debian05:32
wgrantlifeless: Importing Debianish archives into the DB and librarian.05:32
jtvAnd so the datepublished=UTC_NOW is completely bogus05:32
wgrantjtv: It is a lie, and not one with considerable utility.05:33
wgrantKill it.05:33
jtvI had already done that.05:33
StevenKwgrant: http://pastebin.ubuntu.com/577330/ makes me sad05:33
wgrantStevenK: Oh?05:34
wgrantIt makes me happy.05:34
wgrantStevenK: Which line does it fail on?05:35
StevenKself.start_at = spr.id + 105:36
wgrantAh, that's not a new issue.05:36
wgrantIt'll happen whenever there are no SPRs to process.05:37
lifeless200 tasks -> 10 second renders :(05:50
wgrantlifeless: But it at least renders now?05:51
lifelessthats on my laptop05:51
lifelesswhich is an i705:51
lifeless50ms per bugtask hmm05:53
=== almaisan-away is now known as al-maisan
wgrantlifeless: Is wampee running with the second new set of appservers?05:58
lifelesswgrant: yes06:00
jtvwgrant, want to review my branch?  https://code.launchpad.net/~jtv/launchpad/bug-730460-use-spph-factory/+merge/5251506:05
wgrantjtv: Gladly.06:05
jtvwgrant: I split off the factory part to free me up for the real work, so there's not much there.06:10
wgrantlifeless: Could you mentor that?06:11
=== al-maisan is now known as almaisan-away
jtvI'll go grab some food.06:21
=== jtv is now known as jtv-eat
wgrantlifeless: What would you recommend as a Python testrunner these days? I normally use nose, but you probably have better ideas.08:09
=== almaisan-away is now known as al-maisan
lifelesswgrant: testr with subunit.run as the core08:18
wgrantlifeless: subunit.run doesn't do test discovery, does it?08:20
lifelesswgrant: it does if you have python 2.7 or the discovery module installed08:20
wgrantAh, handy!08:21
adeuringgood morning08:35
StevenKwgrant: So, that script doesn't love me.08:52
wgrantStevenK: Oh?08:59
StevenKFeatureError: Single aggregates aren't supported after a  GROUP BY clause08:59
wgrantbigjools: What do you think about replacing cron.daily-ppa with DiskPool improvements?09:00
StevenKwgrant: I think we're going to have to shift back to ids ...09:00
lifelesswgrant: see the oops-repository .testr.conf for inspiration09:00
wgrantStevenK: Yeah, probably.09:00
lifelesswrap the result as I suggested09:00
StevenKwgrant: Will you rescind your issue with it if I do that?09:00
lifelesssee _getExternalthingy in translations for an example09:01
wgrantStevenK: Maybe.09:01
wgrantlifeless: No lp:oops-repository?09:01
lifelesswgrant: https://code.launchpad.net/oops-repository09:03
bigjoolswgrant: sounds reasonable as long as it doesn't slow down the publisher09:03
lifelesswgrant: I haven't setup a 'trunk' yet, nor all the metadata09:03
bigjoolsfigure out where empty dirs are left09:03
wgrantbigjools: The file removal method will just check if the directory it's removing from is empty.09:04
wgrantSo it will slow p-d-r insignificantly.09:05
bigjoolslifeless/stub: can you guys look at my MP with schema changes please?09:20
lifelessbigjools: I'll look in the morning - brain is hurting ;)09:23
lifelessbigjools: I've just enough energy left to send a perf tuesday mail :)09:23
bigjoolslifeless: it is the morning :)  ok, no prob09:24
lifelessmorning schmorning09:24
StevenKwgrant: :-(09:25
stubWhen do we merge db-devel -> devel? I've got a futzed merge proposal up (it will land on devel, but contains stuff currently only in db-devel)09:47
stubbigjools: I'll look now09:47
bigjoolscheers, even09:48
stubbigjools: you targetted to devel, not db-devel09:50
bigjoolsFFS, I didn't, but bzr send did :/09:50
lifelessstub: anytime prior to the pqm freeze09:51
lifelessstub: so anytime in the next 4 weeks09:51
stubbigjools: point me to an updated mp when you have one. its r=stub, patch-2208-52-0.sql09:53
=== Ursinha is now known as Ursinha-afk
stubI think that is anytime prior to reopening db-devel for landings, innit?09:54
bigjoolsstub: ta09:55
lifelessstub: db-devel is open09:55
lifelessstub: we had the freeze for this month already09:55
stubThere are changes on that branch that landed before the freeze that are not in devel09:55
lifelessstub: we merged everything from db-stable09:56
lifelessnight y'all10:00
bigjoolsnn lifeless10:00
stubI can't see a db-devel or db-stable merge since feb10:03
bigjoolsstub: when it scans: https://code.launchpad.net/~julian-edwards/launchpad/publisher-config-db-schema/+merge/5253010:03
stuboic it10:03
stubnfi how my stuff missed going across - it landed on Friday10:04
stubI even got poked to qa it10:05
stubMine can wait, but if an earlier revision got merged we might be missing something important.10:06
wgrantstub: We merged the rev right before yours.10:08
wgrantstub: Yours was un-QA'd at the time.10:09
wgrantWe were already very much out of LOSA time, so we didn't want to push it.10:09
wgrantWe could almost sensibly merge again, though...10:10
stubthere is no particular reason to land it this cycle.10:12
wgrantThere is.10:12
wgrantThe spam from the old one is irritating!10:13
=== al-maisan is now known as almaisan-away
=== almaisan-away is now known as al-maisan
=== vila is now known as vila-afk-lunch
=== al-maisan is now known as almaisan-away
=== henninge_ is now known as henninge
=== henninge is now known as henninge-lunch
=== almaisan-away is now known as al-maisan
=== vila-afk-lunch is now known as vila
jamcan somebody land https://code.launchpad.net/~jameinel/launchpad/use_loggerhead_trunk/+merge/5206212:11
jamAt least, lifeless said on IRC, "Just land the branch" because loggerhead's pqm didn't actually run the test suite12:11
jamso we didn't bother setting it up again yet12:11
jmljam: sure thing.12:14
jmljam: will let you know when the EC2 instance has detached.12:14
jamjml: thanks12:15
jmlhmm. looking at the diff, I'll skip EC212:16
jmllp-land requires a local checkout12:16
jamjml: it is pretty simple :)12:16
jmljam: it's in PQM now.12:18
jamjml: thanks12:18
bigjoolsthis whole Storm Unicode vs RawStr thing is driving me potty12:37
jmlwhat about it?12:39
bigjoolsI can't throw str at unicode and expect it to work12:39
jmljam: "81 queries/external actions issued in 10.73 seconds" -- I want to debug your problem instead of answering email :\12:40
bigjoolsif I try and use RawStr and str() everywhere, somehow unicode gets in....12:40
jmlbigjools: the first thing you said is basically the whole point. A str doesn't have any encoding information, so it can't be used to represent human-readable text.12:42
bigjoolsbut it can ...12:42
jamjml: this is http://pad.lv/OOPS-1893C1040 ?12:43
jmlno, it's just bytes. a_str.decode('utf8') is something that might represent human-readable text12:43
jmljam: yes.12:44
bigjoolsI would not mind if this was consistent everywhere, but it's not12:45
jmlbigjools: can you give an example of where it's not?12:46
jml(Python is lousy at this, but Storm is pretty good)12:47
jmlle sigh12:47
bigjoolsjml: I defined a text column in PG, the storm model was RawStr, and I poked in some data from a form with an explicit str() case.  When Storm reads it back, it blows up with a unicode error.12:48
bigjoolsI'm reverting it all back to unicode now12:48
jambenji: since you approved https://code.launchpad.net/~jameinel/launchpad/suppress-generator-exit-726985/+merge/5241412:49
jamcan you ec2land it?12:49
jamOr does it need a second review because you didn't mark it overall approved.12:49
jamjml: I'd be happy for you to fix it :012:49
jamIt isn't a page I go to often12:50
jmljam: I'll be lucky to have caught up on my email by the end of the day.12:50
jamso not something that is blocking me. But a bit surprising to have it take 14s to load a code overview12:50
benjijam: it doesn't need more approval unless you think it does12:51
jambenji: fine with me. I don't think it is very controversial12:52
jamwgrant might want to look at it. Since he asked me to write it. but it was up there for him yesterday :)12:52
benjiI guess can land it for you if you need.12:52
=== al-maisan is now known as almaisan-away
jambenji: well, if you think it needs ec2land, I would appreciate it. Since I'm not an lp dev, I don't have an ec2 account, etc.12:54
=== almaisan-away is now known as al-maisan
benjiah, gotcha; sure I'll do it for you12:54
wgrantjam: Looks great.12:55
wgrantbenji: Thanks.12:55
jamwgrant: what time is it there?12:55
wgrantjam: Not quite midnight.12:55
jamah, k12:55
jamwasn't sure if it was late enough to be considered "early" yet12:56
wgrantHeh, no.12:57
=== henninge-lunch is now known as henninge
gary_posterlosas, is there a way for me to check how long the current staging update has taken without bothering you?14:17
benjigary_poster: I think devpad.canonical.com:/srv/launchpad.net-logs/staging/sourcherry/2011-03-08-staging_restore.log holds the answer, but I'm not entirely sure how to read it14:20
gary_posterbenji, cool, thanks, looking14:20
jcsackettsinzui: if a project is marked as inactive, that still doesn't free up the name for another project to use, does it?14:22
gary_posterbenji, losas, AFAICT staging started Tue Mar 8 04:14:09 UTC 2011 and ran without errors, and yet 10 hours later we still have no staging. :-( (I synced the logs just to make sure I wasn't missing anything)14:32
gary_posterbenji, just sharing with you in case you were curious :-)14:33
Chex_gary_poster: let me take a look14:33
gary_posterthank you Chex_14:33
benjiI am.14:33
Chex_gary_poster: ok, appears we had a postgres/slony failure, getting you a pastebin14:37
gary_posterthanks Chex_.  Did I just miss it in that log benji mentioned above, or was it somewhere else?14:38
Chex_gary_poster: yes, it fools me often as well, the slony error happens before the end of the process, see here for where it was in the logfile: https://pastebin.canonical.com/44394/14:39
* Chex_ looks at his nick, and frowns14:39
gary_posterah!  ok, sorry Chex_, will be more careful next time14:40
=== Chex_ is now known as Chex
gary_posterI guess I'll go look at the server log, because I have no idea how to remiedaiate that so far14:41
gary_posterjcsackett, I think you were talking to an absent sinzui before, and he is now present14:41
* leonardr also wishes to talk to a present sinzui14:42
jcsackettah, thanks gary_poster.14:42
jcsackettsinzui: if a project is marked as inactive, that still doesn't free up the name for another project to use, does it?14:42
sinzuiI did not know I was offline.14:42
sinzuiNot at all14:42
jcsackettsinzui: clearly, neither did i. :-P14:42
sinzuiWe rename them on request, or we give the old project to the user14:43
jcsackettah, okay. i am looking at https://answers.launchpad.net/launchpad/+question/14831414:43
jcsackettglobal does look like it's not doing anything. but as project-reviewing-overlord, perhaps you would like to weigh in, sinzui?14:45
sinzuijust rename the old project to global-old14:46
gary_posterstub, you around by chance?14:46
gary_posterI'd like to know if there's any remediation you can suggest for https://pastebin.canonical.com/44394/ which is causing staging to be down14:47
sinzuijcsackett:  this issue relates to bug 10650114:48
_mup_Bug #106501: Automatically warn about, then delete, unused projects <chr> <lp-registry> <projects> <Launchpad itself:Triaged> < https://launchpad.net/bugs/106501 >14:48
jcsackettsinzui: i concur.14:48
jcsackettthat process would be nice.14:49
jcsackettso, you think we can offer to make the person the head of global, sinzui?14:51
sinzuijcsackett: just rename the old one.14:52
jcsackettsinzui: cool.14:52
sinzuijcsackett: we really do not think about it. we just get the trash out of the way14:53
jcsackettsinzui: sounds good.14:53
leonardrsinzui, when will you have time to revisit my sampledata problem?14:54
sinzuileonardr: about 45 minutes?14:54
leonardrsinzui, cool14:54
rvbasinzui: Hi ... I've been tasked with bug 727632 and I wonder if you could give me hints on how you think it should look in the UI as I'm still fairly new to the UI (or the code for that matter). I just sent you an email with a few details and a mockup.15:15
_mup_Bug #727632: distro and distro series pages do not specify their owner <derivation> <distributions> <series> <Launchpad itself:Triaged by rvb> < https://launchpad.net/bugs/727632 >15:15
sinzuiDo they have an owner?15:16
sinzuiNo they do not15:16
sinzuidistros and series have registrants.15:16
sinzuiThe owner is always the project maintainer15:17
=== Ursinha-afk is now known as Ursinha
sinzuirvba you can cargo cult the "registering" slot from a lp/registry/templates/product-index.pt into the distro and series pages. The tales  need to pull the series/owner field, but the wording will be "Registered by"15:19
rvbasinzui: isn't this what's currently displayed already?15:20
sinzuirvba: oh. I think soo15:20
sinzuirvba, is the real issue that we do not have this data in the schema15:20
rvbaI confess I'm not sure if the issue is only in the UI or in the data. I though it was only a UI bug.15:21
sinzuirvba, poolie is confused in this bug15:21
sinzuiwe do not have "owners" of these objects, They must be like projects and project series15:22
* sinzui looks at the schema15:22
sinzuirvba. These pages look like the project and series pages. They are correct! I think the issue may be that the schema does not provide the the information, so it is pulled from a surprising field.15:23
rvbasinzui: maybe I'll just correct the wording "Registered" instead of "registered" and ask poolie more details about this then ...?15:25
sinzuirvba distro is missing the registrant field! we are showing the maintainer as the registrant. This is a lie. *All* distros and distros series were registered by a losa, most by mthaddon actually15:26
rvbasinzui: ok I get it. So the information is wrong then.15:27
sinzuirvba: distro has a owner field, but it is actually the registrant. The owner is always the distro maintainer (owner).15:27
sinzuidisrgard that15:28
sinzui*distroseries* has a owner field, but it is actually the registrant. The owner is always the distro maintainer (owner).15:28
rvbaso if we want to display the owner field I just have to pull that info from the corresponding distro (for series)15:29
sinzuirvba: so this bug is about 1, the distro does not know who the registrant is. 2, distroseries stores the registrant in the owner field.15:29
sinzuirvba: Many object store the registrant in the owner field because the PrimaryContext (project or distro) always owns them15:30
* sinzui looks for bug15:30
* rvba looks into the datastructure15:30
sinzuiexcellent. I find duplicate bugs15:33
sinzuirvba: This issue overlaps with bug 153333, bug 188402, bug 20753215:34
_mup_Bug #153333: Clean up owner and registrant attributes <feature> <lp-registry> <registry> <tech-debt> <Launchpad itself:Triaged> < https://launchpad.net/bugs/153333 >15:34
_mup_Bug #188402: Include registrant on people page <feature> <lp-blueprints> <Launchpad itself:Triaged> < https://launchpad.net/bugs/188402 >15:34
_mup_Bug #207532: Product, Project and Series don't have read-only registrant <lp-registry> <tech-debt> <Launchpad itself:Triaged> < https://launchpad.net/bugs/207532 >15:34
rvbasinzui: this is much more involved than I anticipated :-)15:36
sinzuirvba: you do not need to fix everything. those bugs provide context by what we mean by registrant and owner in the schema15:37
sinzuirvba: I updated your bug about what is wrong with distro and distroseries15:37
rvbasinzui: all right, I'll focus on fixing this problem for series and distro series15:38
rvbabut still, since registrant and owner are two different things (?) the UI will have to evolve to present the 2 informations right?15:38
sinzuiThere is one caveat. both distro and distroseries implement IHasOwner, which means they must have an owner field, but in the case of distro we mean maintain, and in the case of distroseries, we mean registrant15:40
sinzuiWe either change the interface / implementation of these objects, or we continue to ignore them.15:40
rvbasinzui: don't you think that changing the interface / implem should be part of the larger refactoring you were just talking about15:43
=== al-maisan is now known as almaisan-away
sinzuirvba: we could remove the registrant field from distros so we stop lying. We could stop there. As pointed out in the other bugs. registrant should be immutable. Since distros series do not have a direct owner, it seemed okay in the past to reuse the field for registrant. but owner is intended to be mutable. It is not possible BTW to change a series owner. We removed the field from edit forms15:46
rvbasinzui: thanks for the clarification15:48
rvbasinzui: I guess I'll fix the interface for now since we (red team) are in feature mode.15:49
sinzuirvba: this could get tricky. set a limit of a few hours for each task and if you do not make progress on the task, shout for help. you are in 6 years of muddled code and It is easy to loose scope15:51
rvbaif I understand correctly to fix it properly we would have to create proper interfaces like IHasRegistrant and refactor quite some code ... plus the data migration.15:51
deryckgmb, ping.  you got a sec?15:52
sinzuirvba: I think that increases scope. I am sure you do not want to retrofit that into all the objects.15:55
rvbasinzui: you're saying I should focus on distributions and distroseries?15:56
sinzuirvba: maybe we want to make add registrant to distroseries and make the property always return the owner. We can export owner as registrant.15:56
rvbasinzui: so no data migration if I understand you suggestion15:57
sinzuirvba: yes the issue is about making the ui clear. You need to add a registrant field to distro. that is a schema..ui change. I do not think we can migrate the data, it is all lies. We usually state the object was registered by ~registy in this case.15:58
sinzuirvba: I think these are two bugs, because you need two branches to fix the different needs of each object and page15:59
gmbderyck: Sure.15:59
deryckgmb, so I retriaged bug 605923 at leonardr's request.  It's the missing archived debian bug bugwatch issue again.16:00
_mup_Bug #605923: bug watches for archived debian bug reports appear not to exist <bugwatch> <lp-bugs> <Launchpad itself:Triaged> < https://launchpad.net/bugs/605923 >16:00
deryckgmb, I was wondering if you could take a look at the bug and just brain dump some info there, for anyone wanting to look into fixing it?16:00
gmbderyck: Sure. I'll take a look at it in a little bit.16:01
deryckgmb, many thanks!16:01
rvbasinzui: so you want to add a registrant field to the distro but still populate the owner field?16:01
sinzuirvba: add registrant to distro, the migration script will set the value to ~regisrty16:01
rvbasinzui: I'm not sure I understand the delimitation of the "2 bugs" properly16:02
sinzuirvba: most distros were registered by kiko, most distroseries where registered by mthaddon. both cases they were admins, they were not involved in the project and no one should contact them about them16:03
sinzuirvba: 1. distro need a registrant field. the page needs to use the registrant field in the registering slot. the migration script will set all existing distro registrants to ~registry16:04
sinzuirvba: 2. distroseries registrant field is the owner, we have choices: a) rename the field field to registrant, b) add a registrant property that always returns the owner, update the interface to export the owner as registrant.16:06
rvbasinzui: all right ... this seems to be good bug to deep further into the code16:07
rvbasinzui: unless you think you need to give me more information I think I'll dive in and come back with question when I'll be stuck16:08
sinzuirvba: happy hacking.16:08
rvbasinzui: s/question/questions/g16:08
rvbasinzui: thx a lot16:09
deryckhenninge, looking at your MP.... should TranslationSharingDetailsMixin also raise NotImplementedError for is_sharing and can_edit_sharing_details?16:12
deryckhenninge, so the API is clear?16:12
henningederyck: Yes, I think that makes sense. I thought about it, too, but I don't know why I discarded the thought.16:13
deryckhenninge, ok, cool.16:13
deryckhenninge, ok, looks good to me otherwise.  I'm not sure I see the value in both the page test and the browser unit test.  they seem very similar....16:33
deryckhenninge, but I guess there is already too much there and it needs updating.16:34
henningederyck: The browser unit test never checks the actual URL, but the page test does.16:34
deryckhenninge, what value is there in the URL?  Just that hitting the URL produces the page we expect?16:35
henningeThe link is still dead at this point, so it's just showing that it is pointing to the right place.16:36
deryckhenninge, sure, that's fine.  Seems like a lot of page test for this.  but no worries.  it's your call.16:37
sinzuihi leonardr: I think I have my head clear and can help. I recall we were looking a test failures that I thought were really because the test was not setup right; sample data is wrong16:37
deryckhenninge, so r=me, with the tiny change mentioned above.16:37
leonardrsinzui: right16:38
henningederyck: thanks a lot ;-)16:38
leonardrsinzui: my code is at lp:~leonardr/launchpad/bug-10633816:38
leonardrthe failure is in patches-view.txt16:40
sinzuileonardr: I saw many tests fail last year when we added code that checked that the package was legit. I fixed them by ensuring they were published in Ubuntu:16:41
sinzui    sourcepackagename='alsa-utils', distroseries=distroseries)16:41
leonardri think that could solve the 'a52dec is not in ubuntu' error16:42
=== jtv is now known as jtv-zzz
leonardrdo you think it makes sense to go through patches-view and get it to stop using sampledata?16:42
sinzuithis test sucks16:43
sinzuiI want to delete the test and declare victory16:43
leonardrthe whole thing? do we have coverage elsewhere?16:44
sinzuiI doubt there is proper test coverage16:44
sinzuiwhere is the failure in this test?16:45
* sinzui runs16:45
leonardrsinzui: there are some problems creating the data in the first place, and then there are problems where the new validation code i added says there are problems with which bugs are conjoined to other bugs16:47
sinzuithis test could use the factory instead16:47
sinzuileonardr: is evolution and a52dec the problems? I think there are missing SPPH in hoary in sample data. We can use the factory to make them genuine ubuntu packages16:49
leonardrok, i'm going to write some code that might or might not work16:49
sinzuileonardr: pause16:49
sinzuiI just got this built. I will play the test and try a trick with the factory and transaction.commit() to put the packages in the right state. Feel free to look at something else or get some tea16:50
leonardrok, i'm working on another project as well16:52
sinzuiexcellent, someone solved this in a story before16:52
sinzuileonardr: sorry. I had make schema. I think I have this fixed. Looks like I fixed the same problem in a bug story last year17:03
leonardrthat's great17:04
m4n1shin https://launchpad.net/+apidoc/1.0.html#person the field hide_email_addresses17:05
m4n1shwhat should it be set?17:05
leonardrm4n1sh: yes17:09
m4n1shleonardr: can it be reflected in the WADL?17:10
m4n1shWADL file17:10
sinzuiunity is a memory hog17:11
leonardryes, we could set type="xsd:boolean" as we have for dates17:11
sinzuiand I think it leaks like a sieve.17:11
m4n1shsinzui: it has a great ALPHA plastered all over17:12
m4n1shleonardr: under which project should I file a bug for this?17:12
leonardrm4n1sh: put it in lazr.restful17:13
sinzuileonardr: I need to reboot to reclaim 1.5G of memory to run the test suite: This is what I did and It think it will fix the issue: http://pastebin.ubuntu.com/577529/17:14
leonardrsinzui, thanks17:14
=== deryck is now known as deryck[lunch]
leonardrsinzui: ok, that got rid of the sampledata problem. i still have the problem that the validation is preventing bugtasks from being created17:27
leonardrmake_bugtask(bug=bug_a, target='hoary', target_is_distroseries_name=True)17:28
leonardrLaunchpadValidationError: This bug is already on Ubuntu. Please specify an affected package in which the bug has not yet been reported.17:28
sinzuileonardr: are we certain that these bugs are unique? They may not given this is a story test17:29
leonardrsinzui: well, the bugs are created within the test17:31
leonardrso any bug tasks for that bug only exist within the test17:31
sinzuiyes, but since the test does note setup isolation from the blocks in each narrative, it is easy to step on something17:31
leonardrsinzui: looking at the calls to make_bugtask, for bug_a we have a task on the 'trunk product series of patchy,17:33
leonardrthe evolution source package (of what?)17:33
leonardrthe a52dec source package (of what?)17:33
leonardrand then we try to create one on hoary17:33
leonardrcould the source-package ones be interfering with the attempt to create a generic hoary bugtask?17:36
sinzuileonardr: possibly. If the validator looked at distroseries before source package17:38
sinzuileonardr: the calls to create SPPH were in hoary. I assume that we are working hoary in the test because it is the current series in sample data17:40
leonardrsinzui: here's the validate code17:41
leonardr        target = None17:41
leonardr        if distribution is not None:17:41
leonardr            validate_new_distrotask(bug, distribution, sourcepackagename)17:41
leonardr        elif sourcepackagename is not None:17:41
leonardr            target = distroseries.getSourcePackage(sourcepackagename)17:41
leonardr        else:17:41
leonardr            target = product or productseries17:41
leonardr        if target is not None:17:41
leonardr            valid_upstreamtask(bug, target)17:41
leonardrthat was the best i could figure out when to call validate_new_distrotask/validate_distrotask/valid_upstreamtasks17:41
leonardryou think source package name should take precedence over distribution?17:41
sinzuiyes I do17:41
sinzuiI think the rules are package before series before primary context17:43
leonardrok, trying that now17:44
leonardrsinzui: 'series before primary content' -> product or productseries should come before distribution?17:44
sinzuiI think it is distro before project. project and distribution are primary context...that is to say they are pillars, they are the first item we traverse17:50
leonardrsinzui: i gotta tell you that everything you say on this topic goes off on a 45 degree angle from me understanding it17:53
sinzuileonardr: bugtasks and conjoined bugs, packages and publishing history are arcane stuff. If the test were good, this would not be a morass.17:55
sinzuileonardr: I write something that I think is the correct test, then delete everything that contradicts it.17:56
leonardrsinzui: but i have no idea how this works. i'm just trying to make an error give a 400 status code when it happens17:56
leonardrlet's see what happens now...17:57
sinzuileonardr: :( this is why these bugs are so old. fixing it means making our applicate make sense17:57
leonardrsinzui: revising the check to my understanding of the precedence doesn't affect the error17:58
leonardri'll paste17:58
sinzuihmm, well maybe we do not need the sourcepackage check. does validate_new_distrotask already do that?18:00
leonardrsinzui: i think it does. can we call validate_new_distrotask on a distro series or can it only be a distribution?18:01
leonardri think it can only be a distribution18:02
leonardrso, if we're passed in a distro series, we still need the sourcepackage check18:02
=== MT-TakinBreak is now known as MTecknology
leonardrsinzui: http://pastebin.ubuntu.com/577553/18:12
leonardri've spelled out the precedence in such explicit detail that i now understand it, and i'm still getting the error18:12
leonardrdoes that code look right?18:12
sinzuileonardr: It looks right, but it makes me weep.18:15
=== deryck[lunch] is now known as deryck
sinzuivalidate_new_distrotask is called twice I know a distroseries is never upstream, yet we validate it with upstream. I think something stinks below your method. I would not ask you to fix it just yet given the difficulties we are having in the branch too18:17
sinzui^ leonardr if your branch starts passing tests now, lets report a bug about this insanity and let it block the value your branch already provides18:17
leonardrsinzui: tests are still not passing18:18
leonardrvalidate_new_distrotask is raising the exception18:18
sinzuileonardr: doh, I do *not* want to block your branch from landing. I prefer to report a bug about the issue and say we are done18:18
leonardram i calling it when i should not be?18:18
sinzuileonardr: maybe.18:19
leonardrsinzui: ok, i understand what's happening18:23
leonardrthis code prohibits you from adding a bugtask to the 'ubuntu' distribution if there is already a bugtask in ubuntu's 'evolution' sourcepackage18:24
leonardris that the correct behavior?18:24
sinzuiI am thinking18:24
sinzuileonardr: I want to say no, but if I were more involved in Ubuntu, I think I would say this is sensible. A bug is Ubuntu is generic. the bug must really be in a package. We know many users cannot guess the correct package. We move the bug to the package during triage...18:26
sinzuibut I still believe I could have a task that is definitely not in the package specified, and I do not know where it is18:27
* sinzui plays on staging18:27
leonardrsinzui: i do not want this branch to change the behavior18:27
sinzuistaging I hate you. May a pox befall your first born18:27
leonardrno, the first born is production18:28
sinzuileonardr: the validator is correct. We do not want a generic bug when we have a bug that is specific. The test needs to be buried alive, then dug back up and shot18:29
m4n1shdone filing the bug https://bugs.launchpad.net/lazr.restful/+bug/73151818:30
_mup_Bug #731518: Set hide_email_addresses in person of type xsd:boolean <lazr.restful:New> < https://launchpad.net/bugs/731518 >18:30
m4n1shanyone can have a look at it18:30
* sinzui looks at what could be deleted18:30
sinzuileonardr: If I find that one template/view is governing all the layout in this test, I will start chopping18:33
leonardrsinzui: ok, let me push my revisions18:33
sinzuileonardr: there is one view and one template. This test is repetitive nonsense. It works on bugtarget. We do not care about each bugtarget, neither does the user according the the bad narrative in the so-called story. We can delete from the second header to the end of the test. "Patches View by Distro" onward is redundant with product.18:40
leonardrm4n1sh, the quickest way to get that into lazr.restful would be if you gave me a branch. you just need to add binary support to WadlFieldAPI.type, and copy some tests from what we do to test xsd:datetime18:41
sinzuileonardr: the story should be about "a bug target, such as a project"18:41
m4n1shleonardr: will do in meanwhile when I get time18:42
leonardrsinzui: ok, do i have r=you on the changes i've already pushed + destroying the rest of that test?18:44
* sinzui looks one more time18:44
* leonardr has now pushed everything18:45
sinzuileonardr: r=me with the fixing of the two issue I think I see in the diff: http://pastebin.ubuntu.com/577568/18:52
sinzuiI seem to have traded a memory leak for a runaway proc. ubuntuone ate 66% of my battery in one hour.18:52
* sinzui restarts to kill procs18:59
leonardrok, bombs away19:01
jmlgary_poster, benji: help requested re https://code.launchpad.net/~jml/launchpad/what-is-in-the-web-ui/+merge/52594 (any other zope experts or interested punters welcome too)19:10
* jml is off for the evening19:10
jmlg'night all. happy hacking.19:10
bachi sinzui19:39
lifelesslets play spot the deploy: http://webnumbr.com/launchpad-critical-bugs19:47
=== ajmitch_ is now known as ajmitch
sinzuihi bac: sorry. I was dealing with a run away proc. you have my attention now20:04
bachi sinzui.  np20:05
baci was just going to ask you some yui test questions.20:05
baci'm trying to get some debugging output.  Y.log() disappears.  i tried console.log() too but it doesn't output to the browser console.  any tips20:06
sinzuium. no. are you saying the out never arrives, or that something clears it?20:07
bacexpected to see it in the browser console but see nothing20:08
bacsinzui: oh, wait, i do see it in the console div, amongst the other output.  i was looking in the error console20:09
sinzuioh, I was nervous asking if you were sure you were looking at the log20:10
thumpersinzui: got a minute to mumble?20:19
bacsinzui: deryck pointed out that commenting out the console in the test_.js dumps everything into the console and is interogatable.20:23
sinzuithumper: I can talk now20:27
lifelesssinzui: https://code.launchpad.net/~sinzui/lazr.restful/json-not-xhtml-0 looks like it isn't actually landed20:55
sinzuilifeless: I do not know what is up with that. I do not see it, but I think I pushed it.20:57
* sinzui tries again20:57
lifelesssinzui: I looked at https://code.launchpad.net/~lazr-developers/lazr.restful/trunk20:57
sinzuiso did I ;(20:58
lifelesswgrant: remerging db-devel would be cutting it -real- fine21:01
thumperleonardr: mumble?21:02
leonardrthumper, yes21:02
sinzuilifeless: sorry, I think I had pushed to the wrong location :(. The fix is in rev 178, version 0.17.421:04
lifelesssinzui: thats cool; are you going to do the lazr.restful update in lp itself?21:06
sinzuileonardr: can you look at the revs in https://code.launchpad.net/~lazr-developers/lazr.restful/trunk ? I am having a panic attack. I think something you landed today is missing because of my rememerge21:06
leonardrsinzui: it's possible, i'll look21:06
sinzuilifeless: please wait for leonardr to be sure I did not loose something21:06
lifelesssinzui: of course21:06
jcsackettlifeless: can you take a look at the reply to the mp of mine that stub added you to? and if you have any further concerns holding it back from being landed, let me know?21:06
lifelesssinzui: I'm hoping we can get the oem bug through qa for the deploy on thursday21:06
lifelessjcsackett: if its a db patch, the requirement is stubs approval - I just audit so I'm in the loop on db schema changes21:07
lifelessjcsackett: per the db schema change policy21:07
jcsackettlifeless: dig, i just saw a pending review for you that i didn't add. :-P21:08
lifelessjcsackett: so, you *should* have added it :) but you don't need my vote21:08
jcsackettright on. :-)21:08
jcsackettlifeless: did i read an earlier comment correct, that we may be remerging db-devel before the rollout?21:09
jcsackettoh good.21:09
lifelesswgrant was expressing enthusiasm for getting more fixes in21:09
jcsackettlifeless: dig. i had some concern there.21:10
leonardrsinzui: i think we may have lost your *own* changes21:12
sinzuileonardr: I really think I lost that last revision. if you have a copy of lazr-restful from you push today. I think you want to repush, maybe with overwrite. bzr cdiff -r176..-1 | less -r only shows my changes21:13
sinzuileonardr: but...I though I had pushed my changes yesterday, lifeless reported they were not there. So I remerged and pushed. So regardless of what happened. I want to be sure both out changes are in the branch21:15
wallyworldleonardr: https://code.launchpad.net/~wallyworld/launchpad/recipe-request-builds-existing/+merge/5238921:19
leonardrsinzui: i'm on revision 178 and my change to _resource.py is present21:19
sinzuileonardr: raise e?21:20
leonardrraise e within the else clause21:21
sinzuiokay I see that, We lost your commit message though21:21
sinzuisee the bottom of the page: https://code.launchpad.net/~lazr-developers/lazr.restful/trunk21:21
sinzuileonardr: I would be happy if you could push your copy to be trunk, then restart my merge so that the changelog is correct21:22
bacsinzui: have time to talk on mumble briefly?21:24
leonardrsinzui: i'm not sure how to do that. 'bzr push lp:lazr.restful' says no new revisions to push21:24
sinzuileonardr: does push --overwrite do nothing?21:25
leonardrlet's see21:25
sinzuileonardr: bzr log -n 0 -r 177..178 shows my remerge swallowed out commit message. So I do not think we lost code, I just messed up the log of histrory21:25
leonardri agree21:26
leonardr--overwrite also does nothing21:26
sinzui:( I suck21:28
sinzuileonardr: thanks for your time. I will try a reverse merge to undo this21:29
leonardrdo you want my original commit message?21:29
leonardrwallyworld: are we talking about 2 extra web service calls, or 2n extra calls?21:34
* wallyworld thinks for a second21:35
wallyworldleonardr: akin to 2n - 1 per archive and 1 per distroseries21:36
leonardri think the current approach is fine21:36
lifelesssinzui: you don't suck, the trunk isn't setup correctly21:36
wallyworldleonardr: cool. it just seems kludgy21:36
bacsinzui: when you have time i'd like to ask you about how yui unit tests find the local JS resources.21:36
lifelessit needs the setting in bzr for no history changes to be set21:36
leonardrit is, but kludgy is better than super-inefficient, and we don't have a good alternative right now21:36
wallyworldleonardr: it would be nice to somehow be able to specify what attributes we wanted to return for elements in a collection_link21:37
wallyworldwhen the link is fetched21:37
leonardrwallyworld: yeah, that's part of expand-and-filter21:37
wallyworldleonardr: is that something currently in progress?21:38
leonardrwallyworld: no, i planned to work on it but then the teams were rearranged. it needs to be scheduled as a large new feature21:38
wallyworldleonardr: imho it a pretty important feature as we move towards more xhr support in lp, so it would be good to see it on the radar :-)21:39
lifelesswallyworld: leonardr: can I but in and ask about the context ?21:39
wallyworldlifeless: i'm retrieveing a collection of build records via the ws api from javascript21:40
wallyworldlifeless: by getting a collection link21:40
wallyworldlifeless: the records contain links to other objects21:40
wallyworldbut i need attributes on those objects and the returned collection elements just have the links21:41
lifelesswallyworld: planning to, or are ?21:41
wallyworldlifeless: i have a mp for review21:41
lifelessI mean, is this in trunk already ?21:41
lifelesswallyworld: so, this sets of red flags for me.21:41
wallyworldme to but there's no infrastructure to support what i need yet21:42
lifelesswhat are the alternatives ?21:42
wallyworldsee leonardr's comment about expand and filter support21:42
wallyworldlifeless: i have an alternative - and am using it in the mp21:42
lifelesssorry, I wasn't clear21:42
lifelesswhat are the alternatives to crippling our appservers when users hit this page21:43
* lifeless adds hyperbole to help the discussion21:43
wallyworldlifeless: but we don't cripple the app servers - i'm not making N+1 calls - the mp uses another way of doing it21:43
leonardrlifeless: wallyworld is using url hacking to make 1 request instead of 2n21:43
lifelessok, whew.21:43
lifelessthats brilliant21:43
leonardri told him that was the right thing to do21:43
wallyworldlifeless: give me some credit - i'm not that stupid :-)21:44
sinzuileonardr: lifeless the history is back in the correct order. My remerge did not increment version.txt so my log fix really was wanted21:44
lifelesswallyworld: well, I read '10:41 < wallyworld> lifeless: i have a mp for review' as meaning the 2n approach21:44
wallyworldlifeless: i was just after a +1 from leonardron the kludgy approach used21:44
lifelesswallyworld: leonardr: thanks for humouring me, totally agree we should use the kludge21:44
wallyworldlifeless: fair enough :-)21:44
lifelesssinzui: great, so - are you going to do the lazr.restful upgrade for lp ?21:45
wallyworldlifeless: yeah, but we will run into more cases requiring it as we expand out xhr support, so I +1 on leonardr implementing expand and filter support into lazr restful :-)21:45
wallyworldif we ever get the time to do such things :-)21:46
leonardrsinzui, lifeless: i have a branch in ec2 land that will upgrade launchpad to 0.17.3, but you probably want 0.17.421:46
sinzuilifeless: I do not what what i do for that. cut a release, push it to pypi, increment egg version in buildhout?21:46
sinzuileonardr: which branch? I will cargo cult21:46
leonardrsinzui: the one you were looking at earlier21:47
leonardrsinzui: here's my 'release script': http://pastebin.ubuntu.com/577617/21:48
sinzuiexcellent. thank you!21:48
lifelesssinzui: cut a release; add to download-cache and commit, then update versions.cfg in lp and merge that to devel.21:50
lifelessgary_poster: hi21:52
gary_posterlifeless, hi21:52
lifelessthis bug I filed21:53
gary_posterbug 731099?21:53
_mup_Bug #731099: structural subscriptions have poor queries on bugs with many bugtasks <story-better-bug-notification> <Launchpad itself:Triaged> < https://launchpad.net/bugs/731099 >21:53
lifelessat the risk of being annoying, I was wondering if you had the time to chat - just here - about it interactively21:54
gary_postersubscription portlet shouldn't use the same code path anymore.  I haven't checked.  That's a bug IMO if it does21:54
lifelessso what I did locally was add 200 bugtasks to a single bug21:54
wallyworldleonardr: you ok to approve the mp so i can push it through ec2? thanks :-)21:54
lifelessthe main bug html rendered without the query I saw - it was the async population of the subscribers portlet that did the big query21:54
gary_posterif you get me that url I'll look at it lifeless.  essentially, here's where I am:21:55
lifeless /bugs/1/+bug-portlet-subscribers-content21:56
_mup_Bug #1: Microsoft has a majority market share <iso-testing> <ubuntu> <Clubdistro:Confirmed> <Computer Science Ubuntu:Invalid by compscibuntu-bugs> <EasyPeasy Overview:Invalid by ramvi> <GNOME Screensaver:Won't Fix> <Ichthux:Invalid by raphink> <JAK LINUX:Invalid> <The Linux OS Project:In Progress> <metacity:In Progress> <OpenOffice:In Progress by lh-maviya> <Tabuntu:Invalid by tinarussell> <Tivion:Invalid by shakaran> <Tv-Player:New> <Ubuntu:I21:56
lifelessis what my local log shows21:56
gary_poster1) the portlet should not use that "let's send things" query.  There's a much simpler approach that gives a reasonable approximation.21:57
gary_posterFurther, Jono and I have tentatively agreed that the subscribers info should probably be deleted because it is no longer accurate enough or interpretable-by-humans enough to be of value, but that's a but of a by the way.21:58
gary_poster2) the best way to get rid of the doubling in query size is to add with support to Storm (which I'd love to get to, but is not currently a priority).  What we can do now without losing functionality is to  make an intermediate query so we don't have to duplicate things.  As I said in the bug report, I'm not convinced it is the right balance of things, but it's not an inner loop, so it probably won't make that big of a22:00
gary_posterthat was probably truncated :-/22:00
gary_posterAs I said in the bug report, I'm not convinced it is the right balance of things, but it's not an inner loop, so it probably won't make that big of a diff either way22:00
lifelessgary_poster: it looked to me eyeballing the query that the inner clauses are able to be optimised out - but that may be my eyes glazing over at hand evaluating OR vs AND precedence22:00
gary_posterThere are two possibilities in your eyeballing22:01
gary_posterthat I see22:01
lifelessgary_poster: specifically, what I saw was (project-or-product and (status-of-filter) OR (project-or-product and (status-of-filter)22:01
gary_poster1) what you said22:02
lifelessgary_poster: and I thought it might be factorable to (status-of-filter and (project-or-product OR project-or-product OR ..)22:02
gary_poster2) I missed something22:02
gary_posterheh, I was going to go on for a while so I went for summaries ;-)22:02
gary_posterlifeless, is there a pertinent bit in the stuff you filed that's an example?  Let's see if I can explain it or not22:03
lifelesscontext -22:04
lifelessEFT JOIN BugSubscriptionFilterTag ON BugSubscriptionFilterTag.filter = BugSubscriptionFilter.id WHERE StructuralSubscription.id IN (1)22:04
lifelessrepeating bits -22:04
lifelessAND ((StructuralSubscription.product = 4 OR StructuralSubscription.project = 4) AND (BugSubscriptionFilt22:04
lifelesserImportance.importance = 20 OR BugSubscriptionFilterImportance.importance IS NULL) AND (BugSubscriptionFilterStatus.status = 10 OR BugSubscriptionFilterStatus.status IS NULL)22:04
lifeless(StructuralSubscription.product = 130) AND (BugSubscriptionF22:04
lifelessilterImportance.importance = 5 OR BugSubscriptionFilterImportance.importance IS NULL) AND (BugSubscriptionFilterStatus.status = 10 OR BugSubscriptionFilterStatus.status IS NULL)22:04
lifeless(repeats with only the product clause changing in each repeeat22:04
gary_posterlifeless, that's because the individual bugtasks have those values, individually22:06
lifelessso we could group this into one complex thing per (importance,status) tuple22:07
lifelesswhich would be about 25 clauses on pathological bugs22:07
lifelessgary_poster: I think we should be data driven about whether this matters22:07
lifelessgary_poster: the /bugs/1/+bug-portlet-subscribers-content url on a machine with the filters will let us gather that data22:07
_mup_Bug #1: Microsoft has a majority market share <iso-testing> <ubuntu> <Clubdistro:Confirmed> <Computer Science Ubuntu:Invalid by compscibuntu-bugs> <EasyPeasy Overview:Invalid by ramvi> <GNOME Screensaver:Won't Fix> <Ichthux:Invalid by raphink> <JAK LINUX:Invalid> <The Linux OS Project:In Progress> <metacity:In Progress> <OpenOffice:In Progress by lh-maviya> <Tabuntu:Invalid by tinarussell> <Tivion:Invalid by shakaran> <Tv-Player:New> <Ubuntu:I22:07
gary_posterlifeless, right; I thought about the optimization and rejected it.  I didn't expect anything like 200 bugtasks.  bug 1 is like that, I take it?22:08
_mup_Bug #1: Microsoft has a majority market share <iso-testing> <ubuntu> <Clubdistro:Confirmed> <Computer Science Ubuntu:Invalid by compscibuntu-bugs> <EasyPeasy Overview:Invalid by ramvi> <GNOME Screensaver:Won't Fix> <Ichthux:Invalid by raphink> <JAK LINUX:Invalid> <The Linux OS Project:In Progress> <metacity:In Progress> <OpenOffice:In Progress by lh-maviya> <Tabuntu:Invalid by tinarussell> <Tivion:Invalid by shakaran> <Tv-Player:New> <Ubuntu:I22:08
lifelessgary_poster: 1 is small -22:09
lifeless      1 |    2122:09
lifeless 230350 |   15922:09
gary_postermy goodness22:09
gary_posterwhy, out of curiosity?22:09
lifelesswhen I get the main page for bug 230350 rendering, I can tell you22:10
_mup_Bug #230350: Missing Debian Maintainer field <Ubuntu:Invalid> <afflib (Ubuntu):Fix Released by saivann> <alac-decoder (Ubuntu):Fix Released by warp10> <axiom (Ubuntu):Invalid> <beneath-a-steel-sky (Ubuntu):Fix Released> <bibletime-i18n (Ubuntu):Fix Released by txwikinger> <binkd (Ubuntu):Fix Released> <bzr-builddeb (Ubuntu):Won't Fix> <capisuite (Ubuntu):Invalid> <chinput (Ubuntu):Fix Released by warp10> <chmsee (Ubuntu):Fix Released> <ciso (U22:10
lifelessIt looks like a very meta bug22:10
lifelesspackages were missing a field, and it got batch filed on all the packages with that issue.22:10
lifelesswe could have bugs with thousands of tasks of this nature22:11
gary_posterwow, blech22:11
gary_posterisn't that indicative of a bug in whatever is using LP?22:11
gary_posterin any case...22:11
lifelessit *is* what multiple tasks are intended for.22:11
lifelessits just a scale that us devs don't inuit22:12
gary_posterIOW, it is reasonable to expect one bug to have action items in 150+ projects?22:12
gary_posterThis is pure curiosity22:13
gary_posterI'm trying to assemble my action item summary in a separate "thread"22:13
gary_posterthere are three action items I can take22:13
lifelessI believe 159 is our largest bug at the moment22:14
gary_posterRight, my question is not whether they exist, but whether they actually serve a useful purpose22:14
wgrantFWIW, performance was fine at the time the bug was filed.22:14
wgrantBut subscription noise stopped us from using them.22:14
lifelessbut this is also self limiting at the moment - the bug page scales at 50ms per task *after* 5 patches to improve it, so falls over well before larger split-outs22:15
leonardrwallyworld: sorry, didn't realize you wanted me to review the whole branch22:15
gary_poster1) if you file a bug about /bugs/1/+bug-portlet-subscribers-content tickling this code path and assign it to me, I'll promise to see if I can address it quickly.  If I can't address it quickly, I'll push it off till bug rotation.22:15
_mup_Bug #1: Microsoft has a majority market share <iso-testing> <ubuntu> <Clubdistro:Confirmed> <Computer Science Ubuntu:Invalid by compscibuntu-bugs> <EasyPeasy Overview:Invalid by ramvi> <GNOME Screensaver:Won't Fix> <Ichthux:Invalid by raphink> <JAK LINUX:Invalid> <The Linux OS Project:In Progress> <metacity:In Progress> <OpenOffice:In Progress by lh-maviya> <Tabuntu:Invalid by tinarussell> <Tivion:Invalid by shakaran> <Tv-Player:New> <Ubuntu:I22:15
lifelessgary_poster: I'll retitle the current bug, because it should be more clear anyway22:16
wallyworldleonardr: sorry. you don't have to. i can get the ocr to do it no problems.22:16
leonardrwallyworld: that would be better, i'm eod22:16
wallyworldleonardr: will do.22:16
gary_poster2) if there's an indication that this code path is causing timeouts, I can try to do the optimization you describe.  Even though I'm fairly skeptical of its utility, I've been wrong plenty before.22:16
lifelessgary_poster: I don't know that there are timeouts on this yet, https://bugs.qastaging.launchpad.net/bugs/230350/+bug-portlet-subscribers-content renders ok22:17
_mup_Bug #230350: Missing Debian Maintainer field <Ubuntu:Invalid> <afflib (Ubuntu):Fix Released by saivann> <alac-decoder (Ubuntu):Fix Released by warp10> <axiom (Ubuntu):Invalid> <beneath-a-steel-sky (Ubuntu):Fix Released> <bibletime-i18n (Ubuntu):Fix Released by txwikinger> <binkd (Ubuntu):Fix Released> <bzr-builddeb (Ubuntu):Won't Fix> <capisuite (Ubuntu):Invalid> <chinput (Ubuntu):Fix Released by warp10> <chmsee (Ubuntu):Fix Released> <ciso (U22:17
lifelessgary_poster: and staging is down as you noted22:17
gary_poster3) I continue to want to reduce the duplication of the SQL size as I was describing in that bug, but I want to do that with the WITH statement.  It is an easy experiment to see if an intermediate query helps in this case, though, so that's another thing we can try.22:19
gary_posterIf you want me to prepare a branch with that change for you to test with (in the 200 bugtask page), I could do so in a min of half hour and a max hopefully of 2 hours.22:19
gary_posterMeanwhile, I have to run.  Please ping me with any bugs you create22:20
gary_posterbye everyone22:20
lifelessgary_poster: ciao22:22
lifelessgary_poster: I've retitled the bug, I think high pri is appropriate and we should see how it behaves on that bug's subscriber portlet on staging when its back22:23
lifelessgary_poster: no point polishing something that is ok for now22:23
sinzuilifeless: leonardr: My bad day continues. I seem to have destroy my lp tree in an errant branching of launchapdlib22:30
sinzuiplease bear/bare/bair/bar with me as I look for where I placed my brain22:31
lifelesssinzui: ><22:32
jcsackettsinzui: i do not appear to have sound. give me a moment.22:33
huwshimisinzui: I appear to be getting very intermittent sound from everyone, restarting mumble22:34
jcsackettyay! mumble issues for everyone.22:34
huwshimisinzui: ugh this is useless22:35
sinzuihuwshimi: I called an end to the stand up. I hope we can talk tomorrow and the monkey god of software is kind22:44
huwshimisinzui: Ah ok, I'm not sure what went wrong. It might have been network issues or something22:45
wgranthuwshimi: jcsackett was having similar issues (although it eventually came mostly good), so I suspect it's not your fault.22:46
jcsackettsinzui: i found the test in question.22:46
jcsackettwgrant: huwshimi: i find mumble is frequently having issues.22:46
sinzuijcsackett: does it look sane and simple. I think I was sane that day I wrote it22:47
jcsackettsinzui: it looks fairly direct, yes.22:47
sinzuihuwshimi: wgrant: yes sound is very dodgy now. At least the new sound-indicator tells me I have not in/out.22:48
sinzuiOh, but it does not tell me when mumble + pulse will throw a wobbly and take my CPU with it22:49
wgrantsinzui: The U1 crasher is fixed now, btw.22:49
wallyworldthumper: leonard has eod. he's fine with the approach since we don't have expand and filter implemented in ws yet. can you +1 it?  https://code.launchpad.net/~wallyworld/launchpad/recipe-request-builds-existing/+merge/5238922:49
sinzuifab. I was thinking of not taking updates for few days22:49
=== Ursinha is now known as Ursinha-afk
lifelessmeh, bug text wrapping breaks my brain23:48
lifelesswgrant: hey23:49
lifeless               ->  Index Scan using securesourcepackagepublishinghistory_sourcepackagerelease_idx on sourcepackagepublishinghistory  (cost=0.00..0.50 rows=1 width=4) (actual time=0.011..0.011 rows=0 loops=1276160)23:49
lifeless                     Index Cond: (sourcepackagepublishinghistory.sourcepackagerelease = sourcepackagerelease.id)23:49
lifeless                     Filter: ((sourcepackagepublishinghistory.archive = ANY ('{1,534}'::integer[])) AND (sourcepackagepublishinghistory.distroseries = 106) AND (sourcepackagepublishinghistory.component = 3) AND (sourcepackagepublishinghistory.status = 2))23:49
lifelesswhats that filter checking for in english23:49
wgrantlifeless: Published sourcepackagereleases in universe, natty(?), primary and partner.23:50
lifelesshmm, I think this is the issue though23:50
lifeless                     ->  Index Scan Backward using bugtask_importance_idx on bugtask  (cost=0.00..116380.60 rows=110413 width=276) (actual time=9.909..2039.202 rows=3788 loops=1)23:50
lifeless                           Filter: ((distribution = 1) AND (status = 10))23:50
lifeless                     ->  Index Scan using sourcepackagerelease_sourcepackagename_idx on sourcepackagerelease  (cost=0.00..4.77 rows=21 width=8) (actual time=0.508..34.643 rows=337 loops=3788)23:50
lifeless                           Index Cond: (sourcepackagerelease.sourcepackagename = bugtask.sourcepackagename)23:51
wgrantWhat's this from?23:52
wgrantComponent bug searches?23:53
_mup_Bug #731679: distribution:+bugs timeout <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/731679 >23:53
lifelessI'm trying to figure out the intent, yes23:53
lifelessI think its 'new bugs only'23:53
lifelessbut https://bugs.launchpad.net/ubuntu/+bugs?orderby=-importance&search=Search&field.status=NEW is snappy23:54
wgrantWhich OOPS is that from?23:55
lifelesswgrant: the first one23:55
wgrantI'd say it was from DistroSeries:+bugs...23:55
wgrantoh, I see.23:57
wgrantOf course.23:57
wgrantA component filter should be all that is necessary. Have you tried that?23:58
wgrantYeah, that works.23:58
wgrantWhere "works" is "times out"23:59

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!