/srv/irclogs.ubuntu.com/2009/11/02/#launchpad-dev.txt

mwhudson  Tear down canonical.testing.layers.TwistedAppServerLayer in 3 minutes 0.938 seconds.01:03
mwhudsonthis seems excessive :(01:03
lifelessprobably swapped out01:03
mwhudsonyeah01:04
thumpermwhudson: got a few minutes to talk about test strategy for a change I ahve01:32
thumper?01:32
mwhudsonthumper: sure01:34
wgrantAs what DB user do the tests run by default?03:04
mwhudsonwgrant: launchpad03:04
wgrantmwhudson: Ah. Thanks.03:05
wgrantHm.03:05
* thumper gains even more dislike of the librarian for tests03:53
wgrantBecause it takes FOREVER to start up, or something more sinister?03:53
lifelesssingleton.slow. functional-not-unit.03:53
mwhudsonor maybe because you have to commit all over the place when using it?03:54
thumperfailed to shutdown in this instance03:57
thumperalso for those other reasons, but in this particular time, for failing to die03:58
mwhudsonthat's a lot of reasons for hate between us04:01
wgrantThere are more.04:01
* thumper EODs for dinner making04:21
mwhudsonprogramming can be terrible some times04:36
wgrantWhat's going wrong?04:37
mwhudsonoh just zope.interface crap04:38
wgrantAh.04:38
stubThe librarian could be mocked for many tests - the transaction rules are well documented and the interface simple. Nobody has bothered though (since it is a per test-run cost rather than a per-test cost?)04:50
lifelessstub: I think you mean stubbed (no pun intended) - Mocks are not suitable for keeping true to the transaction rules.04:52
* wgrant tries to remember the URL to that stubs vs. mocks. vs. fakes thing that was around a while ago.04:53
lifelesswgrant: all are test doubles; mocks assert a sequence of calls with seeded result values04:55
wgrantAh.04:55
wgrantI can never remember which is which.04:55
lifelessstubs are generally limited functionality doubles that do stuff04:55
lifelessand fakes do even less04:55
lifelessanyhow, the rule to remember is 'mocks do not meet contracts' - each and every test has its own mock, or they aren't actually mocks.04:56
lifelesson the upside you can create a mock by decorating a realobject and recording the input/output sequence it sees.04:56
lifelesswhich can be a good way to make mocks, and with a little glue, turn them on/off on a per-run basis04:57
lifelessto let one validate that the mock hasn't altered test validity asynchronously.04:57
mwhudsonit's on mumak.net somewhere i currently can't find05:01
mwhudson_so many_ places reference next_mirror_time :(05:12
MTecknologystub: hi05:14
MTecknologystub: we were just talking about you earlier05:14
mwhudsonand mostly for lame, or at least non-intent revealing, reasons05:15
stubTalking about me for lame and non-intent revealing reasons? O_o05:15
MTecknologyI decided I can either do more crappy documentation in the 30min I'll be awake, start reading an article I need to do a presentation on, tweak my bot some, install stumbleupon and use it, or..... do nothing.... Problem is that none of them sound appealing05:17
MTecknologystub: I was asking about stu and thumper thought I was talking about you. But then we did talk about you and I realized we like the same kind of ice cream05:18
stubstu flavoured ice cream? This is getting more dubious.05:19
MTecknologylol05:20
* mwhudson eods05:21
wgrantstub: Can you have a quick glance at http://pastebin.ubuntu.com/307257/ and tell me if i'm doing anything horrifyingly forbidden ? I'd really really like to get this in before 3.1.10, and need to discuss it with Soyuz people tonight. Basically, I need a way to restrict permissible source package formats by distroseries. This is it.05:25
stubwgrant: We normally use a integer/enum instead of a string where there is a fixed set of allowed options, and strings for free text. I suspect 'format' is not free text and should be an enum.05:27
stubwgrant: The table name sounds a little dubious, but that will shake out in discussions with soyuz. Is this table defining the formats that are allowed for a distroseries or the formats that have been used in a distroseries?05:29
wgrantstub: Right, I wondered about that. But to be consistent, that would mean migrating a column (sourcepackagerelease.dsc_format) from a string to an enum too.05:29
wgrantstub: It's the same scheme used for ComponentSelection and SectionSelection, but I will discuss it with Soyuz. It defines the formats allowed in a distroseries.05:30
stubWe are inconsistent in many places - if you are correct, adding a comment in comments.sql is fine rather than migrating the column now. We can't be as agile with the DB schema as we are with Python.05:30
stuba string field might be fine in this case if it is 'whatever apt or tool x tells us' rather than some value Launchpad needs to actually understand.05:31
wgrantAll values in that column should currently be identical, but it still sounds messy.05:31
wgrantIt does have to be understood.05:32
wgrantAt the moment there is only one valid value. After my branch, there will be three, defining different sets of allowed files in the sourcepackagerelease.05:32
wgrantThe formats are defined by dpkg.05:33
stubSo in general with the Python code we want to avoid "if format == '1.0'" and instead have "if format == PackageFormat.WHATEVER"05:35
stubIf it turns out in discussion that a string is the right choice for this column, I'll want some vague justification.05:36
stubBut no babies will die either way.05:36
wgrantWhat's the best way to do this? Define the DBEnumeratedType and then a dict mapping '1.0', '3.0 (quilt)' etc. to its items? Or does DBEnumeratedType have some builtin way to do this, like lazr.restful seems to use?05:45
sridhergot an error to run > make schema05:49
wgrantsridher: Which error?05:50
sridherhttp://paste.ubuntu.com/307272/05:52
wgrantsridher: Python 2.6 is not supported.05:52
wgrantsridher: Only 2.4 (and nearly 2.5, but it's not merged yet) at this point.05:53
sridherok thank you sir05:53
wgrantsridher: Did you use the rocketfuel-setup script as described on https://dev.launchpad.net/Getting?05:54
sridheryes sir05:55
sridheri got error that "developer-dependency" package not found05:56
wgrantsridher: Which version of Ubuntu are you using?05:56
sridherkarmic05:57
wgrantThat should just work.05:57
wgrantTry running rocketfuel-setup again.05:57
sridherok sir05:57
sridherPackage `launchpad-developer-dependencies' is not installed and no info is available.05:58
wgrantapt-get update05:58
sridherok05:58
sridherhttp://paste.ubuntu.com/307275/05:59
wgrantsridher: You have the launchpad PPA enabled (probably in /etc/apt/sources.list.d/launchpad-dev.list)?06:03
sridheryes sir06:04
wgrantsridher: pastebin the output of 'apt-get update'06:05
sridherhttp://paste.ubuntu.com/307278/06:06
wgrantI do not know what the problem is.06:07
wgrantUnless you are using something other than i386 or amd64.06:07
sridheriam amd6406:07
wgrantWhat is in /etc/apt/sources.list.d/launchpad-dev.list?06:08
sridherdeb http://ppa.launchpad.net/launchpad/ppa/ubuntu karmic main06:09
sridherdeb http://ppa.launchpad.net/bzr/ppa/ubuntu karmic main06:09
wgrantThat's fine.06:09
sridheris it work with python 2.4?06:13
wgrantsridher: Yes. I presume you modified your copy to use python2.6 manually?06:14
wgrantal-maisan: Morning.06:14
al-maisangood afternoon wgrant, how are things?06:14
sridherafter getting an 2.4 error i changed /devel/Makefile version to 2.606:16
wgrantal-maisan: Not too bad. Just finishing off my v3 source format branch. One thing I came across in the tests was that the 'absolutely-anything' upload policy doesn't accept PPA uploads -- do you know if this is deliberate? I need it to for my tests, and enabling PPA uploads in it doesn't break other tests in lp.archiveuploader.06:16
wgrantBut who knows what other tests might lurk elsewhere.06:16
al-maisanwgrant: I don't know off-hand but will take a look.06:17
wgrantal-maisan: (this branch recently became very important, as Debian has started uploading 3.0 (quilt) sources)06:18
al-maisanwgrant: gotcha.06:18
al-maisanwgrant: would the importance be diminished in any way of we sunc'ed lucid from testing as opposed from unstable?06:19
al-maisans/of/if/06:20
wgrantal-maisan: That would delay it by ten days at most, and with the current backlash it's unclear exactly what is going to happen to Lucid syncs.06:21
al-maisanwgrant: I see.06:21
al-maisanwgrant: re. the 'absolutely-anything' upload policy I find this snippet in AbstractUploadPolicy.rejectPPAUploads():06:29
al-maisan        """Reject uploads targeted to PPA.06:29
al-maisan        We will only allow it on 'insecure' and 'buildd' policy because we06:29
al-maisan        ensure the uploads are signed.06:29
al-maisan        """06:29
al-maisanwgrant: why would it be so important for PPA uploads to be signed?06:29
wgrantal-maisan: Seems like absolutely-anything (only used in tests) should be safe, however.06:29
sridherhttp://paste.ubuntu.com/307284/06:29
wgrantal-maisan: I don't know. It works OK once I enable it in the policy, so the code is fine with unsigned uploads.06:30
wgrantThere is some special signature treatment with PPA uploads (stripping signatures from stored .changes), but that seems to handle a lack of signature fine.06:30
al-maisanwgrant: if the policy in question is only used in tests then I guess "It's better to beg forgiveness than ask permission" applies ;)06:31
wgrantal-maisan: OK. Thanks.06:32
noodles775Morning06:48
wgrantMorning noodles775.06:49
noodles775Hi wgrant :)06:53
wgrantnoodles775: Almighty RM, I need your advice.06:53
noodles775thumper: your branch is rc-approved... (just in case you didn't see the email).06:54
noodles775wgrant: for the v3sources?06:54
wgrantnoodles775: Right. Debian has started uploading them, so it's pretty important no.06:54
wgrant+w06:54
noodles775yep, definitely. But it's also the kind of thing that I'd like to have tested on dogfood thoroughly before r-cing, which means it might be best as a CP... but bigjools will be around later...06:55
wgrantnoodles775: The branches are pretty much done (save a conversion from a string to an enum in one place, which I'm finishing off now), but they're bigish.06:55
wgrantRight.06:55
wgrantBut there are DB changes (safe) that are needed.06:56
noodles775He'll be able to check your branch on df...06:56
noodles775ouch06:56
noodles775btw: Thanks so much for gettin in there and fixing those!06:56
wgrantBut assuming those get into 3.1.10, there are CPs required on Soyuz machines and appservers.06:56
wgrantPlus somebody needs to get the new dpkg on the buildds and Soyuz machines... not sure who to talk to about that.06:57
wgrantBut lucid won't open for a few days, and the v3 packages can be survived without for a while, so that's not quite as critical as the DB changes.06:57
noodles775right. Have you got a branch with the db changes ready for review?06:58
wgrantNot on its own yet, as I need to convert a column to an enum.06:58
wgrantThere is a new table, plus some new enum values on SourcePackageFileType that will cause appservers to explode if they don't know about them.06:59
wgrantappservers and builddmaster, that is.06:59
noodles775OK. So assign the db changes to stub+jml, bigjools can take a look when he's in and plan the best course of action.07:01
wgrantWill do. Thanks.07:02
noodles775Thank you!07:02
wgrantnoodles775: Any recommendations for what to rename the table listing permitted formats by distroseries? I've currently called it SourceFormatSelection (because of ComponentSelection and SectionSelection), but I'm sure that's suboptimal.07:05
noodles775wgrant: re-name?07:07
wgrantnoodles775: Well, to you it's just naming it.07:07
noodles775:)07:07
=== noodles775 changed the topic of #launchpad-dev to: This is Launchpad Development Channel | Week 4 of 3.1.10 Release Critical - release manager is noodles775 (flacoste, BjornT)| I am Zero OOPS and So Can You! http://is.gd/4fkLl | PQM is OPEN | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in #launchpad-reviews | Use http://paste.ubuntu.com/ for pastes | This channel is logged: http://irclogs.ubuntu.com/
=== noodles775 changed the topic of #launchpad-dev to: This is Launchpad Development Channel | Week 4 of 3.1.10 | PQM is release-critical Release Critical - release manager is noodles775 (flacoste, BjornT) | I am Zero OOPS and So Can You! http://is.gd/4fkLl | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in #launchpad-reviews | Use http://paste.ubuntu.com/ for pastes | This channel is logged: http://irclogs.ubuntu.com/
=== noodles775 changed the topic of #launchpad-dev to: This is Launchpad Development Channel | Week 4 of 3.1.10 | PQM is release-critical only - release manager is noodles775 (flacoste, BjornT) | I am Zero OOPS and So Can You! http://is.gd/4fkLl | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in #launchpad-reviews | Use http://paste.ubuntu.com/ for pastes | This channel is logged: http://irclogs.ubuntu.com/
noodles775wgrant: SourceFormatSelection seems fine to me - if it's suboptimal, it's because of the other columns that you've based it on. The db reviewers will be in a better position to comment.07:10
wgrantnoodles775: Well, stub said to ask Soyuz...07:10
noodles775wgrant: ah ok. so, as I said, SFSelection seems fine to me. bigjools will comment later I'm sure .07:11
thumpernoodles775: ta07:21
wgrantWhat's the Storm equivalent of an EnumCol?07:28
wgrantStormMigrationGuide has that as an unanswered question down the bottom.07:29
noodles775wgrant: I haven't used it, but http://twistedmatrix.com/users/radix/storm-api/storm.properties.Enum.html07:36
noodles775gr, no, that's not a db enum.07:36
wgrantLooks like it is.07:37
wgrantI see it used for that purpose in PackageCopyRequest, with a utility to convert the DBET.07:37
noodles775Great.07:37
wgrantThanks.07:37
noodles775np.07:37
wgrantnoodles775: Looks like canonical.database.enumcol.DBEnum is what I actually want, as it supported DBETs directly.07:40
noodles775wgrant: good to know.07:41
noodles775wgrant: can you update the migration guide (if you haven't already). Thanks!07:41
wgrantnoodles775: Will do.07:41
adeuringgood morning08:23
wgrantCan I look up a DBItem given a DBEnumeratedType and the item's title?08:25
al-maisanGood morning adeuring08:26
adeuringhi al-maisan!08:27
noodles775wgrant: why not use the actual enum? Like PackagePublishingStatus.PENDING? (but you can do PackagePublishingStatus.getTermByToken('Pending') too?).08:37
wgrantnoodles775: I need to convert a string representation of a format from the .dsc (eg. '3.0 (quilt)').08:38
wgrantI'll try that.08:38
noodles775Great.08:38
wgrantI thought that would go the other way.08:38
wgrantOh, woops. Was wondering why my enum was throwing strange SyntaxErrors. Then realised the names started with digits.08:48
wgrantFORMAT_1_0 rather than 1_0, perhaps?08:48
bigjoolswgrant: good <insert time of day here>, I hear you're working on https://bugs.edge.launchpad.net/soyuz/+bug/29310608:58
mupBug #293106: does not support debian v3 source formats <feature> <motu> <ppa> <soyuz-upload> <Soyuz:In Progress by wgrant> <https://launchpad.net/bugs/293106>08:58
wgrantbigjools: Good morning. I am. Shall we discuss the plan?08:58
bigjoolsplease do08:58
bigjoolsis it blocking anything in the distro that you know of?08:59
bigjoolsI assume some v3 packages need to be synced now?08:59
wgrantNot right now, but Lucid will open in a few days, and the first v3 packages have appeared in Debian.08:59
wgrantThen it will be blocking things, although not critically to start with.08:59
wgrantNow, the code is done and works.08:59
bigjoolsok can this bump to the 3.1.11 milestone?08:59
wgrantI don't think it can.08:59
bigjoolsmassive changes 2 days before a release make me very nervous09:00
wgrantHowever, if we get some safe bits in now, the rest can be CPed to Soyuz machines at your leisure.09:00
bigjools /o\09:00
bigjoolsI love coming back from holidays!09:00
wgrantAll that needs to be applied everywhere is a new table, a few new SourcePackageFileType values, a trivial function on DistroSeries, and four lines in CopyChecker.09:01
bigjoolswgrant: what have you done to fix it?09:01
wgrantThe big invasive changes are only needed on cesium, germanium and cocoplum.09:01
bigjoolshave you got a branch I can see?09:02
wgranthttps://code.edge.launchpad.net/~wgrant/launchpad/distroseries-source-format-selection has it all. I'm just finishing off a change recommended by Stuart.09:02
bigjoolsand why can't it wait a month, do you expect a lot of v3 packages?09:02
wgrantI don't know. But distro people were eeking about it, and Lucid will freeze very early.09:03
wgrantI don't think anyone can say how many v3 packages will appear in the next month and a bit.09:03
bigjoolsis there a workaround to sync them if we can't upload v3?09:04
wgrantThey should be convertible to 1.0 with a little effort, I guess.09:05
bigjoolswgrant: why the new table?09:08
wgrantbigjools: To restrict which series can have the new formats.09:08
bigjoolswhy is that needed?09:08
bigjoolsI don't know anything about the new format BTW :/09:09
wgrantbigjools: Only >= karmic's dpkg supports the new formats.09:09
bigjoolsok09:09
wgrantSo the old chroots will be very unhappy (and maybe chrootwait; I'm not quite sure)09:09
al-maisancould we control the source format selection w/o a schema change, at least initially?09:12
al-maisani.e. by a piece of python logic that ascertains "Only >= karmic's dpkg supports the new formats"09:12
wgrantYou could hardcode the series name, I guess.09:13
bigjoolsew09:13
wgrantYes.09:13
al-maisan.. and clean up things in 3.1.1109:13
bigjoolsI'm not sure what that would buy us, it's still an invasive change09:14
al-maisanno question about that09:15
al-maisanbut we'd at least avoid "last minute" db schema changes09:15
wgrantAbsolutely awful timing, I know, but Debian only revealed its intentions at the end of last week :(09:16
bigjoolsI am still -1 on this going in unless someone from distro steps in and says it must09:18
wgrantRight.09:18
wgrantI'm talking to them now.09:18
bigjoolswhere "someone from distro" = cjwatson :)09:18
bigjoolsthis is a 4k diff branch - that's just insane to put in at this stage09:19
bigjoolserr wait09:19
bigjoolsdiff fail09:19
wgrantLast I checked it was 1.6klines09:19
al-maisan1581 lines of diff09:19
al-maisanstill a massive change09:19
wgrantAnd a tiny fraction of that is needed *right now*.09:19
wgrantBut yes, it is big.09:20
bigjoolsok now I am diffing against db-devel :)09:20
wgrantAhaha.09:20
wgrantThat would do it.09:20
bigjoolswgrant: your sample data change changes the archive table, why?09:21
wgrantbigjools: external_archive_deps change wasn't in there previously.09:21
bigjoolsah you can blame me for that09:21
wgrantYup :P09:21
wgrantSourceFormatSelection is changing to an enum at the moment.09:22
bigjoolsyou're removing the table?09:22
wgrantNo.09:22
wgrantIt's just going to use an enum rather than a string for the format.09:23
bigjoolswgrant: from canonical.database.enumcol import DBEnum09:24
wgrantbigjools: By that I mean I'm rerunning the tests after completing the change.09:25
bigjoolswgrant: what's different with the 3.0 format then?09:28
wgrantbigjools: New types of files.09:29
wgrantAnd massive incompatibility.09:29
wgrantAnd new compression formats.09:30
wgrant(although we're following dak and not supporting lzma, only gzip and bzip2, at cjwatson's direction)09:30
bigjoolslzma dropped entirely?09:31
bigjoolsinteresting09:31
wgrantRight. Trivial regex change to bring it back if we need to.09:31
wgrant(lzma deb support remains)09:31
wgrantOh. Curses. SourcePackageFormat already exists.09:33
wgrantAny suggestions on what to call it?09:34
bigjoolsSourceUploadFormat?09:35
wgrantThat's even more of a lie than the old SourcePackageFormat.09:36
wgrantIt's a property of the source package, and nothing really to do with the upload.09:36
bigjoolsyeah09:37
* bigjools -> OTP09:37
wgrantOTP?09:38
wgrantThis #ubuntu-devel is going to get messy!09:39
wgrant+discussion09:39
wgrant(my best idea is to rename the old SourcePackageFormat to SourcePackageType or something like that, as it is only mentioned on 7 lines, but I guess that would get me even more murdered than I will anyway)09:43
bigjoolswgrant: ok back (On The Phone BTW)10:06
wgrantbigjools: Oh, right.10:06
bigjoolswgrant: +1 on renaming SourcePackageFormat10:06
wgrantbigjools: Great, 'cause I've done it.10:07
bigjoolsheh10:07
bigjoolsSourcePackageType ?10:07
wgrantThat's the one.10:07
bigjoolsperfecto10:07
bigjoolsone day we might even start using it for something useful and support RPM10:07
wgrantPerhaps.10:07
wgrantHm.10:10
wgrantFS corruption.10:10
wgrantThat is not good.10:10
wgrant(product.py started complaining of SyntaxErrors, when I hadn't touched it. it is now full of garbage)10:11
wgrantI wonder what else just got eaten.10:11
bigjoolsext4? :)10:14
wgrantindeed.10:14
wgrantBut the file size changed.10:14
wgrantSo it's not the OMG KITTENS ARE DIEING BUT ONLY THREE PEOPLE HAVE REPORTED IT bug.10:15
wgrantbigjools: Can I safely raise an UploadError in one of NU's error generators if I want to reject the upload and stop right there, or do I have to move the check to another function?10:32
bigjoolsgood question10:32
* wgrant traces it all the way up.10:32
bigjoolssuck it and see I guess10:32
wgrantJust this last failure case to handle...10:33
bigjoolswgrant: EarlyReturnUploadError10:33
wgrantbigjools: Ah, handy. Thanks.10:34
wgrant(NU is too convoluted)10:34
bigjoolsno kidding10:34
bigjoolsit was refactored big-time a couple of years ago10:35
wgrantI recall.10:35
wgrantBut that didn't get finished.10:35
bigjoolsand it improved things a lot, if you can beloeve that10:35
wgrantI cannot.10:35
bigjools:)10:35
wgrantMaybe I will revert the tree later and realise how lucky I am.10:35
jmlahh, nascentupload10:37
wgrantjml: You were privileged to be able to experience it before leaving engineering.10:37
jmlwgrant, indeed.10:38
jmlwgrant, I hope I made an improvement10:38
* wgrant isn't touching those bits.10:38
wgrantNU might not be so bad if it was properly tested, too.10:41
deryckMorning, all.11:00
wgrantbigjools: OK, it seems that most of the rest of my FS survived. I've just finished off that change.11:10
bigjoolswgrant: most of? :)11:12
wgrantbigjools: Well, I've only found that one eaten file. All the packaged files check out fine, my branches are fine, and fsck is happy, so I presume most of it is OK.11:13
wgrantBut it seems odd that there would be only file, given that I haven't written to it.11:13
bigjoolsyes11:14
wgrantbigjools: Broken sample data is currently being fixed :/11:52
bigjoolswgrant: what's broken?11:53
wgrantbigjools: There is a firefox-0.9.2.orig.tar.gz used in tests (note the '-' in place of '_'). That previously worked, as the type check just looked for endswith('.orig.tar.gz'). But now it uses a regex which needs a '_'.11:54
bigjoolswgrant: this is in the data/suite/* stuff?11:55
wgrantbigjools: No, no. archiveuploader would never accept that. This is in a doctest that grabs an existing LFA and SPR.addFile()s.11:56
bigjoolsah ok11:56
* wgrant directs significant volumes of hate at even more crackful sampledata.12:17
leonardrjml, you seem to know a lot about rickspencer3's whereabouts. do you know where in the world is rickspencer3 today?12:29
jmlleonardr, if I had to guess, I'd say he's in the west coast of the continental united states of america.12:30
leonardrhmm12:30
leonardri will hunt him down!12:30
jmlleonardr, I only half-followed your recent conversation with al-maisan -- is exposing a dict over the API really that tricky?12:45
* al-maisan cocks up one ear12:45
leonardrjml: it's never been done before. making the lazr.restful changes to expose a dict should not be *that* difficult12:45
leonardrbut al-maisan wanted to expose a dict _of lists_12:46
jmlleonardr, why is that important?12:46
wgrantWasn't it a dict of Entries?12:46
leonardral-maisan can correct be, but i believe it was a dict that mapped a person to a list of strings12:47
jmlthere are now two interesting branches of this conversation :)12:47
jmlleonardr, isn't a dict of lists a fairly standard json construct?12:48
al-maisanleonardr: actually it was a dict {pocket : object } .. let me check again12:48
jmlal-maisan, it was mapping a pocket enum to a branch object iirc.12:48
al-maisanthat would be right12:48
leonardri don't remember this. i remember people mapped to the types of reviews they'd been assigned to12:49
leonardranyway, the problem is that the arguments to a named operation have to be described in wadl, and recursively defined data types are difficult to define in wadl12:50
jmlleonardr, ahh, I see.12:50
leonardra recursively defined data type is a sign that things are getting too complicated and you should rethink your resource design12:50
jmlleonardr, I can see the point12:51
wgrantAhhhh. Tests pass a little better when one of the data files used in a few tests *hasn't* been corrupted by a stupid filesystem.12:51
jmlas much as I like "for your own good"-style design.12:51
jml*dislike*12:51
jml(one of them Freudian mothers)12:51
al-maisanleonardr, jml: this is what I was trying at that time: http://pastebin.ubuntu.com/300148/12:52
leonardrjml: what i've said is kind of seat-of-pants stuff. my knowledge of that part of lazr.restful is cached on disk.12:52
jmlleonardr, heh12:53
=== henninge is now known as henninge-lunch
=== Ursinha-afk is now known as Ursinha
leonardrsomewhat literally, in that i would have to go into the code and re-learn it12:54
al-maisanleonardr: : so, the problematic part in this case was the return type of the collection returned.12:54
=== noodles775_ is now known as noodles775
al-maisananyway, since that did not work I resorted to: https://code.edge.launchpad.net/~al-maisan/launchpad/35209412:55
=== barry` is now known as barry
jmlleonardr, I know the feeling.13:30
=== bigjools is now known as bigjools-lunch
=== henninge-lunch is now known as henninge
mptWhat's a "package set"?14:17
mptI see <https://dev.launchpad.net/Soyuz/Specs/PackagesetsAndDistroseries> and <https://dev.launchpad.net/Soyuz/Specs/PackagesetsAndDistroseries/OriginalAnalysis>, but they don't seem to mention what a package set is14:19
al-maisanmpt: https://dev.launchpad.net/VersionThreeDotO/Soyuz/StoryCards#packagesetacl14:20
mptahh, security14:20
mptthanks al-maisan14:20
al-maisanyou are welcome :)14:21
Ursinhahey bigjools-lunch :) can you tell me what's exactly the meaning of the pending milestone you have in soyuz? after getting back from lunch, of course :P14:21
=== bigjools-lunch is now known as bigjools
bigjoolsUrsinha: one word: "Celso"14:32
Ursinhabigjools, oh crap14:32
Ursinhabigjools, but do you know his rationale behind it?14:32
bigjoolsUrsinha: it was a place for him to put stuff he didn't want to forget about14:32
Ursinhabigjools, I see14:32
Ursinhabigjools, you're not using it?14:32
bigjoolsand it turned into a general pool of bugs that we need to sort out14:33
bigjoolssort of14:33
Ursinhabigjools, I see14:38
Ursinhabigjools, I asked because I have some bugs that are depending on others to be fixed, not our bugs, and as we don't have a Pending status I considered using a Pending milestone14:39
bigjoolsUrsinha: as you know, things have changed focus recently so it's fallen to the side a little.  But its general concept will be re-used for LP generally soon.14:39
bigjoolsah14:39
bigjoolsgo for it14:40
Ursinhacool14:40
Ursinhathanks bigjools :)14:40
bigjoolsany time14:40
gary_posterbigjools: hi!  I have this on my QA page from Francis.  Am I right that I can push it over to your guys' page? "r9807 [r=adeuring][ui=none][bug=458833] Protect Build attributes using launchpad.Edit. (Landed on behalf of Julian.)"14:44
bigjoolsgary_poster: yes please do14:45
gary_posterbigjools: cool thanks14:45
bigjoolsgary_poster: are you going to stick it on my QA wiki page?14:49
gary_posterbigjools: sorry yes.  Trying to reply to mthaddon on a nother channel then was going to do that.  If you want to do it before me, feel free. ;-)14:50
bigjoolsgary_poster: yeah I can do that, what's your test plan URL?14:50
gary_posterbigjools: thanks https://dev.launchpad.net/FoundationsTestPlan/3.1.1014:51
bigjoolsshoulda guessed that really :)14:51
gary_poster:-)14:51
bigjoolsgary_poster: umm release yer edit lock :)14:52
gary_posterbigjools: sorry done14:52
bigjoolsta14:52
bigjoolsdone14:52
wgrantbigjools: Have you a definitive answer from cjwatson?15:13
bigjoolswgrant: not definitive, but there's no clear requirement that I can see for putting this in 3.1.1015:13
bigjoolswgrant: if the default dpkg format changes then he points out that we're fucked15:14
wgrantbigjools: Right. And that was planned to happen.15:15
wgrantBut might not be any more... hard to tell.15:15
sinzuibac: EdwinGrubbs: barry: I was able to test the product-release-finder. 1 bug is fixed, the other is not. I am going to work on the bad item15:15
bigjoolswgrant: did stub review your db changes15:16
wgrantbigjools: He commented that the table name was a bit odd, but that it was a Soyuz decision, and also that it should use an enum rather than a string.15:17
wgrantThe latter is done, and you seem fine with the former.15:17
bigjoolswgrant: yeah it follows some convention I guess15:18
bigjoolswgrant: so I am thinking that we should land the schema changes only, in case the brown smelly stuff hits the fan this month15:18
wgrantThe cut down (that is, without the big archiveuploader changes) diff (excluding sample data) is ~500 lines.15:18
barrysinzui: cool15:20
bigjoolswgrant: we don't even need that, we can just land the sql patch on db-devel15:20
bigjoolsthat's the thing that would stop a CP if it came to that, so this increases our options15:20
wgrantTrue.15:21
bigjoolsso if you can make a separate branch with that, create an MP, get jml and stub to review it, I'll land it for you15:22
=== james_w` is now known as james_w
=== deryck is now known as deryck[lunch]
bacsinzui, barry: here are the portlets called on the distroseries page:  http://pastebin.ubuntu.com/307675/16:48
bacsinzui, barry: for karmic, all render but the third one:  https://edge.launchpad.net/ubuntu/karmic/@@+table-milestones16:48
sinzuibac: I bet it is the status counts for bugs and blueprints16:49
sinzuibac: barry: The counts are a loop in python, Maybe the code can be made faster, or we can change the model to provide the summary. I prefer the first option because the code is also used to count assignees16:50
sinzuips. you rock bac16:51
=== beuno is now known as beuno-lunch
Ursinhahi sinzui, about the BAD items, I see you're working on bug 415595. do you know about the others?17:25
mupBug #415595: Product release finder should import all files <story-product-release-finder> <Launchpad Registry:In Progress by sinzui> <https://launchpad.net/bugs/415595>17:25
sinzuiWe are working on all bad items17:25
* sinzui has a fix for Bug #41559517:26
mupBug #415595: Product release finder should import all files <story-product-release-finder> <Launchpad Registry:In Progress by sinzui> <https://launchpad.net/bugs/415595>17:26
=== deryck[lunch] is now known as deryck
=== beuno-lunch is now known as beuno
Ursinhasinzui, right! in your opinion, any of these are release blockers?18:11
kfogelhttp://paste.ubuntu.com/307767/18:11
kfogelanyone know why my rocketfuel-get is failing as per above paste?18:12
barrysinzui: i'm back now; what can i look at?18:12
kfogel"bzrlib.errors.BzrError: xmlrpc protocol error connecting to https://xmlrpc.edge.launchpad.net/bazaar/: 502 Bad Gateway" ?18:12
sinzuiUrsinha: I think the timeouts on distroseries defintely are18:12
sinzuiUrsinha: the release finder could slip18:13
Ursinhasinzui, right18:13
sinzuiThe bad CSS must be fixed or we can revert the entire branch18:13
kfogeljml: curious if you recognize the error (see my paste a few lines above)18:16
jmlkfogel, it rings a bell. are you behind a proxy?18:17
kfogeljml: indeed I am.  But I could easily not be.18:17
jmlkfogel, try that.18:18
Ursinhasinzui, are you working in the old bug numbers or have you opened new ones?18:28
Ursinhaby you I mean registry team18:28
sinzuiUrsinha: I reopened the old because because our changes do not represent a partial fix18:29
bacbarry: ping18:38
barrybac: pong18:39
Ursinhasinzui, and the others?18:42
sinzuiI am testing the one that I can test.18:43
sinzuiUrsinha: I do not know how to test salgado's change. I am not even sure it is a regisrty issue18:43
=== danilos is now known as danilos-afk
sinzuibarry: ping18:47
Ursinhasinzui, actually I'm talking about the BAD items you say registry team is working on. I see you reopened the bug of yours, want to know about the others :)18:47
sinzuiThey are inprogress I think18:48
barrysinzui: pong18:48
sinzuiUrsinha: https://edge.launchpad.net/launchpad-registry/+milestone/3.1.1018:48
sinzui^ sort by status18:48
* Ursinha looks18:48
sinzuibarry: I was going to ask for you help with new member + mailing list tests, but I get an oops with my test candidate: https://staging.launchpad.net/~haibunku18:49
Ursinhathanks sinzui18:49
barrysinzui: that oops looks related to my team privacy portlet change18:51
sinzuiyes18:51
barrysinzui: like. some code didn't get updated perhaps?18:51
sinzuibut I saw your style fix on edge it it worked18:52
barrysinzui: does it work in lp.dev?18:52
barry(i should try it)18:52
sinzuibarry: staging is borked19:02
barrysinzui: gotcha19:02
sinzuibarry: db-deel has the right template, but not the right view19:02
sinzuibarry: did you land the fix in two branches?19:03
* sinzui sees the visibility_portlet_class in devel but not db-devel19:03
kfogel_jml: urgk.  this one ring a bell to you? http://paste.ubuntu.com/307835/  "bzrlib.errors.BzrCheckError: Internal check failed: Cannot add revision(s) to repository: missing chk node(s) for parent_id_basename_to_file_id maps"  (I've gotten out from behind the proxy now.)19:03
barrysinzui: i don't think so.  i had two bugs each with a branch but that particular feature should have landed in one branch19:03
barrysinzui: oh, that's sick19:03
jmlkfogel_, umm, no, sorry.19:04
jmlkfogel_, that's #bzr territory19:04
barrysinzui: i'm grabbing db-devel now19:04
sinzuibarry: ../db-devel/lib/lp/registry/browser/person.py line 3248 should be our missing method19:05
barrysinzui: how did registry/templates/team-index.py get the change but not browser/person.py?19:07
barrysinzui: oh wait19:07
sinzuibarry: gremlins from the kremlin? Aliens for Mars? The Illuminati?19:08
barrysinzui: db-devel's team-index.pt did not get the change19:08
sinzuioh, I revered the issue?19:08
kfogel_jml: thought probably -- but you did so well with the last question that I wanted to ask :)-.19:09
barrysinzui: i just bzr-tool-grep'd db-devel.  there's no visibility_portlet_class in the code at all19:09
barrysinzui: so, wtf is staging running?!19:09
=== EdwinGrubbs is now known as Edwin-lunch
Ursinhahi noodles775, are you there?19:52
Ursinhahmm, no19:52
Ursinhais it just me or edge is really slow?19:53
rockstarUrsinha, I've noticed it's really slow as well.19:53
Ursinharockstar, maybe we should poke a losa19:54
rockstarUrsinha, yeah, maybe.  Forking to do that is not something I have bandwidth for right now though.19:55
Ursinhahehe19:55
Ursinhambarnett, hey19:55
mbarnettUrsinha: heya19:56
mbarnettUrsinha: we are in the midst of a rolling update19:56
mbarnettUrsinha: which is why edge is a bit slow19:56
Ursinhambarnett, oic19:56
Ursinharockstar, ^19:56
Ursinhambarnett, for how long will it be like this?19:57
mbarnettUrsinha: i'll check on the progress19:57
Ursinhambarnett, cool19:57
beunobarry, can you think of any reason why we wouldn't want to expose what mailing lists people are subscribed to on ther profiles?19:58
ChexUrsinha: sorry, one of the edge workers was slow to come up, should be OK now, let me know if it isn't?19:59
mbarnettwhat he said!19:59
barrybeuno: for public lists?  i think it would be interesting to know that19:59
Ursinhasure mbarnett and Chex19:59
Ursinhathanks both19:59
Ursinha:)19:59
beunobarry, perfect answer19:59
barrybeuno: facepad!19:59
thumperwhy has no-one booted buildbot into touch?19:59
thumperthe db_devel builder is still fubared20:00
thumperand we're in testfix because of it20:00
thumperit has been over 12 hours!!!!!!20:00
beunobarry, don't reveal my secret plan!20:00
barrybeuno: as long as it's not spacepad :)20:00
beunobarry, do we store ML messages on the db at all?20:02
barrybeuno: in general, no.  we only store ones that are held for moderation20:03
jmlthumper, hi20:03
thumperjml: hi20:03
beunobarry, how complicated would it be to store subjects and authors?20:04
jmlthumper, up for a skype call?20:04
thumperjml: how long do you need?20:04
thumperjml: have normal stand up soon20:04
jmlthumper, ~30 mins. When's your standup?20:05
thumperin 10 minutes20:05
barrybeuno: you'd probably also want message-ids.  i think not complicated, but i don't know how expensive it would be (performance- or space- wise)20:05
beunobarry, and date  :)20:05
barrybeuno: you probably just want to store successfully posted messages, so you'd need to hook into that process, but i think not hard to do20:06
barrybeuno: yep20:06
beunobarry, so we could tell people what kind of traffic a ML has, and what the latest messages where20:06
beunobarry, secondly, are there any good proprietary archiving softwares?  (you've said there aren't any open ones)20:07
barrybeuno: well, if you did that, you'd /have/ to give them links to those messages.  /me remembers our last conversation20:07
jmlthumper, is after the standup ok?20:07
thumperjml: yeah, I think so20:07
barrybeuno: i'm not familiar with any commercial archivers20:07
jmlthumper, cool. can you ping me please?20:07
thumperyep20:08
beunobarry, ok, thanks20:08
mwhudsonmorning20:10
james_wmorning mwhudson20:14
james_wmwhudson: thanks for your work on branch-distro.py20:14
mwhudsonjames_w: np, is everything working ok for you?20:14
james_wseems to be fine20:15
mwhudsoncool!20:15
thumperrockstar: want to host?20:16
rockstarthumper, sure.20:16
rockstarmwhudson, abentley, skype?20:17
mwhudsonrockstar: yep20:17
Ursinhahello gary_poster, could you triage bug 423447, please?20:26
mupBug #423447: Trying to add an email that's already registered as a SSO account but not a person, oopses <oops> <Launchpad Foundations:New> <https://launchpad.net/bugs/423447>20:26
gary_posterUrsinha: a bit busy on buildbot right now20:29
Ursinhaok gary_poster, no problem20:29
thumperjml: ready20:32
jmlthumper, oh sorry20:42
gary_posterUrsinha: triaged20:43
Ursinhagary_poster, thank you20:44
maxbgary_poster: Hi! Good news: ztk-2.5 still fine as of my weekend re-test. Bad news: There's a new test, xx-resetpassword-for-sso-account.txt, on devel that *requires* canonical-identity-provider to pass - bit of a downer for me!21:18
maxb(sorry to pick on you, but salgado isn't here, and it was [r=gary] :-) )21:19
gary_postermaxb: heh and urg21:19
maxbLet me know if you'd like it filed as a bug21:20
gary_postermaxb: I'm swamped ATM.  Make a bug for me, and I'll assign it?  Yes please21:20
maxbFiled as bug 47172421:27
mupBug #471724: Test lib/lp/registry/tests/../stories/foaf/xx-resetpassword-of-sso-account.txt fails if canonical-identity-provider absent <Launchpad Foundations:New> <https://launchpad.net/bugs/471724>21:27
wgrantjml: Can you please review https://code.launchpad.net/~wgrant/launchpad/distroseries-source-format-selection-db/+merge/14304?21:52
jmlwgrant, will do.21:58
jmlwgrant, if not tonight, first thing tomorrow21:58
wgrantjml: Thanks.21:59
jmlwgrant, np22:00
marssilly question: do we still use the lpwindmill.py script for windmill test authoring?22:15
marsif so, where may I find the documentation on the wiki?22:15
marsrockstar, if you are still around: have you written any windmill tests recently?22:19
rockstarmars, I've ported some, am probably going to write some before the sprint, why?22:45
marsrockstar, I've been trying to test the YUI 3 upgrade, and was going to run the windmill suite.  However, I can't run windmill, or start the test authoring tools22:45
rockstarmars, how are you doing it?22:45
marsand I can't find anything about writing windmill tests on the wiki22:46
rockstarmars, I think we need to update them.22:46
marsrockstar, $ bin/py lib/lp/scripts/utilities/lpwindmill.py firefox http://launchpad.dev:808522:46
rockstarmars, if you'd like, I can call you and get you up to speed on the new way of running windmill.22:46
rockstarmars, yeah, it's changed recently.22:46
marsrockstar, that's ok, I don't have skype.  Windmill starts using bin/test, but I'm wondering about authoring22:47
marsok22:47
rockstarmars, no idea how to use the authoring tools.  I've never used them.22:47
marsrockstar, how did you write windmill tests in the past?22:47
rockstarmars, vim22:48
marsrockstar, I mean, did you use the lpwindmill script to set up a test server on port 8085, then start poking at the DOM with the windmill interface?22:49
rockstarmars, no, I just had my dev server and firebug, and would write the test, run it, fix it, etc.22:49
marsok22:50
marsrockstar, after upgraded to Karmic, I'm getting SSL errors when I try to run the windmill suite.  It looks like the bin/test suite is not using port 8085, at least, not to log into the site22:50
rockstarmars, turn off your dev server when you run the tests.22:51
rockstarWindmill and the dev server have never really played well.22:51
marsI thought we fixed that :(22:51
thumpermwhudson: what do you need to qa your last two items?22:51
rockstarmars, not that I've ever seen.22:51
marsI'll try again, doubly sure22:51
mwhudsonthumper: oh i did one22:51
mwhudsonthumper: and the other i need some queries running i guess22:51
mars"Error in sys.excepthook:"  oops.22:52
mars:)22:52
wgrantCan I access the API anonymously yet without patching waddlib to use the browser-accessible root?22:53
marswgrant, you used the word "yet" - have you heard of  a plan to allow such access?22:56
marsjust curious22:56
marsI've been away for a while22:56
rockstarwgrant, I'm not aware of any work done.22:56
wgrantmars: Well, I filed a bug about it. It was scheduled for 3.0 (I think). It didn't happen.22:56
mwhudsondo we still have the problem where when a query times out the time for the query that tripped the limit is not added to the sql time in the oops report23:02
mwhudson?23:02
wgrantI thought that was fixed for 3.0.23:02
wgrantBut I was about to ask about that.23:02
mwhudsonyeah i thought so too23:03
marsrockstar, you were right, was the dev server running as well23:05
wgrantmwhudson: What was the last query?23:06
rockstarmars, yeah, the biggest problem with Windmill is knowing all its quirks.  Once you know all the errors it shows, you know why you made it mad.23:06
rockstarmars, this is also a great example of Stockholm syndrome.23:06
mwhudsonwgrant: select person.* from BugSubscription, Person WHERE Person.id = BugSubscription.person AND BugSubscription.bug = 417842 AND (1=1) ORDER BY Person.displayname23:06
mwhudsonwgrant: unless there is a truly insane number of subscribers that doesn't seem like it should be that slow23:07
marsrockstar, LOL23:07
wgrantmwhudson: That shouldn't be too bad, but I have to wonder why it is executing that at all.23:09
wgrantmwhudson: Since that is loaded via AJAX in a portlet later.23:09
mwhudsonwgrant: yeah23:09
marsrockstar, the funny thing is, I know *exactly* where this tool fits into the big picture now.  I can start thinking of ways to make it better23:10
rockstarmars, cool.  Let me know how I can help.23:10
mwhudsonwgrant: that query is fast apparently so who knows what's going on23:19
wgrantmwhudson: Using the very handy ++oops++ locally, I can see some pretty bad looking queries (selecting all bugsubscriptions for bugs that are duplicates of the current bug, and prejoining all people referred to by those subscriptions)23:21
wgrantThere is absolutely no reason to do that.23:21
mwhudsonwgrant: indeed, but they're acutally pretty quick23:22
wgrantHm. sad.23:22
mwhudsonso it's bad, but not the problem23:22
mwhudsonwgrant: that terribly looking prejoin query takes 77.0 ms23:23
wgrantmwhudson: Blah.23:24
mwhudsonand it would be so nice if a 77ms query was unacceptable but ...23:24
wgrantWhile there are queries going around, how do I convince somebody to run 'SELECT DISTINCT dsc_format FROM sourcepackagerelease' on production? I'd like to see just how broken the data there is, as the early sample data is shocking.23:25
mars:D23:25
mwhudsonwgrant: mbarnett is your friend i guess23:25
mwhudsonwtf23:26
mwhudsonfactory.makeBug() in 'make harness' seems to be broken23:26
wgrantmbarnett: Hi there. Can I convince you to give me the result of 'SELECT DISTINCT dsc_format FROM sourcepackagerelease' on production?23:27
mbarnettwgrant: sure sure23:27
mwhudsonIntegrityError: duplicate key value violates unique constraint "bugnotification__bug__message__unq"23:27
wgrantmwhudson: Awesome.23:27
* wgrant tries.23:27
wgrantConfirmed.23:28
wgrantWTF.23:28
mbarnettwgrant: http://pastebin.ubuntu.com/308005/23:28
wgrantmbarnett: Excellent, thanks.23:29
mbarnettwgrant: welcome23:29
wgrantIt's wrong, but easily fixed when we need to.23:29
* mwhudson makes schema to see if that helps23:29
wgrantmwhudson: It doesn't.23:29
wgrantmwhudson: I've got a fresh one here and it's broken like that.23:30
mwhudsonwgrant: wtf23:33
wgrantmwhudson: That too.23:34
mwhudsonit's used in the tests a fair bit23:34
mwhudsonmaybe differences between test and dev databases23:34
mwhudsonwhat the heck!23:35
wgrantI tried ftest-playground already.23:36
wgrantSame problem.23:36
wgrantThat was my first guess.23:36
mwhudsonbugnotification is empty before this happens23:36
wgrantA subscriber might be doubled up somewhere?23:36
mwhudsonno, the index is just on message and bug23:38
wgrantsubscriber meaning Z3 subscriber, not BugSubscriber.23:39
mwhudsonah23:39
mwhudsonyes possibly23:39
mwhudsonwgrant: yeah, seems possible23:44
wgrantI don't know too much about the subscription system, though.23:45
mwhudsonwgrant: yes, seems all subscribers are added twice in make harness23:49
wgrantmwhudson: Ah, that's easy.23:50
mwhudsonwgrant: easy to fix>23:52
mwhudson?23:52
wgrantmwhudson: If you look at canonical.database.harness._get_locals, you'll see it calls execute_zcml_for_scripts, and then executes script.zcml.23:53
wgrantBut execute_zcml_for_scripts appears to execute script.zcml itself.23:53
* wgrant tests.23:53
wgrantBut that seems far too easy.23:54
marsbad smell in the windmill python suite: the client object has to be passed around everywhere.  Need a context object or something.23:55
wgrantmwhudson: Commenting out the script.zcml invocation in canonical.database.harness._get_locals fixes it, and the appropriate bugnotification is still created.23:57
wgrantBut I must be missing something.23:57
mwhudsonoh i think maybe xecute_zcml_for_scripts executes more zcml than it used to23:58
wgrantPossibly.23:58
wgrantIt certainly executes script.zcml itself.23:58
* wgrant annotates.23:59

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!