/srv/irclogs.ubuntu.com/2010/04/15/#launchpad-dev.txt

maxbwoot, with the tarfile fails cleaned off, it's looking quite good.00:04
maxbI have a one-liner patch that I'd kind of like landed on devel. Is it appropriate to bother with a branch just for this?:00:12
maxb-    >>> http = HTTPCaller(host='launchpad.dev')00:12
maxb+    >>> http = HTTPCaller()00:12
wgrantmaxb: How're you dealing with tarfile? Just fixing the tests in the 2.6 branch?00:14
maxbyes00:15
maxbI've pushed that00:15
wgrantGreat.00:15
lifelessmaxb: nothing lands without a branch; but it doesn't have to be yours :P00:17
maxbI guess my question is: is that too trivial for the reviewer overhead?00:17
lifelessthats what rs= is for :)00:19
maxbright :-)00:21
maxbHrm. valid_absolute_url('whatever://example.com/blah') ---> False in 2.5, True in 2.6, because of stdlb chanes00:21
maxb*changes00:21
* maxb wonders what to do about that00:22
wgrantI was wondering about that one.00:24
wgrantDoes it now ignore the scheme?00:24
wgrantWe already monkeypatch in extra schemes (sftp and bzr+ssh come to mind).00:24
wgrantThis may also cause the DB constraint to be relaxed, which would be seriously seriously bad.00:25
wgrantOK, yes, it's seriously seriously bad.00:26
wgrantOh, wait, that *is* the DB constraint. Oops.00:27
wgrantOr maybe not. i am confused.00:27
* wgrant reads.00:27
maxbwtfness00:28
wgrantI don't know. This seems like a Python bug.00:28
wgrantOr at least a really dangerous behaviour change.00:29
maxbpy2.6 urlparse disregard the uses_netloc list when parsing urls and treats anything beginning with // as netloc based00:29
wgrantYep.00:29
wgrantWTF00:29
maxbWhich is arguably saner than maintaining a list of specific schemes, but changing behaviour is just stupid00:29
wgrantNot just stupid -- very dangerous.00:30
wgrantShould we change both valid_absolute_url implementations to explicitly check the scheme against the monkeypatched uses_netloc?00:31
maxbIt looks to me like the only thing that uses this is c.l.interfaces.validation.validate_url, which already checks against a passed in list of schemes00:33
wgrantmaxb: What about valid_absolute_url in trusted.sql?00:34
maxbThat will suffer the same bug, but is an independent implementation to the one failing the tests00:35
wgrantmaxb: Right. But I would not fix the tested one until we've fixed the other one too, since it appears to be untested.00:36
wgrantAlthough i guess it could be tested, but PostgreSQL still uses 2.5.00:36
wgrantBut no, PL/Python is built for 2.600:37
wgrantSo the DB constraint is untested.00:37
wgrantYay.00:37
wgranthttp://bugs.python.org/issue790400:42
wgrantIt's only a recent change.00:43
wgrantSo they must have decided to change behaviour like that in a stable update.00:43
wgrantAwesomeness.00:43
maxbwgrant: Hi, do you have something in the pipeline towards landing for the foreign branch tdb issues?01:01
wgrantmaxb: I don't. I guess I should prepare the two branches.01:15
wgrantAlthough I wonder if I'm going to cause bzr-hg test failures by ripping out the TDB thing.01:15
wgrantHm, only 8 tests left to fix.01:16
maxbI'm looking at the initZopelessTwice one01:17
wgrantI looked at that before.01:17
wgrantMy initial thought was that the method had been renamed.01:17
wgrantSo the monkeypatch didn't work.01:17
wgrantBut it looks fine.01:17
maxbClose - 2.6 is sending that codepath into a C extension, hence avoiding the monkeypatch01:18
wgrantI saw that evil down the bottom, but then decided to have breakfast first.01:18
maxbI'm in good humour, having returned to the computer having spent the evening in the pub :-)01:20
wgrantAFAICT there are only two remaining issues that are actually mysterious.01:20
wgrantThe leftover thread one, and the _pythonpath one.01:20
* wgrant takes bugs-emailinterface.txt01:21
wgrantOK, so, bugs-emailinterface.txt tests for the bug that http://bugs.python.org/issue7082 fixed.01:40
wgrantThat test is probably insane.01:40
* wgrant might talk to Bugs about that one tonight.01:41
maxbhmm, is emailauthentication.txt really just complaining about a wrapped line?01:46
wgrantmaxb: yes.01:46
wgrantJust looking at that now.01:46
wgrantThe point of the test is to test for whitespace; it tells doctest to not ignore whitespace changes.01:46
wgrantI'm not sure what the point of the test is.01:47
wgrantIt says that we must be careful, because Python unfolds things.01:47
wgrantIt appears that Python no longer does.01:47
wgrantI cannot see an obvious entry in NEWS about this.01:47
wgrantAnd the test does not make it clear what the implications of it are.01:48
wgrantHm, I guess as long as the authenticateEmail succeeds it should be fine.01:48
wgrant- Issue #1670765: Prevent email.generator.Generator from re-wrapping01:48
wgrant  headers in multipart/signed MIME parts, which fixes one of the sources of01:48
wgrant  invalid modifications to such parts by Generator.01:48
mwhudsonargh i hate the puller01:52
mwhudsoni guess i should be happy that i'm deleting chunks of it01:52
wgrantmaxb: Do you think that the carefulness it refers to is the manual boundary handling in lp.services.mail.signedmessage.SignedMessage._getSignatureAndSignedContent?01:52
maxbthat seems plausible01:53
maxbSo perhaps this test failure is inviting us to reconsider whether we still need to do that?01:53
wgrantI guess I'll delete the special case and rerun the tests in 2.5.01:53
wgrantI think so.01:53
maxbCertainly I don't think the failing test has any operational significance01:54
wgrantHm, I guess we can't actually remove it.01:54
wgrantSince only Lucid has the Python fix.01:54
maxbOh, is this a 2.6.x fix?01:55
wgrantSo it probably deserves to have the test for broken folding removed, a bug filed, and _getSignatureAndSignedContent XXXd.01:55
wgrant2.6.5.01:55
maxbugh01:55
wgrant2.6.5rc1, to be precise.01:55
maxbok, concur with your plaan01:56
mwhudsonbiab, again02:02
* maxb wonders what the point of filing a bug for every XXX is02:09
wgrantmaxb: Turns out the bug isn't really fixed.02:17
wgrantIt's just less broken than before.02:17
maxb:-/02:17
wgrantas_string() on the entire message preserves folding, but on the part loses it02:17
wgrantI'll alter the test to look at the part in question, which should work for both.02:17
* maxb is working on tolerance to the apt-ftparchive changes02:18
wgrantI'm not sure if we want to be tolerant.02:19
wgrantIt's going to change the archive. Probably in a good way, but I'd prefer to disable SHA1 and SHA256 for now.02:19
maxbhmm02:20
wgrantBut I can't work out how to do that -- it doesn't seem to respect the config options when they are in the apt-ftparchive config.02:20
wgrantThey may have to be in the apt config instead, and I'm not sure if we can pass a custom one into a-f.02:20
wgrantMind you, I haven't looked at it for two or three months.02:20
maxbInterestingly, there's stuff left in the code from when the same issue affected Packages files, edgy->feisty02:20
maxbSo I was thinking we'd handle it the smae02:20
wgrantWhat happened there?02:21
wgrantJust ellipsised out the extra hashes?02:21
maxbregexed them out, yes02:21
maxberr, string-maniped them out, rather02:21
maxbhmm, well, it's actually 2:20am here, so perhaps I should sleep instead :-)02:23
wgrantHeh, probably.02:23
wgrantNight.02:23
maxbI lean towards letting lucid's apt-ftparchive write the extra sums for the production archive02:23
wgrantIf it's been done before, that's fine.02:24
maxbIt looks to have been done that way before for Packages files, we'd now be doing the analogous change for Sources files.02:24
maxbHrm, have you noticed that soyuz-upload.txt is now dropping a debian 'changelog' file into the root of the LP tree when run?02:26
wgrantI hadn't noticed. But so it does.02:27
mwhudsonabentley: around?03:13
abentleymwhudson, a bit.03:13
mwhudsonabentley: just replying to that merge proposal if you have time to look at that03:14
abentleymwhudson, looking.03:17
pooliespm, could you please run http://pastebin.ubuntu.com/414679/ across some of the logs and paste the first few lines of output?03:29
spmpoolie: http://paste.ubuntu.com/414681/03:37
poolieurk, not quite enough03:38
pooliei guess it has the refererer url at the end?03:39
pooliespm how about http://pastebin.ubuntu.com/414683/03:40
spmpoolie: yeah it does: 1.2.3.4 bugs.launchpad.net - [14/Apr/2010:07:00:04 +0100] "GET /ubuntu/lucid/+bugs?field.searchtext=&orderby=-importance&search=Search&field.importance%3Alist=CRITICAL&field.importance%3Alist=HIGH HTTP/1.0" 200 90458 "-" "Python-urllib/1.17"03:41
spmpoolie: http://paste.ubuntu.com/414684/03:42
poolieok, and what about the whole thing piped through '|sort|uniq -c' ?03:42
spmpoolie: http://paste.ubuntu.com/414686/ <== might be worth checking for skew from robots. maybe.03:44
pooliewow03:44
pooliecould you chain it together with your anti-robot filter that you showed the other day?03:44
pooliethough actually i think it's unlikely there will be too many03:45
pooliebecause there's only a form linking to this url not (i think) generally an A ilnk03:45
poolielink*03:45
pooliethat histogram is pretty damn suggestive btw03:45
pooliei guess that's just over less than a day's data?03:45
spmpoolie: about half of 20 hours worth, for bugs.lp.net; not edge.03:47
spm1 of 2 servers03:47
poolieso something like 10-20% of a day i guess03:48
pooliei wonder if power users tend to use edge?03:48
pooliethanks for helping btw03:48
spmpoolie: np; just don't mention it too much eh? I have a rep as a BOFH to live up to. k?03:50
poolieheh03:50
poolieso would you mind running that on edge, across several days data, just to be sure it's not skewed?03:51
spmadding edge - same server/period - doesn't change the number much; sure.03:51
pooliei mean catting several days of logs into it, if that's possible03:52
poolieso that we get a few thousand hits altogether03:52
spmoh yeah - I was 3/4 thru typiing the above. figured I'd finish and then do several days03:53
poolieoh thanks03:53
spmpoolie: http://paste.ubuntu.com/414692/ about half of nearly 2 weeks worth04:04
wgrantmwhudson: Hi. Half of the unsolved python2.6 failures are Codehosting threading things.05:21
mwhudsonwgrant: oh goodie05:22
wgrantlp.codehosting.puller.tests.test_acceptance.TestBranchPuller.test_hosted_branch_stacked_on_mirrored_branch, in particular.05:22
wgrantIf the .join callback is there, it hangs at that point.05:22
wgrantIf it is not there, it leaves a thread behind.05:22
mwhudsonwell, the latter isn't really a surprise, that's why the join is there :-)05:22
wgrantRight.05:23
wgrantThreads make me sad -- any ideas?05:23
mwhudsonone small one05:23
mwhudsonwgrant: i don't suppose it will make any difference, but if you go to bzrlib/tests/http_server.py:56305:24
mwhudsonwgrant: there's a comment about not having a join in a particular place05:24
mwhudsondoes it work if you add the join there?05:24
mwhudsonwgrant: another idea would be to run gc.collect() before calling join05:25
wgrantmwhudson: Both suggestions hang quite effectively.05:29
mwhudsonwgrant: :(05:32
mwhudsonwgrant: is there an easy ish way i can reproduce this myself?05:33
mwhudsoni have some terrifying gdb hacks lying around i can try to use...05:33
mwhudson(https://code.edge.launchpad.net/~mwhudson/+junk/pygdb if you want to try to adapt them yourself)05:34
wgrantmwhudson: https://dev.launchpad.net/PythonMigrationStatus <- just grab the integration branch, follow 'Setup caveats', and revert the patch which comments out the join.05:34
wgrantFour tests remain broken, two of which we have solutions for. Yay.05:37
mwhudsonnot too bad05:41
stubWhat do BranchRevision rows with a NULL sequence signify again?05:46
mwhudsonstub: off mainline revisions05:46
wgrantstub: *Ouch* at that column type change time.05:54
stubdifficult estimate to make though - it will be much faster on the kick arse hardware, but there are more nodes to update05:55
wgrantIt's a bit sad that 99% of that table is probably duplicated dozens to thousands of times.05:56
pooliewgrant: https://bugs.edge.launchpad.net/malone/+bug/557432/comments/1005:56
mupBug #557432: Hot Bugs algorithm works poorly for some projects, making it a bad default listing <story-bug-heat> <Launchpad Bugs:Triaged> <https://launchpad.net/bugs/557432>05:56
wgrantpoolie: Yeah, I saw the paste.05:57
wgrantRather interesting.05:57
wgrantWe were both right, it seems :/05:57
wgrantawkward.05:57
wgrantpoolie: Note, however, that since the default view is heat, there is less need to order search results by it.05:57
poolietue06:00
poolietrue06:00
pooliebut 42, being about 0.2% of the total, is awfully low06:00
wgrantBut I think the factor of 50 probably outweighs it by a bit.06:00
wgrantRight.06:00
poolieanyhow, i hope we can do more measurements like this06:02
pooliethey're not watertight but they bring something beyond just gut feel06:02
wgrantIndeed. They're much better than guessing.06:02
* mwhudson fills up his hard drive with 2.6 versions of all the eggs06:03
wgrantmwhudson: Weren't they all in download-cache anyway?06:03
wgrantAlthough you may have issues with a missing i386 version of some egg that I forget.06:03
wgrantThe 2.6 i386 meliae egg is missing, that's right.06:04
mwhudsonwgrant: the download-cache mostly contains tarballs06:04
mwhudsonnot built eggs06:04
wgrantOh, so it does.06:04
* wgrant gives up on checkwatches and leaves the last failure to Bugs.06:06
mwhudsonwgrant: now i can run the test!06:17
mwhudsoni wonder if that took long enough :/06:17
mwhudsonwgrant: the test passes for me :/06:22
mwhudsonwgrant: does it only fail as part of running the whole file or something?06:23
wgrantmwhudson: Did you revert the commenting-out that is in the branch?06:23
mwhudsonwgrant: yes06:23
wgrantArgh.06:23
wgrantbin/test -vvt TestBranchPuller06:23
wgrantThat reproduces it for at least three people.06:24
mwhudson./bin/test -vvc  lp.codehosting.puller.tests.test_acceptance06:25
mwhudsonis what i'm running06:25
mwhudsonif using -t makes a difference, i am going to become very angry06:25
mwhudsonTear down canonical.testing.layers.ZopelessAppServerLayer is hanging for me a _lot_ lately06:27
wgrantYour machine hangs in the wrong place :(06:28
mwhudsonand not just on this branch06:28
cody-somervilleWhats the fix for the GpgmeError error in the testRepositorySignatureWithSigningKey test?06:28
mwhudsoni guess it's the sort of thing that rebooting, or at least logging out and in again might fix06:28
mwhudsoni may have stray processes06:28
wgrantcody-somerville: http://paste.ubuntu.com/414737/06:29
cody-somervillemerci06:29
wgrantmwhudson: With http://paste.ubuntu.com/414739/ on top of lp:~launchpad-committers/launchpad/python2.6, both my -t and your -c invocations hang.06:30
mwhudsonit seems test_mirror_imported_branch might have hung06:36
mwhudsonor is just very very slow06:37
mwhudsonlooks pretty hung by now06:39
wgrantBut a different test.06:41
wgrantLovely......06:41
mwhudsonwoot, i have some tracebacks06:42
wgrantgdb?06:42
mwhudsonyes06:42
mwhudsonafter a fashion06:42
mwhudsonwgrant: this is the hung thread: http://pastebin.ubuntu.com/414743/06:42
wgrantDoesn't look very stopped to me.06:44
mwhudson        # Support people who used socket.settimeout() to escape06:44
mwhudson        # handle_request before self.timeout was available.06:44
* mwhudson is suspicious06:44
wgrantWhere's that?06:44
mwhudsonwgrant: handle_request in SocketServer is at least quite different between 2.5 and 2.606:45
wgrantAh.06:45
mwhudsoni guess that if you're select()ing on a server socket that's shutdown, something different happens than if you're accept()ing on a socket that's shutdown?06:47
mwhudsonbecause that seems to be the difference06:48
wgrantCould be, yes.06:48
mwhudsonoh grar06:49
mwhudsoni guess the select() in SocketServer should have self in the error list as well06:49
wgrantIt works if I force the timeout to 10 in SocketServer.06:49
wgrantHmmm.06:50
wgrantMaybe.06:50
wgrantIt also works if self is in all three.06:52
* wgrant tries with it in wlist but not xlist.06:52
wgrantIt doesn't work in just rlist and xlist.06:53
mwhudsoni think this is perhaps the same issue: http://bytes.com/topic/python/answers/851825-socketserver-shutdown-deadlock06:54
wgrantSimilar.06:57
mwhudsonhm06:57
wgrantSo, it works OK with self in rlist, wlist and xlist. But no subset thereof.06:57
mwhudsoni guess it should be fixed in bzr06:57
mwhudsoncan probably fix it by hacking in launchpad though06:57
mwhudsonwell, it should be fixed in python06:57
mwhudsonbut it's a bit late for that06:57
wgrantIndeed... If you can work out the appropriate Launchpad hack, we can probably have the test suite passing under 2.6 in a few hours. Yay.06:58
mwhudsonwgrant: http://pastebin.ubuntu.com/414750/ maybe07:06
wgrantmwhudson: Is that actually the right solution, though?07:07
mwhudsonwell it doesn't seem to work :(07:07
mwhudsonoh no07:07
mwhudsonconfusing slow and hanging tests there07:08
wgrantmwhudson: http://bugs.python.org/issue2302 seems to be a bug about that shutdown thing you linked to earlier.07:08
mwhudsonwgrant: that seems a bit different07:09
mwhudsonwgrant: the problem that i saw was that the server thread went into a select that will never return07:10
wgrantRight.07:10
wgrantSo server_forever was indeed running at the time, so it's probably irrelevant.07:10
mwhudsonnoi07:11
mwhudsonserver_forever is not called07:11
wgrantBah.07:11
mwhudsonby bzrlib07:11
wgrantAnyway, lecture over... I should go home.07:11
mwhudsonhaha07:11
mwhudsonwgrant: do you want me to push that patch somewhere, or .. ?07:12
wgrantmwhudson: If it makes sense to you and probably doesn't just work by accident, I guess push it to the 2.6 branch.07:12
wgrantAt least it's only used in tests; it can't go horribly wrong.07:13
wgrantOK, now I just have to get approval for one change from Bugs, advice from them on another, and we need to work out the best fix for validate_absolute_url.07:16
wgrantThen we are DONE.07:16
wgrantThanks for fixing that one/07:16
* wgrant wanders off.07:16
mwhudsonhm07:22
mwhudsonnot sure the fix is right actually :(07:22
adeuringgood morning07:44
maxb*blink*08:51
maxbregexes are really discouraged entirely in launchpad code?!08:51
maxbis this really true?08:52
lifelessfirst I've heard of it08:53
maxbnoodles775 attempted to convince me of this in a code review08:54
noodles775I can't remember who had mentioned it to me, intellectronica perhaps - as I mentioned on the MP, it's not in the styleguide, so up to you.08:55
lifelessre is generally faster and more reliable than adhoc stuff08:55
lifelessso I would say 'whatever is clearer'08:55
maxbnoodles775: Yeah, I'm not implying you're being unreasonable, I'm just experiencing shock at the concept :-)08:56
lifelessI don't see anything in the reviewer minutes about it either08:56
noodles775lifeless: I don't even know that it's ever come up... it was just something that I was told for a branch I had ages ago that used a semi-complicated re.08:57
noodles775lifeless: that's why I said it was up to maxb :)08:57
noodles775maxb: in which case, do you want me to send it off to land as is?08:58
mrevellMorning!09:01
maxbnoodles775: yes please, if you're happy with that09:11
noodles775maxb: Sure. Done.09:13
wgrantAny Bugs people around? I need advice on what to do with the section of bugs-emailinterface.txt beginning with 'Some mail clients append a filename to the content type of attachments.09:35
wgrant'09:35
wgrantIt breaks in 2.6, because the behaviour that it says should not occur (extracting a filename from Content-Type, rather than Content-Disposition) now occurs due to a Python bugfix.09:36
wgrantIt appears safe to fix the test to expect that the filename is extracted from Content-Type... but I'm wondering why the broken behaviour was tested in the first place.09:37
intellectronicawgrant: let me have a look09:45
wgrant(this is one of the four remaining 2.6 failures)09:46
wgrantintellectronica: Thanks.09:46
intellectronicaactually, adeuring might know more about this. i remember he did some work in this area lately09:46
maxbwgrant: sorry, I just added another two in MailmanLayer to the wiki page :-)09:47
wgrantmaxb: Nooooooo.09:47
maxbwell, I added 3 and fixed 109:47
adeuringhrmmm, well, there is something....  main problem is my bad memory... let me look at the affected code/tests09:48
wgrantI was running that just before I headed home, but it disagreed with being suspended for two hours.09:48
intellectronicaadeuring: maybe i'm wrong, this is about headers in mail. i remembered that you worked on attachments in the webapp.09:48
intellectronicaBjornT: do you maybe remember why this was done? ^^^09:49
adeuringintellectronica: yeah, you are probably right...09:49
intellectronicawgrant: i'm looking myself at the code, but my guess is as good as yours, which is why i'm looking for someone who might know the history of that code.09:50
lifelessthumper: back, perchance?09:51
=== henninge_ is now known as henninge
adeuringwgrant: what breaks in this test?09:52
* maxb has a shocking thought: Python 2.6 for 10.04 == not unachievable09:53
intellectronicawgrant: so i'm not sure if it's really important to ignore the filename supplied with content-type, as long as it doesn't change the behaviour of signatures, right?09:54
wgrantadeuring, intellectronica: http://pastebin.ubuntu.com/414824/ is my fix.09:54
wgrantintellectronica: And the signature depends solely on the MIME representation; not our interpretation of it.09:55
adeuringwgrant: yes, that sould be fine09:55
wgrant(well, assuming a lack of Python bugs, which there is not)09:55
intellectronicawgrant: yes, that looks like the correct fix to me09:55
wgrantintellectronica, adeuring: Thanks.09:55
* wgrant commits to the integration branch.09:55
wgrantmaxb: Ah, I see you still love the tarfile maintainer.09:57
maxb:-)09:58
maxbI suppose I have the consolation that after figuring it out last time, this was a pretty shallow bug09:58
adeuringwgrant: this failure actually means that LP will process mails from Microsoft Entourage somewhat better, once we switch to Python 2.609:58
adeuring(another point is that this program should be banned)09:59
wgrantadeuring: That is not a feature.09:59
wgrantHeh, yes.09:59
maxblol09:59
adeuringwgrant: I agree completely ;)09:59
wgrantmaxb: Ah, so the mailman timeouts are normal? I presumed they were from being asleep for hours.10:01
maxbwell, I'm going by Barry's comment on the wiki page that there are spurious failures if you run those tests not individually10:03
maxbI ran the ones that failed individually and they passed10:03
wgrantadeuring, intellectronica: Also, there is one other Bugs failure that I haven't got to the bottom of yet. Do you have any ideas about test_bug_496988 on https://dev.launchpad.net/PythonMigrationStatus?10:04
intellectronicahmmmm ... it's worth asking allenap and gmb about this one, they might have an idea10:06
* gmb looks10:06
wgrantgmb, intellectronica: Thanks.10:06
intellectronicamaybe the result is that we have to doctor the headers to satisfy the new version of xmlrpclib10:07
gmbintellectronica, wgrant: Yeah, I think that might well be the case.10:07
wgrantintellectronica, gmb: Any specific hints, or shall I go digging?10:10
gmbwgrant, If you give me five minutes I'll have a look at the test. I suspect that some of our testing infrastructure for the XML-RPC tests, which is mostly a looks-like-xmlrpc-but-isn't object isn't playing nice.10:13
BjornTintellectronica: can i remember why what was done?10:13
gmbOkay, something weird's going on...10:15
intellectronicaBjornT: why we had a test for the broken behaviour where a filename specified in the content-type header of a mail is ignored. but unless something comes to mind don't worry about it, i think wgrant has a reasonable fix, now that the behaviour of email parsing was fixed in python2.610:16
gmbOh, the test's moved. that's why I couldn't find it.10:17
wgrantgmb: Yeah, it took me a while too.10:17
gmbSee, this is why we should never, ever refactor things and just keep the whole of LP in launchpad.py10:18
gmbOH DEAR GOD.10:19
wgrantOh dear god?10:19
gmbwgrant, Badly-written test.10:20
wgrantgmb: So it *wasn't* just me.10:20
gmbBut, might be indicative of issues we'll ahve in the future when we switch to 2.6 with doing XML-RPC queries to remote trackers; I don't know.10:20
gmbwgrant, nope.10:20
gmbwgrant, that test actually tries to contact the gnome bugzilla.10:20
wgrantgmb: Oh yes, I noticed that when I tried to run it on the train.10:21
wgrantIt was not happy.10:21
BjornTintellectronica: no, i can't remember why. it seems like an odd test.10:21
gmbwgrant, I think I wrote it, and once I've worked out how to fix it I'll throw myself on my sword.10:21
* gmb makes a note: buy a sword10:21
wgrantHeh.10:21
wgrantApr 15 14:48:12 2010 (31672) Received these actions: create10:22
wgrantApr 15 14:48:12 2010 (31672) List creation error for team: itest-one10:22
wgrantApr 15 14:48:12 2010 (31672) An error occurred; the list was not created: itest-one10:22
wgrantThankyou, mailman, for your descriptive errors.10:22
=== daniloff is now known as danilos
gmbwgrant, Actually, it's not all that bad, because I *think* it might show that we'll have a problem with XML-RPC and bugtrackers when we move to 2.6.10:30
dhasthawgrant, Is there any tool which using launchpad API? like bzr-eclipse, tarmac, apport, ubuntu-dev-tools, and ilaunchpad-shell10:31
gmbwgrant, So, one way to fix it is to monkeypatch lp.bugs.externalbugtracker.get_external_bugtracker to return a Bugzilla instance that doesn't try to connect out. See test_updater.py:131 for an example of us doing this.10:32
gmbwgrant, in this case we want a get_external_bugtracker that returns a subclass of lp.bugs.externalbugtracker.bugzilla.BugzillaAPI whose getExternalBugTrackerToUse() method just returns self rather than contacting the remote server. I *think* that should do the trick.10:34
wgrantdhastha: What do you mean? Aren't those all tools that use it?10:34
dhasthawgrant, sorry.  I have to know, is there any application using launchpad api?10:37
wgrantdhastha: All those examples you gave above do, don't they?10:41
wgrantmaxb: Um, I'm not sure those two mailman tests even pass in 2.5.10:47
wgrantI get the same errors there.10:47
lifelesswgrant: bzr-eclipse, tarmac don't10:47
lifelesswgrant: last I heard, anyhow.10:47
wgrant(yay for tests that are not run by default?)10:48
wgrantlifeless: Hm, I'm pretty sure tarmac does.10:48
lifelessoh, perhaps I mean 'doesn't use my pet feature'10:49
wgrantlifeless: Merge queues that weren't meant to be exposed? :P10:49
lifelesswgrant: meh, they are meant to be according to thumper10:50
lifelessjust not finished10:50
deryckMorning, all.11:03
bigjoolsapt-get dist-upgrade on lucid this morning wants to remove launchpad-developer-dependencies :/11:10
bigjoolsah, python-apt got bumped in lucid11:19
lifelesswhat would cause11:27
lifeless    raise HTTPError(response, content)11:27
lifelesslazr.restfulclient.errors.HTTPError: HTTP Error 414: Request-URI Too Large11:27
wgrantThat's a new one.11:27
lifelessI'm trying to lookup 126 branches by url at once11:27
wgrantAh.11:28
lifelesssquid allows several mb of URI IIRC11:28
* lifeless bugifies11:28
lifelesslaunchpadlib, you think?11:28
wgrantThat looks like a server-side error to me.11:28
maxbbigjools: so it did. uploading....11:29
maxbI have this down to executing 'lpnochange python-apt' now :-)11:29
bigjoolsmaxb: :)  thanks11:30
wgrantmaxb: Those two mailman failures are indeed legit bugs in 2.5 as well. I'm fixing them now.11:30
bigjoolsI have bigger issues, like the upgrader deciding not to install latest versions of some packages ...11:31
maxbbigjools: !11:31
wgrantbigjools: That's normal if there's arch skew.11:31
wgrantAnd when the build farm breaks, there will be arch skew.11:31
bigjoolsthat's probably what it was11:31
bigjoolsI need to install the task for the -desktop package11:31
bigjoolsit  does explain all the frickin breakage!11:32
mwhudsonwgrant: i didn't seem to fix the http server hang :/11:33
wgrantmwhudson: :(11:33
mwhudsoni'll look tomorrow when i'm more alert11:33
lifelessjml: around ?11:55
lifelessmwhudson: or are you still here ?11:55
lifelessgetByUrls is returning a dict mapping strings -> dicts rather than strings->Entries11:55
wgrantlifeless: lazr.restful doesn't support it.11:57
wgrantWell, actually, the WADL implementation doesn't support dicts, so lazr.restfulclient isn't told to interpret them as entries.11:58
lifelessthis is painful11:59
wgrantYes.11:59
lifelesswhat does it take to fix11:59
wgrantleonardr: ^^?12:00
lifelesshes us based12:00
leonardri'm here12:00
lifelessisn't he?12:00
leonardryes12:00
wgrantHe is. Eastern USians are around now12:00
wgrantSome of.12:01
leonardri don't think it would be difficult to add support for that but it's also pretty low on my priority list. it's one of the things i'm going to look at for streamlining the web service12:01
wgrantOr my timezones are wrong. That is more likely.12:02
wgrantadeuring: The test is probably executed once for each bugtarget implementation.12:03
lifelesshmm, 10 minutes to do the branch lookups12:03
wgrantlifeless: This is Launchpad...12:03
lifelessleonardr: this /really/ hurts. Can you suggest workaruonds.12:03
lifelesswgrant: its ~ 1 minute with batches12:04
lifelesswgrant: which I can't use...12:04
leonardrmy usual workaround seems to be the thing that seems to be hurting you--a dict with strings12:04
lifelessleonardr: right; I want a collection with objects12:04
leonardris getByUrls the thing that's published now?12:05
lifelessyes12:05
lifelessleonardr: if you could describe how to approach the problem, in a lplib bug, I might take a peek, or get someone I know to take a peek,.12:06
lifelessleonardr: I think guidance from you is a key step to someone else doing it though12:06
lifelessleonardr: oh, while you are here12:06
leonardrso, general steps12:07
lifelessleonardr: do lplib objects compare == if they have the same canonical url ?12:07
leonardryes, as per https://bugs.edge.launchpad.net/launchpadlib/+bug/26409812:07
mupBug #264098: launchpadlib should implement __eq__ and __ne__ <launchpadlib :Fix Released by leonardr> <https://launchpad.net/bugs/264098>12:07
lifelessleonardr: and are they hashable ?12:08
leonardri don't know12:08
lifelessok12:08
lifelesswhat version of lplib is that fix in ?12:08
leonardrthe lazr.restfulclient NEWS.txt says 0.9.1012:11
lifelessuugh, ok.12:11
lifelessI'll comapre self_link then12:11
lifeless(datacentre has 0.2.3)12:11
leonardris that even a real version? i don't see it in NEWS12:12
adeuringwgrant: right12:13
adeuringwgrant: thanks for the xplataion!12:13
lifelessleonardr: I may have the number slightly wrong12:13
adeuring...explanation...12:13
lifelessleonardr: probably 0.2~bzr35-0ubuntu1 or something12:13
lifelessleonardr: its a port to hardy, after all12:13
leonardri see12:14
leonardrmaybe 0.9.312:14
lifelessno12:14
lifelessprobably one of these:12:14
lifelesspython-launchpadlib | 0.2~bzr25-0ubuntu1 | intrepid/universe | source, all12:14
lifelesspython-launchpadlib | 0.2~bzr35-0ubuntu1 | jaunty/universe | source, all12:14
lifelessI saw the version a few days back, it was definitely 0.2something12:14
leonardrit's possible the version numbers were very different before launchpadlib was split into lazr.restfulclient and launchpadlib12:19
leonardrbut pretty much nothing you want will be in there--including sensible support for dicts, which isn't anywhere yet12:19
* wgrant now understands why MailmanLayer isn't in normal test runs...12:21
lifelessleonardr: sure12:27
lifelessleonardr: I've filed an RT to get 1.5.1 on pqm12:27
lifelessleonardr: you were going to describe how to approach supporting a dict of objects as a return value12:28
leonardrright12:28
leonardr1. create @returns_dict_of similar to @returns_collection_of12:28
leonardr2. come up with a wadl description of what @returns_dict_of means and put it in the wadl12:29
lifelesswhats the difference between a dict and a collection ?12:29
leonardra collection is a list12:29
lifelessok12:29
lifelessif we're using non python names, should we say 'returns_map_of' ? :P12:29
leonardr3. have lazr.restful properly serialize a dict to json (this might work already)12:29
leonardr'collection' is a term from the atom publishing protocol. there's no corresponding term for 'map' so i don't think it matters much12:30
leonardr4. have lazr.restfulclient pick up on @returns_dict_of and handle it properly. old versions won't be able to handle the return value so we should probably do this in a new version of the web service12:31
lifelessleonardr: I can't see an obvious matching bug on launchpadlib12:35
lifelessleonardr: should I file one with this conversation?12:35
leonardrlifeless: https://bugs.edge.launchpad.net/lazr.restful/+bug/48109012:35
mupBug #481090: Cannot define a method that returns a dict <lazr.restful:Triaged> <https://launchpad.net/bugs/481090>12:35
lifelessah, ok12:36
jmlfor this case, you don't want list semantics, you really want dict semantics, I think13:00
jml.13:00
leonardrjml, let me ask you this now rather than as part of a conversation later--how happy would it make you if i added this feature vs other features we've discussed?13:07
jmlleonardr: probably there would be a net change of zero to my happiness13:08
jmlleonardr: the stuff you've got scheduled now is more important. this particular bug annoys me though.13:09
leonardrjml: how about once i get on to streamlining the web service? work on this first, on something else first, or talk to you then?13:11
* jml thinks13:13
jmlleonardr: by streamlining, are you referring to the "publish lazr.restful ops more naturally" task?13:14
leonardrjml, yes13:14
leonardrthis falls under that task, i think13:14
leonardrmore urgently, i need to talk to you about performance at this point13:15
jmlleonardr: ok.13:15
jmlleonardr: I agree it falls under that task, despite the fact that the end-user benefit is greater perceived performance13:16
leonardrjml: how much longer will you be around?13:16
leonardri wonder if i should talk to you first or clean up the performance wiki page13:17
jmlleonardr: probably another eight hours13:17
leonardrok, let me fix up the wiki and we can talk about performance next steps13:17
jmlleonardr: cool.13:17
jmldammit, where's my hasbean delivery?13:31
leonardrjml, i've sorted our performance interventions so far:13:46
leonardrhttps://dev.launchpad.net/Foundations/Webservice/Performance13:46
=== matsubara-afk is now known as matsubara
wgrantbigjools: Did you end up filing an RT for the PPA access log parser?13:57
jmlleonardr: regarding "Request service root conditionally", what was launchpadlib doing before your fix?14:10
leonardrjml: launchpadlib was making a non-conditional request, thus your 'download the wadl every time' problem14:12
jmlleonardr: on every request?14:12
leonardrjml: no, on every startup14:12
jmlleonardr: ahh, ok.14:12
jmlleonardr: sorry, mega-interrupty day14:21
jmlleonardr: can we talk in about three hours time?14:23
leonardrjml, sure14:23
jmlleonardr: thanks.14:23
leonardror at some point--i have interruptions like doctors appointments today14:23
jmlok14:26
jmllifeless, you do know that the last element of an MP url is the db id, right?15:02
lifelessjml: I know it happens to be15:02
lifelessjml: I don't know that its guaranteed to stay that way15:03
lifelessthe self_link docs on branch_merge_proposal in the api doc don't claim either point15:03
jmllifeless, ok. I'm not saying the bug you filed is invalid, just pointing to a potential workaround15:03
lifelessjml: I put that precise workaround in place before filing it ;)15:03
jmllifeless, good good15:04
marsgary_poster, is the current buildbot configuration, specifically the purpose of the various build slaves, documented somewhere?15:34
gary_postermars, not to my knowledge but maybe.  This is the closest we have: https://dev.launchpad.net/Trunk/Glue15:36
wgrantmaxb: What do you think we should do about validate_absolute_url and co.?15:39
wgrantThat's about the only thing left.15:39
marsderyck or gmb, think we can close bug #227824?15:56
mupBug #227824: BugPageTwoZero related branches table inline editing <ui> <Launchpad Bugs:Confirmed> <https://launchpad.net/bugs/227824>15:56
gmbmars, No. You can't edit branches inline.15:56
maxbwgrant: AFAICS It is not a problem that the valid_absolute_url that is being tested now accepts any scheme. It is only the DB constraint that matters.15:56
marsgmb, ok.  The "BugPageTwoZero" stuff confused me.  I thought we were a bit beyond that :)15:57
maxbIn respect of the DB constraint, I think it is dodgy for a constraint named validate_absolute_url to secretly imply a limited list of schemes.15:57
gmbmars, For BugPageTwoZero, read "ThingsWeWantedToDoForTwoZeroButDidntGetRoundTo"15:57
marshehe15:57
maxbI would be in favour of any DB columns which actually need to be scheme restricted growing something more explicit to say so15:57
deryckand there's this nifty inline title editing on launchpad, too.15:58
=== mrevell_ is now known as mrevell
=== deryck is now known as deryck[lunch]
=== leonardr is now known as leonardr-afk
=== beuno is now known as beuno-lunch
=== Ursinha_ is now known as Ursinha
UrsinhaChex, gary_poster, rockstar, bigjools, danilos, sinzui, allenap, production meeting @ #launchpad-meeting in 2 minutes16:58
bigjoolsarg16:59
danilosack16:59
Ursinhabigjools, c'mon, be nice16:59
cody-somervillehmm... has anyone else started to get ImportError: No module named apt_pkg on lucid?17:23
cody-somervilleI'm getting it when trying to run some soyuz tests17:24
jpdscody-somerville: Installed python-apt from the PPA?17:24
james_wcody-somerville: did you install the python-apt from lucid and remove the 2.5 version from the PPA?17:24
cody-somerville0.7.94.2ubuntu6 superseded the version from the PPA it seems, 0.7.94.2ubuntu5launchpad217:25
bigjoolsmaxb said he was going to upload a new one earlier17:28
maxbgrrrr17:28
bigjoolsheh :)17:29
bigjoolscody-somerville: what are you using to install updates?  Mine failed because of the broken package.17:29
maxbWould someone wield their buildd-admin powers and rescore this, please: https://launchpad.net/~launchpad/+archive/ppa/+build/169377117:29
cody-somervilleI use update-manager17:29
bigjoolsmaxb: coming up17:30
bigjoolsgeez lp is slow17:30
cody-somervillemaybe a RBS should be set for that PPA too?17:30
bigjoolsmaxb: powers wielded17:31
maxbRBS?17:34
=== matsubara is now known as matsubara-lunch
=== danilos is now known as daniloff
=== deryck[lunch] is now known as deryck
* bigjools broke devel and is fixing it17:58
bigjoolsexcept gpgme is screwed and my broken tests need it...GAH17:58
=== Ursinha is now known as Ursinha-brb
mrevellnight all18:03
maxbbigjools: hmm?18:06
bigjoolsmight be that I didn't update sourcecode....18:06
maxbIf this is the lucid pygpgme problem, just bzr pull in the pygpgme directory18:07
maxbMy branch to actually bump the revno in sourcedeps.conf is stuck in ec2 somewhere18:07
bigjoolsok will try that if rf-get fails18:07
bigjoolsI <3 my SSD18:08
maxbHrm, I've had 3 branches go through ec2 in ~3h20 today, and the fourth hasn't shown up yet18:09
bigjoolsrevno 49?18:09
bigjoolsmaxb: I did a pull and make in the pygpgme dir but no luck, still getting gpg failures in the test18:11
bigjoolsthe error changed from before though :)18:12
maxbwhat is the failure?18:12
bigjoolsGpgmeError: (7, 32816, u'Invalid argument')18:12
maxbhrm18:12
bigjoolsif you have a working box do you mind running a few tests for me so I can land my testfix?18:13
maxbI can try18:13
bigjoolsmaxb: great thanks, it's here: lp:~julian-edwards/launchpad/ppa-quota-bug-41356318:14
bigjoolsbin/test -cvvt TestPPAUploadProcessorQuotaChecks18:14
bigjoolsmaxb: actually18:15
=== gary_poster is now known as gary-lunch
bigjoolsnever mind - I did a make clean and it works now18:15
bigjoolsthanks18:15
=== beuno-lunch is now known as beuno
sinzuibarry, ping18:45
jmlleonardr-afk, some stuff came up -- let's talk tomorrow18:47
jmlg'night all19:03
=== matsubara-lunch is now known as matsubara
barrysinzui: pong19:20
sinzuibarry, does the holds on staging queue need a kick from an admin to send things to moderation19:21
sinzuihttps://staging.launchpad.net/~launchpad-users/+mailinglist-moderate19:21
sinzui^ I sent three messages, expecting 1 to go to the archive (it did), one to go to moderation, and one to be discarded19:22
sinzuibarry, I wonder if both my missing messages were discarded19:23
barrysinzui: it shouldn't.  i bet they were.  you can check the mailman logs from staging to know for sure19:23
sinzuibarry, the vete?19:24
barrysinzui: check both vette and xlmrpc19:24
sinzuithanks19:24
barrynp19:24
=== leonardr-afk is now known as leonardr
=== gary-lunch is now known as gary_poster
=== ubuntujenkins3 is now known as ubuntujenkins
=== Ursinha-brb is now known as Ursinha
maxbjelmer: Do you remember if your 'update pygpgme to trunk' ever got tested? I'm seeing failures in my python 2.6 environment21:24
mwhudsongood morning21:51
thumpermorning21:59
james_wmorning21:59
leonardrflacoste, gary and i have a question22:18
leonardrhow reliable are the zope events that tell us that a launchpad object changed?22:18
leonardrie. can we be certain that every time an object changes, an event is sent out about it?22:18
flacosteleonardr: unfortunately, that's not magic22:19
flacosteit will only be sent out if the call sites, sends the event22:19
flacosteso not totally reliable22:19
flacostealthough a lot of things break when that doesn't happen22:19
flacosteunfortunately, things have broken in the past22:20
gary_posterflacoste: but we rely on them already, yes?22:20
flacosteyes22:20
flacostefor notifications22:20
flacosteand at some points in the API22:20
gary_posterflacoste: we are intending to rely on them for invalidations of cached webservice representations.  Any reservations?22:20
flacosteseveral22:21
gary_posterheh22:21
flacostethe first one is that not all objects will send events on modification consistently22:21
flacostebecause we don't rely on them for all objects22:21
flacosteso the one for which we rely, we spot breakage22:21
flacostebut for many objects, they don't systematically send events on mutation22:22
flacostei'd say the majority even22:22
gary_posterah :-(22:22
gary_posterthat's a biggie22:22
leonardrflacoste: we're considering cache invalidation options22:27
leonardris there any amount of cache staleness we're willing to tolerate in the web service?22:27
flacosteleonardr: i say it can work, but we'll have a lot of fixup to do, that's my opinion22:27
flacostei need to run now, ttyl22:28
gary_posterleonardr, flacoste let's contonue this on email22:28
leonardrall right22:28
=== matsubara is now known as matsubara-afk
mwhudsonhooray testfix23:17
mwhudsonoh updating sourcecode failed23:17
cody-somervillemwhudson, does it complain about recursive symlink?23:33
mwhudsoncody-somerville: hm?23:33
cody-somervillemwhudson, updating the sourcecode23:33
mwhudsoncody-somerville: no23:34
mwhudsoncody-somerville: at least, not in buildbot23:34
cody-somervilleah23:34
cody-somervilleDoes testRepositorySignatureWithSigningKey fail for anybody else on db-devel?23:34
maxboh, maybe. What revno of pygpgme do you have, though23:35
cody-somerville4823:35
maxbok. I did see that fail, but I wasn't sure whether it was part of the rest of the breakage that is r49 of pygpgme23:36
cody-somerviller49? I just did ./utilities/update-sourcecode and it says 48 is the latest.23:37
mwhudsonfailure in the devel buildbot, someone broke lib/lp/soyuz/browser/tests/archive-views.txt23:38
mwhudsonlosa ping23:46
mbarnettheya mwhudson23:55
mwhudsonmbarnett: can you tell me the state of the launchpad tree on strawberry?23:55

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!