| wallyworld_ | thumper: mumble? | 00:33 |
|---|---|---|
| lifeless | http://searchengineland.com/google-fans-throw-eggs-at-houses-blurred-in-google-street-view-56763 | 00:36 |
| wgrant | lifeless: :( | 01:11 |
| wgrant | You made us the top timeout again :( | 01:11 |
| lifeless | wgrant: there's an easy answer for that | 01:12 |
| lifeless | wgrant: Fix my scaling test for binary packages. | 01:12 |
| wgrant | I looked at that last week. | 01:13 |
| wgrant | But it wasn't obvious what was doing it. | 01:13 |
| lifeless | the test ? | 01:13 |
| wgrant | I couldn't see what was causing all those queries. | 01:13 |
| lifeless | I think the test is flawed, the page doesn't show binaries without sources | 01:13 |
| lifeless | wgrant: fix the test, it will be trivial to see after that ;) | 01:13 |
| wgrant | lifeless: Which test? | 01:18 |
| lifeless | wgrant: the one I landed | 01:19 |
| lifeless | wgrant: sorry to be vague | 01:19 |
| wgrant | Heh, OK. | 01:19 |
| lifeless | look at log for my most recent Archive: landing | 01:19 |
| lifeless | late last week I think | 01:19 |
| lifeless | it adds two tests | 01:19 |
| wgrant | test_source_query_counts and co? | 01:19 |
| lifeless | one for source package scaling which works well - I got the test to demonstrate terrible scaling | 01:19 |
| * wgrant fixes. | 01:20 | |
| lifeless | the binary version *doesn't* demonstrate scaling | 01:20 |
| lifeless | so I think its misspelt somehow | 01:20 |
| lifeless | mwhudson: hi | 01:26 |
| lifeless | mwhudson: test running. you were asking.. | 01:26 |
| mwhudson | lifeless: hello | 01:26 |
| mwhudson | ah yeah | 01:26 |
| lifeless | I replied. | 01:26 |
| lifeless | over there -> | 01:26 |
| mwhudson | didn't realize testtools had discovery | 01:26 |
| lifeless | all the shiny | 01:26 |
| lifeless | AFAIK your options are testtools/subunit .run, trial --reporter=subunit, nose with the subunit plugin. | 01:27 |
| mwhudson | hmm | 01:27 |
| mwhudson | AssertionError: Unable to use discovery, must use python 2.7 or greater, or install the discover package. | 01:27 |
| lifeless | mwhudson: well, have you installed it ? :) | 01:27 |
| mwhudson | no, but i have unittest2 installed | 01:27 |
| mwhudson | is it packaged? | 01:27 |
| lifeless | it may have discover as a private module | 01:27 |
| lifeless | http://pypi.python.org/pypi/discover | 01:28 |
| thumper | wallyworld_: hey | 01:34 |
| wallyworld_ | ho | 01:34 |
| wallyworld_ | thumper: lib/lp/codehosting/vfs/branchfs.py | 01:37 |
| lifeless | === Top 10 Time Out Counts by Page ID === | 02:52 |
| lifeless | Hard / Soft Page ID | 02:52 |
| lifeless | 99 / 5038 Archive:+index | 02:52 |
| lifeless | 75 / 219 BugTask:+index | 02:52 |
| lifeless | 34 / 361 Distribution:+bugs | 02:52 |
| lifeless | 32 / 128 ProjectGroupSet:CollectionResource:#project_groups | 02:52 |
| lifeless | 23 / 378 POFile:+translate | 02:52 |
| lifeless | 17 / 51 DistroSeries:+queue | 02:52 |
| lifeless | 15 / 10 Person:+commentedbugs | 02:52 |
| lifeless | 11 / 2 Person:+bugs | 02:52 |
| lifeless | 7 / 87 Archive:+packages | 02:52 |
| lifeless | 7 / 5 DistributionSourcePackage:+changelog | 02:52 |
| lifeless | wgrant: so, did you make any progress? | 02:52 |
| === lifeless changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 4 of 10.11 | PQM open for 10.12 | firefighting: - | https://dev.launchpad.net/��� | Get the code: https://dev.launchpad.net/Getting | ||
| lifeless | poolie: hey | 03:08 |
| lifeless | poolie: what was the right spelling for LPNET_SERVICE_ROOT ? | 03:08 |
| poolie | hi thar | 03:08 |
| poolie | the trick is that they are now in launchpadlib.uris | 03:09 |
| poolie | lifeless: ^^ | 03:09 |
| lifeless | poolie: got a code fragment ? | 03:09 |
| poolie | bzr pull -d ~/src/hydrazine ☺ | 03:09 |
| lifeless | poolie: I'm writing a blog post | 03:09 |
| lifeless | totally different space | 03:09 |
| lifeless | thats a very robert answer to give | 03:09 |
| lifeless | is it | 03:10 |
| poolie | karma :) | 03:10 |
| lifeless | 'from launchpadlib.uris import LPNET_SERVICE_ROOT' ? | 03:10 |
| poolie | hang on | 03:10 |
| poolie | i wish it was fewer clicks from the project page to the source :/ | 03:10 |
| lifeless | poolie: don't you have it local ? | 03:10 |
| lifeless | I was assuming you could just copy paste a single line | 03:11 |
| poolie | ok | 03:11 |
| poolie | from launchpadlib.uris import (LPNET_SERVICE_ROOT, ) | 03:11 |
| lifeless | thanks | 03:11 |
| poolie | return Launchpad(credentials, service_root, | 03:11 |
| poolie | lplib_cachedir) | 03:11 |
| poolie | etc | 03:12 |
| poolie | btw is it just me or are the apis getting faster too? | 03:16 |
| poolie | also, is there any point filing bugs about api timeouts, or should i assume the oops system will tell you about it | 03:16 |
| lifeless | if they are not on https://bugs.launchpad.net/launchpad-project/+bugs?field.tag=timeout | 03:17 |
| lifeless | then feel free to file | 03:17 |
| lifeless | please include the page id in the title to make it easy to identify dupes | 03:18 |
| lifeless | apis will be getting faster as we fix core issues. | 03:18 |
| lifeless | https://dev.launchpad.net/LEP/PersistenceLayer may interest you | 03:18 |
| mwhudson | lifeless: did you mean to add control characters to the /topic? | 03:37 |
| lifeless | mwhudson: It was a copy paste job. | 03:39 |
| lifeless | mwhudson: they were presumably there already ;) | 03:39 |
| === mwhudson changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 4 of 10.11 | PQM open for 10.12 | firefighting: - | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | ||
| mwhudson | doesn't look like it | 03:39 |
| mwhudson | home time, probably back later | 03:39 |
| lifeless | bai | 03:39 |
| thumper | lifeless: what do we do for a self review again? | 03:42 |
| thumper | lifeless: do I just approve myself? | 03:42 |
| lifeless | you review it youself | 03:43 |
| lifeless | as normal | 03:43 |
| lifeless | just like you'd review someone else | 03:43 |
| thumper | gah | 03:45 |
| thumper | where do I change my login token? | 03:46 |
| * thumper goes to hunt through the email | 03:46 | |
| poolie | qastaging is still down? | 03:47 |
| * thumper stabs mail | 03:48 | |
| lifeless | poolie: its working for me | 03:50 |
| lifeless | poolie: https://qastaging.launchpad.net/ comes up ok, if slow. | 03:50 |
| poolie | bug 680759 | 03:50 |
| _mup_ | Bug #680759: qastaging api 503 errors <api> <Launchpad Foundations:New> <https://launchpad.net/bugs/680759> | 03:50 |
| lifeless | poolie: I think my further updates to that bug, and your mails, crossed paths. | 04:21 |
| poolie | yes they did | 04:22 |
| poolie | is qastaging now the best place to nondestructively test api clients? | 04:26 |
| wgrant | lifeless: Yeah, I got the test to fail. | 04:27 |
| wgrant | Will look at fixing the actual bug shortly. | 04:27 |
| lifeless | wgrant: \o/ | 04:31 |
| poolie | lifeless: just to be clear i'm not complaining, just wanted to flag this in case it wasn't known && matters | 04:31 |
| lifeless | wgrant: can you show me what I should be doing differently ? | 04:31 |
| lifeless | poolie: qastaging is an ok place | 04:31 |
| poolie | you just have to be robust against timeouts | 04:32 |
| lifeless | but its not resourced enough to be a reliable test environment (neither was staging - the DB massively outweighs the RAM on the db server) | 04:32 |
| poolie | fair enough; that's a good practice anyhow | 04:32 |
| lifeless | yeah | 04:32 |
| wgrant | lifeless: Your guess was correct -- the index doesn't show binaries without sources. | 04:32 |
| lifeless | wgrant: whats the diff look like ? | 04:32 |
| poolie | for the sake of my education is RequestExpired also a timeout? | 04:32 |
| lifeless | poolie: yes | 04:32 |
| wgrant | lifeless: http://pastebin.ubuntu.com/535763/ | 04:33 |
| lifeless | poolie: exact error depends on the exact bit that decided time was over | 04:33 |
| wgrant | It would be nice if mBPPH had an arg to do that. | 04:33 |
| lifeless | wgrant: yeah | 04:33 |
| lifeless | wgrant: +1 on any patch doing that :) | 04:34 |
| wgrant | I need to look at the factory and the Soyuz tests and make them match. | 04:34 |
| wgrant | Since at the moment each seems to try to pretend that the other didn't exist at the time of writing. | 04:35 |
| wgrant | lifeless: Bug #676776 should have nothing to do with the buildd-manager. | 04:46 |
| _mup_ | Bug #676776: build stuck with "Uploading build" status [UI] <confusing-ui> <recipe> <Soyuz:Incomplete> <https://launchpad.net/bugs/676776> | 04:46 |
| lifeless | wgrant: julian said 'with the workaround in place builds get stuck and I have to clean them up' | 04:46 |
| lifeless | wgrant: IMBW of course. | 04:47 |
| lifeless | but the symptoms sound exactly as he described | 04:47 |
| wgrant | lifeless: They'll get stuck in PENDING/BUILDING. | 04:47 |
| wgrant | Not UPLOADING. | 04:47 |
| lifeless | there's no xmlrpc calls involved during uploading? | 04:48 |
| wgrant | No. UPLOADING is set once the master has downloaded the files from the slave and thrown them into the upload queue. | 04:48 |
| wgrant | Then it's in p-u's hands, not b-m. | 04:48 |
| lifeless | ok | 04:48 |
| lifeless | thanks for clarifying | 04:48 |
| poolie | "no module gettextpo" indicates out of date sourcecode, i guess? | 07:12 |
| lifeless | yeah | 07:12 |
| lifeless | utilities/update-sourcecode | 07:12 |
| lifeless | + make | 07:12 |
| poolie | yup | 07:17 |
| poolie | and i guess "ImportError: cannot import name LPModerate" would be similar, but update-sourcecode hasn't fixed it | 07:21 |
| lifeless | mm | 07:21 |
| lifeless | dunno about that | 07:21 |
| wgrant | poolie: That's some mailman breakage. make clean usually fixes it for me. | 07:28 |
| poolie | i think it's a test ordering dependency | 07:35 |
| poolie | iow these tests won't pass unless something else runs first in the same process :/ | 07:36 |
| poolie | oh well | 07:36 |
| === almaisan-away is now known as al-maisan | ||
| === al-maisan is now known as almaisan-away | ||
| === almaisan-away is now known as al-maisan | ||
| adeuring | good morning | 08:47 |
| bigjools | morning | 08:59 |
| wgrant | bigjools: Did the log parser eventually finish? | 09:34 |
| bigjools | wgrant: yes but it's got a new problem | 09:34 |
| wgrant | :( | 09:34 |
| wgrant | What is it? | 09:35 |
| bigjools | hang on | 09:35 |
| bigjools | http://librarian.dogfood.launchpad.net/57530685/1z7LhiYuLtwxWSJXAxHNvbyksXZ.txt | 09:35 |
| wgrant | ... pardon? | 09:35 |
| bigjools | it's not exactly a useful traceback :/ | 09:35 |
| wgrant | The real exception wasn't logged at all? | 09:36 |
| bigjools | "Unhandled exception" is all you get | 09:36 |
| bigjools | and then I stopped working on it as I need to debug the b-m | 09:36 |
| === matsubara-afk is now known as matsubara | ||
| === mrevell is now known as mrevell-lunch | ||
| jml | lunch already | 12:01 |
| * jml is not really succeeding at work today | 12:01 | |
| jml | hey | 12:04 |
| jml | can I persuade anyone to test Launchpad against the latest pre-release of Twisted? | 12:04 |
| === al-maisan is now known as almaisan-away | ||
| jml | c'mon, there must be at least ten people with commit access around besides me. | 12:45 |
| bigjools | jml: I can help you | 12:47 |
| jml | bigjools: thanks. Want to test Launchpad against the latest pre-release of Twisted? http://twistedmatrix.com/users/glyph/10.2.0pre3/ | 12:48 |
| bigjools | jml: what are you expecting me to test? | 12:48 |
| jml | Launchpad. Update the version of Twisted that we use, run the tests, see what breaks. | 12:49 |
| bigjools | the test suite is what I needed to hear | 12:50 |
| nigelb | Congrats folks on getting rid of edge :) | 12:50 |
| bigjools | jml: now, I need to learn how to update eggs :) | 12:50 |
| jml | bigjools: doc/buildout.txt (from the root of the checkout) has a guide | 12:50 |
| bigjools | yeah reading that already | 12:51 |
| jml | bigjools: thanks. | 12:55 |
| bigjools | ok that was easy | 12:56 |
| jml | yeah. while I'm still not sold on buildout, it definitely makes upgrading dependencies easier than any other equivalent I've tried. | 12:57 |
| bigjools | jml: ok tests running, I'll let you know in about 189 minutes | 13:05 |
| jml | bigjools: woot. thanks. | 13:05 |
| === mrevell-lunch is now known as mrevell | ||
| bigjools | it'd be nice to just select trial tests | 13:07 |
| jml | + distributionmirror_prober tests! | 13:10 |
| bigjools | heh | 13:10 |
| jml | and tests that use blocking APIs to exercise Twisted servers, I guess. | 13:10 |
| bigjools | maybe it's worth ensuring we do import as TrialTestCase everywhere | 13:11 |
| jml | nah | 13:11 |
| bigjools | gimme a better idea | 13:11 |
| jml | not using trial very soon anyway | 13:11 |
| bigjools | oh, your testtools? | 13:11 |
| jml | yeah | 13:11 |
| bigjools | nyshe | 13:11 |
| jml | run_tests_with = AsynchronousDeferredRunTest | 13:12 |
| jml | (yeah, it's a horrible name) | 13:12 |
| bigjools | ummm | 13:12 |
| jml | or @run_test_with(ADRT) for individual tests. | 13:12 |
| jml | great | 13:33 |
| jml | I have to somehow figure out which revisions lifeless self-reviewed over the last three weeks. | 13:34 |
| * jml plays with log & grep, wishing again that Bazaar had a query language | 13:35 | |
| bigjools | jml: what about using the API? | 13:36 |
| jml | bigjools: it only has by-status filtering, so I might as well check bzr. | 13:37 |
| bigjools | :( | 13:37 |
| bigjools | can you get all his MPs instead? | 13:37 |
| jelmer | jml: bzr search? | 13:38 |
| jml | jelmer: is that actually useful? I thought it was mostly a text search. | 13:39 |
| jelmer | jml: I'm not sure how advanced its query language is. | 13:39 |
| jelmer | jml: What are you trying to find? | 13:39 |
| jml | jelmer: revisions of stable merged from branches authored by lifeless marked as being reviewed by lifeless | 13:40 |
| jml | jelmer: I'm almost done in my search anyway, but it would be nice to know for next time | 13:41 |
| jelmer | jml: Ah. No idea how to do that using the command line. | 13:42 |
| jelmer | But it should be possible using a few lines of Python code and the API | 13:42 |
| jml | jelmer: in general, I wish I didn't have to write so much Python to get stats about trunk landings to PQM-managed branches | 13:44 |
| jelmer | jml: Do you mean the fact that it requires Python code at all? I'd think that it would only be a few lines using the graph API. | 13:46 |
| jml | jelmer: yeah, pretty much. | 13:47 |
| === almaisan-away is now known as al-maisan | ||
| jml | bigjools: any progress on the timeout thing? | 13:54 |
| mrevell | sinzui, Hey, around? | 14:06 |
| sinzui | I am | 14:06 |
| mrevell | sinzui, I was wondering if we could have a call later today. I'm revising the docs for someone who's setting up a new project (so right from registration through to getting each individual app configured) and I wanted to know if you have any views on what I should do. | 14:07 |
| mrevell | sinzui, Also, is there a plan for resolving bug 4449? | 14:08 |
| _mup_ | Bug #4449: Project "title" is redundant with "display name" <tech-debt> <Launchpad Registry:Triaged> <https://launchpad.net/bugs/4449> | 14:08 |
| jml | mrevell: there's a spec by mpt that would be a good starting point. | 14:08 |
| * mrevell looks | 14:08 | |
| jml | mrevell: https://dev.launchpad.net/RegistrySimplifications | 14:09 |
| sinzui | I would love to talk about this. My last effort to improve th page was a disappointment. Maybe docs and inline help will help users accomplish their task | 14:09 |
| mrevell | Yeah, I think a mix is right ... it may also be time to de-mothball the screencasting apparatus, heh. | 14:09 |
| sinzui | mrevell, jml: as a matter of fact, I was considering fixing all the dead attributes during the bug jam. | 14:10 |
| mrevell | Thanks jml | 14:10 |
| mrevell | sinzui, Ah, cool | 14:10 |
| mrevell | sinzui, How does 17.15 UTC sound for a call? | 14:10 |
| sinzui | mrevell, I think I am in a team lead meeting at that time | 14:11 |
| sinzui | mrevell, 15:00 or 16:00? | 14:11 |
| mrevell | sinzui, 15.00 works for me | 14:12 |
| sinzui | okay | 14:12 |
| * jml hugs addDetail | 14:33 | |
| jml | anyone seen anything like this before? http://paste.ubuntu.com/535913/ (librarian db constraint violation) | 14:34 |
| === matsubara is now known as matsubara-lunch | ||
| bigjools | jml: no progress, although I've written 3 lines of code that will lock the slave manager in memory :) | 14:41 |
| jml | bigjools: how will that help? | 14:41 |
| bigjools | jml: I have a theory that it's getting paged out and thus can't respond quick enough to connections | 14:42 |
| bigjools | although I'm starting to doubt that | 14:43 |
| bigjools | but it's worth trying nonetheless | 14:43 |
| === salgado is now known as salgado-lunch | ||
| bigjools | jml: I've also analysed the logs a bit to see if there's any pattern in the builders that are failing | 14:45 |
| jml | a minimal repro client might be a better starting point -- at least then we can completely eliminate lp.buildmaster. | 14:45 |
| bigjools | some are failing a lot more than others | 14:45 |
| jml | bigjools: oh. interesting. | 14:45 |
| bigjools | https://bugs.edge.launchpad.net/launchpad-buildd/+bug/586359/comments/5 | 14:45 |
| _mup_ | Bug #586359: Virtual builders are sometimes very slow to accept connections <Launchpad Auto Build System:Triaged> <https://launchpad.net/bugs/586359> | 14:45 |
| jml | this librarian failure is perplexing | 14:46 |
| bigjools | jml: I also looked at logs from April and the problem is exactly the same there | 14:47 |
| bigjools | jml: when are you getting that librarian error? | 14:47 |
| jtv | henninge: can't get onto the other channel right now, but that export-to-branch failure you pasted was simply yet again the same general config problem. No reason to believe that the script itself hit any real problem otherwise. | 14:47 |
| jml | when running lp.codehosting.codeimport.tests.test_workermonitor.TestWorkerMonitorUnit.test_finishJob_uploads_nonempty_file_to_librarian | 14:47 |
| bigjools | I notice it's in retry_transaction_decorator | 14:48 |
| bigjools | which is suspicious | 14:48 |
| henninge | jtv: it ran fine mostly. | 14:48 |
| jtv | henninge: yes, it was just logging that it was backing off from one branch to avoid a race condition. But that caused the oopsing machinery to log an oops, and that part was broken as elsewhere. | 14:49 |
| henninge | jtv: I filed bug 680908 about what we could do about it. | 14:50 |
| _mup_ | Bug #680908: Script translations-export-to-branch needs its own config section <Launchpad Translations:New> <https://launchpad.net/bugs/680908> | 14:50 |
| jml | it's intermittent | 14:56 |
| jml | but only seems to be a thing w/ my testtools branch... | 14:57 |
| * jml does some bisecting | 14:57 | |
| sinzui | mrevell, mumble? | 14:59 |
| mrevell | sinzui, Sure. | 14:59 |
| jelmer | rockstar: hi, do you know if there's a reviewer meeting today and who's chairing? | 15:04 |
| rockstar | jelmer, no idea. bac would be the guy to ask. | 15:04 |
| jelmer | rockstar: thanks | 15:04 |
| === al-maisan is now known as almaisan-away | ||
| henninge | jtv: Hey | 15:16 |
| === almaisan-away is now known as al-maisan | ||
| sinzui | mrevell, http://curtis.hovey.name/2010/11/02/encouraging-contribution-on-the-project-page/ | 15:19 |
| jml | bigjools: as best as I can tell, (curse this intermittency), it's some interaction between the builder tests and the workermonitor tests. | 15:21 |
| jml | or, to be precise, that's at least one way of reproducing the error. | 15:21 |
| bigjools | jml: suck :/ | 15:21 |
| bigjools | if you run both? | 15:21 |
| jml | if I run both test_builder and test_workermonitor, then it sometimes fails. | 15:22 |
| jml | so far haven't got the error from just running test_workermonitor | 15:22 |
| jml | got a while loop going running it over & over. | 15:22 |
| bigjools | but only sometimes .... urgh | 15:22 |
| jml | yeah. | 15:23 |
| jml | speaking of only sometimes | 15:23 |
| * jml goes downstairs to get some chocolate | 15:24 | |
| === matsubara-lunch is now known as matsubara | ||
| rockstar | sinzui, ping | 15:35 |
| sinzui | hi rockstar | 15:35 |
| === al-maisan is now known as almaisan-away | ||
| rockstar | sinzui, I wonder if you might be able to help me sort out some bug notification/ownership issues, since the bugs guys are gone and you seem to always know about permissions. | 15:36 |
| sinzui | okay | 15:37 |
| * jcsackett hates cleaning up a bad merge state... | 15:37 | |
| rockstar | sinzui, the U1 team isn't getting notified on bugs filed on the u1 client filed through apport. They're trying to find out why. | 15:38 |
| dobey | hrmm, there have been some changes to privacy in lp recently haven't there? | 15:38 |
| sinzui | rockstar, I could be via a structural subscription in a indirect team | 15:39 |
| * sinzui looks | 15:39 | |
| dobey | so it looks like crash reports are all private until the rescanner does some magic | 15:40 |
| dobey | and this seems to be our issue | 15:40 |
| === almaisan-away is now known as al-maisan | ||
| rockstar | dobey, but also that once the rescanner is done, it still doesn't send notifications. | 15:42 |
| dobey | sure | 15:42 |
| dobey | but i'm ok with not getting more e-mail ;) | 15:42 |
| * jml sees more evidence for merging #launchpad & #launchpad-dev | 15:43 | |
| rockstar | jml, +1 | 15:46 |
| sinzui | rockstar, ~ubuntuone-hackers is the bug supervisor or ubuntuone-client. They should be getting all bug reports via email | 15:51 |
| rockstar | sinzui, https://bugs.launchpad.net/malone/+bug/425127 | 15:53 |
| _mup_ | Bug #425127: private bugs in packages people with access to private bugs are subscribed to don't generate emails <story-better-bug-notification> <Launchpad Bugs:Triaged> <https://launchpad.net/bugs/425127> | 15:53 |
| sinzui | I was looking at the project, I will look at the package | 15:54 |
| sinzui | If my reciprocal bug links were in production I would have seen that | 15:55 |
| jml | hah | 15:56 |
| sinzui | rockstar, Ubuntu One hackers has a structural subscription. A team admin can remove it | 15:57 |
| sinzui | https://launchpad.net/ubuntu/+source/ubuntuone-client | 15:57 |
| === salgado-lunch is now known as salgado | ||
| * jml blags | 16:12 | |
| === beuno is now known as beuno-lunch | ||
| jml | :( | 16:39 |
| jml | If I run any subset of the buildmaster.tests.test_builder tests, I don't see the error. :( | 16:40 |
| gary_poster | jml: May I add the Risks section (and the sub-section of experiment, goal/design/result) from https://dev.launchpad.net/Foundations/Proposals/Template to the LEP template so I can delete it ? | 16:44 |
| jml | gary_poster: sure. | 16:45 |
| gary_poster | thanks | 16:45 |
| jml | gary_poster: I think I would like the LEP template to imply that it's good to think of more than one implementation | 16:46 |
| jml | gary_poster: but I think those are definitely good things to have when thinking about an implementation. | 16:47 |
| mars | sinzui, ping, do you have a moment to help with a CHR documentation question? | 16:50 |
| * bigjools shakes fist at Flash plugin | 16:51 | |
| lifeless | jml: I put the things I did into the wiki page | 16:54 |
| sinzui | mars, I am going into a meeting. I can type on irc | 16:54 |
| lifeless | jml: but you can also query for merged mps, and then check the reviewers vs the mp branch owner | 16:54 |
| mars | sinzui, just to clarify, so I don't do the wrong thing for reassigning Launchpad Answers (again) - "Retarget" means "Change the assigned project", correct? | 16:55 |
| jml | lifeless: 'bzr log --line -r date:2010-10-31.. | grep r=lifeless' worked well enough | 16:56 |
| sinzui | mars, yes, retarget does mean change the assigned project or distribution | 16:57 |
| lifeless | jml: sure | 16:57 |
| lifeless | thank you | 16:57 |
| mars | sinzui, perfect, thank you | 16:57 |
| jml | np | 16:59 |
| gary_poster | jml, I wasn't going to include the word "Implementation" in the copy to the LEP because it is confusing. That said, I think of "Workflows" as a bit of an implementation proposal. If the goal is to do blah blah for the webservice, how we accomplish blah blah is described in a workflow. That workflow is a possible approach to the goals. The workflow is a design that has risks. | 17:01 |
| gary_poster | So that's kind of where I think it goes | 17:02 |
| jml | gary_poster: *nod* | 17:02 |
| gary_poster | cool | 17:02 |
| jml | gary_poster: I don't like having workflows in the lep, actually :) | 17:02 |
| gary_poster | jml: gotcha | 17:02 |
| jml | gary_poster: I only added it because I wanted to get beuno to stop shouting | 17:02 |
| gary_poster | but then all the LEP would be about is agreeing on goals? | 17:02 |
| jml | gary_poster: yep | 17:02 |
| gary_poster | hm | 17:02 |
| jml | gary_poster: goals, constraints, requirements | 17:02 |
| jml | analysis, rather than design | 17:03 |
| gary_poster | I see | 17:03 |
| gary_poster | I'd be happy to have that discussion separately, jml | 17:03 |
| jml | gary_poster: but if it gets used for design because managing two docs is too much work, I'm ok with that | 17:03 |
| gary_poster | jml, ok, we'll try separate docs then | 17:04 |
| === al-maisan is now known as almaisan-away | ||
| === beuno-lunch is now known as beuno | ||
| beuno | oh hai jml! | 17:27 |
| jml | beuno: hi | 17:27 |
| gary_poster | lifeless: would you be available for a quick call nowish? | 18:01 |
| lifeless | sure | 18:02 |
| gary_poster | thanks | 18:02 |
| mrevell | I'm off for the day. Night | 18:03 |
| bigjools | night all | 18:07 |
| jml | lifeless: the testing-cabal PPA now has testtools and testrepository building in it | 18:09 |
| jml | lifeless: I've done enough learning for now, so I'm going to open it up for the experts to fix up my mess :) | 18:09 |
| === benji is now known as benji-lunch | ||
| lifeless | jml: cool | 18:26 |
| jcsackett | back | 18:27 |
| jml | you know it's getting serious when you have to draw a truth table to help you debug something | 18:44 |
| lifeless | TTF | 18:45 |
| jml | thirty-two combinations of fail | 18:46 |
| lifeless | TTTTF? | 18:47 |
| jml | yeah | 18:47 |
| jml | lifeless: in case you have any insights: http://paste.ubuntu.com/535913/ is the error I'm seeing. It's during lp.codehosting.codeimport.tests.test_workermonitor.TestWorkerMonitorUnit.test_finishJob_uploads_nonempty_file_to_librarian but the error doesn't appear to happen when the that module's tests are run in isolation | 18:49 |
| jml | lifeless: I can reproduce the error semi-reliably running the buildmaster.tests.test_builder tests before, but I've yet to get close to a minimal set | 18:50 |
| jml | (debugging this one by the numbers, since my brain is too shot for creative debugging) | 18:50 |
| jml | fuller error: http://paste.ubuntu.com/536005/ | 18:51 |
| lifeless | jml: brief call? | 18:52 |
| jml | lifeless: very brief, sure. | 18:52 |
| === benji-lunch is now known as benji | ||
| jml | lifeless: it *looks* like there was another test in test_workermonitor that was dirtying the db via the librarian without declaring as such... | 19:53 |
| jml | lifeless: I have no idea why I could only reproduce the error with test_builder in the mix | 19:53 |
| lifeless | weird | 19:53 |
| lifeless | jml: I'm glad you found it | 19:53 |
| jml | it would be relatively straightforward to put a thing in the librarian layer that checked to see if rows were added ... | 19:54 |
| jml | lifeless: I'm not going to dig any deeper, fwiw. | 19:54 |
| lifeless | jml: fair enough | 19:55 |
| * jml ec2 tests & goes away to eat, drink (medicine) and be lazy | 19:55 | |
| jml | lifeless: thanks for the help. | 19:55 |
| lifeless | anytime | 19:58 |
| === salgado is now known as salgado-afk | ||
| thumper | morning | 20:29 |
| === matsubara is now known as matsubara-afk | ||
| cr3 | can someone explain why some methods in launchpad take a quantity argument instead of using slices? | 21:15 |
| wgrant | cr3: Do you have an example? | 21:21 |
| wgrant | It's possibly webservice-related, but it's hard to say. | 21:22 |
| cr3 | wgrant: grep -r quantity= returns examples such as: IBranchView.latest_revisions, IProductSet.latest | 21:23 |
| cr3 | wgrant: there are some comments relating to the webservice interface, which decorates the functions using call_with(quantity=None), whereas the default is a hard coded value like 10 for example. | 21:24 |
| cr3 | wgrant: I would've expected the other way around though, ie limiting the webservice interface to some hard coded value and allowing internal calls to slice and dice | 21:24 |
| poolie | hi all | 21:29 |
| lifeless | cr3: slices require lazy objects, these are method calls | 21:31 |
| lifeless | as soon as you're not in ResultSet territory, you'll find we don't slice | 21:32 |
| lifeless | and where we do, its generally a bug. | 21:32 |
| cr3 | lifeless: some calls like Branch.latest_revisions return a Storm ResultSet which could be sliced and diced though, haven't looked at many others, but it might just be remnants from the good ol' days | 21:32 |
| wgrant | lifeless: But the methods should be returning (Decorated)ResultSets :/ | 21:32 |
| lifeless | wgrant: not as of yesterday. | 21:32 |
| wgrant | lifeless: How far through the design are you? | 21:33 |
| cr3 | wgrant: do you guys normally use the launchpad DecoratedResultSet rather than the default Storm one? | 21:33 |
| wgrant | cr3: Is there a Storm DRS? | 21:33 |
| lifeless | cr3: when we eager load, yes. | 21:33 |
| cr3 | wgrant: no, the default ResultSet I meant | 21:33 |
| lifeless | cr3: we eager load in many, but not enough places. | 21:33 |
| wgrant | cr3: We use the standard ResultSet in most places. | 21:34 |
| cr3 | lifeless: wait, you use DecoratedResultSet or quantity when eager loading? (I need to finish reading that related thread on canonical-tech) | 21:35 |
| wgrant | DRS is for eager loading. | 21:35 |
| lifeless | DRS for eager loading. quantity when specifying up front what we need (which is a different, not necessarily better or worse) pattern. | 21:36 |
| wgrant | quantity is for broken methods that don't return a ResultSet or derivative. | 21:36 |
| lifeless | s/broken/other/ | 21:36 |
| wgrant | Until yesterday they were broken, now they might not be :P | 21:36 |
| lifeless | wgrant: they weren't broken before | 21:36 |
| lifeless | they are symptoms of our lack | 21:36 |
| lifeless | resultsets do not cache | 21:36 |
| cr3 | wgrant: was there a fix in DRS yesterday? is it in trunk so that I can pull it and see? | 21:37 |
| lifeless | so they aren't suitable to pass into methods | 21:37 |
| wgrant | cr3: No, lifeless is just changing everything. | 21:37 |
| wgrant | EVERYTHING> | 21:37 |
| cr3 | wgrant: just when I thought I understood a thing or two about launchpad :) | 21:37 |
| lifeless | we have several overlapping problems | 21:38 |
| lifeless | firstly, our can-read assertions are scattered all over | 21:38 |
| lifeless | we should have them in one layer so that we don't *return from queries* data the user [whose behalf we are acting on] shouldn't see. | 21:39 |
| lifeless | secondly, templates and logic like canonical_url is per-object | 21:39 |
| lifeless | but for efficiency and to map well to our data storage layer we need to work per-set | 21:39 |
| wgrant | lifeless: is the API going to be similar to the unannounced new webservice model? | 21:40 |
| lifeless | we should have a layer that enforces set based access to the data store and makes it easy to put set/group based idioms higher in the codebase | 21:40 |
| lifeless | wgrant: they have strong parallels and I know that the webservice folk have added to their thoughts the ideas I've put up about the group based idioms in LP's internal code. | 21:40 |
| lifeless | wgrant: Gary expressed a desire to have them be harmonious and perhaps even trivially layerable. | 21:41 |
| lifeless | wgrant: I think they have different constraints, but they should be similar in style even if not in exact detail - initially. | 21:41 |
| lifeless | wgrant: the webservice stuff is on dev.l.n if you're interested | 21:41 |
| gary_poster | wgrant, also fwiw, the new webservice model won't be announced but proposed | 21:42 |
| wgrant | lifeless: Yeah, I've been reading it. | 21:42 |
| wgrant | It looks really great. | 21:42 |
| lifeless | thirdly, the failure mode we have today is death-by-queries | 21:42 |
| lifeless | we should have a mechanism such that when we think we've prepped for a page, we turn off data access | 21:42 |
| wgrant | gary_poster: You're silently working out how to make the webservice fantastic. I think that deserves announcement. | 21:43 |
| lifeless | cr3: ^ | 21:44 |
| gary_poster | :-) thanks wgrant. (where "you" includes a bunch of people.) And I hear you. I'll do it. | 21:44 |
| lifeless | cr3: so I've proposed an explicit persistence layer | 21:44 |
| cr3 | lifeless: death-by-queries reminds me of an idea I had recently about unit testing where it might be desirable to decorate or provide some assert methods to make sure a certain number of queries are being performed, ie to detect when fetching a list of items results in a query per item for example | 21:44 |
| lifeless | cr3: it will only expose things the current user can see, will return objects that have no backend-connection so cannot trigger bad behaviour by mistake | 21:45 |
| lifeless | cr3: search for HasQueryCount in the lp code base | 21:45 |
| cr3 | lifeless: awesome! will do | 21:45 |
| lifeless | cr3: the persistence layer will also be a clear boundary, which we can switch out for tests that are not testing the data storage story | 21:46 |
| wgrant | gary_poster: My main remaining gripe is the lack of transactions, but that's probably impossible to do, and it's mostly mitigated by the collection fixes. | 21:46 |
| lifeless | wgrant: batch PATCH gives you near-enough IMO | 21:46 |
| wgrant | Exactly. | 21:46 |
| gary_poster | wgrant, not impossible--leonardr talks about how to do it in his REST book--but not terribly attractive IMO | 21:46 |
| lifeless | wgrant: we're -not- going to expose sql transaction objects on the net. | 21:46 |
| gary_poster | yay | 21:46 |
| wgrant | lifeless: Batch PATCH and the atomic collection retrieval does almost everything needed, I think. | 21:47 |
| lifeless | yeah | 21:47 |
| cr3 | lifeless: that reminds me, I didn't notice any unit tests for the database functions themselves, ie no database/tests directory for example. I decided to have tests specifically for my database code in my latest project | 21:47 |
| lifeless | cr3: lib/canonical/database/ftests/ ? | 21:47 |
| wgrant | cr3: How are they different from the model tests, given that the layers are one? | 21:48 |
| lifeless | there are some big questions about where 'is allowed to be related' logic should go in my new layer | 21:48 |
| lifeless | but I think things like 'when a bug changes notify X Y and Z' very clearly do not belong in it. | 21:49 |
| cr3 | lifeless: well, I would expect tests to exercise each of the functions defined in trusted.sql for example, which it doesn't look like lib/canonical/database/ftests/ is for | 21:49 |
| lifeless | equally 'who will be notified' isn't a storage problem, its built on it. | 21:49 |
| lifeless | cr3: oh. mmm, thats kindof bootstrap code. | 21:50 |
| lifeless | sure, you could test it. It might be nice to do so. | 21:50 |
| cr3 | wgrant: not necessarily true, some database integrity should be caught at the database layer rather than the application layer, so exercising the model layer does not validate database integrity per se | 21:51 |
| cr3 | wgrant: I also find it easier when defining a function in sql to have tests specifically exercise that function rather than going through the model layer | 21:53 |
| lifeless | cr3: this asumes you have different layers. | 21:54 |
| lifeless | cr3: triggers are evil though and we should get rid of ours. They are workarounds for poor ORM support. | 21:54 |
| lifeless | perhaps. We can keep them now we have them, but we've triggered storm bugs by using them, and will in the future) | 21:54 |
| cr3 | lifeless: do you mean problems refreshing the cache in storm when triggers update objects without the orm knowing? | 21:56 |
| lifeless | I mean bugs. | 21:56 |
| lifeless | like updating all columns rather than changed objects | 21:56 |
| lifeless | changed columns that is. | 21:56 |
| cr3 | lifeless: oh, has that been fixed yet? I've experience that problem myself and it's been a pain :( | 21:57 |
| lifeless | in 0.17 | 21:57 |
| lifeless | it broke other stuff, which is fixed in 0.18 | 21:57 |
| cr3 | lifeless: thanks for the info, will have a look | 21:57 |
| lifeless | you may find https://dev.launchpad.net/LEP/PersistenceLayer my high level problem statement interesting | 21:58 |
| cr3 | lifeless: you're ambitious: # None of our tests of Launchpad functionality require a real database to execute. | 21:59 |
| lifeless | we can't tell if its complete if we don't do that | 22:00 |
| cr3 | lifeless: I would've gone for some ratio or some rough best effort figure, I would be afraid of the 80-20 rule where you'll spend 80 percent of your time finishing off the 20 percent remaining tests hitting the database | 22:01 |
| cr3 | lifeless: whereas I'd consider it a tremendous success if 80 percent of tests didn't hit the database, testing would be blazingly fast (modulo windmill and doctests) | 22:02 |
| lifeless | cr3: if we're going to do it, we need to -do it- | 22:04 |
| lifeless | cr3: the last thing we need is a half (or 80%) deployed thing. | 22:04 |
| lifeless | that would leave the remaining 20% still existing : having 20% of our pages be terribly slow is not acceptable. | 22:04 |
| cr3 | lifeless: another way of looking at it is 20% of each page being slow, which still means 5 times quicker than before. but I can appreciate your point of view though, and I suspect you already have a design in mind that actually makes it feasible. in lifeless we trust :) | 22:06 |
| wgrant | What do Landscape and U1 do? | 22:09 |
| wgrant | It seems that we are trying to solve all sorts of problems that they probably solved years ago. | 22:10 |
| lifeless | landscape makes heavy use of 'collections' idioms. | 22:10 |
| lifeless | In my discussions with jkakar he has expressed interest in what I've sketched out, and said they suffer the same things we do. | 22:10 |
| lifeless | so, for them, they work harder at performance. Its not intrinsically easier for them. | 22:10 |
| lifeless | U1, AIUI, are built on django | 22:11 |
| wgrant | And U1 uses Couch and all sorts of other evil, I suppose. | 22:11 |
| lifeless | which has a less strictly-object flavour and so is less driven to the pathology Launchpad is | 22:11 |
| lifeless | but they've had plenty of requests taking thousands of queries. | 22:11 |
| lifeless | I'm not aware of any Canonical web property that /has/ this solved pervasively, nor solved at the scale (of data model complexity) that Launchpad needs. | 22:12 |
| lifeless | ... I've been asking :) | 22:12 |
| wgrant | Has anyone else? | 22:13 |
| lifeless | hibernates HQL, sqlalchemy's QL - these are great things to look at. | 22:13 |
| wgrant | Right. | 22:14 |
| lifeless | functional approaches, and things like map-reduce are very useful inspiration too | 22:14 |
| poolie | how/when is the sampledata updated? | 23:52 |
| lifeless | fridayish | 23:53 |
| poolie | iow why has my merge from devel got large sampledata changes compared to devel? | 23:53 |
| lifeless | oh | 23:53 |
| poolie | in https://code.launchpad.net/~mbp/launchpad/dkim/+merge/41689 | 23:53 |
| lifeless | when commits are made | 23:53 |
| lifeless | did you do 'make sampledata' | 23:53 |
| mwhudson | large diffs tend to come from (sometimes tiny) version differences in postgres | 23:59 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!