[00:01] huwshimi: http://t.co/O68m8Ur7 if you haven't seen this sort of thing before [00:03] mwhudson: Oh, yeah, I saw that :) [00:03] good good [00:07] lifeless: My bug supervisor change, I'd wager [00:08] lifeless: So, on one hand it's just one query, but on the other, I'd prefer to reduce it. [00:08] However, I think this branch is large enough. [00:09] StevenK: if its one Full Stop, its ok. If its one per row, aieee. [00:09] i think his talk is insufficiently smug [00:09] i know some people who could help though [00:11] lifeless: One full stop. The test case adds 5 bug tasks. [00:12] StevenK: they all have different supervisors etc? [00:12] StevenK: [paranoid] [00:12] I doubt it. [00:29] wgrant, ok looking at jubany now [00:30] poolie: Thanks. [00:30] StevenK: it may not matter, make an educated assessment [00:30] is banana the front end for both edge and lpnet? [00:30] StevenK: I'm just noting the possibility that the test isn't able to assess a regression here [00:30] s/the/a [00:30] yes [00:30] poolie: One of. [00:30] actually its primary [00:31] normally all traffic goes through it [00:31] So nutmeg usually sees next to no traffic? [00:31] because haproxy doesn't know how to cooperate in a cluster [00:31] yeah [00:31] any traffic nutmeg gets it sends to banana [00:31] if banana goes down nutmegs backup path is direct to the appservers [00:32] And if nutmeg drops, banana doesn't care. [00:33] right [00:35] To clarify, clients send to both. [00:35] But nutmeg normally forwards to banana. [00:36] They should really be the frontends for all LP services. [00:36] But codehosting/PPA are a bit special at the moment, unfortunately. [00:36] And mailman. [00:36] nutmeg apache forwards to banana [00:37] banana apache forwards to banana haproxy [00:37] haproxy then does the same dance because we can't rely on apache [00:37] i like the fact they're actually related [00:37] anyhow, udd certainly does try to use lpnet [00:37] restfulclient's api is so wacky perhaps it's not having the intended effect [00:38] hm [00:38] that is our ip though [00:39] poolie: So, with this change to the bug portlet, the ordering has changed from alphabetical, to usage count. Would you add a test for that, or should I just delete these current (now mostly useless) tests? [00:40] i would add a test for it [00:41] it's probably not so much likely they will come back in the wrong order as that it will blow up [00:41] poolie: Ok thanks [00:41] a python test obviously [00:43] wgrant, it certainly thinks it has an lpnet connection [00:43] poolie: Yeah :) [00:43] poolie: Hmm. I wonder what's using edge, then. [00:43] That IP address looks like jubany, doesn't it? [00:43] Yes [00:43] wgrant the apache log line you quoted is to api.l.n, not edge [00:44] Erm, really? [00:44] So it is. [00:44] I may have synced the wrong log. That would be bad...... [00:44] AHEM [00:44] Yes [00:44] That was an lpnet log. [00:44] Sorry [00:45] Nothing from package-import in the actual edge logs. [00:47] that's ok :) [00:48] wgrant: so, I nagged brian for nought? [00:51] lifeless: No, I found bdmurray in my correct analyses a couple of weeks ago. He's the second-top user on real edge [00:51] phew. tanks [00:53] didrocks 28614, bdmurray 10237, ubuntuqa 5671 [00:53] Top three from yesterday [00:55] * StevenK looks forward to edge.launchpad.net being NXDOMAIN [00:57] wgrant: want to mail didrocks ? [00:59] Once I have more data. [01:01] I think didrocks' thing is new, actually. [01:01] It's showing up every day now. [01:01] But wasn't there when I last checked a couple of weeks back, IIRC. [01:22] bbs, grocery shopping time [01:27] hm [01:28] i'm trying to get its tests to run in semi isolation [01:49] hm, launchpad's TestCase now depends on having rabbit running? [01:58] wgrant, https://code.launchpad.net/~mbp/launchpad-buildd/lpbuildd-tests/+merge/82837(small) [01:58] wgrant, https://code.launchpad.net/~mbp/launchpad-buildd/lpbuildd-tests/+merge/82837 (it's pretty small) [01:58] pht [02:18] poolie: TestCase? shouldn't. [02:19] poolie: specific tests? yes, absolutely [02:28] poolie: Any reason to continue bumping to a new version so rapidly? [02:28] poolie: Also, your wraping in debian/control's Depends line is sort of screwy. [02:43] mwhudson: is there a generic django facility for 'I need an odd bit of metadata like a timestamp for my whole project' [02:43] mwhudson: ? [02:45] lifeless: Can you elaborate a little more about what you want to do with it? [02:46] I'm just coming in on the conversation now, so I might be being unhelpful :) [02:47] huwshimi: that was the entire conversation [02:47] huwshimi: I have a single datetime I need to store for python-oops-tools [02:47] huwshimi: (that is lp-oops.canonical.com) [02:47] huwshimi: it records the point in time which pruning has been done to [02:48] lifeless: And you just need a way of storing that timestamp? [02:48] and reading it back, yes [02:53] lifeless: There isn't a generic global config table or anything of that sort. You could do it by setting up a simple model and could probably use the unique flag. I'm just checking. [02:54] yeah, I can do it manually [02:54] was just wondering if I could avoid that overhead [02:55] lifeless: Actually, you might be able to use the site model [02:55] lifeless: You might be able to hook something on to that [02:56] looks equivalently complex - http://stackoverflow.com/questions/2821702/how-do-you-extend-the-site-model-in-django [02:57] lifeless: yeah, quite possibly [02:57] lifeless: It's the only unique site wide table though [02:59] lifeless: It might still be simplest to add another model that is unique to a Site to store the timestamp [03:00] huwshimi: I very much doubt that the oops app supports multiple sites as-it is [03:01] huwshimi: so I'm going to do the simplest thing I can, which is a model with one row [03:02] lifeless: Sure, I just thought that would be a clean way to do it. That way you can just do Site.objects.get_current().my_timestamp [03:03] A default site is created when you set up the django project [03:03] I see; it probably is pretty clean [03:03] however, I've gotten all my stuff setup in the mean time [03:04] lifeless: Yeah, well doing it another way would be fine too [03:05] lifeless: Oh actually it probably would have been Site.objects.get_current().configs.my_timestamp [03:05] lifeless: It's just how I would have done it if I was setting up project wide configs [03:05] lifeless: not sure i understand [03:06] * mwhudson catches up [03:06] lifeless: model with one row sounds easiest, tbh [03:26] lifeless, yep does fairly clearly depend on rabbit [03:26] will file a bug i guess [03:27] poolie: that doesn't make sense to me, as our unit tests would die horribly wouldn't they ? (the ones with no layer) [03:28] maybe none of them have no layer? [03:28] istr rabbit says it can't be torn down [03:28] so things with no layer always have it accidentally running? [03:29] anyhow, https://bugs.launchpad.net/launchpad/+bug/892938 has a simple demonstration [03:29] <_mup_> Bug #892938: lp.testing.TestCase inherently depends on rabbit < https://launchpad.net/bugs/892938 > [03:29] argh django. datetime should *always* have tzinfo. What are you *thinking* [03:30] Django has a lot of similar moments. [03:30] lifeless: you assume much by mentioning thought [03:30] lifeless: https://code.djangoproject.com/ticket/17062 is fun [03:31] oh hey look, that's fixed apparently [03:31] hah [03:31] nice job with the bug mail, trac [03:31] 19 hours ago [03:31] wgrant, i don't know, do you think i should stay on the same debian version? [03:31] it just seemed likely to cause less confusion if i did increment it [03:31] especially as it's sometimes rebuilt and might lose the recipe version etc [03:32] mwhudson: thanks for the link ! [03:32] also, do you think i should not wrap the depends? [03:32] or just wrap them more tidily [03:33] poolie: Wrap them more tidily. [03:33] I think we should bump the version for releases. [03:33] Not for every revision. [03:34] what is a release in this context? [03:34] every time we ask it to be deployed? [03:34] mwhudson: does delete() return the row count deleted ? [03:34] anyhow the basic thing is ok with me [03:35] lifeless: don't know [03:35] i'm not sure when it will actually get deployed so i suppose i'm being optimistic [03:35] any other comments on that patch? [03:35] hah, win [03:35] File "/home/robertc/source/launchpad/oops-tools/working/parts/django/django/db/models/query.py", line 430, in delete [03:35] "Cannot use 'limit' or 'offset' with delete." [03:35] AssertionError: Cannot use 'limit' or 'offset' with delete. [03:36] Is that surprising? I didn't think postgres supported that. [03:36] MySQL does on UPDATE, IIRC. [03:37] I thought LIMIT worked with DELETE? [03:38] no, the surprising thing is that django generated it in the query [03:38] and barfs on it [03:38] :) [03:38] rather than e.g. doing a subselect to identify the rows to delete [03:39] removed: lib/canonical/launchpad/icing/style-3-0.css [03:39] It cannot be! [03:39] lifeless: And you're suprised Django is naive? [03:39] wgrant: Wait. jtv will accidently resurrect it via lint changes. [03:39] Heh [03:40] anyone around that might be able to help with increasing the size of a pap ? [03:40] err ppa I mean [03:40] Dr_Who: How urgent is it? [03:40] wgrant: Did that just get merged? [03:41] huwshimi: Yep [03:41] well I'm trying to get a large set of packages built against libjpeg-turbo so if things continue well, we'll be able to replace libjpeg with libjpeg-turbo in precise [03:41] (it's about a 2x-4x perf boost for intel and armel) [03:41] funnier in scotland [03:42] if tomorrowish was possible wgrant that'd be awesome [03:42] Dr_Who: Probably best to ask at https://answers.launchpad.net/launchpad/+addquestion [03:42] ok thanks === wgrant changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugtasks: 292 [03:43] wgrant: Oh, so it did. [03:43] wgrant: https://code.launchpad.net/~lifeless/python-oops-tools/prune/+merge/82840 [03:44] Grar, I hate tests that touch DB users in bad ways. [03:45] Like TestPersonSetMerge [03:55] wgrant: https://code.launchpad.net/~lifeless/python-oops-tools/bug-892917/+merge/82841 [03:56] lifeless: How does oops-tools' prune differ from oops-datedir-repo, apart from having an implied path and DB-backed last prune date? [03:57] lifeless: As for the other branch, don't you need to change the current model too>? [03:59] wgrant: https://code.launchpad.net/~lifeless/python-oops-tools/bug-892914/+merge/82842 [03:59] wgrant: yes, I do, will fix that [04:00] wgrant: oops-tools prune only prunes the db, doesn't touch disk. [04:00] wgrant: for two reasons; A) if it does delete everything, I wanted a Plan B. and B) two-phase commit is hard. [04:00] wgrant: so I just plan on us running both pruners. [04:01] wgrant: other than UI cruft like help text, the duplication is ~ 4 lines long. [04:01] wgrant: well below my generalise-it threshold [04:04] lp is in testfix mode or something? [04:04] lifeless: Ah, it should probably say that. [04:04] lifeless: It currently says it deletes stuff from the repository. [04:04] poolie: No [04:05] Commit message [[r=mbp] delete canonical.buildd, now its moved to launchpad-buildd] does not match commit_re [(?# Strong suggestion: use ``bzr lp-land`` or ``ec2 land``. Manual submission notes: your submission message needs at least one of [r=...] [rs=...] or [release-critical=...] AND at least one of [no-qa] [incr] [rollback=...] [bug=...].)(?is)(?=(\s*\[[^\]]+\])*\s*\[(release-critical=[^\]]+|rs?=[^\]]+)\])(?=(\s*\[[^\]]+\])*\s*\[(incr| [04:05] no-qa|bugs?=\d+(,\s*\d+)*|rollback=\d+)\])] [04:05] huh [04:05] poolie: Needs a QA tag [04:05] it's associated with a bug [04:05] pht [04:05] poolie, hi [04:05] poolie, so, i'll definitely need to get a version of LP running locally? [04:06] yeah that's a good idea [04:06] dev.launchpad.net has scripts for how to do it [04:06] it is mostly scripted but it may take a while [04:06] or, rather, it's all scripted, it will take a while, it is possible but unlikely the scripts may get snagged and then we can help you get them going again [04:07] poolie, guess i can always have a go at designing the source-controlled wiki feature i'm interested in too then :-) [04:07] test_poppy failed again :/ [04:07] that would be great [04:07] i discussed that with mgz or somehting [04:07] a few weeks ago [04:07] poolie: Hm, never seen that. [04:07] i was just starting to put in markdown support on the weekend [04:07] poolie: Apart from your email. [04:08] poolie, oh awesome. in what area? using python-markdown i guess [04:08] yes [04:08] poolie, mind linking me to the appropiate bug so i can now when to begin my efforts? :-) [04:10] wgrant: 'repository'. But yes, I will change it to say 'database' to avoid confusion [04:11] Noldorin, https://code.launchpad.net/~mbp/launchpad/391780-markdown/+merge/82832 [04:11] that is the markdown thing [04:11] i suggest you get some smaller patches in first to warm up [04:11] and get familiar with how it works [04:11] poolie, yeah, like the download types thing? [04:11] yeah [04:11] I appear to have forgotten how to merge a working branch with our main branch. Can I just do "rocketfuel-get" and then "merge ../devel" from my branch? [04:12] huwshimi: That will work, yets. [04:12] poolie, ooh so you've already finished it, jsut ready to mergew [04:12] yes [04:12] wgrant: tweaked help pushing now [04:13] wgrant: Is that not the recommended way? Or am I reading too much into the way you said that? [04:13] wgrant: I'm going to cook dinner; I would like to land all three and deploy tonight, for hopefully obvious reasons [04:13] i've finished a small part of it [04:13] lifeless: Have you also fixed the model lengths? [04:13] lifeless, i will look at the mail bug now [04:14] wgrant: yes, pushed that an age ago [04:14] huwshimi: Some discourage rocketfuel-get, but I think it's fine. [04:14] lifeless: Thanks. [04:14] are you suggesting that i broke it, or are you just keen to accept my offer to look at it? [04:14] wgrant: thank you for noticing [04:14] poolie: bit of column A, bit of column B [04:14] wgrant: Ah right, it seems to work for me, but I don't do very complicated things [04:14] poolie: there is some guilt by association with that area of the code, but also just that you are familiar with it, which is a Good Thin [04:15] g [04:16] wgrant: our oopses compress to about 100K - thats using a 7MB as reference [04:16] wgrant: I propose to add gz compression to amqp2disk to tide us over to a server with more disk [04:17] lifeless: If we indeed turn rabbit on by Wednesday, that sounds like a good workaround. [04:17] poolie, i'm most encouraged by how active this channel is :-) [04:17] bedtime now [04:17] but i'll have a go at that soon [04:17] and poke a dev if necessary [04:18] * wgrant renames reconnect_stores() to reconnect_stores_DONT_YOU_DARE_USE_THIS() [04:19] night folks [04:19] Night Noldorin. [04:20] bye [04:23] happy hacking [04:24] wgrant: right, really off cooking. Can you ack you're reviewing those? [04:24] lifeless: Yu[ [04:25] Yup [04:25] thanks. [04:27] huwshimi, would you be so kind as to read https://code.launchpad.net/~mbp/launchpad/888353-microformats/+merge/82767 for me [04:27] poolie: sure [04:27] * huwshimi reading [04:27] and https://code.launchpad.net/~mbp/launchpad/meta-description/+merge/82769 [04:37] could someone please try running lp.poppy.tests.test_twistedsftp.TestSFTPServer.test_file_creation on oneiric? [04:38] poolie: I think that looks ok, but I don't really know enough about how google handles that stuff [04:40] it's hard to tell for sure ahead of time [04:40] until we see how they parse the actual pages [04:40] do you think it will be any worse? [04:40] self.assertEqual(os.stat(file_name).st_mode, 0100644) [04:40] AssertionError: 33204 != 33188 [04:40] poolie: ^ [04:40] i thought so [04:40] thanks StevenK [04:40] poolie: Not sure how it could be [04:41] poolie, StevenK: Yeah, there are a couple of tests that break due to the insane default umask change. [04:41] I have a list somewhere. [04:41] oh ok [04:41] https://bugs.launchpad.net/launchpad/+bug/892955 [04:41] <_mup_> Bug #892955: lp.poppy.tests.test_twistedsftp.TestSFTPServer.test_file_creation fails on oneiric

< https://launchpad.net/bugs/892955 > [04:42] do we actually care? [04:42] We will in a few months. [04:42] Until then we just curse Ubuntu for doing bad things. [04:43] wgrant: It's now 002? [04:43] It is. [04:43] Madness. [04:44] I thought we set the umask inside poppy? [04:48] poolie: re depends on rabbit - no, it doesn't [04:49] poolie: your environment claims rabbit is available, but it isn't [04:55] what environment? [04:57] lifeless: But you want the oldest, not the first loaded. [04:57] Since AFAICT you use the real age everywhere else. [04:57] Not the loaded age./ [04:59] lifeless, oh i see [04:59] but having configuration data for rabbit does not imply it's running [05:00] ± [05:16] wgrant: for pruning we want real not claimed [05:16] poolie: yes it does :) [05:16] poolie: if its configured, and not running, thats an environmental issue [05:17] * wgrant shops for a reviewer for https://code.launchpad.net/~wgrant/launchpad/reconnect_stores-is-a-horrible-person/+merge/82844 [05:18] +81/-188 [05:18] lookin [05:24] lifeless: Thanks. [05:29] poolie: From line 59 is my test I've added, does this look ok? https://code.launchpad.net/~huwshimi/launchpad/tag-cloud-removal-709009/+merge/81689 [05:31] lifeless, so re "environmental problems" i just want to be able to run the tests within launchpad's buildout environment [05:31] that seems like a reasonable thing to want [05:32] poolie: which tests? [05:32] the buildd tests [05:32] post splitout [05:32] they should be independent [05:32] it Must Not import from launchpad [05:32] and launchpad Must Not import fro it [05:32] *from* it. [05:32] pht [05:33] I don't know what pht means here. But LP is a horrendous environment with a deep stack of assumptions in its code and structure - things like the config system are merely part of it. [05:34] I don't see it as *at all* interesting to let importing things from LP work for other projects: I see it as interesting to extract code from LP such that that code can be reused sensibly. [05:34] i'm just trying to gradually unpick the horro [05:34] anyhow [05:34] with one trivial exception, they do no longer import from lp [05:34] i will fix that now [05:34] i guess it needs its own buildout config for things it does depend upon [05:34] I know, and I'm glad you're doing that - it doesn't mean that using an LP specific test helper in a misconfigured environment is reaosnable [05:35] (yes, 'development' is misconfigured - deliberately so, until make run or bin/run are executed) [05:35] ok, whatever [05:35] if using it outside of bin/test is not supported, i'll close the bug [05:36] its not [05:37] fixing things so that tests really can run in isolation, get an automatic config etc would be wonderful - but I think you'd need to start at the other end of the chain : the symptom you ran into is in ok code, its the rest of the stack that is flawed. [05:37] amongst other things, your developer db would be wiped, etc, running with the 'development' config. [05:38] yeah [05:38] i guess bzr has or had problems where test classes counted on setup being done by bzr selftest [05:38] i think those are bugs [05:39] but, perhaps this is not a good place to start [05:39] it's closed [05:39] certainly, and I'd totally accept 'LP Tests don't work outside of bin/test' as a bug - and be happy if you want to start on that yak shaving exercise [05:39] its deep and nasty; both jml and I have spent weeks stabbing at it and making various partial improvements [05:40] for instance, currently you must have a test runner that can reinvoke itself because there is no guarantee for an arbitrary testA, testB pair, that testA can run in the same process as testB === jtv is now known as jtv-eat [06:23] ok [06:23] soryr for the misunderstanding [06:23] lifeless, i'm happy to say launchpad-buildd no longer imports launchpad at all [06:24] there are still tests in launchpad that import lpbuildd [06:24] production code does not [06:25] lifeless, could you read https://code.launchpad.net/~mbp/launchpad/892427-service-failure/+merge/82766 for me? [06:52] allenap, hi? [08:05] allenap, hi? [08:06] Hello poolie, allenap should be here in about 1h. === rvba changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: rvba | Critical bugtasks: 292 [08:11] ooh [08:11] rvba, how about some reviews? [08:11] poolie: yep, I just saw that you have a few branches up for review ;) [08:12] :) [08:12] i felt inspired on the weekend [08:12] I see that! [08:12] by how bad some search results were, etc === jtv-eat is now known as jtv [08:47] rvba, thanks [08:48] np [08:49] i had an idea there was meant to be no space after the chevron but i can't see anything that says so [09:12] Morning poolie, sorry I forgot to change the topic :-/ [09:14] np [09:14] was just going to say i ended up finishing off the rabbit startup failure patch you suggested [09:14] raphael reviewed it [09:15] Hello === almaisan-away is now known as al-maisan [09:16] Morning mrevell. [09:37] rvba: Was it you who was working on preloading specific jobs for Builder:+history? [09:37] mrevell, i think the only really scary bit of https://bugs.launchpad.net/launchpad/+bug/391780 [09:37] <_mup_> Bug #391780: Support Markdown "stack overflow" style hyperlinks markup

< https://launchpad.net/bugs/391780 > [09:37] is hooking it into the edit widget [09:37] wgrant: yep [09:37] and it may not be hard [09:38] rvba: Are you aware of the large OOPS regression? [09:38] wgrant: oops, no… [09:38] and i guess see if lifeless has any real objection [09:38] 2285 InconsistentBuildFarmJobError: Could not find all the related specific jobs. Bug: https://launchpad.net/bugs/891600 [09:38] <_mup_> Bug #891600: InconsistentBuildFarmJobError: Could not find all the related specific jobs. < https://launchpad.net/bugs/891600 > [09:38] 13 https%3A//launchpad.net/ubuntu/karmic/%2Bsource/libatasmart/%2Bbuilds (SourcePackage:+builds) [09:38] OOPS-0307a9c4a7c9430eeb6a7952a3fb138c, OOPS-1e170c1660c5e4b23a4fa8db662b267c, OOPS-2ae836060fa3b0c90416132f78e4c6cc, OOPS-2d3458326e2328888803cce42c71ae2f, OOPS-4220ac65deb478aa8b47f4ddc65290d9 [09:44] wgrant: sorry, was otp. [09:44] wgrant: I'm looking into it right now. [09:46] wgrant: I remember wondering about this but I made the new code do what the old code did: raise an InconsistentBuildFarmJobError if no specific job can be found… I really wonder how that's a problem now. [09:48] how hard/easy is it to override a utility for the duration of a test? [09:48] rvba: Hmm [09:48] wgrant: do you think this is so bad that we need a cowboy to stop the bleeding? [09:49] rvba: No, it's been around for a few days. [09:49] poolie: lp.testing.fixtures has ZopeAdapter and ZopeViewReplacement fixtures. ZopeUtility would be similar, I imagine. [10:05] yeah, i think this will work [10:05] surely this is the whole point of the complicated dependency injection thing? [10:05] s//a [10:06] Hah [10:08] it's a religious thing? [10:08] poolie: In its defence, raw_sendmail does have a comment saying that it's untested. [10:12] the court finds this argument unpersuasive [10:15] wgrant: I think I'll need your help to pin point where the problem is… having this InconsistentBuildFarmJobError means that one of the specific jobs could not be found: is that possible? I mean I'm trying to understand if the bug is in the way I fetch the specific jobs or if the condition to raise this error is too strict. [10:20] rvba: That is a little odd. [10:20] rvba: I note in the query log of https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-0307a9c4a7c9430eeb6a7952a3fb138c that the first few BFJ IDs in query 138 are duplicated. [10:22] Right. [10:22] Which would probably do it. [10:23] Since it's not going to get duplicates back. [10:23] So the counts will differ. [10:24] Very true. I wonder where this duplication comes from [10:24] * wgrant scratches head [10:24] Ah, rendering error [10:24] * wgrant kicks WebKit. [10:25] rvba: It's also already doine tonnes of BFJ/PB queries before it gets to that. [10:25] Yes, I just saw that. [10:25]