| wgrant | Hm, I didn't know enough data was exposed to create such a thing. | 00:02 |
|---|---|---|
| kb9vqf | Is the PPA build system open sourced as well? | 00:04 |
| wgrant | Yes. | 00:04 |
| wgrant | https://dev.launchpad.net/Soyuz/HowToUseSoyuzLocally | 00:04 |
| kb9vqf | Thanks! | 00:04 |
| === Ursinha is now known as Ursinha-afk | ||
| jml | wgrant, that's 100% through the API | 00:13 |
| jml | wgrant, using anonymous access only. | 00:13 |
| jml | wgrant, although the numbers are wrong, for some reason that yet eludes me | 00:13 |
| wgrant | jml: Oh, using date_confirmed and its kin? | 00:14 |
| jml | yep | 00:14 |
| wgrant | kb9vqf: I've just tweaked that page a bit, trimming obsolete sections and that sort of thing. | 00:15 |
| kb9vqf | OK, thanks--I was still in the configuration section anyway ;-) | 00:15 |
| kb9vqf | wgrant: Any idea what might have caused this: http://pastebin.ubuntu.com/447456/ | 00:18 |
| kb9vqf | All other pages seem to work fine, but the home page doesn't | 00:18 |
| kb9vqf | (https://launchpad.dev/) | 00:18 |
| wgrant | kb9vqf: You're using an empty DB without sample data? | 00:19 |
| kb9vqf | I trimmed out a lot of the sample data | 00:19 |
| kb9vqf | Guess I trimmed something I shouldn't have ;-) | 00:19 |
| wgrant | Try cronscripts/update-stats.py | 00:19 |
| wgrant | That may fix it, but it may not work unless you have at least one project. | 00:19 |
| kb9vqf | I was wondering if that would be the case | 00:19 |
| kb9vqf | Thanks for the cron hint as well | 00:19 |
| kb9vqf | One more thing...regarding changing all the instances of the word "Launchpad" to something else, would I be lucky enough for that to be controlled by a single file somewhere? | 00:22 |
| kb9vqf | Or will I have to resort to sed | 00:22 |
| wgrant | sed, probably. | 00:22 |
| wgrant | I use http://pastebin.ubuntu.com/447458/ and http://pastebin.ubuntu.com/447459/ on an empty DB (without any sample data at all). | 00:23 |
| wgrant | Creates most of the critical celebrities. | 00:23 |
| wgrant | Updates the statistics, that sort of thing. | 00:23 |
| wgrant | And the second populates Ubuntu. | 00:24 |
| * kb9vqf spent an entire day trying to get the celebrities in place | 00:24 | |
| kb9vqf | I'll try the scripts as soon as Launchpad finishes rebuilding | 00:24 |
| ajmitch | wgrant: you have those scripts somewhere other than pastebin? | 00:29 |
| ajmitch | they look useful | 00:29 |
| wgrant | ajmitch: Not really. | 00:42 |
| wgrant | They just sit around in ~/launchpad/utils and occasionally get unbroken when I get fed up with how terrible the sample data is. | 00:42 |
| lifeless | jml: wow | 00:47 |
| lifeless | jml: lovely visualisation; can you do the bazaar project group too ? | 00:47 |
| jml | lifeless, I want to. You might be able to figure out how to do it yourself if you branch lp:~jml/+junk/convergence | 00:48 |
| lifeless | jml: I'm sure I can :) . Uhm this seems like a lptools thing, if you want to make a home for it | 00:49 |
| jml | lifeless, I'm deferring the problem of figuring out how to have couch views that I can give a parameter to. | 00:49 |
| jml | (probably the answer is something horrible) | 00:49 |
| lifeless | jml: this is using offlinelaunchpadstuff ? | 00:50 |
| jml | lifeless, no. | 00:50 |
| jml | I wanted to, and then I spent a bunch of time making the offlinelaunchpad stuff better in subtle, peripheral, important ways and then I got on with this. | 00:50 |
| wgrant | jml: Launchpad didn't collapse under the load of generating that graph? | 00:51 |
| lifeless | heh | 00:51 |
| lifeless | wgrant: no | 00:51 |
| jml | wgrant, not as far as I can tell. | 00:51 |
| lifeless | wgrant: we generate more load internally than the API servers can cause ;P | 00:51 |
| wgrant | What was it that was causing the DB issues? | 00:52 |
| lifeless | more load on master than desired, not enough on the slaves. | 00:54 |
| lifeless | which was due to replication lag being stale | 00:54 |
| wgrant | Ah. | 00:54 |
| lifeless | which was due to the replication lag monitor not running. | 00:54 |
| wgrant | Oops. | 00:55 |
| lifeless | which wasn't fixed immediately because its not, itself, monitored. | 00:55 |
| lifeless | Which is because we don't have a 'must have monitoring on day 1' rule. | 00:55 |
| lifeless | or something like htat. | 00:55 |
| lifeless | (such a rule might be more trouble than its worth) | 00:56 |
| jml | meh | 01:02 |
| lifeless | jml: ? I'm saying not everything is important enough to monitor, in my suggestion that a hard rule might be trouble. | 01:02 |
| jml | apart from the 162 bug tasks that are counted as "New" on that graph due to API exposure, I don't know why there are 500 bug tasks that have no "date_foo" field but are not actually "New" | 01:03 |
| jml | lifeless, the "meh" was in response to my internal angst :) | 01:03 |
| lifeless | ah :) | 01:03 |
| lifeless | by all means, mehangst away | 01:03 |
| wgrant | jml: Incomplete? | 01:04 |
| jml | wgrant, that's the 162 | 01:04 |
| wgrant | Ah. | 01:04 |
| thumper | jml: hi | 01:11 |
| thumper | jml: are you skypeable? | 01:11 |
| jml | thumper, only for fun and social reasons :) | 01:11 |
| thumper | jml: I have something of strategic importance | 01:11 |
| thumper | heh | 01:11 |
| thumper | probably neither fun nor social | 01:11 |
| thumper | jml: are you on leave? | 01:12 |
| jml | thumper, yeah. | 01:12 |
| thumper | when are you back? | 01:12 |
| jml | thumper, thus dicking about with desktopcouch and graphs | 01:12 |
| jml | thumper, back Monday | 01:12 |
| thumper | ok | 01:12 |
| thumper | remind me Monday | 01:12 |
| jml | thumper, will do. look at the shiny. http://people.canonical.com/~jml/convergence/ | 01:12 |
| thumper | I saw | 01:12 |
| thumper | lots of bugs | 01:12 |
| thumper | you didn't graph won't fix :) | 01:13 |
| jml | thumper, yeah I did! | 01:13 |
| thumper | really? | 01:13 |
| jml | thumper, it's the invisible bit at the bottom :) | 01:13 |
| wgrant | What about Opinion? | 01:13 |
| jml | thumper, or rather, wontfix are considered done for the purpose of this graph | 01:13 |
| wgrant | Or is that not actually in production yet. | 01:13 |
| thumper | jml: your key only shows five statuses | 01:13 |
| jml | thumper, it deliberately doesn't show non-open bugs. | 01:14 |
| thumper | It'd be interesting to see our rate of fixing ... | 01:14 |
| thumper | has it slowed down / sped up ... | 01:14 |
| thumper | jml: a first derivative graph of the bugs graph would also be interesting | 01:15 |
| jml | thumper, well, you can kind of see that by the "Fix committed" line at the bottom. | 01:15 |
| thumper | jml: at what rate are bugs being filed | 01:15 |
| jml | thumper, some of that stuff is on this (old) set of graphs: http://people.canonical.com/~jml/lp-bugs/ | 01:15 |
| jml | thumper, I guess now that I've figured out the relevant technologies and don't have to do staging database queries to get this information, I can add those. | 01:17 |
| * thumper nods | 01:18 | |
| * thumper goes back to wrapping bugs | 01:18 | |
| jml | thumper, first derivative isn't useful with so many lines -- I think you only want "open" and "closed" then | 01:18 |
| * thumper stabs ORMs in the face | 01:22 | |
| wgrant | Ridiculously inefficient SQL usage? | 01:22 |
| thumper | yes | 01:23 |
| thumper | and the hoop jumping to make it more efficient | 01:23 |
| thumper | part of what I need to talk to jml about | 01:23 |
| wgrant | I wish I could tell it that I wanted it to precalculate various attributes. | 01:23 |
| wgrant | Rather than having to do that manually, which makes everything about 10000000000x times uglier. | 01:23 |
| thumper | like what? | 01:24 |
| wgrant | For example, the publisher needs to retrieve lots of SourcePackagePublishingHistorys, their SourcePackageReleases, their SourcePackageReleaseFiles, and various other collections. | 01:25 |
| wgrant | If I want to do that in less than hundreds of thousands queries (quite literally), I have to formulate a query manually and can't use the objects' normal attributes. | 01:26 |
| lifeless | yes | 01:26 |
| lifeless | orms are evil | 01:26 |
| lifeless | db query interfaces - like storm /has/ - are very nice | 01:27 |
| wgrant | I should be able to tell Storm to precalculate SPPH.spr, SPR.files, and that sort of thing. | 01:27 |
| lifeless | but magic attributes - very hard to get the right balance | 01:27 |
| lifeless | wgrant: you can | 01:27 |
| wgrant | There is the prejoin implementation in the SQLObject wrapper. | 01:27 |
| lifeless | wgrant: there's a couple of hooks; not sure if the precise one you need is present | 01:27 |
| wgrant | But it doesn't do collections. | 01:27 |
| lifeless | wgrant: theres a 'when getting this from the DB do xyz' hook | 01:27 |
| lifeless | wgrant: I'm pretty sure | 01:28 |
| maxb | heh, did someone give launchpad/ppa a biased buildscore? :-) | 01:32 |
| lifeless | don't think so | 01:33 |
| maxb | I just uploaded stuff and it seems to have bypassed the entire queue :-) | 01:34 |
| lifeless | rotfl | 01:34 |
| wgrant | That is a little suspicious. | 01:36 |
| wgrant | I suppose we can delete most of the PPA now. | 01:37 |
| maxb | s/suspicious/convenient/ :-) | 01:37 |
| wgrant | Mm, although I guess we need to wait until 10.06 is out the door. | 01:37 |
| lifeless | is there a karma<->priority matchup ? | 01:38 |
| wgrant | lifeless: Only inasmuch as people with more karma are more likely to be able to hit up buildd admins :P | 01:40 |
| maxb | "Automatic retry of builds waiting on dependencies is disabled pending resolution of a performance problem" -- o rly? | 01:41 |
| wgrant | It's back now. | 01:41 |
| wgrant | Was announced on launchpadstatus, I thought. | 01:41 |
| maxb | Also, https://edge.launchpad.net/~lamont/+archive/psql8.3/+packages seems to have gone into an endless loop of depwait -> retry -> depwait -> retry ... | 01:42 |
| maxb | i.e. they're being erroneously retried | 01:42 |
| wgrant | If you ever catch a build log or dependencies value from any of them, please keep a note of it. | 01:43 |
| wgrant | There are some known bugs, but not any that should affect PPA builds. | 01:44 |
| maxb | the lpia build there is building again now | 01:44 |
| wgrant | Yeah. | 01:44 |
| wgrant | Oh, if it depwaits I guess it will finish soon. | 01:44 |
| wgrant | Forgot that detail. | 01:44 |
| wgrant | If buildd-manager ever wakes up, that is. | 01:46 |
| wgrant | I must fix it one day. | 01:47 |
| maxb | If there's a buildd-admin willing, can I get a rescore on at least one build in https://edge.launchpad.net/~maxb/+archive/launchpad/+packages ? I'd like to get a confirmed success before I sleep, so I can give them to stub tomorrow | 01:48 |
| thumper | ✁☹ | 01:50 |
| thumper | ✁☹ | 01:50 |
| thumper | ✁☹ | 01:50 |
| thumper | ✁☹ | 01:50 |
| thumper | ✁☹ | 01:50 |
| wgrant | Ah, debhelper >=7. | 01:50 |
| thumper | grrrrraaaaahhhhhhhhhhhh!!!!!!!!!!!!! | 01:50 |
| wgrant | So it's probably the pocket-naïveté. | 01:50 |
| wgrant | thumper: What has Storm done now? | 01:51 |
| * jelmer waves to thumper and wgrant | 01:51 | |
| wgrant | Or Soyuz? | 01:51 |
| wgrant | Morning jelmer. | 01:51 |
| * jelmer ducks | 01:51 | |
| thumper | I'm using lazr.delegate to wrap a bug so it doesn't hit the db for its tasks | 01:51 |
| thumper | but getBugTask(self, target) uses self.bug_tasks | 01:52 |
| thumper | so in order to get that method to the cached tasks, | 01:52 |
| thumper | I either have to copy the implementation | 01:52 |
| thumper | or violate one of our rules for imports | 01:52 |
| thumper | and import Bug into code.browser | 01:52 |
| thumper | to go Bug.getBugTask(my_wrapped_bug, target) | 01:53 |
| wgrant | So you're precalculating the collection? | 01:53 |
| thumper | yes | 01:53 |
| wgrant | Well. | 01:53 |
| thumper | in a single query to get the linked bugs and all their tasks | 01:53 |
| wgrant | One easy way out is to abuse the fact that Storm doesn't notice when you assign a list. | 01:53 |
| wgrant | So you clobber bug.bug_tasks with your precalculated list. | 01:54 |
| wgrant | Evil, yes. | 01:54 |
| wgrant | But so are the other methods. | 01:54 |
| thumper | eh? | 01:54 |
| thumper | that is evil | 01:54 |
| spm | maxb: did you get that rescored? I'll do it now? | 01:54 |
| maxb | spm: yes please | 01:55 |
| spm | ahh. slony. hell yes I'l rescore those. | 01:55 |
| * thumper goes for the lesser of three evils | 01:56 | |
| maxb | :-) | 01:56 |
| spm | maxb: powa has been abused and used: https://edge.launchpad.net/~maxb/+archive/launchpad/+builds | 01:57 |
| maxb | neat, and they've been dispatched already :-) | 01:58 |
| spm | see. I am useful for *some* things. ;-) | 01:58 |
| maxb | Now I just need to talk lamont into using my builds instead of redoing it | 02:00 |
| wgrant | Doesn't Hardy have 8.3 already? | 02:01 |
| spm | the backport for hady-cat that we do? generally that's little more than trivial textual changes and getting a localised rebuild of same. | 02:02 |
| maxb | An 8.3 what? | 02:08 |
| maxb | These builds were about getting the most recent upstream of slony built for (hardy,lucid)*(8.3,8.4) | 02:09 |
| wgrant | Ah, right, Slony. | 02:09 |
| maxb | which it now has :-) | 02:09 |
| thumper | queries down from 294 to 93 using cached bug tasks | 02:56 |
| thumper | (for my test branch) | 02:56 |
| ajmitch | thumper: you managed to mangle it to use the right property? | 03:08 |
| thumper | no | 03:08 |
| thumper | well kinda | 03:08 |
| thumper | too much special code IMO | 03:08 |
| thumper | now I see most of the queries being in the menu link formatters :( | 03:09 |
| thumper | it should not be doing that :( | 03:09 |
| thumper | I thought that some old fix of salgado fixed that | 03:10 |
| * thumper wishes sinzui was around to explain | 03:10 | |
| sinzui | Yes it should not be doing that | 03:11 |
| sinzui | I crafted a menu two weeks ago to avoid subscription checks. each menu call in the template was a call to the db | 03:11 |
| sinzui | thumper, I think menus should be singletons or we use a utility that manages to menus for each object to ensure there is only for instance for each request | 03:13 |
| thumper | sinzui: ah, so it is known that it is somewhat fucked then? | 03:13 |
| sinzui | You and I know this | 03:14 |
| thumper | heh | 03:14 |
| thumper | ok, how to fix this in my branch? | 03:14 |
| thumper | ah | 03:14 |
| thumper | I know why it is screwed in this moment | 03:14 |
| sinzui | I was very tempted to risk toppling all menus to fix the performance issues working with milestones on pages | 03:14 |
| thumper | the context object is the unwrapped object | 03:15 |
| thumper | the view.context is the decorated object | 03:15 |
| thumper | I bet the pages are using context/menu rather than view/context/menu... | 03:15 |
| thumper | I'm caching the queries needed by the menu | 03:15 |
| thumper | in my wrapped object | 03:15 |
| * thumper checks the templates | 03:15 | |
| sinzui | I think each template fragment is creating the menu on demand or for it's chunk, but that same menu for the context may be instantiated 30 times | 03:16 |
| thumper | right | 03:17 |
| sinzui | I think salgado had a hack to put the data in the request so that templates shared the menu, but I do not see evidence of that | 03:17 |
| thumper | sinzui: can I override the globs for the template? | 03:18 |
| thumper | sinzui: I want to set the context to be my wrapped context | 03:18 |
| thumper | rather than the unwrapped one | 03:18 |
| thumper | sinzui: do you know where the template render code is actually called? | 03:18 |
| sinzui | render() for the fmt:link? | 03:19 |
| thumper | I mean the code that sets the 'context' for the page template render | 03:19 |
| sinzui | ah | 03:19 |
| sinzui | That would be sidnei's chameleon engine | 03:20 |
| thumper | are we using chameleon? | 03:20 |
| sinzui | yes | 03:20 |
| * sidnei raises an eyebrow | 03:20 | |
| thumper | where is the code? | 03:20 |
| sinzui | in an egg | 03:20 |
| thumper | :( | 03:20 |
| * sinzui saw it while purging py2.5 eggs today | 03:21 | |
| thumper | surely we call it somewhere with a dictionary of locals right? | 03:21 |
| thumper | that is what I want to override | 03:21 |
| sidnei | sinzui, chameleon is not enabled, though the code is there | 03:21 |
| sinzui | :( | 03:21 |
| * thumper hunts in webapp for calling pagetemplates | 03:21 | |
| thumper | sidnei: unless you can tell me where to look | 03:21 |
| sinzui | thumper, are these LaunchpadView views? | 03:22 |
| thumper | yep | 03:22 |
| sinzui | I believe we hacked __call__ | 03:22 |
| sidnei | thumper, i might if sinzui doesn't beat me to it | 03:22 |
| thumper | :( | 03:23 |
| thumper | there seems to be much magic there | 03:23 |
| sidnei | there's a ton of magic involved yes. it left me scratching my had last time i looked at it. | 03:24 |
| sidnei | s/had/head | 03:24 |
| thumper | __call__ calls render | 03:24 |
| thumper | render calls self.template() | 03:24 |
| thumper | template is a property that returns self.index | 03:24 |
| thumper | self.index seems opaque | 03:25 |
| sinzui | thumper, during traversal, the view is construct. adapters from zcml marry the named view to a template and create the final view call, and it monekypatches __facet__ because using ILayer would have been too easy | 03:25 |
| sidnei | thumper, self.index is the template that gets registered by the zcml directive iirc | 03:25 |
| thumper | sinzui: do I have to read the entire publisher file? | 03:26 |
| sinzui | No, look for __facet__ I scream every time I see that monster | 03:26 |
| thumper | ENOTFOUND | 03:27 |
| * sinzui ponders | 03:27 | |
| thumper | __launchpad_facet__ | 03:28 |
| sidnei | sorry for not being more helpful, it's almost midnight here and im trying to wrap up a large refactoring | 03:29 |
| sinzui | thumper, yes, in metazcml | 03:30 |
| thumper | ✁☹ | 03:32 |
| ajmitch | are there any tests around for scripts/ftpmaster-tools ? | 03:36 |
| lifeless | hah | 03:36 |
| ajmitch | I was afraid of that | 03:37 |
| thumper | return mapply(ob, request.getPositionalArguments(), request) | 03:38 |
| thumper | publication.py line 436 | 03:38 |
| ajmitch | on a related note, is launchpad.net running on lucid | 03:39 |
| sinzui | yes | 03:39 |
| sinzui | as of today, dev is on 2.6 | 03:40 |
| sinzui | python 2.6 | 03:40 |
| ajmitch | ok, I just saw some duplicated code in sync-source.py to remove then, I'm just changing some of the version matching in there :) | 03:41 |
| spm | sinzui: it is? oh *dev* is. right. prod is still running on hardy. | 03:41 |
| sinzui | all lpnet is hardy? | 03:42 |
| lifeless | ajmitch: different questions | 03:42 |
| lifeless | ajmitch: 'works on 2.6' | 03:42 |
| lifeless | ajmitch: != 'deployed on 2.6' | 03:42 |
| lifeless | ajmitch: we can't remove compat stuff until its deployed on 2.6 | 03:42 |
| sinzui | We this will be a very exciting rollout. I am glad I am not the release manager for 10.06 | 03:42 |
| spm | sinzui: I hope so, or else it's been upgraded without me knowing about it; which'd be fun :-) | 03:42 |
| ajmitch | lifeless: in this case, it's code duplicated from python-debian that is in lucid | 03:42 |
| ajmitch | rather than anything python 2.6-specific | 03:42 |
| lifeless | ajmitch: we're runniong on hardy | 03:43 |
| ajmitch | which is mostly what I was asking | 03:43 |
| lifeless | yeah, you asked the right question; but a little more clarity that it wasn't a proxy for python version would have helped ;) | 03:45 |
| ajmitch | I did clarify, just a little too late :) | 03:45 |
| spm | :-) | 03:45 |
| ajmitch | it was bug 520508, fwiw | 03:46 |
| mup | Bug #520508: Duplicate copy of split_gpg_and_payload() <soyuz-upload> <tech-debt> <Soyuz:Triaged> <https://launchpad.net/bugs/520508> | 03:46 |
| * thumper frowns | 03:49 | |
| thumper | I can't find it! | 03:50 |
| thumper | damn it | 03:51 |
| thumper | the publication code calls request.getPositionalArgs | 03:51 |
| thumper | which defers to self._args of the request | 03:51 |
| thumper | which gets set with positional= kwarg | 03:51 |
| * thumper is frustrated | 03:51 | |
| thumper | going back to changing context/menu to view/context/menu | 03:52 |
| thumper | for some reason my context is getting wrapped twice | 04:07 |
| thumper | \o/ | 04:15 |
| thumper | down to 63 queries | 04:15 |
| * thumper looks for dupes | 04:15 | |
| lifeless | nice | 04:22 |
| thumper | down from almost 300 | 04:24 |
| thumper | still dupes though | 04:24 |
| thumper | and we have 20 queries before we even hit object traversal | 04:24 |
| thumper | I think with some effort we could drop that to under 1- | 04:24 |
| thumper | 10 | 04:24 |
| wgrant | thumper: Ew, what are the pre-traversal queries? | 04:36 |
| wgrant | That's mildly insane. | 04:37 |
| thumper | :) | 04:37 |
| thumper | want me to pastebin it? | 04:37 |
| wgrant | Well, I have an exam in 20 minutes, so it's probably best that I don't look at it :P | 04:38 |
| thumper | http://pastebin.ubuntu.com/447557/ | 04:38 |
| thumper | sorry | 04:38 |
| thumper | there goes the study | 04:38 |
| wgrant | Heh. | 04:38 |
| thumper | which exam? | 04:38 |
| wgrant | Logic Completeness and Incompleteness. | 04:38 |
| ajmitch | oh that sounds fun | 04:39 |
| wgrant | SELECT AccountPassword.account, AccountPassword.id, AccountPassword.password FROM AccountPassword WHERE AccountPassword.account = %s | 04:39 |
| wgrant | Why would it being doing that? It's not meant to know that passwords exist. | 04:39 |
| thumper | wgrant: go to your exam | 04:46 |
| thumper | ew | 04:53 |
| thumper | removeSecurityProxy(db.branch).__class__.displayname.__get__(db) | 04:53 |
| thumper | I think that passes the wrapped object (db) into the property getter as self | 04:53 |
| thumper | it is pretty revolting | 04:54 |
| stub | Hmm.... """sourcepackagerelease components "not null" site:bugs.launchpad.net """ in google finds me the bug I want on index pages, but not the bug itself. | 05:05 |
| madscientist159 | wgrant: You still around? | 07:19 |
| madscientist159 | wgrant: I tried to run your script to initialize the empty database, but all I got was this: | 07:20 |
| madscientist159 | http://paste.ubuntu.com/447607/ | 07:20 |
| === madscientist159 is now known as kb9vqf | ||
| kb9vqf | Anyone have an idea why I am getting that permission error? I've tried as root and as the postgresql user... | 07:24 |
| kb9vqf | wgrant: I managed to get around the problem by temporarily granting all access to the karmacategory table and sequence, so my above comments are just an FYI | 07:55 |
| kb9vqf | Also FYI, the second script will notrun, complaining about "lp.registry.interfaces.distroseries" not existing | 07:57 |
| kb9vqf | Ideas for debugging this oops? | 08:07 |
| kb9vqf | v | 08:07 |
| kb9vqf | http://pastebin.ubuntu.com/447619/ | 08:07 |
| kb9vqf | It happens only when trying to go to a project page such as https://launchpad.dev/launchpad | 08:07 |
| kb9vqf | Everything else seems to work great | 08:08 |
| adeuring | good morning | 08:18 |
| kb9vqf | wgrant: Here's a fixed version of the second script: http://pastebin.ubuntu.com/447622/ | 08:20 |
| === almaisan-away is now known as al-maisan | ||
| === thumper changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 1 of 10.06 | PQM is open but ec2 land/-s is broken | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes | ||
| wgrant | kb9vqf: Ah, yeah, sorry, forgot about the karmacategory perm fix. | 09:20 |
| wgrant | And DistroSeriesStatus was recently moved and renamed to SeriesStatus, and apparently that version of the script hasn't been fixed yet. | 09:20 |
| wgrant | And it looks like you need to run rmy second script now, or all projects explode. | 09:20 |
| wgrant | That's pretty inconvenient. | 09:20 |
| kb9vqf | wgrant: Did you see my fixed version of the second script ;-) | 09:22 |
| wgrant | Aha. | 09:23 |
| kb9vqf | I figured out the projects exploding...my local installation is actually working quite well ATM | 09:23 |
| kb9vqf | Thanks to your scripts ;-) | 09:23 |
| kb9vqf | Still working on the PPA builders though | 09:23 |
| * kb9vqf crosses fingers | 09:23 | |
| wgrant | What's wrong with them? | 09:24 |
| kb9vqf | Nothing yet | 09:24 |
| kb9vqf | Just don't have the software installed yet | 09:24 |
| wgrant | Ah. | 09:24 |
| kb9vqf | I fumble-fingered a move command earlier and deleted the past 12 hours of work | 09:24 |
| wgrant | Well, beware, I wrote those instructions too, so... | 09:25 |
| kb9vqf | :-) | 09:25 |
| wgrant | I might throw the scripts into a branch and push it up. | 09:26 |
| kb9vqf | Be aware that there is an issue (syntax error) in one of the Launchpad python files the first script calls...something to do with the distribution IIRC | 09:27 |
| kb9vqf | Sorry I don't have the filename handy, that was several hours ago | 09:27 |
| kb9vqf | There was a colon in the function declaration when it was really supposed to be a comma | 09:27 |
| wgrant | Hmm, odd. I guess I'll find out in a few minutes when I test the whole lot. | 09:28 |
| noodles775 | kb9vqf: yes, a testfix branch was just landed. (The issue was the result of a bad-but-non-conflicting merge) | 09:28 |
| wgrant | However, I think the ubuntu.currentseries.displayname crash is a legitimate bug. | 09:28 |
| wgrant | Let's see... | 09:28 |
| kb9vqf | Well, it went away after the second script was modified and run | 09:28 |
| wgrant | Yeah. | 09:29 |
| wgrant | But I meant the second one to be optional; that's why it's separate. | 09:29 |
| kb9vqf | Oh, OK | 09:29 |
| wgrant | Ah, it's not a real bug that will show up in production :( | 09:30 |
| kb9vqf | How does the blog widget on the Launchpad home page operate? Can I redirect it to my blog? | 09:38 |
| wgrant | It's... hardcoded. | 09:38 |
| kb9vqf | Oh | 09:38 |
| kb9vqf | How can I remove the entries that are there right now? | 09:38 |
| wgrant | grep around and find the template that has them. | 09:39 |
| kb9vqf | In the DB or in the files? | 09:39 |
| wgrant | The files. | 09:39 |
| wgrant | The entries themselves are hardcoded. | 09:39 |
| kb9vqf | Ohhhh | 09:39 |
| wgrant | Not the blog feed that it pulls from. | 09:39 |
| kb9vqf | Shall I say that seems rather...odd? | 09:40 |
| kb9vqf | ;-) | 09:40 |
| wgrant | Yes. | 09:40 |
| kb9vqf | Is there a list anywhere of how often each cron script should be run? | 09:53 |
| * kb9vqf doesn't want to waste system resources by running every minute if its not needed | 09:53 | |
| wgrant | That's something they've not revealed. | 09:56 |
| kb9vqf | OK, trial and error it is then | 09:58 |
| kb9vqf | wgrant: Are there any instructions for setting up a build daemon on a remote machine? | 10:06 |
| kb9vqf | I have a rather large cluster here of identical machines I'd like to use | 10:06 |
| wgrant | kb9vqf: Just install launchpad-buildd on them and point LP at them. | 10:08 |
| wgrant | But watch out. lp-buildd is unauthenticated. | 10:08 |
| kb9vqf | I have a dedicated private network, so that's not a problem ;-) | 10:08 |
| kb9vqf | (Physically secured too) | 10:08 |
| kb9vqf | So I have to install the lp-branches/devel stuff on each build machine then? | 10:08 |
| wgrant | No. Just grab the launchpad-buildd .deb that is created. | 10:09 |
| kb9vqf | What about the chroot image? | 10:09 |
| kb9vqf | Oh..is that image system wide and uploaded to each builder on-demand? | 10:09 |
| wgrant | Right. | 10:10 |
| wgrant | The chroot is sent out for each build. | 10:10 |
| wgrant | (although it's cached on the slave) | 10:10 |
| kb9vqf | That makes a ton more sense | 10:10 |
| kb9vqf | Thanks! | 10:10 |
| wgrant | https://code.launchpad.net/~wgrant/launchpad/bootstrap-db-from-scratch contains all the changes and fixed scripts for bootstrapping. | 10:11 |
| kb9vqf | Subscribed myself so I won't lose it :-) | 10:11 |
| kb9vqf | Has anyone else actuallydeployed Launchpad other than Canonical? | 10:12 |
| kb9vqf | Just out of curiosity | 10:12 |
| wgrant | I don't believe so. | 10:12 |
| kb9vqf | So I might be the first | 10:12 |
| kb9vqf | :-) | 10:12 |
| wgrant | Anyone who tried would have probably run into huge problems and asked here. | 10:12 |
| wgrant | Everyone who has asked here has given up. | 10:12 |
| kb9vqf | Hmmm....Librarian | 10:19 |
| kb9vqf | It doesn't seem to be started from "make run" | 10:19 |
| wgrant | noodles775: Have the getBuildRecords timeouts been fixed? I've only had a few emails from cron today, rather than the usual several dozen. | 10:19 |
| wgrant | It still times out sometimes, but not as much. | 10:19 |
| wgrant | kb9vqf: It probably only listens on localhost. | 10:19 |
| kb9vqf | Yes; I'm getting this though | 10:20 |
| kb9vqf | Librarian upload failed: [localhost:58090]: [Errno 111] Connection refused | 10:20 |
| wgrant | Hm. | 10:20 |
| wgrant | Is that from manage-chroot? | 10:20 |
| kb9vqf | Yup | 10:20 |
| wgrant | You've not touched the config? | 10:20 |
| kb9vqf | Nope | 10:20 |
| wgrant | make run definitely normally starts it. | 10:20 |
| kb9vqf | That's what I thought too; here's what is in the config file right now: | 10:21 |
| kb9vqf | [librarian] | 10:21 |
| kb9vqf | download_url: http://launchpad.dev:58080/ | 10:21 |
| wgrant | Hm, that's not the default, is it? | 10:21 |
| kb9vqf | Yes it is | 10:22 |
| wgrant | Mm, so it is. | 10:22 |
| kb9vqf | At least that's what came from Bazaar | 10:22 |
| kb9vqf | Should it be localhost? | 10:22 |
| wgrant | Well, if you want to make really sure that it doesn't work from any of your other buildds :P | 10:22 |
| kb9vqf | No thanks, I'll pass :-P | 10:23 |
| kb9vqf | It doesn't seem to be running at all; there is nothing with "lib" in ps aux | 10:23 |
| wgrant | It's a twistd with librarian.tac in argv. | 10:24 |
| wgrant | So try killing and restarting make run. | 10:24 |
| kb9vqf | That's what I'm doing now | 10:24 |
| wgrant | So. | 10:24 |
| * kb9vqf is reminded of Windows..."When acting strangely, simply reboot!" | 10:25 | |
| wgrant | I guess you need to override upload_host, download_host, restricted_upload_host and restricted_download_host in the librarian section in your config file. | 10:25 |
| wgrant | As well as making it somehow not listen on localhost only. | 10:25 |
| wgrant | Or stick a proxy in front of it. | 10:25 |
| kb9vqf | The reboot fixed it | 10:26 |
| kb9vqf | And I was thinking of doing a proxy, it's three lines in Apache ;-) | 10:26 |
| noodles775 | wgrant: no, well not specifically. There has been some changes since yesterday that reduce the db load, so that might be the reason you're seeing less timeouts. | 10:26 |
| kb9vqf | Can I run i386 buildd on a 64-bit installation of Ubuntu? | 10:27 |
| wgrant | kb9vqf: Yes. | 10:27 |
| kb9vqf | Good | 10:27 |
| wgrant | You'll need to tweak the arch in /etc/launchpad-buildd/default, though. | 10:27 |
| wgrant | It defaults to the system arch. | 10:27 |
| kb9vqf | Yeah, I noticed that | 10:27 |
| kb9vqf | Wasn't sure if it was a read-only attribute so to speak | 10:27 |
| wgrant | You can also drop additional config files alongside 'default' to run multiple lp-buildds on one host. | 10:28 |
| wgrant | Handy for handling multiple archs. | 10:28 |
| wgrant | They cannot, however, share a filecache. | 10:28 |
| kb9vqf | Cool | 10:29 |
| kb9vqf | Now I can put those 8 cores to good use | 10:29 |
| kb9vqf | Things not to do...try to run all of the cron scripts in parallel at once | 10:43 |
| kb9vqf | :-{ | 10:43 |
| wgrant | Um, yeah, bad idea. | 10:48 |
| kb9vqf | wgrant: FYI, I got a "canonical.launchpad.utilities.celebrities.MissingCelebrityError: ubuntu-bugzilla" from one of those scripts | 10:52 |
| kb9vqf | You might want to add it to the DB building script | 10:52 |
| wgrant | kb9vqf: Note that the DB bootstrap script has lots of comments in the celebrity list for stuff that isn't created yet. | 10:57 |
| wgrant | You really don't want to be running scripts unless you know what they do. | 10:57 |
| kb9vqf | I glanced through it and verified the database; just didn't look real hard at which celebrities were being created | 10:58 |
| kb9vqf | Since that is a bit of a black box to me right now | 10:58 |
| wgrant | Right. But you should be working out what the cronscripts do before you run them. | 10:58 |
| kb9vqf | That is true | 10:58 |
| wgrant | Ewwwwwwwwwwwwwwwww. | 11:00 |
| kb9vqf | The main problem was launching them all at once; I will probably have to create groups of them (e.g. the PPA updating scripts could run every minute, etc) | 11:00 |
| wgrant | I'd never dared to look at why ubuntu-bugzilla was a celebrity. | 11:00 |
| wgrant | I... regret discovering why. | 11:00 |
| kb9vqf | Do I want to know? | 11:01 |
| wgrant | Well, once upon a time, Ubuntu used Bugzilla. Then all the bugs were imported into LP, with bugwatches pointing at the old Bugzilla bugs. | 11:01 |
| wgrant | Now, normally bugwatches are updated every day or so. | 11:01 |
| wgrant | But this doesn't make sense for Ubuntu Bugzilla, since it doesn't exist any more. | 11:02 |
| wgrant | So.... there's a hack to skip updating bugwatches if their tracker's name is the same as the ubuntu-bugzilla celebrity's. | 11:02 |
| kb9vqf | Ewwww | 11:02 |
| ajmitch | almost as ewww as the old ubuntu bugzilla was | 11:04 |
| wgrant | It wasn't *that* bad. | 11:07 |
| kb9vqf | If /var/tmp/poppy does not exist, should I just create it? | 11:25 |
| wgrant | Yes. | 11:25 |
| wgrant | I thought that was documented. Hmm. | 11:25 |
| wgrant | Yeah, the first process-upload.py call in the "Upload a source to the PPA" section will create /var/tmp/poppy. | 11:26 |
| wgrant | Or should, at least. | 11:26 |
| kb9vqf | It gave me an error until I created the directory manually | 11:27 |
| wgrant | Hrmph. | 11:27 |
| kb9vqf | One other question...how does Launchpad handle Email sending? Can I just point it to my SMTP server? | 11:28 |
| wgrant | Pretty much. But you'll need to make sure any email addresses in the code or DB are really what you want, and aren't going to spam people. | 11:30 |
| kb9vqf | OK. And the configuration file for that is schema-lazr.conf or something else? | 11:30 |
| wgrant | You shouldn't alter schema-lazr.conf itself. | 11:30 |
| wgrant | You should probably create a new config, perhaps based on the development one. | 11:30 |
| wgrant | Overriding more things. | 11:30 |
| kb9vqf | Ah, OK | 11:31 |
| wgrant | If you look really hard you might be able to find the old production configs and use bits of them as examples ;) | 11:31 |
| kb9vqf | I need to catch some sleep...the build farm looks like it is active and cron jobs are partially set up...I'll do the actual build test tomorrow | 11:40 |
| kb9vqf | Thanks for all your help so far! | 11:40 |
| deryck | wgrant, ping. still around? | 11:50 |
| wgrant | deryck: Sure. | 11:53 |
| wgrant | Hm, translation template jobs are on now? | 12:03 |
| henninge | wgrant: you mean if they are working? | 12:07 |
| wgrant | I haven't seen dozens on the farm before. | 12:07 |
| wgrant | But most of the i386 builders are doing them at the moment. | 12:07 |
| henninge | wgrant: well, they are restricted to run only on i386 | 12:09 |
| wgrant | Well, yes, but I hadn't seen any on production for weeks. | 12:09 |
| wgrant | Also, are they restricted to only happen every $INTERVAL? | 12:10 |
| wgrant | Or will they run for every rev? | 12:10 |
| henninge | AFAIK they run on every rev. | 12:10 |
| henninge | jtv might be surer about the details. | 12:10 |
| wgrant | erm. | 12:10 |
| jtv | hi wgrant | 12:10 |
| wgrant | Evening jtv. | 12:11 |
| * henninge goes to lunch | 12:11 | |
| jtv | wgrant: we still need a lot more logging and tracking for these jobs. :/ One thing we found some time ago was that there's been a change in gettext that makes our front-end vetting think a lot of pure-gettext branches are intltool-based. | 12:12 |
| wgrant | jtv: Yeah, well, we have the infrastructure to keep track of them after they're done now. | 12:13 |
| wgrant | Once that's done, it probably becomes easier to impose quotas, too. | 12:13 |
| wgrant | Since you can't actually tell at the moment if an attempt has been made lately... | 12:14 |
| jtv | It'd be nice to have some form of bundling. | 12:14 |
| wgrant | Hm? | 12:14 |
| jtv | Where we don't record separate jobs for every revision, but the need to re-process a given branch. | 12:14 |
| wgrant | Ah. | 12:14 |
| jtv | If you get a dozen revisions in quick succession, there shouldn't be a need to run a dozen jobs. | 12:15 |
| wgrant | Right. | 12:15 |
| wgrant | That seems easy enough to do, even now. | 12:15 |
| jtv | Perhaps, yes. | 12:16 |
| wgrant | Is the porting of SPRBs and TTBJs to the new infrastructure going to happen in the foreseeable future? | 12:17 |
| jtv | Frankly, I don't know. We're focused on something else at the moment. | 12:19 |
| === lifeless_ is now known as lifeless | ||
| bigjools | wgrant, jtv: I expect buildd admins will insist so that those jobs show up in the history | 13:00 |
| === matsubara-afk is now known as matsubara | ||
| jtv | bigjools: it'd definitely make sense to do, yes. And we want to do it... question is when. | 13:00 |
| wgrant | Plus it means we can delete a whole lot of code. | 13:01 |
| wgrant | And simplify a whole lot of other code. | 13:01 |
| wgrant | And make logging suck less. | 13:01 |
| bigjools | win win win | 13:07 |
| bigjools | henninge has a decision to make ;) | 13:08 |
| henninge | bigjools: I would love to if I knew about what ... | 13:09 |
| bigjools | henninge: "porting of SPRBs and TTBJs to the new infrastructure" | 13:09 |
| bigjools | TTBJs in your case | 13:09 |
| henninge | also, I am currently listening to a Microsoft general manager talking about "Microsoft and Opennes" ... | 13:09 |
| bigjools | Microsoft has always been open | 13:10 |
| === henninge is now known as henninge-linuxta | ||
| bigjools | ...to making more money | 13:10 |
| === Ursinha-afk is now known as Ursinha | ||
| Ursinha | Chex, gary_poster, rockstar, bigjools, henning, sinzui, gmb: production meeting on #launchpad-meeting @ Freenode in 30 minutes | 16:32 |
| Chex | Ursinha: thanks for reminder | 16:32 |
| rockstar | Ursinha, correction: 28 minutes. | 16:32 |
| gary_poster | thanks | 16:32 |
| rockstar | :) | 16:33 |
| Ursinha | rockstar, thanks for that :) | 16:33 |
| gmb | Ursinha, Thanks. | 16:36 |
| Ursinha | Chex, gary_poster, rockstar, bigjools, henning, sinzui, gmb: production meeting on #launchpad-meeting @ Freenode now | 17:00 |
| === beuno is now known as beuno-lunch | ||
| === matsubara is now known as matsubara-lunch | ||
| === beuno-lunch is now known as beuno | ||
| === matsubara-lunch is now known as matsubara | ||
| kb9vqf | Any idea why Launchpad won't accept my key import? | 19:27 |
| kb9vqf | "Launchpad could not import your OpenPGP key" | 19:27 |
| === al-maisan is now known as almaisan-away | ||
| kb9vqf | Any idea why Launchpad won't accept my key import? | 19:44 |
| kb9vqf | "Launchpad could not import your OpenPGP key" | 19:44 |
| kb9vqf | This is weird... "format '1.0' is not permitted in lucid." | 19:52 |
| kb9vqf | What does that mean? | 19:52 |
| kb9vqf | It came from a PPA upload rejection notice | 19:52 |
| === Ursinha is now known as Ursinha-food | ||
| kb9vqf | I have sucessfully uploaded a package to my PPA on my local Launchpad installation and it is marked as published, but for some reason it won't start building | 21:03 |
| kb9vqf | It's stuck on "Needs building" | 21:03 |
| kb9vqf | Any thoughts? | 21:03 |
| maxb | Have you read wgrant's documentation on making a local soyuz work? | 21:12 |
| kb9vqf | Yes, I followed it | 21:13 |
| kb9vqf | It seems like is almost working, the build machines show up in /+builds as idle, I added both the i386 and amd64 chroot images | 21:13 |
| kb9vqf | But the build itself does not fire off | 21:14 |
| kb9vqf | maxb: is there anywhere I should look for logging info? | 21:14 |
| maxb | Not sure, sorry. | 21:15 |
| kb9vqf | maxb: If I haven't signed the code of conduct could it do thise? | 21:15 |
| kb9vqf | this? | 21:15 |
| maxb | huh. I'd expect it to refuse the upload | 21:16 |
| kb9vqf | maxb: I'd like to try signing it and see what happens, but I get "Launchpad could not import your OpenPGP key" when I try to add my key | 21:17 |
| kb9vqf | Any thoughts on that? | 21:17 |
| maxb | I'm sure wgrant documented how to sign the CoC | 21:17 |
| === Ursinha-food is now known as Ursinha | ||
| kb9vqf | maxb: Well I think I know why it accepted my upload; I ran the upload acceptor with absolutely-anything | 21:19 |
| kb9vqf | But I don't know if the CoC is checked again afterwards before the package is built | 21:20 |
| kb9vqf | So what piece of Launchpad actually dispatches the builds? | 21:26 |
| mars | lifeless, ping, is there any chance you could approve https://code.edge.launchpad.net/~tseaver/subunit/add_distutils_support/+merge/26653 ? | 21:45 |
| mars | lifeless, s/approve/review/ | 21:45 |
| lifeless | yup its on my todo | 21:46 |
| mars | lifeless, cool, thank you. | 21:46 |
| lifeless | is this work relasted for you? its currently in my personal pile | 21:49 |
| lifeless | mars: ^ | 21:58 |
| mars | lifeless, relasted? | 21:58 |
| mars | ah, related | 21:59 |
| lifeless | related | 21:59 |
| mars | lifeless, yes, it is blocking my landing of https://code.edge.launchpad.net/~mars/zope.testing/fix-subunit-utf8-traceback-reporting/+merge/27086 | 21:59 |
| lifeless | how so ? | 21:59 |
| mars | if you read Tres' comment, you'll see that he is waiting for PyPI subunit. | 22:00 |
| mars | When that is available, then zope.testing's setup.py and buildout.cfg can be updated with conditional subunit[test] dependencies | 22:00 |
| mars | And then my patch will be sane: you will need subunit to run the zope.testing test suite | 22:02 |
| lifeless | ah | 22:02 |
| lifeless | there is a test tools bug thats related | 22:03 |
| mars | as Tres put it, then we can "ensure that the [subunit] features don't bitrot" | 22:05 |
| mars | gary_poster, ec2 land did not eat my branch. It just took an hour to deliver my change from ec2 to PQM :/ | 22:06 |
| mars | It fell through a timerift on the way over or something | 22:06 |
| gary_poster | mars, well, time rifts are interesting, and a lock of eating the branch is a good thing :-) | 22:08 |
| gary_poster | lack | 22:08 |
| kb9vqf | So what piece of Launchpad actually dispatches the builds? | 22:31 |
| lifeless | sidnei: around ? | 22:32 |
| lifeless | tres has said he is ok with https://code.edge.launchpad.net/~mars/zope.testing/fix-subunit-utf8-traceback-reporting/+merge/27086 landing as long as you or someone else with subunit tests it; mars has said it may break other builds right now as the additional tests he adds aren't conditional. | 22:33 |
| lifeless | mars: you're going to fix that right ? | 22:33 |
| mars | lifeless, yes, I certainly will now | 22:33 |
| === matsubara is now known as matsubara-afk | ||
| sidnei | lifeless, im about to leave. i can land it in the morning. | 22:51 |
| lifeless | sidnei: cool | 22:52 |
| lifeless | thanks | 22:52 |
| thumper | rockstar: call now? | 23:09 |
| rockstar | thumper, sure. | 23:09 |
| thumper | rockstar: and I suppose we have to mumble? | 23:10 |
| rockstar | thumper, yes.... :( | 23:10 |
| jml | lifeless: fwiw, that thing I did to make a pretty graph has grown a script that makes similarly pretty graphs given a csv file like YYYY-MM-DD,number,number,number,... | 23:16 |
| lifeless | nice | 23:16 |
| jml | I guess I should productize it, or something. | 23:17 |
| lifeless | meh | 23:19 |
| lifeless | tweet it | 23:19 |
| lifeless | your job is done | 23:19 |
| jml | hah | 23:25 |
| ajmitch | yay for documentation about testing soyuz | 23:25 |
| jml | at least now it has a README | 23:26 |
| jml | good bye | 23:26 |
| lifeless | fly well | 23:28 |
| wgrant | ajmitch: What about it? | 23:51 |
| ajmitch | wgrant: that I'm glad it's there, it was helpful | 23:52 |
| wgrant | Ah, you've actually tried it? | 23:53 |
| ajmitch | yes | 23:53 |
| ajmitch | at least the accepting source packages & publishing them, not the building | 23:53 |
| ajmitch | so that I could test out this change to sync-source.py: | 23:54 |
| ajmitch | http://bazaar.launchpad.net/~ajmitch/launchpad/fakesyncs/revision/10992 | 23:54 |
| wgrant | Ah, right. | 23:55 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!