| lifeless | fun - OpenSSL: error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol | 00:38 |
|---|---|---|
| lifeless | Unable to establish SSL connection. | 00:38 |
| rick_h_ | wallyworld__: so just an fyi along the lines of what I was thinking. https://pastebin.canonical.com/75528/ is where I'd head to try to make sure the privacy.js is decoupled from the information_type.js aside from listening to the event. | 00:39 |
| lifeless | erm wtf | 00:40 |
| lifeless | telnet> open 10.0.3.24 443 | 00:40 |
| lifeless | Trying 10.0.3.24... | 00:40 |
| lifeless | Connected to 10.0.3.24. | 00:40 |
| lifeless | Escape character is '^]'. | 00:40 |
| lifeless | HELO | 00:40 |
| lifeless | <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> | 00:40 |
| lifeless | <html><head> | 00:40 |
| lifeless | <title>501 Method Not Implemented</title> | 00:40 |
| lifeless | :) | 00:40 |
| wallyworld__ | rick_h_: looks ok at first glance. the get_banner_text function can probably be taken off the namespace now with these changes | 00:41 |
| lifeless | right, thats better. | 00:41 |
| rick_h_ | wallyworld__: right, not complete | 00:41 |
| rick_h_ | wallyworld__: but I wanted to kind of clear up the idea/where I'm trying to head to make sense of it more | 00:42 |
| lifeless | whoever had that apache not working to lp.dev before, if you're using lxc etc, make sure your apache config is listening on *:443, not 192.160.0.88:443 | 00:42 |
| wgrant | lifeless: I added a feature to Makefile to do that, which I added to the LP LXC docs | 00:43 |
| wallyworld__ | rick_h_: looks ok so far but i probably need to get into the code a little more. i'd run your changes by jc since he did some banner work a few months ago | 00:43 |
| lifeless | wgrant: I was giving the lp-setup inline docs a spin | 00:43 |
| lifeless | wgrant: they are incomplete | 00:44 |
| rick_h_ | wallyworld__: yea, honestly I'm afk for a week so I'd not worry about it too much. Just hoping to make me seem les insane | 00:44 |
| lifeless | (because it goes for test-suite capability) | 00:44 |
| wallyworld__ | rick_h_: np. it's all good | 00:44 |
| * rick_h_ swears he's not lost it yet | 00:44 | |
| wgrant | lifeless: Right | 00:51 |
| lifeless | how do I set up the combo loader stuff ? | 01:14 |
| wgrant | lifeless: Should just need to set 'js.combo_loader.enabled default 0 true' | 01:15 |
| wgrant | May need to make combobuild, but the usual targets should do that | 01:15 |
| rick_h_ | lifeless: which part? the actual hosting bit? or just enabling? | 01:15 |
| lifeless | I have a fresh lxc | 01:15 |
| lifeless | I've run lp-setup update | 01:15 |
| lifeless | and sudo make install | 01:15 |
| rick_h_ | then it should be setup to combo build ootb. I thought the combo loading FF was on by default. | 01:15 |
| lifeless | I'm getting reference errors | 01:15 |
| lifeless | [13:14:08.562] ReferenceError: YUI is not defined @ https://launchpad.dev/product-name-100011/+milestone/unique-from-factory-py-line878-100017/+index:53 | 01:16 |
| lifeless | [13:14:08.562] ReferenceError: LPJS is not defined @ https://launchpad.dev/product-name-100011/+milestone/unique-from-factory-py-line878-100017/+index:110 | 01:16 |
| wgrant | Check your browser's HTTP request log | 01:16 |
| rick_h_ | lifeless: make combobuild ? | 01:16 |
| lifeless | [13:14:08.330] GET https://launchpad.dev/+combo/?yui/yui/yui-min.js&lp/meta.js&yui/loader/loader-min.js [HTTP/1.1 200 OK 226ms] | 01:16 |
| wgrant | Aha | 01:16 |
| wgrant | Go to that URL | 01:16 |
| wgrant | It probably says "missing" everywhere | 01:16 |
| lifeless | /* yui/yui/yui-min.js */ | 01:16 |
| rick_h_ | and if it's missing then check build/js/... | 01:16 |
| lifeless | /* [missing] */ | 01:16 |
| rick_h_ | which is what combobuild does | 01:16 |
| lifeless | /* lp/meta.js */ | 01:16 |
| wgrant | Yeah, make combobuild would be my suggestion | 01:17 |
| lifeless | make run doesn't do it ? | 01:17 |
| wgrant | Perhaps lp-setup uses a custom target that doesn't include it | 01:17 |
| wgrant | It should... | 01:17 |
| lifeless | make combobuild | 01:17 |
| lifeless | utilities/js-deps -n LP_MODULES -s build/js/lp -x '-min.js' -o \ | 01:17 |
| lifeless | build/js/lp/meta.js >/dev/null | 01:17 |
| lifeless | utilities/check-js-deps | 01:17 |
| rick_h_ | and jsbuild_watch which will should get lined up with combobuild | 01:17 |
| lifeless | ctrl-refresh, same stuff in that url | 01:17 |
| wgrant | Hm | 01:17 |
| wgrant | make inplace | 01:17 |
| lifeless | there are a lot of LP modules present | 01:17 |
| rick_h_ | lifeless: anything in build/js? | 01:17 |
| lifeless | its not an empty combo response | 01:17 |
| wgrant | Oh | 01:17 |
| lifeless | var LP_MODULES = (function(){ | 01:18 |
| wgrant | Do you have a YUI version flag set? | 01:18 |
| lifeless | f'r instance | 01:18 |
| lifeless | no | 01:18 |
| lifeless | neither does qastaging | 01:18 |
| wgrant | Right, it's not necessary | 01:18 |
| rick_h_ | LP_MODULES is from the build/js/lp/meta.js so that's a good sign | 01:19 |
| wgrant | lifeless: Does the /srv/launchpad.net/convoy symlink point to a reasonable location? | 01:19 |
| rick_h_ | lifeless: so is there anything in build/js/yui ? | 01:19 |
| rick_h_ | maybe some failure to get that dep right? | 01:19 |
| lifeless | lrwxrwxrwx 1 robertc robertc 59 2012-09-28 01:18 convoy -> /home/robertc/source/launchpad/lp-branches/working/build/js | 01:20 |
| lifeless | ls build/js/yui | 01:20 |
| lifeless | yui-3.3.0 yui-3.5.1 | 01:20 |
| wgrant | So that's all fine. Must be a buildout issue | 01:20 |
| wgrant | Huh | 01:20 |
| wgrant | yui should be a symlink to yui-3.5.1, shouldn't it? | 01:20 |
| lifeless | there are two instances of 'missing' in the https://launchpad.dev/+combo/?yui/yui/yui-min.js&lp/meta.js&yui/loader/loader-min.js response AFAICT | 01:20 |
| rick_h_ | no, should have build/js/yui symlink to 3.5.1 | 01:20 |
| rick_h_ | lifeless: right, yui-min.js and load-min.js are coming from the yui libary | 01:21 |
| rick_h_ | https://launchpad.dev/+combo/?yui-3.5.1/yui/yui-min.js&lp/meta.js&yui-3.5.1/loader/loader-min.js will work since you have that dir | 01:21 |
| wgrant | Is build/js/yui a symlink? | 01:21 |
| rick_h_ | but you should have the yui symlink to yui-3.5.1 as the 'default' yui version and that's missing somehow | 01:21 |
| rick_h_ | wgrant: right | 01:22 |
| lifeless | build/js$ ls -ld yui | 01:22 |
| lifeless | drwxr-xr-x 2 robertc robertc 4096 2012-09-28 01:18 yui | 01:22 |
| lifeless | its a directory not a symlink | 01:22 |
| rick_h_ | lifeless: so yea, that's the issue | 01:22 |
| lifeless | rm it ? | 01:22 |
| rick_h_ | no, you need it, it needs to be the symlink, loading up my lxc to look at what makes that symlink sec | 01:22 |
| wgrant | On my lpsetup-created instance it's a symlink as expected | 01:22 |
| lifeless | well, this is bind mounted | 01:23 |
| lifeless | as you'd expect | 01:23 |
| lifeless | I've had this working tree for yonks | 01:23 |
| wgrant | Ah, so it's an existing checkout? | 01:23 |
| wgrant | Right, so I'd make clean_buildout build | 01:23 |
| lifeless | of course | 01:23 |
| rick_h_ | right, clean it up and rerun then | 01:23 |
| rick_h_ | right, combo-rootdir will do the symlink if it doesn't exist. Since you have a directory it's failing to generate it | 01:24 |
| rick_h_ | not sure how you got the directory there, but that's causing it to fail. a clean_buildout and make should work | 01:25 |
| lifeless | ok, thats a lot bigger | 01:25 |
| lifeless | still funky | 01:25 |
| lifeless | 1000 unassigned bugs is 11 seconds to render | 01:26 |
| lifeless | thats very interesting | 01:26 |
| wgrant | Bugs and people have a few enums | 01:27 |
| lifeless | I get Y.lp is undefined still | 01:30 |
| wgrant | Any [missing]s? | 01:30 |
| lifeless | yes | 01:30 |
| lifeless | between meta.js and yui/loader/loader-min.js */ | 01:30 |
| wgrant | So meta.js is missing? | 01:31 |
| lifeless | /* lp/meta.js */ | 01:31 |
| lifeless | /* [missing] */ | 01:31 |
| lifeless | needed a make combobuild in that recipe you gave me | 01:32 |
| === matsubara is now known as matsubara-afk | ||
| lifeless | 5.56 seconds now | 01:33 |
| lifeless | so the 10K was noise of some sort | 01:33 |
| lifeless | e.g. sitting on battery ;) | 01:33 |
| lifeless | 40% time reduction neuterin the tal in lib/lp/registry/templates/milestone-macros.pt | 02:10 |
| lifeless | ah, late eval queries | 02:11 |
| lifeless | storm cache strikes again I bet. | 02:11 |
| * lifeless looks for the hammer | 02:11 | |
| wgrant | storm_cache_size | 02:12 |
| wgrant | configs/development/launchpad-lazr.conf | 02:12 |
| lifeless | yeah | 02:12 |
| lifeless | do you remember the prod size? | 02:12 |
| lifeless | [and why can't we just toss the damn thing..] | 02:12 |
| wgrant | 10000 I think | 02:12 |
| wgrant | We need *a* size | 02:13 |
| lifeless | well | 02:13 |
| lifeless | why? | 02:13 |
| wgrant | At least for scripts | 02:13 |
| lifeless | we discard the cache after every request | 02:13 |
| wgrant | Sure, for the webapp the limit can be quite ridiculous | 02:14 |
| lifeless | set a ulimit for scripts, so ones that try to do too much in one pass can fail hard | 02:14 |
| wgrant | For scripts it might want to be more sensible. | 02:14 |
| lifeless | or just have a non-discarding cache with a low fixed limit for scripts | 02:14 |
| wgrant | Hm | 02:14 |
| wgrant | Why do we still have two storm_cache_size keys... | 02:14 |
| wgrant | I thought I removed all the duplicates | 02:15 |
| lifeless | 2.5 seconds with minimal td entries | 02:15 |
| wgrant | What'd you change? | 02:15 |
| lifeless | neutered all the bug data access | 02:16 |
| wgrant | Ah | 02:16 |
| lifeless | getting a baseline for doing that many td rows | 02:16 |
| lifeless | 2.7 seconds with the first cell filled in | 02:16 |
| lifeless | same amount of HTML output | 02:16 |
| lifeless | running several times and taking min of course | 02:17 |
| lifeless | <td class="icon left"> | 02:19 |
| lifeless | <span class="sortkey" tal:content="bugtask/bug/id" /> | 02:19 |
| lifeless | <span tal:content="structure bugtask/image:icon" /> | 02:19 |
| lifeless | </td> | 02:19 |
| lifeless | ^ 0.2 seconds over 1000 rows | 02:19 |
| lifeless | vs | 02:22 |
| lifeless | <td class="icon left"> | 02:22 |
| lifeless | <span class="sortkey" tal:content="bugtask/bug/id" /> | 02:22 |
| lifeless | </td> | 02:22 |
| wgrant | So, while I clean up this storm_cache_size stuff I think I might remove the development override | 02:23 |
| wgrant | It doesn't make much sense | 02:23 |
| lifeless | +1 | 02:24 |
| wgrant | We keep having to work around it in tests | 02:24 |
| wgrant | Heh | 02:25 |
| wgrant | In fact, the main tests that override the limit are for ProjectMilestone:+index... | 02:26 |
| lifeless | wgrant: where is structure defined | 02:29 |
| wgrant | lifeless: In templates? | 02:29 |
| wgrant | Or in our Python? | 02:29 |
| wgrant | It's a TAL keyword; it's not defined as such anywhere | 02:30 |
| lifeless | the implementation | 02:30 |
| lifeless | emitSubstitution in talgenerator.py - closing in on it | 02:32 |
| wgrant | Right, it's in zope.tal | 02:32 |
| wgrant | Or chameleon if the page uses that | 02:33 |
| wgrant | But I believe that one's still zope.tal | 02:33 |
| lifeless | def do_insertStructure_tal(self, (expr, repldict, block)): | 02:34 |
| lifeless | whoa | 02:52 |
| lifeless | 18K permission checks | 02:52 |
| lifeless | I /really/ want to rip that out entirely | 02:52 |
| wgrant | We would need a pretty significant rewrite for that to work | 02:53 |
| wgrant | Plus the checks should be mostly cached, so very very fast | 02:53 |
| lifeless | 5.5K calls are to block implicit flushes, block implicit flushes makes 18K calls to checkPermission | 02:54 |
| wgrant | "makes" | 02:55 |
| wgrant | It's a decorator | 02:55 |
| lifeless | I know | 02:55 |
| wgrant | Ah :) | 02:55 |
| lifeless | there are getattr's | 02:56 |
| lifeless | that are blocking flushes | 02:56 |
| lifeless | and checking permissions | 02:56 |
| lifeless | 5.5% of runtime in that stack | 02:56 |
| lifeless | so - 'very very fast' - not so much | 02:56 |
| lifeless | ahha | 02:57 |
| lifeless | wgrant: low hanging fruit: | 02:57 |
| lifeless | _get_sqlobject_store | 02:57 |
| lifeless | wgrant: spot what it does | 02:57 |
| wgrant | The import could be a problem if we were multithreading, but we're not really... | 02:58 |
| lifeless | also block_implicit_flushes is bad too | 02:58 |
| lifeless | wgrant: its a lock | 02:58 |
| lifeless | wgrant: synchronisation primitives are not that cheap | 02:58 |
| wgrant | Sure | 02:58 |
| lifeless | wgrant: also, prod runs two threads - xmlrpc + http | 02:59 |
| lifeless | so we are | 02:59 |
| wgrant | But does the profile show that's an issue? | 02:59 |
| mwhudson | importing even existing modules in a tight loop is surprisingly slo | 02:59 |
| mwhudson | w | 02:59 |
| wgrant | Right, and the xmlrpc cohabitation is the root of most of our annoying undiagnosable issues | 02:59 |
| StevenK | mwhudson: Namespacing causes problems, news at 11? | 02:59 |
| wgrant | Not namespacing... | 02:59 |
| lifeless | wgrant: http://paste.ubuntu.com/1231593/ | 03:02 |
| wgrant | Indeed | 03:02 |
| wgrant | Although I'd prefer to look at untangling the imports, that should work | 03:03 |
| lifeless | wgrant: its about 300ms over the request. | 03:04 |
| lifeless | wgrant: from a quick timeit + back of napkin | 03:04 |
| wgrant | Hm | 03:05 |
| wgrant | I've seen _get_sqlobject_store show up before in profiles, but hadn't actually looked at it as there were other dominating factors to fix first | 03:06 |
| wgrant | But I haven't profiled in several months | 03:06 |
| wgrant | Anyway, storm_cache_size is now 10000 everywhere in devel | 03:07 |
| wgrant | I meant to do that as followup to my DB config rework branches last year | 03:08 |
| wgrant | Or whenever it ws | 03:08 |
| wgrant | But never got around to it, apparently | 03:08 |
| bigjools | how often is process-upload running these days? | 03:15 |
| lifeless | ok so its hard to tell | 03:20 |
| lifeless | but it looks like it saved 100ms | 03:20 |
| lifeless | wgrant: ^ if you want to do it at some point | 03:20 |
| lifeless | I've got to context switch here | 03:20 |
| lifeless | hard to tell because of the massive variance | 03:21 |
| wgrant | bigjools: */5 for sources | 03:29 |
| wgrant | lifeless: Yeah, and it really really depends on the page whether it's a big issue | 03:29 |
| wgrant | I might try untangling that stuff | 03:29 |
| lifeless | it would be great to make imports barf | 03:30 |
| lifeless | install an import hook | 03:30 |
| bigjools | ta | 03:30 |
| lifeless | hmmm, dunno if cached imports would trigger | 03:30 |
| wgrant | lifeless: We have a lot of inline imports, so we can't do it generally | 03:31 |
| lifeless | wgrant: we could set a cap | 03:31 |
| lifeless | wgrant: 500 imports in a request, then boom. | 03:31 |
| wgrant | Yeah | 03:31 |
| lifeless | that would be about 5ms | 03:32 |
| lifeless | more or less | 03:32 |
| wgrant | I was initially thinking we could do a per-import cap | 03:32 |
| wgrant | But as you say, I'm not sure it'll actually trigger on repetition | 03:32 |
| wgrant | Maybe | 03:32 |
| lifeless | I'm not sure it will trigger at all | 03:32 |
| lifeless | we could hack the bytecode | 03:32 |
| lifeless | but folk might consider that objectionable | 03:32 |
| wgrant | Heh | 03:32 |
| wgrant | Or we could just ban inline imports in lp.services.webapp... | 03:32 |
| mwhudson | __import__ is called for cached imports | 03:33 |
| mwhudson | says science | 03:33 |
| lifeless | mwhudson: tried monkey patching it ? | 03:33 |
| lifeless | mwhudson: and we were actually speculating about pep whatsisthing hooks | 03:33 |
| mwhudson | oh | 03:34 |
| mwhudson | yeah, i just overwrote __builtin__.__import__ | 03:34 |
| mwhudson | dunno about the hooks | 03:34 |
| lifeless | anyhow replacing __import__ with a thread-local-counting abomination would be effective. | 03:37 |
| lifeless | mwhudson: want to write one | 03:37 |
| lifeless | ? | 03:37 |
| mwhudson | i suppose i could try | 03:39 |
| mwhudson | where would the output go? same place as the query count? | 03:39 |
| lifeless | mwhudson: Oh, I was thinking super simple: Raise a Mother* exception :) | 03:40 |
| mwhudson | oh, when it crosses 500 or something? | 03:40 |
| lifeless | mwhudson: needs a call to install it, probably one to uninstall it, a per-thread 'request starting, kill me after X imports', and 'request finished, don't kill me anymore' | 03:41 |
| lifeless | mwhudson: ^ is how I would do it. | 03:41 |
| lifeless | http://pypi.python.org/pypi/pyjack/ is kindof related | 03:43 |
| lifeless | mwhudson: might want a fifth call to change the threshold mid-request | 03:43 |
| wallyworld__ | wgrant: if you want to look https://code.launchpad.net/~wallyworld/launchpad/duplicate-bug-warning-xss-1057630/+merge/126849 | 03:43 |
| lifeless | mwhudson: would let us featureflag it | 03:43 |
| lifeless | mwhudson: hmm, could minimise down to 4 calls: - install/uninstall/reset_count/set_threshold(None-or-an-int) | 03:44 |
| lifeless | mwhudson: make the set an attribute access, and you've got the ability to read-back too, for oops integration if we care. | 03:48 |
| mwhudson | lifeless: i guess there isn't a bug for this? | 03:49 |
| lifeless | nup | 03:49 |
| lifeless | man, I can't get used to this fast ->dbstable bb :) | 03:52 |
| mwhudson | how do you guys set things up so you can access launchpad running in an lxc from outside it? | 03:53 |
| mwhudson | is it just a matter of putting the right bits in /etc/hosts outside the container? | 03:53 |
| wgrant | wallyworld__: Might it be worth mustaching this instead? This still injects dup_id directly, which is only safe today because of constraints we place on this particular piece of data | 03:53 |
| wgrant | mwhudson: Right, Makefile takes a variable to make Apache listen on *, and then I drop stuff in /etc/hosts | 03:54 |
| mwhudson | make install LISTEN_ADDRESS=* ? | 03:54 |
| wgrant | Should be on https://dev.launchpad.net/Running/RemoteAccess | 03:55 |
| wgrant | But that looks like it | 03:55 |
| mwhudson | seems to be | 03:55 |
| mwhudson | ProgrammingError: function pg_last_xact_replay_timestamp() does not exist ? | 03:58 |
| mwhudson | oh | 03:58 |
| wgrant | You fail at 9.1 | 03:58 |
| mwhudson | i probably haven't updated launchpad-dependencies in the container for eons | 03:58 |
| wgrant | Yeah | 03:58 |
| mwhudson | The following packages have been kept back: | 03:58 |
| mwhudson | ... launchpad-database-dependencies | 03:58 |
| mwhudson | hm | 03:58 |
| wgrant | dist-upgrade? | 03:59 |
| mwhudson | oh right | 03:59 |
| mwhudson | yes, seems happier | 03:59 |
| wgrant | Then you'll probably want to purge 8.4, then drop and recreate the 9.1 cluster to get it on port 5432 | 03:59 |
| wgrant | Then rerun utilities/launchpad-database-setup and all will be good | 03:59 |
| mwhudson | ah yes | 04:01 |
| StevenK | lifeless: I think db-stable should die. It's not clear what should happen to db-devel, but it's awfully heavyweight for what we want it for. | 04:02 |
| mwhudson | i guess what's actually going to happen here is that i'll get to the point where i could do some launchpad development | 04:03 |
| lifeless | StevenK: how would you assess 'db and code work well together' ? | 04:03 |
| mwhudson | and then it'll be time to go home | 04:03 |
| StevenK | mwhudson: Are you coming back to us? | 04:04 |
| mwhudson | StevenK: not formally :) | 04:04 |
| StevenK | mwhudson: Oh, so you're going to tease us with a few bug fixes? :-) | 04:05 |
| mwhudson | i would still like to kill the branch puller | 04:05 |
| StevenK | Can you kill the branch scanner too? | 04:07 |
| mwhudson | Get:68 http://archive.ubuntu.com/ubuntu/ lucid-updates/main libmysqlclient16 5.1.63-0ubuntu0.10.04.1 [1,898kB] | 04:07 |
| mwhudson | whut | 04:07 |
| mwhudson | the path to that is less clear i think | 04:08 |
| StevenK | mwhudson: stub had a pretty brilliant idea, I thought | 04:08 |
| StevenK | mwhudson: A DB backend for bzr so when you push a branch, it directly hits the DB, no muss, no fuss | 04:08 |
| mwhudson | ah yeah | 04:09 |
| stub | I thought I was trolling? | 04:09 |
| mwhudson | i think i first heard that idea from mark, actually :-) | 04:09 |
| mwhudson | i don't know if he was trolling either | 04:09 |
| mwhudson | it would be awesome, but also a great deal of work | 04:09 |
| lifeless | bzr is a db | 04:09 |
| stub | In theory, it is a great idea. | 04:09 |
| stub | yeah, what he said | 04:09 |
| lifeless | so, ITYM *another* DB backend | 04:09 |
| StevenK | Fine, a *postgresql* backend | 04:10 |
| mwhudson | create table versionedfile (...) | 04:10 |
| mwhudson | what could go wrong | 04:10 |
| wgrant | So then the question becomes "oh god why" | 04:10 |
| mwhudson | Configuring postgresql.conf to use port 5433... | 04:13 |
| mwhudson | oy you | 04:13 |
| wgrant | Did you purge 8.4 and drop the existing 9.1 cluster? | 04:13 |
| mwhudson | yes | 04:14 |
| wgrant | :( | 04:14 |
| mwhudson | oh | 04:14 |
| mwhudson | maybe i didn't purge all of 8.4 | 04:14 |
| stub | yeah, oh | 04:14 |
| mwhudson | ok, that's better | 04:15 |
| mwhudson | woo | 04:20 |
| jtv | stub: looks like my format-relative-imports branch didn't come through. :( I got the EC2 success message, but no landing. | 04:20 |
| mwhudson | um | 04:22 |
| mwhudson | the importfascist still exists? | 04:22 |
| wgrant | Yes. | 04:22 |
| wgrant | It doesn't do very much | 04:22 |
| wgrant | But it still exists | 04:23 |
| wgrant | And someone angered it earlier this week | 04:23 |
| stub | jtv: I'll try the landing step again. I too got the success message. | 04:23 |
| wgrant | stub, jtv: We were in testfix for several hours | 04:23 |
| wgrant | Probably got caught in that | 04:23 |
| mwhudson | database_root = 'canonical.launchpad.database' | 04:23 |
| stub | jtv: Do you have the mp url handy? | 04:24 |
| * mwhudson spots some dead code | 04:24 | |
| mwhudson | oh yay, bootstrapping | 04:29 |
| jtv | stub: https://code.launchpad.net/~jtv/launchpad/format-relative-imports/+merge/124878 | 04:30 |
| mwhudson | heh | 04:32 |
| mwhudson | https://code.launchpad.dev/applets -> 522 imports | 04:32 |
| mwhudson | or 1713 for the first request after app server startup :) | 04:33 |
| wallyworld__ | wgrant: i could look at mustache i guess. we know the id is an int, so i saw no need to do anything special with it | 04:37 |
| mwhudson | lol what | 04:41 |
| mwhudson | one of the main offenders is get_current_browser_request doing "from zope.security.management import queryInteraction" inline | 04:41 |
| mwhudson | which is probably only to keep lazr.restful from unconditionally depending on zope ? | 04:41 |
| lifeless | wallyworld__: trusting the integrity of over the wire data is risky | 04:42 |
| wgrant | mwhudson: Heh | 04:42 |
| lifeless | mwhudson: lazr.restful shoul always depend on zope | 04:43 |
| wallyworld__ | i guess i need to be more paranoid | 04:43 |
| wgrant | lifeless, wallyworld__: It's not so much the integrity of the wire. It's the fact that changing something on the server will subtly make unnecessarily fragile client code vulnerable | 04:43 |
| lifeless | mwhudson: the folk making it work for django were being optimists | 04:43 |
| mwhudson | yes | 04:43 |
| lifeless | wgrant: I didn't talk about integrity of the wire :) | 04:43 |
| lifeless | wgrant: and yes, I agree. | 04:43 |
| mwhudson | i think reality has won there now? | 04:43 |
| lifeless | mwhudson: yes | 04:43 |
| lifeless | mwhudson: if it were to be made separate, it would need to be several more separate projects with callbacks etc | 04:44 |
| wgrant | wallyworld__: eg. in a tonne of places today we inject URLs containing product names unescaped, because people thought it should be safe | 04:44 |
| wgrant | wallyworld__: It's a reasonable assumption, except that it means we have to be really really ultra-paranoid about project names | 04:44 |
| wallyworld__ | sure, but a name is not an integer | 04:44 |
| wgrant | Because if a quote gets in one somehow, we're fucked | 04:44 |
| wgrant | wallyworld__: Bugs have nicknames that are used in some situation | 04:44 |
| wgrant | Eg. bug gbcw | 04:45 |
| wgrant | Or did we remove them, I can't remember. | 04:45 |
| wgrant | http://launchpad.net/bugs/gbcw :) | 04:45 |
| wgrant | Anyway | 04:45 |
| wgrant | Client code shouldn't be unnecessarily fragile | 04:45 |
| wgrant | Partly because it's possible it'll change | 04:45 |
| wgrant | But mostly because people will copy it | 04:45 |
| wgrant | And an argument that people *won't* copy it isn't really valid here, since someone clearly did copy this from elsewhere :) | 04:46 |
| wgrant | Launchpad is really good at convincing people to copy code :( | 04:47 |
| wallyworld__ | wgrant: changes pushed | 04:54 |
| wgrant | wallyworld__: That looks much safer and even shorter, thanks. | 04:55 |
| wgrant | _show_comment_on_duplicate_warning still needs the same treatment, though | 04:55 |
| wallyworld__ | np, thanks for the suggestion | 04:55 |
| wallyworld__ | ah, ok, bug id | 04:56 |
| wallyworld__ | will do | 04:56 |
| wgrant | The mustache implementation isn't precisely fast, but it should be more than fine for a dialogue like this | 04:56 |
| wgrant | wallyworld__: Well, you also extracted the title setting | 04:56 |
| wgrant | So you can merge that back in and save a line | 04:56 |
| wgrant | As well as handling bug_id paranoidly | 04:56 |
| wallyworld__ | yeah, i extracted it because it failed too | 04:56 |
| wgrant | Indeed, thanks for finding that one. | 04:57 |
| stub | wgrant, lifeless: Can you think of why Deryck needs this index, when we didn't need similar ones for bug_sharing_policy and branch_sharing_policy? | 04:58 |
| stub | https://code.launchpad.net/~deryck/launchpad/product-specification-sharing-policy-idx-1057617/+merge/126728 | 04:58 |
| wgrant | stub: No. It's possible it's for a garbo job, but I thought that was already done. | 04:59 |
| wgrant | So I'm not quite sure what's going on there | 05:00 |
| wgrant | It may be for adding a NOT NULL constraint or something like that | 05:00 |
| wgrant | (and also we established that for a product garbo job it's not worth it) | 05:00 |
| stub | Right. In which case a partial index might be preferable, but whatever | 05:00 |
| mwhudson | ok | 05:04 |
| mwhudson | boringly, i can report that most of the inline imports are in zope.tal :( | 05:04 |
| lifeless | mwhudson: really? FAARK | 05:04 |
| wallyworld__ | wgrant: right, done | 05:04 |
| wgrant | mwhudson: In, or inside? | 05:04 |
| lifeless | mwhudson: cause, thats not peformance sensitive or anything | 05:04 |
| mwhudson | err | 05:05 |
| mwhudson | actually | 05:05 |
| mwhudson | pause that thought :) | 05:05 |
| wgrant | wallyworld__: Safer and two lines shorter. r=me. Thanks! | 05:06 |
| wallyworld__ | thank you | 05:06 |
| mwhudson | wgrant, lifeless: for your confusion: http://pastebin.ubuntu.com/1231712/ | 05:13 |
| wgrant | Rather odd. | 05:13 |
| mwhudson | if it 'helps' | 05:13 |
| mwhudson | _entnd_re = re.compile('&(#[0-9][0-9]*)(?![0-9;])') | 05:13 |
| mwhudson | i think the only conclusion possible here is that python is terrible? | 05:14 |
| lifeless | mwhudson: wtf is that doing importing ? | 05:14 |
| wgrant | Hm | 05:15 |
| wgrant | See _sre.c's call func | 05:15 |
| wgrant | Oh, that's just the module | 05:15 |
| wgrant | nevermind | 05:15 |
| lifeless | thats compiled already | 05:16 |
| lifeless | tal doing string joining won't be helping | 05:17 |
| lifeless | a list accumulator form of it would help I suspect | 05:17 |
| mwhudson | /* not a literal; hand it over to the template compiler */ | 05:17 |
| mwhudson | filter = call( | 05:17 |
| mwhudson | SRE_PY_MODULE, "_subx", | 05:17 |
| mwhudson | PyTuple_Pack(2, self, ptemplate) | 05:17 |
| mwhudson | ); | 05:17 |
| mwhudson | from _sre.c | 05:17 |
| wgrant | Oh | 05:18 |
| wgrant | Indeed | 05:18 |
| mwhudson | aaasd | 05:18 |
| mwhudson | is firefox super crashy for anyone else in precise currently? | 05:18 |
| wgrant | It's fine for me | 05:18 |
| mwhudson | hm | 05:18 |
| wgrant | Though I've run with Firebug disabled since the globalmenu issues | 05:18 |
| mwhudson | i guess my profile has eaten itself somehow | 05:18 |
| mwhudson | ah yea | 05:18 |
| mwhudson | firebug got me for a while too | 05:19 |
| mwhudson | anyway, on that depressing note, home time | 05:21 |
| wgrant | :) | 05:21 |
| wallyworld__ | wgrant: another one https://code.launchpad.net/~wallyworld/launchpad/confirmation-dialog-xss-1057901/+merge/126855 | 05:43 |
| wgrant | Ah right | 05:44 |
| wgrant | wallyworld__: r=me | 05:45 |
| wallyworld__ | thanks | 05:45 |
| === almaisan-away is now known as al-maisan | ||
| wgrant | IStoreSelector and co really have quite a few callsites | 06:26 |
| lifeless | ... | 06:39 |
| StevenK | Destroying SQLBase and SQLObject sounds rather more fun | 06:40 |
| wgrant | Not destroying | 06:40 |
| wgrant | Just moving | 06:40 |
| wgrant | To disentangle the import that lifeless was working around | 06:40 |
| StevenK | wgrant: Maybe you could destroy Storm and switch to SQLAlchemy | 06:40 |
| wgrant | Heh | 06:40 |
| wgrant | There we go | 06:42 |
| wgrant | Circular imports gone | 06:42 |
| wgrant | Without global hacks :) | 06:42 |
| StevenK | Blink | 06:56 |
| StevenK | How horrible is the diff? | 06:56 |
| wgrant | Only about 4000 lines | 06:58 |
| wgrant | Net -10 :) | 06:58 |
| wgrant | lib/lp/services/job/celeryjob.py: from lp.services.job.celeryjob import CeleryRunJob | 07:30 |
| lifeless | wgrant: ! | 07:37 |
| adeuring | good morning | 07:44 |
| lifeless | speaking of long requests | 09:04 |
| lifeless | https://oops.canonical.com/oops/?oopsid=OOPS-ddb3a4e6e473461ac93b92604efcb918 | 09:04 |
| lifeless | wheee | 09:04 |
| === adeuring changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: adeuring | Firefighting: - | Critical bugs: ~300 | ||
| === al-maisan is now known as almaisan-away | ||
| czajkowski | mgz: is this possible ? https://answers.launchpad.net/launchpad/+question/209652 | 10:31 |
| mgz | czajkowski: dependency resolution doesn't care about where packages come from | 10:38 |
| mgz | I'm not sure of bzr-builder needs extra config to look at ppas, will have a quick look | 10:40 |
| mgz | so, I don't think you can do remote recipe builds that need things from ppas, but it should just work locally | 10:43 |
| mgz | there might be specific launchpad configuration for that, generally just the main archive and sibling packages in the current ppa get used when building | 10:45 |
| czajkowski | mgz: ah ok | 10:45 |
| mgz | it's not clear from the question exactly what hes trying to do | 10:46 |
| mgz | but there's probably a way to do it :) | 10:46 |
| czajkowski | mgz: thank you | 10:53 |
| === bac changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: adeuring,bac | Firefighting: - | Critical bugs: ~300 | ||
| bac | hi adeuring. nothing in the review queue today. \o/ | 12:11 |
| === matsubara-afk is now known as matsubara | ||
| === almaisan-away is now known as al-maisan | ||
| tumbleweed | so, what happens after my merge proposal is approved? I assume someone needs to run the test suite and land the branch? | 13:17 |
| cjwatson | tumbleweed: Yes - I can land it for you now if you think it's ready | 13:20 |
| cjwatson | tumbleweed: Unless you want to add more tests per stub's comment | 13:20 |
| tumbleweed | the questions that lead to that comment were more about LP style than anything else | 13:21 |
| tumbleweed | but yes, I suppose I did intend to add tests after finding out how I was supposed to handle it | 13:21 |
| cjwatson | tumbleweed: Could you set a commit message too? | 13:21 |
| tumbleweed | cjwatson: does it need to be in a particular format? | 13:22 |
| cjwatson | No, just a description. | 13:23 |
| tumbleweed | will do | 13:23 |
| cjwatson | The tools will add necessary metadata to it. | 13:23 |
| cjwatson | So let me know once you're happy with the test coverage ... | 13:24 |
| czajkowski | jcsackett: can you look at https://answers.launchpad.net/launchpad/+question/209247 please | 13:24 |
| jcsackett | czajkowski: sorry, that one's outside the realm of what i can parse--i haven't done anything with packaging. :-/ | 13:33 |
| jcsackett | czajkowski: looks like wgrant is helping them out so far though. | 13:33 |
| czajkowski | in theory he should be sleeping | 13:33 |
| czajkowski | *theory* | 13:33 |
| jcsackett | well, i have mentioned his name. i have noticed many people in those timezones seem to appear like magic when mentioned. :-P | 13:34 |
| czajkowski | lol | 13:34 |
| wgrant | They probably want to be sent to #ubuntu-packaging or so. | 13:34 |
| wgrant | As it's now nothing to do with Launchpad. | 13:35 |
| jcsackett | czajkowski: see? magic. | 13:35 |
| jcsackett | wgrant: thanks, i'll post that as a reply to the question. | 13:36 |
| czajkowski | jcsackett: I actualy poked you with the wrong link :/ https://answers.launchpad.net/launchpad/+question/207802 | 13:36 |
| * jcsackett laughs | 13:36 | |
| jcsackett | ok, let me take a look at that one. | 13:36 |
| czajkowski | jcsackett: you mock it's been a long week | 13:36 |
| jcsackett | czajkowski: not mocking, just laughing. we all have moments like that, right? :-0 | 13:36 |
| jcsackett | s/:-0/:-)/ | 13:37 |
| czajkowski | lol | 13:37 |
| * czajkowski laughs | 13:37 | |
| tumbleweed | cjwatson: Added 1 test, everything else was already covered. And set a commit message | 13:39 |
| cjwatson | tumbleweed: OK, it's off to EC2 now; you'll get e-mail in four hours or so | 13:45 |
| tumbleweed | thanks | 13:45 |
| jcsackett | czajkowski: so we have a couple of bugs related to branch scanning failing, which looks to be what's happened here. on my own branches, i usually just delete the branch on LP and push a new copy to lp. | 13:45 |
| czajkowski | nods | 13:45 |
| jcsackett | czajkowski: however, they may be leery of doing that. | 13:45 |
| czajkowski | some font like them delted | 13:45 |
| jcsackett | czajkowski: it may be worth it for them to push new copies of those branches under new names temporarily to see if those scan right. in the mean time, you can point them at bug 904683 which tracks the issue. | 13:47 |
| _mup_ | Bug #904683: Updating branch seems to last forever <Launchpad itself:Triaged> < https://launchpad.net/bugs/904683 > | 13:47 |
| jcsackett | czajkowski: there is also bug 1018460, but i gather this may have scanned properly initially, and so this isn't applicable. | 13:47 |
| _mup_ | Bug #1018460: Failure during initial branch scan increases scan time enormously <branch-scanner> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1018460 > | 13:47 |
| jcsackett | czajkowski: and yeah, with big main branches i totally can see them not wanting to delete it--just sharing my experience. not really recommending it in this instance. :-P | 13:48 |
| === al-maisan is now known as almaisan-away | ||
| abentley | deryck: Another formatting-related test failure that I get even on stable: http://pastebin.ubuntu.com/1247615/ | 14:38 |
| wgrant | abentley: You're running with 2.7 | 15:01 |
| wgrant | We only run the test suite on Python 2.6 | 15:01 |
| === adeuring changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: bac | Firefighting: - | Critical bugs: ~300 | ||
| abentley | wgrant: True, but yesterday deryck reported he could not reproduce my test failure even with 2.7.3 on Precise. | 15:02 |
| === Ursinha is now known as Ursinha-afk | ||
| wgrant | Hmm | 15:03 |
| wgrant | That's certainly odd | 15:03 |
| abentley | wgrant: That was a different failure: http://pastebin.ubuntu.com/1230926/ | 15:03 |
| wgrant | That should always fail | 15:04 |
| wgrant | As there are no %s in the format string... | 15:04 |
| abentley | wgrant: Yet it hasn't. Both the test and implementation code are years old. | 15:05 |
| wgrant | Ah, that branch is never meant to be hit | 15:05 |
| abentley | wgrant: But the test is exercising that branch. | 15:06 |
| wgrant | Indeed | 15:06 |
| wgrant | So I am very confused | 15:06 |
| abentley | Me too. | 15:07 |
| abentley | wgrant: And today's failure is equally bonkers, because "summary" behaves like a mapping in line 5, but not in line 6. | 15:08 |
| wgrant | Hmm | 15:11 |
| wgrant | File "/home/wgrant/src/launchpad/branches/sandbox/lib/lp/code/model/branchmergeproposal.py", line 418, in setStatus | 15:11 |
| wgrant | That's from 2.6 | 15:11 |
| wgrant | raise AssertionError('Unexpected queue status: ' % status) | 15:11 |
| wgrant | AssertionError: Unexpected queue status: | 15:11 |
| wgrant | So I guess it must be treating status (which is an enum value) as a zero-length sequence | 15:11 |
| wgrant | And as for the other failure, I guess 2.7 is just more strict about what it wants from a mapping for formatting | 15:13 |
| wgrant | It is pretty odd, though | 15:13 |
| abentley | wgrant: How do you run with 2.6? | 15:15 |
| wgrant | In a Lucid LXC | 15:15 |
| abentley | wgrant: Ah. | 15:15 |
| wgrant | >>> 'foo' % BranchMergeProposalStatus.SUPERSEDED | 15:16 |
| wgrant | 'foo' | 15:16 |
| wgrant | >>> 'foo' % object() | 15:16 |
| wgrant | Traceback (most recent call last): | 15:16 |
| wgrant | >>> iter(BranchMergeProposalStatus.SUPERSEDED) | 15:16 |
| wgrant | Traceback (most recent call last): | 15:16 |
| wgrant | How odd | 15:16 |
| wgrant | It's not iterable, but it's 0-length? | 15:16 |
| abentley | wgrant: in what sense 0-length? It doesn't support len(). | 15:20 |
| wgrant | abentley: If it didn't think it was an iterable, then it would interpret it as a single thing to inject into the format string | 15:21 |
| wgrant | If it thought it was an iterable but wasn't 0-length, it'd complain that there were too many arguments | 15:21 |
| wgrant | So it must think it's a 0-length iterable | 15:21 |
| wgrant | Despite not being iterable | 15:21 |
| === matsubara is now known as matsubara-lunch | ||
| === Ursinha-afk is now known as Ursinha | ||
| === matsubara-lunch is now known as matsubara | ||
| sinzui | bac do you have time to review https://code.launchpad.net/~sinzui/launchpad/merge-non-active-person/+merge/127041 | 18:09 |
| bac | sinzui: now that i've enjoyed a fine lunch i do. | 18:14 |
| sinzui | thank you | 18:15 |
| bac | sinzui: done. thank you. | 18:21 |
| sinzui | thank you bac | 18:21 |
| abentley | deryck: I got a test failure on EC2 that I can't reproduce locally. Any suggestions? | 19:16 |
| deryck | abentley, can you email me or paste me the failure? I'll take a look when back, going for the kids from school now. | 19:18 |
| deryck | abentley, back now…. looking at the mail | 19:55 |
| abentley | deryck: I did a second run (using -t) and it passed. Just saw that now. | 19:55 |
| deryck | abentley, ah ok, cool | 19:56 |
| abentley | i.e. ec2 test -o '-t externalbugtracker-comment-imports.txt' | 19:56 |
| abentley | deryck: I hope the initial failure was due to an issue in devel that has since been resolved. | 19:57 |
| deryck | abentley, that's what I was wondering, if it was just timing with another failure not related to you. | 20:00 |
| abentley | deryck: I have no data either way. | 20:00 |
| sinzui | sigh. buildbot fellover again on a different test | 20:12 |
| sinzui | jcsackett, I will force this build. | 20:12 |
| czajkowski | evening | 20:30 |
| abentley | deryck: In your add-product-information-type-1046467 branch, the sampledata appears to contain NULLs: http://pastebin.ubuntu.com/1248311/ | 20:46 |
| deryck | abentley, looking. | 20:51 |
| abentley | deryck: (I'm getting that oops because I added a proper information_type to the model definition, but I can just mark it nullable for now.) | 20:53 |
| deryck | abentley, ah | 20:53 |
| deryck | abentley, I did this the same way I did specification_sharing_policy and relied on the garbo job, or else fixed data locally myself to play around. | 20:54 |
| abentley | deryck: That makes sense, I just figured you might want to fix the sampledata before landing, since you're changing it anyhow. | 20:56 |
| === gary_poster is now known as gary_poster|away | ||
| deryck | abentley, ah, fair point. I got the success email an hour or two ago now, though. let me see if it merged. | 20:58 |
| deryck | heh, I got my success mail from buildbot before I could finish registering for UDS and check pqm submit mails. :) | 21:10 |
| === matsubara is now known as matsubara-afk | ||
| deryck | Have a nice weekend everyone. | 22:01 |
| cjohnston | flacoste: ping | 23:43 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!