/srv/irclogs.ubuntu.com/2012/02/23/#launchpad-dev.txt

wgrantlifeless: Thoughts on the timeout thing?00:04
lifelesswhich thing00:04
wgrantSee here 50 minutes ago00:05
lifelessuhm, why is the controller destroyed?00:05
lifelessoh, paging it00:05
lifelessin00:05
lifeless..00:05
lifeless..00:05
lifeless..00:05
wgrantController destruction and soft request timeouts are both end-request events.00:06
lifelesssoft time out is checked at end of request00:06
mwhudsonit's removed in the endRequest event handler right?00:06
wgrantYep00:06
lifelessyou want the controller info in the oops00:06
mwhudsonwoo components00:06
lifelessso no, thread local values won't help00:06
lifelessyou either want the events ordered, or one event00:06
StevenKAh, so I can't delete a team I own? WTF?00:07
wgrantWell, we don't have the controller info now, and I wanted a quick way out :)00:07
wgrantBut OK00:07
lifelesswgrant: poolie put the flag stuff into the oopses for hard oopses00:07
poolielifeless, i haven't forgotten the timeline stuff :)00:09
pooliejust so you know00:09
poolieit's still within shouting distance of the top of stack00:09
lifelesspoolie: thanks, I know; it was only last week we chatted00:09
wgrantMaybe I can do the soft timeout in afterCall or so instead.00:11
wgrantUnfortunately I've mostly forgotten how publication works.00:12
wgrantBah00:13
wgrantGitHub has hung Firefox00:13
lifelesso/00:14
mwhudsonclearly you should be using safari00:14
wgrantwallyworld_: Ready to talk?00:21
wallyworld_wgrant: sure, i'll start mumble00:21
=== elmo_ is now known as elmo
wgrantIn [4]: lp = Launchpad.login_anonymously('fewfw', 'dev', 'devel')00:38
wgrantIn [5]: lp.access_policy_services00:38
wgrantOut[5]: <lazr.restfulclient.resource.Collection object at 0x106abcec>00:38
wgrantwallyworld_: ^^00:38
* StevenK wonders why deleting a team he owns brings up "There is 1 error"00:39
wgrantwallyworld_: http://pastebin.ubuntu.com/853451/00:39
wallyworld_wgrant: i can hear you01:03
wallyworld_wgrant: i'll restart mumble01:04
StevenKNow, why can't I delete a team I just created?01:06
lifelessdelete? Delete? DELETE?01:08
wgrantLPCONFIG=development utilities/create-lp-wadl-and-apidoc.py --force lib/canonical/launchpad/apidoc/01:09
StevenKlifeless: It says "There is one error" but gives no indication what that is01:10
lifelessStevenK: prod? qastating? dev?01:13
StevenKlifeless: qas01:16
StevenKhttps://qastaging.launchpad.net/~foobarbazquux01:16
wgrantAh01:16
wgrantTeam deletion won't work01:16
wgrantAt all01:16
wgrantBecause qastaging's DB is beyond all reasonably definitions of fuckedness.01:16
StevenKRight01:16
StevenKWhich means person merge is utterly fucked too01:17
lifelessits being rebuilt as we speak01:17
wgrantStevenK: Deletion is merge.01:17
StevenKwgrant: I'll mark bug 934778 as untestable01:17
_mup_Bug #934778: Typo in informational message after deleting a team <docs> <merge-deactivate> <qa-needstesting> <trivial> <Launchpad itself:Fix Committed by stevenk> < https://launchpad.net/bugs/934778 >01:17
StevenKAt the end of the day, it was a mechnical change anyway01:17
lifelessthe robots are revolting ?01:18
StevenKsourcherry is, anyway01:18
StevenKwgrant: Are you done with wallyworld_?01:28
wgrantNot even close :(01:29
wgrantStevenK: We're done now.01:43
wgrantBut I'm about to have lunch unless you need something.01:43
StevenKwgrant: Go and have lunch, we can chat about LimitedView after lunch.01:45
wgrantStevenK: Does it provide any immediate benefits?01:46
wgrantOr should we deploy now without it?01:46
StevenKIt's not flagged, so if it works as intended it should remove the confusing UI01:47
wgrantOh, has the template been modified?01:47
StevenKNot as yet01:47
wgrantThen how can it remove the confusing UI?01:47
wgrantNothing uses LimitedView on Bug, does it?01:48
StevenKI thought the template had been fixed by sinzui01:48
wgrantDidn't you just say the template hadn't been modified?01:48
wgrantHow was it tested without an adapter?01:48
* StevenK was going by what wallyworld_ told him01:49
wallyworld_hmm?01:49
wallyworld_yes, the base page template supports rendering entites with limited view01:49
wallyworld_it won't render the main or side slots for example01:50
wgrantAh01:50
wallyworld_just put in a message saying "you cannot see this"01:50
wgrantNot sure it will actually work, though.01:50
wgrantThe view init will probably trip over not having View01:50
wgrantNeeds testing01:50
wallyworld_works for teams, and yes, if init touches anything meeding view, you're stuffed01:50
wallyworld_need to add permission checks if that's the case01:51
wallyworld_in the init01:51
StevenKwgrant, wallyworld_: http://nowhiring.com.au/424936+job+Prime+Minister+of+Australia+ACT.aspx01:53
wallyworld_urgh, asp01:53
wallyworld_i might forward the link to krudd01:54
StevenKHah01:54
StevenKI hear he has a bunch of free time now.01:54
wallyworld_wgrant: the top level stuff and navigation rules all work nicely02:16
wgrantwallyworld_: You have a trivial service?02:26
wgrantStevenK: Did you change the traversal check?02:37
wgrantStevenK: I can't traverse to bugs on which I should have LimitedView.02:37
StevenKwgrant: There's a traversal check? :-(02:38
wallyworld_wgrant:  yes, there's a trivial service02:40
wgrantStevenK: That's how it 404s.02:42
wgrantStevenK: You know that this is testable locally, and the view will probably need changes, right?02:43
wallyworld_wgrant: StevenK: the traversal check has already been changed to only require limitedView IIRC02:44
wgrantThe launchpadlib collection check was. I don't know about the bug traversal check.02:45
wallyworld_i thought it had but could be wrong02:47
StevenKwgrant: So, I think I was led down the garden path a bit by wallyworld_02:50
wallyworld_StevenK: it all works fine for teams02:50
wallyworld_and bugs should be similar02:50
StevenKwgrant: If you point me at the traversal check, I'll look into fixing it02:50
wallyworld_if it needs fixing02:50
StevenKwallyworld_: wgrant has already stated it does02:51
wallyworld_do teams use different traversal?02:51
StevenKwgrant: As it is, r14855 is safe to deploy, but it doesn't work.02:51
wallyworld_the traversal stuff that was changed was in the base publisher02:51
wallyworld_so it should work for most things, no?02:51
StevenK[13:37] < wgrant> StevenK: I can't traverse to bugs on which I should have LimitedView.02:52
wgrantwallyworld_: Yes, it's separate.02:52
StevenKwallyworld_: ^02:52
wgrantOnly two types of objects have the forbidden-equals-404 check AFAICT02:52
wallyworld_hopefully only bugs is different then02:52
wgrant        # Get out now if the user cannot view the bug. Continuing may02:53
wgrant        # reveal information about its context02:53
wgrant        if not check_permission('launchpad.View', bug):02:53
wgrant            return None02:53
wgrantAh crap.02:53
wgrantStevenK: You are in for a world of pain.02:53
wgrantStevenK: You need to introduce a new view, basically.02:53
wgrantStevenK: On IBug02:53
wgrantSince IBug doesn't have a +index of its own -- it redirects to the default task.02:53
wgrantWhich is private information.02:53
StevenKPerhaps we should introduce a NotSharedWithUserView or something?02:54
wgrantIn related news, multitask bugs are hideous.02:54
wgrantAnyway, let's deploy.02:56
* wgrant deploys.02:56
StevenKwgrant: Including or excluding r14855?02:57
wgrantIncluding.02:57
wgrantNo reason to exclude, really.02:57
StevenKOr a reason to rollback02:57
wgrantWell02:58
wgrantBug's security declarations are terrible.02:58
wgrantBut I don't think it's too holy.02:58
* StevenK sighs02:59
StevenKWhere did my route to the DC go?02:59
wgrantlolxetel03:00
wgrantStevenK: you need to fix Bug's security declarations before you continue.03:01
wgrantWe can deploy now, but you must do nothing further on this until you fix them.03:02
StevenKWhich you just said were terrible.03:02
wgrantYes03:02
wgrantThey are more terrible than I expected.03:02
wgrantNot fatal, but not good.03:02
StevenKSo, my current plan was to look at Curtis' bug and branch so I can unblock him03:02
StevenKBut I can't reach the DC03:02
StevenKThis is reminding me of when wgrant couldn't.03:03
wgrantqa-meh-itll-do03:04
StevenKqa-wcpgw03:04
StevenKAh. I think I'm getting TTL expired03:05
StevenKBecause L3 are a bunch of idiots03:05
StevenKI think v6 traffic is immune, which is nice03:08
wgrantThe DC is also immune to v6, though :(03:09
StevenKHmmm, the route has changed. I think Exetel may have noticed, since their core is now dropping everything03:12
StevenK 19.|-- 4.69.134.77                                 88.9%     9  244.9 244.9 244.9 244.9   0.003:14
StevenKMuppets.03:14
StevenK 20.|-- 4.69.137.77                                 88.9%     9  312.1 312.1 312.1 312.1   0.003:14
StevenK 21.|-- ???                                         100.0     8    0.0   0.0   0.0   0.0   0.003:14
StevenKYay, I can hit the DC again03:26
StevenKNice how it took Exetel 30 minutes to drop their route to Telstra, since they are the cause.03:26
StevenKWhat the?04:30
StevenKLook at any bug reported against LP and hover over "Launchpad itself"04:30
wgrantStevenK: Mmm, delicious 2007.04:38
wgrant(2007 + bugs, but still 2007)04:42
StevenKwgrant: So, what needs to happen with the bug security adapters?04:52
wgrantStevenK: They leak tonnes of stuff that needn't be leaked.04:52
wgrantGrab https://bugs.qastaging.launchpad.net/api/devel/bugs/718213/duplicate_of?ws.accept=application/json as anonymous04:53
_mup_Bug #718213: Can't access due to content. <Internet Archive - Tech Support:New> < https://launchpad.net/bugs/718213 >04:53
wgrantIt's a dupe of bug #1234, which I've made private.04:53
_mup_Bug #1234: Gina is an unmaintainable mess of command line options, environment variables and shell scripts <lp-foundations> <Launchpad itself:Fix Released by debonzi> < https://launchpad.net/bugs/1234 >04:53
StevenKLots of that JSON is redacted05:00
wgrantAnd lots is not.05:01
StevenKwgrant: What do you want to see disappear?05:02
wgrantPretty much all of it.05:02
wgrantI don't have access to the bug.05:02
wgrantI don't really need to see much.05:02
StevenKI think the issue is the <allow ... in the ZCML05:02
wgrantIndeed.05:03
StevenKMost of them should drop to launchpad.View ?05:03
StevenKWhich means anonymous users can't fetch them for public bugs eithwer05:03
wgrantI would suspect so.05:03
wgrantOh?05:04
StevenKOh, I'm wrong?05:04
wgrantlaunchpad.View is held by anonymous users for public bugs.05:04
wgrantI would hope.05:04
wgrantIf it's not it's a bug.05:04
StevenKSo, id needs to stay, as does private.05:04
wgrantRegrettably.05:05
StevenK[16:06] < Solver> Dodo announced THE INTERNET to Telstra and they accepted it05:07
wgrantHeh05:07
StevenKwgrant: ^ Cause of my DC unreachability05:07
* StevenK kicks wallyworld_05:08
wallyworld_ouch05:08
StevenKThat's what you get for importing ILaunchpadCelebrities into bugs.security05:08
wallyworld_did i do that?05:09
StevenK14186.3.3   ian.boo | from zope.component import getUtility05:09
StevenK                    |05:09
StevenK                    | from lp.app.interfaces.launchpad import ILaunchpadCelebrities05:09
wallyworld_i should have used personroles05:10
wallyworld_not sure why i didn't05:10
wallyworld_moment of madness05:10
StevenKuser is already an role05:10
StevenKwallyworld_: I'm putting up a small branch to fix05:11
wallyworld_ok. i thought by what you were saying you'd do it as a drive by05:12
StevenKNo, I don't want to cause other issues in a branch where I'm moving 30 attributes05:14
wgrantStevenK: Be warned, there may be 10 or more tonnes of test fallout.05:15
StevenKwgrant: Yes, which is why I don't want to do any drive-bys05:16
* wallyworld_ stabs unity/compiz/whatever... restart time :-(05:16
StevenKwallyworld_: https://code.launchpad.net/~stevenk/launchpad/less-celebrites-bug-security/+merge/9431305:19
wallyworld_looking05:20
wallyworld_StevenK: r=me05:21
wallyworld_thanks for fixing05:21
wgrantBah05:24
wgrantI wish we were on postgres 9.105:24
StevenKTossing through ec2 just to see if there are any failures05:24
wgrantBah05:46
wgrantBugHeatUpdater is only 300 times faster.05:46
StevenKOnly05:46
wgrantYes, only.05:47
wgrantWas hoping for more than 1000x.05:47
wgrantMust still be something wrong.05:47
stubA penchant for wishful thinking? :-)05:47
wgrantOh, blah, the bug->task heat mirror trigger.05:48
wgrantForgot about that.05:48
nigelbwgrant: https://twitter.com/#!/nigelbabu/status/172561507148763136 :D06:01
nigelbI hope I didn't convey that lp is only 300 times faster...06:01
wgrantHeh06:01
wgrantOh oops.06:12
nigelbMan, "What Could Possibly Go Wrong" should be like the key phrase for LP.06:29
StevenKwgrant: Only 41 failing tests.06:41
StevenKwgrant: Right, http://pastebin.ubuntu.com/853644/ looks pretty good, and has no failing tests.07:00
StevenKPity default_bugtask, getBugTask and userCanView had to be made public too07:01
wgrantStevenK: default_bugtask does not need to be public.07:01
StevenKThe factory goes boom if it is not07:01
wgrantThat's not an excuse.07:02
wgrantThe factory can log in as an admin or remove the security proxy :)07:02
StevenKuserCanView is the excusable.07:02
StevenKs/the //07:02
wgrantYes.07:03
StevenKgetBugTask is not, right?07:03
wgrantUnless you are extremely convincing, it is indeed not excusable.07:04
wgrantI would expect the only public attributes to be id, private, and userCanView07:05
wgrantPossibly not even private :)07:05
StevenKI'm leaving private for the time being07:05
wgrantYeah07:05
wgrantprivate isn't leaking07:05
wgrantIt's just dirty07:06
StevenK  File "/home/steven/launchpad/lp-branches/less-bug-leakage/lib/lp/testing/factory.py", line 1682, in makeBug07:06
StevenK    bugtask = bug.default_bugtask07:06
StevenKForbiddenAttribute: ('default_bugtask', <Bug at 0xd21c150>)07:06
wgrantwith admin_logged_in()07:07
wgrantOr rSP07:07
wgrantProbably rSP07:07
wgrantDepending on whether you're returning the task or not.07:07
StevenKNo, we're not returning it07:08
StevenKUsing it for date_closed and milestone only07:08
wgrantThen you can just unproxy it and nobody will care :)07:08
StevenKThis branch is *tiny*07:13
StevenK+7/-907:13
GStevenK: and you've got 2 lines for the future :P07:15
StevenKI suspect I have a lot more than that.07:15
StevenKBoth wgrant and I tend to remove a lot of code.07:16
wgrantzomg08:31
wgrantBugHeatUpdater caught up08:31
wgrantFor the first time in more than a year08:32
wgrantMaybe we will stop having so many oopses now.08:32
StevenKsteven@liquified:~/launchpad/lp-branches/less-bug-leakage% subunit-stats < lp.log | grep Failed08:32
StevenKFailed tests:    30708:32
wgrantUhoh08:32
wgrantAh08:32
wgrantNot so bad08:32
StevenKAlmost all of them are default_bugtask08:33
wgrantThat's why I haven't preemptively euthanised you yet.08:33
StevenKOh?08:33
StevenKWhy would you want to do that?08:33
wgrantBecause if they were 307 different failures, you'd want it.08:34
StevenKHaha08:34
wgrantstub: http://people.canonical.com/~wgrant/launchpad/bug-legacy-mirror.sql is my current plan for migration of bugs to the new privacy model. Lets us safely run both schemas in parallel during the migration without worrying about syncing in the app. Rather abhorrent, I know, but it's fast and temporary.08:50
stubWe have too much data to migrate in a 5 minute window?08:51
adeuringgood morning08:52
wgrantstub: Given the interactions with already unperformant bug search, we'd like to be able to revert to the old ways quickly if things don't perform optimally.08:53
stubIf it is good enough to get past staging, can we tell if any performance issues on production are because of the new schema or because of existing issues?08:54
wgrantHeh, have you tried a bug search on staging? :)08:55
stubSo we have a new feature we have no way of qaing ?08:56
wgrantMost recent risky performance tweaks have been done by deploying them with feature flags, releasing to launchpad, then beta-testers, then everyone.08:57
wgrantBecause staging just isn't good enough.08:57
stubI see a fairly complex trigger, so I'm wondering if we can avoid it and the potential for bugs it brings.08:57
wgrantAnd for stuff like bug search, where there are thousands of different queries...08:57
wgrantHeh, yeah. There is that issue.08:57
stubIs the compatibility code more likely to cause troubles than the new work :)08:58
stubWill we be hiding use of the new schema behind a feature flag, so most users continue to use the old schema?08:58
* stub can't even load bugs.staging.launchpad.net...08:59
wgrantstub: Yes, it will all be behind feature flags.08:59
stubSo we must have the two models running in parallel then.08:59
wgrantWell, we basically have to. Even if we want to just cut over, we'd have to have horrifying views or hybrid appserver code to cope after the change.09:00
wgrantBecause we don't have code changes at the same time.09:00
stubPopulating one schema asynchonously is sucky, so I think we are stuck with a trigger like you have09:00
wgrantI figure doing it this way makes that much easier and safer.09:00
wgrantAnd lets us back out.09:00
stubyup09:00
wgrantThe trigger is sucky, yes.09:00
=== czajkowski changed the topic of #launchpad-dev to: https://launchpad.net/ | Help contact: czajkowski | Launchpad is an open source project: https://dev.launchpad.net/ | This channel is logged: http://irclogs.ubuntu.com/ | User Guide: https://help.launchpad.net/ | Support: https://answers.launchpad.net/launchpad | For packaging help: join #ubuntu-packaging
czajkowskigah09:01
=== czajkowski changed the topic of #launchpad-dev to:
stubMuch better :)09:02
wgrantstub: The idea is that we get the new schema running in parallel, adjust the bug search code to optionally use it, do lots of performance testing. Based on the results of that, probably implement a flattened bug+bugtask table (which I've already done some testing on, with very promising results). Once we're comfortably on the new schema, dump the old one and make the new one writable and we're in a happy place.09:02
czajkowskistub: great start to my day eh09:03
=== stub changed the topic of #launchpad-dev to: <Insert Coffee to continue>
stubwgrant: Yes, this seems the sanest approach.09:04
wgrantstub: Thanks, just wanting to get your veto before I do much more :)09:06
stubczajkowski: I have your attempts in scrollback if you don't have the history :)09:06
=== elmo changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugtasks: 4*10^2
czajkowskistub: elmo thanks.09:07
wgrantAw09:08
* czajkowski goes and makes another cuppa tea 09:08
stubwgrant: urgh... using EXECUTE to ensure the query gets planned, rather than using the static plan generated when the procedure was compiled :-(09:09
wgrantstub: I know :(09:09
wgrantI'm not sure what the actual prepared plan is.09:09
wgrantBut it takes 20ms rather than 150us09:09
stubI just make elmo buy me more hardware.09:10
wgrant:)09:10
wgrantstub: The indices are partial on product IS NOT NULL and distribution IS NOT NULL, so I guess it can't use them unless it knows there aren't any nulls.09:13
stubwgrant: Adding redundant 'product is not null' clauses might help there.09:14
wgrantIndeed.09:16
* wgrant tests.09:16
wgrantIndeed, that works.09:18
wgrantEven a few hundred microseconds faster.09:18
wgrantI guess it will avoid the present prod planning problems, too.09:19
=== almaisan-away is now known as al-maisan
wgrantIdeally I'd use conditional triggers rather than filtering in plpgsql, but that's not available until 9.0 :(09:21
StevenKHeh, wallyworld_ has landed his two step picker again09:24
wgrantYep09:24
StevenKShall we go for two rollbacks? :-)09:24
wallyworld_i fixed it09:24
wgrantLooked over it this morning, and there're no glaring issues this time.09:24
StevenKs/this time/yet/ ? :-P09:24
wgrantAnd a global potential XSS fix.09:24
wgrantCan't argue with that :)09:24
StevenKWe might even deploy it tomorrow09:25
wallyworld_wgrant: btw, the launchpadlib api doesn't work with the new services stuff :-(09:25
wgrantwallyworld_: Does so. What doesn't?09:25
wallyworld_wgrant: lp.load('/services') fails09:25
wgrantThere's no reason this shouldn't work, but it might take a bit of poking.09:25
wgrantwallyworld_: What error?09:25
wallyworld_invalid xml id #service_factory09:26
wallyworld_because there's no collection exported09:26
wgrantSinister.09:26
wgrantDo you have a branch?09:26
mrevellHello09:26
wallyworld_wgrant: just writing a test or two, give me a few minutes09:26
wgrantwallyworld_: Sure09:26
wallyworld_mrevell: g'day09:26
wgrantwallyworld_: We should be able to make lp.services.access_policies work as well.09:26
wallyworld_wgrant: hope so09:27
wallyworld_mrevell: only a few more days for you to put up with bigjools \o/09:27
mrevellwallyworld_, We have a band waiting at the air port to see him off. That and big banners with a "Never darken these shores again".09:29
wallyworld_hah09:29
wallyworld_mrevell: but now *I* need to deal with him09:29
wallyworld_no fair09:29
wgrantstub: Want to rebuild bug_fti for hopefully the last time for a while? :)09:40
wgrantstub: The new BugHeatUpdater took 1.5 iterations to do the entire backlog, so that should be it.09:40
wgrant(currently at 67% bloat)09:41
wgrantHm09:46
wgrantI wonder if I can make update-pkgcache massively faster with SPPH.spn09:46
wgrantThat would be nice.09:46
wgrantStevenK: Those 307 errors look pretty easily fixable?10:06
wgrantblink10:10
wgrantThere's no SPPH index on (archive, distroseries) :(10:10
StevenKwgrant: I'm not sure, I haven't investigated. I can rSP a lot, but that doesn't fill me with happiness.10:10
wgrantStevenK: In the factory it's fine.10:10
wgrantThe factory can commit any evil it wants, as long as the objects it returns are proxied.10:10
StevenKThose 307 are with the factory fixed.10:10
wgrantAh10:10
wgrantI'll be making the factory commit a loooot of evil in coming months.10:11
wgrantBut it should be very fast evil :)10:11
StevenKwgrant: Oh, and that 307 is just -m bugs10:15
StevenKI'm suspecting more fallout10:15
wgrantShould be relatively minor elsewhere.10:19
wgrantI actually expected more.10:19
wgrantGiven that you're fixing buggy security declarations from 2005.10:19
wallyworld_wgrant: wip, https://code.launchpad.net/~wallyworld/launchpad/pillar-access-service-infrastructure/+merge/9433810:20
wallyworld_wgrant: i used named utilities to get the services10:20
wallyworld_so the servicefactory is very simple10:21
wallyworld_to add a new service, just define it in the zcml10:21
wallyworld_and that's it10:21
wgrantwallyworld_: Ah10:21
wgrantThat's why it won't work in launchpadlib.10:21
wgrantIt won't be in the WADL10:21
wgrantYou should still be able to say lp.load('/services/accesspolicy'), but not lp.services.accesspolicy10:22
wallyworld_the servicefactory  attribute10:22
wallyworld_load doesn't work10:22
wallyworld_lp.load('/services') neither10:22
wgrantHm10:22
wallyworld_undefined xml id10:22
* wgrant tries.10:22
wgrantDid you actually rebuild your WADL?10:22
wallyworld_yes, several times10:23
wgrant:(10:23
wallyworld_i may have missed something10:23
wallyworld_i'm happy how the webservice side of things is working10:23
wgrantIt's also possible that launchpadlib is being hilarious with WADL caching.10:23
wallyworld_maybe10:23
wallyworld_bbiab, gotta feed the dog10:23
wgrantwallyworld_: In [5]: lp.load('/services/accesspolicy')10:31
wgrantOut[5]: <access_policy_service at https://api.launchpad.dev/devel/services/accesspolicy>10:31
wgrantwfm10:31
wallyworld_wgrant: what, it worked for you?10:31
wallyworld_one thing i had to do was define a url for the service10:32
wallyworld_otherwise it would complain10:32
wgrantIndeed, it will do that.10:32
wallyworld_i guess i'll try make clean and do it again10:32
wallyworld_but i've done that already10:32
wgrantJust blow away lib/canonical/launchpad/apidoc and make build again10:33
wallyworld_wgrant: can you invoke a method?10:33
wgrantwallyworld_: No. There's a launchpadlib bug about that, IIRC>10:33
wgrantBug #68176710:33
_mup_Bug #681767: Can't use relative URLs with launchpadlib.Launchpad - generates URI object that breaks wadllib <lp-foundations> <launchpadlib :Triaged> < https://launchpad.net/bugs/681767 >10:33
wgrantReasonable workaround.10:34
wgrantIn [14]: aps = lp.load(str(lp._root_uri) + '/services/accesspolicy')10:34
wgrantIn [15]: aps.getAccessPolicies()10:34
wgrantOut[15]: u'[{"index": 0, "description": "Everyone can see this information.\\n",10:34
wallyworld_wgrant: so, you can't go lp.load('/services/accesspolicy').get('xxxx') ?10:34
wgrantetc10:34
wallyworld_ah ok10:34
wallyworld_well, i'm pleased it all works for you10:35
wallyworld_now to try for me10:35
wgrantwallyworld_: Heh10:35
wallyworld_this is good10:35
wgrantwallyworld_: So, you can probably define a generic browser:url for services.10:35
wallyworld_i'm liking what we can do with this10:35
wgrantLet me try.10:35
wallyworld_i have10:35
wallyworld_already10:35
wgrantAh10:35
wgrantI thought you meant a specific one.10:35
* wgrant actually reads the code.10:35
wgrantAh, cool.10:36
wallyworld_wgrant: all you need to do to add a service is add a bit of zcml for it10:36
wallyworld_done10:36
wallyworld_really easy :-)10:36
wgrantWould be nice if we could get it into the WADL, but that would require being able to define custom adapters, or some runtime interface patching.10:36
wallyworld_yeah, a future branch10:37
wgrantStill, the current launchpadlib way is not too bad :)10:37
wallyworld_yes, it works10:37
wallyworld_not too fugly10:37
wallyworld_wgrant: it fails for version='devel'10:40
wallyworld_when i do version='beta' it works10:41
wgrantwallyworld_: That's what I'm using.10:41
wgrantlp = Launchpad.login_anonymously('fewfw', 'dev', version='devel')10:41
wgrantBlow away your launchpadlib cache10:41
wallyworld_In [1]: from launchpadlib.launchpad import Launchpad10:41
wallyworld_In [2]: lp = Launchpad.login_with('testing', service_root='https://api.launchpad.dev', version='devel')10:41
wallyworld_In [3]: ap = lp.load('/services/accesspolicy')10:41
wallyworld_where's the cache?10:41
wgrant~/.launchpadlib/api.launchpad.dev/cache10:41
wallyworld_ok, didn't even know that was there10:41
wgrantMight as well dispose of ~/.launchpadlib/cache as well, if it's there10:42
wgrantI think it's deprecated.10:42
wallyworld_wgrant: yay, it was the supid cache al lalong :-(10:43
* wallyworld_ add a launchpadlib test to the branch10:43
wallyworld_s/add/adds10:43
wgrantHeh10:44
wallyworld_i wasted sooo much time due to that :-(10:44
wgrantThat's what caches are for :)10:45
wallyworld_but cahces need to be properly maintained :-/10:46
wgrantPssht, details.10:46
wallyworld_sigh10:46
* wallyworld_ has more unity / compiz grief10:47
wgrantOh10:52
wgrantThis just gets better and better10:53
wallyworld_wgrant:  i was offline briefly. what's "this" ?10:53
wgrantwallyworld_: update-pkgcache10:53
wgrantUnrelated :)10:53
wallyworld_ok :-)10:54
wgrantJust a 20h daily script that I want to fix.10:54
wallyworld_20h. there's still 4 to go :-)10:54
wgrantIt used to be 25h10:54
wgrantUntil I fixed it a bit.10:54
wgrantBut it still does about 1000x too many queries.10:54
wallyworld_25h is ok if we slow down the rotation of the earth10:54
wgrantThat's a good idea.10:55
wgrantOh look that query is now 400 times faster10:56
wgrantsigh10:56
=== matsubara-afk is now known as matsubara
* wgrant aims to get it down to 5 minutes or less.11:03
wgrantAh, it's back up to 23 hours :(11:07
=== danhg_ is now known as danhg
=== jtv is now known as jtv-afk
=== al-maisan is now known as almaisan-away
=== jtv-afk is now known as jtv
=== almaisan-away is now known as al-maisan
=== mrevell_ is now known as mrevell
=== rick_h_ changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: rick_h* | Firefighting: - | Critical bugtasks: 4*10^2
=== jcsackett changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: jcsackett, rick_h* | Firefighting: - | Critical bugtasks: 4*10^2
abentleyrick_h_: Could you please review https://code.launchpad.net/~abentley/launchpad/bulk-insert/+merge/94271 ?14:03
rick_h_abentley: sure thing14:03
abentleyadeuring: I'm ready to join in on the Job system revamp.  Can you bring me up to speed?14:04
adeuringabentley: i only created a basic branch:14:05
* adeuring os tryingto fins it again...14:06
adeuringabentley: lp:lazr.jobrunner14:06
adeuringabentley: mumble14:07
adeuringabentley: sorry,mylaptop crashed14:21
abentleyadeuring: no worries.14:21
czajkowskihttps://bugs.launchpad.net/launchpad/+bug/936989  could someone point me in the right direction of where I'd find the answer to this ?14:23
_mup_Bug #936989: Launchpad doesn't properly redirect if ubuntu-bug user is allready logged <apport-bug> <i386> <oneiric> <Launchpad itself:New> < https://launchpad.net/bugs/936989 >14:23
czajkowskiplease14:23
sinzuiczajkowski, I do not know what to say. I can confirm that Lp did redirect me 15 minutes ago14:24
czajkowskisinzui: likewise, but i do know from time to time i've had glitches also14:25
czajkowski:/14:25
czajkowskinot today when I try and replicate it though14:25
czajkowskisinzui: good morning btw :)14:25
sinzuiis it? I do not think I have ever had a good morning. I am happy to have survived the night though14:26
sinzuiczajkowski, I wonder if the user has multiple profiles. The console and launcher are selecting different profiles14:27
flacosteadeuring, deryck: are we done with bug #14:28
flacosteexec'ing bin/maas in the run script would be enough to satisfy14:28
flacostesupervise.14:28
flacosteraarh14:28
flacostepaste, i hate you14:28
deryckheh14:28
czajkowskisinzui: I'm not a great sleeper and can function on about 4hrs sleep pretty well.14:29
deryckadeuring, did you chat with bryce yesterday about it?14:29
flacosteadeuring, deryck: i meant to say: are we done with bug #829074? any news from the stakeholders?14:29
_mup_Bug #829074: Show bugs that are not known to affect "official" upstream <bugs> <escalated> <qa-ok> <Launchpad itself:Fix Released by adeuring> < https://launchpad.net/bugs/829074 >14:29
adeuringderyck,flno, i did not yet talk with bryce14:29
deryckadeuring, please try to get that done today so flacoste can close out the request completely.14:30
adeuringok14:30
sinzuiczajkowski, are you gloating or challenging me. On my 25th birthday my metabolisms slipped from 5th gear to 3rd. I need 6 hours of sleep or more.14:32
flacostethanks adeuring14:35
czajkowskisinzui: hey I'd like more don't get me wrong. I just seem to have missed the lesson on how to sleep through a night!14:35
deryckvideo games and melatonin14:37
deryckczajkowski, lesson learned ^^ :)14:37
czajkowskideryck: I do my loco work at night that helps, next week I get the go ahead to go back to the gym, which should kill me and help me sleep! :)14:38
sinzuideryck, Does that combat children hoping into bed? What about fighting cats? I wake up with scars, maybe I referred a cat fight in my sleep14:39
derycksee cats, that's your problem right there.14:39
deryckand I've grown immune to kids toes in my ribs somehow. :)14:40
czajkowski lol14:41
lifelessflacoste: did you have an opinion on the oops-on-local-404 thing ?14:53
adeuringabentley: new attempt to mumble?14:55
abentleyadeuring: sure.14:56
abentleyadeuring: http://pastebin.ubuntu.com/854075/14:57
flacostelifeless: whether to show OOPS id on those?15:01
lifelessflacoste: and whether to even make an OOPS15:01
lifelessflacoste: because of linkification, its not uncommon to have bad links in-site that we don't control15:02
flacostelifeless: right15:02
flacostewe are not doing much with those OOPS right now15:02
lifelessI've been torn for a while, I think its not really a win ATM15:02
flacosteso dropping them wouldn't have any impact15:02
lifelesscan always turn it on again later15:02
flacosteexactly15:02
lifelessok, cool15:02
rick_h_abentley: did jelmer review the other one for you or was that just a drive by?15:13
abentleyrick_h_: OTP15:16
czajkowskisinzui: in the LP licience reviews, I've 2 maintained by us, both look a bit dubious , later on when you can would15:17
czajkowskiyou please15:17
sinzuituxubuntu can be deactivated15:17
* sinzui hopes weiqi is real15:18
czajkowskinods15:19
sinzuiczajkowski, I think there is a problem with gnome-go. It is real, we should own it.15:19
sinzuiczajkowski, The code import is odd, not wrong, just odd15:20
czajkowskisinzui: I dont feel so bad now for asking, I managed to do the others this morning/so far but those two just looked odd to me15:20
sinzuiI happen to know the registrant is the baltix maintainer, and his branch in owned by baltix. I assume that branch is his distro version. The branch is also the upstream import.15:21
sinzuiczajkowski, I see the branch has a recipe and looking at the source, it has a debian dir: http://bazaar.launchpad.net/~baltix/gnomego/trunk/files/head:/src/15:23
sinzuiczajkowski, We want to set this branch as the focus of development for gnomego. I think baltic will want to create a separate branch in time and maybe an extra recipe to make upstream integrate into baltix15:24
sinzuiczajkowski, visit https://code.launchpad.net/gnomego15:25
czajkowskiok15:25
sinzuiczajkowski,  copy the branch URL (lp:~baltix/gnomego/trunk) then follow the link to configure code hosting15:26
sinzuiPaste the link to the branch field and Update to be done15:26
czajkowskilets hope I dont break things!15:27
=== kirkland` is now known as kirkland
sinzuiczajkowski, I think the second option in that form is fundementally broken (Create a new, empty branch in Launchpad and link to this series). Creating an empty branch does not really help anyone. We should remove the feature to register an empty branch15:28
czajkowskisinzui: I do the change code hosting, but owner, It picks the teams I'm on, so I select the registery ?15:29
sinzuiIt want's to change the owner? That is odd15:31
sinzuiI would pick ~registry, but I do not think we should be changing the owner of the branch unless we are asked to15:31
sinzuiThat would break recipes15:32
sinzuiczajkowski, maybe I am doing this the hard wary. See the " set it now" link on https://code.launchpad.net/gnomego15:33
sinzuiczajkowski, This simpler for want us to paste ~baltix/gnomego/trunk into the field an Update15:34
czajkowskisinzui: ahh thats a lot easier15:34
czajkowskithanks sinzui15:34
czajkowskican I have a brain dump of sinzui and lifeless please on launchpad topics!15:34
abentleyrick_h_: I thought it was a review, but it looks like technically, he didn't vote.  Anyhow, it didn't get a LOC waiver from lifeless, so it won't be going in.15:35
sinzuiI think my information is ordered by iChing, broken into topics about food. I do not think many people can make sense of it.15:35
czajkowskisinzui: oh food, food I cna understand!15:36
=== al-maisan is now known as almaisan-away
=== matsubara is now known as matsubara-lunch
=== Ursinha is now known as Ursinha-nom
=== Ursinha-nom is now known as Ursinha
rick_h_jcsackett: if you get a sec can you peek at https://code.launchpad.net/~rharding/launchpad/watch_jsbuild/+merge/94379 ?16:52
rick_h_jcsackett: note that it's dep on a branch that's in ec2 land atm16:53
jcsackettrick_h_: sure. talking with danhg right now about some text changes. after that i can take a look at yours.16:53
rick_h_np, ty much16:53
abentleyadeuring: As of r7, you don't need virtualenv, "make develop" will set you up and "make check" or bin/python setup test will run the test suite.17:32
adeuringabentley: cool17:32
jcsackettrick_h_: i see pyinotify added to the versions; you're going to update sourcedeps with that before landing, i assume?17:40
rick_h_jcsackett: yes, should be there alreday17:40
rick_h_already*17:40
jcsacketthrm.17:40
jcsackettperhaps i didn't update today.17:41
rick_h_just did it before the MP17:41
rick_h_but I did update and make sure it came down17:41
jcsackettrick_h_: dig.17:41
jcsackettrick_h_: I'm unfamiliar with pyinotify; i see you have a method for IN_CREATE and IN_MODIFY but they both use event.mask == pyinotify.IN_MODIFY; is that correct?17:44
rick_h_jcsackett: ah, let me update that sorry.17:45
=== deryck is now known as deryck[lunch]
jcsackettrick_h_: no worries. lemme know when the fix goes up.17:48
rick_h_jcsackett: yea, will be a few. I actually had bigger plans there I got sidetracked and forgot about oops17:49
czajkowskinn folks18:03
=== matsubara-lunch is now known as matsubara
lifelessmorning for reals18:39
rick_h_howdy lifeless18:40
lifelesshey rick_h_18:40
=== deryck[lunch] is now known as deryck
james_wdoes anyone know where the session db sql sanitising code that LP uses lives?18:48
lifelessyes18:49
james_wah, got it18:49
lifelessand now you do too :)18:49
james_wunfortunately it's not all that helpful18:49
lifelessits very simple18:50
james_wbecause I can't assume a separate db18:50
lifelessI'm sure django will have different needs18:50
lifelesscan you assume a particular substring match18:50
james_wI'm not sure what a good substring would be18:51
lifelesswell, whats your session table ?18:51
james_wwe can assume the session table name18:51
james_wprobably18:51
james_w'django_session'18:52
lifelessseems unlikely to mask legitimate traffic, and if it does unlikely to be harmful18:53
james_wwith 'session key' and 'session data'18:53
james_wjust if "django_session" in query: query = "<session query>" ?18:54
lifelessthats what I'd do18:54
lifelessI mean, you can be more fancy, but will it make a real difference to you?18:54
james_wprobably not18:55
lifelessif someone wants to attack the system, *and* cause oopses, *and* hide a query, they can put django_session in as user data18:55
lifelessbut that wouldn't mask every query I expect18:55
lifelessyou could look at what django compiles for the query and do a tighter match - e.g. 'FROM django_session'18:56
james_wthe other one would be user passwords18:56
james_wwhich could be more annoying to have redacted18:56
lifelessyou're not using openid ?18:57
james_wyes18:57
james_wbut it doesn't mean that it should be ignored in a librar18:57
lifelessso, same principle will apply18:58
lifelesspossibly admins will have local accounts for you18:58
rick_h_jcsackett: ok, updated https://code.launchpad.net/~rharding/launchpad/watch_jsbuild/+merge/9437919:11
lifelessrick_h_: did you consider putting your watch thing as a separate tool? It seems totally unrelated to LP itself19:11
rick_h_jcsackett: I meant to go back and implement the meta.js generation when you add a new file19:11
rick_h_lifeless: yes, except that the "knowledge" of how things need to get built isn't easy to pull out some how that I can think of19:11
rick_h_since it needs to know to regex the filename, copy over the bug/code/etc app part, and map it over to the build/js/lp/$app/xxx19:12
rick_h_lifeless: but yea, I'm hoping this and hte lpjsmin can enable the other projects to keep common good practice JS/combo load build patterns19:12
lifelessthat logic seems needed for the main build part too; so presumably there is a mapper function we call, which it could use.19:12
rick_h_lifeless: yea, except currently that 'logic' is in a combo-rootdir shell script19:13
lifelessso, this duplicates it into python ?19:13
rick_h_a little bit, this requires convoy package to be avail to run and such19:13
rick_h_this ONLY takes care of the combo loader, it doesn't touch launchpad.js19:13
* lifeless thinks you're building up a LOC debt rapidly :)19:13
rick_h_that's too slow since it has to be completely rebuilt19:14
rick_h_well, but the idea was that I ripped out the jsmin code, decrease maint by making JS dev faster19:14
lifelessI like the idea19:14
rick_h_so yes, this adds a new script file that's + LOC19:14
rick_h_but with the jsmin branch I've - LOC as well19:14
lifelessI suggest having a mapper function, putting the watch tool somewhere separate (e.g. convoy perhaps? they might benefit from it) and having it be trivially callable with our custom mapper19:15
lifelessI worry - a lot - when we add dev tools to the LP tree19:15
lifelessthey are typically entrenched and very hard to reuse when we do that.19:15
rick_h_lifeless: understand, and thought of the callable. Do you have a locationsuggestion for that? I didn't see one in my initial looking19:15
lifeless(which existing entrenchment you're running into)19:15
lifelessrick_h_: lp.services.js.something ?19:16
rick_h_lifeless: yea, I think we're in agreement in principle for sure, just hard time implementing19:16
rick_h_would a new service dir with just an __init__.py exposing the callable be ok then?19:17
lifelessfine by me19:17
rick_h_and about the variable defs? the paths? just move those into that callable then? They're a bit buried19:17
lifelesswhere are they defined at the moment, and are the alterable ?19:18
rick_h_I guess I can start there, no worse off.19:18
lifelessI nearly said variable but then the world exploded19:18
rick_h_right now they're in the top of the script19:18
rick_h_so we'd still need something Make can call, but I guess that can be python -m lib.lp.services.js.run_jswatch or something19:18
rick_h_right now those build paths and such are part of all the build/makefile/buidlout process19:19
rick_h_so it just feels strange to move them down into tree19:19
lifelessrick_h_: a buildout template can be just: #!\nfrom lib.lp.services.js import path_map\nfrom convoy.buildwatch import watcher\ndef __main__():watcher(path_map)\n19:20
rick_h_lifeless: right, yea. Let me know it out and I'll see if things fall into place.19:20
lifelessmake can then just call bin/jsbuildwatch or whatever the name is again19:20
rick_h_so I'm looking to add a jsbuild_watch python module that's external/reusable requiring a callback to be sent to it right?19:20
rick_h_lifeless: right, cool19:21
lifelessyeah19:21
lifelesssomething which you can write a tiny bit of code to use in e.g. maas, or u1, or ...19:21
rick_h_lifeless: right, that might make sense to take back to convoy. I'll check it out.19:21
rick_h_jcsackett: ok, so nvm on the review :) thx19:22
jcsackettrick_h_: fair. :-p19:26
james_whttps://code.launchpad.net/~james-w/python-timeline-django/obscure-session-info/+merge/9443919:26
rick_h_james_w: I don't think we can review that one for you19:29
james_wrick_h_, that's ok thanks19:30
james_wrick_h_, I was just noticing it here because I was discussing it a little earlier in here19:30
james_wsomeone else on my team will do the actual review19:30
james_wsorry for not being clear19:30
rick_h_james_w: ah ok cool. Wanted to try to get my OCR hours in but no buttons there19:30
james_w:-)19:31
rick_h_lifeless: re last-mod header, what header would you send the content hash back with?19:57
rick_h_lifeless: I was noticing that I was geting 304 not-modified responses from some static files, but nothing out of the combo loader so got down this path of why/how to get the combo loader files treated the same as static files19:57
rick_h_lifeless: nvm, so you mean set the hash as the etag vs last-mod header19:59
lifelessayup20:04
lifelessof course this means you have to honour INM as well - or just set good cache headers and put a squid in front20:05
lifelessrick_h_: actually, thats better - for *us* we never edit a file, so we want to just set cache-control: 100000 or something20:06
lifelessand expires: now+1 year20:06
rick_h_lifeless: yea, so I ran into this on an outside app I'm using convoy for20:06
lifeless(more or less, I can dig up the canonical make-it-never-expire reference fo you layer)20:06
rick_h_the trouble I was getting is that while I set expires headers way out, it would still request because it didn't have a last-mod date in the resp20:06
lifelessrick_h_: ah cool. So, LMT can be done, but IMO etag is safer20:07
lifelessup to you20:07
rick_h_I didn't try an etag header though. Trouble with that is that convoy streams the response, so getting the etag header is going to be a pita20:07
lifelesstechnically you can set it at the end, but I know of -no- web stacks that handle trailers correctly20:08
rick_h_lifeless: yea, thanks for the heads up. I'll have to look into it some more.20:08
lifelessno worries20:08
rick_h_lifeless: do you have 5 to do Q&A on that though? I'm wrapping my head around part of it and think it might be faster to voice vs type?20:09
lifelesssure, gimme a few and I'll ping you20:09
rick_h_ty20:09
=== matsubara is now known as matsubara-afk
lifelessrick_h_: ok20:27
rick_h_lifeless: tech of choice?20:27
lifelessone that works? skype? hangouts? you know, anything proprietary20:28
rick_h_hah, gotcha20:28
abentleygary_poster: adeuring and I are working on a new lazr package: lp:lazr.jobrunner.  I am having trouble getting it working in developer mode: http://pastebin.ubuntu.com/854512/  Do you have any advice?20:44
gary_posterabentley, on call, will look soon20:45
abentleygary_poster: thanks.20:45
lifelessabentley: is it lazr.runjobs or lazr.jobrunner ?20:53
abentleylifeless: Doh!20:54
lifelessabentley: I don't know if you and adeuring have seen https://dev.launchpad.net/CreatingNewProjects, but https://launchpad.net/lazr.jobrunner suggests you haven't21:00
lifelessabentley: could you run through that checklist and update the project and metadata and so forth? thanks!21:00
abentleylifeless: sure thing.21:01
lifelessabentley: (I assume buildout worked for you now ?)21:02
abentleylifeless: Yes, thanks.21:02
lifelesscool21:02
lifelessrick_h_: btw have you had a chance to play with pybars?21:39
gary_posterabentley, heh, I had just reviewed and was going to ask the same name question.  Glad it's working.21:42
abentleylifeless: btw, going with handlebars/pybars makes a lot of sense to me.21:43
abentleylifeless: esp. given that handlebars is an expected YUI 3 technology.21:44
lifelessabentley: thanks22:03
lifelessabentley: http://blog.fastmail.fm/2012/02/20/building-the-new-ajax-mail-ui-part-2-better-than-templates-building-highly-dynamic-web-pages/ - seems to me that handlebars could compile down to DOM operations just as well22:12
lifelesswhich would be nice22:12
sinzuiwallyworld_, We should update this page with your recent experience https://dev.launchpad.net/API/ImplementingAPIs22:12
lifelesssinzui: wallyworld_: when is your standup?22:13
StevenKNow22:13
sinzuiright now22:13
lifelessI would have joined yesterday but it was a trauma22:13
StevenKYou're welcome to join22:13
sinzuiwe are in mumble in purple22:13
abentleylifeless: It could, and that could be an advantage for big loops.  Handling unescaped might be a bit of a challenge, though.22:19
sinzuiwallyworld_,  here are the cluster of bugs. I think they overlap and maybe some are duplicates, Bug #590705, Bug #615244, Bug #723753, Bug #72395923:12
_mup_Bug #590705: New Build api exposed via 'beta' rather than 'devel' <api> <lp-soyuz> <soyuz-build> <Launchpad itself:Triaged> < https://launchpad.net/bugs/590705 >23:12
_mup_Bug #615244: syntax for declaring *when* an API is exported <lp-foundations> <Launchpad itself:Triaged> <lazr.restful:Triaged> < https://launchpad.net/bugs/615244 >23:12
_mup_Bug #723753: defaulting to exporting new web service methods on all web service versions makes web service mistakes dangerous <lazr.restful:Triaged> < https://launchpad.net/bugs/723753 >23:12
_mup_Bug #723959: Separate the floating "latest" version from the latest version-in-progress <lazr.restful:Incomplete> < https://launchpad.net/bugs/723959 >23:12
wallyworld_sinzui: thanks23:12
wallyworld_hmmm. none of those appear to exactly describe the issue.23:22
* wallyworld_ considers filing a new bug23:25
wallyworld_bug 93991023:32
_mup_Bug #939910: Need to export entry in version "beta" but it's only needed for "devel"  <lazr.restful:Triaged> < https://launchpad.net/bugs/939910 >23:32

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!