/srv/irclogs.ubuntu.com/2011/08/14/#launchpad-dev.txt

nigelbI still can't figure out why I started hacking on LP before I read about Zope 3.02:33
nigelbFinally I'm reading Zope 3 intros :P02:33
* StevenK purges 3GiB of postgres logs so that /var actually has some free space.04:53
* nigelb twitches at GiB04:57
StevenKnigelb: You don't like {K,M,G,T}iB?05:00
StevenKCan't remember what comes after T. E?05:00
lifelessEca05:21
lifelessbah05:21
lifelessExa05:21
StevenKRight, so it is E05:22
lifelessor is it P05:22
StevenKThen, P, then Z, and then OMGBBQWTFBIG05:22
lifelessit might be P E05:22
lifelessah E P05:22
lifelesswgrant: we don't build-in a structure of partitioned namespaces (partly because teams have no project affiliation etc05:24
lifelesswgrant: so folk need to remember05:24
lifelesswgrant: however, this one has me confused too ;P05:32
nigelbStevenK: My dislike is for the 'i'06:20
lifelessnigelb: you're a no i kinda guy ?06:21
nigelblifeless: Bwahaha. No.06:22
nigelbSo, I noticed something weird today.06:26
nigelbPotentialy a mintor security thing.06:26
nigelbWhen I go to https://launchpad.net/~canonical-sysadmins, I see a <hidden> under "Subteam of"06:27
nigelbIsn't our policy to to show "its not even there" when something isn't supposed to be seen?06:27
nigelbs/show/do'g06:28
nigelbs/show/do/g06:28
lifelessthats an adapter kicking in and preventing disclosure06:35
lifelessyes, it shouldn't show at all, but its a cosmetic, not a security, concern06:35
lifelessplease do file a bug06:35
nigelbcool, will do06:36
wgrantlifeless: It is actually a security concern, slightly.06:49
wgrantI filed it a couple of years ago.06:49
wgrantIt's in alphabetical order.06:49
wgrantSo you can determine the display name.06:49
nigelbwgrant: there's already a bug?06:50
wgrantI think so. Trying to find it...06:50
lifelesswgrant: you can guess at possible display names ;)06:51
wgrantlifeless: Then you can create teams on staging to do a search.06:51
nigelbwgrant: bug 32913206:51
nigelb?06:51
_mup_Bug #329132: Strange string under "Subteam of" in LOSAs' team page <confusing-ui> <lp-registry> <registry-people> <Launchpad itself:Fix Released by bac> < https://launchpad.net/bugs/329132 >06:51
wgrantHah06:52
wgrantWhat a fix.06:52
wgrantI thought I filed a similar one, though...06:52
* nigelb stabs Zope 306:53
nigelbTrying to get it to work. Refuses to work.06:54
wgrantWhat are you doing to the poor creature?06:54
nigelbUsing Benji's quick start guide06:54
wgrantnigelb: I guess you should file a new one rather than reopen the old one.06:54
nigelbwgrant: Filing now06:55
wgrantnigelb: You may have some issues with using that.06:55
nigelbOh.06:55
wgrantnigelb: Since Zope 3 doesn't exist any more.06:55
nigelbwtf.06:55
wgrantIt's... complicated.06:55
wgrantMost of the underlying libraries now make up what is known as the Zope Toolkit, or ZTK.06:55
wgrantThe UI (ZMI and other stuff that's mentioned there) is not part of the ZTK, and was abandoned for a time.06:56
wgrantBut then was made into its own project atop the ZTK, BlueBream.06:56
nigelbAll I wanted was to get a hands on feel for zope minus the complications of Launchpad so I can figure out Launchpad faster.06:56
wgrantNot sure if that is still active.06:56
wgranthttp://bluebream.zope.org/screencasts.html may be helpful.06:56
wgranthttp://bluebream.zope.org/doc/1.0/index.html too06:57
wgrantLooks like the docs don't make me want to die any more. Which is a good start.06:57
nigelbOh, cool!06:57
nigelbThank you06:57
nigelbIs that closer to what LP is using?06:57
wgrantLP doesn't use BlueBream, but it uses many components of the ZTK.06:58
wgrantNot all the latest versions, but mostly within the last 18 months.06:58
wgrantIt also uses some stuff that BlueBream uses but that is outside the ZTK.06:58
nigelbOk, so it should give me a good basic idea.06:58
wgrantYup.06:58
nigelbRight now, I'm coding on LP with extensing grepping and asking here.06:59
nigelbAnd grepping my logs of asking here.06:59
wgrantPreviously I would have suggested that you obtain, by some means, a copy of Web Component Development with Zope 3. But it was already obsolete when I started, and it's going to not be practically usable now.06:59
wgrantGood in-depth teaching of the concepts, though.07:00
wgrantMost LP engineers do what you do, I think :)07:00
wgrantThey learn Zope 3 by drowning in LP.07:00
nigelbHah07:00
wgrant"It is recommended to use a custom-built Python for working with BlueBream."07:01
wgrantGaaaaah stupid Python people.07:01
nigelbheh07:01
wgrant(if following the bluebream getting started guide, I would use virtualenv to install bluebream without polluting your system)07:02
nigelbI always use virtualenv for this kinda stuff07:03
lifelesswgrant: ah, nifty07:03
wgrantFrom Wikipedia: "BlueBream is in active development and is now considered a stable framework, used on production projects worldwide, most notably Launchpad."07:03
nigelbWHAT?07:04
wgrantBut we don't use the whole thing :(07:04
wgrantMaybe they've altered the scope of BlueBream to be everything that was in Zope 3 that is not in the ZTK.07:04
wgrantLast I heard it was primarily the ZMI.07:04
lifelessnigelb: my preferred solution for 'LP is hard to understand' is to clean things up07:04
lifelessnigelb: e.g. by deleting lots o fcode07:05
nigelblifeless: My problem is my web development experience is in Django. I'm trying to figure out how is the sort of right way to to do things in zope.07:05
lifelessnigelb: ah07:05
nigelbIts probably a day or two of reading up07:05
lifelessnigelb: so do you want the good news or the bad nes ?07:05
* StevenK deleted something like 2,800 lines of code yesterday07:05
nigelbI figured its time I invested that07:05
nigelblifeless: I'll take the bad news first07:05
lifelessnigelb: the right way of doing things is the wrong way for LP07:05
nigelbExpected.07:06
nigelbYay!07:06
lifelesssorry, that was mistyped. But hilarious.07:06
lifelessnigelb: the right way of doing things in zope is the wrong way for LP07:06
nigelbStevenK: When I meet at some point, I'll buy you a drink for that.07:06
lifelessis what I meant to say.07:06
nigelbSmall difference :)07:06
nigelbStevenK: *meet you07:06
lifelesstwo words, but significant impact :>07:07
wgrantlifeless: Well, everything except persistence is not too bad.07:07
lifelesswgrant: except that *everything* is driven by persistence.07:07
lifelesswgrant: also you forgot configuration.07:07
nigelbWell, so I'm probably looking at bare basic concepts. Like tal thingy or zcml.07:07
nigelbBoth of these I still don't really understand, but I can read and guess07:08
wgrantlifeless: Except ZCA and views and TAL and ZCML and the general architecture.07:08
lifelesswgrant: tal and views are impacted by persistence; zcml is because of the model:interfaces interactions.07:08
wgrantOnly slightly.07:09
wgrantlifeless: Also, do we want to roll back that spaces in URLs thing?07:09
lifelesswgrant: zca also interacts with persistence in the same way, fwiw07:09
lifelesswgrant: yes, we do.07:09
wgrantShouldn't have to roll back production, since we're almost deployable.07:10
wgrantAlso there's also an additional 1000 Loggerhead OOPSes that we might care about.07:10
lifelesswgrant: are you still on leave ?07:10
wgrantYes.07:10
lifelessheh07:10
lifelesswgrant: I was going to ask if you felt like doing the rollback07:10
nigelbIf wgrant is *this* productive when he's on leave...07:10
wgrantqemu-kvm's broken gdbstub is broken enough that I will look at work, but not do it :)07:10
nigelblifeless: I saw the wiki LEP. Has it gone stale or is it still alive?07:11
lifelessnigelb: its confused07:11
nigelbBecause I'd like to propose extracting stuff from readthedocs and bring it into LP.07:11
StevenKlifeless: Oh, the loggerhead rollback?07:11
lifelessStevenK: not loggerhead, url ascii rules07:11
wgrantloggerhead's broken too, but I think that might be two rollouts ago.07:12
wgrantNot quite sure.07:12
lifeless993 RuntimeError: dictionary changed size during iteration ?07:12
lifelessprobably bzrlib07:12
wgrantWe didn't upgrade bzrlib, did we?07:13
wgrantStill 2.3.307:13
wgrantWe did upgrade loggerhead.07:13
lifelesswe need a bug at the least07:15
lifelessand probably a rollback of loggerhead too07:15
* wgrant curses lifeless a bit.07:18
lifelesswgrant: ?07:18
wgrantThe only thing left writing to BugTask's target key attributes directly is the bugsummary tests.07:18
wgrantremoveSecurityProxy ftw.07:19
lifelesswgrant: hah. I didn't write those :P07:19
wgrantGUILTY07:19
wgrantWe might be able to put NULL Project to sleep on Monday...07:20
StevenKOh, nice07:21
StevenKwgrant: Can I land populate-bprc now?07:21
lifelessStevenK: why wouldn't you ?07:22
wgrantWell, it almost has a veto.07:22
wgrant(not from me!)07:22
StevenKlifeless: From a certain GSA who won't be named saying it sounded like an epicly bad idea07:22
lifelesshow so ?07:23
lifelesswhat was the concern ?07:23
lifelesshttps://bugs.launchpad.net/loggerhead/+bug/82613607:23
_mup_Bug #826136: dictionary changed size during iteration <oops> <loggerhead:Triaged> < https://launchpad.net/bugs/826136 >07:23
StevenKlifeless: That the data does not and should not go into the main DB07:24
lifelessStevenK: thats not a concern07:24
lifelessStevenK: thats a conclusion07:24
lifeless:)07:24
StevenKlifeless: After wgrant and I talked about it, we decided to wait for a bit, since if we kick off the migration, it's quite difficult to migrate the data back out again07:25
lifelessStevenK: I don't follow.07:25
wgrantIt's slightly awkward.07:25
wgrantNot quite difficult :)07:25
StevenKlifeless: It's a large amount of data that will land into launchpad_main -- it will sit like a red wine stain. Once it's there, it will require persistance to move07:27
lifelessStevenK: so the concern is size?07:28
lifelessStevenK: why isn't deleting it as simple as a loop doing delete LIMIT 50 ?07:29
StevenKlifeless: No. My small concern is migration pain07:29
StevenKMy other concern is I'm not clever enough to implement step 3 and 4 of the grand plan07:29
lifelessthey being?07:30
StevenK3. Hook into the publisher to create and delete rows in BPRC and BPP when it publishes or superseded a BPR.07:30
StevenK4. Hook into the publisher to generate contents file, and turn off the current contents generation script.07:31
lifeless3 seems bogus07:31
wgrantBoth are.07:31
StevenKEh?07:31
wgrantNeither should be in the publisher, I don't think.07:31
wgrantBPRC lifetime is the same as file lifetime.07:32
lifeless4 could be in the publisher, or separate07:32
wgrantSo they should certainly not be removed by the publisher.07:32
wgrant4 is possibly OK, but not required, and there's already enough in the publisher IMO :)07:32
lifeless3 is bogus because there can be N published entries, and trying to add just-in-time or delete just-after-use will be massively racy.07:32
StevenKThe garbo job should certainly keep up with new BPRs, but I'm concerned about bloat in BPRC/BPP07:34
wgrantThe BPF expiry script will remove them.07:34
StevenKwgrant: Ah, that reminds me.07:34
StevenKwgrant: The populate-bprc garbo job returns BPRs that do not have any BPFs. Do not want.07:35
wgrantIndeed, although in production there shouldn't be any.07:35
StevenKwgrant: Hah, maybe.07:36
lifelessStevenK: so here is a question folk might like to answer07:36
StevenKlifeless, wgrant: So, land the garbo population job and then think about the next step while it populates?07:36
lifeless'when was /usr/bin/foo introducted to the archive'07:36
lifelessStevenK: we might want to gc things for space considerations07:37
lifelessbut having enough data to ssatisfy the use case is the first step07:37
StevenKWe certainly do want to GC things07:37
lifelessStevenK: what was the gsa's concern ?07:37
StevenKlifeless: That it doesn't belong in the main DB07:37
lifelessStevenK: You can fit *every* file in *every* arch in *every* published deb, ever, in < 5GB07:37
lifeless[excluding PPAs, but they are (for now) still minimal)07:38
StevenKlifeless: Modulo bugs :-(07:38
lifelessStevenK: yes, but having extra rows would be fairly obvious07:42
lifelessStevenK: I don't think gc is a given.07:42
* StevenK fixes the conflicts the branch has with devel07:44
lifelessrev 13687 is the rollback08:17
StevenKlifeless: We don't have much QA to do, wallyworld_, bigjools, 2 * danilo08:18
StevenKWe might be able to deploy on Monday afternoon UK time08:18
lifelessnot morning ?08:26
StevenKlifeless: Depends on how quick the QA is done -- I was being pessimistic.08:30
=== almaisan-away is now known as al-maisan
=== al-maisan is now known as almaisan-away
nigelbgah, qastaging is timing out, known?15:00
jelmernigelb, hey15:12
jelmernigelb, I don't think it is - perhaps it's just updating?15:12
nigelbjelmer: hm, maybe.16:05
nigelbI'll just try a while later :)16:05
lifelessnigelb: timing out on all pages or just one ?20:08
lifelessnigelb: some timeouts are (unfortunately) 'normal' for [qa]staging20:08
lifelessaieeeee23:30
lifelessIUnloggedException. Sob.23:30

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!