[00:00] Oh. [00:01] I see why this happens now. [00:01] There's a proper fix, but this reversion is probably better. [00:01] (for now) [00:01] why? [00:02] wallyworld: ^^ [00:02] wgrant: ? [00:03] wallyworld: I (and everyone else, it seems) was misunderstanding the code. write_htpasswd is given the name of a temporary file which was just created, with mode 600. [00:03] Not the old .htaccess file. [00:03] So this makes sense. [00:04] I now understand why that patch works. [00:05] no solution to my problem? [00:06] thumper: i need that cowboy eyeballed asap. see mp ^^^^^^^^^ [00:06] thumper: only one losa left and he is waiting [00:06] weather15: What happens if you try to retrieve testopenid.dev from the server? [00:06] wallyworld: The MP looks empty to me :/ [00:06] wallyworld: looking [00:06] Ah, there. [00:06] wgrant: fixed. i was being dumb [00:07] Service Temporarily Unavailable [00:07] weather15: You killed 'make run'? [00:07] wallyworld: you aren't really reverting the fix are you? [00:07] wgrant see this: http://pastebin.ubuntu.com/533655/ [00:07] thumper: no. but that effectively what it does [00:07] it a new branch [00:07] bad comments i guess [00:08] ah... [00:08] Preferably don't merge that fix, but please cowboy it. [00:08] what is really changing here? [00:08] it will be cleaned up. i justed wanted to get the cowboy done [00:08] wgrant: doesn't open(..., 'w') overwrite? [00:08] thumper: Yeah, but it doesn't set the mode. [00:08] thumper: see pastebin http://paste.ubuntu.com/533638/ [00:09] wgrant: so really what is the difference for the cowboy? [00:09] thumper: What do you mean? [00:10] thumper: The problem is that write_htpasswd is being called on a tempfile. [00:10] The tempfile is 600. [00:10] wallyworld: pong [00:10] thumper: open(..., 'w') truncates, but does not set the mode to the 644 that we require. [00:10] thumper: This is unobvious, and apparently a drive-by cleanup a week or so ago. [00:12] abentley: hi, just was looking for some eyeballs to review a cowboy. see mp a few lines above ^^^^^^ thumper is onto it [00:12] if the file doesn't exist at all (in the case of a newly privatized PPA) is the file going to be initially created with the correct permissions? [00:13] wgrant what can I do? [00:13] mbarnett: That's irrelevant. It creates a new tempfile, writes to that, then renames it over the top of the old one, if any. [00:13] (to attempt to be atomic) [00:14] wgrant: I just don't see why this is any different [00:14] wgrant: how confident are you that this works? [00:15] thumper: It deletes the file and creates it again. [00:15] Umask is 0022 [00:15] and the existing file is 600 why? [00:15] I've tested it locally, and it's just the same as the old code which worked. [00:15] thumper: Because that's what mkstemp does. [00:15] mbarnett: do the cowboy plz [00:15] Yay. [00:16] thumper: thanks [00:16] wgrant: why is the existing filename mkstemp? [00:16] wgrant: I'm just trying to understand [00:16] thumper: did we want the cowboy to become the accepted fix [00:16] wallyworld: probably not [00:16] wallyworld: in fact, it's something that looked wrong when I reviewed the code, and I said so at the time. [00:17] wallyworld: I think it should be looked at as it isn't obviously wrong [00:17] thumper: generate-ppa-htaccess creates a new temp file (using mkstemp), calls write_htpasswd on it, then renames it to .htpasswd. [00:17] thumper: thanks. [00:17] thumper: abentley: np. who should i assign the bug to? [00:17] I can't see the original MP (it's a private branch), but it looks like something which could have been fixed as part of a review. [00:18] wallyworld: don't assign it to anyone [00:18] patched [00:18] wallyworld: but pass it on to soyuz [00:18] mbarnett: thanks! [00:18] now we need to update a subscirption and make sure it worked! [00:18] mbarnett: Have you fixed existing file modes? [00:18] thumper: ack [00:18] wgrant: would it be better to look at the mode of the file rather than depend on the umask? [00:18] thumper: Of course. [00:18] And it should be tested. [00:18] wgrant: yeah, all were set properly [00:19] But I lack time to do that right now. [00:19] wgrant: re-running my find to make sure no more snuck in [00:19] Traceback (most recent call last): File "bin/killservice", line 23, in import lp.scripts.utilities.killservice File "/home/weather15/launchpad/lp-branches/devel/lib/lp/scripts/utilities/killservice.py", line 25, in from canonical.lazr.pidfile import ( ImportError: No module named pidfile make: *** [initscript-stop] Error 1 [00:19] ??? [00:21] wgrant: Yeah, it was considered a security vulnerability that we were publishing private PPAs before setting up their auth. [00:21] abentley: I know the bug and fix, but I can't see the comments in the MP. [00:22] I was wondering if the problematic change was a review fix. [00:23] no errors in the logs on the subsequent ppa generation runs. gonna change a subscription now and make sure it is happy [00:26] wgrant: it was. [00:26] Hah. [00:28] wgrant: rev 11842.1.12 of stable. [00:30] ... why did I not think to check the revision history. [00:36] and the build farm seems to still be alive. [00:49] Is PQM untestfixed at the moment? [00:52] who knows? [01:21] seiflotfy: hey [01:21] hi poolie [01:21] hi all [01:21] seif is interested in feeding launchpad actions into zeitgeist [01:21] so [01:21] which i think would give an incredible experience [01:21] "that bug i touched the other day" [01:21] Using Zeitgeist as a backend in Launchpad will allow users to have a personal timeline of their activities regarding themselves or specific project. This will allow easier tracker of when things were done commited and view a chronicle of the project in terms of all (blueprints/bugs/etc..) mashed together. [01:21] To do so Zeitgeist will just need 1 minor change which is use a different SQL DB to allow multiple read and write. [01:21] seif i just saw this general topic of zeitgeist-like-things was mentioned in the Economist last week [01:22] really [01:22] where? [01:22] iiiiiiiiiiiiiiiiithe magazine [01:22] sorr [01:22] sorry typo [01:22] sor [01:22] ok [01:22] fixed the keyboard [01:22] sorry [01:22] ok [01:22] ........... [01:22] :) [01:23] seiflotfy, what's the db backend of zeitgeist? Can it be replicated? [01:23] sqlite [01:23] james_w, we have an extension that forwards zeitgeist stuff to couchdb [01:23] seiflotfy: http://www.economist.com/node/17388382?story_id=17388382 [01:24] james_w, its easy to exchange the backend [01:24] the trick is in what shoudl be stored [01:24] seiflotfy, so LP could write a different backend to use a more appropriate technology for it? [01:25] yes [01:25] i was thinking of postgre sql [01:25] ok, what would zeitgeist provide in that situation, over some hacked up timeline view? [01:25] james_w, i dont understand you question [01:26] seiflotfy, just trying to understand what benefit using zeitgeist itself would have [01:26] hello [01:26] yeah [01:26] I agree that the functionality would be good for LP, but I'm unsure whether the code itself makes sense [01:26] james_w, i dont understand the question though [01:26] if you're talking about having zeitgeist directly access the launchpad pgsq, that's probably not feasible [01:26] no no no [01:26] are you talking about having zeitgeist run on the client, or also having it run in the datacentre? [01:27] poolie, it shoudl run on the datacenter with its own DB [01:28] james_w, the code is stable [01:28] seiflotfy, my concern would be that zeitgeist was designed for activity tracking on desktops. Using it in a webapp is quite different. I want to separate the concept from the implementation. [01:28] and being used by canonical already for unity [01:28] james_w, not really [01:28] how about having lp emit data into couch, in the dc, and then having zeitgeist pull some per-user parts of that? [01:28] we developed it for using it with webapps too [01:29] james_w, i can track twitter stuff easily [01:29] james_w, poolie , zeitgeist supports exchangable backends per default :) [01:30] so the DB where the data is stored is not a problem [01:30] its about how it is stored [01:30] james_w, I already started mapping out how user activities on launchpad will be logged [01:30] and tbh its pretty easy [01:30] I'd suggest that you send it to the mailing list, you'll reach a larger fraction of LP developers that way [01:32] james_w, sure thing [01:32] http://pastebin.ca/1995138 [01:33] poolie, this is how a bug report would look like [01:33] i can save this as it is in zeitgeist [01:33] this will allow me to query a timeline for user/project/team/bug [01:35] ok.. [01:35] spiv and i might get some lunch, biab [01:35] agree with james, sending it to the list too could be good [01:35] seiflotfy, what about privacy? [01:36] james_w, in which sense [01:37] james_w, its easy to set the privacy issues of who gets to see what as a layer between zeitgeist and launchpad [01:37] its a simple logic [01:37] right, but you need to account for that [01:37] bugs can be private [01:37] depends on how the team is set up [01:38] thumper: could you plz stamp this so i can progress it https://code.edge.launchpad.net/~wallyworld/launchpad/person-mergequeue-listview/+merge/41122 [01:38] james_w, the logic part for privacy is an issue that should be taken seriously and I agree on that, but its easier than oyu think [01:38] james_w, can u give me the launchpad list [01:41] seiflotfy, I'm not saying it's hard, I'm saying it's a requirement for LP, so if your proposal doesn't mention it then it will be the first objection that people make [01:41] james_w, taken into consideration [01:41] james_w, the problem i dont have it all thought out yet [01:41] i would like to do it with the launchpad team [01:41] think it through with u guys [01:42] https://launchpad.net/~launchpad-dev [01:45] james_w, join the team or write to launchpad-dev@lists.launchpad.net ? [01:46] I think you have to do the former to do the latter [01:50] ok [01:50] i am applying to join then [02:09] === Top 10 Time Out Counts by Page ID === [02:09] Hard / Soft Page ID [02:09] 198 / 62 Person:+commentedbugs [02:09] 138 / 5176 Archive:+index [02:09] 78 / 267 BugTask:+index [02:09] 44 / 308 Distribution:+bugtarget-portlet-bugfilters-stats [02:10] 36 / 1 Bug:EntryResource:newMessage [02:10] 28 / 267 Distribution:+bugs [02:10] 26 / 53 Archive:+copy-packages [02:10] 21 / 135 POFile:+translate [02:10] 17 / 24 DistroArchSeries:+index [02:10] 15 / 10 ProjectGroup:+milestones [02:10] Yay, we aren't winning any more :D [02:11] sum the two figures ;) [02:13] :( [02:28] seiflotfy: yeah, privacy seems to be the hard thing [02:28] wallyworld: Pong [02:28] poolie, not at all [02:29] i think it can be done as a logic between launchpad and the server-zeitgeist [02:29] excellent [02:29] poolie, its pretty easy [02:29] StevenK: hi, no matter now. there was an cherry pick i wanted to get eyeballed. all sorted, thanks [02:29] poolie, i am sending out an email to the launchpad and zeitgeist lists [02:29] wallyworld: I'm not working this week in any case :-) [02:29] this would get us into good discussion :) [02:29] StevenK: lucky bastard. you on leave? [02:30] wallyworld: Yup, until Monday [02:30] StevenK: well get off irc then and enjoy your holiday :-) [02:37] poolie, i sent out an email to the dev list :) [02:39] poolie, i am off to bed again [02:39] its 3:40 am here [02:39] ok [02:39] sleep! :) [02:39] n8 === seiflotfy is now known as seiflotfy|sleepi [02:41] wallyworld: can you have a look at bug 674581 [02:41] <_mup_> Bug #674581: Sourceforge Mercurial Import Fails [02:42] and say whether it's a dupe or a known limitation? [02:42] poolie: ok, i'll look [02:44] poolie: not sure - i'm a bit uneducated as yet on this stuff. perhaps thumper could comment? ^^^^ [02:44] ah... whut? [02:45] thumper: could you take a quick look at poolie's bug above. i can dig further. i thought you may just look at it and know what was happening based on past experience [02:46] wallyworld: most hg imports fail [02:46] around 80% [02:46] bzr-hg is not really going anywhere [02:46] ok. so it's a limitation in our importer [02:46] I'm looking at the imports [02:50] wallyworld: the only thing I could suggest is to try the import locally [02:51] thumper: thanks. poolie see thumper's comment ^^^^^^ [02:51] it's failing to connect to the server [02:52] poolie: I see that, have you tried it locally? [02:52] i can connect tothe url [02:52] i didn't actually try bzr-hg [03:02] * thumper tries === Ursinha-bbl is now known as Ursinha-afk [03:07] poolie: bzr-hg fails miserably with that import [03:07] I've commented on the bug [03:07] I think that the server is looking at the client type [03:07] hg client works fine [03:07] bzr gets the http redirect [03:08] and so fails on the unknown bzrdir format [03:08] poolie: I added a bzr-hg task [03:10] i see === timrc is now known as timrc_zzz [04:30] anyone seen 'Module canonical.launchpad.webapp.login, line 440, in expireSessionCookie [04:30] value = request.response.getCookie(session_cookiename)['value'] [04:30] on launchpad.dev logging in ? [04:30] TypeError: 'NoneType' object is unsubscriptable
' [04:34] No. [04:34] I blame Cassandra! [04:38] hah [04:38] so, did you gain much from my notes? [04:41] Mm. It looks good for some stuff (like the session bits you mentioned). But I need to look into it more deeply to see how it could feasibly replace much more than that. [04:50] * thumper EODs [04:58] sinzui: ping [05:00] wgrant: its what, 4pm for you ? [05:00] lifeless: 4pm, yeah. [05:00] hmm, actually, I can just look at my mail in the morning. [05:00] I have a fix for Person:+commentedbugs [05:01] Its a one-word fix. [05:01] Hah. [05:01] https://code.launchpad.net/~lifeless/launchpad/malone/+merge/41127 === almaisan-away is now known as al-maisan [08:32] good morning === adeuring1 is now known as adeuring [08:46] danilos: Hi! [08:50] wallyworld_, wgrant: Thanks for dealing with the permissions issue crisis! I'm looking at a proper fix now. [08:51] jelmer: no problem at all. i updated the incident report with a few extra details which i also included in the email [08:53] wallyworld_: Thanks. === didrocks1 is now known as didrocks [09:17] hello [09:37] jelmer: Great. [09:37] jelmer: An unobvious bit of code, that one :/ [09:37] jml: Thanks for landing that. [09:37] wgrant: my fix seems to have worked [09:38] bigjools: Yup, looks OK today. [09:38] no, to work out wtf they take so long to respond [09:38] bigjools: The "fix" being the removal of failure count assessments? [09:38] now* [09:38] Or increasing timeouts? [09:38] yes [09:38] nothing would have got failed if the assessment method had been left [09:38] Huh. [09:39] Interesting. [09:39] * bigjools will land the fix properly today [09:39] So you increased the TCP timeouts across the board? [09:40] all xmlrpc connections [09:40] Hmm. [09:41] that's the *connection* timeout [09:41] Can we grab logs from palmer and co. and see what was happening around the time of the timeouts? [09:41] Also, do we have resource graphs from the non-virt builders? I'm guessing not. [09:41] yeah I'll sort that out [09:41] :( [09:41] we do [09:41] ! [09:41] Yay. [09:41] There may be hope yet. [09:41] as in like the one I showed you for the virtual ? [09:42] No, I mean load and memory usage and stuff. [09:42] oh that [09:42] ummm [09:42] I'll check [09:42] do you know which ones (apart from palmer) to save me looking them up? :) [09:42] All of them, I believe it was. [09:43] One or two may have survived. But I don't recall which. [09:58] wgrant: it did it automatically [09:59] jml: Huh? [09:59] wgrant: I think there was some kind of mail queue delay [10:00] Ah. [10:00] wgrant: because another branch that I had forgotten about also landed. [10:00] Heh. [10:00] stub: any idea why our makefile has "schema" depending on "clean_codehosting"? [10:01] No idea [10:01] Having it depend on build definitely looks like overkill—we don't _really_ need new WADL to build a new DB, do we? [10:02] Or should I say 'because that Makefile has become utterly unmaintainable' [10:02] Too late—I'm hacking & slashing at it. [10:02] I don't know when we need WADL. [10:02] jtv: make schema will be deleting branches, so it probably makes sense to clean up their directories too... [10:03] wgrant: make schema deletes branches? That's surprising. [10:03] jtv: It resets your DB. [10:03] So... [10:04] wgrant: I'm waiting with bated breath for you to finish that sentence! [10:04] jtv: Is Codehosting going to be wonderfully happy if it tries to create a branch and there's already something there? [10:05] wgrant: I still fail to see the connection between that and the DB schema. [10:05] jtv: 'make schema' will remove all your extra branches. If you create a new one, it's going to use an ID that's already on disk. [10:06] wgrant: nope, I'm still not getting it. What are my "extra branches"? [10:06] jtv: Any branches you create on top of the sampledata. [10:06] Oh, you're saying it needs to reset the local codehosting state in conjunction with the schema. [10:06] Which are probably the same as those that are in the directories that clean_codehosting deletes. [10:07] (Why not the local librarian state though, I have to wonder) [10:07] Yes, schema also depends on clean_codehosting. [10:07] Hmm. clean seems to remove just about everything except the dev librarian. [10:08] Oh. [10:08] schema removes the dev librarian. [10:14] ISTM schema may only need to depend on $(PY) and clean_codehosting. [10:14] Not even compile? [10:15] Looks like you may be right. [10:16] I can't see compile doing anything more fundamental than UI-level work. [10:17] Or wait, no, compile also did mailman. [10:17] (Which it could do perhaps 10 seconds faster with a judicious -j option, but the change I have in PQM right now should make that sort of pointless) [10:18] But I imagine mailman is not necessary for what we want from "make schema." [10:18] I would hope not. [10:18] Unless something imports it. [10:19] Which ew. [10:19] We'll see, when my current branch clears PQM. [10:19] What does it do? [10:19] Not break when you "make -j2" :-) [10:19] Hah. [10:20] It's a pretty even break between "generate WADL" and "do everything else," so gratifyingly close to a 2× speedup. [10:20] benji & I got worked up about build speeds last night. He may have a way to skip the half a million optparse "add_option" calls from the WADL generation. [10:20] Unexpected? It was to me. [10:21] In a 4-minute profiling run of wadl generation, add_option accounted for about one minute. [10:21] wow [10:21] (Profiling added less than 100% runtime overhead for me, so estimate the gains at 30s) [10:21] jtv: Oh, last time I tried to profile the WADL generation it didn't finish... [10:22] See? Things aren't so bad these days! :-) [10:22] Heh. [10:23] It could be worth doing separate wadl generation runs for separate API versions, though single-cores shouldn't sniff at the repeated zcml overhead either. [10:24] Ironically, "make" would be just the tool for limiting that job to only the files that need updating. But the finer details are hidden away in a python script. [10:24] Yes :( [10:45] That was quite a mail backlog. [11:47] jml: did you change something recently in testtools? I am getting errors about "can't import StartsWith" and I've dist-upgraded and updated-sourcecode... === matsubara-afk is now known as matsubara [11:48] And updated download-cache and remade? [11:48] I've never had to do that before [11:48] And the dozens of other dependency mechanisms? :D [11:48] that's the nice thing about standards, there's so many to choose from [11:48] But really, testtools is an egg. Update download-cache and run make. [11:49] utilities/update-sourcecode does all that [11:49] Doesn't that just update sourcecode? [11:49] rocketfuel-get is the one that does it all. [11:49] it just calls the latter [11:49] I think you have it the wrong way around. [11:49] last time I looked, anyway [11:49] rocketfuel-get calls update-sourcecode. [11:49] that's what I said [11:49] update-sourcecode pulls everything in sourcecode/ [11:56] make clean seems to have fixed it [11:56] the Windows solution [11:56] Oh. [11:57] Morning, all. [11:57] wotcha deryck [11:57] bigjools: There was an email about that a week ago. [11:57] An egg was changed. [11:57] ahhhhhhh I remember now [11:58] I fixed it on one box but not my other [11:58] * bigjools goes back to tracking down PoolFileOverwriteErrors [11:58] :( [11:58] Copies? [11:59] trying to work that out but I 99% suspect so [11:59] :( [12:00] wgrant: https://launchpad.net/~rhpot1991/+archive/ppa/+packages [12:00] see mythbuntu-lirc-generator [12:01] I wonder if it's down to .debs moving between sources [12:05] bigjools: Let's see.. [12:05] That doesn't sound likely. [12:05] *sigh* why do we put up with this internet thing? [12:05] But this is Soyuz. [12:06] wgrant: the files were put in the librarian a few hours apart [12:06] currently hunting down where they are published [12:06] bigjools: I am confuse. [12:07] bigjools: That file isn't in any of the other sources in the PPA. [12:07] 2 files with the same name [12:07] yep [12:09] bigjools: Something is really screwed. [12:09] There was never a binary by that name published. [12:09] Of any version. [12:09] On any arch. [12:09] So WTF is it doing on disk? [12:10] It's not a half-written file? [12:11] no idea [12:11]