=== Ursinha is now known as Ursinha-afk [00:16] wgrant: it is running; but appears it is rather beautifully broken. possibly from the reroll [00:29] wgrant: ok. that looks to be working again. reverted back to prior the rollout and seeing stuff happen again. I call shenanigans [00:30] spm: As you might have noticed, I've been dropping in and out a bit. What's happened since I LOSA-pinged? [00:30] lala. I hadn't. [00:30] wgrant: it is running; but appears it is rather beautifully broken. possibly from the reroll [00:30] wgrant: ok. that looks to be working again. reverted back to prior the rollout and seeing stuff happen again. I call shenanigans [00:30] is all you may have missed [00:31] Would it be revealing too much if someone was able to tell me what is the lp servers run? [00:31] hardy right now [00:31] my 2c observation is a misconfig/code mess - looks like it's trying to talk to a local postgres instance, vs the db servers [00:31] or do you mean hardware? [00:32] Nope, hardy is a good answer [00:32] Are there plans to upgrade to lucid? [00:32] yes [00:32] I see [00:33] OK. [00:33] What's it dying with? [00:33] Argh. === wgrant_ is now known as wgrant [00:35] * wgrant crosses fingers, and asks again: How's it failing? [00:35] wgrant of the dropouts: psycopg2.OperationalError: could not connect to server: No such file or directory Is the server running locally and accepting connections on Unix domain socket.... [00:38] So, it seems that my entire Optus cable node drops out whenever I ask how it's broken :P [00:39] But that looks like a broken config. [00:40] * ajmitch would blame the censorship [00:42] spm: Ahem. So, nothing else is showing that sort of failure? [00:43] wgrant: not yet........ [00:44] spm: Because that doesn't look even slightly Soyuzy. [00:44] I refactored buildd-manager lots during 10.04, but nothing that low-level. [00:45] this seems to be a change between the roll; and the reroll; i've reverted to the version from the roll; and that's fine. so... the opportunitys for brokenness are reduced. [00:45] Why python over php or another language? [00:45] Oh, so it's actually OK now? [00:46] AdamDV-iPod: Um, well, PHP is... PHP. [00:46] and the configs are the same revno; so not config based. my 2c. [00:46] spm: Bah. I was going to accuse the prod configs. [00:47] :-) [00:49] Haha [00:51] spm: Has it died again, or is it just busy processing uploads? [00:51] it's not done much for a few minutes. [00:51] manually stopped; chasing stuff with jelmer [00:51] Whereas it should be doing lots every 5 seconds. [00:51] Ahh. [00:52] who isn't here btw. he is 'away'. says so in his irc client. [01:02] Launchpad depends on got-core. How ironic? [01:02] huh? [01:02] AdamDV-iPod: ah, you mean git-core? [01:03] it's for testing git imports [01:03] Yea. Stupid touch screen. [01:03] Oh [01:03] Lp supports git imports? [01:03] AdamDV-iPod: That's probably to test the git import functionality [01:04] The functionality never ends! [01:06] the kitchen-sink module is my personal favourite [01:07] Is there really one? [01:07] Subversion has a kitchen_sink.c :-) [01:08] maxb: I have no idea; but I wouldn't be terribly surprised :-) [01:12] AdamDV-iPod: we do everything except what people really want, like wikis and dashboards [01:12] * thumper tries not to be too bitter [01:13] must be coffee time [01:13] Haha [01:13] People want wikisband dashboards eg? [01:14] AdamDV-iPod: I don't grok what you just said [01:14] Err, eh. I hate the iKeyboard [01:14] Translation: People want wikis and dashboards now, ey? === Guest44380 is now known as AdamDV2 [01:15] Guest44380: how many ids do you have? [01:15] Nah, Nickserv hates me. [01:15] I have AdamDV and TuxIce [01:15] Pidgin randomly cuts out, and doesn't identify to nickserv. [01:15] luckily someone is working on wikis :) [01:16] well that sucks [01:16] Ooh nice :) [01:16] And yes, it does. However, its my only option. I loathe Empathy. [01:16] thumper: Ooh, I wasn't aware of that - who? [01:16] although since they aren't a priority, it is an idle / evening task [01:16] jelmer: me [01:16] :) [01:16] :D [01:17] jelmer: I should have something that will serve a bzr branch as a wiki soon [01:17] Rocketfuel is installing obsolete versions of postgresql, apparently. [01:17] jelmer: it is my evening fun [01:17] AdamDV2: yeah, not yet updated to 8.4 [01:17] thumper: that sounds really cool [01:17] AdamDV2: it does work with 8.4 [01:17] I see. [01:17] * thumper uses 8.4 [01:17] * AdamDV2 uses MySQL [01:18] * thumper needs food and coffee [01:18] Heh [01:18] thumper: It looks like Dulwich trunk itself works but the combination dulwuch+bzr-git is broken against http repos atm :-/ [01:18] :( [01:18] Presumably once we go lucid and 2.6 for real, there'll be somewhat more motivation to get on with the move to 8.4 [01:18] jelmer: which versions should I take then? [01:19] maxb: there is no reason not to use 8.4 now [01:19] thumper: the last version I know at least works is from 13 april [01:19] maxb: it was just that stub hadn't gotten around to it [01:19] jelmer: which versions are they? [01:19] jelmer: cause I really wanted 909 of bzr-git [01:19] jelmer: oh, and you commit *a lot* [01:20] I have *never* had apt-get take 1 hour, 15 minutes and going. [01:20] You guys aren't exaggerating when rocketfuel says "This will take awhile" [01:21] thumper: bzr-git 889, dulwich 562 [01:21] AdamDV2: you obviously don't upgrade to the beta OS early [01:21] I don't usually upgrade. [01:22] jelmer: [rs=jelmer] upgrade to lp:bzr-git 899 [01:22] thumper: alternatively I can look at fixing the issue with http, it shouldn't be too hard [01:22] jelmer: from lp's bzr-git branch [01:22] jelmer: if you can fix http, I'll upgrade bzr-git on LP asap [01:24] thumper: I don't really have time to do that now though, probably friday evening [01:24] jelmer: ok, np, lets look to upgrade early next week [01:24] jelmer: I can add it to my Monday tasks [01:24] thumper: are you going to be at UDS? [01:24] jelmer: nope [01:25] * thumper afk for food [01:25] thumper: This is where I am confused. Why are the things that would make LP less completely unattractive not priorities? [01:26] wgrant: best answer for that is "ask jml" :) [01:26] * thumper takes laptop to coffee ship [01:26] shop [01:27] Also, I should look at wikkid at some point. [01:27] * wgrant looks at wikkid. [03:23] wgrant: I've just pushed my work from the last few days into trunk [04:02] Something deeply weird is happening to the Debian package syncer: https://edge.launchpad.net/debian/+source/subversion/+publishinghistory [04:09] it's been happening for awhile, I think [04:10] StevenK has been looking at it [04:10] at least with regards to out-of-date stuff :) [04:12] the old version is due to debian being a bit strange, and having 2 different versions of source packages, see this in 'rmadison -u debian subversion' [04:19] oh, it's picked the version in hurd-i386 [04:20] ugh [04:20] ugly, isn't it? [04:21] Do you know where the bug might be found (I assume there is one) - registry? soyuz? [04:23] most likely related to bug 568745 which I filed [04:23] Bug #568745: Debian unstable record of gpt missing [04:23] * ajmitch should put in the info about duplicate info in the Sources.gz from debian [04:30] * AdamDV2 curses [04:51] ajmitch, maxb: But gina has been able to handle multiple versions for either a few weeks or a couple of days. [04:52] StevenK: That was one of your first branches, wasn't it? [04:52] wgrant: I think so [04:52] wgrant: that subversion +publishinghistory is showing the first version listed in Sources as being the latest [04:53] ajmitch: That suggests that it just hadn't picked it up before. [04:53] Probably because StevenK's change hadn't landed yet. [04:53] So when his change landed, it picked up both versions. But the real latest version was already there, so it wasn't recreated. [04:53] So only the old one was new, so it appears to be the latest. [04:54] that sounds reasonable [04:54] it just looks a little odd when https://edge.launchpad.net/debian/+source/subversion has that old version as the latest upload [04:54] Yes, that is slightly COMPLETELY INSANE. [04:54] Right. The bug is that gina is of the opinion that gpt 1.0.4-3 already exists in the database, so it doesn't import it [04:55] But it's right. [04:55] that's special [04:55] It's not a bug. [04:55] Oh, wait, different one. [04:55] wgrant: It does not exist for Debian sid. [04:55] at the time I filed that bug the sync hadn't been requested for lucid either [04:57] Hmm. [04:58] StevenK: What suggests that it thinks it's already there? [05:03] wgrant: Because of the log messages [05:09] StevenK: Pfft. [05:19] wgrant: You don't have much faith in logs now, do you? :-) [05:21] StevenK: I might point out that this is gina we are talking about. [05:21] I don't have much faith in it, let alone its logs. [05:21] wgrant: Point [05:25] wgrant: So I think the bug is in _getSource in lib/lp/soyuz/scripts/gina/handlers.py [05:26] wgrant: Since that function is effectively the one that decides if the package has been imported and can be skipped, or hasn't, and needs to be imported. [05:28] StevenK: See, I looked at that, but it looks fine. [05:29] What is the log message that tells you that it's skipping it? [05:29] I only see a log call when it isn't being skipped. === AdamDV2 is now known as AdamDV|ZzZz [05:33] And, well, https://edge.launchpad.net/debian/+archive/primary/+copy-packages?field.name_filter=gpt&field.status_filter=&field.series_filter= makes it pretty clear that 1.0.4-3 was never in that archive. [05:33] And _getSource's query looks correct to me. [05:34] StevenK: So, I doubt it's skipping it because it's already there -- unless you have a log message to prove it. [05:34] Just getting logs synced [05:49] 2010-05-06 02:06:20 INFO SourcePackageRelease already published with no chang [05:49] es as Pending [05:49] wgrant: ^ [05:50] 2010-05-06 02:06:20 INFO gpt already exists in the archive [05:50] Oh, interesting. [05:50] It really could do with logging the version string, though. [05:53] StevenK: So, there's only one source version in unstable, but are you sure that's not from a stable/testing run? [05:56] wgrant: Exactly the same message for sid, lenny and squeeze [05:58] Hrmph. [06:02] wgrant: But yes, the SQL reads cleanly and looks right [06:02] StevenK: Have you tried convincing it to spew SPR/SPPH versions and IDs? I am really really confused at how this can go wrong. [06:03] My first thought was that it didn't know about archives. [06:03] But it does. [06:03] It restricts the location correctly. [06:07] wgrant: I can't recall how to pull the SPPH out, since the SQL returns the SPR [06:07] StevenK: The thing prints out 'Pending', so it has the SPPH somewhere there too. [06:08] But that's probably in publish_sourcepackage. [06:08] I was thinking in _getSource [06:08] A debug print of what it thinks is actually in the archive [06:09] Right, but if you're going to get a patch applied, it's probably best to get as much debugging in as possible. [06:09] I'd be printing out the version we're searching for, and the found SPR and SPPH IDs. [06:11] wgrant: http://paste.ubuntu.com/428746/ [06:12] wgrant: I don't think that's enough, but I welcome suggestions [06:12] StevenK: SPR IDs too, please. [06:13] wgrant: It's .id? [06:15] StevenK: Yes. http://paste.ubuntu.com/428748/ would also be handy. [06:17] wgrant: http://paste.ubuntu.com/428750/ [06:17] StevenK: LGTM === spm changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 4 of 10.04 | PQM is open | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes [07:06] wgrant: LGTM ? Lets Go To Mothers???? [07:07] Looks Good To Me [07:08] * spm holds up the humour meter near StevenK ... hrrm. reading near 0.... [07:09] So, yes, hopefully that patch will tell us something less useless. [08:03] good morning === almaisan-away is now known as al-maisan === AdamDV is now known as Guest31208 [09:22] Morning Launchpadderisers. [09:26] wgrant: Still here? [09:29] StevenK: I am. [09:30] wgrant: Right, so we get this: [09:30] 2010-05-06 07:15:51 INFO SourcePackageRelease already published with no chang [09:30] es as Pending (SPPH 478847) [09:30] 2010-05-06 07:15:51 INFO gpt already exists in the archive [09:30] * wgrant hunts out SPPH 478847 [09:30] I have it, going to paste [09:30] gpt 1.0.4-1 in sid [09:30] gpt | 1.0.4-1 | debian | sid [09:31] Intriguing. [09:31] So... you're sure it's looking for -3? Does the log say so? [09:31] My debug message doesn't fire at all :-( [09:31] Oh, duh [09:32] * StevenK asked spm for the wrong thing [10:58] archive.txt is too long. [11:00] yep (well, *exceptionally* long... all the doc tests are long :/) [11:00] Heh. === mrevell is now known as mrevell-lunch === salgado-afk is now known as salgado === al-maisan is now known as almaisan-away === jelmer is now known as Guest34328 === Guest34328 is now known as ctrl === ctrl is now known as jelmer____ === mrevell-lunch is now known as mrevell [13:27] Hrm. [13:27] When the datacentre upgrades to lucid, as a side effect of the updated dpkg, the form of the quilt metadata in all source format 3.0 package import branches is going to change [13:28] From the .dpkg-source-applied file to the .pc/ directory [13:28] maxb: Hm, it's not already running a very recent dpkg? [13:29] Not *that* recent [13:30] The change is in 1.15.5.4 [13:30] Oh, really really recent. [13:32] This would be a james_w thing, right? [13:32] yes [13:48] eh? === Ursinha-afk is now known as Ursinha === almaisan-away is now known as al-maisan [14:07] BjornT, can you help me track down the apache configuration for launchpad in production? [14:08] i thought it would be in lp-production-configs but that only has launchpad configs [14:10] leonardr: i think you need to ask a losa for that [14:10] losas, will you induct me into this secret? [14:20] james_w: Once the udd importer starts running on lucid, lucid's dpkg will be producing a different form of quilt metadata in unpacked sources, so there will be spurious churn in the first versions of every 3.0 package imported after the change [14:20] I don't think there's anything to be done about that, though [14:21] no, I don't think so [14:21] In a perfect world we'd move the format change into its own revision, but ETOOMUCHFAFF === Ursinha is now known as Ursinha-sprint [15:47] leonardr, you free? [15:47] manish: i'm actually about to go afk for a bit. are you available at around 12:30? [15:47] leonardr, 12:30 UTC? [15:48] manish: sorry, that was stupid [15:48] in about an hour and a half [15:48] leonardr, sure [15:48] great === leonardr is now known as leonardr-afk === salgado is now known as salgado-lunch === leonardr-afk is now known as leonardr === beuno is now known as beuno-lunch [17:19] wgrant, I'm happy to answer an email, or talk at UDS or what have you [17:23] leonardr, am back. Whenever you are free, just tell me [17:23] manish, i'm back [17:23] leonardr, I was just checking your mail again. It is highly insightful. I never thought of that situation [17:24] the major question is have is what I mentioned at the end of the second mail I sent. How will the apps know about the new API? this makes Apps API specific? [17:24] manish: let me pull up your second email to respond in detail [17:25] but, it's true that there's a limit to the server-side changes that a client can hide from an app [17:25] leonardr, i see, this was the mail I was talking about https://lists.launchpad.net/launchpad-dev/msg03314.html [17:25] thanks for link [17:25] the advantage of the launchpadlib approach is that even in the worst case, where a client based on launchpadlib totally breaks, launchpadlib itself does not break [17:26] leonardr, Yeah. i can see that. Probably that caching is taken care by the library (launchpadlib) rather than the app [17:26] another questions is that how will I come to know what all versions of API are present? [17:27] means via code? not manually [17:27] manish: that's a very good question. right now there is no machine-readable advertisement of the available versions [17:27] i would really like to add one, and if you need it, that would give me a reason to fight for time to add it [17:27] or maybe we can parse the +apidoc page to check the existence, but it would be a brutal thing [17:27] exactly [17:28] i want to publish a json equivalent of the +apidoc page at http://api.launchpad.net/ [17:28] another problem with LP is that the WADL file is present behind OAuth [17:28] it should be publicly available [17:29] manish: i agree. the service root and the version list should not be protected [17:30] leonardr, what I intend to make is a component which takes in WADL file and then converts it to a library which can be loaded at runtime [17:31] and another component which checks the API version and calls the above component [17:31] that sounds pretty good. let me respond to your message in order [17:31] leonardr, sure [17:32] so we covered #1 a little bit [17:32] a client written against version 1.0 that uses checkBugIsValid will not break when we release version 2.0 with checkBugValidity [17:32] it will break, but that's not the time when it will break [17:32] it will break when 1) someone tries to upgrade it to use 2.0 instead of 1.0 [17:33] or when 2) we end-of-life version 1.0 [17:33] our goal is to work with developers so that they upgrade their applications (and fix problems like this) well before an old version reaches end-of-life [17:33] ^^ point. The most apt one [17:36] so, now on to #2 with that for context [17:36] wait, does that answer your question about having multiple backends? [17:37] the idea is that you will have one backend and you will gradually upgrade it to keep it up to date with launchpad [17:37] since launchpad is always available there's no need to have a 1.0 version and a 2.0 version. once you port your app to 2.0 you can get rid of the 1.0 backend [17:38] leonardr, I meant that there is a backend for example named foo [17:38] and the app just tells it what version of API it is built for [17:38] and the backend foo passed it the library which is linked [17:38] ah, i see [17:38] I havnt done it, just planning to do it this way [17:38] yes, the app will say it's based on version 2.0, and you will give it a library based on the current wadl file for version 2.0 [17:39] ^ exactly [17:39] this should work fine [17:39] I think this way there wont be hundreds of copy of the library [17:39] the one caution i would give you is to make sure you take advantage of whatever caching features are present in your http client [17:39] and every app doesnt need to be bundled with a copy of library [17:40] so that you don't retrieve the wadl file every time [17:40] yeah. it would be surely there. It [17:40] when the app requests for example v2.0 of API [17:40] then foo would hunt for library for it [17:40] if not present, it would download the WADL and generate the library [17:41] and then passed [17:41] the generated library is stored in the library cache [17:41] means relevent location as per the OS [17:41] ok, i just want to be clear that the wadl file for 2.0 may change once a month when a new launchpad is rolled out, or even more often than that on edge [17:41] you will need to occasionally check for changes and possibly re-download the wadl and regenerate the library [17:42] is it so? [17:42] once a stable API is released, it should be stable [17:42] any new changes should be in a new version. [17:43] for most new features, it's easier for us to just add them to all versions than to specifically exclude them from old versions [17:43] another problem with WADL is the enormous size [17:43] it takes minutes to download it [17:43] you will need to build this flexibility in anyway, to allow clients to access the 'devel' service [17:43] something has to be done here [17:43] manish: this is why i talked about the caching [17:44] we have made a number of improvements [17:44] i am working on one final improvement, which is compressing the wadl in transit [17:44] leonardr, that's cool [17:44] (i implemented this a long time ago and only recently discovered that it didn't work) [17:44] let me quickly outline the levels of caching [17:45] the first time a user starts an application the full wadl will be downloaded [17:45] once i make the compression fix, this will be about 100k instead of over 1 megabyte [17:45] Yes. based on the app API version [17:45] yes, per version [17:45] this wadl file will be used for 1 week (unless it is the devel wadl, but you won't use the devel wadl in a real application) [17:45] no requests for the wadl at all [17:46] during that week [17:46] at the end of that week, the client will make a _conditional_ request for the wadl [17:46] if the wadl changed in that week (because we rolled a new version of launchpad), the client will download the new wadl--another 100k [17:46] if the wadl has not changed (because launchpad has not been updated), the client will download nothing [17:46] and will use the existing wadl for 1 more week [17:47] well, how to achieve this? [17:47] some headers in the request? [17:47] yes. it is already implemented in launchpadlib + httplib2 [17:47] i want to make sure you implement it too, or use an existing http client library that understands caching and conditional requests [17:47] so you can receive the same benefit [17:48] I mean does the request to fetch WADL returns a specific header which contains a specific code? [17:48] yes [17:48] like the revision no of the API [17:48] sort of. there are two headers [17:48] then I could do a HEAD request and read this revision no [17:48] Cache-Control: max-age=604800 [17:48] ETag: "004e1871cab3546c277f7ebae72513058bc6b50f" [17:48] the ETag is analogous to the revision no [17:48] the Cache-Control tells the client that it can use this document for 1 week [17:49] okay. For each WADL revision, ETag is different? [17:49] yes [17:49] the ETag must change when the document changes [17:49] okay. Sort of Hash value? [17:49] exactly [17:49] so there's a week when you can just use this document from the cache without requesting it again [17:49] at the end of the week, you can make a conditional request [17:50] GET /1.0 [17:50] If-None-Match: "004e1871cab3546c277f7ebae72513058bc6b50f" [17:50] this tells the server to compare the ETag you give it in If-none-Match against the current ETag [17:50] if the ETags are the same, that means the wadl has not changed [17:50] the server will send: [17:50] 304 Not Modified [17:50] Cache-Control: max-age=604800 [17:50] that means you can use the existing wadl for another week [17:51] if the ETags differ, the server will send [17:51] 200 Ok [17:51] Cache-Control: max-age=604800 [17:51] ETag: "some-new-etag" [17:51] and then the new wadl, which you can cache for a week [17:51] you can make a HEAD request, but then if it turns out the wadl has changed, you will have to make a GET request anyway [17:51] so you might as well make a conditional GET request [17:51] does this make sense? [17:52] Absolutely. :) [17:52] great [17:52] It clears so many of my doubts [17:52] like i said, hopefully you can use an existing C# http client that understands these rules [17:52] they are standard parts of http [17:52] but if not, they are fairly simple to implement [17:52] It's pretty simple to implement this [17:53] yeah [17:53] If-None-Match headers solves so many of my doubts [17:53] otherwise I would have gone crazy [17:53] yes, it's very useful [17:54] my biggest fear was what about the people who are under slow internet connection [17:54] or who pay per kb [17:54] so compression of WADL is very useful [17:54] yes, it's very bad that the compression system has been broken for so long [17:54] unless you have any questions, i'm going to go back to work on that [17:55] Nope. Nearly all doubts solved which I had presently [17:55] great [17:55] ping me again if you have more questions [17:55] leonardr, sure. === salgado-lunch is now known as salgado [18:08] nytol [18:08] mars, ping [18:13] salgado, ping [18:14] hi sinzui [18:14] salgado I am hacking on bug 575317. I cannot delete the account because it is [18:14] used required by the OpenIDSummary table. I think I need to update that table [18:14] to point the entries to the remaining account. This may be pointless because [18:14] other schema parts I do not know about also link to Account. I could change [18:14] the openid of the merged account to ensure authentication falls back to email [18:14] Bug #575317: Person merge must delete merged account [18:14] address === beuno-lunch is now known as beuno [18:16] salgado, maybe our OpenIdRPSummary table is obsolete. Lp is not the real OpenId provider [18:16] it is obsolete, yes [18:16] fab [18:17] but you're right that there may be other tables referencing the row you want to delete [18:18] I guess changing the OpenID identifier of the merged account is the easier solution, while we don't get rid of the Account table completely [18:18] sinzui, pong [18:20] There is a function that allows us to change the openid. I could use it to make the merge account unmatchable [18:20] mars, salgado has confirmed my discoveries about account. ^ I will instead change the account's openid identifier so that there is no match--fallback to the email address to select the Lp person [18:21] sinzui, cool [18:27] mars, I will prepare a script to that we can run in production that will update the merged accounts. [18:32] Chex, the script sinzui described above should make it unnecessary to delete that account you were trying to delete [18:33] Chex, yes it will. I have will not know the rules of what must change until I complete the test I am writing === EdwinGrubbs is now known as Edwin-lunch === al-maisan is now known as almaisan-away [22:01] * maxb is amused by the Code/Branches backtracking [22:04] maxb: I wasn't particularly happy with the change when it happened [22:04] I like Code better too [22:29] Can someone please EC2 https://code.edge.launchpad.net/~wgrant/launchpad/bug-576168-no-disabled-latest-ppas/+merge/24812? Vanished instance count so far: 1 [22:42] sinzui: About bug #576478: it doesn't make sense to have Packagings at all for virtual packages like that. [22:42]