[13:36] When I run "ubuntuone-client-applet" in a terminal, I see a notify-osd popup saying "There was a fatal error in Ubuntu One. This mayb be a bug in the software. Please click on the Ubuntu One icon in your panel to report a bug." [13:37] -- [13:37] Guillermo Gonzalez [13:37] [13:37] -- [13:37] Guillermo Gonzalez [13:37] [13:37] -- [13:37] Guillermo Gonzalez [13:37] [13:37] ooops [13:37] I've flushed all the settings and reinstalled a few times to try to sort that. [13:37] verterok: :) [13:37] jblount: hi [13:37] verterok: hi [13:37] jblount: could you check the logs? [13:38] jblount: ~/.cache/ubuntuone/log/syncdaemon.log [13:38] verterok: tailing now... [13:38] verterok: http://pastebin.ubuntu.com/258640/ [13:39] verterok: The oauth log: http://pastebin.ubuntu.com/258641/ [13:39] jblount: could you paste a few more lines :) [13:39] ? [13:39] jblount: looks like a loop [13:40] verterok: On the sd log? [13:40] jblount: yeap [13:40] jblount:there is an error in the oauth logs [13:40] verterok: http://pastebin.ubuntu.com/258642/ [13:41] verterok: Yeah, I wonder if that is the problem-causer [13:41] jblount: syncdaemon is running, pleae confirm : ps aux | grep ubuntuone [13:42] jblount: I'm sure we have a bug report with a similar traceback [13:42] Moin. [13:42] verterok: "/usr/bin/python /usr/lib/ubuntuone-client/ubuntuone-syncdaemon" is running [13:42] CardinalFang: j0 [13:44] jblount: so the oauth dbus thingy is having problems, I can't help with that, could you paste a bit more of the oauth logs? [13:45] verterok: Sure, http://pastebin.ubuntu.com/258643/ is all of it === rmcbride_ is now known as rmcbride4 === rmcbride4 is now known as rmcbride [13:49] jblount: please, check the contents of /etc/xdg/ubuntuone/oauth_urls [13:50] jblount: you should have a [default] section with 4 rows [13:50] verterok: that should've probably ben [DEFAULT] [13:50] verterok: AFAIK, DEFAULT is special, but default is not [13:51] (special as in "magically appears in all sections") [13:52] Chipaca: uh? oauthdesktop isn't usign configglue, just ConfigPArser [13:52] verterok: that's CP magic, not CG magic [13:53] verterok: I don't seem to have this anymore. I did fiddle with it at some point last week trying to do something unrelated, how can I bring this back? [13:53] “Know Thy Standard Library” :-D === chs_ is now known as chs [13:53] Chipaca: :) [13:53] jblount: reinstall the client package maybe? [13:54] verterok: Will try again, although I think I have since having this problem. [13:54] it's probably considered a configfile [13:54] you might have a oauth_urls.dpkg.new or something [13:54] Chipaca: but when I change config files, apt barf about what version I want to keep [13:54] Chipaca: donde? [13:55] /etc/xdg/ubuntuone/oauth_urls.dpkg-new ? [13:55] jblount: qué te hacés el hispanoparlante, ahora! :) [13:55] Chipaca: :) [13:56] Chipaca: This file, it doesn't come back with reinstall, so I guess it is treated like a config file. I also don't have a .dpkg-new in that directory (/etc/xdg/ubuntuone). [13:56] Just "auth_registration.d" and "syncdaemon.conf" [13:56] Chipaca: a dpkg-reconfigure? [13:57] verterok: don't think so [13:57] jblount: dpkg -P python-ubuntuone-client; apt-get install python-ubuntuone-client [13:57] * jblount is once again learning that he shouldn't fiddle with random files [13:59] * Chipaca goes for some tea [13:59] jblount: he next time you need to change a file in /etc/xdg/foo/bar.conf, just add a new file in: ~/.config/foo/bar.txt [13:59] jblount: in the case of u1, ~/.config/ubuntuone/ :) [13:59] s/he/the/ [13:59] verterok: Thanks! [13:59] jblount: if the app is using xdg correctly it should pick the file in your home [14:02] verterok: Chipaca 's suggestion seems to have worked, I just went through the oauth process and sd seems to be working now. [14:02] jblount: cool [14:51] meh, oauth :( [14:54] thisfred, CardinalFang: so, afaict, things we need to do for desktopcouch for the freeze (which statik would like finished today if possible): make dc.records use oauth (lp:~sil/desktopcouch/dc-records-oauth, blocked on couchdb patch), make pairing exchange oauth tokens (chad), make DC startup kick off continuous replication (chad, bug 416581), work out how to get cloud oauth tokens into DC (me/thisfred). Is that it? [14:54] Launchpad bug 416581 in desktopcouch "Start replication to paired servers on desktopcouch startup" [Undecided,New] https://launchpad.net/bugs/416581 [15:00] MEETING BEGINS [15:00] HAPPY MONDAY SAY ME IF YOU WANT TO OR IF STATIK IS THE BOSS [15:00] haha [15:01] statik: :) [15:01] me [15:01] me [15:01] me [15:01] me [15:01] me [15:02] dobey, urbanape? [15:02] me [15:02] CardinalFang ? [15:02] urbanape is out being famous today [15:02] me [15:02] DONE: desktopcouch merges, packaged couchdb-0.10 branch, landed chads spawning branch and broke edge rollouts [15:02] TODO: Erlang packaging work (enabling tests) to unblock main inclusion of couchdb [15:02] BLCK: nope [15:02] aquarius, you have the stick [15:02] ⚀ DONE: talk to chad about pairing app actually doing oauth token exchange and save into management db and DC startup kick off continuous replication to all paired servers; discover that chad is doing all my work; win [15:02] ⚁ TODO: piston oauth in snowy; learn about process groups; review desktopcouch changes from past week; be baffled as to why oauth isn't working but only sometimes; talk to cardinalfang/thisfred about what remains to be done for desktopcouch [15:02] ⚂ BLOCKED: couchdb patch which lets OAuth read users from the ini file doesn't seem to work [15:02] "I ask you to kill teknico and you can't even do that one simple thing" - Robert Vaughn, Superman 3 [15:03] DONE: installed and configured the new Dell notebook, investigated a problem in markgsaye's phone_setup branch, started a branch to fix the problem [15:03] TODO: completing and landing the mentioned branch, working on the phones web ui [15:03] BLOCKED: none [15:03] next: jblount [15:03] superman kill me? ha! [15:03] DONE: reviews, fixed client (reminded self to stop fiddling with random config files that in my home directory) [15:03] TODO: LAND BRANCHES THAT FIX THE WEB UI, take Tuesday and .5 of Wed off [15:03] BLOCKED: nope [15:03] rodriogo_: rocknroll [15:03] • DONE: Tomboy default sync server work. Started adding url fields to evo-couchdb contacts. Played a bit with liboauth [15:03] • TODO: Add more tests in couchdb-glib test suite. More openSUSE packaging. Add social services accounts config to about-me. Talk to Ara about writing mago tests for evo-couchdb. Propose couchdb-glib/evo-couchdb for GNOME 2.29. Store UUIDs for postal addresses. Conflict resolver tool in pair tool. oAuth authentication and signing of all couchdb-glib requests. Finish adding URLs to contact records in evo-couchdb [15:03] • BLOCKED: none [15:03] next: dobey [15:03] ☭ DONE: Finished #330769, Buenos Aires sprint and back [15:03] ☭ TODO: Preferences dialog, Fork OAuth [15:03] ☭ BLCK: None. [15:03] CardinalFang: surprise! [15:03] and fear [15:04] * aquarius laughs [15:04] DONE: Vacated. Accidentally thought about d-c replication and worked on that. [15:04] TODO: Make advertisement/replication daemon. [15:04] BLOCKED: OAuth is still a mystery. Maybe this doesn't block. [15:04] and an almost fanatical devotion to the pope [15:05] the pope with an eye [15:05] MEETING ENDS [15:05] hey dobey, how was Buenos Aires? practiced your Spanish? :) [15:05] Well, I do admire Alexander Pope mightily. [15:05] rodrigo_, good work on the liboauth stuff :) [15:05] rodrigo_, did you see the blog post about macaco contacts integrating with couchdb yesterday? [15:05] aquarius: well, nothing done yet, just trying to understand the liboauth API [15:05] statik: no, in planet ubuntu? [15:06] haha, oauth [15:06] rodrigo_: a little spanish, not much time outside the hotel though [15:06] some dude suggested an ubutnu one service that tracks your laptop, too, which was interesting [15:06] aquarius: track it how? [15:06] "Last IP address is: foo." [15:07] dobey: you had at least some good meat, didn't you? [15:07] rodrigo_, http://www.themacaque.com/?p=226 [15:07] rodrigo_: yeah, we went to the meat emporium on monday [15:07] statik: SteveA and I talked with him some weeks ago, I even sent him a mail but had no answer, so good he keeps working on it [15:07] rodrigo_, http://jldugger.livejournal.com/32566.html [15:07] rodrigo_, i registered on the site and left a comment talking about desktopcouch, but it hasn't been moderated yet [15:07] * aquarius goes for a cigarette before hassling thisfred and cardinalfang for a full list of what needs doing for a DC 0.3 release [15:08] aquarius: ah, cool, anti-theft :D [15:08] * CardinalFang interprets that as an invitation to think about it. [15:09] statik: yeah, he should use desktopcouch [15:12] hrmm [15:13] aquarius: a cool idea, certainly, but needs some major changes: since ubuntuone needs a logged in user to run, the thief will have gone "WTF is this Umbongo? I'll format and install windows, before ubuntuone would have the chance to report an IP. [15:14] and this, ladies and gentlemen of the jury, is why we need ubuntuone in the kernel! [15:14] ;) [15:14] thisfred, CardinalFang: so, afaict, things we need to do for desktopcouch for the freeze (which statik would like finished today if possible): make dc.records use oauth (lp:~sil/desktopcouch/dc-records-oauth, blocked on couchdb patch), make pairing exchange oauth tokens (chad), make DC startup kick off continuous replication (chad, bug 416581), work out how to get cloud oauth tokens into DC (me/thisfred). Is that it? [15:14] Launchpad bug 416581 in desktopcouch "Start replication to paired servers on desktopcouch startup" [Undecided,New] https://launchpad.net/bugs/416581 [15:14] thisfred: No, the kernel does not survive a install windows. [15:14] thisfred: it *has* to go into the BIOS! [15:14] thisfred, Yeah, but we'd have to lower the Python stack space too far. :( [15:14] thun_: yeah I understand that: but likely the thief would boot the machine at least once [15:14] before reinstalling [15:15] statik: I just submitted patch for tomboy with u1 as default server. At the end, discussing with Sandy we decided not to allow several sync servers, since tomboy is not ready for that, so it just defaults to having u1 as default. For more than one sync server, it will be done in next cycle [15:15] thun_: but +1 on ubuntuone in the BIOS [15:15] :D [15:15] likely the thief would steal your information before wiping the drive and selling it on ebay [15:15] thisfred, Though, think about what a system desktop-couch would get us. [15:15] * thun_ would be happy to get a kde version first:-) [15:15] rodrigo_, they're going to ship tomboy in gnome with u1 as default sync server? [15:15] aquarius: no, it's a patch for our package [15:15] likely, your laptop is probbly suspended or in hibernation, and so you are already logged in anyway [15:15] rodrigo_, oh, right, heh.i thought: blimey. :) [15:16] aquarius: I didn't even suggest Sandy to accept the patch :D [15:16] though likely, the thief will probably be smart enough to not have it hit the network when thieving your data off the disk, before wiping it clean and stuffing it on ebay [15:16] rodrigo_, that is great [15:16] aquarius, I can't get both pairing+oauth and the replication thing done in the next 8 hours. [15:17] is liboauth the C library version? [15:17] dobey: you'd be surprised. Most people who feel they need to resort to stealing physical objects aren't geniuses [15:17] One of them, sure. [15:17] statik: well, I would have liked to not lose that much time trying to make it work with several servers, to just submit such an easy patch (https://bugs.edge.launchpad.net/ubuntu/+source/tomboy/+bug/416865), but well, it's done now :) [15:17] CardinalFang, that's why I'm trying to nail down the list, so we can divide up the work ;) First thing is: is there anything else that needs to be on that list and isn't? (thisfred, statik, your comments on that invited too) [15:17] Launchpad bug 416865 in tomboy "Need to have ubuntuone as default sync server" [Undecided,In progress] [15:17] dobey: yes [15:17] dobey: If they were they'd be stealing intellectual property. That's where all the big money is, I hear. ;) [15:18] thisfred: exactly. though you can't really sell open source IP to anyone for any big money [15:18] or i would already have a house or 5 [15:18] aquarius, i'm working on unblocking couchdb going into main by enabling tests in the erlang packages, and am standing by to build you a new couchdb package when we get a fixed oauth patch [15:19] aquarius: the latest couchdb package in karmic includes the changes feed? [15:19] aquarius: I can't think of anything else, though I envision going through the list itself may sprout a few side items [15:19] aquarius: well, or the one in our PPA, which I think was the one being updated this morning [15:19] rodrigo_, certainly the one in our ppa does. I'm not sure about whether that's actually in karmic or not yet. (statik will know) [15:19] ah ok, that's enough for testing [15:19] aquarius, it's in our beta ppa now [15:19] damnit [15:20] rodrigo_, it's not the global _changes feed [15:20] statik: it's per database? [15:21] meh [15:22] * dobey hopes autogen works this time [15:22] stupid hard disks failing [15:23] rodrigo_, i'm not quite sure [15:27] hrmm [15:28] aquarius: what's the url for the changes feed? [15:29] rodrigo_, http://localhost:portnumber/dbname/_changes [15:29] or _changes?feed=continuous for continuous updates [15:30] aquarius: seems to not work for me then, I guess I don't have the latest [15:30] * rodrigo_ checks [15:30] ah, the db didn't exist :D [15:36] statik, so, I suspect, we won't have all this stuff finished today, so no dc0.3 today :( [15:42] aquarius, maybe we can do a 0.3pre with whatever we can get finished today [15:42] statik, yep; jan is planning to look at the patch in an hour or so [16:10] aquarius: so [16:10] I'm thinking reuse the keys to the filesync car [16:10] agreed entirely [16:10] but...can you sign up for u1 data without u1 filesync? [16:10] which everyone will have after they installed u1 [16:10] aquarius: not in this alpha you can't is my thinking [16:11] kk. So, then, keys already exist. In order to sync your data with U1, then, you just need to run something on the desktop. [16:11] My thought is: add a button to the pairing app with "pair with Ubuntu One" on it. [16:11] aquarius: yes, that's a logical enough place for now [16:12] and maybe forever [16:12] and that button would...get the keys from the keyring and write a paired server record for U1 [16:12] and that's it? [16:12] I really like that it seems we're gonna be able to treat it just like any other pairing almost all of the way [16:13] that's the plan [16:13] aquarius: yes, then the other side needs work [16:13] that way U1 is not magic and special [16:13] other side? [16:13] cloud [16:13] what work does it need? [16:13] replication needs to readch out and create databases *anyway* [16:13] we need a way to put those keys in the .ini/_users db [16:14] when a user signs up [16:14] why? just put 'em in there when the punter first signs up for u1. [16:14] right [16:14] that's what I mean [16:14] I'm not sure where those keys are created [16:14] but that's easy, and not really dtc related [16:15] *nod* [16:15] so it's just easy then, right [16:15] add one button to the pairing app which gets the keys from the keyring. Done. [16:17] it can't be that easy. [16:17] what does u1fs put in the keyring? [16:17] right, so that's doable [16:19] just the oauth token [16:19] wow, erlang builds itself 4 times while bootstrapping [16:19] dobey, yep, got it, perfect. [16:19] although i think we should probably do it better than we do [16:21] but we can change that later [16:24] hey tcole [16:24] howdy [16:28] aquarius: ok, so: wrt desktopcouch, what can I do right now to help? I have only one very minor desktopcouch issue assigned to me, which I can work on, but it's not a new feature, and basically only code cosmetics [16:29] so, if there is nothing concrete, I'll go on with cloud issues, and respond to your cries of despair and help out wherever appropriate [16:29] thisfred, you can talk to jan when he looks a the patch, and test that it works, since I have to go and get my daughter shortly...but that's not a right-now thing. Hm, what else is on the list? [16:30] thisfred, what will be the URL of my couch in the cloud? [16:33] aquarius: good point: we need to decide that and get a proxy up I suppose [16:35] what would be kinda neat to have is ubuntuone.com/couchdb/foodb [16:35] like /files and /notes [16:35] where the login would redirect, but no idea if that's possible with oauth [16:35] *nod* [16:36] if not ubuntuone.com/[u1accountid]/[foodb] works [16:37] I don't think an ugly url matters really [16:37] we won't be opening futon or anything like that [16:38] nothing needs to go there but replication [16:38] you can pretty much do anything you want with oauth [16:39] right [16:40] I'm not sure what we want to do though. Since we don't need human readable urls, I think we need to do what is at the intersection of least work and best debuggability [16:40] so less redirection is probably better [16:42] going to pick up my daughter [16:42] bbiab [16:42] aquarius: later [16:43] thisfred, am about half done on the Ubuntu-One-button-in-pairing-app [16:44] aquawesome [16:51] * CardinalFang acquires lunch. [16:54] * dobey does some evil to test his branch [16:59] crap [16:59] need to make syncdaemon fork/exec (daemonize) itself on start [17:01] start-stop-daemon is a built in program to do that [17:03] that's a server thing [17:03] i need it to fork/exec when i run it from the applet [17:19] sigh [17:19] and my desktop just locked up hard [17:20] probably from my attempt at trying to fork/exec syncdaemon [17:20] but i really must get some lunch now, bbiab [17:43] Ubuntu One/My Files was removed? [17:53] huh, nevermind, it seems like Nautilius changed the bookmark on me [18:02] J_Litewski: Sorry no one has responded, but I belive that is a recent change from us, we are trying to make it easier to find "where do I put these files" [18:03] np, I was having problems with it syncing yesterday and today my bookmark changed, so I was think my computer was starting to spaz on my [18:04] *me [18:09] jtatum_: :), if it makes you feel better, I had some video card driver weirdness this morning, and was thinking something along the same lines. [18:47] dobey: Is there a reason we're using "allow modification" instead of "allow changes" in the sharing interface on the desktop? [18:47] (I'm making the web ui match right now, and modifications seems like a yucky computer word) [18:49] i don't think they had computers when middle english was in widespread use [18:50] thisfred, I have thought of something else that needs doing: port discovery logic. the port number is now output to the log file, and parsing that is better than walking proc [18:50] dobey: What do you think about swapping it for changes instead of modifications ? [18:50] Ooo. [18:50] aquarius, Do we have the PID also? [18:50] CardinalFang, yep, it's written to the pidfile. why? [18:51] jblount: i don't know. don't really have time to fuss about it either [18:51] dobey: That's fair. I'll leave it alone for now. [18:52] aquarius, Just curious. I want to make sure that the data is right, still. Walking /proc ensures that the port is recent info. Log files can be stale forever. [18:52] CardinalFang, yeah, but walking /proc is a horrid hack :) [18:53] Chipaca or verterok: ping [18:53] dobey: pong [18:54] aquarius, Agreed. I'd be happier delegating that to a program that does the same, and that everyone would scream about if it broke. "ps" or "netstat" maybe. Poor Unix needs better process management functions. [18:55] aquarius: ah [18:55] Chipaca: how hard would it be to make syncdaemon "daemonize" (fork/exec itself) on start? [18:55] well, shouldn't be too hard, and should make things more stable. [18:55] aquarius: Should I get on that then> [18:55] ? [18:56] thisfred, I think so, yes -- it's a nice self-contained bit [18:56] aight [18:56] dobey: not hard [18:56] dobey: why? [18:57] dobey: not hard, if we for a moment can pretend we are not hours away from feature freeze [18:57] dobey: so, why? [18:58] Chipaca: so that it can exit(0) and not block the applet when i do subprocess.Popen() to run it when dbus gives me back an error [18:58] but maybe i am just trying to go about this the wrong way in the first place [18:59] dobey: well, dbus shouldn't give you back an error. What error is it giving you back? [18:59] Chipaca: me personally? nothing unless i somehow force a failure by simply making it so dbus can't activate syncdaemon [19:00] dobey: I don't know if it's the right way, but you can avoid blocking on a subprocess [19:00] oh, btw CardinalFang: I took out the random stopping of couchdb in the tests. I know I said before I wanted to test the stop and automatic start, but I'll think of a more controlled way. Perhaps I'll just copy one other simple test, and run a stop at the beginning. [19:00] Chipaca: but lots of people are getting the "Fatal Error" warning at start up, because i guess syncdaemon starts slower than the dbus reply timeout or something, and dbus throws its hands in the air and reports an error saying it doesn't know what's going on [19:01] dobey: the solution changes dramatically depending on that "or something" [19:03] dobey: according to bug #62763 the timeout was raised from 25 to 40 seconds. Both values are immensely huge compared to the u1sd startup times we're seeing. [19:03] Launchpad bug 62763 in dbus "dbus activation timeout too short" [Low,Fix released] https://launchpad.net/bugs/62763 [19:03] Chipaca: well given beuno's most recent report, it seems like it fails, and then starts up and sends the StatusChanged signal with READY_WITH_LOTS_OF_THINGS_TO_DO_AND_MAYBE_THERE_IS_SOME_NETWORK (or whatever that state is) :) [19:04] verterok: dbus registration happens before local rescan, right? [19:04] Chipaca: yes [19:04] quite [19:04] Chipaca: i don't think it's an activation timeout issue [19:04] Chipaca: but rather, message reply timeouts [19:05] basically the general "dbus isn't really async, but you can try and use it like it is" problem [19:05] dbus isn't async? [19:06] dbus is async [19:06] dbus functions can be sync or asycn. [19:06] but it's usually used in a fake-sync mode [19:07] CardinalFang: not really sync, the client just blocks waiting the async reply :/ [19:07] right, but dbus itself is an asynchronous message-passing system [19:07] verterok, Dang! I guess you can think of syscalls doing the same! [19:08] CardinalFang: :) [19:09] CardinalFang: I mean, it's async by nature, but looks like the sync api is the most used one. but as it don't gurantee delivery, IMO don't make sense to use a sync api ;) [19:10] It does not retry? [19:10] * CardinalFang is no dbus expert. [19:12] CardinalFang: I'm also not a dbus expert :) [19:12] CardinalFang: but I don't think retry is builtin, in the sync api :/ [19:13] dbus method calls are sync [19:13] if you want async, you have to make your sync dbus method call do something in a timeout/thread/something, and have it call a signal [19:14] and have the method call itself return immediately, and return void [19:15] aquarius, Do you have an API in mind for Bug#416581 data? [19:15] dobey: no you shouldn't need to run the method call in thread, syncdaemon dbus tests run in a single thread and use dbus calls without blocking [19:16] CardinalFang, api? [19:16] verterok: no no, you misunderstood me [19:16] aquarius, Yeah, I need to read those records, so I wondered if you were just designing the data or updating a branch also. [19:16] dobey: you mean the endpoint? [19:16] verterok: in the "server" that implements the method call, you have to stick the actual work in a timeout/thread, and return immediately, otherwise it will block [19:17] CardinalFang, ah, the record format is as in the bug, I think: server, oauth, database_name_prefix, pull_from_server top level keys [19:17] dobey: hmm, don;t know really. I hate threads, twisted FTW! [19:17] :) [19:17] aquarius, Er, okay. So I will write code. Rgr. [19:18] any dbus method call that doesn't immediately return void, is not async :) [19:18] CardinalFang, just seen your comment. database_name_format seems fine to me. What do you mean by "wary of changing hte name"? [19:18] which is why we get the "did not get a reply" errors for the get_root call and such [19:19] eh [19:19] verterok: you said i can make popen not block? [19:19] CardinalFang, how are you getting a uuid for the pairing identifier? [19:20] aquarius, if the remote database name isn't the same as here, then an uneducated pairing later could (AFAIK) blossom into infinite databases. foo -> pfx-foo. pfx-foo -> pfx-pfx-foo ... [19:20] in C i might agree... i don't see how in pydoc [19:20] CardinalFang, yeah, but we have to do that, because your contacts db and my contacts db may well be ont he same U1 couch server [19:20] CardinalFang, hence u1accountid/contacts [19:21] dobey: how are you using Popen? [19:21] dobey, does the desktop *know* your U1 account ID in any way? [19:21] aquarius, I intended to use one of the functions in the uuid py module. [19:21] aquarius: no, all we know is the oauth access token [19:21] dobey, that's what I thought [19:22] thisfred, any suggestions as to where I should get the database name prefix from, since the desktop doesn't know the u1 account ID? I could oauth to a page on the web server which displays it... [19:22] verterok: verterok http://pastebin.ubuntu.com/258827/ [19:23] CardinalFang, str(uuid.uuid4()) ? [19:23] aquarius, that sounds right, uuid4() [19:23] dobey: p.wait() is the blocking bit [19:24] verterok: how do i know if it failed to start up? [19:25] dobey: you need to check p.returncode [19:25] aquarius, I don't yet understand why we need prefixes, and if we do need them to keep data straight on servers we own, perhaps we should change the server to keep them arranged without d-c changes. This means changing Erlang code, I'm sure. [19:26] aquarius: sry phone [19:27] verterok: if the process is still running, p.returncode is None [19:27] dobey: yes [19:27] and it blocks [19:27] CardinalFang, we need them because up at u1 we'll have (conceptually) one couch server with all the databases on, so we need to differentiate your contacts db from mine [19:28] CardinalFang, I honestly don't think it's a problem -- when we set up replication, we have to specify the DB names for each end anyway [19:28] CardinalFang, so we just specify xxx for local name and db_prefix+xxx for remote name [19:28] dobey: calling returncode blocks? [19:28] dobey: it's an attribute lookup :/ [19:28] well it's not a call [19:29] but yes the applet is still blocking without p.wait() [19:29] dobey: right, it's an attribute lookup [19:30] dobey: possiblt the p.stderr.readlines() call [19:30] *possibly [19:30] yes, well readlines() also blocks [19:30] and returncode is not None, so... [19:30] :( [19:30] dobey: != 0 right? :) [19:31] aquarius, at U1 server, use oauth token supplied earlier to map to U1 account name, and prefix it there silently. If I can change a record and/or code on my box and shove data into your database, then you might regret it. [19:31] CardinalFang, that's why we want per-database authentication :) [19:31] dobey: maybe moving that into a thread, and block the thread instead of the mainloop? [19:32] ah ok [19:32] * aquarius proposes lp:~sil/desktopcouch/pair-with-u1, which adds "Ubuntu One" to the pairing app. CardinalFang, a glance from you at that would be good :) [19:32] "if returncode and returncode != 0" seems to be ok [19:32] why not "if returncode"? 0 is falsy, isn't it? [19:33] aquarius: if we don't know the u1account id we're sort of screwed, unless we do a single url that redirects, or have a service to find out [19:33] aquarius: the account id isn't findoutable from the filesync either? [19:33] aquarius: are negative numbers true? [19:33] thisfred, nope. filesync only knows the oauth tokens. [19:34] dobey, yep [19:34] aquarius, thisfred: if we are doing oauth, shouldn't the server side be able to handle the account mapping transparently? [19:34] dobey, on the other hand, 0.1 is also false. Er. :-) [19:34] aquarius, like some kind of really simple proxy that knows if the client is syncing the contacts db signed with elliot's token, then it's really the elliot_contacts database (i'm simplifying, but you get the idea) [19:34] statik, only if we guarantee that everyone's oauth token is unique, which we don't currently do as far as I know? [19:35] aquarius: uhm, i don't think exit() allows you to exit with float values? [19:35] tcole, do we guarantee that all generated oauth tokens are unique? (I'm not sure who did the oauth server side implementation for u1fs) [19:35] we guarantee that they are unique, otherwise the whole system would fall apart i think, since the token uniquely identifies a user [19:35] dobey, I don't know what you're using it for, I'm just sayin' :-) [19:36] aquarius: i just have to check the returncode of Popen() :) [19:36] statik: no we don't :) [19:36] dobey, oh, then you oughta be fine :) [19:37] I don't think we guarantee so much as assume that really big randomly generated numbers are unique [19:37] or at least, oauth.py certainly doesn't guarantee uniqueness [19:37] but we have database constraints and so on [19:37] aquarius, # parse "a=b&c=d" to {"a":"b","c":"d"} What if b or d has an equals-sign in it? Can it? Are you sure nothing can have an ampersand in it? [19:37] right, but if we were to assume that they are unique we wouldn't be creating an *extra* problem. got it :) [19:37] if we ever issued two users the same oauth token, the world would end [19:38] CardinalFang: shouldn't b or d be percent-escaped? [19:38] CardinalFang, that key should only contain oauth_token=whatever&oauth_token_secret=whatever, and u1fs relies on that. my parsing algorithm is relatively simple, I admit it :) [19:39] I wonder how u1fs does it? [19:39] aquarius, If you change that x.split("=") to have 1 as a second parameter, I'd be happy. [19:39] oh, it doesn't, it hands it directly to oauth.OAuthToken [19:39] CardinalFang, wisdom, I will do :) [19:39] dobey, aquarius: i just verified, we have UNIQUE db constraints on the server side in all the right places on the oauth tables [19:41] CardinalFang, so...maybe you're right and we don't need the prefix. Clever. [19:41] verterok: thanks :) [19:41] CardinalFang, although you have invoked a vapourware interpreting proxy, but that's ok ;-) [19:41] :) [19:42] dobey: np [19:42] ok, now to pick one of thse million bugs to declare "the winner of having all the others be a duplicate of it" [19:44] too many bugs [19:45] aquarius, instead of print "couch fail" , please use logging.exception("couch fail") . We get the error logged this way. [19:46] CardinalFang, ooh, I forgot that. It needs to do something a lot better than print something :) [19:46] aquarius, after that, I approve. [19:46] CardinalFang, like give some user feedback. I'll do that now. Meant to come back to it ;) [19:46] Rgr. [19:56] CardinalFang, any reason why you used a Dialog rather than a MessageDialog in pair? [19:56] aquarius, Er, no reason. Ignorance, most likely. [19:56] CardinalFang, kk :) [19:57] messagedialog == pain [20:00] ugh, allergies :-/ [20:00] aquarius, rather, there may be a reason, but I do not remember what it is any more and can't tell you it. [20:01] CardinalFang, does the logging module have a way of logging a traceback? Or do I have to construct a message myself from the traceback and the logging.exception() the message? [20:02] aquarius, .exception groks it. [20:02] CardinalFang: aquarius: when I find out the port number from the log file, do we really need find_pid() anymore? (I think I'm going to leave well enough alone, but wouldn't we be able to use get_port to see if a db was running? something for the future...) [20:02] thisfred, we want to make sure that the log file isn't stale, I say. [20:02] thisfred, you need find_pid because the logfile might be stale [20:03] thisfred, and you need to know whether couch is already running or not to know whether to start it up or point at the existing one [20:03] CardinalFang, is it clever enough to just do logging.exception() inside an except handler? Or do I need to grab sys.last_traceback or whatever it is? [20:04] sure, but with a stale log, we could just to a call to the url, and see it's not running [20:04] (or e, of course) [20:04] anyway, I'm leaving get_pid in, [20:04] it's just that it seems silly, since the actual pid is then aggressively ignored by all the code [20:04] aquarius, Hmm, I think it must be inside an exception handler. Any manner of exceptions are thrown in mundane calls, and you don't want to make it easy to get the wrong one. [20:05] "Oh darn, I logged the StopIteration instead." [20:07] CardinalFang, winner, it works fine, I think :) [20:07] CardinalFang, and pushed [20:10] aquarius, Looks good. [20:11] aquarius, Is that all we need for writing pairing, even among local machines? [20:11] CardinalFang, I think so! [20:12] CardinalFang, unless you think I'm missing something. There is obviously your find-the-port daemon cleverness thing [20:13] CardinalFang, bug for the vapourware proxy server: https://bugs.edge.launchpad.net/ubunet/+bug/418266 [20:13] Error: This bug is private [20:13] I *think* this is almost doable completely inside apache, although it probably isn't. I'd be interested in thoughts on how best to have it implemented [20:13] aquarius, all I need is a local list of all the UUIDs of hosts we pair with, and my UUID so I can yell it onto the network. [20:14] CardinalFang, so, for local ones, "server" would be blank and instead there'd be a "server_uuid"? [20:15] It's the UUID you already added. [20:15] But blank, yes, is fine. [20:16] CardinalFang, hang on, confused now. I'd have thought that the U1 record would have "server: ubuntuone.com", and the local record would have "server_uuid: {93457934573}" [20:17] aquarius, Yes, that's fine. The ID is all I care about. It's a filter-list for IDs I see advertised nearby. [20:18] CardinalFang, cool [20:18] how do I run a subset of tests with 'make check' in ubuntuone-client? [20:19] tcole, "trial mod.ule.path" [20:19] tcole, you may need to set PYTHONPATH . [20:20] could you give me a specific example please? [20:21] hm, I think I got it [20:22] thisfred, so...the vapourware redirecting proxy...how does the proxy get a list of oauth-tokens<->userids? [20:23] aquarius: that info would be in postgres right? [20:23] thisfred, I imagine so, yes :) [20:24] so the vaproxy could be a simple (sic) twisted process, or a django view [20:24] * CardinalFang votes for "vaporoxy" [20:24] thisfred, I can think of two ways of doing this: 1. have the proxy be a program, which accesses postgres, works it out, and returns a 301 redirect, 2. dump the data to a HUGE RewriteMap, and have apache do it, which might be more efficient [20:25] if we're using Apache [20:25] I like 2 [20:25] yeah, the problem comes with: how often do you re-dump it? [20:26] If maps are smartly written, then good with 2. Apache is correct, not fast, so it may be dumb iteration searching. [20:28] I'm thinking that, at least initially, a small twisted app is better. Optimise later. [20:31] * CardinalFang talks to his dog about moving the computation out to the user, by putting the ID in at pair time, and using a specific hostname that we need only check. So, I'd connect to cmiller.hashedSecretAndUsernameWeCanCheckWithoutDb.ubuntuone.com . [20:31] * CardinalFang 's dog hates the idea. [20:31] Stupid dog. [20:38] oi [20:38] your dog is wisdom [20:41] I wasn't talking about authentication, just the multiplexing to databases. All addresses point to same box. SHA224 is cheaper than DB lookup. [20:42] CardinalFang, I don't get how it'd work [20:44] CardinalFang, also, it's going to be quite handy for later on when there's web access to your couch in the cloud to be able to tell everyone, the place you point at is ubuntuone.com/yourdatabase without any identifying features at all, so the instructions are identical for everyone [20:45] aquarius, Throw out the hash bit. It's not important. Instead, put the username in the hostname we connect to. HTTP 1.1 gets the hostname. FIrst part of hostname is database that we'll prefix with. The rest is normal, except we don't need to map the oauth key to the username. [20:45] CardinalFang, the problem is that the desktop doesn't *know* the username [20:45] oh nice [20:45] python crashed :( [20:46] aquarius, Right, but it can. Pairing is once. Replication is hundreds of times per day. [20:46] CardinalFang, that's broadly the same thing as db_prefix, then. :) [20:47] CardinalFang, and it can know the username if we set up an oauthable URI which returns the username, which means that you need to be online, and you need to do an http request -- admittedly it saves us a DB hit per replication, though [20:47] aquarius, Yeah, but an idiot user messing around in the database can't hose up their network with one-new-database per replication cycle with this. [20:48] aquarius, my dog just bit me, so I'm giving up this idea. It's probably dumb anyway. [20:49] * aquarius grins [20:52] ok, so...records uses oauth (can't test without couch patch, being looked at by jchris). pairing storing oauth tokens (being done by CardinalFang). daemon to start continuous replication requests (being done by CardinalFang). set up pairing with U1 (done by me, branch up for review). vaporoxy to make U1 pairing work (not being done by anybody atm). parse couch logfile for port (being done by thisfred). Is there anything else that needs doing for a [20:52] DC 0.3 release? [20:52] i.e., should I be working on something now, or can I stop for today? [20:53] aquarius: I think you can stop. parse couchfile almost done. I think it works, but it broke all my test hackery to not use the user's db [20:54] also the new contacts picker's tests used the user' [20:54] s db once again [20:54] thisfred, so you're happy to catch up with jchris about the ini-file bug? [20:54] aquarius: yeah, I'll ask him to ping me [20:54] aquarius, Eh, I think that's all. Good night! [20:55] when there's something to test [20:55] aquarius: have an excellent night! [20:55] kk. thanks, all. Sorry I didn't get more done. [20:55] yeah, slacker [20:55] :P [20:55] :) === __till is now known as till [21:45] CardinalFang: port discovery from logfile branch propoz0rd [21:48] thisfred, that log file may be huge. [21:49] CardinalFang: yep, but it's being read on the user's machine [21:49] but a suggestion to make it more efficient is very welcome [21:49] I just attacked it very high level first [21:50] seeking in the file is an option [21:50] I have code that does something similar in cloud_server (which I was about to replace) [21:51] thisfred, hrm, how about "for line in log_file" and "if line.startswith(Apache CouchDB has started on http://'):" and no "break" -- catch all port lines, and last one wins. [21:51] anyone know of a way to make readline/readlines behave lazily when reading from the end backward? [21:51] or something like it [21:51] I do not. [21:52] CardinalFang: unfortunately the lines do not start with that, but yeah, I can do that [21:52] thanks [21:53] startswith() is O(1), and "in" is O(n), I'm sure. [21:54] thisfred, if you're okay with the read-the-whole-file thing, then just a big string and rfind() should work. [21:54] CardinalFang: pushed [21:54] * statik goes on a desktopcouch merge and test rampage [21:55] CardinalFang: yeah, but all the lines start with non unique crap [21:55] Bah! [21:55] anyway in shouldn't take that long when the lines aren't long [21:56] I could do a regex [21:56] not sure if that is a big win though [21:56] sigh [21:57] It's not big enough to bother with. [21:59] thisfred, so, suppose you find nothing. Do you want to raise a ValueError? [22:00] CardinalFang: any error that we can catch, but a better suggestion than ValueError is also welcome [22:02] thisfred, well, the name of the function starts with "get_" so I wouldn't mind if it returned None on finding nothing. [22:02] also I only catch IOError now [22:02] yeah, perhaps that is fine [22:04] Do we need get_pid() any more? It's a strange way to make sure the db is running if we don't need the PID value. [22:04] i think it uses that to poke through proc to get the port [22:05] but i guess if that's fixed with some dbus thing upstream or whatever now, it's not needed [22:05] * CardinalFang boggles. [22:05] pushed fixed [22:06] CardinalFang: that's what I was saying: we don't [22:06] CardinalFang: except we need a way to detect that the log is stale and/or the couchdb isn't running [22:10] if we can think of a way to do that with less complexity then great, but all I could think of was while tries < 1000: urllib2.urlopen(uri:port_read_from_log) which may be less complex but also less good [22:12] thisfred, okay, let's not invent problems. This looks good to me. If we hear of problems, we can address them. [22:13] CardinalFang: sounds good to me. Let someone else invent the problems for once! ;) [22:14] Geez, I was hoping to get further today. I have only one karmic machine so far so hard to test the zeroconf stuff. /me eyeballs jblount and pfibiger for tomorrow. [22:17] * thisfred is debating whether he has the skillz to start on the vaporoxy. [22:17] I don't have a dog to tell me "no stupid" [22:18] although as luck would have it, that could change tonight, as a very unsaynoableto puppy was found by one of our neighbors last week, and she still hasn't been claimed [22:21] thisfred, the proxy isn't needed for feature freeze is it, since it's on the server side? [22:21] thisfred, i was kinda hoping we could do it with something as simple as mod_rewrite [22:21] since the db name is in the URL [22:22] are we going to be doing the replication back to ubuntuone.com over SSL? [22:24] (if you answer no, you're fired) [22:24] :) [22:24] NO! [22:24] maybe [22:24] sounds like a good idea [22:25] i would hope so [22:25] in the sense that replicating all my contacts' information over plaintext sounds like a bad idea :) [22:26] statik: we need to decide the urls, because depending on what they look like we need the client side to know the user's u1 account id [22:26] shouldn't [22:27] you should be able to just point at a script on the server and have it redirect based on the OAuth authorization [22:27] thisfred, the reason i was asking about SSL is because i'm unsure whether desktopcouch/python-couchdb supports SSL properly [22:27] and then just point cllient at https://ubuntuone.com/couch/dbname [22:28] right, should be able to, so is that our choice? (I'd vote for that, but I'm not sure what implications it has) [22:28] so the proxy needs to rewrite the URL based on the user [22:28] i vote for it [22:28] and i think aquarius voted for it earlier [22:29] statik: since aquarius made couchdb-python speak oauth, I'm optimistic: it exposes the httplib internals pretty much all the way. [22:29] i guess that means the proxy needs to read the oauth signature on the request, so we can't do it in plain old apache mod_rewrite [22:31] thisfred, so i want to use a separate subdomain so that we have more control over how to route the requests and the SLA on the load balancers/proxies can be different from the main website, but i'm voting for what you said [22:31] statik: I don't think we use couchdb-python for the replication, so if couchdb itself can handle ssl replication, we're good [22:32] we're probably in trouble then [22:32] on the server side we will use apache as a proxy to handle the SSL unwrapping [22:32] but something on the client side needs to be able to speak SSL [22:33] statik: well, python-couchdb is not a candidate for that, since it always talks to couchdb, and not another python-couchdb on the other end [22:33] couchdb doesn't do SSL? [22:34] that seems like fail [22:34] statik: ah you mean to do the SSL wrapping, rather than unwrapping [22:35] thisfred, yes. i see what you mean about python-couchdb not being in that part of the replication path though [22:35] that we could (ab)use couchdb-python for. I was thinking what about all the replication with other couches than the cloud [22:35] i guess maybe thats handled by ibrowse, and it does handle ssl after all - it's what is causing the export restriction notification to be filed for couchb [22:41] * thisfred pages through couchdb mailing list archives [22:47] Can someone confirm for me that the "stop sharing" button in the info popup doesn't work? [22:49] statik: ibrowse has ssl support, but couchdb doesn't use it for anything other than uuid generation. Couchdb leaves ssl to the proxy like a lot of other things. [22:50] statik: so we can maybe make it work by subverting couchdb-python for the client<->cloud replication, but client<->client not so much [22:50] and we break the cloud is just another paired couchdb a little more [22:50] and kittens cry [22:51] yeah, we should support ssl between client<->client too [22:51] but bad luck to the kittens, if we can do it [23:15] apache on every client won't fly, probably [23:16] Good night, all. [23:17] later monseigneur Fang