[03:27] hmm.. if I change part of file is the whole going to be reuploaded or only the part of it? [08:56] hi duanedesign [09:02] ok, time to write diagnostic script for .cache/ubuntuone/ file permissions [09:22] bug 491950 - added to ubuntuone-client-diagnose [09:22] Launchpad bug 491950 in ubuntuone-client "100% cpu usage calling calcuate_hash on same file over and over (dup-of: 407762)" [High,Triaged] https://launchpad.net/bugs/491950 [09:22] Launchpad bug 407762 in ubuntuone-client "ubuntuone-syncdaemon maxes out system resources when trying to hash a file with root permissions" [Medium,Triaged] https://launchpad.net/bugs/407762 === Urda_ is now known as Urda [12:20] hi, is it possible to make certain files public? [12:21] sebi`, it will soon. The feature is mostly done, we're figuring out some deployment details [12:21] stay tuned! [12:21] oh, darn [12:22] but nice to hear it's going to be implemented \o/ [12:23] also, it might sound silly, but will there ever be a non-linux client, as in, for windows XP, or Mac for example? (please don't hurt me for asking that :P) [12:24] sebi`, sure, there is an on-going effort to build a windows client [12:24] there will be a sprint during pycon [12:24] neat-o [12:24] :D [12:25] sebi`, we're anticipating all your needs [12:25] what else you got? :) [12:25] nothing so far :D [12:25] but thank you for your time [12:26] you're welcome [12:41] ok, I guess we'll need a script that parses the syncdaemon.log of current version to find what state syncdaemon is, since "About a month ago Ubuntu One just stopped working." requires syncdaemon.log to find possible reason. [12:42] rye, do it! [12:43] beuno, thinking on how to do that atm... [12:43] :) [12:43] rye, verterok is always a good person to talk to when you want to do funky things, he's an ex-gentoo user ;) [12:43] i just hate repeating "Please upload syncdaemon log", scan it, only to find that it is in STANDOFF_WAITING_FOR_THE_END_OF_THE_WORLD [12:47] * verterok looks [12:48] verterok, erm, no, it is just an ongoing conversation, nothing to look on so far [12:50] rye: there are other ways to know the state of syncdaemon :) [12:50] rye: e.g: dbus-send --session --print-reply --dest=com.ubuntuone.SyncDaemon --type=method_call /status com.ubuntuone.SyncDaemon.Status.current_status [12:50] rye: that should work in all ubuntu installs :) [12:54] verterok, oh, thanks! That is nice thing, I want to be able to parse log files so that there is clear sequence of events (file spotted, file uploaded, uuid is that one), state changed @ 1 AM, error occured, etc. :) [12:54] oh [12:54] i know i know i know [12:55] syncdaemon needs some method to enable debugging right _now_. [12:55] rye: parsing the config is a good idea [12:55] rye: a method? at runtime? [12:55] verterok, yup [12:56] verterok, i.e. i have a bug. syncdaemon does not want to upload files. we instruct the user to enable debug, add more files, then package this into a nice fine tarball and send to us, if this does not help, then restart syncdaemon. it may not be thaaat useful, though. [13:00] rye: could you file a bug about this requirement? :) [13:11] verterok, which exactly? runtime debug or log parser? :) [13:11] rye: runtime debug :) [13:12] rye: regarding the log parser, maybe facundobatista alsready have some scripts :) [13:13] verterok, btw, end users might also want to see what's happening behind the scenes w/o enormous numbers of uuids that don't give much info about filename unless one scans through the logs with a magnifier (or grep) ... [13:14] rye, what you said is logical: people want to see what's going on with the client [13:14] rye, how many files already uploaded/downloaded, how many are still in the queue [13:14] rye, at which speed everything is working, etc [13:14] facundobatista, frankly speaking I'd like to know what's actually happening too :) [13:15] and what Blocked(filename) means... it all is in the sources... thought [13:15] though [13:15] rye, however, referring the user to the logs it's a poor solution: we should provide a GUI for that [13:15] facundobatista, yep, to show the current, aaand a log reader to show the past :) [13:17] rye, mmm [13:17] btw, is anybody here running syncdaemon enabled on session login? [13:17] rye, let's take for example a torrent client... how many times did you want to see the past? [13:17] it looks like syncdaemon eats my harddrive when it starts leading to an increased waiting time... [13:18] rye, in any case... I don't have any scripts for the logs [13:18] facundobatista, ok, but if the torrent client removed the entries for already downloaded file then there should have been some kind of logs [13:18] erm [13:19] rye, or it shouldn't remove the entries for already downloaded files ;) [13:19] torrents not apply here - they stay seeding forever so the only thing that is important is the speed charts, to show that to ISP and convince them that they really have a shaper in place :) [13:19] but files are uploading and we forget about them [13:19] facundobatista, in any way, runtime debug switch seems to be easier to implement [13:20] rye, yeap [13:20] facundobatista, rye, I hear quickly is great for building quick UIs [13:20] rye, anyway... for GUI enhancements you should talk with Chipaca or beuno [13:20] rye, I just say that logs shouldn't be the way to see what's going on for final users [13:23] facundobatista, beuno, verterok, ok so back to syncdaemon running LR on login - have you experienced any elevated hdd activity and gdm-login-to-usable-session time? [13:23] rye: it's a known issue that LR takes a lot of time in some cases [13:23] rye, I showed ubuntu devs my bootchart graph, and they all cursed at u1 [13:24] grrr [13:24] not good [13:24] my session login takes up to 1 minute [13:24] w/ syncdaemon eating ( "D" ) my drive. [13:25] oh, my boot time is about 11 seconds from power on to desktop :) [13:25] beuno, rye: that's the applet/nautilus starting syncdaemon earlier than they should [13:26] verterok, ok, what about trunk? [13:26] rye: you can't start syncdaemon on login using trunk :) [13:26] rye: there is no applet [13:26] verterok, I can :) [13:26] verterok, so what? [13:26] rye: hehe, yes you can. [13:26] verterok, applet is no longer the launcher :) [13:27] rye: who is launching it now? [13:27] verterok, rye, https://devpad.canonical.com/~beuno/beuno-laptop-lucid-20100212-2.png [13:27] verterok, u1sdtool can launch it but even when syncdaemon itself is launched it starts doing LR which greatly impacts the ability to do something before it calms down [13:28] rye: yes [13:28] * rye becomes sad because he has 75 seconds of bootcharting [13:28] rye: ATM, there is no way to avoid the LR at startup [13:29] verterok, but that can be read in chunks, much like js calls are chained so that browser does not get too sad about executing all that 12 billion lines at once [13:30] verterok, freeing the hd for some time to perform the task... [13:32] rye: it's being done in chunks :/ [13:33] rye: one of the options is to change the filesystem manager layer to make it thread safe, so we can do the LR in a different thread [13:34] verterok, i trapped on bug 409972 - could that improve anything? By reading one file instead of tons of small open()s and read()s ? [13:34] Launchpad bug 409972 in ubuntuone-client "Use cPickle hack to speed up syncdaemon start" [Medium,Triaged] https://launchpad.net/bugs/409972 [13:34] rye: no, that's a different issue [13:34] verterok, LR is local rescan, not metadata loading.... [13:35] grrr [13:35] i got it [13:35] rye: on startup, syncdaemon builds a data structure that holds all the sync states/transitions [13:35] local rescan is traversal of all the files and folder to say "hi, file!" [13:35] rye: that bug is to pickle that data structure on the first startup, to speedup the next one [13:35] rye: actually local rescan traverse all files *and* loads metadata [13:36] rye: the issue with the startup isn't local rescan actually, it's the metadata loading of filesyetem manager [13:36] rye: FilesystemManager builds an index on startup, and needs to read all the metadata to do it :( [13:37] verterok, With all the buzz around improving startup time we forget that there's still time-to-usable-desktop and that is heavily impacted by running anything disk-intensive right at startup... [13:39] rye: that's the only block-until-finish bit on syncdaemon startup [13:39] verterok, one seems to be enough :) [13:40] rye: indeed [13:40] rye: and we are awere of this problem [13:40] rye: I did a little experiment, and replaced the current metadata storage with sqlite [13:40] rye: the result wasn't a big improvement :( [13:44] beuno, was that lucid install mentioned in bootchart a clean one or upgrade from Karmic ? [13:45] rye, upgrade from karmic to lucid [13:45] after a tweak [13:45] rye, #476711 could improve LR timings [13:45] I got it 10 seconds down [13:45] http://launchpadlibrarian.net/39214382/beuno-laptop-lucid-20100213-2.png [13:45] beuno, any problems w/ ureadahead not generating pack files? [13:45] rye, not atm, no [13:45] also, I have an SSD HD [13:46] facundobatista, ah, stat() on every file found? [13:47] beuno, still, my upgrade went not that smooth, i don't have plymouth displaying nice things, ureadahead is broken and tmp is not cleaned on start (IT SHOULD BE TMPFS!) [13:47] rye, yeap [13:48] rye, wierd. It was flawless for me [13:49] beuno, thanks for SSD hint, is your /tmp in memory or on disk? [13:50] rye, disk [13:55] hey mattgriffin [13:57] rodrigo_, ping [13:58] I'm working on the notes branch [13:58] and I have a questionfor you when you get back [14:06] rodrigo_, ok, so notes are saving now. Just need to make the flow nice and fix quirks: lp:~beuno/ubuntuone-servers/dont-autosave [14:07] I'm interested in something you said about revisions and not sending them creating problems [14:16] aquarius, maybe you can shed some light on it ^ [14:33] beuno, cool! [14:34] beuno, the problem is that when saving notes yto couchdb, you need to specify the _rev you are updating [14:35] beuno, and the autosave code didn't take them into account [14:35] beuno, so if there is no autosave, it should be ok, just sending the POST when the user pressed 'Save' should do it correctly [14:38] rodrigo_, could you enlighten me on why the manual save won't cause this? [14:39] beuno, because the autosave code doesn't know the ID of the document AFAIR [14:39] the _rev, sorry [14:40] rodrigo_, so this is what I did to make it work: https://pastebin.canonical.com/27865/ [14:41] beuno, not sure if you need the format=json part? but if it works, it looks good to me :-) [14:41] beuno, have you submitted the branch for merging? [14:41] rodrigo_, not yet, it needs a lot of clean up [14:41] rodrigo_, the json bit [14:41] is because otherwise the response is an HTML [14:41] ah, ok [14:41] and everything goes BOOM [14:42] I need to let you go into edit mode, make the buttons pretty, and maybe a spinner while it saves :) [14:42] how urgent is this? [14:42] I can split it into 2 branches [14:42] if you can, that would be great [14:43] will do [14:43] it's a bit urgent, because the web UI is broken [14:43] but if you want to have all in 1 branch, that's ok with me [14:43] rodrigo_, I'll see how time consuming it is [14:43] start with the basics [14:43] and cut off when I get to a reasonable amount of time spent :) [14:43] ok :) [14:43] thanks! [14:44] de nada! [14:44] good thing its a holiday in the US today [14:44] gave me an extra day before going knee-deep into phone sync [14:44] hehe [14:53] rodrigo_, is this the traceback we're trying to fix with this? https://pastebin.canonical.com/27866/ [14:54] beuno, no, but if you get that, I guess we should fix it [14:54] rodrigo_, I get that with the new implementation :) [14:54] beuno, I can fix it if you want, it needs to 'if "Tomboy" in annotations [14:55] beuno, right, I changed the code to use dc's get_application_annotations, which doesn't have the has_key [14:55] rodrigo_, if you can throw a quick patch at me, it would be great [14:55] beuno, but since that part wasn't working, I didn't change it it seems [14:55] or change in your branch and I'll merge [14:55] beuno, ok, let me see [14:58] http://www.youtube.com/results?search_query=ubuntuone&search_type=&aq=f [14:58] beuno, ok, patched (but not tested) in my branch, so merge it [14:58] interesting [14:58] rodrigo_, merging, will loet you know in a sec [14:59] beuno, ok [15:00] rodrigo_, new error! https://pastebin.canonical.com/27868/ [15:01] hmm [15:02] rodrigo_, that's missing stuff [15:02] one sec [15:03] ah [15:03] rodrigo_, https://pastebin.canonical.com/27869/ [15:05] hmm, I'm lost, that should work :-( [15:06] beuno, see line 426, that works, so not sure what the difference is?? [15:07] hrm [15:09] rodrigo_, that does: if existing_doc is not None and "_rev" in existing_doc._data: [15:10] I wonder if that check makes the diff? [15:11] that just adds the _rev field if it's an update [15:11] but the error says a DC.Record is not serializable [15:11] right [15:11] I get that error when editing and saving [15:11] aquarius or thisfred should know better what the error means, since it's in DC code [15:11] so it probably should add the rev as well, no? [15:12] (even if its unrelated) [15:12] but the 2 calls to .put_record use a python list [15:12] beuno, if it's a new document, no, you can't make up a revision [15:12] couchdb will assign it automatically [15:12] aha [15:12] ok, good to know === jamlata-afk is now known as jamalta [15:17] so, to put_record you pass a Record(list values) and internally DC converts that to a Record, and so it shouldn't be serializing it if it can't be serialized [15:17] not sure what's going on, sorry :-( [15:17] the 2 calls to put_record seem to use the same format [15:17] I can't figure it out either :/ [15:18] aquarius, ^^ [15:18] * aquarius reads the backscroll [15:18] aquarius, the meat of it is: https://pastebin.canonical.com/27869/ [15:18] wtf? [15:18] on saving an edited note [15:18] how have you created a nonserialisable dictionary? :) [15:19] aquarius, https://pastebin.canonical.com/27869/ [15:19] aquarius, I'm special that way :) [15:19] aquarius, see lib/u1/web/notes/views.py#161 [15:19] beuno, can you pastebin the value of doc just before line 161 in web/notes/views.py? [15:20] sure [15:20] beuno, you stuffed mergeable list that you've received originally into the record? [15:21] aquarius, where would print statements be outputted to? [15:22] rye_, I haven't changed anything in python [15:22] beuno, no idea. I always do raise Exception(str(doc)) :-) [15:22] beuno, aquarius - the same at client side: http://paste.ubuntu.com/376910/ [15:22] so it IS possible, i have filed a bug, searching... [15:23] rye_, hang on, what's that paste? [15:23] rye_, ah, that replicates the problem. cheers. [15:23] aquarius, nice trick: https://pastebin.canonical.com/27871/ [15:24] aquarius, bug 510223, though you are already using the server code... [15:24] Launchpad bug 510223 in desktopcouch "The new version of update_fields() should be backported from the server code" [Medium,Fix committed] https://launchpad.net/bugs/510223 [15:24] weird [15:24] thisfred? [15:24] CardinalFang? [15:24] no thisfred [15:24] CardinalFang, ping? [15:24] I don't know much about update_fields [15:25] aquarius, hmm, so the problem is the doc["application_annotations"] = record.application_annotations? [15:25] aquarius, I guess that needs to be converted to a list? [15:25] thisfred has told me to do... wait, searching... [15:26] something odd about that. It might not be possible to ship a_a around that way [15:26] aquarius, hi. USians have holiday today. I've seen that bug, and I don't understand it yet. I want to compare the code in py couchdb mod. [15:26] old_record.application_annotations["Tomboy"] = application_annotations [15:26] record_id = CouchDbNote.notes_db.put_record(old_record) [15:27] so no update_fields but my application_annotations is just a regular python dict [15:27] CardinalFang, oh, sorry, pal, didn't realise it was a holiday! [15:29] * beuno fixes js while the experts figure this out [15:30] Jan 20 20:11:54 so it needs to be a dictionary :( [15:30] Jan 20 20:12:02 you can do this: [15:30] Jan 20 20:12:05 sorry, http://paste.ubuntu.com/359656/ [15:30] Jan 20 20:12:20 db.update_fields(record_id, { [15:30] Jan 20 20:12:20 "application_annotations": application_annotations._data }) [15:30] Jan 20 20:12:40 but boy that method stinks [15:30] hrm [15:31] I guess it's better then to do doc["applications_annotations"] = record.application_annotations._data ? [15:31] and then just the put_record(Record(data=doc)) ? [15:32] db.update_fields(record_id, { "application_annotations": application_annotations._data }) should work [15:32] having just tested it [15:32] aquarius, but we'll have to list all the fields [15:32] mark it with a big UGH THIS IS A WORKAROUND nastiness :) [15:33] and wouldn't doc["applications_annotations"] = record.application_annotations._data work also? [15:33] * beuno is prepared to merge [15:33] go beuno go! :D [15:35] merging! [15:35] man [15:35] working on U1 is *so* much more fun than launchpad... [15:35] you can actually get something done in an hour! [15:35] rodrigo_, nothing to merge! [15:36] beuno, ah, you meant merging from my branch? [15:36] let me change that then [15:36] rodrigo_, anywhere! [15:36] as long as it has the fix :) [15:37] is it possible for syncdaemon to be running as root? [15:38] beuno, ok, merge now [15:38] i see the config entry im_ok_with_being_root_pretty_please_let_me_be_root.default = False and i think whether it does anything... [15:39] rodrigo_, works [15:39] ! [15:39] beuno, cool! [15:39] thank you rodrigo_, aquarius, rye_ [15:39] on with it === rye_ is now known as rye [15:40] beuno, let me know when you propose the branch, and I'll review it [15:42] rodrigo_, will do, I need to fix a few things on the javascript side first so people don't hav to use firebug to edit notes :) [15:42] re "UGH THIS IS A WORKAROUND". I remember having seen "This should not be done this way, it needs to be rewritten ASAP!" in a file that had this since 2004. Nobody knew what was actually wrong there so it was left as is :) [15:42] rye, yeah, comments you never revisit don't work :-) [15:43] ok, I'm off to pay the rent. I'll be back in 30-40' [15:43] rye, if you put "FIXME" then pylint bitches at you about it, which is why I do it :) === beuno is now known as beuno-lunch [15:44] Hi all. I posted quite a lot of messages on a bug this weekend. I would like some feedback/advice on it. [15:44] (https://bugs.launchpad.net/bugs/497143) [15:44] Ubuntu bug 497143 in desktopcouch "CouchDatabase.execute_view should take an optional extra parameters dictionary" [Low,In progress] [15:45] It is basically a minor correction to a solution by aquarius [15:45] rodrigo_, I will finish this today, I need to be free tomorrow for phone sync ;) [15:51] beuno-lunch, ok, if you finish the js, I can continue on the othjer issues, if you haven't done them [15:57] aquarius: we don't use pylint on the client, and pyflakes doesn't complain [16:02] why oh why rhythmbox, did you stick some of my quite old mp3s in "recently added"? === beuno-lunch is now known as beuno [16:53] rodrigo_, sure, I think it's just js now [16:53] will let you know as I progress [16:54] ok [17:28] rodrigo_, bug 494322 and 501020 feel like dupes [17:28] Launchpad bug 494322 in ubuntuone-servers "editing notes in ubuntu one web interface doesn't have a "Save" button" [Low,In progress] https://launchpad.net/bugs/494322 [17:28] Launchpad bug 501020 in ubuntuone-servers "Tomboy note sync and web-ui for notes are broken" [High,Confirmed] https://launchpad.net/bugs/501020 [17:28] what's your take? [17:29] beuno, they are indeed [17:29] I will do the honors [18:08] * rye is slow today, will shutdown now to prevent damage to the source code [18:14] rodrigo_, getting closer [18:14] need to figure out why the save call returns HTML [18:14] and I think we have a working notes web UI which saves [18:50] rodrigo_, got a minute? [18:51] I've traced the remaining issue down to a URL not returning json when it should [18:52] aquarius, maybe you can help as well :) === teknico is now known as teknico_away [19:24] rodrigo_, branch proposed! [20:29] http://www.facebook.com/album.php?aid=157659&id=122313196726&saved#!/photo.php?pid=4039110&id=122313196726 [20:29] whoot. [20:31] dobey, look at that! one step closer from uninstalling Firefox! [20:32] too bad I'm not working on a facebook app [20:32] one step at a time [20:33] Well, with FB, I don't also have to build the API. It's already mostly all there. :) [20:51] beuno, cool, reviewing it now [20:51] rodrigo_, thanks. I'm unsure about a specific part of it, I mentioned it in the MP [20:53] rodrigo_, the branch went deeper than I expected, I can tell you that [20:54] :) [20:55] it also made me realize I need to spend 2 or 3 days cleaning up CSS, javascript files and templates [20:55] they're all clustered into sections [20:55] the app is still small enough to fix it in a reasonable time frame [20:55] in 6 months, no way [20:56] * beuno eyeballs jblount to pair with him in the neat future [21:08] beuno, approved! [21:09] rodrigo_, woooo! thanks, on its way to PQM [21:10] beuno, ping some losa when it gets merged, so that it gets deployed asap [21:10] rodrigo_, this will need to get cherrypicked into production? [21:11] beuno, not sure what the process is, it depends what other revisions there are before [21:11] beuno, so better ask a losa :-) [21:11] rodrigo_, will do [21:15] rodrigo_, what if we let it roll out to edge first, test it, then cherry pick? [21:15] beuno, yeah, sounds good [21:15] or is production broken enough that whatever will be better? [21:15] super [21:15] well, web ui editing of notes is completely broken [21:16] but going through edge should be ok, it gets rolled every 4 hours there, I think [21:16] yeah, I'll let all the moving parts settle, we should have it on edge tomorrow, and have statik and other around to help us with the cherrypick [21:18] ok === jamalta is now known as jamalta-afk === jamalta-afk is now known as jamalta === jamalta is now known as jamalta-afk