[09:37] <popey> morning ubuntuoners
[10:02] <duanedesign> morning popey
[10:16] <rye> mornings!
[10:19] <duanedesign> hello sir. Good weekend I hope
[12:01] <duanedesign> /8/9
[12:02] <duanedesign> oops
[12:02] <rye> duanedesign, 0
[12:04] <rye> So, I'm testing user designated folders. That means that you can get any folder in your /home and say "I want to have this in Ubuntu One". The syncdaemon supports this, the web ui does not support it at the moment.
[12:04]  * rye checks what happens next ...
[12:11] <popey> rye: does it share all folders below the one you choose or just the folder you select?
[12:11] <rye> popey, all the hierarchy, I believe
[12:11] <popey> cool
[12:11]  * popey would be interested in testing that
[12:12] <kklimonda> is the gstreamer0.10-fluendo-mp3 package going to be moved to main now that the music store depends on it?
[12:14] <rye> popey, this is also a pre-requisite to "publish the file" so that it is accessible via HTTP by everyone
[12:14] <rye> testing that as well
[12:14] <popey> neat
[12:20] <duanedesign> rye: Is the pyinotify error followed by gibberish in syncdaemon.exception.log telling us anything?
[12:21] <duanedesign> EX: http://paste.ubuntu.com/386257/
[12:22] <rye> duanedesign, grab that and cat it
[12:22] <rye> 2010-02-18 08:52:41,543 - pyinotify - ERROR - The path /home/damns/Ubuntu One/Ð½Ð¾Ð²Ð°Ñ Ð¿Ð°Ð¿ÐºÐ° of this watch <Watch wd=2 mask=3064 auto_add=False proc_fun=None path=/home/damns/Ubuntu One/Ð½Ð¾Ð²Ð°Ñ Ð¿Ð°Ð¿ÐºÐ° dir=True > must not be trusted anymore
[12:22] <rye> 2010-02-18 21:19:57,259 - pyinotify - ERROR - The path /home/damns/Ubuntu One/Ð½Ð¾Ð²Ð°Ñ Ð¿Ð°Ð¿ÐºÐ°-wrong-path of this watch <Watch wd=2 mask=3064 auto_add=False proc_fun=None path=/home/damns/Ubuntu One/Ð½Ð¾Ð²Ð°Ñ Ð¿Ð°Ð¿ÐºÐ°-wrong-path dir=True > must not be trusted anymore
[12:22] <rye> and this is where we need someone who knows what that means
[12:23] <duanedesign> aha
[12:23]  * rye summoned someone... waiting for someone to appear
[12:44] <rye> https://answers.edge.launchpad.net/ubuntuone-client/+question/102820 ? - Previously saved files have disappeared ?
[12:46] <duanedesign> rye: that reminds me. I wanted to ask about. https://answers.edge.launchpad.net/ubuntuone-client/+question/102607
[12:47] <duanedesign> was unsure the correct answer on that one
[12:47] <rye> duanedesign, yes, for now the answer - syncdaemon will panic and write No space left on device to the logs until it fills up the remaining space with the logs
[12:59] <rye> duanedesign, that's why i linked the question to it
[13:35] <webm0nk3y> pgraner ping
[13:35] <pgraner> webm0nk3y: hello
[13:35] <webm0nk3y> pgraner, hi. you filed #528698?
[13:36] <webm0nk3y> bug #528698
[13:36] <pgraner> webm0nk3y: yep
[13:36] <webm0nk3y> there's mup
[13:36] <webm0nk3y> hehe
[13:36] <webm0nk3y> pgraner, need some more details
[13:36] <pgraner> webm0nk3y: sure
[13:36] <webm0nk3y> pgraner, in the Web UI what does it show as far as storage space used?
[13:37] <facundobatista> pgraner, question: did you have two or three machines involved in the incident?
[13:38] <webm0nk3y> pgraner, can you tell you have our *complete* attention? :)
[13:38] <pgraner> webm0nk3y: 251.2 KB Used (0.0%) today, on Friday it was 0 KB Used (0.0%)
[13:39] <webm0nk3y> pgraner, ok, so it sounds like it is syncing new files right?
[13:39] <pgraner> facundobatista: There are 3 machines, I had two previously set up and running (zorak & desktop), I was trying to add a third on Fri (air)
[13:39] <pgraner> webm0nk3y: nope I have killed the sync daemon from all my boxes
[13:40] <facundobatista> pgraner, you gave me logs about the first one and the laptop
[13:40] <webm0nk3y> pgraner, k
[13:40] <facundobatista> pgraner, I would need logs from the second machine to discard the client in the equation
[13:40] <pgraner> facundobatista: I gave you the ones from desktop & air
[13:40] <webm0nk3y> facundobatista, I'm going to setup a new machine and see if I have similar issues
[13:40] <pgraner> facundobatista: ok standby
[13:40] <facundobatista> pgraner, thanks
[13:50] <pgraner> webm0nk3y, facundobatista: bug updated with logs and info
[13:50] <webm0nk3y> pgraner, thank you
[13:52] <facundobatista> pgraner, thans!
[13:52] <facundobatista> *thanks
[14:30] <webm0nk3y> pgraner, are you positive you logged in with the same account when you setup the new computer?
[14:31] <pgraner> webm0nk3y: yep I only have *one* U1 account and its all done with openid
[14:31] <pgraner> webm0nk3y: ie. I only have one openid account which is my canonical email addr
[14:32] <webm0nk3y> pgraner, did any local files on your other machines get deleted?
[14:33] <pgraner> webm0nk3y: the only onces got deleted on desktop,  zorak was powered off (its my netbook) and when I brought it up I killed syncdaemon so it couldn't do any damage
[14:33] <webm0nk3y> pgraner, and I assume all files got deleted?
[14:34] <webm0nk3y> pgraner, in the bug report, you mentioned that directories showed up. Can you explain that a little more?
[14:34] <pgraner> webm0nk3y: When I saw the notifier pop up and say it was syncing files and I new there were no changes, I looked on the local system and saw things going byebye, I killed the syncdaemon
[14:35] <webm0nk3y> pgraner, ok
[14:35] <webm0nk3y> pgraner, did you share your root folder with anyone?
[14:35] <pgraner> webm0nk3y: When I went to add air (the 3rd machine) after adding the computer to U1, it took me to the web interface, thats when I noticed I had a directory hierarchy but no files in the dirs
[14:36] <pgraner> webm0nk3y: Hell NO, this is my working canonical data
[14:36] <webm0nk3y> pgraner, ahh...
[14:46] <webm0nk3y> pgraner, when you got to https://one.ubuntu.com/account/machines/, which machines show up?
[14:46] <pgraner> webm0nk3y:
[14:47] <pgraner> Computer	Date Added
[14:47] <pgraner>   air (tt3KVTmRwgM8qRXh5PhN)	 26 Feb 2010
[14:47] <pgraner>   desktop (0CbL4P0jkQRj0GzTjNSw)	 17 Feb 2010
[14:47] <pgraner>   zorak (9K4HbMNXxGfb6R70FLm5)	 17 Feb 2010
[14:47] <webm0nk3y> pgraner, thanks
[14:54] <webm0nk3y> pgraner, when you setup air, this was a new machine right? not an upgrade and not on a computer that had Ubuntu One on it?
[14:59] <FND> aquarius: can I bug you for a minute? I understand the DesktopCouch database requires authentication - but it's not simple username/password, but fanc
[14:59] <aquarius> FND, yes
[14:59] <dobey> oauth
[15:00] <FND> *fancy stuff which isn't easy to reproduce independently? background is that I'm working on a "regular" CouchDB app but would wanna allow users to connect to DC, optionally
[15:00] <FND> has anyone done anything like this before?
[15:00] <dobey> it's not that fancy
[15:00] <dobey> :)
[15:01] <FND> I remember seeing discussions about python-keyring and stuff, which might have scared me off
[15:01] <aquarius> FND, is your app Python?
[15:01] <FND> yes
[15:01] <pgraner> webm0nk3y: yep new Lucid install
[15:02] <aquarius> FND, then the easiest thing to do is use desktopcouch.records.server. You don't have to use the records API, but desktopcouch.records.server takes care of providing roughly the python-couchdb interface and handling all teh oauth stuff for you.
[15:03] <FND> ah great - so I can do try: import desktopcouch.records.server except ImportError: import couchdb.server ?
[15:04] <FND> so it will work on a headless server with python-couchdb, on Ubuntu desktop with DC
[15:04] <Chipaca> MEETING BEGINS
[15:04] <Chipaca> me
[15:04] <CardinalFang> me
[15:04] <urbanape> me
[15:05] <teknico> me
[15:05] <dobey> me
[15:06] <vds> me
[15:07] <Chipaca> beuno: aquarius: rodrigo_: jblount: last call
[15:07] <beuno> me
[15:07] <rodrigo_> me
[15:07] <beuno> Chipaca, why not just ping us to start with?  :)
[15:07] <Chipaca> beuno: you should have MEETING BEGINS ping you :)
[15:08] <beuno> Chipaca, good idea!
[15:08]  * Chipaca rolls eyes
[15:08] <beuno> (typical manager delegating work)
[15:08]  * Chipaca is but an egg
[15:08] <teknico> delegating work to machines, isn't it all we do here?
[15:08] <Chipaca> DONE: lats of talk, little coding. Less than I would've liked on oauthdesktop.  TODO: *lots* more talk. Shove oauthdesktop off on somebody else if possible (bad manager, doing coding! bad!). Talk some more; if you're in desktop+ and haven't talked with me by EOBD, ping me.  BLOCKED: I refuse to be blocked  NEXT: CardinalFang
[15:09] <aquarius> me
[15:09] <CardinalFang> I wish xchat groked alert lines.  Mine alerts on "meeting" OR "begins".  Quite a pain.
[15:09] <teknico> (well, in addition to silly talk)
[15:09] <CardinalFang> DONE: fixed up and merged find-pid branch.  Merged get-port program branch.  Merged docs change that elucidates put_record mutation of argument.  Lots of merging!
[15:09] <CardinalFang> TODO: Help kenvandine get d-c packaged.
[15:09] <CardinalFang> BLOCKED: None.
[15:09] <CardinalFang> next, urbanape, winner of three gold medals!
[15:09] <urbanape> DONE: Finished phones/ ajax branch, submitted for review.
[15:09] <urbanape> TODO: Land it, start in on a new contacts/ branch.
[15:09] <urbanape> TOTO: I bless the rains down in Africa!
[15:09] <urbanape> BLOCK: None
[15:09] <urbanape> teknico: europe
[15:09] <teknico> DONE: reviewed and integrated another branch from beuno; looked into local sync problem
[15:09] <teknico> TODO: finish fixing the local sync problem; removing the ME contact from the contacts web ui; showing the web desktop+ guys around our phone sync code
[15:09] <teknico> BLOCK: none
[15:09] <teknico> next: dobey
[15:09] <teknico> urbanape, I rope?
[15:09] <dobey> ☺ DONE: Found/fixed bugs in StorageThrottlingFactory move, More CP button work
[15:10] <dobey> ☹ TODO: Finish branch to add Connect to CP, Finish the rest of the CP work
[15:10] <dobey> ☹ BLCK: None.
[15:10] <dobey> vds: che l'italiano dici?
[15:10] <vds> DONE: investigated on problems with web/phones/test_views, fixed funambol json connector, updated trunk to use new funambol_cared revno, investigated on zombie contact again but not had time to really focus on it
[15:10] <vds>  TODO: re-land patch_13, continue the invetigating on contact zombies
[15:10] <vds> BLOCKED: no
[15:10] <vds> beuno: please
[15:11] <beuno> DONE: Landed the better-phone-index branch, dupe detection and merging is about to land
[15:11] <beuno> TODO: Kill the me contact, fix the oopses related to it, figure out why I can no longer sync my phone on production
[15:11] <beuno> BLOCKED: No (finally!)
[15:11] <beuno> aquarius, bring it home
[15:11] <aquarius> DONE: numerous small music store things, like translatable names; phone meetings; apiservers rollout
[15:11] <aquarius> TODO: work with rodrigo on download progress; make library page links work; #oppdev talk
[15:11] <aquarius> BLOCKED: only got ten fingers
[15:11] <aquarius> rodrigo_, go for it
[15:11] <rodrigo_> • DONE: Vacation. Notes problems fixing. Music store downloads polling
[15:11] <rodrigo_> • TODO: Conflict resolver tool in pair tool. Make sandy's snowy test suite work with our server (http://git.gnome.org/cgit/snowy/tree/api/tests.py). Discuss with jdo and aquarius about oauth token per app, not per machine? Add jslint tests to check. & in note titles.
[15:11] <rodrigo_> • BLOCKED: no
[15:12] <rodrigo_> nobody next, right?
[15:12] <CardinalFang> aquarius, spacebar <= forehead
[15:12] <aquarius> CardinalFang, to quote the late great Eric Morecambe, the spaces between the notes might be the important thing, but you still need the notes
[15:14] <dobey> notes do help pay the rent.
[15:14] <jblount> DONE: Some html and css stuff for the music store, trying to figure out a way to better handle the phone sync instructions
[15:14] <jblount> TODO: Phone sync instructions parsing and learning. Keep Lernid on in the background to learn how to be a better developer during "Opportunistic Developers" week.
[15:14] <jblount> BLOCKED: Never!
[15:15] <dobey> vds: yeah, my italian sucks :)
[15:15] <vds> dobey: nope! :)
[15:15] <dobey> heh
[15:16] <dobey> that was valid italian?
[15:16] <Chipaca> MEETING ENDS. Thanks everybody!
[15:17] <teknico> aquarius, John Cage would disagree ;-)
[15:33] <facundobatista> pgraner, I may be looking to something strange...
[15:33] <facundobatista> pgraner, http://paste.ubuntu.com/386297/ and http://paste.ubuntu.com/386298/ seems to be equal
[15:33] <facundobatista> pgraner, do you have two equal logs in your disk?
[15:38] <pgraner> facundobatista: they are not the same: look at http://paste.ubuntu.com/386362 that is the ls -al of the log directory, they are a few bytes difference
[15:40] <pgraner> facundobatista: also it might be helpful if you guys put hostnames into the log files, it really gets confusing trying to figure out what log came from where
[15:42] <facundobatista> pgraner, they have similar sizes because they rotate per size
[15:43] <facundobatista> pgraner, the problem is that I don't have a relationship between your pastebin and the real log name
[15:45] <pgraner> facundobatista: pastebin 386297 == syncdaemon.log.2010-02-23_08-50-33 & pastebin 386298 == syncdaemon.log.2010-02-23_11-12-11
[15:47] <facundobatista> pgraner, ok
[15:49] <pgraner> facundobatista: they are easy to coordinate, I ran the following to get the pastebin entries: for file in sync*; do pastebinit -i $file -b http://paste.ubuntu.com; done
[15:50] <pgraner> facundobatista: so if you look at the list in the bug they are in order with the zero length files removed
[15:50] <facundobatista> pgraner, the problem is that "sync*" is disk order, not alphabetical one
[15:51] <CardinalFang> kenvandine,  https://edge.launchpad.net/desktopcouch/trunk/0.6.2
[15:51] <CardinalFang> kenvandine, new release of desktopcouch
[15:51] <pgraner> facundobatista: nope they came out exact
[15:51] <pgraner> pgraner@zorak:~/.cache/ubuntuone/log$ for file in sync*; do echo $file; done
[15:51] <pgraner> syncdaemon-exceptions.log
[15:51] <pgraner> syncdaemon-exceptions.log.2010-02-17_19-51-14
[15:51] <pgraner> syncdaemon-exceptions.log.2010-02-19_09-44-27
[15:51] <pgraner> syncdaemon-exceptions.log.2010-02-20_08-54-17
[15:51] <pgraner> syncdaemon-exceptions.log.2010-02-20_09-03-46
[15:51] <pgraner> syncdaemon-exceptions.log.2010-02-23_08-50-32
[15:51] <pgraner> syncdaemon.log
[15:51] <pgraner> syncdaemon.log.2010-02-23_08-50-33
[15:51] <pgraner> syncdaemon.log.2010-02-23_11-12-11
[15:51] <pgraner> syncdaemon.log.2010-02-23_12-25-35
[15:51] <pgraner> syncdaemon.log.2010-02-26_15-45-03
[15:51] <pgraner> syncdaemon.log.2010-02-27_15-46-43
[15:52] <pgraner> facundobatista: either way pastebinit returns the pastbin URL, just match them up throwing out the zero length file names and you'll have the proper matching
[15:52] <facundobatista> pgraner, thanks!
[15:53] <facundobatista> pgraner, question... I'm seeing strange stuff in these logs
[15:53] <facundobatista> pgraner, these files you once had in the server... when you uploaded them?
[15:53] <facundobatista> Feb-23, or before?
[15:53] <pgraner> facundobatista: before
[15:54] <facundobatista> pgraner, I'm looking into the files of your second machine, and they appear to be new in the disk
[15:54] <pgraner> facundobatista: I had desktop & zorak on karmic and did an upgrade to lucid which broke syncing
[15:54] <facundobatista> pgraner, I mean, the sytem is finding all the files new in the disk, not having metadata at all for them
[15:54] <pgraner> facundobatista: so I removed the two from ubuntu one
[15:54] <facundobatista> pgraner, how do you removed them?
[15:54] <pgraner> facundobatista: deleted the Ubuntu One directory
[15:55] <pgraner> facundobatista: readded the computer to U1
[15:55] <facundobatista> pgraner, how do you deleted the Ubuntu One directory?
[15:55] <pgraner> facundobatista: then synced, and it moved all the files from the server back down
[15:55] <facundobatista> pgraner, do you deleted the folder using nautilus? command line? other programs like mc?
[15:55] <pgraner> facundobatista: I did a rm -rf "Ubuntu\ One"
[15:56] <facundobatista> pgraner, ok
[15:56] <pgraner> facundobatista: and did the same process to the other computer
[15:56] <pgraner> facundobatista: both were working fine for a few weeks
[15:56] <pgraner> facundobatista: then on Fri I went to add air into the mix and noticed that files were missing off the server
[15:57] <facundobatista> pgraner, "rm -rf" generates a delete on each file, and that removes the files in the server
[15:57] <pgraner> facundobatista: the files were still there for weeks after I did that
[15:58] <pgraner> facundobatista: the U1 daemon was not running at the time nor was I on a network
[15:58] <facundobatista> pgraner, maybe because the client didn't have the opportunity to remove them from the server
[15:58] <facundobatista> pgraner, the daemon has stuff inside to know what you did while he was not running... he later found the deleted files, and removed everything
[15:59] <pgraner> facundobatista: your missing the point after I did that on a clean Ubutnu One dir is synced all the files on the server back down to the client
[15:59] <facundobatista> pgraner, in which machine?
[16:00] <pgraner> facundobatista: then why did it sync everything for weeks, even when I added new files, then one day wake up and decide to delete everything but leave only the directory structure?
[16:00] <pgraner> facundobatista: on both
[16:01] <facundobatista> pgraner, we have a bug that is hitting us all of the time, and that I saw in your logs, that the client gets stuck
[16:01] <facundobatista> so the deletion may have been latent
[16:01] <facundobatista> pgraner, then, something happened that restarted the daemon
[16:02] <webm0nk3y> reboot
[16:02] <facundobatista> pgraner, and the deletion was there (we call it 'trash'... those files that we tried or should have tried to delete but we couldn't)
[16:02] <facundobatista> so, we got the trash, and removed all the files
[16:02] <facundobatista> pgraner, at the same time it found the stuff in disk, and tried to upload
[16:03] <facundobatista> pgraner, that surely created a mess and maybe because that we're seeing dirs and no files
[16:03] <facundobatista> pgraner, also, we prioritize metadata and not content, so makedirs may have got to the server first
[16:03] <pgraner> facundobatista: strange I was syncing for weeks with no prob
[16:04] <pgraner> facundobatista: I added lots of data and it was working fine
[16:05] <facundobatista> pgraner, regarding trash, I found 972 lines like:
[16:05] <facundobatista> 2010-02-23 08:52:57,280 - ubuntuone.SyncDaemon.local_rescan - INFO - unlink from trash: share_id: ''   parent_id: '9cbead56-f9e9-41e6-bb56-bae9df502961'   node_id: '0bf10521-f6a1-4c01-9536-39c9b4d7a6ad'
[16:05] <facundobatista> that's why I pointed to that...
[16:05] <webm0nk3y> looks like the syncdaemon started up last tuesday then
[16:06] <pgraner> webm0nk3y: startup? It should have been running, I used to be able to tell by the applet, but now its not there anymore
[16:07] <pgraner> webm0nk3y: so your telling me one machine was not connected then magically started and whiped everything out?
[16:10] <webm0nk3y> pgraner, I meant on the machine where the files were deleted
[16:24] <pgraner> webm0nk3y: I did the same procedure on both computer exactly I deleted the U1 directory while offline, removed the computers from the U1 web interface, reauthorized them and they both synced with all the files that were on the server and worked for weeks
[16:24] <pgraner> webm0nk3y: and I did the procedure one computer at a time and waited for the first computer to sync prior to doing the 2nd one
[16:27] <facundobatista> pgraner, in your second machine, the SD process started on 2010-02-23 08:50:33,253, and it had all those trash full, waiting
[16:30] <pgraner> facundobatista: at this point I don't know, this should have been a simple. "I want to get rid of my local crap and give me whats on the server" operation
[16:30] <facundobatista> pgraner, mmm... I understand
[16:30] <statik> hi kenvandine, CardinalFang: I understand there are some desktopcouch bugfixes that need to get released and uploaded to lucid, and I'd like to help with that. where can I start?
[16:31] <facundobatista> pgraner, discard everything locally and start from scratch
[16:31] <kenvandine> statik, last i looked, the fix branch was still pending review
[16:31] <facundobatista> pgraner, removing the files is not the way to achieve it
[16:31] <kenvandine> statik, but i am quite anxious to get it into lucid
[16:31] <kenvandine> it makes gwibber suck way less :)
[16:31] <facundobatista> Chipaca, in your GUI plans, did you thought about having some way to trigger something like ^ ?
[16:32] <CardinalFang> kenvandine, statik, I think all is ready.  I closed the milestone and uploaded a source tarball an hour ago.
[16:32] <kenvandine> excellent
[16:32] <kenvandine> statik, can you upload that?
[16:32] <statik> kenvandine: yessir I will get on it now
[16:32] <kenvandine> awesome
[16:32] <statik> CardinalFang, thanks
[16:32]  * kenvandine loves statik having upload rights :)
[16:32]  * Chipaca does too
[16:33] <Chipaca> facundobatista: something like what, sorry?
[16:33] <kenvandine> CardinalFang, thanks for getting that release done, it seems to really make gwibber start reliably
[16:33] <Chipaca> facundobatista: ah, the "just nuke everything, already"?
[16:33] <CardinalFang> kenvandine, my pleasure.
[16:34] <facundobatista> Chipaca, a "discard everything and sync back with what's in the server" button
[16:34] <CardinalFang> statik, I want to help test the package before you upload.
[16:34] <facundobatista> Chipaca, maybe with a shorter name
[16:34] <Chipaca> facundobatista: sounds like something we could do for mutant mantis
[16:34] <statik> CardinalFang: ok, I'll upload to the hackers nightly PPA before putting it to lucid
[16:34] <CardinalFang> Rawk.
[16:35] <statik> and talk with you about testing
[16:35] <statik> i need a 500GB SSD, I keep having to delete build trees
[16:35]  * kenvandine could use one too
[16:36] <Chipaca> statik: easy, get 500GB of RAM, and just make a ramdisk
[16:36] <kenvandine> *easy*
[16:36]  * kenvandine chuckles
[16:36] <statik> that is some ceo-level handwaving chipaca. careful or you'll get promoted
[16:37]  * Chipaca rubs his hands gleefully
[16:37] <Chipaca> excellent.
[16:45] <jblount> statik: Do you have an extra drive bay?
[16:46] <statik> jblount: no, not even a cd drive.
[16:47]  * statik hugs his tiny x200s
[16:47]  * jblount is quietly contemplating losing his disc drive for a small ssd / big hd combo
[16:52] <pfibiger> jblount: is it a hard mod?
[16:52]  * jblount looks for the url
[16:52] <jblount> pfibiger: I don't think so
[16:53] <pfibiger> ah, found it
[16:53] <pfibiger> looks easy
[16:53] <statik> kenvandine, any particular gwibber bug that I should mention in this desktopcouch changelog?
[16:53] <pfibiger> $200 gets you the mounting hardware plus an external enclosure for the dvd drive you pull out
[16:54] <pfibiger> so $200 + $300 for the 80gb intel ssd.
[16:54] <pfibiger> hmm
[16:55] <kenvandine> statik, one sec
[16:58] <kenvandine> statik, 519444
[16:58] <statik> thanks
[17:05] <CardinalFang> statik, I put some notes in the release changelog, fwiw.
[17:08] <duanedesign> if someone has some corrupted metadata. The error they are getting 11087e34.... Would the directory structure be 1/10/87/11087e34...
[17:11] <duanedesign> i am used to seeing numbers like 617e34...which is easy ~/.local/share/ubuntuone/syncdaemon/vm/shares/6/1/7/617e24...
[17:12] <statik> CardinalFang: oh, thanks!
[17:14] <statik> CardinalFang: I'll use that instead of what i was writing up by looking at the commits. too bad uscan doesn't download that from launchpad, i had totally missed it
[17:30] <rye> duanedesign, i don't think so... could you please give the bug number? I believe syncdaemon does only a/b/c/abc metadata partitioning (if this can be called this way)
[17:33] <duanedesign> rye: ok thank you...
[17:33] <duanedesign> bug 530092
[17:37] <rye> ok, hacked a script for detection of blank notes, now need to make the fix
[17:37] <rye> duanedesign, unmarked that as private since that report does not contain anything personally identifiable more than launchpad login could give
[17:41] <duanedesign> rye: so the key error is caused by blank notes?
[17:41] <rye> duanedesign, the pickle loading seems to be related to fsm, which means that something has probably removed... file_shelf.py, _load_pickle raises KeyError if there is except (IOError, OSError)
[17:41] <rye> duanedesign, no, that was unrelated to this bug report, we are going to provide the steps on how to fix blank notes
[17:41] <duanedesign> lol, ok
[17:41] <duanedesign> :P
[17:42] <rye> duanedesign, and I decided to create the script to make people with 10+ notes happy
[17:44] <rwlyonsjr> Is anyone having problems with the web interface of Ubuntu One?
[17:46] <beuno> rwlyonsjr, more specifically?
[17:46] <rwlyonsjr> When I log into the web interface, I intermittently can not see any of my files
[17:47] <rwlyonsjr> I am also not able to sync my notes at all.... I saw the post on the blog that said they were switching things to read-only, but I can not see anything at all
[17:54] <rwlyonsjr> hello??
[17:54] <Technoviking> would the Ubuntu One note issue be affecting syncing with tomboy
[17:55] <rwlyonsjr> Technoviking, I did not see anything in the post that would indicate that...they only said it would be read only
[17:56] <rye> rwlyonsjr, Technoviking, the notes are now in read-only on the web interface and Tomboy sync is failing due to a different issue that is now identified and being worked on
[17:56] <rye> rwlyonsjr, are you receiving a Something Wrong (500) page?
[17:57] <rwlyonsjr> rye, thanks! Any info on files not showing up at all on the web interface.... The problem is intermittent...
[17:58] <rye> rwlyonsjr, hm
[17:59] <rwlyonsjr> rye, when I log on to the web interface it shows I have no files at all, then I hit refresh and they came back.... another refresh and they were gone again... Just in and out like that, with no real pattern
[18:02] <rye> rwlyonsjr, confirming
[18:02] <rye> rwlyonsjr, no, wait, this is my testing account, it has 0 files
[18:03] <rwlyonsjr> rye, I just tried again... same results for me.... No files, I have not refreshed yet
[18:04] <rwlyonsjr> with a refresh it comes back again.... Looking at the issue more, it may be that the "My Files" folder is not showing up on first log-in for me.... It goes to the "Shared with me Folder", with no other folders viewable...
[18:13] <rwlyonsjr> rye, were you able to duplicate the issue?
[18:14] <rye> rwlyonsjr, how many files do you have in ~/Ubuntu One ?
[18:14] <rwlyonsjr> rye, standby
[18:15] <rwlyonsjr> rye, about 50-75, some of them are in folders.... it is not a lot of data
[18:16] <rye> rwlyonsjr, ok, I think I know the reason and it has something to do with web interface aborting information loading when it takes more time than expected. Since for every file a new background request is created this may lead to the problems.
[18:17] <rye> desktop+ guys, need info on why web interface would want to load every file info separately via separate XMLHttpRequest
[18:17] <rye> even firefox has a limit on concurrent connections to the server
[18:19] <rwlyonsjr> ok..... is there something I can do to help?
[18:19] <dobey> jblount, urbanape, beuno: ^^
[18:20]  * beuno looks up
[18:20] <beuno> right
[18:20] <beuno> we need to re-work /files
[18:21] <beuno> we have a call on Wed to start planning that  :)
[18:25] <urbanape> rye: basically, historical reasons. That part of the code hasn't changed since first implemented, even though the layout (and a lot of the js accompanying it) has
[18:25] <urbanape> and what beuno said.
[18:26] <rye> urbanape, beuno, because that started to time-out for me in my IRC logs folder which has only 180 files.
[18:26] <beuno> rye, can yuo test on edge as well?
[18:27] <beuno> there's some new code from jdo that makes listing faster
[18:27] <urbanape> rye: I'm not disputing it, just making excuses. *grin*
[18:37] <rye> thisfred, may I poke you about update_fields again, you told me that it is possible to use ['application_annotations','Tomboy','some-property'] as a key for update_fields. May I inquire about the syntax of update_fields in this case?
[18:37] <rye> beuno, testing the edge
[18:38] <rye> rwlyonsjr, could you please check your files on https://edge.one.ubuntu.com as well ?
[18:38] <rwlyonsjr> rye, sure.... Standby
[18:39] <rye> beuno, I like the compiz effect on non-responding window when I load my IRC logs... :)
[18:40] <Technoviking> rye: thanks
[18:41] <rwlyonsjr> rye, The problem persist there, however it does not happen as often at all.... Instead of every other refresh, I can refresh several times before my files vanish
[18:41] <thisfred> rye, can I get back to you on that? have to briefly go out to walk the dog now
[18:41] <rye> thisfred, say hello to the dog from me :)
[18:41] <thisfred> will do bbiab
[18:41] <rye> hm, maybe python keys in dicts do not need to be scalars like in Perl?...
[18:43] <rye> nope... TypeError: unhashable type: 'list'
[18:45] <rye> Technoviking, the notes are now under heavy fixing. There are issues with WYSIWYG editing and saving as well as displaying the result of the save. Tomboy note sync is also affected by HTML to XML conversion and note blanking that has happened earlier and now we are putting up the solution for this.
[18:57] <thisfred> rye: keys can be composite, but have to be immutable types, so you can't have lists
[18:57] <thisfred> rye, but tuples work
[18:58] <Kandanalor> hi
[18:58]  * rye went to find out what tuple means...
[18:59] <Kandanalor> i hve a little question about storing and editing large files in the cloud
[18:59] <thisfred> rye, ('foo', 'bar', 'baz') rather than ['foo', 'bar', 'baz']
[18:59] <thisfred> so for that method:
[18:59] <Kandanalor> is only the part that i edited downloaded and uploaded or the whole file?
[18:59] <thisfred>     def update_fields(self, record_id, fields, cached_record=None):
[19:00] <thisfred> rye: fields could be of the form: {('application_annotations','Tomboy','some-property'): new_value}
[19:01] <dobey> Kandanalor: currently, the entire file is uploaded again
[19:02] <rye> thisfred,   File "/usr/lib/pymodules/python2.6/simplejson/encoder.py", line 260, in iterencode
[19:02] <rye>     return _iterencode(o, 0)
[19:02] <rye> ValueError: keys must be a string
[19:03] <thisfred> rye, you are using update_fields, correct? You can't set keys like this in a record directly
[19:03] <rye>    path = ('application_annotations', 'Tomboy', 'last-sync-revision')
[19:03] <rye>         self.notes_db.update_fields(self.uuid, \
[19:03] <rye>                 { path: self.last_sync_revision + 1 })
[19:03] <rye> thisfred, ^
[19:04] <thisfred> rye, and you're using desktopcouch trunk?
[19:05] <dobey> Kandanalor: and it should only get uploaded again when you close the file
[19:05] <rye> thisfred, i am using lucid lynx version, and I'd really want to have it working in karmic as well :)
[19:06] <thisfred> rye: it will only work if update_fields in server_base.py looks like this: http://friendpaste.com/3XtkfLWeUNaXHv1JE7qWva
[19:06] <thisfred> I am not sure whether that has been uploaded to lucid yet
[19:07] <dobey> Kandanalor: you don't need to /msg me
[19:07] <Kandanalor> k
[19:08] <Kandanalor> so does anybody know when editing parts of a file without downloadig and reupping the whole file would be available?
[19:08] <dobey> Kandanalor: there's some ideas floating around about how to do diffs, but it only would resolve part of your issue
[19:09] <Kandanalor> yeah ok
[19:09] <dobey> Kandanalor: primarily, a large file, such as an encrypted fs would still only get diffed and synced after it was closed. we wouldn't want to sync arbitrary bits at random points while the file was open
[19:09] <dobey> that could be bad
[19:09] <Kandanalor> yes I understand
[19:10] <rye> thisfred, no, my server_base.py does not look like this
[19:10] <thisfred> rye, ah, then it does not yet work like that... :(
[19:11] <Kandanalor> think Ill have to think about this problem maybe I find a solution that works for me
[19:11] <rye> thisfred, i believe I will need to read all info from couchdb, then update fields and save that... hitting json exception on the way... probably...
[19:11] <Kandanalor> then I just wanted to thank you for your work. I think you do a great thing to the community!
[19:11] <Kandanalor> so bye
[19:12] <Kandanalor> :)
[19:12] <thisfred> rye: if you just load the document, change the field, and save it, you won't get any json exceptions
[19:12] <dobey> Kandanalor: thanks
[19:13] <thisfred> rye: the only thing that can happen, that the update_fields method protects against somewhat, is a conflict, if something else has written to the same record in the meantime
[19:13] <rye> thisfred, but I am changing application annotations. isn't that special for couchdb?
[19:13] <rye> dc
[19:14] <thisfred> rye: not for couchdb, for desktopcouch it's special in that you have to access them like this: record.application_annotations['Tomboy']['some_field'] but under the hood, application annotations is just a regular json dictionary. We just want to hide it from people who don'
[19:15] <thisfred> t need it
[19:15] <thisfred> rye: i.e if you loop through all the fields of a record, application_annotations won't show up
[19:16] <rye> thisfred, awesome. put_record works with simple fetch doc, update doc, put doc
[19:17] <rye> now I need to check whether that works in carmic
[19:17] <thisfred> rye: I think it will
[19:18] <thisfred> rye: though you should have a try: except: around the put, that can handle a resource conflict
[19:19] <thisfred> depending on how bad it is to get an error sometimes
[19:27] <Bookman> Can .u1conflict files be deleted safely?
[19:33] <duanedesign> Bookman: The way to resolve conflicts is to pick from the original file (if there is one) and the conflict file that you wish to keep and to delete the conflict file or move the conflict file over the original one, depending on your choice.
[19:38] <rye> rodrigo_, joshuahoover: http://paste.ubuntu.com/386500/
[19:39] <rye> rodrigo_, joshuahoover: tomboy needs to be shut down prior to executing this script. I am still learning python, so, probably it does not pass any pylint tests... hm, need to check that
[19:40] <joshuahoover> rye: nice work!
[19:40] <rye> wow, 7.13/10
[19:40] <joshuahoover> so tomboy needs to be shutdown...anything else?
[19:41] <rye> joshuahoover, no, it will make sure desktopcouch is started properly, then it will find out the dir for tomboy within xdg directories, read all the notes that are there, load these notes from couchdb and check whether content is blank. If it is, then we bump the last-sync-revision in couchdb and try to act naturally.
[19:41] <rye> joshuahoover, let me check this in karmic...
[19:42] <rye> and it would be really nice if someone with python background had a look at that :)
[19:42] <joshuahoover> rye: ran fine in my one karmic env...i didn't have any notes that were blank previously
[19:42] <rye> joshuahoover, now you have :-P :)
[19:42] <rye> ok, /me is back to being serious
[19:45] <rye> wow, it worked in karmic as well
[19:46] <rye> this script does not check resource conflict since it does not expect anything to modify the notes between the read and save operation
[19:47] <thisfred> rye: for a script like this I think that's fine: if you get a conflict error, just run it again
[19:47] <thisfred> depending on who the target audience is
[19:48] <joshuahoover> thisfred: we'd point users to run this who were affected by bug #524339
[19:48] <thisfred> joshuahoover: right, in that case maybe mention that conflict errors could occur and aren't a real problem
[19:49] <rye> joshuahoover, but we will still need to have a hope that the user has not compacted their database
[19:49] <joshuahoover> thisfred: and just run it again if you do get conflict errors?
[19:49] <thisfred> they'd be rare enough that I don't think it warrants defensive coding against them in such a script
[19:49] <thisfred> yes
[19:49] <joshuahoover> thisfred: cool
[19:49] <joshuahoover> rye: but isn't that an issue either way? manual or using your script?
[19:50] <rye> joshuahoover, true
[19:50] <joshuahoover> rye: so, i think this script is MUCH better than the manual steps, which are a huge hassle for most users, especially if there are a lot of notes needing fixed
[19:51] <rye> joshuahoover, but if they have synced these blank notes to the couchdb then things start to get more complicated, we will need to dig through revisions and find those which are not blank. And I don't know how to browse through revisions in dc, only futon...
[19:52] <joshuahoover> rye: ah, true
[19:55] <joshuahoover> rye: so thoughts on how to best guide users through this?
[19:56] <joshuahoover> rye: seems like we could have them try your script first (least painful) and then if it doesn't we suggest...????
[20:00] <wharp> I'm following the quickly tutorial and it says that if the (desktop couch) page I'm redirected to doesn't work, I probably just need to clear the browser cache. I've tried that and its still not working, also tried in a different browser. What else could be wrong?
[20:03] <joshuahoover> wharp: what is the url of the page you're redirected to?
[20:05] <wharp> http://localhost:35608/_utils
[20:08] <joshuahoover> wharp: and it comes up with an error?
[20:09] <joshuahoover> thisfred, rye: if the futon web interface doesn't come up, what do you do? does that mean couchdb isn't running?
[20:09] <rye> joshuahoover, if this doesn't work we suggest go to futon and browse through revisions via 'Older revision' and then bump the version there
[20:09] <thisfred> joshuahoover: what does it say? asking for a login?
[20:10] <joshuahoover> thisfred: trying to help wharp ^^
[20:10] <wharp> page not found
[20:11] <wharp> what/where is futon?
[20:11] <rye> joshuahoover, if couchdb is running then futon is running, but it might not be running exactly on the port that is written in couchdb html file
[20:11] <thisfred> page not found implies couchdb not running yes, unless you have done something to remap localhost, if that is even possible
[20:11] <rye> wharp, http://paste.ubuntu.com/386516/
[20:11] <thisfred> wharp: futon is couchdb's admin web interface
[20:12] <joshuahoover> rye: browse through revisions via 'Older revision'? not sure how to tell a user to do that without wading through (potentially) tons of documents
[20:12] <wharp> well file:///home/wes/.local/share/desktop-couch/couchdb.html loads
[20:12] <thisfred> ah, yes, the portnumber may be out of sync
[20:12] <rye> wharp, that script will launch firefox and connect to your desktopcouch instance, w/o couchdb.html file which may be out of date
[20:12] <wharp> ok
[20:13] <rye> joshuahoover, there is [ Showing revision 14 of 14 ] and [  ← Previous Version | Next Version →  ] links on every document that is in couchdb in futon
[20:13] <rye> thisfred, is it possible to browse through older revisions in desktopcouch?
[20:14] <joshuahoover> rye: ah, ok, i didn't see that before :)
[20:14] <thisfred> rye: no, I don't think so
[20:14] <wharp> ah, that works, so I should just upset the couchdb.html with the correct port number?
[20:15] <thisfred> rye: I think if you have the exact revision number, you can still access them by that url, but that's sort of a back door
[20:15] <rye> thisfred, futon can :)
[20:15] <thisfred> rye, ok, if futon can do it, it's probably an intentional feature
[20:15] <rye> thisfred, ?revs_info=true
[20:15] <thisfred> I thought it was discouraged to rely on them
[20:16] <thisfred> rye ah ok
[20:16] <rye> and "_revs_info":[{"rev":"18-e8f76906531ed01e37b51972e25a95f0","status":"available"} ... ]
[20:16] <thisfred> that will work for regular http access as well then
[20:16] <rye> not sure how to ask for that but that might be a nice to have feature in such cases...
[20:17] <thisfred> rye, it will only work as long as you don't compact though. After that, the revision numbers will be there, but not the actual revisions
[20:17] <thisfred> that's probably what status available means
[20:18] <joshuahoover> thisfred: i can't remember, do we automatically compact the database for the user?
[20:18] <thisfred> rye, so if you do a regular http call into couch instead of through futon, the ?revs_info=true parameter will work. I'm not sure if we can do this through python
[20:19] <thisfred> joshuahoover: AFAIK not yet
[20:19] <joshuahoover> thisfred: well, that's good in this case :)
[20:19] <thisfred> yep :)
[20:19] <rye> joshuahoover, no, we don't compact, but some users (like me) like to press whatever buttons they find...
[20:19] <thisfred> another argument against premature optimization ;)
[20:20] <rye> joshuahoover, thisfred i believe there are plans to start compacting users database much like replication is being performed...
[20:21] <joshuahoover> rye: yes, i think that is correct
[20:21] <thisfred> rye: yes, and we definitely should, since we'll use up the entire hard drive if we don't
[20:22] <rye> thisfred, true... btw, is there any particular reason why gwibber started putting the messages to couchdb, not only configuration and accounts but messages
[20:23] <rye> It puzzled me while I was writing http://blog.rtg.in.ua/2010/02/ubuntu-one-internals-notes.html but I came up with nothing really useful
[20:23] <thisfred> rye: because they could, I guess ;) I think the messages make more sense to store in d-c than the configuration, myself: configuration should be in easy to reach places. Flat text files, preferably
[20:23] <thisfred> since if stuff breaks, you don't want to have to learn how to use couchdb first
[20:24] <thisfred> But I may be in the minority there. Doesn't mean the majority's right though ;)
[20:26] <thisfred> rye: desktopcouch is about storing structured data, and sharing it between computers (and, in the future, users) I think of twitter messages as data, more than I would application configuration
[20:27] <thisfred> of course it generates quite a bit of data and traffic, but I think that's good, we'll shake out some more bugs soon
[20:40] <joshuahoover> rye: ok, so i'm going to recommend people run your script first, if that doesn't work, then i'm going to give steps for going through the revisions (noting this will only work if the database has not been compacted)...for those steps, will the user find the revision with the content (versus the notes that are blank) and set the last-sync-revision to the maximum plus 1?
[20:46] <rye> joshuahoover, let me try that....
[20:46] <rye> :)
[20:48] <rye> joshuahoover, that won't work out of the box, the _rev will need to be updated as well
[20:48] <joshuahoover> rye: can that be set manually? (the _rev)
[20:49] <rye> joshuahoover, hm, does not work... still resource conflict
[20:49] <joshuahoover> rye: just the _rev max + 1?
[20:50] <joshuahoover> rye: yeah, that's what i thought....then the user would probably be best to modify the content and increase the last-sync-revision to max + 1? i think that will up the _rev and take care of the last-sync-revision
[20:50] <joshuahoover> rye: this is a pretty horrible process for the user
[20:51] <rye> the user will need to copy the last non-empty content to the latest revision and then my script will be able to bump the revision but this is too much work
[20:51] <joshuahoover> rye: yeah...hmmm...
[21:12] <rodrigo_> rye, wow, thanks for the script!
[21:13] <rye> rodrigo_, feel free to tell me what you think about the code itself :)
[21:48] <CardinalFang> Aw, I got an email messaage, ""Comee back to Dropbox!!"
[21:49] <beuno> CardinalFang, ask them if they have integration with Ubuntu One
[21:49] <beuno> :)
[21:51] <rwlyonsjr> I had to get off line earlier, but is there still a problem with the web interface displaying files every time?
[22:37] <joshuahoover> thisfred: do we limit couch syncing by using the "excluded_names" key in the ubuntu_one_pair_record doc within the management database?
[22:38] <thisfred> joshuahoover: yes, that has the names of databases that should not be synced
[22:39] <joshuahoover> cool, kudos to duanedesign for providing that answer in our forums!
[22:39] <joshuahoover> thisfred: is that documented somewhere already?
[22:39] <thisfred> joshuahoover: not sure, that's a question for CardinalFang/aquarius
[22:40] <joshuahoover> thisfred: ah, ok...i was creating an faq since this was the first i'd ever seen/heard about this
[22:40] <thisfred> yeah, my guess would be no
[22:41] <CardinalFang> joshuahoover, It's not documented because it's not well supported or tested.  It's also not thought-out very well.
[22:41] <joshuahoover> CardinalFang: ah, ok :)
[22:42] <CardinalFang> We provide no API for anyone to query or change it, and it is pairing-specific.
[22:43] <aquarius> https://answers.edge.launchpad.net/desktopcouch/+question/95878
[22:43] <aquarius> answers that question
[22:44] <joshuahoover> aquarius: great! thanks!
[23:07] <aquarius> CardinalFang, why would desktopcouch-service be pegging the CPU
[23:07] <aquarius> ?
[23:08] <CardinalFang> aquarius, Hrm, I don't know.
[23:09] <CardinalFang> aquarius, I'm afk for 30 minutes.  I'll be back to see if you say anything.
[23:10] <aquarius> we think it's d-bus
[23:13] <duanedesign> i have two bug reports with similar erors. I am looking for any help to better understand the issue :) _ssl.c:326: error:0B084009:x509 certificate routines:X509_load_cert_crl_file:PEM lib
[23:13] <duanedesign> bug 529168
[23:19] <duanedesign> bug 522285
[23:29] <duanedesign> well, its family time. i leave my client open if anyone has any input. Have a nice evening all.
[23:30] <duanedesign> oh and aquarius i enjoyed your session today, good job.