/srv/irclogs.ubuntu.com/2011/01/25/#ubuntuone.txt

TrickyJGood morning :)00:30
* karni eods, c u tomorrow everybody01:21
erosscan someone give me the dummy's skinny on how this works? i just bought an album and it's transferring to the ubuntu-1 storage area.. what can i do after that, re-download again to local?02:35
erossseems to be slow or stalled02:35
erossnm looks like the files are on some cloud02:38
erosshow do i get at them02:40
jderoseaquarius: okay, got oauth stuff basically working, but i'm trying to understand why you some times append an Authorization header, other times use a signed url?11:37
aquariusjderose, um...can't remember :)11:39
aquariusjderose, when I originally started I used signed URLs because it's easier (you can test hte URL in your browser directly), but that doesn't work with futon because futon pages introspect their own URL to do stuff, and having loads of extra query parameters screwed that up11:40
aquariusso i went to headers instead11:40
aquariusbut I'm not sure why I didn't go entirely to headers11:40
jderoseyeah, i tried it just using the header, but it's not working11:41
jderoseaquarius: so ouath doesn't sign the entire request the way the amazon aws api dose, correct?11:42
jderoseaquarius: mean, oauth just working from the uri, not the headers of the request11:43
* jderose is getting tired :)11:43
aquariuscorrect; an oauth signature is composed of the URL, the HTTP method, a timestamp, and a nonce.11:43
aquariusthe body and the other headers are not part of the signature calculation11:44
duanedesignhello aquarius11:44
jderoseokay, gotcha11:44
duanedesignmorning all11:44
aquariusduanedesign, heya!11:44
jderosemorning duanedesign!11:44
aquariusjderose, ya, I'm not sure why it doesn't work with headers all the time either. It ought to :)11:45
duanedesignhello jderose11:45
jderoseaquarius: but based on your emminse11:45
jderosedamnit, can't type11:46
duanedesignjderose: were you in talking to karni the other day?11:46
jderoseaquarius: but based on your immense oauth knowledge, it should work with just headers?11:46
duanedesignjderose: oh, and hello :)11:46
nessitahello everyone11:46
jderoseduanedesign: i don't know who karni is, so i don't think so :)11:47
duanedesign:)11:47
aquariusjderose, well...it should, yes, in theory. I'm not sure why it doesn't :)11:49
jderoseaquarius: well, i'll poke around some more.  i'll let you know if i figure it out.   thanks!11:50
karnihi guys. duanedesign, I probably pinged you with some guys question, didn't I :)11:54
jderoseaquarius: ug, perhaps i found out why... looks like the request headers are immutable from Python! crap.11:56
jderosewell, time for bed. later!11:56
duanedesignhey karni !12:01
karniduanedesign: holla :)12:01
=== zyga is now known as zyga-coffee
=== zyga-coffee is now known as zyga
karnibbl12:22
ralsinagood morning!12:25
nessitahi ralsina12:25
ralsinahola naty12:27
=== teknico is now known as teknico_away
ralsinanessita: any reviews for me?12:46
nessitaralsina: yes! https://code.launchpad.net/~nataliabidart/ubuntuone-control-panel/confirm-before-delete/+merge/4732412:47
ralsinacool, will review in 10' after updates download12:47
nessitaralsina: awesome!12:53
ralsinaAfter that I have to run a review on windows so the day becomes markedly less fun ;-)12:53
nessita:-)12:55
nessitaso, ralsina, I need to consult something with you. I'm about to do a release for the control panel, and, the services tab is not working right since the call to list all desktopcouch replication is (still) blocking, and seems like there is an issue with the plugins where the destopcouch service is frozen or deadlocked when starting up12:56
ralsinaouch12:57
nessitaso basically the u1cp backend freezes for 30 seconds~ or maybe more12:57
nessitawhich is aweful12:57
nessitamy proposal is the following:12:57
ralsinaYou could do a release without that tab12:57
nessita* hide the 'Services' tab for this release12:57
ralsina:-)12:57
nessita* define a way of strating DC in a thread or similar for next release to not depend on dc blockages12:58
alecunessita, can you point me at that bit of code?12:58
* pedronis back12:58
nessitaalecu: u1/cp/replication_client.py12:58
ralsinanessita: check canonicaladmin you have good news there ;-)12:58
alecu_ack12:58
nessitaalecu_: the problem is the problem with this line:      64         result = replication_module.ReplicationExclusion()12:59
alecuI see12:59
ralsinanessita: you have a comment by rye in that branch, did you see it?12:59
nessitanopes, let me check13:00
nessitaI've been in a call with i-vanka up to just now13:00
ryeralsina, i believe I should not make such comments in branch reviews13:01
nessitaI thinki it was a joke :-)13:01
ralsinarye: hahaha ok then :-)13:01
* ralsina thought the "no" button was disabled or something13:01
nessitaalecu: I think we need to make the replication_module.ReplicationExclusion call in a thread, I don't know how to join the thread without blocking the caller13:02
ryeralsina, commented again clarifying what I meant in case I get back to that review in a week and have no idea what I meant at that time.13:03
ralsinarye: jokes are always ok in principle for me, so don't worry :-)13:04
ralsinanessita: add a repeating timer that checks to see if the thread is running.13:04
ryeralsina, well, I will try to re-read the thing I am posting since the way I wrote that clearly indicates something bug-related and then "Approved". yeah, right...13:04
ryenot joking today anymore13:05
nessitaralsina: and the timer runs where?13:05
ralsinanessita: event loop?13:05
nessitaralsina: in the glib loop, you will say13:05
ralsinanessita: exactly13:05
nessitaralsina: but we're trying to make this g-agnostic13:05
ralsinanessita: there's no way to have a g-agnostic timer, I'm afraid13:05
ralsinanessita: you could create a class that can be patched for other event loops, so at least it's always in one place13:06
nessitaralsina: so, only the backend executable knows about glib, we may need to pass the timer class as class creation parameter down to the lower layers?13:06
ralsinaOr you could use unix signals but that's hell on a handbasket13:06
ralsinanessita: right13:06
* ralsina hates even the sound of this13:07
nessitaralsina: I'm pondering the fact that desktopcouch *should* start in a few seconds13:07
nessitanot in 30. There is a bug there.13:07
ralsinanessita: yes, but still, blocking for 5 seconds is not really acceptable either13:07
nessitaso, how much effort we want to put in the cp considering that DC is not working?13:07
ralsinaIs there a way to check non-blocking if it's running?13:07
nessitaralsina: not a clean way that I know of13:08
nessitaother than asking for pidof or something like that13:08
ralsinanessita: not that much, I'd say. First we should make it start as fast as possible, and if that is not accpetable, then we hack with threads13:08
ralsinaElse, it's premature optimization13:08
ralsinathis is in the control panel backend or in the frontend?13:09
nessitaralsina: backend13:09
ralsinathe backend could just start dc in background and the frontend would wait for it. We can use glib on the frontend13:10
nessitawhich is ugly since the UI is not freezed but shows 'loading' screens everywhere13:10
ralsinaOr am I saying nonsense?13:10
nessitaralsina: you are :-) the frontend accesses the backend thru dbus, so the frontend only does method call and connects callbacks to signals13:10
ralsinanessita: but, as an exception to keep the backend glib-free... ok, forget it :-)13:11
nessitathe backend need to return immediatly after a call (which is not doing right now in this case)13:11
ralsinanessita: right, that's why I said the backend should just start it and forget it.13:11
ralsinaok, looking at the code forget it, I would break half of it13:12
ralsinaSo, back to "hide the tab, complain about startup time, don't sweat it on control panel yet"13:13
ralsinamaybe it will start in .2 seconds next week ;-)13:13
alecunessita, ralsina: I've started digging into desktopcouch and couchdb python modules, and it seems we may run all that in a thread.13:13
nessitamy point exactly13:13
nessitaalecu: not sure what you mean13:14
ralsinaalecu: but how will the backend notify the frontend that it's running?13:14
alecuralsina, what do you mean "running"?13:14
ralsinaready?13:14
alecuby a dbus signal!13:15
alecuthe frontend makes a dbus call, and expects a dbus signal when it's done.13:15
alecuor am I missing something?13:15
ralsinaso the second thread emits the signal?13:15
ralsinacan we do that?13:16
alecuralsina, yes: python dbus bindings are thread safe13:16
ralsinaalecu, nessita: so no need to join the thread then13:16
alecu(btw: the desktopcouch code uses the couchdb module, and that speaks http, by using raw sockets, so we won't have any issues there running it in a thread)13:17
alecuyup, no need to join threads.13:17
ralsinaalecu: if you think that's not hard to do and it can be done today, it would be great13:18
ralsinanessita: any objections?13:18
ralsinaand we do the release tomorrow but with a nice new feature.13:19
nessitaralsina, alecu: I don't see how a thread will fire a dbus signal. Let me explain:13:19
nessitawe have these layers: dbus_service (signals, among other stuff), backend, replication_client (desktopcouch)13:20
nessitahow DC, running in the 3rd layer will fire stuff in the first layer?13:20
alecunessita, by passing a callback from the first layer into the third.13:20
nessitaright13:21
alecunessita, "here, call this function when you are done."13:21
alecunessita, and that method just fires the signal.13:21
alecudoes it makes sense?13:22
nessitait makes sense, but what am I thinking is:13:22
alecuralsina, do you want me to work on it, or should I keep working on my aggregation branch?13:22
nessitaI have no free slot this week to do that (I'm very very busy with the shares stuff), and the release needs to go out today13:23
ralsinaI say we keep this solution in mind13:23
ralsinaand don't do it yet13:23
* ralsina cries a little bit13:23
nessitaralsina: I prefer you do aggregation and if you're free, the ZG listener bugs, that they are messing with syncdaemon logs a lot (in a sense they add confussion)13:23
nessitaI meant alecu :-)13:24
ralsinanessita: yes13:24
ralsinaand anyway it may not even be necessary to do this thread stuff13:24
nessitaI would hide the services tab to make a release for the alpha213:24
ralsinanessita: yes13:24
ralsinaso we have a plan now. <captain picard>make it so</captain picard>13:25
nessitaand restore it after, and chase CardinalFang for some input. He proposed a branch yesterday but the test suite is failing a lot in that branch13:25
alecunessita, both hide and disable the backend code, right?13:25
ralsinanessita: what branch was that?13:25
alecudisable the backend *call.13:25
nessitaalecu: just hide the tab, the code will no get executed until you enter the tab13:25
alecunessita, cool.13:25
nessitaralsina: https://code.launchpad.net/~cmiller/desktopcouch/service-must-not-call-self-over-dbus/+merge/4726213:26
nessitaralsina: seems like the newly added plugin code is making the service startup deadlock13:26
ralsinaright13:27
ralsinaugh :-(13:27
nessitaralsina: my second objection to that branch is that the fix has no tests...13:27
ralsinawhat would you test, that the port you get over dbus is the same as the port you put there? :-)13:28
nessitaralsina: first of all, I'd test that no internal API breaks13:28
ralsinaadd a None default to couchdb_port and get it by dbus if it's not passed could be a good idea13:29
ralsinaSo we keep the API stable13:29
ralsinathen again, I come from C++ so I have a stable API fetish13:30
nessitamy biggest concern is that there is already some tests for the load_plugins function13:31
nessitaand this branch breaks that (or should break)...13:31
ralsinaapproved confirm-before-delete13:31
nessitahow come CardinalFang is getting all green?13:31
nessitaralsina: thanks13:31
ralsinaah?13:31
ralsinaall what green?13:34
nessitaall tests pass for him13:36
nessita(he said)13:36
ralsinaweird13:37
* ralsina checks13:37
ralsinarunning the tests now. couchdb started in about 2 seconds, BTW13:41
CardinalFangralsina, nessita, I think I have that fixed.  Making tests work has been a problem.13:42
ralsinaCardinalFang: cool13:42
ralsinaI got a bunch of errors already13:43
CardinalFangMy kid is really grumpy, so it's hard to get started this morning.  I'll be back in a bit.  Hopefully before my 9AM standup-meeting.13:43
ralsina"    from desktopcouch.application import local_files13:43
ralsina" rings any bells?13:43
ralsinaOf course the error is "cannot import local_files" ;-)13:44
nessitaI got that too13:44
CardinalFang(Someone in my house mixed chlorine bleach and ammonia about a day ago, and we all felt bad for a while.)13:44
nessitabut not in trunk...13:44
nessitaouch!13:44
ralsinathat mix is incredibly dangerous, you know13:45
CardinalFangYes.  We all know.  Someone wasn't attentive.13:45
ralsinaIn fact, the byproduct of that was used as a chemical weapon in WW113:45
* ralsina never has ammonia at home, just in case13:46
CardinalFangIndeed.  In different proportions, it tends to make explosives too.13:46
ralsinayup13:46
ralsinaAs in "that is rocket fuel" explosive.13:46
CardinalFangSo, a fun evening in the Miller household, all in all.13:47
* CardinalFang afk.13:47
beunois the canonical irc network down or is it me?13:48
nessitabeuno: I'm connected13:48
ralsinabeuno: it's you13:48
beuno:/13:48
beuno07:48 -!- Irssi: warning SSL handshake failed: server closed connection13:49
ralsinaalecu CardinalFang dobey mandel nessita thisfred vds: standup in 11'13:49
mandelok13:49
thisfredackack13:49
nessitayeah13:50
* ralsina forgot someone?13:50
alecume14:00
nessitame14:00
ralsiname14:00
vdsme14:00
nessitaCardinalFang, dobey, mandel, thisfred?14:01
thisfredme14:02
dobeymeh14:02
mandelme14:02
ralsinaalecu, start14:02
alecuDONE: counters for (done/total) operations progress, aggregation based on threshold and timeout. Spent a lot of time chasing an issue that looks like a twisted bug.14:02
alecuTODO: discuss with thisfred how to pass this info to upper layers, plan for UDFs/shares, isolate twisted issue.14:02
alecuBLOCKED: no14:02
alecuLOVE & HATE: twisted.internet.task.Clock14:02
* alecu hands a big fishbowl of twisty snakes to nessita14:02
nessitaDONE: had some issues with syncademon that I debug a bit and reported. Worked on bug #706906 and bug #706888, landed branches for bug #692772. Had a call with cparrino to review all the strings in the control panel.14:02
nessitaTODO: call with i-vanka to sync up re control panel, fix strings as per cparrino review, add marketing link in the control panel (u1 support, askubuntu, tiwtter, facebook). Release u1cp and ussoc.14:02
nessitaBLOCKED: nopes14:02
nessitaLOVE: chocolate cake14:02
nessitaNEXT: ralsina14:02
ubot4Launchpad bug 706906 in ubuntuone-control-panel (Ubuntu Natty) (and 2 other projects) "Token info is logged (affects: 1) (heat: 6)" [High,Triaged] https://launchpad.net/bugs/70690614:02
ubot4Launchpad bug 706888 in ubuntuone-control-panel (Ubuntu) (and 1 other project) "Prompt for confirmation before removing a device (affects: 1) (heat: 6)" [Medium,Triaged] https://launchpad.net/bugs/70688814:02
ubot4Launchpad bug 692772 in ubuntuone-control-panel (Ubuntu) (and 1 other project) "Visual improvements (affects: 1) (heat: 102)" [High,Triaged] https://launchpad.net/bugs/69277214:02
ralsinaDONE: reviews, talked to lots of people, did my canonicaladmin work, read code.14:03
ralsinaTODO: reviews, management stuff, someday actual coding ;-)14:03
ralsinaBLOCKED: no14:03
ralsinavds!14:03
vdsDONE: second branch for #701029 proposed third ready to start14:03
vdsTODO: continuing with the views and keep checking that the back end does what it is supposed to do14:03
vdsBLOCKED: no14:03
vdsthisfred: please14:03
* ralsina doesn't LOVE or HATE anything yet14:03
thisfredDONE: messaging branch work, desktopcouch reviews, web and mobile discussions TODO: land messaging branch | discuss with alecu whatever just scrolled by BLOCKED: no14:03
thisfreddobey: a vous!14:03
dobeyλ DONE: swap day14:03
dobeyλ TODO: 3rd party apis?, evaluate SRUs for maverick, lucid bugs14:03
dobeyλ BLCK: None.14:03
dobeymandel14:03
mandelDONE: Finished FileSystemNotificaion. Finished implementation of IPC yet got stuck in a multithreading issue (some lock in a file I cannot find.)14:03
mandelTODO: Debug, debug, debug... Move to ubuntu sso on windows.14:03
mandelBLOCKED: debugging, is that being blocked?14:03
mandelHATE: Windows file system and kernel.14:03
mandeleom?14:03
nessitaone comment:14:03
mandelfirst!14:04
ralsinamandel: comments first14:04
nessitawe would be making releases today, so if you have any pending branch let us know14:04
mandelhehehe14:04
nessitadobey: can you make releases for u1sp and u1client today?14:04
ralsinaI would like a 3-line report in private of each of you about how you are doing with your assignments, so I don't have to think too hard for team leads ;-)14:05
nessitaralsina: ack14:05
=== teknico_away is now known as teknico
alecuralsina, on it.14:06
dobeyi don't think there have been any changes to protocol since the last release, but probably14:06
ralsinaargh, what's the magical command to clean debris when I abort run-tests on a desktopcouch branch?14:07
dobeyunless someone committed some change that breaks the world again14:07
alecuralsina, rm -rf?14:07
ralsinadobey: talking about breaking the world ;-)14:07
ralsinaalecu: / ? ;-)14:07
alecu:-)14:07
ralsinadobey: it seems your use-devtools branch was a bit... under-announced?14:08
nessitadobey: I just said u1sp in the sense of "if needed"14:08
ralsinadobey: please post a quick mail on our malining list when you change how others should do things mmmkay?14:08
dobeyralsina: but how others should do things didn't change. "make test" and "make lint" still work exactly the same.14:09
nessitadobey: chicharra guys use ./contrib/test *a lot* to run selective tests14:10
nessitaselectively*14:10
ralsinadobey: what nessita said14:10
dobeysigh14:15
ralsinadobey: that's why lucio sent that email today, he was frustrated by that late last night.14:17
ralsinanessita: I get errors with CardinalFang's branch too, but not at all like yours.14:17
nessitaralsina: right, mine starts with test_start_migration hanging, and then the rest is all a consequent mess, I think14:18
* dobey thinks people should pay more attention to merge proposals if they care about changes14:24
nessitais impossible to keep track of every merge proposal14:24
=== teknico is now known as teknico_away
=== IAmNotThatGuy is now known as M0hi
vdsralsina, dobey https://pastebin.canonical.com/42279/15:52
dobeyvds: that's nightlies on lucid, right?15:54
vdsdobey, yep15:55
=== teknico_away is now known as teknico
nessitaivanka1: ping16:17
ivanka1sorry nessita - am in meetings - maybe ping otto?16:18
nessitaivanka1: sure, no problem16:19
dobeylunch time16:20
=== ralsina is now known as ralsina_lunch
=== beuno is now known as beuno-lunch
=== nessita1 is now known as nessita
=== ralsina_lunch is now known as ralsina
* karni is back17:16
vadi2The 'More' link  on the ubuntuone interface, in Chrome, is not doing anything upon clicking anymore.17:34
karnibeuno-lunch: say a user wants to sync only on WIFI. we're on Mobile. we start the app, there's a Toast 'Not syncing on Mobile'. should a file download (upload) upon being tapped (uploaded) on Mobile connection? I think 'yes' is a sensible answer. That way you can still use the app without syncing at all.17:37
=== beuno-lunch is now known as beuno
beunokarni, hrm17:44
beunowell17:44
beunoyes, but only if we say "Tap to start syncing"17:44
karnibeuno: you mean, we shouldn't be able to donwload a single file (not doing sync beforehand) ?17:45
beunowait, scratch what I just said17:46
karniuhm17:46
beunoyes, I think connectivity is for auto-sync17:46
karniright17:46
beunomanual sync should always happen17:46
karniwait, so we should add a 'Sync' option somewhere, shouldn't we (in case sync on network type XXX is disabled)17:47
beunokarni, yeah, maybe we should17:48
karnibeuno: perhaps a 'refresh/sync' button in the action bar in case the sync is set to manual, or is not set to perform on current nework type :) ?17:49
beunoyeap, sounds good17:50
karnigood!17:51
ralsinadobey ping17:53
dobeyhi17:55
ralsinahi dobey17:55
ralsinaI have a patch from Lucio that should go in syncdaemon but he feels he is not familiar enough with the code to do a real fix, and it should be done by facundo, who is on holiday17:56
ralsinabut... the patch does fix something for nessita, and should not break anything (famous last words, I know)17:57
ralsinahttps://pastebin.canonical.com/42289/17:57
ralsinaWhat do you think?17:57
dobeylooks ok to me.17:58
nessitaralsina: it fixes a real bug that happened to hit me ;-)17:58
ralsinanessita: I know it's a real bug :-)17:58
dobeyat least, having the "restart the upload" code cancel any current upload of that file, makes perfect sense to me17:59
ralsinacool, can you merge it?17:59
mandelthisfred: you ran away from #desktopcouch you evil…. hehe17:59
dobeyralsina: lucio and facundo are both on holiday?18:00
ralsinalucio is here, facundo is on holiday18:00
thisfredmandel: oops, that was unintentional, I promise!18:00
mandelthisfred: branch is done fixing the pep8 issues, here it is: https://code.launchpad.net/~mandel/desktopcouch/fix_pep8_from_windows/+merge/4743318:01
thisfredmandel: you rock!18:01
ryedobey, is there anything else I can do to get lucid-backport-work into lucid? btw -there's a long partial file fix branch proposed against your lucid-backport one for sd18:01
mandelthisfred: I guesses so, but it was funny nervertheless hehe :)18:01
dobeyralsina: then i guess he should propose a branch for it :)18:01
ralsinabut lucio says he doesn't understand the area enough for a real fix, this is a temporary one until facundo comes back18:01
mandelthisfred: some of the errors where very stupid ones, like not allowing a white space infront of [… that is lame18:02
mandelthisfred: need to go, if you need anything else, ping me here and I'll see it later :)18:02
thisfredmandel: no, that's pep8, that's how you write python :)18:02
dobeyrye: i saw the proposal but haven't had time to look at it yet. lucid is in SRU freeze atm. and there are some other fixes that need to go into stable-1-2 as well18:02
mandeltodos: laters!18:02
dobeythisfred: that's what he said. it's lame :)18:03
thisfredmandel: you always break after [ or (18:03
thisfreddobey: it prevents confusion. Some pep8 decisions are pretty arbitrary, but I think this one is unambiguously the right thing™18:04
thisfredmakes it immediately clear the statement is ongoing, even before scanning the next line18:04
dobeyralsina: i don't want to merge a "temporary fix" to trunk without a proposal and linked bug; if it's worthy of going in trunk, it's worthy of a proposal. how critical/important is the bug? just one hit from nessita, or lots of users reporting it? can it just wait for facundo's return to be fixed?18:07
dobeyrye: do those two bugs not exist in maverick? if not, do you know when they were fixed?18:09
ralsinadobey: I agree with you. I'll ask lucio to propose a branch and if he can't for whatever reason, I will.18:10
ralsinaBut no, it can't wait until facundo comes back :-(18:10
ralsinadobey: https://code.launchpad.net/~lucio.torre/ubuntuone-client/add-upload-cancel/+merge/4743518:23
ralsinaI approved it so with your seal of approval it's in18:23
dobeyi think nessita should review/test it, given that she is hitting the bug18:25
nessitaI am doing it rightnow18:25
nessitaanyways, the bug is not reproducible so no I'm not able to fieldtest it18:26
dobeyoh18:26
ryenessita, what bug?18:45
nessitarye: Bug #70523118:47
ubot4Launchpad bug 705231 in ubuntuone-client "ActionQueue.unqueue gives exceptions.ValueError: list.remove(x): x not in list (affects: 1) (heat: 127)" [High,Confirmed] https://launchpad.net/bugs/70523118:47
ryenessita, hm, i had that18:48
nessitarye: you did?!?!?! did you report it?18:49
ryenessita, well, at that time i was in Unlink() everything mode so.. no. Checking now...18:50
karnibeuno: I've been working on connectivit/sync now. One question about the action bar button though. I'm no longer sure if the sync/refresh button should be there. Sync is more an app-wide operation rather than folder-level. Or at least, volume-level. I think we should either 1. let user manually sync single volume 2. let the user sync manually, but from the Dashboard options menu, for instance.19:20
karnibeuno: any thoughts/ideas?19:20
beunokarni, I think general sync, and individual files19:21
* karni processes in context of the app19:21
karnibeuno: "general sync" :: Menu -> Sync [visible in case sync is disabled or preferred network not available]19:22
karnibeuno: "individual file" :: file -> context menu -> Sync19:22
karnibeuno: now, the thing is..19:23
karnibeuno: to know if a single file has been updated since the last use, we need to ask for the volume generation19:23
* beuno nods19:23
karnibeuno: so we basically initiate sync process for a single volume. I could bring that down to a single file,19:23
beunoI think I'm mostly referring to downloading a specific file, rather than syncing, but it doesn't change a lot19:24
karniright.. it doesn't, because we have to know the new nodeId for that file19:24
karni(if it has changed)19:24
* beuno nods19:24
karniverterok: can I request nodeState info in old-style if I use set-capabilities:generations ?19:25
karniverterok: i.e. if my client sets the caps to generations (among others), can I still ask for info on a single file ?19:25
karniprobably that would involve setting another caps of nodeState, but I'm not sure beuno if they won't conflict19:25
karni__lucio__: can you have a look at the question I asked verterok ? ↑19:26
* beuno does not know a lot about that part of the code19:26
verterokkarni: you already have the info...why do you need to do a query on a single node?19:26
karniverterok: imagine I don't ;) ok, so the thing is19:27
karniverterok: let's assume the user doesn't want to sync all the files19:27
karnibeuno: but wants to fetch one of them. if it was updated, we don't know the new nodeId19:27
karniops that was to verterok19:27
karniverterok: I know this is very different from the "PC" version, but it's been like that evar hahah :)19:28
karnipartial sync - is not what U1 was designed for19:28
verterokkarni: if you don't know the node id how do you have the file? :)19:28
karniverterok: bah, I meant the new contentHash, sorry19:28
karniverterok: we need to know the new hash to fetch the file, don't we?19:28
karnione sec.. I should have checked that before asking19:28
karnigetContent(volume, node, hash, null, os) yes19:29
karniverterok: we need to know the hash of the file to get it. we may now the previous hash, but we don't know if it has changed since.19:29
karniverterok: so the question is - can I fetch the file in AndroidU1 style, using a client that I've told to use 'generations' capability19:30
verterokkarni: why not use getdelta?19:30
karniverterok: if I use getDelta, I'll get meta info of all the files from the volume, right?19:31
verterok__lucio__: ^ I'm wrong about using getdelta instead of query19:31
verterokkarni: yes19:31
verterokkarni: just to make it simpler...but I asume you can use query with a generations-enabled client...the issue here is that query is deprecated19:33
__lucio__karni, if you want to do limited access, just use the rest api19:33
verterokkarni: it will go away sooner or later (I hope)19:33
karniverterok: aha :/19:33
__lucio__if you can travel to the future19:33
verterokjajaja19:33
karnithat's not funny ;d19:33
karniyou have the REST for me? ;d19:33
beunokarni, we can do that19:33
karni:)19:33
beunovds, ping19:33
verterokkarni: __lucio__ has a rent-a-time-machine shop  :p19:34
karni:O19:34
beunokarni, stalk vds about it19:34
karniare you kidding me guys :D establishing a connection is such a PITA, and we have the rest api working ;D?19:34
beunohe's been secretly working on them19:34
karnihahahah19:34
karnioh man19:34
karniI mean, I know he has, but didn't now on what stage it is19:35
beunowell19:35
__lucio__karni, keep in mind that the only way we have for notifications (ie, realtime sync) is with the connectin established19:35
karnianyway, verterok  - yes, that's what I need. query on a generations-enabled client... and that'll depricate, as you say19:35
beunowe could prioritize based on what you need, maybe19:35
karni__lucio__: yes, that's correct19:35
karni__lucio__: but 6-7 seconds of setup (best case) is still long for a mobile app19:35
karnisetup = connection+auth+setup19:35
__lucio__karni, so you plan on dropping realtime?19:36
karni__lucio__: the thing is, we never assumed realtime. keeping that connection alive all the time, and the service in background, is a phone-killer19:37
karniCPU/batt/etc, no way we could do that. so, instead,19:37
vdsbeuno, pong19:37
karni__lucio__: we settled for periodic sync, starting from 5 min, up to a daily sync19:37
__lucio__karni, scanning each directory to find the diff is a no-no19:37
vdshi karni19:37
karnivds: hello! just a sec my friend :)19:37
__lucio__karni, you need generations for this.19:38
karni__lucio__: what do you mean. I'm using generations. the thing is,19:38
karni__lucio__: we perform a down-sync at the moment of favourite items only19:38
karni__lucio__: I'm not doing a local-rescan every time I sync. not at all ATM actually19:38
__lucio__karni, this is all about server rescan19:39
karni__lucio__: perhaps I misunderstand something?19:39
karni__lucio__: ok. so, whenever I ask for delta, it does a rescan?19:39
__lucio__karni, ideally, when you connect every 5 minutes or whatever, you should say "tell me all the changes since X"19:39
karni__lucio__: that's what I do :)19:39
karni__lucio__: and if the user is not using the app ATM, I drop the connection ASAP when I'm done syncing.19:40
__lucio__karni, then i dont follow what do you want to use query for19:40
karnivds: sorry to keep you waiting.19:40
karni__lucio__: Imagine a user is on Mobile connection, not WIFI. and it's expensive (happens)19:40
__lucio__yes19:40
vdskarni, np for me but I'll be EODing in 30 mins, I'll be online tomorrow all day19:40
karni__lucio__: And he wants to download a single file, without pefroming full sync.19:40
karnivds: ACK!19:41
karni__lucio__: If the user wants 1 file19:41
karni__lucio__: he needs to have it's new hash. Thus, we want this info without calling getDelta for a whole volume, which is a waste of bandwith19:41
karnivds: ok, so it seems I got an ACK from the beuno for you to reveal your rest api secrets to me19:42
karnivds: not sure if I'll be converting to the rest at this very moment, but this is probably the way Ubuntu One Files for Android will go19:42
beunoand more importantly, to coordinate and see what would help karni the most19:42
karniexactly. thank you beuno19:42
vdskarni, there are no real secrets :) it's just that I'm not done yet :) but sure we can talk about the api19:43
karnivds: The thing is, U1 was never designed for a partial sync. And that's what I'm forcing it to do ATM :)19:43
__lucio__karni, maybe i am mixing stuff because i dont think we have a "get delta" for the rest api. you would need that if you want to use the rest api.19:43
vdskarni, the rest apis is not going to do any sync19:43
karni__lucio__: perhaps. that's what you know, but I dont :)19:44
karnivds: I'd like to hear what it can do, though :)19:44
vdskarni, REST CRUD on all the volumes19:44
karnivds: read. good. can we fetch info about a single file?19:45
vdskarni, yes19:45
karnivds: I assume the same goes to volumes? (i.e. know the volume last generation)19:45
karni__lucio__: plus, queries over https are much more lightweight than an on-going connection. I know I sometimes can sound confusing, but it's all due to the fact I'm thinking the android/mobile-way, which is quite different from the regular syncdaemon19:48
karnivds: don't timeout on me friend, I soo need to know that or I won't be able to sleep hahahh19:49
karnivds: can we do the same with volumes? fetch the last gen. number?19:49
vdskarni, yes, sorry :)19:49
karninp np :)19:49
karnithis is great!19:49
karnivds: now, the most important question hahah. what's up with the getDelta since X with the rest api :)? can we do that?19:50
karnivds: i.e. if we already asked what's the last gen, can we know which files changed19:50
vdskarni, I have no idea what is it,19:50
vdsaquarius, ^^19:51
karnivds: it means, if I know a volume generation is 419:51
karnivds: and the server holds generation 1019:51
karnivds: if I ask for delta(4) it means, "what files have changed since generation 4?"19:51
beunoif it doesn't it shouldn't be hard I think19:51
karnivds: so basically, it's a list of iles19:51
* karni nods19:51
__lucio__all the support is in the dal19:51
__lucio__should be easy19:52
__lucio__and it would be great to have that19:52
vdskarni, it's not in the original specs I guess we can add it, I'm glad to work on it, we need to check with all the others if/when we want to do it19:52
karniThe thing is guys, I was poking people (aquarius, beuno, other) about the rest api since some time already, because I know it would be much more lightweight and suitable for the android app.19:53
vdskarni, I hope to reach a good point this week and deliver REST CRUD on everything apart from share19:53
karnivds: that would be sooo awesome, so swesome19:53
beunovds, karni would be a good sample developer to test these apis on *wink*19:54
karniSo, although the client with generations is cool, REST over http is what I've been looking forward to for quite a while :)19:54
karnibeuno: definitely19:54
vdsbeuno, as I said I'll be more than happy to go ahead and spend more time on this, we need to check what the schedule says19:54
aquariuskarni, the problem with the rest API is that it's not suitable for notification of changes, and I don't want you to repeatedly poll the rest API. Getting notification of a changed file as soon as it happens is what the storage protocol is *for*19:54
ralsinavds: I have 560MB of dev dependencies to download now. Don't worry, I'll do the review  but it may take an hour or two.19:55
beunovds, right. Just so you know, you're back with the web team for 4 weeks to so starting next week  \o/19:55
vdsbeuno, I know that :D :D19:55
karniaquarius: indeed. the thing is, the connection gets *really* slow in the evenings.19:55
beunoso get some sleep this weekend!19:55
aquariuskarni, why?19:56
vdsahahahaha :)19:56
karniaquarius: so slow that I would rate the app 1/5 if I was a regular user. unusable.19:56
aquariuskarni, that doesn't happen on the desktop, does it?19:56
karniaquarius: I'm not sure, but it's repeatable19:56
ralsinaE: Package 'python-canonical-payment-client' has no installation candidate19:56
ralsina... what am I missing here?19:56
beunoaquarius, it does19:56
beunoit's slow api servers19:56
karniaquarius: you never know, I don't observe it so closely. plus19:56
beunoralsina, the PPA configured?19:56
aquariuskarni, and you are as sure as hell not going to provide a better service if you repeatedly poll the rest API once every ten seconds :)19:57
pedronisralsina, is not really possible to test server stuff on natty19:57
karniaquarius: new connections are slowe, persisting connections don't get so affected19:57
ralsinapedronis: ooook19:57
pedronisralsina, the deps are not built yet19:57
beunoralsina, want me to pick up that review for you?19:57
pedronisetc etc19:57
ralsinabeuno: please19:57
karniaquarius: definitely, but I don't plan to poll every 10 seconds ;)19:57
ralsinabeuno: my lucid and maverick VMs need rebuilding :-(19:57
aquariuskarni, and there's no way to ask the rest API "what has changed since X"...and there's not planned to be, either, because if you want that info then that's what the storage protocol is for :(19:57
karnibeuno: vds: __lucio__ ↑ :<19:58
beunowell19:58
* ralsina has destroyed more ubuntu installs this month than on the last 5 years19:58
beunoit could be19:58
beunothis is something we need to talk about19:58
pedronisaquarius, well it would not be hard to had get delta to the rest api19:58
karniaquarius: ok, perhaps I should share more details19:58
pedroniss/had/add19:58
karniaquarius: regular connect (connect+auth+setup) takes 6-7 seconds usually19:58
aquariuspedronis, I agree it's not hard technically. What concerns me is that providing that flat out encourages people to poll the rest API regularly, which is not what we want :(19:59
karniaquarius: that get's over 2-3 minutes in the evenings19:59
__lucio__aquarius, the only reason the protocol still survives is because its a great interface for managing versioned clients and because we need a channel to push stuff.19:59
__lucio__if we have continuos updates on the phones, reason one goes away19:59
karniaquarius: each volume getDelta takes another.. 1-2 minutes per volume (just the meta)19:59
__lucio__if we dont need notifications, reason 2 goes away19:59
vdsralsina, check if that can be traded with LP karma points ;)19:59
pedronisaquarius, well long term will need to put rate limiting on the apis anyway, that's what people end up doing anyway19:59
aquarius__lucio__, nah, there's another reason: the protocol is good for notifying you instantly that something has changed in a long-lived connection.19:59
karniaquarius: if you have few volumes, the initial start+sync takes... 6+ _minutes_ in the evenings20:00
aquariuskarni, that's quite shit, I agree20:00
__lucio__aquarius, yeah, thats what i mean by push20:00
karniaquarius: yes, and we're not keeping that connection live on Android20:00
karniaquarius: we do periodic sync20:00
karniaquarius: 5 min, ..., up to daily20:00
aquarius__lucio__, we do not have that in the rest API. We could do, but we'd just have to invent it from scratch, which is something that needs thinking about and planning. It's not something where we can just throw it together in two minutes :)20:01
karniaquarius: thus, we drop the connection *if* the user is not using the app at the moment20:01
aquariuskarni, I didn't know about the api servers taking 6 minutes to connect :(20:01
karniaquarius: thus, you get notifications -- as long as you're using the app or it's during the scheduled periodic sync20:01
__lucio__get delta should not take 2 minutes, it does not to me, and on average we finish a delta under a second20:01
aquariusbeuno, is the delay on the api servers just because they're overloaded?20:01
aquariusbrb20:01
__lucio__aquarius, unless we are broken, which is not all the time, its not 6 minutes20:01
aquarius__lucio__, karni. karni, __lucio__. You disagree on this. ;-)20:02
karni__lucio__: that has been tested multiple times in live environment over wifi20:02
__lucio__aquarius, being on ec2 also impacts a lot. we cant do more parallel queries to speed things up because of losa issues20:02
__lucio__karni, yeah, same here, and i get to also see the result of every single get delta that is done20:02
karni__lucio__: my full sync takes ~30 seconds including conn+setup+auth20:03
karni__lucio__: in the evenings, it takes ages.20:03
karni__lucio__: how can I prove that?20:03
verterokkarni: what does setup+auth involves?20:03
karniverterok: it's just connect(), authenticate() and setup() [set caps]20:04
karninothing else20:04
karniverterok: methods from your Client (to be specific)20:04
verterokkarni: and that takes 30 seconds?20:04
karniit's not my custom stuff where I do tons of sh.t :)20:04
karniverterok: no, +sync20:05
karniverterok: that takes 5-7 seconds20:05
karni6-720:05
karniconnect, authenticate, setup takes 6-7 seconds. no less. never.20:05
karnino wifi20:05
karni*on20:05
karniverterok: 30 seconds was an example of my U1 connect+sync20:06
karnilet me check how things run ATM20:06
__lucio__right now, from nothing to working, 8 seconds20:06
__lucio__anyhow20:07
verterokkarni: do you know/remember if we are using the "lite" version of protobuf?20:07
__lucio__we know we have performance issues20:07
karniverterok: I think we are20:07
__lucio__and we are working to get a more stable service20:07
verterokkarni: k, thanks20:07
* karni checks the app20:08
karni__lucio__: I confirm, at the moment things run smoothly20:09
karniI can ping you guys when things go slow next time for me.20:09
karniaquarius: beuno: I imagine it's not a one day or even one week matter, but I would definitely want at least to look into the API - one of the reason havinb query() depricating soon [thus, not being able to sync one file without syncing a whole volume's meta]20:15
aquariuskarni, indeed, but if we're gonna make the rest API be a complete and total replacement for the storage protocol, then we need to plan it out right, and none of that planning has happened yet. :)20:16
karniaquarius: beuno: plus, we're not keeping that connection majority of the time anyway. It would have too huge impact on the battery life.20:16
karniaquarius: I understand that.20:17
aquariusI'm seriously thinking about how we shouls use c2dm for this sort of thing...20:17
karniaquarius: Right, as we talked before.20:17
karniaquarius: Even if we get notified via c2dm, connect takes 6-7 secs with current ubuntuone-java-storageprotocol implementation. (not that is bad! it's an excellend piece of great code)20:18
karniUnless we would use the REST API at that stage.20:19
aquariuskarni, yeah, but six seconds is not that much :)20:19
aquariuswe could just use c2dm for notifications, never ever ever ever ever poll, and just use the rest API.20:19
karniaquarius: true, unless we hit the performance issues again20:20
aquarius(what happens to c2dm messages if you're switched off? are they stacked, or thrown away?20:20
karniaquarius: indeed, that's what I meant20:20
karniaquarius: google servers make sure it gets to the receiver of the msg20:20
karniaquarius: they are aggregated20:20
verterokkarni: 6-7 sec to do the connect+auth is a lot...the python storageprotocol takes 2 sec to do it20:20
karniverterok: the biggest part is the auth20:21
karniverterok: it produces tons of GC..20:21
aquariuskarni, hm...c2dm to notify of a change, plus the rest api, might be a really good way to do it.20:21
karniverterok: When I see like.. 12 lines of GC in the logcat, I know it's authenticate() -- that's what taking most of the time20:21
verterokkarni: so, lets fix it if that's the problem...2 sec is acceptable?20:21
karniverterok: hell yea! :D20:21
karniverterok: but, there's a little but.20:22
karniverterok: what worries me is not as much those 6-7 secs, but performance issues in the evenings. atm all runs nicely20:22
beunokarni, we're working on that20:22
beunowe have a plan20:22
karnibut previous evenings the app was unusable (including yesterday)20:22
karnibeuno: perfect!20:22
beunoand it's being executed20:22
beunoso don't worry about that in the medium-term20:22
karniverterok: if so ↑, then 2 secs would be great!20:23
karnibeuno: uhm :)20:23
karniaquarius: that's how majority of the android apps run I think. it's so lightweight.20:23
aquariusbeuno, your thoughts on c2dm+rest? would require us implementing c2dm push in the server, plus a dependency on Google, but it'd make the client really lightweight :)20:24
beunoaquarius, well, I think that's probably the way to go eventually20:24
beunoand not just for this20:24
beunoit will obviously not happen this cycle because of all the infrastructure needed20:25
karniyup20:25
beunoso we need to figure out how to deliver, and plan for that20:25
aquariusbeuno, yeah, that's what I think. I'm not sure that the rest API is the place to do streaming instant notifications, but I agree it's the place to get and put and manipulate files and folders20:25
karniaquarius: I think c2dm+rest is the perfect solution. Not sure what's a replacement of c2dm on an iPhone though ;D20:26
karniaquarius: I think apple suggest regular Push for such apps on an iPhone. Just saying.20:27
beunoso20:27
beunoYOGA20:27
beunobbl20:27
karniOk guys, I'll ping next time we have a slowdown (if we do, hopefully not), and it'd be great to speed up that auth on the java-storageprotocol side (maybe it's the oauth library, maybe it's android problem itself, we've been there with verterok once already.. )20:28
verterokkarni: FWIW, on my desktop conn+auth+setcaps takes 3.5 seconds20:29
karniverterok: compare that to a 355MHz procesor ;)?20:29
karniperhaps on a Snapdragon 1GHz phone, it takes less than 6 secs20:30
karniaquarius: you have some fancy google phone, don't you20:30
karniverterok: I'll use my friend to test tomorrow ;)20:31
verterokkarni: ok, I'll try to profile this thing20:31
karniverterok: lovely20:31
aquariuskarni, I do. Nexus S.20:32
karniaquarius: do you have U1F installed? (doesn't have to be the latest version)20:32
verterokkarni, aquarius, beuno: we should compare json vs protobuf, in case you plan to move to the rest api to get more speed json parsing can be quite expensive if you don't use the right parser20:33
verterokand with that, I will go back to my stuff20:33
verterok:)20:33
karniverterok: thank you :) bye bye!20:33
aquariuskarni, I don't, because since I got this new phone you were working on a new version so I was holding off :)20:33
karniaquarius: ha! then I'll keep the new version for few more days while I'm wrapping up sync and connection settings, I want it to look good for you.20:34
karniaquarius: I'll use my friends desire to test :)20:34
* karni gets some food20:35
nessitaralsina: ussoc 1.1.9 is now released, so is u1cp 0.8.020:37
ralsinanessita: yay!20:37
nessita(packages already in main repo)20:37
nessitabye all!21:01
=== ivanka is now known as ivanka-train
karni__lucio__: you around? connect()+authenticate()+setup() [set caps] took 59 now22:01
karni__lucio__: seconds, that is.22:01
karniaquarius: what exactly is "description: doing server rescan" ? (PC syncdaemon) what is happening at that stage? client is rescanning the server? [I'm on 10.04]22:05
beunokarni, yes, 10.04 has a slow server-side rescan22:05
__lucio__karni, 10 seconds for me22:05
beuno10.10 is super quicj22:05
beuno*quick22:05
karni__lucio__: thanks, let me recheck22:06
karnibeuno: aha, thanks22:06
__lucio__karni, as beuno said, if you have 10.04, then forget about it, its a pain.22:06
karni__lucio__: now it's fine. <10secs for sure. looks like your plan is working. if I'll notice major slowdown, I'll ping you.22:07
karni__lucio__: ACK. I was just testing how fast does my PC syncdaemon connect+auth (which was fast)22:07
karni__lucio__: hmm. it took 15 secs (still, not bad). it's pretty undeterministic considering different server load at any given moment. however, still slower than PC.22:09
karnianyway, thanks guys. back to work.22:09
__lucio__karni, what took 15 secodns?22:13
karni__lucio__: conn+auth+set_caps . forget it, it's still close to 10, which you had :)22:13
karni__lucio__: I'll let you know when it gets considerably slower :)22:14
karni(hopefully not ^^)22:14
__lucio__cool22:14
karni__lucio__: thanks for your support and feedback22:14
__lucio__np22:15
dobeylater all, out for now22:34
karnibye dobey22:34

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!