/srv/irclogs.ubuntu.com/2011/06/03/#ubuntuone.txt

faganmorning08:59
ralsinamorning fagan09:10
faganralsina: morning09:10
faganSo when do you want to walk me through the testing stuff09:12
mandelmorning :)09:13
faganmorning mandel09:13
mandelralsina, fagan: can you run the tests on linux for : https://code.launchpad.net/~mandel/ubuntuone-client/provide_credentials_management/+merge/6262709:17
ralsinamandel: sure!09:17
faganmandel: sure09:17
mandelthx ^ 209:18
ralsinaand god morning!09:18
mandel:)09:21
mandelralsina, fagan: did the tests pass?09:29
ralsinamandel: still running09:29
mandelralsina: wow, they are slow...09:29
ralsinayep09:29
ralsinafagan: about the testing stuff, I am probably the worst one to walk you through it ;-)09:31
ralsinabut in any case, yesterday, or the day before, nessita told you where the tests for that piece of code are.09:31
faganautogen and make worked fine09:32
mandelfagan: and make check?09:32
faganralsina: well I know where the tests are I just want a walkthough of mocker and how to use it in relation to the code and all that09:32
faganmandel: running now09:32
ralsinafagan: read the other tests. If you don't understand something, ping me09:33
ralsinanot all of course, just one or two ;-)09:33
ralsinamandel: make test is successful09:34
ralsinaBTW: that thing has exactly 2222 tests!09:34
mandelralsina: dammed, take a look at the merge proposal, it fails on tarmac...09:34
faganErrors09:34
ralsinamandel: let me check it09:34
mandelfagan: got errors? can you copy paste them?09:34
faganill have to redo the tests my terminal doesnt hold logs longer than 999 lines or something09:35
faganim on 11.10 so that might be one of the issues though09:35
faganif the tests are passing for ralsina09:36
ralsinamandel: I tested the branch, not the merged branch09:36
ralsinaprobably you have to merge to see the failures09:36
mandelralsina: can you try that?09:36
ralsinaplus, I will surely not get "exceptions.OSError: Too many open files" since I have that set waaaaay high09:36
ralsinaI am trying the merged branch now09:37
faganok give it 10 minutes and ill give the complete log09:37
ralsinaonce unit testing starts taking 10 minutes ... oh, well.09:43
mandelI really dont understand it, mine merged with trunk works ok09:44
ralsinamandel: the too many open files error may be tarmac doing something09:45
mandelyes, I suspect so…09:45
mandelwhich is a pain in the ass09:45
fagantoo many open files09:46
mandelfagan: you got that too?09:46
faganand there is another about defer timing out09:46
faganyep09:46
ralsinafagan: pastebin plz09:47
faganralsina: im still getting it09:47
faganralsina: takes ages09:47
faganI just was looking down what was there09:47
mandelso, any luck?09:52
faganmandel: still going09:52
ralsinamandel: still running09:53
ralsinaPASSED (skips=1, successes=2223)09:53
mandelthen I dont understand...09:53
ralsinamandel: so, can't help you with that :-(09:53
mandelleches!!09:54
ralsinamaybe fagan can give you shell access to his box?09:56
mandelralsina: the wird things is also to get this: exceptions.AttributeError: 'generator' object has no attribute 'addBoth'09:57
ralsinathat's weird, yes09:57
mandelthat does not make sense… the too many open files ok, but this..09:57
ralsinacould be twisted versions?09:57
mandelI dont think there should be any issues, also I'm running N as tarmac should be doing09:58
mandeland between 10 and 11 there is not such a diff09:58
ralsinaI am on N too09:58
faganim running O09:59
ralsinabut I am not fully up to date! Something else to try...09:59
faganso it might be a different version09:59
ralsinamandel: trying with latest devtools etc10:00
mandelralsina: I'm gonna see if I'm not up to date, but I should..10:00
ralsinamandel: ok10:00
faganhmmmmm so from what I remember about reading and writing to files you are only allowed to open a few buffers at any given time is that the issue thats coming up10:02
fagan9000 lines and counting of the log10:09
ralsinafagan: the limit on open files on linux is like 8192 files by default10:09
ralsinamandel: got errors now!10:10
mandelralsina: me too, seems I was not up to date...10:10
faganralsina: but the buffer limit is like 8 and 2 are open by default10:10
ralsinathe too many open files one10:10
ralsinafagan: no10:10
mandelralsina: bloody hell…. I wonder if this happens in a clean branh from trunk10:10
mandelralsina: if it does we are blocked...10:11
ralsinafagan: there is no such thing as buffers separate from files, that was only true in, like DOS 4.010:11
* fagan looks though his notes to find the bit that explains this 10:11
ralsinamandel: well, another thing to try, but I am betting on devtools being the guilty party10:11
mandelralsina: I hope so… the ssue s from pyinotify, which is an evil beast10:11
ralsinamandel: I only had a few updates, and devtools is the only obvious one that could hurt this10:12
ralsinaalso memory usage has gone to hell so I am guessing tests are not cleaning after themselves10:12
mandelralsina: yes, but I wonder what is the trial runner doing… it could be that the tests are not correctly tearing down10:13
mandelralsina: hehe though the same, great minds...10:13
ralsinadobey was workin on tardown last night, too10:13
mandelok, so we are block regarding tarmac until we roll back and we push the old one to tarmac10:13
mandelright?10:13
ralsinaI think so10:16
ralsinaor maybe dobey looks at it and says "doh"10:16
ralsinawe can stack branches (scary)10:16
ralsinawhy the hell did I accept a plane ticket that leaves at 8AM? Was I drunk?10:18
faganralsina: yeah what I was remembering was io streams10:21
mandelralsina: dammed, stacking branches is dangaerous, we have already done that...10:21
ralsinamandel: thus the scary part10:21
faganralsina: 3 are open by default not 2 as well10:22
ralsinadobey should be here in ~4 hours10:22
mandelralsina: when does dobey arrive?10:22
ralsinafagan: what do you mean "io streams"?10:22
mandelralsina: dammed you are too fast for me :)10:22
faganralsina: stdio, stderr, stdin10:22
ralsinastdin/stdout/stderr? Those are regular file descriptors in linux (and most OSs I suppose)10:22
faganralsina: and when you open a file it creates a buffer like those10:22
fagan(they are just buffers)10:23
ralsinayes, file IO is buffered. That doesn't mean you can't open 8192 files. Try it :-)10:23
ralsinaha, the tests have killed my VM :-(10:23
faganralsina: well you can open them but you cant use them all at once maybe the OS makes that seem a little more seemless than it is10:24
ralsinafagan: try it!10:24
ralsinastart 4000 threads writing to 4000 files10:24
faganralsina: actually that definitely will kill itself10:24
ralsinaor one thread with 8000 open files and write to them round robin :-)10:24
faganralsina: did that in class one day10:24
ralsinafagan: t r y i t10:24
faganhah10:25
ralsinatry the 8000 files one. 4000 threads need eef to run ;-)10:25
ralsinas/eef/beef/10:25
faganralsina: well it will work if you write them in a round robin since it will only have 1 buffer at 1 time10:25
ralsinafagan: and how do you write to two files at once without threads?10:26
faganif I did like 20 at 1 time the OS would probably switch between them10:26
ralsinaand no, you will have 8000 buffers, unless you flush them10:26
faganwell there would be some OS magic there id say10:27
ralsinafagan: again. Try it. It's 10 lines. I will wait.10:27
faganill do it a bit later10:27
* ralsina brings out the subborn-o-meter10:27
faganstill waiting on the tests10:27
ralsinadamn, sTubborn-o-meter10:27
ralsinafagan, if you got the error, kill it, we already reproduced it10:27
faganah ok then10:28
faganactually im getting some different ones too10:28
faganlike there are a few attribute errors10:29
ralsinadamn, had to hard-reset the VM :-(10:29
mandelralsina: we can revert the last merge in ubuntuone-dev-tools to see if it works, I'll do that10:32
ralsinamandel: cool10:32
faganralsina: did you get the attribute errors too?10:32
ralsinafagan: can't tell since it died10:33
ralsinafagan: BTW: turns out ubuntu stats with an 1024 open file limit10:34
faganralsina: interesting id still like to see how its handled kernel level10:34
fagan(my guess is they move around the buffers and give each one time)10:35
ralsinafagan: you seem to have the impression that buffers are some scarce resource. They are not.10:35
faganralsina: yeah I know I just got a flash of the C class where they said you might have problems in some OSes10:36
ralsinaand the whole purpose of them is to make writing to files return faster. You can even not use them.10:36
ralsinafagan: yes, "in some OSes"  :-)10:37
faganIm going to let the tests finish so I can see if there are any other errors10:38
faganralsina: well windows in different times and some versions of unix that are floating about10:38
faganwell probably not any one in major use10:39
ralsinawindows seems to be 2048 by default10:39
faganralsina: that the same for dos-9810:39
ralsinaok, 512 by default, and you can raise it to 204810:39
ralsinaof course if you app has more than 100 open files, you are probably doing something wrong10:40
faganwell it wouldnt make much sense in the desktop case10:41
fagansince it would slow everything down10:42
faganand there wouldnt be anything that would need to be doing things all at once like that with files10:42
faganbut a server might have a problem if they were doing something weird10:42
faganother than that I cant see a use for it10:43
ralsinafagan: servers either use a multiprocess model (thus avoiding these limits) or ask you to raise them, or use massive caching to memory, or mmap files, or... I mean these limits have been there for 40 years, not a new problem ;-)10:43
faganralsina: true10:44
fagan11300 lines in the log :D10:45
faganralsina: race? http://paste.ubuntu.com/617390/10:46
ralsinafagan: once you get the state of the program as broken as it is when 1000 tests are not closed, anything can happen, usually errors lose relevance after a while :-)10:47
faganthere are a good few other errors than just the too many files thing10:47
faganah ok10:47
ralsinaunless that error is one of the firsts, in which case, yeah10:48
faganok then I was thinking it could have been interesting to see what errors it cmes up with10:48
fagan*comes10:48
faganjust to make sure I pastebinned the output it takes a while to load though http://paste.ubuntu.com/617393/10:53
faganI killed it though10:53
mandelralsina: so far with r 31 of dev tools tests pass, I'll test all of them from there to see when it brakes10:56
ralsinamandel: cool, at least we have a clue10:56
ralsinaI wish bzr had a bisect command like git10:56
* fagan likes10:57
* fagan likes to say "make one" when people wish 10:58
* mandel goes for a small break while tests run10:59
* fagan takes a nap10:59
ralsinalunch break for me11:02
* mandel back11:14
mandelralsina: looks like the culprit is the very last revision of ubuntuone-dev-tools, version 34, we can revert to version 33 and propose for merge..11:33
duanedesignmorning all12:00
duanedesignhmmmm, http://www.dorianselimi.com/2011/06/justin-bieber-gnulinux-os/ /me thinnks i will put this on my roomates laptop12:06
duanedesignfacundobatista: if you get a second today I had a question about {'is_directory': 'T', 'changed': 'LOCAL', 'has_metadata': 'T'} and what that is telling us.12:08
facundobatistaduanedesign, ok12:09
facundobatistaMuy buenos días a todos!12:09
facundobatistaduanedesign, well, not ok, that combination has no sense, :)12:10
facundobatistaduanedesign, do you have a branch of ubuntuone-client at hand?12:10
duanedesigni do12:12
duanedesignfacundobatista, just fetched the latest revision12:13
facundobatistaduanedesign, ok, open the ubuntuone/syncdaemon/u1fsfsm.ods file12:15
facundobatistaduanedesign, that is how we programed the Sync state machine12:16
facundobatistaduanedesign, the general desing of the file is that for each Event, you have an input state, an action, and an output state12:16
facundobatistathat input state is the combination of:12:17
facundobatista- the node has metadata locally12:17
facundobatista- the "changed" state (NONE means not changed, LOCAL means that has local changes, SERVER means that there're server changed pending to update locally)12:18
facundobatista- is_directory means that if it's a dir or a file12:18
facundobatistaand then you have some parameters according to the hash informed in the Event (if any) and errors (if any)12:18
facundobatistaduanedesign, so far so good?12:18
duanedesignyes, thank you!12:19
facundobatistaduanedesign, ok, now to be more specific about "changed"12:19
facundobatistathe "changed" attribute is not a real one, it's a combination of:12:19
facundobatista- if local_hash (the hash we have locally for the node) is equal or different to the server_hash (the hash we locally know that the server has)12:20
=== teknico is now known as teknico_away
facundobatista- if the node has a "partial" component (that means that it's being downloaded from the server)12:20
facundobatistaLOCAL means that it has no partial, and the hashes are different... in other words, that we're *uploading* the node12:21
facundobatistaduanedesign, which has no sense at all talking about directories: we never upload directories12:22
* duanedesign nods12:22
facundobatistaduanedesign, if you check the .ods, there's no input in any line for that combination (that's why the log says "Invalid input state"12:22
facundobatista)12:22
facundobatistaduanedesign, furthermore, there's no output state for that combination12:22
facundobatista(columns L to N)12:23
facundobatistaduanedesign, so, it's very very strange how the user gets to have that12:23
duanedesignfacundobatista: thank you that is very helpful12:24
facundobatistaotoh, we have seen cases like this already, and even more strange ones... and we also have seen a lot of cases of corrupted metadata in strange ways because of disk failing, energy cuts, etc12:24
duanedesignfacundobatista: is their a way to get more info about the particular file(s) that might be corrupted? Or is the user likely to just need to 'rm' the metadata folder and have it recreated?12:26
mandelfacundobatista: have you seen the failures from tarmac?12:31
mandelfacundobatista: the current version of ubuntuone-dev-tools is broken… and we wont be able to land anything until the changes are reverted and installed in the tarmac machine12:31
ralsinamandel: or wait one more hour for dobey and ask him to look at it12:35
mandelok12:36
facundobatistamandel, which failures from tarmac?12:38
facundobatistaduanedesign, you can always make the user to dump the metadata to a readable form12:38
mandelfacundobatista: let me find and example12:38
facundobatistaduanedesign, see 'contrib/dump_metadata.py'12:39
facundobatistaduanedesign, if you want to see more info there, it's easy to add12:39
mandelfacundobatista: last comment: https://code.launchpad.net/~mandel/ubuntuone-client/provide_credentials_management/+merge/6262712:39
mandelfacundobatista: all those are due to u1trial issues12:39
* mandel walks dog12:40
facundobatistamandel, exceptions.AttributeError: 'generator' object has no attribute 'addBoth' ?12:41
facundobatistamandel, ah, ok, yes, it seems to be a ubuntuone-dev-tools problem12:45
nessitagood morning everyone!13:00
facundobatistaHola nessita13:04
nessitahola facu13:05
ralsinahola nessita13:13
nessitahola ralsina, que tal?13:14
ralsinanessita: saying goodbye to people here, readying myself fr london13:15
nessitaralsina: when do you travel? today or tomorrow?13:15
ralsinatomorrow 8AM13:15
nessitathat's early :-)13:15
ralsinayeah, I don't know what I was thinking13:16
ralsinaI have to leave for the airport at 5AM13:16
faganralsina: I have to leave at 7 for my plane even though it leaves at 1113:21
fagandamn busses13:21
faganI think next time im going to book the flights around 5 so it would be easier13:22
=== teknico_away is now known as teknico
Chipacaralsina: you were thinking "yeah 8am sounds right, i get up early, have breakfast and say my goodbyes, but then have all the day in london"13:31
Chipacaralsina: forgetting that 8am flight means getting up at stupid am :)13:31
ralsinaChipaca: it probably will be "ok, I stay up all night then go to the airport like a zombie, mistakenly take a plane to Burkina Faso, end being torn apart by hyenas"13:32
ralsinaI am setting my expectations low13:33
fagansounds like a very cool place to get a weird plane to13:35
ralsinafagan: you can get there for U$S 2000 from Dublin. Only a 18 hour flight.13:42
ralsinaThen again, the older name "Upper Volta" is way cooler13:43
faganwow you could charter a jet for that price13:44
ralsinafagan: round trip13:44
ralsinait costs about the same as coming from argentina to turkey13:44
faganralsina: wow are you paying that or are you going through some other airport13:45
* fagan sees why mark owns his own jet 13:45
ralsinafagan: I am not going to london via argentina if that's what you are asking ;-)13:45
faganralsina: arent your family going back13:46
fagan(from turkey)13:46
ralsinafagan: yeah, tickets were not cheap. About 2.7K for both of them13:46
faganwow couldnt you get a connecting flight for cheaper13:47
faganIm pretty lucky everywhere in europe is pretty cheap13:47
ralsinafagan: that involves a 4-hour stop in brazil. There are not many places to connect between S.am and europe ;-)13:53
faganralsina: ah ok I forgot about timezones13:54
ralsinafagan: basically, you have to pay for a plane that's nice enough to cross the atlantic at it's longest13:54
faganralsina: and that isnt going to be entirely full13:54
ralsinafagan: yeah. While in europe flights you can fly cheap, small planes that can stop every 1000km.13:56
ralsinafagan: so, it's like comparing a ferry and a transatlantic ship13:57
faganralsina: yeah I can get to the US for about 600 depending on the time of day13:57
faganralsina: and anywhere in europe for around 200 max13:58
ralsinathat's about 6000km shorter than going to argentina13:58
ralsinaand inside europe, over 10000km shorter13:58
thisfredI've been doing the economy plus upgrades recently. (Basically you pay for an exit seat of a little more leg room) Totally worth it for me13:58
faganyeah its hard to compare a trip across the world13:59
ralsinathisfred: in british I was able to get that by being first on the online checkin13:59
thisfredgood to know, I'm flying BA as well :)13:59
ralsinaon turkish they reservethose seats until "real" checkin ;-(13:59
faganim going ryan air this time :(13:59
ralsinathisfred: you can pay 25 euros to be able to do it before the 24 hours13:59
ralsinafagan: you are fliying 90 minutes :-)14:00
faganralsina: ryan air are the devil14:00
fagan:D14:00
thisfredralsina: 75$ for me, but they wouldn't let me, because of something with  the booking (I guess because it was booked through the agent(14:00
faganand im going into standsted which doesnt have a tube or train to it I dont think14:00
ralsinathisfred: really? ouch!14:00
thisfredralsina: anyway, the exit seats were already all gone. I'm hoping this plane has economy plus section (the newer ones of this type do)14:01
ralsinaistanbul is only 1000lm farther away from argentina than london is. Surprising.14:01
thisfredwhich will be something like 200$ to upgrade to, but still worth not having to amputate my legs on arrival14:01
ralsinathisfred: yeah, that's much nicer. Specially you being so tall14:01
ralsinait seems my plane is half empty so I can probably just pick when I am in the plane14:02
thisfredI wish they would just make the seats higher, there's room there, and that would solve everyone's problems14:02
ralsinaand 4 hours is short enough14:02
ralsinathisfred: all we short guys would get embolisms :-)14:02
thisfredyeah, for anything under 6 hours I won't bother14:02
thisfredmaybe have foot rests14:03
ralsinaI am galled they dare call the metal loop things below the seats foot rests.14:03
ralsinaI would have to detach my feet to use them14:03
thisfredOh they have foot rests? :)14:03
ralsinathisfred: not really ;-)14:04
thisfredI'm usually with my knees by my ears as it is :)14:04
ralsinaI need wider seats, but not because of my belly, but because of my shoulders :-(14:04
thisfredanyway, London's only 7 hours, which is survivable. B.A. was really painful at 11 hours14:04
ralsinaI end in a semi-permanent shrug :-)14:05
ralsinaI have a 16 hour return flight next week...14:05
* ralsina considers upgrades14:05
thisfredyeah that too: my arms are too long for the arm rests so I have to tuck them in and hunch14:05
thisfredralsina: the economy plus seats really are awesome for me, but I don't know that they're wider14:06
nessitaOOPS14:06
thisfredralsina: upgrading on such a long flight will get you a meeeeellion miles though14:06
nessitame14:06
thisfredme14:06
ralsinathe business upgrade is ridiculously expensive, so I will try economy plus14:06
ralsiname14:06
mandelme14:07
ralsinafagan?14:07
nessitadobey, fagan?14:07
=== m_conley_away is now known as m_conley
nessitaDONE: folder subscription is now landed, started debugging DBus failures on test suite14:07
nessitaTODO: more DBus debugging14:07
nessitaBLOCKED: not yet14:07
nessitaNEXT: thisfred14:07
thisfredDONE: Bug #791927, reviews, reinstalled laptop14:07
thisfredTODO: possibly assist with couchdb diagnostic session | u1-unity bugs14:08
thisfredBLOCKED: no14:08
thisfredNEXT: ralsina14:08
ubot4`Launchpad bug 791927 in desktopcouch (Ubuntu) "apport hook in source package not installed (affects: 1) (heat: 6)" [Medium,Confirmed] https://launchpad.net/bugs/79192714:08
ralsinaDONE: pinged lots of people, organized my trip, reviews, not much else14:08
ralsinaTODO: travel14:08
ralsinaBLOCKED: no14:08
ralsinamandel?14:08
mandelDONE: Look at why tarmac reject branches Ubuntuone-dev-tools rev 34 is broken, we have to revert to 33.14:09
mandelTODO: Land branches. Work on sdtool won windows.14:09
mandelBLOCKED:not really14:09
mandelno one next I suppose...14:09
nessitadobey and fagan, go!14:09
dobeyme14:10
dobeyλ DONE: debugging horribly broken twisted+dbus test stuff14:10
dobeyλ TODO: more debugging, bug #789300, bug #771488, expenses14:10
dobeyλ BLCK: None.14:10
ubot4`Launchpad bug 789300 in ubuntuone-dev-tools "DBusTestCase needs to work with Qt main loop as well (affects: 1) (heat: 6)" [Critical,In progress] https://launchpad.net/bugs/78930014:10
ubot4`Launchpad bug 771488 in ubuntuone-dev-tools "u1trial should unset GTK_MODULES (affects: 1) (heat: 5)" [Medium,In progress] https://launchpad.net/bugs/77148814:10
mandeldobey: have you seen the state of tarmac?14:11
nessitaexpenses!!!14:11
* nessita goes to canonicaladmin14:11
mandeldobey: looks like the last revision of u1trial is very broken14:11
nessitamandel: to me that looks like an env issue (Too many open files)?14:11
dobeynessita: no, it's weird14:11
mandelnessita: if yo revert tone rev it is fixed14:11
* ralsina has not yet passed the expenses from the BUENOS AIRES sprint. Oh well.14:12
nessitamandel: you get the same error in your computer?14:12
mandelis nothing to do with the env, it can be be reproduce if you update u1trial to the latests version14:12
mandelnessita: ^14:12
ralsinanessita: mandel, fagan and I with latest devtools14:12
dobeymandel: which is weird14:12
nessitadobey: did you land something yesterday?14:12
ralsinadobey: tests are not tearing down and it's reaching 1024 files open. It also eats tons of ram14:13
mandeldobey: I do know it was the latests, I have tested with 31 up to find when as the issue14:13
faganme whoops14:13
mandeldobey: yeah, it crashes my testing vm hehe14:13
dobeynessita: land something where?14:14
nessitadobey: devtools14:14
* fagan was trying to get the pdf of the tickets14:14
faganDONE14:14
fagan* Got plane ticket pdf14:14
fagan* tested mandel's branch14:14
fagan* tried to get the branch tests fixed14:14
faganTODO14:14
fagan* Get the branch merged in and the tests finished14:14
faganBlocked14:14
fagan* nope14:14
dobeynessita: no14:14
nessitaweird14:14
faganI just found out that I have to pay extra for a bag14:14
fagan:/14:15
fagandamn ryan air14:15
dobeymandel: on trunk, or just on your branch?14:15
mandeldobey: trunk14:16
mandeldobey: if you go back to rev 33 it works, so there most be a bug in the latests merge14:17
mandeldobey: I think the issue is with the fact that you removed the @inlineCallbacks14:18
dobeyno14:18
mandelbut I'm not 100% sure...14:18
dobeyit's not14:18
dobeythe issue is that ALL OUR TESTS ARE COMPLETELY BROKEN14:19
mandeldobey: oh, you know already what is going on?14:19
jdobrienthe sky is falling!14:19
dobeyno i have no idea what's going on14:19
nessitaI know I'm fixing control panel and sso tests14:21
dobeynessita: have you gotten anywhere with that?14:21
nessitadobey: yes, I'm very close to have that done14:22
fagannessita: fix my tests too while your there :D14:22
nessitadobey: we were using the dbus session worng14:22
nessitadobey: I; m thinking we may wanna patch dbus. SessionBus with lambda: self.bus in devtools14:22
* fagan is willing to donate 2 beers in london for the cause 14:22
nessitadobey: that way we don't have to fix a crazy amount of tests that use dbus.SessionBus14:23
nessitadobey: what do you think?14:23
faganIm doing the break it and try again approach to fixing my set of tests14:23
dobeynessita: hrmm, that's an interesting idea14:23
dobeynessita: i wonder if we should also start a system bus daemon, and do similar with SystemBus()14:24
nessitadobey: I don t know of any app that uses the SystemBus, but maybe?14:24
nessitadobey: I will propose a branch, first confirming this "fix" the u1cp test suite14:25
dobeynessita: well, NM is on the system bus14:25
dobeynessita: or maybe we should just have the one daemon, and force SystemBus to point to it as well14:25
mandeldobey, nessita: are this dbus issues in the tests realted with the errors we have in ubuntuone-client, or is it completely diff?14:26
nessitamandel: I think your issues are the consecuence of a timing issue that was exposed by some changes that added some yields to setUp and tearDown14:27
dobeymandel: i have no idea why it is happening14:28
dobeychill and give me time to actually look at it already :)14:28
ralsinaI have to go pack, and have some quality family time. So, EOD for me. If anyone needs reviews, ping me about them, I may do them tomorrow.14:30
mandeldobey: Oh, I'm very chill, you have my trust 100% I was just wondering :)14:34
dobeymandel: http://www.youtube.com/watch?v=KCujz2HzE7M14:34
dobeywell, ok, not a great video of that scene, since the fool edited it to repeat the "just take it easy" bit14:36
nessitadobey: pacthing dbus.SessionBus works well! (though there is still some tiny bugs to fix, like releasing the bus name)14:38
dobeyhuh14:39
dobeynothing else should have to be changed for that; if it's a problem then the self.bus.close() is not being called properly14:40
dobeywhich means tearDown is not working properly14:40
* dobey watches top running tests in u1client trunk14:40
dobeyugh14:41
dobeyit is in constant diskwait here14:41
dobeyprobably in swap :(14:41
dobeyhrmm, it is leaking like 2M/sec14:42
dobeybut isn't totally outrageous yet14:43
dobeyonly at ~120M right now14:43
nessitadobey: hum... no14:43
nessitadobey: I added an assertion before the self.bus.close14:43
nessitaand that is failing, let me remove it14:43
dobeywell yes, that would break :)14:44
dobeyok, and there, tests in u1client blew up for me now too14:44
dobeywtfh14:45
nessitadobey: confirmed self.close releases the name14:45
dobeyok14:50
dobeyso what the heck changed that caused u1client to go nuts14:50
dobeybecause i tested my branch several times before proposing it, on sso, cp, and u1client :(14:50
dobeymandel: so, nessita is fixing cp tests right now, so let's wait and see where she gets with that, and go from there, so we can fix stuff correctly, instead of just continually trying to make stuff work by throwing stuff at a wall and seeing what sticks :)15:03
nessita+115:04
nessitaI'm close15:04
nessitarunning final check...15:10
mandelsounds gooh15:10
mandelgood :P15:10
thisfredheh. The dog just scared the bejeebus out of a Jehova's Witness when she tried to jump past me to say hello15:10
dobeythisfred: haha. the beast is here!15:12
nessitadobey, mandel: these two branches, when run together, work like  a charm: https://code.launchpad.net/~nataliabidart/ubuntuone-dev-tools/fixit/+merge/63386 and https://code.launchpad.net/~nataliabidart/ubuntuone-control-panel/testingit/+merge/6338415:19
nessitaI will now use the devtools branch within SSO suiteand see what happens15:19
nessitabut first, I will make some mate!15:19
* nessita brbs15:19
* nessita is back15:27
dobeynessita: hrmm, if you also self.patch dbus.SystemBus, do any tests break?15:27
nessitadidn't try, let me check!15:28
nessitamy money is that they don't15:28
* dobey bets desktopcouch might, but also might not15:28
nessitaglib tests do not break15:29
nessitarunning qt now15:29
nessitaall green15:29
nessitadobey: want me to push that change?15:29
dobeyplease15:30
dobeyhrmm, that branch doesn't make u1client tests work right again though15:30
dobeyso clearly it is not the removal of the inlineCallbacks that broke u1client :)15:30
dobeymandel: can you try? https://code.launchpad.net/~nataliabidart/ubuntuone-dev-tools/fixit/+merge/6338615:31
nessitadobey: I'll bet wee need to tweak something on u1client end15:33
nessitadobey: I can take a look after checking ussoc ones15:33
dobeywell we do need to tweak u1client for sure15:35
dobeyit is pretty broken regarding tests :(15:35
nessitadobey: all green in ussoc without any tweak15:36
nessitahum though I'm getting pep8 issues, I will fix those15:36
nessitadobey: storage protocol all green as well15:46
nessitaonto u1client now15:46
nessitamandel: trivial review for https://code.launchpad.net/~nataliabidart/ubuntu-sso-client/testingit/+merge/63389 please?15:47
fagannessita: I got the review15:48
dobeynessita: pep8 fix approved15:48
dobeyfagan: no need15:48
faganawh15:48
nessitathanks all!15:49
faganwell too many approvals is better than too little15:49
nessitaindeed15:49
nessitatons of errors in u1client! YEAH15:49
* nessita fixes15:49
dobeythisfred: http://pastebin.ubuntu.com/617595/ <- got this in desktopcouch tests, i suspect that should not pass, like it does :)16:01
* thisfred looks at the test in question16:02
thisfreddobey: I think this may have changed with the latest checkin. I have no idea why a meaningless statement would be mocked.16:06
dobeyno idea either16:07
dobeywell, i think the issue is that it's not moacker.expect()ed?16:08
dobeyctx is a mock object i suspect, and something is trying to access the db_dir attr on it16:08
thisfreddobey: but there is a line to that effect in the test16:09
thisfredI guess it's the way to mock attribute access if you're not interested in passing back any result16:10
thisfredit's weird though. We should as CardinalFang16:11
thisfredask16:11
thisfredguess he's en route already?16:11
thisfreddobey: so maybe this mock is not in the right place:16:12
thisfred        self._ctx.db_dir  # searching for files    # pylint: disable=W010416:12
thisfredand it was not fixed because the test does not go red16:12
thisfredand it does not go read because mocker does not fail on unexpected meaningless statements?16:13
thisfredgo red16:13
mandelnessita: on it16:14
nessitamandel: is done, no need16:14
nessitathanks though16:14
thisfreddobey: also none of those tests actually assert anything16:14
mandelouch16:14
jdobrienthisfred, so what do they do?16:15
jdobrienthisfred, def <test name>()16:15
jdobrienpass16:15
thisfredwell, since they mock a bunch of stuff, they do test assumptions about called code and calling order16:16
dobeythisfred: hrmm16:16
dobeythisfred: it should fail, mocker raised MatchError :)16:16
dobeythisfred: something is trapping the exception, it seems16:17
thisfreddobey: I agree16:17
thisfreddobey: well, it might be because of the forking, although it *looks* like that is mocked out properly16:17
dobeyah that could be16:18
dobeyyeah i don't know why it would just do foo.attr as a statement in python, that makes no sense :)16:18
dobeyis that actually inside of a statement or, a new line of one?16:19
dobeynessita: needsfixing on superfluous comments on devtools branch16:21
dobeyhrmm, i think i need to get lunch now16:21
nessitadobey: looking16:22
nessitamakes sense, fixing16:23
nessitapushed16:24
thisfreddobey: So I think it may be just mocker. This test passes: http://pastebin.ubuntu.com/617619/16:26
thisfreddobey: although I don't even get the warning there16:26
dobeythisfred: that test doesn't do self.dummy.foo though16:27
dobeythisfred: you need to poke an attr on the object, not the object itself16:28
thisfredyeah and that definitely fails16:28
thisfreddobey: I don't see that error on trunk though, so maybe it does have to do with the new devtools16:31
thisfreddobey: you have trunk of dc, right?16:32
dobeyyes16:33
dobeyi was running tests with nessita's devtools branch, to check whether the patching of dbus.SystemBus would break the dc tests or not16:33
thisfredright, so I guess it does, though I don't see how a ctx attribute and dbus should be related. Unless the dbus session is part of the test context, which it may be16:34
dobeybut even then it shouldn't mess up that expectation in mocker16:36
dobeythisfred: i got the same issue in trunk without using nessita's branch, though16:36
dobeyweird16:36
dobeyanyway, i need to get lunch16:36
dobeybbiab16:36
dobeynessita: +1 on devtools, but i'd like another set of eyes to look at it in case i missed something.16:38
dobeyok, really off now16:38
nessitadobey: of course16:38
nessitadobey: and I want u1client tests to pass first as well:-)16:38
nessitathisfred: ping16:57
thisfrednessita: pong16:59
nessitathisfred: would you know why the tests under u1clinent -> eventlog -> test_zglog.py inherit from DBusTwistedTestCase? I don't see any dbus involved17:00
thisfrednessita: I do not, that's an alecu question, but *maybe* we used to talk to zg through dbus, and then later changed that?17:03
nessitamaybe17:04
nessitaI'm removing the dbus thingy, from my POV, the current code use no dbus at all17:04
thisfreddo the tests pass if you remove the DTTC inheritance?17:04
nessitano, but the error is unrelated, and makes me realize that the test_log_does_not_err_when_daemon_not_started is wrong17:04
thisfredah ok. well I'm fine with removing it, let me know if I can review17:05
nessitaso I think that somethings are working thanks to timing issues, not thanks to the proper writting of those :-)17:05
thisfredoops. I did it again.17:10
nessitalet's dance britney!17:11
thisfredone thing I really like about compiz/unity: CTRL+ALT+numeric keypad to position windows17:14
thisfrednessita: btw, I have the system monitor in unity also17:14
jdobrienthisfred, how do you set that up?17:14
jdobrienoh...nm im using a laptop without a numpad17:15
nessitathisfred: how?17:15
thisfrednessita: sudo apt-add-repository ppa.launchpad.net/indicator-multiload/stable-daily17:15
jdobrienoh that's cool17:15
jdobrienuseless on a laptop though17:15
thisfredthen install indicator-multiload17:15
nessitaahhhh17:16
jdobrienbbiab17:16
thisfredjdobrien: ah yeah, hadn't thought of that yet. Working on my laptop I usually do everything fullscreen anyway tho17:16
thisfredthe only time I ever split the screen there is within emacs17:18
dobeythisfred: they should have named that PPA "oxymoron" instead18:09
thisfredI think that's the code name ;)18:10
nessitadobey: ping19:04
dobeynessita: hi19:04
nessitadobey: what's the rationale behinf  "len(bus_names) > 2". My question is, why 2 and not, let say, 3?19:04
nessitabehind*19:04
dobeyugh; rabbitmq19:05
nessitadobey: I'm thinking that in syncdaemon we may need more than 2 since we have at least 2 faked services being registered: SyncDaemon and NetworkManager19:05
dobeynessita: because dbus-daemon itself has one name, and our self.bus has one name, so that should be 219:05
nessitahum.....19:05
nessitadobey: what about when we register both SyncDaemon and NetworkManager? we're using 3 names total19:06
dobeynessita: even still, at that point in the code (DBusTestCase.setUp), there should only be 2, because the fake services shouldn't yet be registered19:06
nessitayou're right19:06
nessitaarghhhhh :-)19:06
* nessita keeps debugging19:07
dobeyso if they're already getting registered, there's a problem in the __mro__ :)19:07
dobeynessita: are you hitting this problem in a test case that has MI?19:07
nessitadobey: when running test_dbus alone, I get no issues at all. But when I run test_dbus with the rest of the suite (except those other dbus dependent tests), I'm getting this list_names:19:08
nessitahttp://pastebin.ubuntu.com/617722/19:08
nessitadobey: the first test that is run from test_dbus fails in the setUp (I added a pdb right before raising the InvalidSessionBus error)19:09
nessitawell, that is having len() > 3 becasue I was testing my theory, which is wrong19:09
dobeyi wonder where those are coming from then. seems like some other things are creating bus connections elsewhere?19:10
dobeywell, there are 4 there, so still more than 3 :)19:10
nessitaright19:10
nessitawith len() > 2 I get the same list19:10
nessitadbus.Array([dbus.UTF8String('org.freedesktop.DBus'), dbus.UTF8String(':1.0'), dbus.UTF8String(':1.1'), dbus.UTF8String(':1.2')], signature=dbus.Signature('s'))19:10
nessitawhich is SO ODD, since if I move the test_dbus away, all the rest (non dbus suite) pass19:11
nessitaso that means they are not leaving stuff around19:11
dobeywell19:11
dobeyit is weird, yes19:11
dobeywhat it means, is that without test_dbus, something else is connecting to the bus; but not using DBusTestCase, most likely :)19:12
dobeytests.platform.linux.test_messaging19:13
dobeyi wonder if that is one of them19:13
nessitalet's see19:14
dobeynessita: have you looked in d-feet? i presume the processes with those connections are all the same python process from u1trial?19:14
nessitadobey: I did, and you presume correctly19:14
dobeyalso, test_unity might be one of them19:14
dobeytests.platform.linux.test_unity19:14
nessitaI moved all the tests away, and I will be adding one by one to see which one is guilty19:15
nessitatest_dbus + test_unity, all green19:15
* nessita keeps going19:16
=== m_conley is now known as m_conley_away
nessitatest_session is, for now, one the guilty ones19:27
nessitafixed already, let's keep adding more test files19:27
nessitadobey: can you think why test_os_helper is messing with dbus?19:28
dobeyi don't know what it does19:29
nessitadobey: well, it does nothing weird expect importing gio19:30
nessitadoes that ring any bell? I'm not familiar with gio19:30
dobeyno, that should be fine19:30
dobeyone second19:30
dobeynessita: maybe some of the actual unity/notification/zg code is getting run, that makes a dbus connection, and it never gets cleaned up?19:33
fagandobey: was that branch you did for the new keyring spec merged?19:33
nessitadobey: from the test_os_helper, you say?19:33
dobeywhat branch?19:33
dobeynessita: yes, since it's doing file deletes/moves/etc...19:33
nessitaperharps19:33
dobeynessita: and zg is all about logging those :)19:33
fagandobey: I reviewed one a while back that fixes the gnome 3 ppa thing with the keyring19:33
dobeyfagan: i have made no such branches; and the one fix which did land does not fix all the problems19:34
nessitadobey: True. SInce that;'s the only test pending to fix, I'll debug by running/adding one test at a time19:34
dobeyand i've been dealing with more critical issues at the moment19:34
fagandobey: ah ok I was just wondering if it was fixed or not for 11.1019:35
=== m_conley_away is now known as m_conley
dobeyno it's not fixed19:36
dobeynot completely anyway19:36
fagancool19:36
dobeynessita: find it? :)20:05
nessitadobey: not yet, I'm running the whole suite to confirm that's the only missing piece to fix :-)20:05
nessitaso far so good!20:06
dobeyok20:07
dobeygreat!20:07
dobeyit's not taking 2 GB of RES is it?20:07
nessitaOH NO20:07
nessitaI was so close20:07
nessita  DBusTwistedTestCase20:07
nessita    runTest ... > /home/nessita/canonical/u1/devtools/fixit/ubuntuone/devtools/testcase.py(245)setUp()20:07
nessita-> raise InvalidSessionBus('Too many bus connections: %s (%r)' %20:07
* nessita cries20:07
nessitadobey: I'll check20:08
nessitaok, there are stuff outside platform that are doing nasty stuff with dbus20:09
nessitalet's narrow this down...20:09
nessitatests/api is empty, I'll remove that dir20:10
fagansee you all on monday20:20
faganor sunday :D20:20
nessitadobey: is the bus being created in the dbustestcase a copy of the real session bus?20:46
nessitaI would guess no, but want to confirm20:46
dobeyno20:47
dobeynessita: the whole point is to avoid talking to anything on the real session bus, and breaking things20:49
nessitaof course20:49
nessitabut when running together tests from tests/syncdaemon and tests/platform, the single first test that uses DbusTestCase fails having 25 things in list_names20:50
nessitahum, I think I *may* have a clue20:53
nessitamy god, tets/platforms is an import nightmare20:53
nessitaan AWEFUL one20:54
dobeyheh20:57
dobey:)20:58
nessitadobey: you asked before, resident memory is at 150m and growinf20:58
nessitagrowing20:58
nessita20020:59
nessita230 and seems "stable"21:00
dobeynessita: ok, sounds like stuff is still pretty badly broken; protocol from nightlies might help if you update to it21:00
nessitaack21:00
nessitafound an ungly stuff:21:02
nessita     23 class TestStatusIPC(MockerTestCase, IPCTestCase):21:02
nessitathat is surely making disasters21:02
nessitasince IPCTestCase == DBusTwistedTestCase21:02
nessitaand setUp only calls      28         super(TestStatusIPC, self).setUp()21:02
jdobriennessita, oops21:05
dobeynessita: right, but super() should call all the parent classes21:06
dobeynessita: problem is the ordering probably mucks up the __mro__21:06
nessitadobey: yeah, but it will not yield on the setUp() from DBusTwistedTestCase21:06
nessitawhich needs to be yield on21:06
dobeynessita: simply switching it to (IPCTestCase, MockerTestCase) might fix it21:07
nessitanopes, we need to yield on IPCTestCase.setUp()21:07
dobeynessita: right, so it also needs to have the yield and inlineCallbacks :)21:07
nessitayeap21:07
dobeyyou have to call super()21:07
dobeyright tcole?21:07
nessitaI'm switching to remove the multiple inheritance21:08
nessitais not needed21:08
dobeyok21:08
tcolewe need both super and chaining21:08
nessitabut thanks21:08
tcolebut if removing spurious MI can make things better that's good too21:08
nessitayeap21:09
dobeytcole: by super *and* chaining, you mean both super(Child, self).foo() and Parent1.foo(), Parent2.foo()?21:09
tcoleno, by chaining I mean chaining the deferred callbacks returned from setUp21:10
tcolevia yield or simple addCallback21:10
tcoleer, yield+inlineCallbacks21:10
tcolethere needs to be an unbroken callback chain built that encompasses the entire class hierarchy21:10
dobeyright, ok21:10
dobeyso yield super() + inlineCallbacks, as i said :)21:11
nessitaso, I will run some more tests and then I'll give up21:28
nessitaI need to do some laundry and to prepare my suitcase21:29
nessitadobey: I guess we can pair up next week and try to nail this21:29
nessitaI'm close, but not close enough21:29
dobeynessita: well i don't think you will need to make more changes to devtools to fix client, at this point?21:29
nessitanot at all21:30
nessitadevtools is correct (tm)21:30
nessitathe problem is in our tests21:30
dobeytcole: could you maybe review nessita's devtools branch?21:30
nessitatcole: https://code.launchpad.net/~nataliabidart/ubuntuone-dev-tools/fixit/+merge/63386 (thanks!)21:31
nessitadobey: in the mean time, I'm running the whole suite (until it blows up, it will be soon enough) with the new u1sp21:32
tcoleI'll have a look21:32
nessitaI will report RES usage: so far 130<21:32
nessita130m21:32
nessitaand growing21:32
nessita16021:32
nessita20021:33
nessitaoops, it jumped to 30021:33
dobeyhmm21:33
tcolenessita: +121:34
nessitatcole: thanks!21:34
* dobey has to do laundry and pack and all that jazz too, still21:36
nessitaok, I'm off21:37
nessitaI will push this branch and hope that the time make it work (?)21:37
dobeyheh21:37
dobeymeh21:46
tcoledobey: hmm, unfortunately the rabbitmqctl breaks things on my machine ... apparently there's not a one-size-fits-all fix21:47
tcoles/breaks/change breaks/21:47
dobeywhat does python -c 'import socket; print socket.hostname()' give you?21:48
dobeyand what does that resolve to?21:48
nessitaok, I'm gone (for real now)21:49
nessitabye all! see ya next week21:49
dobeycheers nessita21:50
thisfredyeah, I'm about to head out too, unless there's something I can help anyone with right now?22:04
=== m_conley is now known as m_conley_away
thisfredgoing, going, gONE22:07
dobeyi'm out too; see all you u1 hackers on sunday!22:30
jo-erlendis there some good documentation of the client protocol and the storage protocol, if they are different things? They're implemented in Python? I thought I'd have a look at it and see if I have anything to contribute.23:42

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!