[08:59] morning [09:10] morning fagan [09:10] ralsina: morning [09:12] So when do you want to walk me through the testing stuff [09:13] morning :) [09:13] morning mandel [09:17] ralsina, fagan: can you run the tests on linux for : https://code.launchpad.net/~mandel/ubuntuone-client/provide_credentials_management/+merge/62627 [09:17] mandel: sure! [09:17] mandel: sure [09:18] thx ^ 2 [09:18] and god morning! [09:21] :) [09:29] ralsina, fagan: did the tests pass? [09:29] mandel: still running [09:29] ralsina: wow, they are slow... [09:29] yep [09:31] fagan: about the testing stuff, I am probably the worst one to walk you through it ;-) [09:31] but in any case, yesterday, or the day before, nessita told you where the tests for that piece of code are. [09:32] autogen and make worked fine [09:32] fagan: and make check? [09:32] ralsina: well I know where the tests are I just want a walkthough of mocker and how to use it in relation to the code and all that [09:32] mandel: running now [09:33] fagan: read the other tests. If you don't understand something, ping me [09:33] not all of course, just one or two ;-) [09:34] mandel: make test is successful [09:34] BTW: that thing has exactly 2222 tests! [09:34] ralsina: dammed, take a look at the merge proposal, it fails on tarmac... [09:34] Errors [09:34] mandel: let me check it [09:34] fagan: got errors? can you copy paste them? [09:35] ill have to redo the tests my terminal doesnt hold logs longer than 999 lines or something [09:35] im on 11.10 so that might be one of the issues though [09:36] if the tests are passing for ralsina [09:36] mandel: I tested the branch, not the merged branch [09:36] probably you have to merge to see the failures [09:36] ralsina: can you try that? [09:36] plus, I will surely not get "exceptions.OSError: Too many open files" since I have that set waaaaay high [09:37] I am trying the merged branch now [09:37] ok give it 10 minutes and ill give the complete log [09:43] once unit testing starts taking 10 minutes ... oh, well. [09:44] I really dont understand it, mine merged with trunk works ok [09:45] mandel: the too many open files error may be tarmac doing something [09:45] yes, I suspect so… [09:45] which is a pain in the ass [09:46] too many open files [09:46] fagan: you got that too? [09:46] and there is another about defer timing out [09:46] yep [09:47] fagan: pastebin plz [09:47] ralsina: im still getting it [09:47] ralsina: takes ages [09:47] I just was looking down what was there [09:52] so, any luck? [09:52] mandel: still going [09:53] mandel: still running [09:53] PASSED (skips=1, successes=2223) [09:53] then I dont understand... [09:53] mandel: so, can't help you with that :-( [09:54] leches!! [09:56] maybe fagan can give you shell access to his box? [09:57] ralsina: the wird things is also to get this: exceptions.AttributeError: 'generator' object has no attribute 'addBoth' [09:57] that's weird, yes [09:57] that does not make sense… the too many open files ok, but this.. [09:57] could be twisted versions? [09:58] I dont think there should be any issues, also I'm running N as tarmac should be doing [09:58] and between 10 and 11 there is not such a diff [09:58] I am on N too [09:59] im running O [09:59] but I am not fully up to date! Something else to try... [09:59] so it might be a different version [10:00] mandel: trying with latest devtools etc [10:00] ralsina: I'm gonna see if I'm not up to date, but I should.. [10:00] mandel: ok [10:02] hmmmmm so from what I remember about reading and writing to files you are only allowed to open a few buffers at any given time is that the issue thats coming up [10:09] 9000 lines and counting of the log [10:09] fagan: the limit on open files on linux is like 8192 files by default [10:10] mandel: got errors now! [10:10] ralsina: me too, seems I was not up to date... [10:10] ralsina: but the buffer limit is like 8 and 2 are open by default [10:10] the too many open files one [10:10] fagan: no [10:10] ralsina: bloody hell…. I wonder if this happens in a clean branh from trunk [10:11] ralsina: if it does we are blocked... [10:11] fagan: there is no such thing as buffers separate from files, that was only true in, like DOS 4.0 [10:11] * fagan looks though his notes to find the bit that explains this [10:11] mandel: well, another thing to try, but I am betting on devtools being the guilty party [10:11] ralsina: I hope so… the ssue s from pyinotify, which is an evil beast [10:12] mandel: I only had a few updates, and devtools is the only obvious one that could hurt this [10:12] also memory usage has gone to hell so I am guessing tests are not cleaning after themselves [10:13] ralsina: yes, but I wonder what is the trial runner doing… it could be that the tests are not correctly tearing down [10:13] ralsina: hehe though the same, great minds... [10:13] dobey was workin on tardown last night, too [10:13] ok, so we are block regarding tarmac until we roll back and we push the old one to tarmac [10:13] right? [10:16] I think so [10:16] or maybe dobey looks at it and says "doh" [10:16] we can stack branches (scary) [10:18] why the hell did I accept a plane ticket that leaves at 8AM? Was I drunk? [10:21] ralsina: yeah what I was remembering was io streams [10:21] ralsina: dammed, stacking branches is dangaerous, we have already done that... [10:21] mandel: thus the scary part [10:22] ralsina: 3 are open by default not 2 as well [10:22] dobey should be here in ~4 hours [10:22] ralsina: when does dobey arrive? [10:22] fagan: what do you mean "io streams"? [10:22] ralsina: dammed you are too fast for me :) [10:22] ralsina: stdio, stderr, stdin [10:22] stdin/stdout/stderr? Those are regular file descriptors in linux (and most OSs I suppose) [10:22] ralsina: and when you open a file it creates a buffer like those [10:23] (they are just buffers) [10:23] yes, file IO is buffered. That doesn't mean you can't open 8192 files. Try it :-) [10:23] ha, the tests have killed my VM :-( [10:24] ralsina: well you can open them but you cant use them all at once maybe the OS makes that seem a little more seemless than it is [10:24] fagan: try it! [10:24] start 4000 threads writing to 4000 files [10:24] ralsina: actually that definitely will kill itself [10:24] or one thread with 8000 open files and write to them round robin :-) [10:24] ralsina: did that in class one day [10:24] fagan: t r y i t [10:25] hah [10:25] try the 8000 files one. 4000 threads need eef to run ;-) [10:25] s/eef/beef/ [10:25] ralsina: well it will work if you write them in a round robin since it will only have 1 buffer at 1 time [10:26] fagan: and how do you write to two files at once without threads? [10:26] if I did like 20 at 1 time the OS would probably switch between them [10:26] and no, you will have 8000 buffers, unless you flush them [10:27] well there would be some OS magic there id say [10:27] fagan: again. Try it. It's 10 lines. I will wait. [10:27] ill do it a bit later [10:27] * ralsina brings out the subborn-o-meter [10:27] still waiting on the tests [10:27] damn, sTubborn-o-meter [10:27] fagan, if you got the error, kill it, we already reproduced it [10:28] ah ok then [10:28] actually im getting some different ones too [10:29] like there are a few attribute errors [10:29] damn, had to hard-reset the VM :-( [10:32] ralsina: we can revert the last merge in ubuntuone-dev-tools to see if it works, I'll do that [10:32] mandel: cool [10:32] ralsina: did you get the attribute errors too? [10:33] fagan: can't tell since it died [10:34] fagan: BTW: turns out ubuntu stats with an 1024 open file limit [10:34] ralsina: interesting id still like to see how its handled kernel level [10:35] (my guess is they move around the buffers and give each one time) [10:35] fagan: you seem to have the impression that buffers are some scarce resource. They are not. [10:36] ralsina: yeah I know I just got a flash of the C class where they said you might have problems in some OSes [10:36] and the whole purpose of them is to make writing to files return faster. You can even not use them. [10:37] fagan: yes, "in some OSes" :-) [10:38] Im going to let the tests finish so I can see if there are any other errors [10:38] ralsina: well windows in different times and some versions of unix that are floating about [10:39] well probably not any one in major use [10:39] windows seems to be 2048 by default [10:39] ralsina: that the same for dos-98 [10:39] ok, 512 by default, and you can raise it to 2048 [10:40] of course if you app has more than 100 open files, you are probably doing something wrong [10:41] well it wouldnt make much sense in the desktop case [10:42] since it would slow everything down [10:42] and there wouldnt be anything that would need to be doing things all at once like that with files [10:42] but a server might have a problem if they were doing something weird [10:43] other than that I cant see a use for it [10:43] fagan: servers either use a multiprocess model (thus avoiding these limits) or ask you to raise them, or use massive caching to memory, or mmap files, or... I mean these limits have been there for 40 years, not a new problem ;-) [10:44] ralsina: true [10:45] 11300 lines in the log :D [10:46] ralsina: race? http://paste.ubuntu.com/617390/ [10:47] fagan: once you get the state of the program as broken as it is when 1000 tests are not closed, anything can happen, usually errors lose relevance after a while :-) [10:47] there are a good few other errors than just the too many files thing [10:47] ah ok [10:48] unless that error is one of the firsts, in which case, yeah [10:48] ok then I was thinking it could have been interesting to see what errors it cmes up with [10:48] *comes [10:53] just to make sure I pastebinned the output it takes a while to load though http://paste.ubuntu.com/617393/ [10:53] I killed it though [10:56] ralsina: so far with r 31 of dev tools tests pass, I'll test all of them from there to see when it brakes [10:56] mandel: cool, at least we have a clue [10:56] I wish bzr had a bisect command like git [10:57] * fagan likes [10:58] * fagan likes to say "make one" when people wish [10:59] * mandel goes for a small break while tests run [10:59] * fagan takes a nap [11:02] lunch break for me [11:14] * mandel back [11:33] ralsina: looks like the culprit is the very last revision of ubuntuone-dev-tools, version 34, we can revert to version 33 and propose for merge.. [12:00] morning all [12:06] hmmmm, http://www.dorianselimi.com/2011/06/justin-bieber-gnulinux-os/ /me thinnks i will put this on my roomates laptop [12:08] facundobatista: if you get a second today I had a question about {'is_directory': 'T', 'changed': 'LOCAL', 'has_metadata': 'T'} and what that is telling us. [12:09] duanedesign, ok [12:09] Muy buenos días a todos! [12:10] duanedesign, well, not ok, that combination has no sense, :) [12:10] duanedesign, do you have a branch of ubuntuone-client at hand? [12:12] i do [12:13] facundobatista, just fetched the latest revision [12:15] duanedesign, ok, open the ubuntuone/syncdaemon/u1fsfsm.ods file [12:16] duanedesign, that is how we programed the Sync state machine [12:16] duanedesign, the general desing of the file is that for each Event, you have an input state, an action, and an output state [12:17] that input state is the combination of: [12:17] - the node has metadata locally [12:18] - the "changed" state (NONE means not changed, LOCAL means that has local changes, SERVER means that there're server changed pending to update locally) [12:18] - is_directory means that if it's a dir or a file [12:18] and then you have some parameters according to the hash informed in the Event (if any) and errors (if any) [12:18] duanedesign, so far so good? [12:19] yes, thank you! [12:19] duanedesign, ok, now to be more specific about "changed" [12:19] the "changed" attribute is not a real one, it's a combination of: [12:20] - if local_hash (the hash we have locally for the node) is equal or different to the server_hash (the hash we locally know that the server has) === teknico is now known as teknico_away [12:20] - if the node has a "partial" component (that means that it's being downloaded from the server) [12:21] LOCAL means that it has no partial, and the hashes are different... in other words, that we're *uploading* the node [12:22] duanedesign, which has no sense at all talking about directories: we never upload directories [12:22] * duanedesign nods [12:22] duanedesign, if you check the .ods, there's no input in any line for that combination (that's why the log says "Invalid input state" [12:22] ) [12:22] duanedesign, furthermore, there's no output state for that combination [12:23] (columns L to N) [12:23] duanedesign, so, it's very very strange how the user gets to have that [12:24] facundobatista: thank you that is very helpful [12:24] otoh, we have seen cases like this already, and even more strange ones... and we also have seen a lot of cases of corrupted metadata in strange ways because of disk failing, energy cuts, etc [12:26] facundobatista: is their a way to get more info about the particular file(s) that might be corrupted? Or is the user likely to just need to 'rm' the metadata folder and have it recreated? [12:31] facundobatista: have you seen the failures from tarmac? [12:31] facundobatista: the current version of ubuntuone-dev-tools is broken… and we wont be able to land anything until the changes are reverted and installed in the tarmac machine [12:35] mandel: or wait one more hour for dobey and ask him to look at it [12:36] ok [12:38] mandel, which failures from tarmac? [12:38] duanedesign, you can always make the user to dump the metadata to a readable form [12:38] facundobatista: let me find and example [12:39] duanedesign, see 'contrib/dump_metadata.py' [12:39] duanedesign, if you want to see more info there, it's easy to add [12:39] facundobatista: last comment: https://code.launchpad.net/~mandel/ubuntuone-client/provide_credentials_management/+merge/62627 [12:39] facundobatista: all those are due to u1trial issues [12:40] * mandel walks dog [12:41] mandel, exceptions.AttributeError: 'generator' object has no attribute 'addBoth' ? [12:45] mandel, ah, ok, yes, it seems to be a ubuntuone-dev-tools problem [13:00] good morning everyone! [13:04] Hola nessita [13:05] hola facu [13:13] hola nessita [13:14] hola ralsina, que tal? [13:15] nessita: saying goodbye to people here, readying myself fr london [13:15] ralsina: when do you travel? today or tomorrow? [13:15] tomorrow 8AM [13:15] that's early :-) [13:16] yeah, I don't know what I was thinking [13:16] I have to leave for the airport at 5AM [13:21] ralsina: I have to leave at 7 for my plane even though it leaves at 11 [13:21] damn busses [13:22] I think next time im going to book the flights around 5 so it would be easier === teknico_away is now known as teknico [13:31] ralsina: you were thinking "yeah 8am sounds right, i get up early, have breakfast and say my goodbyes, but then have all the day in london" [13:31] ralsina: forgetting that 8am flight means getting up at stupid am :) [13:32] Chipaca: it probably will be "ok, I stay up all night then go to the airport like a zombie, mistakenly take a plane to Burkina Faso, end being torn apart by hyenas" [13:33] I am setting my expectations low [13:35] sounds like a very cool place to get a weird plane to [13:42] fagan: you can get there for U$S 2000 from Dublin. Only a 18 hour flight. [13:43] Then again, the older name "Upper Volta" is way cooler [13:44] wow you could charter a jet for that price [13:44] fagan: round trip [13:44] it costs about the same as coming from argentina to turkey [13:45] ralsina: wow are you paying that or are you going through some other airport [13:45] * fagan sees why mark owns his own jet [13:45] fagan: I am not going to london via argentina if that's what you are asking ;-) [13:46] ralsina: arent your family going back [13:46] (from turkey) [13:46] fagan: yeah, tickets were not cheap. About 2.7K for both of them [13:47] wow couldnt you get a connecting flight for cheaper [13:47] Im pretty lucky everywhere in europe is pretty cheap [13:53] fagan: that involves a 4-hour stop in brazil. There are not many places to connect between S.am and europe ;-) [13:54] ralsina: ah ok I forgot about timezones [13:54] fagan: basically, you have to pay for a plane that's nice enough to cross the atlantic at it's longest [13:54] ralsina: and that isnt going to be entirely full [13:56] fagan: yeah. While in europe flights you can fly cheap, small planes that can stop every 1000km. [13:57] fagan: so, it's like comparing a ferry and a transatlantic ship [13:57] ralsina: yeah I can get to the US for about 600 depending on the time of day [13:58] ralsina: and anywhere in europe for around 200 max [13:58] that's about 6000km shorter than going to argentina [13:58] and inside europe, over 10000km shorter [13:58] I've been doing the economy plus upgrades recently. (Basically you pay for an exit seat of a little more leg room) Totally worth it for me [13:59] yeah its hard to compare a trip across the world [13:59] thisfred: in british I was able to get that by being first on the online checkin [13:59] good to know, I'm flying BA as well :) [13:59] on turkish they reservethose seats until "real" checkin ;-( [13:59] im going ryan air this time :( [13:59] thisfred: you can pay 25 euros to be able to do it before the 24 hours [14:00] fagan: you are fliying 90 minutes :-) [14:00] ralsina: ryan air are the devil [14:00] :D [14:00] ralsina: 75$ for me, but they wouldn't let me, because of something with the booking (I guess because it was booked through the agent( [14:00] and im going into standsted which doesnt have a tube or train to it I dont think [14:00] thisfred: really? ouch! [14:01] ralsina: anyway, the exit seats were already all gone. I'm hoping this plane has economy plus section (the newer ones of this type do) [14:01] istanbul is only 1000lm farther away from argentina than london is. Surprising. [14:01] which will be something like 200$ to upgrade to, but still worth not having to amputate my legs on arrival [14:01] thisfred: yeah, that's much nicer. Specially you being so tall [14:02] it seems my plane is half empty so I can probably just pick when I am in the plane [14:02] I wish they would just make the seats higher, there's room there, and that would solve everyone's problems [14:02] and 4 hours is short enough [14:02] thisfred: all we short guys would get embolisms :-) [14:02] yeah, for anything under 6 hours I won't bother [14:03] maybe have foot rests [14:03] I am galled they dare call the metal loop things below the seats foot rests. [14:03] I would have to detach my feet to use them [14:03] Oh they have foot rests? :) [14:04] thisfred: not really ;-) [14:04] I'm usually with my knees by my ears as it is :) [14:04] I need wider seats, but not because of my belly, but because of my shoulders :-( [14:04] anyway, London's only 7 hours, which is survivable. B.A. was really painful at 11 hours [14:05] I end in a semi-permanent shrug :-) [14:05] I have a 16 hour return flight next week... [14:05] * ralsina considers upgrades [14:05] yeah that too: my arms are too long for the arm rests so I have to tuck them in and hunch [14:06] ralsina: the economy plus seats really are awesome for me, but I don't know that they're wider [14:06] OOPS [14:06] ralsina: upgrading on such a long flight will get you a meeeeellion miles though [14:06] me [14:06] me [14:06] the business upgrade is ridiculously expensive, so I will try economy plus [14:06] me [14:07] me [14:07] fagan? [14:07] dobey, fagan? === m_conley_away is now known as m_conley [14:07] DONE: folder subscription is now landed, started debugging DBus failures on test suite [14:07] TODO: more DBus debugging [14:07] BLOCKED: not yet [14:07] NEXT: thisfred [14:07] DONE: Bug #791927, reviews, reinstalled laptop [14:08] TODO: possibly assist with couchdb diagnostic session | u1-unity bugs [14:08] BLOCKED: no [14:08] NEXT: ralsina [14:08] Launchpad bug 791927 in desktopcouch (Ubuntu) "apport hook in source package not installed (affects: 1) (heat: 6)" [Medium,Confirmed] https://launchpad.net/bugs/791927 [14:08] DONE: pinged lots of people, organized my trip, reviews, not much else [14:08] TODO: travel [14:08] BLOCKED: no [14:08] mandel? [14:09] DONE: Look at why tarmac reject branches Ubuntuone-dev-tools rev 34 is broken, we have to revert to 33. [14:09] TODO: Land branches. Work on sdtool won windows. [14:09] BLOCKED:not really [14:09] no one next I suppose... [14:09] dobey and fagan, go! [14:10] me [14:10] λ DONE: debugging horribly broken twisted+dbus test stuff [14:10] λ TODO: more debugging, bug #789300, bug #771488, expenses [14:10] λ BLCK: None. [14:10] Launchpad bug 789300 in ubuntuone-dev-tools "DBusTestCase needs to work with Qt main loop as well (affects: 1) (heat: 6)" [Critical,In progress] https://launchpad.net/bugs/789300 [14:10] Launchpad bug 771488 in ubuntuone-dev-tools "u1trial should unset GTK_MODULES (affects: 1) (heat: 5)" [Medium,In progress] https://launchpad.net/bugs/771488 [14:11] dobey: have you seen the state of tarmac? [14:11] expenses!!! [14:11] * nessita goes to canonicaladmin [14:11] dobey: looks like the last revision of u1trial is very broken [14:11] mandel: to me that looks like an env issue (Too many open files)? [14:11] nessita: no, it's weird [14:11] nessita: if yo revert tone rev it is fixed [14:12] * ralsina has not yet passed the expenses from the BUENOS AIRES sprint. Oh well. [14:12] mandel: you get the same error in your computer? [14:12] is nothing to do with the env, it can be be reproduce if you update u1trial to the latests version [14:12] nessita: ^ [14:12] nessita: mandel, fagan and I with latest devtools [14:12] mandel: which is weird [14:12] dobey: did you land something yesterday? [14:13] dobey: tests are not tearing down and it's reaching 1024 files open. It also eats tons of ram [14:13] dobey: I do know it was the latests, I have tested with 31 up to find when as the issue [14:13] me whoops [14:13] dobey: yeah, it crashes my testing vm hehe [14:14] nessita: land something where? [14:14] dobey: devtools [14:14] * fagan was trying to get the pdf of the tickets [14:14] DONE [14:14] * Got plane ticket pdf [14:14] * tested mandel's branch [14:14] * tried to get the branch tests fixed [14:14] TODO [14:14] * Get the branch merged in and the tests finished [14:14] Blocked [14:14] * nope [14:14] nessita: no [14:14] weird [14:14] I just found out that I have to pay extra for a bag [14:15] :/ [14:15] damn ryan air [14:15] mandel: on trunk, or just on your branch? [14:16] dobey: trunk [14:17] dobey: if you go back to rev 33 it works, so there most be a bug in the latests merge [14:18] dobey: I think the issue is with the fact that you removed the @inlineCallbacks [14:18] no [14:18] but I'm not 100% sure... [14:18] it's not [14:19] the issue is that ALL OUR TESTS ARE COMPLETELY BROKEN [14:19] dobey: oh, you know already what is going on? [14:19] the sky is falling! [14:19] no i have no idea what's going on [14:21] I know I'm fixing control panel and sso tests [14:21] nessita: have you gotten anywhere with that? [14:22] dobey: yes, I'm very close to have that done [14:22] nessita: fix my tests too while your there :D [14:22] dobey: we were using the dbus session worng [14:22] dobey: I; m thinking we may wanna patch dbus. SessionBus with lambda: self.bus in devtools [14:22] * fagan is willing to donate 2 beers in london for the cause [14:23] dobey: that way we don't have to fix a crazy amount of tests that use dbus.SessionBus [14:23] dobey: what do you think? [14:23] Im doing the break it and try again approach to fixing my set of tests [14:23] nessita: hrmm, that's an interesting idea [14:24] nessita: i wonder if we should also start a system bus daemon, and do similar with SystemBus() [14:24] dobey: I don t know of any app that uses the SystemBus, but maybe? [14:25] dobey: I will propose a branch, first confirming this "fix" the u1cp test suite [14:25] nessita: well, NM is on the system bus [14:25] nessita: or maybe we should just have the one daemon, and force SystemBus to point to it as well [14:26] dobey, nessita: are this dbus issues in the tests realted with the errors we have in ubuntuone-client, or is it completely diff? [14:27] mandel: I think your issues are the consecuence of a timing issue that was exposed by some changes that added some yields to setUp and tearDown [14:28] mandel: i have no idea why it is happening [14:28] chill and give me time to actually look at it already :) [14:30] I have to go pack, and have some quality family time. So, EOD for me. If anyone needs reviews, ping me about them, I may do them tomorrow. [14:34] dobey: Oh, I'm very chill, you have my trust 100% I was just wondering :) [14:34] mandel: http://www.youtube.com/watch?v=KCujz2HzE7M [14:36] well, ok, not a great video of that scene, since the fool edited it to repeat the "just take it easy" bit [14:38] dobey: pacthing dbus.SessionBus works well! (though there is still some tiny bugs to fix, like releasing the bus name) [14:39] huh [14:40] nothing else should have to be changed for that; if it's a problem then the self.bus.close() is not being called properly [14:40] which means tearDown is not working properly [14:40] * dobey watches top running tests in u1client trunk [14:41] ugh [14:41] it is in constant diskwait here [14:41]