[06:19] <didrocks> veebers: hey, around?
[08:27] <didrocks> hey sil2100, how are you?
[08:39] <sil2100> didrocks: hey! Still looking at two failure types that popped up, but I'm fixing up the problems thomi mentioned in my autopilot additions yesterday
[08:40] <didrocks> sil2100: did you look at the tests results from yesterday?
[08:40] <didrocks> sil2100: seems it's repeatable
[08:40] <didrocks> sil2100: ps-unity-autopilot-release-testing, intel in particular
[08:40] <didrocks> sil2100: 40 failures, that's what prevent us from releasing today
[08:41] <didrocks> sil2100: I wonder if it's not linked to thomi's change few days ago with autopilot dbus communication
[08:42] <sil2100> didrocks: yes, I looked into the preview failures, and those seem to be problems with introspection, so maybe indeed it's related - I'll take care of it now then and take care of the branch fixes later
[08:42] <didrocks> sil2100: thanks, it seems more important to me as well :-)
[08:42] <didrocks> sil2100: once you got something, I can rerun a build + tests, thanks!
[08:42] <sil2100> didrocks: what version of autopilot was used during the build?
[08:43] <didrocks> sil2100: trunk version, as the misc task published, you can check that in:
[08:45] <didrocks> sil2100: job/ps-unity-autopilot-release-testing/label=autopilot-intel/85/artifact/results/artifacts/machine-config/dpkg-list.log
[08:45] <didrocks> like https://jenkins.qa.ubuntu.com/job/ps-unity-autopilot-release-testing/label=autopilot-intel/85/artifact/results/artifacts/machine-config/dpkg-list.log
[08:45] <sil2100> Thanks!
[08:45] <didrocks> yw :)
[08:46] <didrocks> sil2100: but yeah, we are taking latest trunk for autopiloting unity
[08:46] <didrocks> (basically everything which built in the ppa)
[08:52] <seb128> hum
[08:52] <seb128> didrocks, cyphermox: did you see bug #1124941 ?
[08:53] <didrocks> seb128: yeah, I didn't update yet, I'm wondering why tests passed though yesterday
[08:54] <didrocks> seb128: seems like, as I was telling, coverage of integration tests for indicators is low
[08:54] <seb128> didrocks, I'm updating, let's see
[08:56] <seb128> didrocks, seems to work for me
[08:56] <didrocks> seb128: it seems to be only the non default indicators
[08:56] <didrocks> like the ones in vala?
[08:57] <seb128> well, that title seems to indicate a g-i issue
[08:57] <didrocks> oh python
[08:57] <didrocks> I meant
[08:57] <seb128> ok, I can confirm with onboard
[08:58] <seb128> $ onboard
[08:58] <didrocks> (still updating)
[08:58] <seb128> 2013-02-14 09:58:06,280:ERROR:root: Could not find any typelib for AppIndicator3
[08:58] <didrocks> is the typelib shipped?
[08:58] <seb128> /usr/lib/i386-linux-gnu/girepository-1.0/AppIndicator3-0.1.typelib
[08:58] <seb128> it has been multiarched
[08:58] <seb128> but gir doesn't support multiarch
[08:58] <didrocks> indeed
[08:59] <didrocks> seb128: well, as it doesn't impact our default, I would rather wait for cyphermox ^
[08:59] <didrocks> seb128: the python indicator is crashing from start, right?
[08:59] <seb128> didrocks, it fails to import appindicator so it's not working
[09:00] <seb128> didrocks, well "doesn't impact our default", I can't get onboard on screen, not sure how that will play out for the nexus
[09:00] <didrocks> yeah, so an autopilot tests with a dummy python indicator process should be easy to do
[09:00] <seb128> no onscreen keyboard on a touch device makes the device less useful ;-)
[09:00] <seb128> yeah
[09:00] <didrocks> seb128: indeed, but if we fix it right away without any tests, we won't have it, I can bet on it
[09:00] <seb128> can we revert?
[09:01] <didrocks> seb128: that's an option
[09:01] <seb128> cyphermox is probably after eod at this time
[09:01] <didrocks> right, he's back in ~3 or 4 hours generally
[09:01] <seb128> so we will need to wait at least some 5-6 hours to see that being worked on
[09:01] <seb128> ok
[09:01] <seb128> well, let's see how much people are impacted by it
[09:02] <didrocks> seb128: well, I think we won't have any other daily isos for the nexus before that TBH
[09:02] <seb128> right, it's just people dist-upgrading their nexus every day
[09:02] <seb128> let's see how long it takes for ogra to come ping ;-)
[09:02] <didrocks> indeed, we'll get it fixed soon anyway
[09:02] <didrocks> heh :-)
[09:05] <didrocks> seb128: even better, this can be a dummy test during package build
[09:05] <didrocks> seb128: just trying to import the lib
[09:05] <didrocks> hum, no, it needs to be installed
[09:08] <seb128> didrocks, ok, onboard seems to work, it's just the indicator which is broken
[09:08] <seb128> didrocks, I didn't try ubiquity but that recommends the gir as well so it might have broken the unity panel
[09:08] <didrocks> seb128: well, for this, we need a new ISO in that case anyway
[09:09] <seb128> didrocks, in any case seems like both are non stoppers for today, we should get it fixed for tomorrow's iso though ... let's see if cyphermox can add the test
[09:09] <seb128> otherwise let's revert at eod
[09:09] <seb128> works?
[09:09] <didrocks> seb128: yeah, should be simple, will ping him as soon as he's available :)
[09:09] <seb128> cool
[09:09] <didrocks> seb128: sounds like a good plan :)
[09:24] <xnox> seb128: /me had reports that ubuntu daily iso boots straight into "try ubuntu" instead of ubiquity.
[09:24] <seb128> xnox, that seems a different issue
[09:24] <xnox> the error message is cryptic "failed to activate consolekit". Fair enough.
[09:25] <didrocks> yeah, that appindicator introspection has anything to do with this :)
[09:25] <xnox> i was diffing the manifests and didn't find any obvious suspects.
[09:27] <mmrazik> smspillaz: btw. it looks like setting the tests to False didn't help:
[09:27] <mmrazik> https://jenkins.qa.ubuntu.com/job/compiz-clang-ci/75/build=pbuilder,distribution=raring,flavor=amd64/console
[09:28] <smspillaz> mmrazik: hmm, let me have a look
[09:28] <smspillaz> mmrazik: thanks for the quick response
[09:28] <mmrazik> the strange thing is that I've seen this thread stuff yesterday on pbuilder while I was trying something unrelated (I tend to use lp:compiz for experiments)
[09:28] <smspillaz> ah okay, don't owrry about that job
[09:28] <smspillaz> CI was already blocked on something else
[09:28] <mmrazik> ok
[09:29] <smspillaz> I unblocked that in a new MP, and then the clang bug was the problem :p
[09:29] <mmrazik> then lets see if it happens somewhere else too
[09:29] <smspillaz> hang on, I'll get you the relevant job
[09:29] <smspillaz> mmrazik: https://code.launchpad.net/~compiz-team/compiz/compiz.fix_1124313/+merge/148193
[09:29] <mmrazik> smspillaz: ok. I'll start that one too
[09:30] <smspillaz> great, thanks
[09:30] <smspillaz> mmrazik: btw, do you know anyone at canonical who might be a good llvm contact?
[09:30] <mmrazik> but it will probably take a while. There is one more -ci job in the queue
[09:30] <mmrazik> no idea :-/
[09:30] <smspillaz> we should probably report this bug to the clang people
[09:30] <smspillaz> okay, thats fine
[09:32] <smspillaz> mmrazik: btw, last time I checked all of the xorg-gtest tests are now passing in CI, bar one, which was failing because of a distro patch \o/
[09:32] <mmrazik> nice
[09:49] <sil2100> Strange things, suddenly it's as if sometimes things are not pushed for introspection
[09:50] <didrocks> sil2100: do you think the autopilot change can be the issue?
[09:58] <sil2100> didrocks: hard to say, but it doesn't seem like it though, but still looking
[09:58] <sil2100> didrocks: didn't see any direct dependency between the code changes and the case we have there
[09:59] <didrocks> interesting
[09:59] <didrocks> sil2100: still seems a timing issue for some, look at ati and nvidia, they are in better shape
[10:00] <sil2100> Problematic thing is, it's not that all the preview-related tests are broken, since only some of them fail
[10:00] <didrocks> but "reliably", intel is having more errors
[10:00] <didrocks> so I would more think of a timing issue
[10:00] <sil2100> And when they fail, well, it seems that /Unity/DashController/DashView/PreviewContainer/PreviewContent[id=1208]/* returns no children ;/
[10:01] <sil2100> Originally I thought yes, that it *might* be a timing issue - but when I found this debug output, autopilot tries introspecting with this query 10 times over 10 seconds, so a timing issue is rather improbable
[10:03] <didrocks> weird…
[10:05] <sil2100> It makes no sense, since there's no way of running away from introspection, every preview gets AddChild'ed whether it likes it or not
[10:06] <sil2100> But maybe hm
[10:06] <sil2100> But maybe somehow there are two PreviewContainers opened by mistake, and during the failure, the 'inactive' one is being fetched?
[10:07] <sil2100> Not sure if that's even possible
[10:08] <didrocks> sil2100: possible that it's a real regression, yeah
[10:20] <sil2100> didrocks: indeed it might be a really rare regression, since I noticed another strange thing that might fit to my strange theory
[10:21] <didrocks> sil2100: well, not "rare" in the sense it's not happening regularly on a machine
[10:21] <sil2100> All preview tests work up until a point, after which it suddenly stops working every time
[10:21] <sil2100> Now the theory:
[10:21] <didrocks> sil2100: if you look at yesterday's run and the one before, we still have that on intel. Maybe the machine is either slower or faster than others :/
[10:21] <didrocks> yeah?
[10:23] <sil2100> hm, to have my theory fully thought out, I need to see how does AP know what id to query for, one moment
[10:26] <sil2100> Ok, so, this theory might be flawed as I still don't have enough unity introspection knowledge ;p But let's say up until a point, we're using a single PreviewContent introspection object for all previews
[10:27] <sil2100> Then, suddenly, after a point (probably because of a test that has been executed at a time), unity is forced to kill the previous PreviewContainer and create a new one (maybe when looking at previews in a different lens?)
[10:28] <sil2100> The new one is now used by Unity, while the old one, well, it's empty
[10:28] <sil2100> But autopilot is still using the old one, since that's the one he allocated at the beginning - that's why we get those query errors for the same id all the time
[10:29] <sil2100> This could make sense even, since I see that before breaking the preview tests, there is a 'files lens' preview test being executed that failed: test_files_lens_preview_open_close
[10:29] <didrocks> sil2100: that's a valid theority, now, we need to map that to a change in either autopilot or unity :)
[10:29] <didrocks> sil2100: last time the tests passed was rev 3140 in lp:unity
[10:29] <sil2100> didrocks: looking at intel and ati, comparing those two in regards to test_files_lens_preview_open_close, which looks strange to me (dee errors etc)
[10:29] <sil2100> ACK!
[10:30] <didrocks> yeah ;)
[10:30] <sil2100> didrocks: and it seems that test_files_lens_preview_open_close did not fail on ati, so it might be indeed the root cause of the failures - something is getting broken when that test fails
[10:30] <didrocks> sil2100: the corresponding autopilot version was 130 FYI
[10:30] <didrocks> sil2100: yeah, more than possible :)
[10:49] <sil2100> didrocks: anyway, now I see that my previous theory might have been more correct - but enough theoritizing, I'm looking now all the time if I'm right and how to fix that
[10:50] <didrocks> sil2100: thanks a bunch, good hunt! and keep me posted :)
[11:36] <sil2100> hoh, I think I found something
[11:37] <sil2100> Maybe
[11:47] <smspillaz> mmrazik: it looks like clang is just completely screwed in the distro
[11:47] <mmrazik> smspillaz:  so its not just me with that impression...
[11:48] <smspillaz> mmrazik: it fails on compiling gtk-window-decorator too
[11:48] <smspillaz> mmrazik: we should probably just disable the job for now :/
[11:48] <mmrazik> smspillaz: ok
[11:48] <smspillaz> hopefully someone will notice
[11:48] <mmrazik> smspillaz: done
[11:48] <smspillaz> mmrazik: awesome, thanks
[12:05] <sil2100> !
[12:12] <didrocks> sil2100: "!" as found anything? ;)
[12:15] <smspillaz> sil2100: https://code.launchpad.net/~compiz-team/compiz/compiz.fix_1124313/+merge/148433 hopefully CI should go green soon
[12:15] <smspillaz> clang is being a pain
[13:11] <smspillaz> man this is the weirdest thing
[13:11] <smspillaz> if you close a child process' stdout, it hangs the next time it calls XSync
[14:05] <sil2100> HAH! Reproduced the issue FINALLY
[14:06] <sil2100> Damn, but still, this is one nasty son of a ***
[14:08] <sil2100> I'm so close ;)
[14:14] <sil2100> smspillaz: still waiting on CI on that merge request...
[14:18] <smspillaz> sil2100: yeah, it seems to be backed up
[14:18] <smspillaz> sil2100: did you find the intel crash ?
[14:26] <mterry> sil2100, a test failed with http://10.97.0.1:8080/job/ps-unity-autopilot-release-testing/label=autopilot-intel/lastCompletedBuild/testReport/unity.tests.test_panel/PanelTitleTests/test_panel_title_doesnt_change_with_switcher_Single_Monitor_/
[14:26] <mterry> sil2100, which looks like a buggy test: no hud object
[14:27] <sil2100> mterry: will look into that one in a moment
[14:27] <mterry> sil2100, no rush  :)
[14:27] <sil2100> Yes, I think it's a malformed test
[14:27] <sil2100> mterry: I'm on the verge of fixing the preview test failures ;p
[14:32] <mterry> sil2100, nice  :)
[14:37] <cyphermox> monring
[14:37] <cyphermox> ugh
[14:38] <didrocks> hey cyphermox, mterry
[14:38] <didrocks> cyphermox: you can still publish the indicator stack btw :)
[14:38] <didrocks> cyphermox: and ping so that libcolumbus is NEWed
[14:38] <cyphermox> didrocks: did you pull on AA ?
[14:38] <cyphermox> ack!
[14:38] <didrocks> cyphermox: yep, all done :)
[14:38] <cyphermox> was libappindicator fixed too?
[14:38] <didrocks> cyphermox: yeah, you need to ack the MP
[14:39] <cyphermox> ok
[14:39] <didrocks> cyphermox: and the unity branch with the additional test should be merged now :)
[14:39] <mterry> hi
[14:40] <cyphermox> didrocks: oh, but wait this is going to take a bit more to land the fix as well..
[14:41] <didrocks> cyphermox: hum, what do you mean?
[14:41] <didrocks> cyphermox: I would say, let's manually publish indicator if you are happy with the packaging change
[14:41] <didrocks> and ack the MP (looks fine to me)
[14:41] <didrocks> then, once landed, rebuilding on libappindicator
[14:42] <cyphermox> well, libappindicator-autolanding needs to run, and then everything needs to be rebuilt
[14:42] <cyphermox> ah, sure, that way works too ;)
[14:42] <didrocks> cyphermox: only rebuild what you need :)
[14:42] <cyphermox> err, yeah, but I meant we could wait for the libappindicator fix to land before publishing
[14:43] <didrocks> cyphermox: well, I would say, let's publish first, it's taking 30s :)
[14:44] <didrocks> cyphermox: we'll probably have another manual publishing for libappindicator as it's a packaging change
[14:45] <cyphermox> ok, I see
[14:45] <cyphermox> there, libcolumbus will publish
[14:45] <cyphermox> ugh
[14:45] <cyphermox> not sure you'll see me next week -- I'm coughing up my lungs
[14:51] <cyphermox> didrocks: how come I don't see it in the queue? :)
[14:52] <didrocks> cyphermox: which queue?
[14:52] <cyphermox> libcolumbus in unapproved?
[14:52] <didrocks> cyphermox: remember that we sync the list of copy on the server?
[14:53] <cyphermox> yeah
[14:53] <cyphermox> oh, you still need to do a manual job?
[14:53] <cyphermox> or is this waiting on a cron task?
[14:53] <didrocks> cyphermox: no, it's a cron, running every 15 minutes
[14:53] <cyphermox> ok
[14:54] <didrocks> cyphermox: see https://wiki.ubuntu.com/DailyRelease/StackPublish#Copy_to_distro ;)
[14:54] <didrocks> "Then, on the archive admin machines, we have a cron using this copy script which:
[14:54] <didrocks> and so on…
[14:54] <mterry> cyphermox, you need your lungs!  don't cough them up...  (feel better!)
[14:55] <didrocks> I heard lungs are handy sometimes :)
[14:56] <cyphermox> yeah, kind of useful
[14:56] <cyphermox> excuse me while I medicate
[15:10]  * didrocks notes to be 10000 feets away from cyphermox next week :)
[15:15] <cyphermox> hehee
[15:19] <cyphermox> let's finish up with libappindicator and then I'll go lie down for a while, and I'll be much better by Saturday
[15:23] <didrocks> cyphermox: get better! :-)
[15:24] <cyphermox> did you new libcolumbus?
[15:27] <didrocks> cyphermox: I will wait for seb128 to do it, I've done the packaging, it would be unfair :)
[15:27] <didrocks> cyphermox: did you notice the misc stack needs manual publication as well, btw?
[15:28] <cyphermox> yeah, getting to it
[15:28] <didrocks> grand!
[15:28] <didrocks> now that libappindicator is merged, you can build it :)
[15:31] <cyphermox> don't we need -daily to run first, to you know, build and snapshot to the ppa?
[15:31] <cyphermox> otherwise it won't have the new "automatic snapshot" line to match with for prepare-packages?
[15:31] <cyphermox> oh wait, I suck
[15:31] <cyphermox> nevermind me
[15:32]  * cyphermox publishes misc
[15:32] <didrocks> \o/
[15:32] <cyphermox> ./cu2d-run -R indicators libappindicator
[16:03] <andyrock> popey, ping
[16:05] <popey> andyrock: pong
[16:06] <andyrock> popey, is there a lp bug for the focus issue?
[16:06] <popey> pass, om26er found it initially but dunno if he filed one
[16:07] <popey> happy to file it if you need it
[16:07] <andyrock> k
[16:07] <om26er> popey, andyrock I didn't file the report yet.
[16:08] <om26er> popey, if you are doing i'll confirm, else I can report it now
[16:10] <popey> om26er: go for it
[16:16] <om26er> andyrock, popey bug 1125331
[16:16] <andyrock> om26er, thx
[16:16] <om26er> yw
[16:19] <om26er> andyrock, in raring the separator between the dash and the launcher is missing most of the times, that's a regression from 12.10
[16:19] <om26er> the white line
[16:19] <andyrock> om26er, ask dednick_ ;)
[16:19] <andyrock> om26er, i fixed that bug 2-3 times in the last cycle
[16:20] <om26er> ouch
[16:20] <andyrock> om26er, i'm sure he can fix it in a couple of mins
[16:20] <andyrock> ;)
[16:21] <om26er> andyrock, I'll log a bug in launchpad then
[16:23] <om26er> the fullscreen preview is deadly slow. Jay was looking into that when he was still working
[16:39] <om26er> reported as bug 1125346
[17:04] <didrocks> sil2100: so, in your hunt, did you get lucky, unlucky? Should we all take our pay and we'll be never able to release unity again? ;)
[17:18] <didrocks> anchorchangesnodeinstance.cpp
[17:18] <didrocks> ooops, focus issue
[17:18] <cyphermox> didrocks: so, tests look fine I think
[17:18] <cyphermox> not perfect, but we're not going to reach perfection
[17:19]  * didrocks looks
[17:19] <cyphermox> just waiting for armhf to finish building nao
[17:19] <didrocks> cyphermox: yeah, we would need again a sil2100 to get the tests fixed :)
[17:19] <didrocks> cyphermox: but otherwise: agreed
[17:19] <didrocks> once armhf is finished building, you can force publishing
[17:20] <cyphermox> yeah
[17:47] <sil2100> didrocks: I analysed analysed and finally found the reason behind all those preview failures - it's a rarely reproducible issue, but when it happens, well, it breaks everything
[17:47] <sil2100> didrocks: dednick helped me find a probable fix
[17:47] <sil2100> I'm testing it now, but the testing process is, well, hard to do ;)
[17:48] <didrocks> sil2100: you ROCK! :-)
[17:48] <didrocks> sil2100: if you want, we can still get it in and try a manual rebuild (or wait for next daily)
[17:48] <didrocks> as it seems intel likes triggering it :p
[17:58] <sil2100> didrocks: maybe let's get it properly tested and wait for the tomorrow's daily
[17:58] <sil2100> didrocks: you can fire up the tests if you want, since it's probable that the bug won't even happen for the second time
[17:59] <sil2100> brb, valentines
[17:59] <didrocks> sil2100: hum, it did happen for 2 times, as I told you more than once :p
[17:59] <didrocks> sil2100: it happened yesterday and today on intel ;)
[18:00] <didrocks> sil2100: to be able to test, we need to merge it though