/srv/irclogs.ubuntu.com/2014/05/12/#ubuntu-ci-eng.txt

=== rsalveti_ is now known as rsalveti
=== nik90_ is now known as nik90
imgbot=== trainguard: IMAGE 22 building (started: 20140512 02:10) ===02:09
imgbot=== trainguard: IMAGE 22 DONE (finished: 20140512 03:25) ===03:24
imgbot=== changelog: http://people.canonical.com/~ogra/touch-image-stats/22.changes ===03:24
=== ev_ is now known as ev
* sil2100 is flashing09:03
Mirvsil2100: I can get the error even when running the single add_photo test now. but I don't seem to have big problems creating an album / adding a photo manually.09:09
Mirvsil2100: something funky though sometimes, like the animation that opens album doesn't open up fully.09:10
Mirvsil2100: I wonder if it depends on the amount of images, and maybe on whether camera app AP were run before since they take a photo?09:10
Mirvsurely there are some bugs in gallery-app, no doubt about that, but it's not probably that it was broken in #17, fixed in #18-#19 and broken again in #20-#21. I'd more guess that it has been the same but the test errors out in certain environment.09:12
sil2100hmmm09:23
sil2100Mirv: thanks! I wonder why it suddenly started being a problem again09:24
popeycihelp: can someone help me with 91.189.93.70:8080/job/generic-mediumtests-utopic/119/console ? from https://code.launchpad.net/~carlos-mazieri/ubuntu-filemanager-app/app-devel-pre1/+merge/21795809:27
popey"E: dpkg was interrupted, you must manually run 'sudo dpkg --configure -a' to correct the problem.09:27
popeyseems broken.09:27
psivaapopey: let me take a look09:27
popeythanks09:28
=== ev changed the topic of #ubuntu-ci-eng to: Ubuntu CI Engineering Team | Vanguard: ev | CI Train support - US: robru, cyphermox, rsalveti - EU: sil2100, Mirv, didrocks | CITrain support no answer: use mup bot after 30 minutes, but choose right timezone | Known issues: s-jenkins restart needed
dbarthhi09:31
dbarthsil2100: i have a new request line 32, to rebuild some peripheral OA packages, just in case09:32
dbarth+ 2 upstream bug fixes in there09:32
sil2100dbarth: ok, still waiting for the MP list and/or sources list :)09:32
Mirvdbarth: don't set the request to "Ready?" "Yes" before it's actually ready, since there'll be immediate ping on the another channel :) (I mean, currently the MP:s aren't listed yet)09:32
dbarthuh, the mp list is in my open tabs...09:37
dbarthsil2100: done09:38
sil2100dbarth: thanks! Assigned!09:40
alf_ev: Hi! Any ETA for s-jenkins restart? (I assume this will fix the stuck armhf/mako jobs?)09:45
sil2100Mirv, ogra_, psivaa: so, I got the same ofono list-modems crash on messaging-app, weather app tests still running10:03
sil2100davmor2: how's it looking so far?10:04
davmor2sil2100: well being as I've been on it for a whole 4 minutes ;)10:04
sil2100davmor2: woot, let's ship it!10:04
sil2100:D10:04
davmor2sil2100: my day starts technically as 11:00 so 5 minutes ago :)10:05
sil2100Hey hey, 11:00 was an hour and 5 minutes ago!10:05
sil2100;)10:05
* sil2100 different timezone10:06
davmor2sil2100: yeah but not in the real timezone :P10:06
sil2100pfff!10:07
popeyev: https://code.launchpad.net/~popey/ubuntu-clock-app/fix-1315318/+merge/218067 is failing - i know fginther was setting it up to test on devices rather than desktop, can you help or do I need to wait for him?10:08
davmor2sil2100: I'm just transferring my data over fresh10:08
sil2100Damn, 6 failures on weather app10:08
sil2100davmor2: could you do some additional tests for weather-app?10:08
davmor2sil2100: sure10:09
sil2100Thank you :)10:09
sil2100The weather app autopilot tests make me angrrry10:16
sil2100Mirv: ok, anyway, could you fill in a bug for this add_photo issue? :)10:16
sil2100Mirv: let's poke Bill once he's up with that10:17
davmor2sil2100: minor bug number http://davmor2.co.uk/~davmor2/screenshots-phone/device-2014-05-12-111506.png  I'm assuming that red dotty thing should be to the right of the last letter,  It starts under the magnifying lens which I think is the issue10:17
sil2100davmor2: I wonder what's up with that dotty, I saw it during messaging app tests today as well - when did they add that?10:18
davmor2sil2100: recently10:18
davmor2maybe the keyboard changes10:18
sil2100Don't think it's OSK related, don't remember seeing that change - maybe in UITK?10:19
davmor2sil2100: could be10:20
=== vrruiz_ is now known as rvr
nik90sil2100: I got an update on the https://bugs.launchpad.net/bugs/1314533. It seems recently there was a regression in the UITK which causes pagestack to behave differently. I have been coordinating with t1mp_ to fix this. The fix for the UITK regression has been merged into staging. I am just waiting on it to be merged to trunk and pushed to the phone. When10:30
nik90that happens, hopefully the clock apps tests should start passing.10:30
ubot5Ubuntu bug 1314533 in Ubuntu Clock App "Test fails on mako #3 utopic" [High,Confirmed]10:30
nik90sil2100: The long alarm save times do not seem to happen any more on the latest utopic images after the recent EDS alarm status fix by renato.10:31
nik90sil2100: https://bugs.launchpad.net/ubuntu-ui-toolkit/+bug/131673610:31
ubot5Ubuntu bug 1316736 in Ubuntu UI Toolkit "Tabs current selection changes when pushed down a PageStack" [Critical,Fix committed]10:31
sil2100nik90: excellent! Thanks for the info10:32
sil2100nik90: do you think the UITK regression could have affect on some other tests of other applications?10:33
nik90sil2100: it could affect application that depend on pagestacks..so yes10:33
sil2100hm, I guess weather app doesn't seem to be using pagestacks10:41
Mirvsil2100: so there's already a bug from you https://bugs.launchpad.net/gallery-app/+bug/1304950 , I wonder if that should be reused or a new one filed10:41
ubot5Ubuntu bug 1304950 in gallery-app "Autopilot test flakyness in test_add_photo" [Low,New]10:41
Mirvwell that claims it's fixed so I guess a new one is the answer10:41
sil2100Mirv: hm, I think re-opening might be possible, but maybe a new one could be cleaner. Is the failure looking similar?10:42
sil2100i.e. MediaSelector issues?10:42
Mirvsil2100: nope, I think this one is different even though the same test.10:43
sil2100Then a new bug please :)10:43
davmor2seb128: who deals with the interaction between online-accounts and the setting-app?  It is doing some really strange stuff of late10:48
Mirvargh, what is LP doing to my bug report..10:49
Mirvsil2100: bug #1318596 (with funky hidden comments in between when I tried to hide one and offer a fuller copy-paste in the next comment, but LP decided to for some reason hide _all_ my comments, even the new ones)10:50
ubot5bug 1318596 in gallery-app "add_photo AP test errors out" [Undecided,New] https://launchpad.net/bugs/131859610:50
sil2100uh10:53
sil2100I didn't know one could hide comments in overall10:53
Mirvmaybe LP just wanted to firmly guide me towards using attachment also for the second log10:54
=== MacSlow is now known as MacSlow|lunch
davmor2popey: can you try a quick test.  Send a message to a phone, have an audible ping, reply to the message from the indicator, Send a second message, lose the audible ping, and lose the ability to return the message from the indicator11:02
seb128davmor2, mardy11:02
davmor2seb128: thanks11:02
davmor2mardy: I'll be upsetting you shortly11:03
mardydavmor2: mmm... any way to avoid it?11:03
davmor2mardy: No11:04
mardydavmor2: OK, if it's fate, I'll accept it ;-)11:05
davmor2mardy: Is there any work on improving how the accounts app interacts with Settings app.  Currently it's a bit meh.  If you open setting, open accounts, add a new account, add a new <anything_but_ubuntuone> account, when the account setup is mostly complete the account and settings app both seem to minimise, then if you click on the accounts window it completes the account setup, if you hit back you get a black screen for 30 seconds, and the11:09
davmor2n it has minimised the setting app rather then switching back to it11:09
davmor2mardy: there are bugs for this that date back so trusty I was just wondering if there was any work scheduled for it11:11
davmor2s/so/to11:11
=== gatox is now known as gatox_brb
evpopey: sorry for the delay. Today is back to back calls. I'm not sure I follow what the problem is. If I step through the maze of Jenkins links, I end up here: http://s-jenkins.ubuntu-ci:8080/job/generic-click-autopilot-runner-mako/153/#showFailuresLink . It looks like you have real test failures.11:12
mardydavmor2: yes, this will fix it: https://code.launchpad.net/~mardy/ubuntu-system-settings-online-accounts/signon-ui/+merge/21758911:14
davmor2mardy: fantastic :) thanks11:14
popeydavmor2: hm, just sent a message, got a ping and notification, nothing showing in incoming indicator, next text just disappeared, no ping, nothing11:16
mardydavmor2: if you have time to try that branch and leave a comment, BTW, that would be really appreciated, because for reviews I have to borrow alex-abreu from webapps, and he's always busy :-)11:16
popeydavmor2: however i see both in the messaging app11:16
davmor2mardy: will do latter today11:16
davmor2popey: as do I and I can reply from the messaging app just not the indicator11:17
davmor2sil2100: ^11:17
Mirvsil2100: so, the touch input patch will (naturally) go into utopic first to qtbase, but the patch is touching code (XCB platform plugin) that is not installed on the devices11:18
popeyev: that's a vpn link which is no good for community people to debug, is there a public url that non-canonical people can get to?11:19
Mirvsil2100: oh, a correction, it's installed but not used since Mir is used instead11:19
evpopey: https://jenkins.qa.ubuntu.com/job/generic-click-autopilot-runner-mako/153/#showFailuresLink11:20
popeythanks11:21
popeynik90: ^ see these clock failures? is this the alarms taking too long to save still?11:21
* nik90 looking11:22
nik90popey: It could the UITK regression I was talking about above https://bugs.launchpad.net/ubuntu-ui-toolkit/+bug/131673611:22
ubot5Ubuntu bug 1316736 in Ubuntu UI Toolkit "Tabs current selection changes when pushed down a PageStack" [Critical,Fix committed]11:22
nik90popey: after creating an alarm, AP needs to go to the alarm tab to check the alarm count to confirm the alarm creation. However due to the regression, it goes to the clock tab after saving alarm and thus AP gets the wrong alarm count11:23
popeyah11:23
popeyt1mp_: do you know when the fix for bug 1316736 will land?11:24
ubot5bug 1316736 in Ubuntu UI Toolkit "Tabs current selection changes when pushed down a PageStack" [Critical,Fix committed] https://launchpad.net/bugs/131673611:24
t1mp_popey: we start the new landing today11:30
popeyok, thanks11:30
psivaapopey: re: 91.189.93.70:8080/job/generic-mediumtests-utopic/119/console. still working on it. awaiting access to the server.11:33
popeythanks11:33
cjwatsonogra_: Can you remind me where the custom "zip" binary (/srv/cdimage.ubuntu.com/utouch-android/zip) used in touch image builds comes from?11:37
cjwatsonOh, it's identical to /usr/bin/zip on nusakan11:37
ogra_cjwatson, we dont use, build or provide zips anymore ... just rip it out11:37
popeyhttps://imgur.com/cekokVt anyone else see that orange pip thing in the search box? davmor2 ?11:38
cjwatsonogra_: It's certainly still called as part of the build11:38
ogra_hmm, it shouldnt11:38
cjwatsonBut I can at least reduce it to /usr/bin/zip, which simplifies deployment11:38
cjwatsonIt's called from the ubuntu_data script11:39
davmor2popey: yes see #ubuntu-app-devel and the point I made earlier :)  zsombi commented on it11:39
davmor2popey: nik90 pointed it out11:40
popeyk11:40
ogra_cjwatson, yeah, but we shouldnt call it, i thought i had disabled it :/11:40
ogra_ugh11:40
cjwatsonogra_: We shouldn't call ubuntu_data, or we shouldn't call zip?11:40
ogra_http://cdimage.ubuntu.com/ubuntu-touch/daily-preinstalled/current/ looks really odd11:40
ogra_cjwatson, we shouldn't produce and promote zips11:40
cjwatsonI'll get rid of the stale trusty images from there11:41
* sil2100 in a meeting for now11:41
ogra_system-image only needs the three img files and a rootfs tarball ... people doing stuff with the images at home should use rootstock which takes the same input files11:41
ogra_so the zips are obsolete11:41
cjwatsonogra_: OK, I'm certainly happy to bin that since it will simplify deployment11:42
=== gatox_brb is now known as gatox
ogra_yeah, and it was a mess ...11:42
cjwatsonI suspect that mark-current didn't ditch the trusty images because goldfish and maguro have gone away and it didn't realise that's OK11:43
ogra_rsalveti, you dont use the zips anywhere anymore in the emulator, right ?11:43
ogra_(i'm pretty sure he doesnt, but confirmation is better)11:43
davmor2sil2100: so it looks like there are some serious issues with the incoming indicator and it looks to be getting worse not better11:44
cjwatsonogra_: Do I need to archive a copy of the last goldfish and maguro images?11:44
ogra_no11:45
cjwatsonWorks for me11:45
ogra_system-image has the latest images that are still supported ... that should be enough11:45
thostr_sil2100: why can't I cancel silo 9 build any longer?11:47
thostr_davmor2: which indicator is that?11:50
ogra_thostr_, the envelope11:51
ogra_(messaging)11:51
davmor2thostr_: indicator-messaging.  You sometime get a notification, other times you don't, sometimes you can reply to message other times you can't11:51
davmor2thostr_: sometimes you get an audible ping sometimes you don't etc etc11:52
thostr_davmor2: I see11:52
ogra_looking at that issue and at the failing alarms/apppointment notifications, i wonder if we arent still seeing some bits of the Qt5 bug that stopped rendering11:52
ogra_it feels a bit like that was only half fixed11:53
cjwatsonogra_: I can commit http://paste.ubuntu.com/7452005/ or thereabouts once rsalveti confirms11:57
ogra_yep, looks good ...11:58
alf_ev: Hi! Any ETA for s-jenkins restart? (I assume this will fix the stuck armhf/mako jobs?)11:59
evretoaded: ^ are you aware of this?12:00
=== MacSlow|lunch is now known as MacSlow
alf_ev: BTW, I have no idea if a restart is the right thing to do, just read that in the channel header and thought that perhaps it would help with the stuck armhf/mako jobs too12:01
=== josepht_ is now known as josepht
=== josepht changed the topic of #ubuntu-ci-eng to: Ubuntu CI Engineering Team | Vanguard: josepht | CI Train support - US: robru, cyphermox, rsalveti - EU: sil2100, Mirv, didrocks | CITrain support no answer: use mup bot after 30 minutes, but choose right timezone | Known issues: s-jenkins restart needed
thostr_can anybody cancel silo 9 build for me (seems my powers are not enough any longer...)12:11
alf_josepht: ^^ FYI (stuck armhf/mako jobs)12:11
josephtalf_: I'll take a look12:13
alf_josepht: thanks12:13
josephtalf_: do you have a link to a stuck job?12:14
Mirvbzoltan1: you should have landing-018, not sure why the sheet is not updating12:15
bzoltan1Mirv: thanks .. the Silo18 is not yet ready for playing12:16
alf_josepht: e.g. http://s-jenkins.ubuntu-ci:8080/job/mir-team-mir-development-branch-autolanding/652/ is stuck since Friday waiting for a mako builder to become free12:16
alf_josepht: but there are a lot of other stuck jobs (in red) if you go to the main CI page12:17
alf_josepht: s/mako builder/mako executor/12:18
=== alan_g is now known as alan_g|lunch
Wellarksilo9 is "stuck" at "Building" simply because there is an infinite dependency-wait for gcc-4.7 on ppc64el12:20
Wellarkrebuild would pull in a packaging modification which does not try to use gcc-4.7 on ppc64el12:20
thostr_can anybody cancel build of silo 9?12:21
retoadedev, josepht: just looking through the information now.12:21
sil2100davmor2: are those new regressions? I mean, was it working better in our earlier promoted images?12:23
Mirvbzoltan1: did you mean your branch is not ready or that yes for you too you don't see the spreadsheet updated?12:26
bzoltan1Mirv:  I do not see the spreadsheet updated and yes my branch is ready12:26
sil2100brb, lunch12:26
Mirvbzoltan1: running manually then, https://ci-train.ubuntu.com/job/landing-018-1-build/28/console12:27
Mirvsil2100: FYI ^ spreadsheet does not seem to update12:27
bzoltan1Mirv:  thanks12:27
didrockssil2100: FYI, I'm restoring the anonymous access that webops removed without asking us12:30
didrockson ci train12:30
Wellarkmaybe they had a valid reason for that..?12:30
didrocksWellark: just to workaround the redirection12:31
didrocksfor login users12:31
Wellarkok.12:31
didrocksWellark: but that removes the fact that a public process should be visible by everyone12:31
Wellarkdidrocks: thostr_ says he lost his ability to rebuild silo9.. any idea why that happened?12:31
didrocksev: as per mailing list12:32
didrocksWellark: not sure, I think sil2100 will look at for that one12:32
didrockshe needs to become familiar with the system :)12:32
didrocksWellark: maybe linked to the change the webops guys did…12:32
thostr_...once he's back from lunch... :(12:32
didrocksthostr_: ping him directly or mup him12:32
didrocksthostr_: not nice he ignored you12:32
thostr_didrocks: already did so...12:32
didrocksmup?12:32
thostr_no, justed pinged12:33
didrocksthostr_: sms sent12:33
thostr_didrocks: thanks!12:33
didrocksyw12:34
didrocks(anonymous reworks)12:36
didrocksev: sil2100: as per email, I restored it ^12:36
ogra_josepht, i clicked rebuild on https://code.launchpad.net/~ogra/ubuntu-ui-toolkit/improve-haptics/+merge/218966 three days ago, there is still no trace of a re-testing of the CI bot today (i just clicked rebuild again, but seems nothing changed)12:38
josephtogra_: I'll take a look12:39
Mirvthostr_: cancelled 009 build12:39
ogra_thanks12:39
ogra_:)12:39
thostr_Mirv: thanks!12:39
Mirvthostr_: but the spreadsheet seems stalled or something. so did you need a reconfigure as well, or how was it? I can do it on the backend side.12:40
WellarkMirv: just a rebuild12:40
Wellarkis needed12:40
Wellarkthe list of MP's has not changed12:40
didrocksMirv: would be good to figure out why thostr_ can't cancel anymore though, he should12:40
thostr_Mirv: didrocks: somebody stole my permissions: "strehl-t is missing the Job/Build permission"12:40
Wellarkflashback of the Nokia times..? just wait for the HR email.. ;)12:41
Mirvthostr_: :( I was happy that my rights were preserved in the reorganization12:41
Mirvthostr_: Wellark: rebuild started at https://ci-train.ubuntu.com/job/landing-009-1-build/70/console12:41
WellarkMirv: thanks!12:42
josephtogra_: we're investigating an issue with stuck jobs on s-jenkins now, this is possibly related.12:42
thostr_Mirv: so, what happened between Friday and today?12:42
ogra_josepht, ah, great ... as long as its not me i'm all fine :)12:42
Wellarkwell, at least Finland lost to Latvia and Russia on ice-hockey.. that's what happend!12:43
dbarthsil2100: not sure if it's a known problem, but i seem to have lost build privs on jenkins12:46
thostr_dbarth: welcome to the club12:46
Wellarkdbarth: thostr_ has the same problem12:46
ogra_Wellark, ice-hockey has no future anyway in a world with global warming :P12:46
dbarthdidrocks, Mirv: adding myself to the list ^^12:48
=== josepht changed the topic of #ubuntu-ci-eng to: Ubuntu CI Engineering Team | Vanguard: cihelp | CI Train support - US: robru, cyphermox, rsalveti - EU: sil2100, Mirv, didrocks | CITrain support no answer: use mup bot after 30 minutes, but choose right timezone | Known issues: s-jenkins restart needed
didrocksdbarth: first time you try to do something since Friday?12:49
thostr_dbarth: try to logout/loging couple of times12:49
dbarthyes12:49
dbarthoh12:49
thostr_dbarth: and try to close your browser in worst case12:49
thostr_dbarth: after doing that in random order it works again for me12:49
didrocksyou can go on the webops channels, they did change something since Friday12:50
dbarthindeed, that works12:50
didrocksev: FYI ^12:50
Mirvthostr_: Friday and today? ok, no idea, I only know what I read from e-mails from ev etc.12:50
Wellarkhmm.. why is jenkins always creating a clean environment for each MR when it prepares source packages for upload to silo ppa?12:53
davmor2sil2100: yes it was working more reliably previously12:53
Wellarkit should just a be simple "merge & bzr bd -S" no?12:53
didrocksWellark: you have build-deps requirement in debian/rules12:54
didrocksdebian/rules clean may need debhelper with a higher version the machine running it is on12:54
didrocksor python12:54
didrockslike dh_python12:54
didrocksor python3, go, blablabla…12:54
=== alan_g|lunch is now known as alan_g
davmor2sil2100: on a plus side weather seems fine12:58
Wellarkdidrocks: right. OK.. I haven't educated myself on which targets in rules are relevant when simply building a source package12:58
rsalvetimorning12:58
rsalvetiogra_: cjwatson: yeah, we don't use zip files anymore, safe to remove them12:59
ogra_yay12:59
cjwatsonrsalveti: Great, will do, thanks12:59
=== jhodapp_ is now known as jhodapp
=== josepht changed the topic of #ubuntu-ci-eng to: Ubuntu CI Engineering Team | Vanguard: josepht | CI Train support - US: robru, cyphermox, rsalveti - EU: sil2100, Mirv, didrocks | CITrain support no answer: use mup bot after 30 minutes, but choose right timezone | Known issues: s-jenkins restart needed
=== tedg is now known as ted
sil2100didrocks, Mirv: are the spreadsheet problems still happening?13:30
sil2100cyphermox: looking13:32
Mirvsil2100: still no updates, line 33 has a silo and line 31 (004) has finished building, from what I know13:33
sil2100Mirv: ok, looking then13:33
fgintherpopey, I'm looking into you're earlier pings13:33
popeythanks fginther13:33
sil2100didrocks: could you just check if the triggers are still set-up on your side of the spreadsheet? :)13:48
sil2100didrocks: since manual execution seems to work, but the auto-refreshing doesn't seem to work13:48
fgintherpopey, I'm now looking at the latest comment on https://code.launchpad.net/~popey/ubuntu-clock-app/fix-1315318/+merge/21806713:50
davmor2thostr_: for the lack of audible notification popey wrote https://bugs.launchpad.net/ubuntu/+source/indicator-messages/+bug/131757913:50
ubot5Ubuntu bug 1317579 in indicator-messages (Ubuntu) "No notification sound on sms since #15 mako utopic image" [High,Confirmed]13:50
fgintherpopey, does that imply that the plan is to attempt to merge this again once the sdk change lands?13:51
davmor2thostr_: I'll add bugs for the other issues13:51
popeyfginther: yeah, nik90 commented on that here earlier.13:51
popeyI think so.13:51
ogra_oh, i just got mail for https://code.launchpad.net/~ogra/ubuntu-ui-toolkit/improve-haptics/+merge/218966 which tells me it got merged ... the LP UI tells me something completely different though13:53
ogra_and there seems to be a CI bot run that negated the "Approved" it did itself 4 min before13:53
Wellarkugh.. when was the last unity8 landing?13:57
Wellarkoh, wait.. never mind13:58
nik90popey, fginther: https://code.launchpad.net/~popey/ubuntu-clock-app/fix-1315318/+merge/218067 has got nothing to do with the alarm failure. So that MP can be merged. I was referring to https://bugs.launchpad.net/bugs/1314533 earlier here. That requires the UITK fix.14:06
ubot5Ubuntu bug 1314533 in Ubuntu Clock App "Test fails on mako #3 utopic" [High,Confirmed]14:06
rsalvetisil2100: hey, want to land a change in ubuntu-touch-session and trigger a new image after that so we can get a working emulator image14:11
rsalvetisil2100: are you planning to spin a new image already?14:11
sil2100rsalveti: hi! We didn't land anything important recently, so I guess we can wait for your change and then triggering a new image, no problem14:12
rsalvetisil2100: awesome, thanks14:12
rsalvetiogra_: ubuntu-touch-session is part of silo-0214:12
rsalvetibut I'd guess this silo is not yet ready for landing14:13
ogra_rsalveti, yeah, thats mterrys nemesis landing :) goes on since montha and months14:14
rsalvetisil2100: ogra_: should we flush the silo then?14:14
ogra_talk to mterry to stack the changes on to of yours instead14:14
ogra_*top14:15
sil2100rsalveti: we can do what we usually do - we do an 'ignore conflicts' assignment14:15
ogra_or that14:15
sil2100rsalveti: since silo 002 is anyway waiting for so long, they anyway need to rebuild constantly14:15
rsalvetisil2100: right, cool then14:16
ogra_sil2100, rsalveti oops, see the ML ... someone should defintely ping mterry :)14:22
ogra_(since he just sent out a request for testing exactly that silo)14:22
rsalvetiright, but we can land this and trigger a rebuild14:23
ogra_sure14:23
rsalvetisil2100: would need a silo for 34 then :-)14:26
rsalvetiquick one14:26
sil2100Sure, uno momento14:26
rsalvetiogra_: https://code.launchpad.net/~rsalveti/ubuntu-touch-session/removing_runlevel_lightdm/+merge/21921014:26
ogra_approved on both levels14:27
rsalvetiawesome14:28
fginthernik90, thanks for clarifying. I'll double check with popey on getting this landed14:30
didrockssil2100: you didn't answer to thorst still, but it seems it has been figuring it out14:30
didrockssil2100: I can remove the triggers14:30
didrocksand let you define yours14:30
didrocksif that can help14:30
sil2100didrocks: thostr_ said some issue was 'resolved', so I didn't know hat to answer to ;)14:31
sil2100didrocks: yeah, lets migrate them to my side14:31
sil2100What triggers do you have defined right now?14:31
=== alan_g is now known as alan_g|tea
thostr_sil2100: problem was that something was broken with my login so I couldn't trigger builds any longer... anyway, works again14:37
sil2100thostr_: ok! It seems I've been in a meeting when you pinged, good thing it's resolved already14:37
thostr_sil2100: we just had some testing rounds on silo 13 and discovered some issues... so, if you need more silos right now feel free to take it for time being as I'm not sure if we still get a fix today14:37
sil2100thostr_: thanks for the info - we're a bit low, but I think nothing urgent needs a silo right now14:38
sil2100Anyway, I still have to try and fix up the auto-refresh of the spreadsheet14:38
thostr_sil2100: ok, up to you.14:38
=== alan_g|tea is now known as alan_g
cyphermoxsil2100: so is anything missing for line 18 after all?15:11
sil2100cyphermox: no no, spreadsheet issues, let me re-trigger the assignment then15:11
cyphermoxoh, alright then15:12
=== pete-woods is now known as pete-woods-lunch
sil2100Sorry for that ;)15:12
gatoxseb128, hi! i've fixed the issues in this branch: https://code.launchpad.net/~diegosarmentero/ubuntu-system-settings/click-notifications/+merge/218644 and just tesed it in the phone and it works, could you review it?15:13
seb128gatox, hey, sure15:14
seb128gatox, laney was wondering earlier about the "blank page" branch status as well, not sure if you saw15:14
gatoxseb128, yes... i'm working in that one, i can reproduce it, but i don't know what is happening yet15:15
seb128k15:15
gatoxseb128, i hope to propose a branch for that today15:15
seb128great15:15
seb128let's do a landing once that happens15:16
gatoxack15:17
sil2100popey: do you know who is responsible for the shorts-app?15:18
sil2100davmor2: so, I'm a bit 'all over the place' today, but what do you think about #22 promotion-wise?15:19
popeyyes15:22
popeywhat's up?15:22
davmor2sil2100: so the sms incoming issues I think are the only worsening item but it works fine from the messaging app.  So I think it is fine15:22
sil2100popey: just sent an e-mail regarding the recent autopilot test failures to the ML, want to know who to mention as 'person of contact'15:25
popeysil2100: well, the app hasn't changed, so is this an AP fail or a UITK fail?15:25
popeyballoons: ^15:25
sil2100popey: yes, I already mentioned that in the e-mail, just want to know who's best to mention as PoC15:26
popeyme15:26
popeyunless it's an AP fail, in which case balloons ☻15:26
popeywhere  are the failures?15:26
sil2100;)15:26
sil2100Let me show you the bug15:26
popeyci.ubuntu.com/smokeng/utopic/touch/mako/22:20140512:20140506.1/7968/sudoku_app/ is good15:26
sil2100https://bugs.launchpad.net/ubuntu-rssreader-app/+bug/131870015:26
ubot5Ubuntu bug 1318700 in Ubuntu RSS Feed Reader App "Autopilot tests for shorts_app fail since utopic image #20" [Undecided,New]15:26
balloonsdoesn't appear to be a feed issue at first glance15:28
balloonsthat has happened before and could be unexpected15:28
popeylooks like it's trying to open the toolbar and failing15:28
sil2100The only risky thing that changed in the image is UITK, so I CC'd poor bzoltan about that in the e-mail15:28
popeydo we not test apps when we land UITK updates?15:29
bzoltansil2100: ohh...15:29
bzoltanpopey: sure I do ... according to the testplan15:29
popeygreat, is shorts on that list?15:29
popey(rssreader)15:30
bzoltanpopey: Yes15:30
bzoltanpopey: https://wiki.ubuntu.com/Process/Merges/TestPlan/ui-toolkit15:31
bzoltanpopey: shoot if you want to add/remove/change anything15:31
popeygreat!15:33
popeywonder why it fails on AP tests in the lab then.15:33
sil2100Not only in the lab15:34
sil2100I ran the tests locally on my mako #22 and it failed, 3 failures15:34
balloonsI can try the older image15:35
balloonsimage #19 I guess15:35
bzoltanpopey: I was instructed to ignore failures what are marked red on the dash or fail on the stock image15:36
popeydavmor2: do you have a bug for sms not appearing in indicator incoming?15:38
davmor2popey: tedg looks like he just reassigned yours so I'm about to write it to the same process I'm assuming they are a similar root cause15:39
popeyk15:39
=== gatox is now known as gatox_lunch
balloonsthe older image seems to run rssreader fine15:44
davmor2sil2100, popey: https://bugs.launchpad.net/ubuntu/+source/telephony-service/+bug/131872415:57
ubot5Ubuntu bug 1318724 in telephony-service (Ubuntu) "indicator-message is not always showing new message notifications" [High,New]15:57
sil2100davmor2: thanks!15:57
sil2100davmor2: do you know if it's visibly worse than, let's say, the last promoted image?15:57
sil2100I'll mention it in the e-mail in overall15:58
ogra_sil2100, still in another meeting, might be late15:58
sil2100ogra_: ACK15:58
davmor2sil2100: previously you would miss out on audio pings from time to time but you always got the blue envelope and osd now that isn't happening any more15:58
sil2100huh, why did that suddenly break so badly :|15:59
davmor2sil2100: the audio pings is popeys bug which I've highlighted in this bug15:59
davmor2sil2100: it might not of been suddenly, it might just be that the audio ping missing was the first signs of a race and it was hit hard enough to show the other issues16:00
cyphermoxsil2100: got news for me? still Google issues?16:00
sil2100cyphermox: it should have been assigned, let me recheck the spreadsheet16:00
sil2100hmmm16:00
cyphermoxI checked and refreshed, etc, no joy yet ;)16:01
rsalvetisil2100: ogra_: ubuntu-touch-session is in, we can trigger a new image anytime16:01
ogra_rsalveti, i'll do right after the landing meeting16:01
rsalvetiogra_: great, thanks16:01
sil2100cyphermox: ok, so it seems the auto-refresh still doesn't work ;/16:01
balloonsoO I see the source of failures for shorts :-)16:07
balloonstime to investigate16:08
balloonsplars, is weather app a reported bug somewhere?16:09
plarsballoons: no idea16:09
plarsballoons: I thought so, but I don't have it16:09
balloonsI'd not seen or known they existed until now16:09
plarsballoons: it was mentioned in the 'Landing team 09.05.14' email for sure16:11
plarsballoons: there's not a buglink that I see though16:12
plarswhich is weird since there's one for filemanager and clock there16:12
balloonsplars, in the mail I see only clock and fm.. or I'm crazy :-)16:13
plarsballoons: it's mentioned above in the text16:14
sil2100ev: ;)16:15
ev:-P16:15
balloonsstill don't see it. blah16:16
ogra_image build triggered ...16:17
sil2100ogra_: just reminding so that it's not forgotten! #22 promotion plz ;) kthxbye16:17
ogra_promotion running16:18
sil2100Thank you!16:19
davmor2ogra_: man you should of forgot ;)16:19
ogra_hah16:20
ogra_i never forget anything ....16:20
ogra_... i just store it in a place where i never look for it :P16:20
ogra_hmm, i wonder why the promotion goes so slow today16:21
davmor2ogra_: it's because although you typed in the name of the script to run you didn't remember to hit enter after it ;)16:22
ogra_argh !16:23
ogra_sil2100, there is no 22 :P16:23
ogra_system-image claims ...16:23
sil2100huh?16:24
ogra_yeah, weird16:24
ogra_it just failed with a python traceback16:24
sil2100ogra_: how come? Even your script says there is one!16:24
ogra_yeah, tell that to s-i16:24
ogra_:P16:24
imgbot=== trainguard: IMAGE 23 building (started: 20140512 16:25) ===16:24
sil2100My device as well ;p16:24
ogra_aha16:24
sil2100Maybe we broke it after that nasty hack with #20?16:24
ogra_now it worked16:24
sil2100phew16:25
sil2100;)16:25
ogra_no, that was no hack at all16:25
ogra_the hack is that we try to keep the versions in sync actually :)16:25
ogra_by design system-image wouldnt do that16:25
ogra_anyway16:26
ogra_=== IMAGE #22 Promoted ===16:26
ogra_bah, or not ... at least not completely ... it didnt promote the x86 emulator image16:28
* ogra_ sighs16:28
ogra_ok, fixed16:29
sil2100o/16:31
sil2100Thanks :)16:31
davmor2ogra_: haha :)  I like it when a fix comes together16:34
=== fginther is now known as fginther|lunch
=== fginther|lunch is now known as fginther
=== gatox_lunch is now known as gatox
=== pete-woods-lunch is now known as pete-woods
=== alan_g is now known as alan_g|EOD
balloonspopey, new reminders is proposed in store17:07
popeyi see it17:13
* popey tests17:14
popeyballoons: dpm approved reminders17:20
dpm*\o/*17:20
robrubzoltan, silo 19 ready for your request on line 3517:21
bzoltanrobru: thanks!17:21
robrubzoltan, you're welcome!17:21
=== josepht changed the topic of #ubuntu-ci-eng to: Ubuntu CI Engineering Team | Vanguard: cihelp | CI Train support - US: robru, cyphermox, rsalveti - EU: sil2100, Mirv, didrocks | CITrain support no answer: use mup bot after 30 minutes, but choose right timezone | Known issues: s-jenkins restart needed
imgbot=== trainguard: IMAGE 23 DONE (finished: 20140512 17:35) ===17:34
imgbot=== changelog: http://people.canonical.com/~ogra/touch-image-stats/23.changes ===17:34
=== fginther changed the topic of #ubuntu-ci-eng to: Ubuntu CI Engineering Team | Vanguard: fginther | CI Train support - US: robru, cyphermox, rsalveti - EU: sil2100, Mirv, didrocks | CITrain support no answer: use mup bot after 30 minutes, but choose right timezone | Known issues: s-jenkins restart needed
=== fginther changed the topic of #ubuntu-ci-eng to: Ubuntu CI Engineering Team | Vanguard: fginther | CI Train support - US: robru, cyphermox, rsalveti - EU: sil2100, Mirv, didrocks | CITrain support no answer: use mup bot after 30 minutes, but choose right timezone | Known issues: -
=== jhodapp is now known as jhodapp|brb
Wellarkcould someone with the powerz trigger a rebuild on silo9? thanks!18:30
=== jhodapp|brb is now known as jhodapp
rsalvetiogra_: davmor2: something bad happened with latest image it seems, music and video scopes are always empty, and the other custom scopes are not showing up (weather channel, etc)18:32
davmor2rsalveti: Hey dude in your mail it say utopic-proposed should it not be devel-proposed?18:32
robruWellark, done: https://ci-train.ubuntu.com/job/landing-009-1-build/71/console18:32
rsalvetidavmor2: both should work same way, but devel-proposed is indeed better18:33
davmor2rsalveti: I see music18:33
davmor2rsalveti: and grooveshark18:33
rsalvetihm, empty here (after first flash with bootstrap)18:33
davmor2rsalveti: did you have internet connection on start up?18:34
rsalvetiyes18:34
davmor2rsalveti: if not once you have do a search18:34
davmor2rsalveti: so wikipedia has content, weather has content, amazon has content18:35
rsalvetinothing yet, let me reboot18:35
rsalvetidavmor2: did you upgrade or did a clean flash?18:35
davmor2rsalveti: clean install of 21 and an ota from that image18:36
rsalvetiright, 21 was fine here18:36
davmor2I can do a fresh install though18:36
rsalvetidavmor2: cool, that would be nice18:36
rsalvetilet me reflash again18:36
rsalvetibut the x86 emulator has the same issue18:36
davmor2rsalveti: oh hang on the latest image is 23 I'm on 22 ofcourse18:36
rsalvetialso with a clean image18:36
davmor2rsalveti: 22 was the promotion.  Let me try an update first and see what happens18:37
davmor2rsalveti: can you confirm you are indeed on 23 and not 2218:37
rsalvetiyeah, 2318:37
rsalveti22 was fine18:38
Wellarkrobru: thanks!18:38
davmor2rsalveti: yeah I forgot there was an update released right after they promoted 2218:38
robruWellark, you're welcome18:38
davmor2rsalveti: you did a bootstrap right?  ota from 22 shows content in everything18:40
rsalvetiyeah, clean install18:40
davmor2rsalveti: right give me a few minutes then18:41
rsalvetidavmor2: yeah, flashed 23 again, same issue18:48
davmor2rsalveti: personally I blame the guy who authorised all the changes for i386 ;)18:49
rsalveticannot get metdata from remote registry: unity::scopes::NotFoundException: Registry::get_metadata(): no such scope (name = com.canonical.scopes.onlinescopes)18:49
davmor2rsalveti: yeap broken18:50
davmor2rsalveti: http://davmor2.co.uk/~davmor2/screenshots-phone/device-2014-05-12-195120.png18:52
rsalvetiyeah, same here18:52
rsalveti    unity::ResourceException: Error downloading https://dash.ubuntu.com/smartscopes/v2/remote-scopes? - server replied: Internal Server Error18:54
rsalvetiwonder if the server is up still18:55
bzoltanrobru: The Silo19 is good to go18:57
robrubzoltan, alright, publishing18:57
bzoltanrobru: thanks18:57
robruyou're welcome!18:58
robruboiko, we're out of silos, so I'm freeing a stagnant one for you now19:14
boikorobru: nice! I will try to be fast on testing this one too, thanks19:15
robruboiko, thanks, will ping you once it's ready19:15
robrusoon19:15
robruboiko, alright, building! https://ci-train.ubuntu.com/job/landing-001-1-build/45/console19:21
boikorobru: thanks19:22
robruyou're welcome19:22
t1mp_we have some uitk failures on the dashboard, for example this one http://ci.ubuntu.com/smokeng/utopic/touch/mako/23:20140512.1:20140506.1/7982/ubuntuuitoolkit/1111500/20:08
t1mp_but I don't see anything in the console log what went wrong20:08
t1mp_is someone around who can help with that?20:08
t1mp_fginther: ^any ideas?20:09
=== t1mp_ is now known as t1mp
fginthert1mp, looking20:09
t1mpthanks20:10
t1mpfginther: on http://ci.ubuntu.com/smokeng/utopic/touch/mako/23:20140512.1:20140506.1/7982/ubuntuuitoolkit/ there are a bunch of failures20:10
t1mpfginther: the ones I checked seem to fail in the same way, in launch_test_application20:10
bzoltanfginther: t1mp: I have just run the AP suite for the silo18 and the UITK was all OK20:11
fgintherplars, can you provide any insight into the uitk failures in the dashboard ^ ?20:29
plarslooking20:30
fgintherplars, the uitk failures show up in the last 20 test cases (and they all failed with a similar signature)20:30
fgintherI can't find any evidence of a qml crash or OOM killer or...20:30
plarsthey all passed on flo, so it would be weird to have them on mako and not flo20:31
fgintherplars, the autopilot failure indicates that the process ID was not present when it asked for it from dbus20:33
plarsfginther: yeah, which is surprising since there was no crash20:33
plarsfginther: not much I can see from this really, I could rerun them though to see if it's reproducible20:33
fgintherplars, worth a shot, I can't think of any other logs to examine at this time20:35
robruboiko, how's silo 1 going? need a hand with testing?21:03
boikorobru: nope, testing it already, should finish soon21:03
robruboiko, cool thanks21:03
bzoltanrobru: I just finished the test of the silo1821:12
robrubzoltan, great, thanks!21:12
robruoh,needs qa signoff21:12
bzoltanrobru:  I have pasted all the AP logs to the Silo18 page21:12
robruToyKeeper, can you test silo 18 when you get a chance?21:12
bzoltanrobru: ToyKeeper: this time I made my homework better :) I have linked there all the zillions of test results21:13
robrubzoltan, thanks bzoltan! I'll publish when ToyKeeper says so21:13
ToyKeeperHmm, new branch?  (I think the previous one landed already)21:14
robruToyKeeper, yeah, looks like a big new feature if you read line 33 in the spreadsheet21:14
bzoltanToyKeeper:  here is the MR https://code.launchpad.net/~bzoltan/ubuntu-ui-toolkit/landing_120514/+merge/21917921:16
ToyKeeperYeah, looking at it now.21:16
ToyKeeperbzoltan: Thanks for including all the extra detail.  :)21:17
bzoltanToyKeeper: I try to speed it up :)21:17
plarsfginther: t1mp: I still get 20 failures when I rerun it21:27
fgintherplars, thanks. I have a freshly flashed mako, I'll give it a try there21:28
Wellarkrobru: sorry to bug you, but could you kick off another rebuild of silo9 ?21:35
WellarkI discovered a bug which is now fixed.21:35
=== jhodapp is now known as jhodapp|afk
bzoltanToyKeeper: I drop out to catch few hours sleep... t1mp might be at yourservice for few minutes still. leave me lines if there is  anything wrong with the UITK21:48
ToyKeeperbzoltan: Will do...  looks good so far though.21:49
robruWellark, sure thing: https://ci-train.ubuntu.com/job/landing-009-1-build/72/console21:55
boikorobru: sorry for the delay, but landing-001 is tested and ready to go22:11
robruboiko, no worries! publishing!22:11
fgintherplars, t1mp, I didn't see any failures running the uitk tests stand alone.22:12
asacbzoltan: did you see the toolkit failures on dashboard that plars confirmed?22:13
plarsfginther: hmm, I can run it a third time, but that's really weird22:13
asacrobru: any idea if someone knows whats going on?22:13
fgintherplars, I'm going to try again with the same list of apps that were tested by jenkins22:14
plarsfginther: on that last run, I had it just do a fresh install and *just* do the ubuntuuitoolkit test22:14
asacplars: ah you guys are investigating if its infra?22:14
fgintherplars, oh22:14
plarsasac: I don't see any way it's an infrastructure issue at this point22:15
asacplars: http://people.canonical.com/~ogra/touch-image-stats/23.changes22:15
asaclooks suspiciously small22:15
robruasac, no i'm not up to speed on the uitk issue. just waiting for toykeeper to do qa signoff on the existing uitk silo22:15
plarsasac: could be that it's just sometimes failing, sometimes not - could even be from a previous build but we were lucky enough not to hit it before22:15
asacrobru: will that fix the APs?22:15
asaci am not sure if pumping uitk in if its currently red is the best approach :)22:16
asacbut havent followed discussion22:16
robruasac, yeah, i don't see mention of it in the MP: https://code.launchpad.net/~bzoltan/ubuntu-ui-toolkit/landing_120514/+merge/21917922:16
asacrobru: guess we might want to check with ToyKeeper if she has checked the APs22:16
t1mpasac: the changes fix some AP failures in a few applications, but they wouldn't fix the failures that we see on the dashboard now22:16
asacif those are still green for her that would be interesting and might indicate that its fixed22:16
ToyKeeperWorking on it, but it takes a while.22:17
asact1mp: what are those failyres about?22:17
asact1mp: do we know yet?22:17
plarsI kicked off a 3rd run in the lab, we'll see what happens there22:17
t1mpasac: the failures on dashboard? I have no idea. I checked the logs and to me it looks like they are failing when starting a new qml app, before the test really does anything22:17
asact1mp: this seems to be NEW and reproducible on image http://people.canonical.com/~ogra/touch-image-stats/23.changes22:18
asacnot many things changed. we shouldnt leave the opportunity pass to check what really happened that caused this change of behaviour22:18
asacat least i would rather say: stop, what happened :)22:18
t1mpasac: none of those packages look familiar to me (I don't know what they do or if we depend on them)22:19
asacwhat would be next step and who should do that?22:20
robruasac, yeah, none of those appear related to uitk22:20
asacplars: thought you reproduced that locally?22:22
asact1mp: are you running the AP test suite on the uitk silo?22:22
asacif so, are those tests failing for you there too?22:22
fgintherplars, I'm trying again locally with just uitk, this time with a flash included22:24
plarsasac: no, my phone here is tied up with a different test, but as soon as I can I'm going to try it here too22:25
t1mpasac: I didn't (and its past midnight for me so I'm not starting new tests now), but zoltan did and he didn't have UITK failures22:25
asact1mp: do you know when he did the last rrun? was that on top of #23?22:25
asacplars: well, if zoltan didnt see it and if fginther didnt see it and given the very small list of changes that feels very unlikely to have an impact on this22:26
asaci guess our best guess is double checking what is up in infra for a bit first :)22:27
t1mpasac: from the timestamp of the pastebin http://pastebin.ubuntu.com/7454338/ it seems a few hours ago22:27
asacso yeah, what you are doing is probably right22:27
asacalso check if any type of change was rolled to the infra22:27
t1mpasac: I am not sure which image he used22:27
asacafter 22 image run22:27
asacphablet-test-run could echo some meta info about the image it runs on i guess :)22:28
t1mpasac: yes good idea22:28
asacunfortunately sergio isnt on :)22:28
asact1mp: filed bug 131883622:30
ubot5bug 1318836 in phablet-tools (Ubuntu) "add some verboseness about what image phablet-test-run is run on" [Undecided,New] https://launchpad.net/bugs/131883622:30
asachehe22:30
t1mpasac: :)22:31
asacplars: maybe there is ac rash?22:32
asachmm. not accoding to dashboard22:32
asaci thoght maybe woopsie upload changed behaviour on crash in a way that causes more noise22:32
asacnot: just a desparate straw to refer this back to a real change, rather than infra flakiness :P22:33
t1mpplars: you could reproduce the failures? can you run a single test, for example ubuntuuitoolkit.tests.custom_proxy_objects.test_checkbox.ToggleTestCase and see how it fails?22:35
asacplars: arent our tests getting spread to different devices? maybe the red zone was on a broken device?22:36
asacwell, might not explain why we have two greens before the last red22:36
asacassuming that those are not null ops :)22:36
asaci mean the green ones :)22:37
t1mpcould this mean something? /var/log/syslog: {{{22:38
t1mpMay 12 21:13:33 ubuntu-phablet kernel: [ 1452.086555] healthd: battery l=99 v=4374 t=27.4 h=2 st=2 c=-162 chg=u22:38
t1mpall the failed tests I checked so far have some healthd in the syslog22:39
asacmaybe heat?22:39
asacChickenCutlass: ^22:39
asacis that a throttle message?22:39
ChickenCutlassasac: healthd is very chatty22:40
ChickenCutlassasac: we need to quiet it down22:40
ChickenCutlassasac: should have nothing to do with it.22:40
ChickenCutlassget those messages on all phones22:40
asact1mp: guess red herring22:41
* asac wonders why we dont see those for the green tests22:41
ChickenCutlassyou should always see those22:41
asacdo we only extract syslog info for failed tests?22:41
asacfginther: do you kno?22:41
t1mpno uitk failures on flo22:45
* t1mp eod22:49
asacthanks!22:50
asacrobru: so not sure, but so guess if ToyKeeper cant see the AP failures either we can continue oeprating assuming that plars and CI team will fix it somehow magically. if this persists tomorrow we can still have folks look deeper22:51
robruasac, sure thing, I'll run some tests myself shortly (just flashing now)22:51
asacfginther: plars: guess you agree with that assess/recommendation?22:51
robrubfiller, got you silo 14 for keyboard stuff22:53
bfillerrobru: cool, thanks22:53
robrubfiller, you're welcome22:53
fgintherasac, agreed, the manual testing is all indicating that nothing is broken (my 2nd run passed as well)23:04
fgintherasac, and I'm pretty sure the syslog data is only saved for failed tests23:04
=== fginther changed the topic of #ubuntu-ci-eng to: Ubuntu CI Engineering Team | Vanguard: cihelp | CI Train support - US: robru, cyphermox, rsalveti - EU: sil2100, Mirv, didrocks | CITrain support no answer: use mup bot after 30 minutes, but choose right timezone | Known issues: -
* fginther steps away again, will check IRC in a couple hours23:05

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!