[06:08] hey Mirv, how are you? [06:10] didrocks: hey, fine. I've been fixing 10+ prepare jobs this morning for head/saucy, but there's still work to do [06:10] Mirv: sil2100 didn't finish it and didn't email you? [06:11] didrocks: I don't think he worked on those, I saw the same prepare jobs yellow as yesterday [06:11] (no e-mail) [06:11] * didrocks wonders why fixing prepare jobs is taking so long. [06:11] Mirv: urgh :/ [06:11] ok, I'll talk with him [06:11] I'm looking at why the jobs FTBFS [06:12] didrocks: oh, I started using the Stack status for trusty and filed first FTBFS bugs https://docs.google.com/a/canonical.com/spreadsheet/ccc?key=0AuDk72Lpx8U5dHFtUmlPOUtCRk8zR2dtaEpIbUVhMmc#gid=3 [06:12] Mirv: ah, excellent! looking :) [06:16] * didrocks adds https://bugs.launchpad.net/ubuntu/+source/mediascanner/+bug/1243536 [06:16] Ubuntu bug 1243536 in mediascanner (Ubuntu) "mediascanner FTBFS in trusty" [Undecided,New] [06:19] and https://bugs.launchpad.net/autopilot-gtk/+bug/1243538 [06:19] Ubuntu bug 1243538 in autopilot-gtk "autopilot-gtk FTBFS on trusty" [Undecided,New] [06:21] jibel: hey! [06:26] didrocks: the prepare jobs are now fast to fix (I'm now at ~15/20), there was just so much to do before that. on Monday I started with the thought there's just small bits needing to be done, while actually most of the transition was not done at all. [06:26] Mirv: urgh, ok… I hope we'll get better at next release opening [06:27] Mirv: there are some in saucy as well it seems [06:27] Mirv: I'm pocking people to get their FTBFS fixed [06:28] didrocks: yeah.. as I was away for a few days, and you mentioned something along the lines that it's about done but sil missed a few, I was under wrong impression. during Mon & Tue then the actual transition was done with me + sil + fginther. [06:28] didrocks: yeah, I've fixed around 8/14 of the saucy prepare job problems now [06:29] Mirv: ok, good luck :) [06:29] I'm more wondering why sil2100 didn't email/continue yesterday [06:29] because the FTBFS were not handled [06:29] nor the prepare jobs [06:29] nor the -check issue (no access to archive.ubuntu.com) [06:31] vila: did you get any email? this is blocking us [06:31] vila: from magners-orchestra, I can't ping archive.ubuntu.com [06:31] unfortunately, as sil2100 didn't share any info (I guess he pinged retoaded privately), we don't know where we stand [06:31] asac: FYI ^ [07:01] didrocks: can't ping from m-o, filtered by the IS firewall, I use wget for that kind of tests, and wget archive.ubuntu.com works [07:01] didrocks: so, what's the issue ? [07:02] vila: http://10.97.0.1:8080/job/autopilot-trusty-daily_release/9/label=autopilot-nvidia/console for instance [07:03] /var/log/upstart/otto-setup.log: Could not resolve 'archive.ubuntu.com' [07:03] vila: seems to work from outside the container [07:03] that doesn't sound like a firewall problem - you wouldn't be getting a DNS failure in that case [07:03] (using wget) [07:03] Hi [07:03] more likely busted resolv.conf or something [07:03] didrocks: retoaded had this to say (I though he was fixing that): [07:04] sil2100, looking through the console output in the job then looking at dx-autopilot-nvidia it appears that the LXC network bridge (lxcbr0) doesn't have an interface assigned so it doesn't have a network path outside if itself. [07:04] nor are there any iptables rules in place to route the network traffic from lcxbr0 to eth0 [07:05] This was close to my EOD [07:05] sil2100: hum, weird, something changes on the machine, we got a successful snapshot (and it was the raring machines) [07:06] hmm [07:06] jibel: do you remember we are doing any bridge manual setup on the hosts for otto? ^ [07:06] sil2100: btw, Mirv is finishing the prepare jobs, (apparently he didn't find any branches) [07:07] didrocks: urhg :-( [07:07] didrocks, we don't do anything the bridge is handled by lxc in the upstart job lxc.conf [07:07] * didrocks continues on libaccounts-glib FTBFS [07:07] maybe a regression from latest lxc? [07:07] vila: are you on it? ^ [07:08] didrocks: not for now, I'm on upstream-merger for trusty but I can switch if needed [07:08] lxc-net.conf actually [07:09] didrocks: which also involve otto and a container so I may run into the same issue... [07:09] vila: ok, so upstream-merger is blocked as well? [07:09] didrocks: for trusty yes (unless fginther did something I'm not aware of) [07:10] sil2100: I noticed that you had two branches, but there were 17 prepare job problems all in all - now fixed [07:10] Mirv: all fixed? \o/ (saucy as well?) [07:10] didrocks: as for cu2d on phones, I've seen MPs from doanac but no firm confirmation is was ready to be plugged... [07:11] vila: ok, keep us posted once he's online. I guess the priority is upstream-merger/otto, while we are working with upstream to fix FTBFS [07:11] didrocks: /me nods [07:11] didrocks: yep, saucy as well [07:12] Mirv: thanks a lot! seems it wasn't that long after all ;) not sure why sil2100 wasn't able to finish it [07:12] (the accounts-glib FTBFS is really puzzling) [07:12] didrocks: still, for my sanity/understanding, http://10.97.0.1:8080/job/autopilot-trusty-daily_release is a step (or several) further than the setup we did with you jibel for otto ? [07:13] vila: it's the otto job, the one running the tests [07:13] but it can't get to archive.ubuntu.com for any reason inside the container [07:13] didrocks: I thought we were able to create a container which includes an apt-get update right ? [07:13] so the tests are failing [07:13] vila: right, on Monday [07:13] didrocks: so that's a regression right ? [07:13] vila: sounds like it [07:13] didrocks: ok, thanks [07:14] * vila restores a tiny bit of sanity [07:14] didrocks: could it be something similar to the [[:space:]] issue encountered on the phone yesterday ? [07:16] hmm, was the dashboard CSS updated or sis my browser just decide to use a completely different font on its own ? [07:17] vila: maybe [07:17] s/sis/did/ [07:19] system-image-cli -c trusty-proposed -v [07:19] ... [07:19] [systemimage] Oct 23 09:19:13 2013 (22849) Already up-to-date [07:19] HMPF [07:20] root@ubuntu-phablet:/# system-image-cli -i|grep channel [07:20] channel: saucy-proposed [07:20] that doesnt look right [07:21] ogra_: there are multiple files for the channels [07:22] ogra_: at least 2 [07:22] so I won't be surprised there is a bug somewhere [07:22] didrocks, well, the above should switch me to trusty-proposed [07:22] according to the docs [07:22] ogra_: right, maybe it's just updating the wrong file :) [07:22] and i know it worked for pat mcgowan yesterday [07:22] ah, interesting [07:22] RO image? [07:22] oh, wait [07:22] i tested something ... [07:23] * ogra_ checks for writable_image [07:23] right, its rw [07:23] * ogra_ removes and reboots [07:23] ok, was worth looking… [07:24] doesnt change even in ro [07:25] Mirv: there were more branches, I pasted like 8 branches to cyphermox before going EOD [07:27] * ogra_ phablet-flash'es .... hoping his data wont be wiped [07:27] didrocks: I pasted all the branches that needed merging to cyphermox, and I see 2 of those didn't merge in because of merger failures [07:28] sil2100: branches that you handled? [07:28] argh, so duplicate work because of no email/communcation :/ [07:29] hm, could be, I thought Mathieu handled those as I pasted those to him - but let me check what's the status with that [07:31] Mirv: which branches did you have to fix? As I fixed all yellow prepare jobs that I saw yesterday, at least preparing merges for those from the Trusty side [07:35] sil2100: btw, any opinion on #ubuntu-unity? [07:39] vila: ok, seems 1.0.0~alpha2 is guilty [07:39] downgraded to alpha1 and no issue [07:39] (lxc) [07:43] didrocks: lxc on the host right ? [07:43] vila: right [07:44] then, there is another issue I'm debugging [07:44] aptsources.distro.NoDistroTemplateException: Error: could not find a distribution template for Ubuntu/trusty [07:44] hum, lsb-release is fine though [07:44] good, the use case for upstream-merger is slightly different as we use saucy on the host so I may be immune, will know soon [07:45] vila: I'm going to hold lxc upgrade [07:45] didrocks: rings a bell, heard something I didn't followup closely about patching distro-data manually :-/ [07:46] vila: well, ubuntu.csv contains 14.04 LTS,Trusty Tahr,trusty,2013-10-17,2014-04-17,2019-04-17 [07:47] * vila sighs [07:47] didrocks: something else then :-/ Sorry [07:47] sigh [07:48] so switching to trusty with pahbelt-flash works, but all installed apps are gone [07:48] thats annoying :/ [07:48] didrocks: python-apt-common needs updating [07:48] ogra_: I got that even without switching (i.e. image 101) [07:49] wgrant: ah, that's the one! thanks :) I can do it [07:49] ogra_: but I had to redo the restoring part of the backup manually (adb got stucked but I copied the backup.tar.gz during phablet-flash, lucky me) [07:50] Amusingly there's a new one stuck in proposed because bzr-builddeb autopkgtest failed. [07:51] ah, indeed :) [07:51] vila, well, by the looks of it it didnbt even do a backup, it just didnt touch my homedir [07:52] ogra_: so it failed to restore it , didn't phablet-flash hang ? [07:52] nope [07:52] just went through [07:52] i also didnt get a full image ... which i thought i was supposed if i swithc channels [07:53] * ogra_ will wait for stgraber [07:53] sil2100: right, I only saw the unmerged ones. so also fix needing in trusty were libcolumbus, autopilot, mtp, compiz, and in saucy some 10+ more [07:59] didrocks: can you redeploy head unity8 + mir for the unity-mir move? I can't deploy the latter since I don't have rights to lightdm [08:00] Mirv: sure, only head, right? nothing to deploy for saucy? [08:01] didrocks: I didn't touch saucy for those, and mir is already disabled there [08:01] Mirv: reminder: when we move/removed things, you need to delete by hand the jenkins job [08:01] didrocks: I remember that [08:01] Mirv: ok, deployed [08:02] thanks, cleaned the unity-mir from unity8 [08:02] thanks ;) [08:04] Mirv: right, saucy! I only posted the trusty merges to cyphermox it seems! [08:05] sil2100: can you relaunch the jobs where the prepare jobs failed? [08:05] just to ensure we are all good on that one at least [08:05] Mirv: sorry about that, we duplicated work because of that [08:05] didrocks: the saucy failures you mean? Sure [08:06] sil2100: I don't think there was any duplicates beside the autopilot + ui-toolkit, since you didn't start on the saucy, plus the trusty ones I did didn't have branches from you? [08:06] FYI unity-system-compositor was missing from head before didrock's redeploy. I'll now compare all stacks side-by-side head vs. saucy and clean up eg. removed jobs [08:08] didrocks: it seems all of those stacks were already ran and are now green for saucy \o/ [08:09] didrocks, Mirv: what should we do with stacks that have no components in them anymore? [08:09] (in saucy) [08:09] Should we remove those altogether? [08:09] sil2100: and head? [08:10] sil2100: if you can remove them from config + jenkins, I guess that will help clarifying, yeah [08:10] thanks Mirv, sil2100 [08:10] didrocks: clean as well [08:11] grrr, I can't reproduce the bzr-builddeb failure on saucy, even upgrading latest python-apt [08:11] will need a trusty chroot I guess [08:11] didrocks, Mirv: ok, I'll clean up the empty stacks then, as we see now that more or less things are working right now [08:12] didrocks: ! [08:12] didrocks: I'm just checking my mail and I got a mail from Larry about the lxc issues, let me take a look [08:13] didrocks: hah, let me forward it to you [08:13] didrocks: what was the lxc failure? [08:13] didrocks, any word on unity-mir and lxc-android-config release? [08:13] didrocks: it seems the issue is a bit bigger [08:13] cjwatson: I didn't debug it yet, just downgraded, I'm on the bzr-builddeb autopkgtests failing which is blocking otto as well [08:14] but hard to reproduce, can't start in the isolated env (following the doc from http://developer.ubuntu.com/packaging/html/auto-pkg-test.html#executing-the-test) but http://cloud-images.ubuntu.com/trusty/current/trusty-server-cloudimg-amd64-disk1.img doesn't exist yet [08:14] didrocks: forwarded the e-mail, seems like retoaded forwarded the problem to Larry and he made an investigation [08:14] not sure what adt is using then [08:14] Saviq: we need first to be able to run the test infra [08:14] IIRC adt-virt-lxc isn't actually packaged yet [08:14] Saviq: and trusty seems not ready for it yet, so working on it [08:14] didrocks, :/ [08:14] didrocks, k thanks [08:15] ok, I think I have to upgrade to trusty to hope reproducing the bzr-builddeb autopkgtests issue [08:16] at least, the 3 errors seems to have a common ground [08:17] the first one seems more hectic [08:18] vila: getting any progress on your side on the upstream-merger one? [08:19] ok head is up-to-date, with the only exception being cordova-ubuntu which will need some more info before adding it - ie. is the 3.0 branch ready to be released (I think not, there are among else hard-coded reference to 2.8 in the SDK) [08:19] * cjwatson would like to make it clear that the fact that you're blocked on python-apt in trusty is another casualty of Mark being slow to announce the name [08:19] didrocks: it anyway seems that the DNS issues we're encountering are some strange apparmor issues [08:20] If we'd had the new name in a timely fashion then we could've had support for it in saucy [08:20] Mirv: yeah, I saw cordova-ubuntu trunk when I was doing the splitting and didn't see a debian/ directory then even [08:20] Not sure how it is now [08:21] sil2100: I'm sure the webapps team will let us know when they want it - they've promised that they'll also update the sdk at that time [08:21] cjwatson: I think it's clear to all of us TBH [08:22] cjwatson: I'm trying to look at what exactly broke in lxc after the meeting [08:22] didrocks: Yeah, I just wanted to hammer it home :) [08:22] ;) [08:23] TBH, if we get everything running (including CI) less than a week after the release name was given, it's already an achievement TBH [08:23] diverging all branches, and so on… [08:23] we are not rolling, so we can't have a "no time interruption" as if we had it. I think people should understand that [08:24] I agree, but there's always pressure to be faster and I really resent having to break promises I've given because of something as simple as a name [08:24] didrocks: almost ready to do a run but I lack some context from fginther :-/ [08:24] And really, if we didn't have that blocker as we've had for the last three cycles, we could be open again almost immediately [08:25] oh 3? I was thinking it was only 2… [08:25] Three [08:25] We got raring's name about a day before quantal released, which was too late to add proper support to quantal [08:26] And even the cycle before that, we only had three days' notice, which wasn't enough [08:26] The cycle before that (so precise's name) we had eight days' notice, which was still less than I'd like but it was manageable [08:26] The cycle before *that* we had 52 days' notice [08:26] Mirv: right [08:28] I don't think I'll be on this morning's standup, I've had five hours' sleep due to kids invading the bed and I don't think I can manage coherent conversation in actual speech [08:28] didrocks: it failed, that's progress ;) The trusty container doesn't exist, now to find where it's create... [08:28] *created [08:28] cjwatson: no worry ;) [08:29] didrocks: it failed with an understandable error which is nice [08:29] heh [08:31] vila: joining? [08:31] didrocks: yup [08:51] popey, mind to test trusty-proposed ? we dont have call/sms tests for that yet [08:52] ogra_: sure thing [08:52] thx [08:52] i think didrocks tested the rest already [08:52] * popey OTA's [08:52] from saucy ? [08:53] no, from 14.04 [08:53] ah, k [08:53] I mean, OTA from saucy would be nice [08:53] but I suspect we're not planning to support that? [08:53] bug 1243573, bug 1243577 [08:53] bug 1243573 in ubuntu-system-settings (Ubuntu) "Timezone setting is gone after upgrade to Ubuntu Touch touchy" [Undecided,New] https://launchpad.net/bugs/1243573 [08:53] bug 1243577 in system-image (Ubuntu) "All apps are gone after upgrade to trusty" [Undecided,New] https://launchpad.net/bugs/1243577 [08:53] thats non OTA though ... [08:54] yeah, i flashed clean trusty-proposed on monday iirc [08:54] maybe tues [08:54] i tried to switch the channel from cmdline using system-image-cli -c ... but that didnt work :( [08:54] it pulled the channel properly from what i saw with -v but told me i'm up to date === vrruiz_ is now known as rvr [09:02] ogra_: sms and calls work fine in both directions [09:02] awesome, thanks [09:02] didrocks, asac ^^^ [09:07] ogra_: please promote then! :) [09:07] popey: ^ [09:07] will do [09:07] (or try to, not sure if the script behaves ... first trusty image etc) [09:15] ogra_: pay attention to what might have broken after :) [09:15] i am sure all phablet-flash will go crazy hehe [09:16] vila: so after a reboot, still nothing (worked on the nvidia machines, not on others FYI): http://10.97.0.1:8080/job/autopilot-trusty-setup_otto/label=qa-intel-4000/6/console [09:16] * didrocks will classify that as "unknown other issues" [09:16] asac, i wont switch the devel or devel-proposed channels, thats stgraber stuff ... [09:16] * ogra_ will just promote trusty-proposed to trusty [09:17] ogra_: lemme know when and I'll test reflashing without --no-backup [09:19] didrocks: makes sense since update_host ... update the host and the container so get the latest lxc no ? [09:19] vila: lxc in the container -> we don't really care [09:19] it's only the host one which is used [09:20] didrocks: update_host update the *host* [09:20] popey, didrocks, asac trusty 3/20131022.1 promoted [09:20] vila: yeah, but as told in the hangout, I hold lxc [09:21] vila: just try apt-cache policy on the machine to check [09:21] lxc: [09:21] didrocks: I'll start with that once done with u-m [09:21] Installed: 1.0.0~alpha1-0ubuntu11 [09:21] Candidate: 1.0.0~alpha2-0ubuntu3 [09:21] Version table: [09:21] 1.0.0~alpha2-0ubuntu3 0 [09:21] 500 http://us.archive.ubuntu.com/ubuntu/ trusty/main i386 Packages [09:21] didrocks: ack [09:21] *** 1.0.0~alpha1-0ubuntu11 0 [09:21] 100 /var/lib/dpkg/status [09:21] for instance [09:21] ogra_: thanks! [09:21] ogra_: thanks, will test [09:22] didrocks: for u-m update_host needs a tweak since it want to create a container for the host release (aka saucy here when I want a trusty container) [09:23] vila: are you using update_host to create the container? not otto directly? [09:23] didrocks: I'm using nothing for now ;) Looking at update_host for at least getting which parameters otto wants [09:27] asac: status email sent === psivaa is now known as psivaa-afk [09:30] didrocks: container building [09:33] didrocks: lxc_container: command get_init_pid failed to receive response harmless ?lxc-ls [09:33] grrr [09:33] ogra_: right, so flashing from saucy to trusty wipes /opt/click.ubuntu.com ☹ [09:33] didrocks: thanks! [09:33] do we have a bug for that? [09:33] popey, yup [09:34] popey, bug 1243577 [09:34] bug 1243577 in system-image (Ubuntu) "All apps are gone after upgrade to trusty" [Undecided,New] https://launchpad.net/bugs/1243577 [09:34] ta [09:40] didrocks: 2013-10-23 05:36:55,304 INFO Starting container 'trusty-amd64-20131023-0529' [09:40] lxc_container: command get_init_pid failed to receive response [09:40] didrocks: rings a bell ? [09:41] vila: can be anything meaning that lxc-start didn't work [09:42] ack, looking into that already, thanks [09:42] https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1181136 [09:42] Ubuntu bug 1181136 in lxc (Ubuntu) "Empty log file when a container is started with the API" [Medium,Confirmed] [09:43] didrocks: great :) subscribed... read.... :-/ [09:45] vila: there was another one I can't find right now about no log at all in stdout when using the python bindings [09:46] didrocks: seems to be related to the pre-start.sh indeed: [09:46] + [ -e /sys/fs/cgroup/memory/lxc/memory.use_hierarchy ] [09:46] + echo 1 [09:46] lxc-start: Script exited with status 1 [09:46] again the same issue :/ [09:47] whic is weird since that's commented out [09:47] vila: is it really? we reverted IIRC [09:48] didrocks: let me check on dx-autopilot-intel, on ps-radeon-hd8350 otto is up to date with lp:otto [09:48] vila: dx-autopilot-intel is a saucy machine, right? [09:49] roght [09:57] didrocks: ~ubuntu/otto is up to date, but ~jenkins/otto is not, stooopid vila [10:03] vila: you ps-radeon-hd8350? [10:03] didrocks: http://10.97.0.26:8080/job/autopilot-testrunner-otto-trusty/9/console so, tests are running but seem to be failing [10:03] vila: some are passing, so sounds good [10:03] vila: what stack is it? [10:04] didrocks: suspicious messages /var/log/syslog: Oct 23 10:01:36 trusty-amd64-20131023-0529 kernel: [60980.789436] [drm:radeon_dvi_detect] *ERROR* DVI-I-2: probed a monitor but no|invalid EDID [10:05] didrocks: http://10.97.0.26:8080/job/autopilot-testrunner-otto-trusty/9/parameters/? [10:05] didrocks: so unity8 IIUC [10:06] didrocks: I used the same parameters as fginther did on the previous runs [10:06] vila: making sense for unity8 to fail [10:06] didrocks: I think I'll stop on that one as I've already far too many unknowns to check with fginther [10:07] will look at qa-intel-4000 instead [10:08] vila: ok, thanks [10:21] didrocks: clarification, when you say 'I hold lxc' you mean on the host or in the archive ? I think we were having two discussions intermixed, me talking about ps-radeon and you talking about qa-intel. [10:22] didrocks: on qa-intel the container creation is failing so my question is: can use update_host there or will that bring the latest lxc back ? [10:22] didrocks: or will I get the same absence of usable error message by running update_host manually :-/ [10:26] vila: i hold the package on previous lxc on the 3 machines [10:26] well, lxc not updated and same messages... [10:26] didrocks: right, how do you achieve that ? [10:27] vila: apt-mark hold [10:27] didrocks: thanks [10:27] yw [10:33] * vila breaks for lunch [10:40] didrocks, vila: how's progress with otto? Any luck? [10:54] sil2100: I guess it's for vila [11:20] sil2100, didrocks: I don't think it solved itself during my lunch but I'll know that soon ;) [11:21] ;) [11:22] right, still the same, the container creation fails... with not a single msg when done from update_host, I need to drill down [11:24] meanwhile another round of autobuilds is progressing nicely [11:25] Mirv: were you able to contact anyone from upstream for https://bugs.launchpad.net/libfriends/+bug/1243527 ? [11:25] Ubuntu bug 1243527 in libfriends "libfriends FTBFS on trusty" [Critical,New] [11:27] sil2100: didier pinged robru about it [11:30] sil2100: (as it says on the stack status page) [11:30] sil2100: robert will ping ken in the morning [11:31] Mirv: ah! Right, it's been such a long time since we used that, I need to get my habits back [11:34] didrocks: I was finally able to do a proper retrace, it seems apport has various bugs in handling non-English locale that don't even show up as such. bug #1243665 "QUbuntu: Could not create application instance" so might be even mir or such problem [11:34] bug 1243665 in qtdeclarative-opensource-src (Ubuntu) "qmlscene crashed with SIGABRT in raise()" [Undecided,Confirmed] https://launchpad.net/bugs/1243665 [11:34] right, so apparmor DENIED on host's /var/log/syslog during lxc-create AFAICS http://paste.ubuntu.com/6288480/ [11:36] didrocks: you did reboot after downgrading lxc right ? [11:39] vila: yeah, on 2 of the 3 machines [11:39] but as it's failing on 2 [11:40] didrocks: I'm on qa-intel-4000 for now, pinged stgraber in #ubuntu-server [11:42] didrocks: the denied are for operation="mount" on /dev/input /dev/dri but the weird thing is that the pid (16121) is the same for all creations but I can't find that process in 'ps fax' [11:47] damn, re-trying again, the apparmor msgs don't come back... === alan_g is now known as alan_g|lunch [11:58] didrocks: no log files under /var/log/lxc for any container I tried to create, the last one if for trusty-i386-20131023-0008 but has been updated 10 minutes ago ??? [12:10] vila: you are trying to create with otto? [12:11] or lxc-create directly? [12:13] otto [12:14] hoping to find some msgs somewhere and avoid issues reproducing the way otto calls lxc-create, will try to do that now [12:15] *sigh* I pointed to https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1181136 [12:15] Ubuntu bug 1181136 in lxc (Ubuntu) "Empty log file when a container is started with the API" [Medium,Confirmed] [12:15] vila: that why I pasted that bug ^ [12:16] didrocks: for the lxc log itself, doesn't mean nothing can be found elsewhere [12:17] vila: what other non lxc logs you were expecting to get in /var/log/lxc? [12:18] didrocks: no idea, that 's why I looked and it's still weird that an old log file modification time changes, but I won't dig why as I doubt it's related === psivaa-afk is now known as psivaa [12:32] didrocks: using otto directly doesn't provide more output so I need to drill down to the lxc-create command but otto "prepares" a lot of stuff first (which are relevant is still unclear...) [12:34] didrocks: otto -d gives some hint but still fails too quickly.... that rings a bell [12:35] 2013-10-23 12:32:59,910 INFO Container 'trusty-i386-20131023-1216' started [12:35] 2013-10-23 12:32:59,911 ERROR An error during creation occurred, trying to cleanup the container: The container didn't st [12:35] 910 started, 911 ERROR, that's a little too fast [12:36] vila: the issue is clearly in lxc creation [12:36] and as per bug I posted above, you won't get any info from the container in lxc [12:37] due to the python bindings [12:37] so you need to start lxc-create by hand [12:37] didrocks: which otto doesn't display nor the context it requires either as per This method creates a new container from scratch. We don't want to use [12:37] the create method from LXC API because the bootstrapping phase is not [12:37] required and replaced by using an Ubuntu image directly. [12:38] vila: otto can't display the lxc logs as per bug :/ [12:38] vila: it's call lxc-start [12:38] calling* [12:38] didrocks: but it could display which lxc-create command it want to execute or are you .... no lxc-create ?? [12:38] at all ? [12:39] no lxc-create, lxc-start on the container [12:39] so a classic lxc-start -n [12:39] so you look at what container it created last [12:39] you mv broken. [12:40] and sudo lxc-start -n [12:41] pfew...... mv broken. good to get the context creted by otto, I see [12:42] vila: broken. is to not reuse the broken container [12:42] but not removing it [12:42] so that you can still debug [12:42] and you stay on latest good container [12:43] yeah, got that, confused because the last debug session I saw was Mirv extracting the delta, different case [12:44] didrocks: and if I haven't read lxc-create when you wrote lxc-start earlier.... [12:45] didrocks: lxc-start succeeds.... [12:45] vila: well, the second time, I wrote lxc-create TBH (because of broken mind)… [12:46] didrocks: nevermind, let's go ahead and fix that crap if we can without stgraber help [12:47] didrocks: hmm, the container just logged me out and terminate [12:47] because it finished its upgrade [12:48] didrocks: now I'm confused. If it starts properly and halt properly, what are we searching for ? [12:48] vila: are you sure it's doing everything properly? [12:48] if it didn't exit 1, yeah, there is an issue [12:49] if you are starting the right container [12:49] didrocks: no, not even sure I started with good one, so let me re-start that from scratch, update_host, use that one [12:56] didrocks: http://paste.ubuntu.com/6288876/ [12:57] vila: it exits 1, right? [12:58] didrocks: lxc-start ? 255 [12:58] vila: so… not 0, hence the command failing I guess [12:59] didrocks: but it's not the same behavioras when run from otto (,910 -> ,911 earlier) I'd say [12:59] behavior as [12:59] weird [12:59] not a spanish variant ;) [13:00]