[05:11] <pitti> Good morning
[05:55] <didrocks> good morning
[06:01] <pitti> bonjour didrocks
[06:03] <didrocks> hey pitti, how are you?
[06:07] <pitti> didrocks: quite well, thanks! juggling systemd, langpacks, and urgent apport features :)
[06:07] <didrocks> good luck!
[06:07] <pitti> et toi ?
[06:08] <didrocks> I'm fine, thanks
[06:24] <larsu> morning :)
[07:11] <pitti> hey larsu, wie gehts?
[07:12] <seb128> hey pitti & desktopers
[07:12] <seb128> pitti, wie gehts?
[07:12]  * pitti ^5s seb128, gut danke! und dir?
[07:12] <seb128> pitti, larsu is off for the morning, he's have spanish classes this week (and shifted hours)
[07:12]  * seb128 ^5s pitti back, good, danke!
[07:12] <pitti> seb128: ah, I missed his "good morning" earlier
[07:54] <seb128> hum
[07:54] <seb128> so that iso doesn't boot in kvm :-/
[07:54] <seb128> "systemd: Started udev Coldplug all Devices."
[07:54] <seb128> it just sits there
[07:55] <seb128> 13s in the boot
[08:06] <Laney> hello!
[08:07] <didrocks> morning Laney
[08:08] <seb128> hey Laney
[08:11] <seb128> pitti, is boot blocking on "systemd: Started udev Coldplug all Devices." symptomatic of some known class of system misconfiguration/bug?
[08:13] <Laney> hey didrocks & seb128
[08:13] <seb128> Laney, how is u.k today?
[08:15] <Laney> cloudy, rain later
[08:15] <Laney> and nl?
[08:18] <seb128> sunny today!
[08:18] <seb128> and not too windy, which is nice
[08:22] <pitti> seb128: not really -- that sounds like a kernel driver issue usually, but hard to tell without more logs
[08:22] <pitti> seb128: drop "quiet splash" and boot with "debug systemd.debug-shell"?
[08:23] <seb128> pitti, k
[09:18] <seb128> pitti, it hangs on http://imagebin.ca/v/25WyUYshyvpy ... no way to go to vt8 or something
[09:28] <pitti> seb128: hm, is that with a new kernel or something? since when did that happened?
[09:30] <Laney> bet a thousand euros it's the personal image
[09:30] <Laney> :)
[09:30] <seb128> Laney, you won!
[09:31] <seb128> pitti, it's a candidate of snappy personnal image I'm trying to build in kvm
[09:31] <seb128> it's the first image build that worked so can't compare to "previous versions/since"
[09:32] <pitti> seb128: ah, ok; and no VT switching? seems that something is breaking stuff really hard then :/ could you try moving /etc/sysctl.d/ away for testing if it's one of the knobs there which trigger a kernel fault or so?
[09:47] <seb128> pitti, how do you I move things away without a command line? can I do that from grub?
[09:47] <seb128> booting with upstart it hits cloud-init expection errors and seem to bug on that
[09:47] <pitti> seb128: you can't boot rescue either?
[09:47] <seb128> some mkdir failing
[09:47] <pitti> seb128: you could boot with init=/bin/bash
[09:48] <pitti> then mount -o remount,rw /
[09:48] <pitti> and h4ck stuff there
[09:48] <seb128> let me try recovery
[09:48] <pitti> I can help you in a bit, I'm just rather swamped ATM
[09:49] <seb128> don't worry, thanks for the hints
[09:49] <pitti> and Laney's "fix udisks" club is becoming a real threat too
[09:49] <pitti> seb128: in the mean time, feel free to scp it to people.c.c.?
[09:49] <seb128> pitti, it's a 10G image, going to take a while on my dsl line, going to do that over lunch
[09:50] <pitti> seb128: convert it to qcow2?
[09:50] <pitti> seb128: please don't upload a raw image :)
[09:50] <seb128> k
[09:50]  * seb128 googles
[09:51] <seb128> qemu-img convert -f raw -O qcow2
[09:51] <pitti> seb128: qemu-img convert -O qcow2 xx.raw xx.img
[09:51] <pitti> seb128: you can then also use -snapshot
[09:51] <pitti> when running the VM, to avoid modifying it
[09:52]  * didrocks has some easy-to-use scripts for this if interested
[09:52] <seb128> didrocks, converting?
[09:52] <seb128> shoot! :-)
[09:52] <didrocks> seb128: not converting, but starting with/without snapshot, using hw accel…
[09:53]  * Laney hugs pitti 
[09:53] <Laney> feel free to punt that if you have some hints about where to look
[09:53] <didrocks> seb128: http://people.canonical.com/~didrocks/vms/vmcommon
[09:53] <seb128> didrocks, I'm interested!
[09:53] <didrocks> seb128: http://people.canonical.com/~didrocks/vms/ rather
[09:53] <seb128> didrocks, 'ci
[09:53] <pitti> fun how everyone has their own :)
[09:53] <didrocks> seb128: make them all executable, but vmcommon
[09:54] <seb128> k
[09:54] <pitti> http://people.canonical.com/~pitti/scripts/vm and http://people.canonical.com/~pitti/scripts/vmbr here
[09:54] <didrocks> Usage: /home/didrocks/bin/vm [--master] <image> [vm options...]
[09:54] <pitti> so that I can do "vm /my/ubuntu.img -snapshot"
[09:54] <pitti> and vmbr is with using the lxcbr0 bridge for "proper" networking
[09:54] <Laney> you crazy people
[09:54] <didrocks> interesting, mine are -snapshot by default, unless you specify --master to modify base image
[09:54] <Laney> I just load it up in virt-manager :P
[09:55] <didrocks> (as I'm using snapshot way more)
[09:55] <pitti> didrocks: yeah, I tend to do that too, good idea
[09:55] <didrocks> the vm-net is inspired by the commands you gave me some times ago, seems similar to vmbr :)
[09:55] <pitti> yeah, "vm" just searches a free port and uses that for ssh redirection, vmbr does the bridge thing
[09:57] <didrocks> Laney: remember that you are the one liking tiling wm :)
[10:57] <larsu> hi pitti :)
[10:57] <larsu> man it's nice out
[11:00] <ogra_> in berlin too ?
[11:04] <pitti> c'était gris en Bavière d'hier :(
[11:11] <larsu> ogra_: ya. Perfect.
[11:12] <ogra_> kassel too :)
[11:12] <larsu> nice :)
[11:24] <pitti> Laney: udisks is an actual regression; investigating now
[11:24] <pitti> perhaps related to newer kernel/util-linux/udev, as udisks2 itself hasn't changed in a while
[14:28] <seb128> pitti, ok, so I managed to get a debug console, tons of jobs fail to start :-/
[14:28] <seb128> journalctl has a lot of "failed dependencies", is there a way to tell which one?
[14:29] <seb128> also trying to start e.g getty@tty7 returns a job for ... canceled"
[14:29] <larsu> it is configured to only allow 6 gettys
[14:29] <larsu> this is on touch?
[14:30] <pitti> seb128: can you pastebin a journalctl output?
[14:30]  * pitti needs to run for some errands, back in 20
[14:31] <seb128> larsu, ignore getty, I can list any other job in the same situation, rc-local.service if you prefer
[14:31] <seb128> pitti, not easy, I'm in a debug console of a kvm without much access to outside
[14:31] <pitti> seb128: systemctl --failed
[14:31] <seb128> larsu, and no, it's not touch, it's snappy personnal try
[14:31] <seb128> pitti, that lists 38 failed jobs
[14:32] <seb128> but mostly mount ones
[14:32] <seb128> which I know why they fail, but that shouldn't block the system to start, should it?
[14:32] <larsu> depends on which ones fail I guess
[14:33] <seb128> larsu, /writable touch rw overwrites
[14:33] <seb128> not system partitions
[14:33] <larsu> hm?
[14:33] <seb128> hum?
[14:33] <larsu> huum?
[14:33] <seb128> what do you "hm?" about?
[14:33] <larsu> not sure what you mean by "/writable touch rw overwrites"
[14:33] <seb128> the fstab includes bindmounts for touch
[14:33] <seb128> which don't work on snappy
[14:34] <seb128> but I don't really care if those locales stay ro
[14:34] <seb128> so those jobs failing is fine to me (if that doesn't block other things)
[14:34] <larsu> ah, that makes sense
[14:34] <larsu> haven't dug that deep into systemd yet
[14:34] <larsu> but probably it's configured to not boot when mounts fail
[14:35] <seb128> is there a "retry all failing jobs?" ;-)
[14:35] <larsu> I don't think so, no :)
[14:36] <larsu> there's probably some mount target that you can activate though
[14:36]  * larsu looks
[14:38] <larsu> seb128: what does `systemctl list-units --type=targets --all` say?
[14:38] <larsu> especially for the local-fs.target
[14:39] <seb128> "unknown unit type or load state 'targets'"
[14:39] <larsu> sorry, it's --type=target
[14:40] <seb128> all is not-found
[14:40] <larsu> not good :/
[14:40] <seb128> syslog as well
[14:40] <seb128> everything else is loaded
[14:40] <seb128> local-fs is loaded inactive dead
[14:40] <seb128> which I guess is not good
[14:41] <larsu> indeed
[14:41] <larsu> should be active :)
[14:41] <larsu> you can try reactivating it, but I'm not sure why that would work now
[14:42] <seb128> yeah, a dependency job failed
[14:42] <seb128> but it doesn't say which one
[14:42] <larsu> not even in the jounral?
[14:43] <larsu> journalctl -f in a separate terminal is really handy btw
[14:44] <seb128> yeah, just too much content in there
[14:44] <seb128> like it lists 50 jobs that fail
[14:46] <seb128> k, that's ridiculous, I guess I need to clear out the fstab/writable issue first and see what remains
[14:46] <pitti> seb128: if fstab mounts fail that aren't marked with "nofail", they will fail local-fs.target and with it pretty much everything else
[14:46] <seb128> pitti, yeah, I guess it's the issue there
[14:47] <seb128> pitti, can I edit fstab and tell "retry all the jobs"? ;-)
[14:47] <seb128> or retry start local-fs ?
[14:47] <pitti> seb128: (untested) "systemctl default"
[14:47] <pitti> seb128: but might be easier to just reboot :)
[14:48] <pitti> seb128: systemctl status local-fs.target doesn't say which dependency failed? it shold
[14:51] <seb128> pitti, hum, are changed to qcow2 images stored/persistent?
[14:51] <pitti> seb128: normally yes, unless you start qemu with -snapshot
[14:52] <pitti> usually better to start with -snapshot, as you can hack away without destroying the image
[14:52] <seb128> k
[14:52] <seb128> pitti, http://picpaste.com/qemu-7fiqV7Bv.png
[14:53] <didrocks> or if you use my script… :)
[14:53]  * seb128 tries reboot
[14:54] <pitti> seb128: ok; then "systemctl --failed --type=mount" should list the failed ones
[14:54] <seb128> pitti, yeah, there is like 38 of those
[14:55] <seb128> ok, boots now to what looks like a display server fail, progress!
[14:57] <seb128> didrocks, do you remember why you didn't include http://bazaar.launchpad.net/~ubuntu-core-dev/livecd-rootfs/trunk/view/head:/live-build/ubuntu-core/hooks/20-extra-files.chroot in the ubuntu-desktop-next dir?
[14:57] <seb128> that seems the issue
[14:57] <seb128> the /writable dir is missing so all the mount job fail
[14:57] <seb128> jobs*
[14:58] <pitti> seb128: ah, nice!
[14:58] <pitti> ln -sf /proc/mounts /etc/mtab
[14:59] <pitti> oh for $DEITY's sake, can we just completely eradicate /etc/mtab?
[14:59] <pitti> it seems snappy is the one thing that doesn't need to be concerned about 30 year old tools for disks/mounts administration?
[14:59] <didrocks> seb128: once sec, looking (sorry, finishing up opening a BTS issue on licensecheck)
[14:59] <seb128> didrocks, thanks
[15:00] <seb128> pitti, yeah, I just want the "mkdir -p /writable"
[15:00] <seb128> mvo_, ogra_, http://bazaar.launchpad.net/~ubuntu-core-dev/livecd-rootfs/trunk/view/head:/live-build/ubuntu-core/hooks/20-extra-files.chroot ... see pitti's comment about mtab
[15:00] <didrocks> seb128: maybe a mistake. I remember to have deleted one file on purpose, but I don't think it was that one
[15:01] <pitti> mvo_, ogra_: specifically, did you just add /etc/mtab "because we always did", or does something on snappy actually need it? if the latter, can we fix that?
[15:01] <ogra_> seb128, yes, and ? i'm fully with him
[15:01] <ogra_> pitti, no, i had to add it else mountall tries to set it up
[15:01] <pitti> (I'm volunteering for fixing it, FTR)
[15:01] <ogra_> we discussed that, remember
[15:01] <pitti> ogra_: aah, ok
[15:01] <seb128> ogra_, so it's needed?
[15:01] <pitti> ogra_: so that was just to quiesce mountall
[15:01] <ogra_> it needs to be a sym,link
[15:01] <pitti> but that doesn't affect snappy, just touuch?
[15:01] <ogra_> we used to do that on the s-i server when re-packing
[15:02] <ogra_> does systemd have anything handling mtab ?
[15:02] <ogra_> i think you said no
[15:02] <pitti> ogra_: our package has a "fixup" job to replace a file with a symlink
[15:02] <pitti> upstream, no
[15:02] <ogra_> then snappy wont need it
[15:02] <pitti> the last bit of util-linux was fix in 2.25 I believe (but we only turned on that option in wily)
[15:03] <pitti> ogra_: ah, I just see that we do need to adjust /lib/systemd/debian-fixup
[15:04] <pitti> we don't need /lib/systemd/system/debian-fixup.service in snappy at all, so if we are looking for some 0.01 s optimization somewhere.. :)
[15:04] <seb128> ogra_, pitti, so should I keep it or delete it?
[15:05] <pitti> seb128: delete it, and I'll fix debian-fixup.service
[15:05] <ogra_> pitti, btw, i discussed mtab with slangasek when i added that ... (we are trying to get rid of it since at least 5 years) and i think he said it would probably time to re-visit that ... iirc there are still some debs that rely on it and iots format he said
[15:05] <pitti> it'll fail on a r/o image until then, but that's harmless
[15:05] <seb128> pitti, k, what happens without your fix?
[15:06] <ogra_> seb128, /etc isnt writable ...
[15:06] <pitti> seb128: debian-fixup.service will likely fail as it can't create the /etc/mtab link
[15:06] <ogra_> so whatever would try to create it would fail
[15:06] <seb128> k, but that doesn't block the boot?
[15:06] <pitti> seb128: no, should be fine; it's just a wants
[15:06] <seb128> great
[15:07] <ochosi> Laney: pressed the power button on my new xps13 for the first time one minute ago
[15:07] <seb128> ogra_, mvo_, do you want it out of ubuntu-core as well?
[15:07] <seb128> ochosi, oh, you received it? great :-)
[15:07] <ochosi> yep! :)
[15:07] <ogra_> seb128, i dont reallyy care, just leave it in touch
[15:07] <ochosi> first time i see ubuntu OEM
[15:07] <Laney> ochosi: nice, good luck ;-)
[15:08] <ochosi> Laney: well i've already downloaded a 15.04 iso, but i still wanted to see what the ootb experience was
[15:08] <ogra_> the point of this hook is to make it possible to use plain cdimage rootfses without them being re-packaged on the s-i server
[15:08] <ogra_> i thinnk it will make mountall fall over or at least unhappy on the phone
[15:11] <ochosi> Laney: is it just Dell or do you ship chrome by default these days?
[15:11] <Laney> haha of course not
[15:11] <seb128> ochosi, just oem images
[15:11] <Laney> that's them
[15:11] <ochosi> thought so, just wanted to be sure
[15:12] <seb128> we have discussions every now and then about chromium by default but it never managed to make it default
[15:12] <ochosi> yeah, basically it'd mean replacing tb as well, no?
[15:12] <ochosi> at least to get the full benefit of dropping XUL
[15:12] <seb128> no
[15:13] <seb128> yeah, well that's not the main driver motivation to change
[15:13] <seb128> the people who want chromium default just think that it has more users/is more popular
[15:14] <ogra_> well, it will soon have less ads :)
[15:14] <ogra_> (given the new firefox ad model)
[15:16] <ochosi> Laney: whoa, who made BIOS look like Windows 95??
[15:33] <seb128> pitti, ok, next problem, the vm boots to what looks like x failing to start, but it loops on that and no way to go to a vt or anything, any clue how to debug?
[15:33] <seb128> recovery does the same
[15:34] <seb128> tried to boot with init=/bin/bash but that hangs boot and doesn't give me a shell
[15:34] <pitti> seb128: can you boot with console=ttyS0, and with qemu option "-serial stdout"
[15:34] <pitti> sorry, "-serial stdio"
[15:34] <pitti> seb128: then you will get a serial console and the boot messages there
[15:35] <pitti> and don't need to rely on the internal VTs
[15:36] <seb128> pitti, great, danke
[15:38] <seb128> arg
[15:38] <seb128> don't ctrl-C on that std :p
[15:38] <seb128> it stops the qemu
[15:40] <pitti> seb128: yeah :/
[15:40] <pitti> seb128: do you get some useful output and a getty there?
[15:41] <seb128> pitti, yes, basically xorg fails to start as expected
[15:41] <seb128> it just do it in loops for ever
[15:41] <seb128> and in recovery mode as well? (that feels wrong)
[15:41] <pitti> so lightdm keeps restarting? or the autologin does?
[15:41] <pitti> yes, recovery shouldn't start lightdm and default.target and all that
[15:42] <seb128> lightdm keeps restarting as far as I can tell
[15:43] <ogra_> seb128, hmm you dont create the firmware and modules mountpoints ?
[15:43] <ogra_> (why ?)
[15:43] <seb128> ogra_, is that needed and why?
[15:43] <seb128> because I don't understand why that's here
[15:43] <seb128> and it's undocumented
[15:43] <ogra_> to give your system access to modules and firmware files :)
[15:43] <seb128> so I try without and if it's needed I'm going to know why and document it
[15:43] <ogra_> that is where the initrd mounts them
[15:43] <ogra_> s/mounts/bind mounts/
[15:44] <seb128> so it's needed?
[15:44] <ogra_> if you plan to use any modules or firmware ... yes
[15:44] <ogra_> well, if you always only use system-image re-packed rootfses it will currently still work because the s-i server re-packs the whole rootfs just to add these three dirs
[15:45] <ogra_> but we are planning to drop that code from there
[15:45] <ogra_> this hook was in preparation of the droppage
[15:45] <seb128> oh ok
[15:45] <ogra_> so if you want it future proof, better create these two dirs too
[15:46] <seb128> I was going to say, livecd-rootfs copy things to those dirs
[15:46] <seb128> k
[15:46] <seb128> noted for the next upload, thanks
[15:46] <ogra_> no, it doesnt ... they are just nodes for mountpoints
[15:46] <ogra_> (firmware and modules come from the device tarball, separate from the rootfs)
[15:47] <seb128> ogra_, well, on the booted image those dirs exist and have content from the kernel image deb
[15:47] <ogra_> http://bazaar.launchpad.net/~ubuntu-system-image/ubuntu-system-image/server/view/head:/lib/systemimage/generators.py#L525
[15:47] <ogra_> thats the code in system-image the hook replaces
[15:47] <ogra_> (and which we will drop)
[15:48] <seb128> pitti, yeah, according to the journal lightdm does "Failed with result 'exit-code'" and it keeps retrying
[15:48] <seb128> ogra_, k, thanks
[15:51] <mvo_> seb128: could you do me a favour and reject my accidental golang-gettext from vivid-proposed. that was a mistake
[15:52] <seb128> mvo_, done
[15:52] <mvo_> ta
[15:53] <seb128> yw
[15:56] <ochosi> Laney: oh wow, installing 15.04 cleanly doesn't seem like much fun... (uefi and no wifi)
[15:56] <ochosi> and sry, i'll stop spamming you with this now
[15:56] <Laney> ochosi: I just got bcmwl on there manually
[15:57] <Laney> and it is installed in efi mode
[15:57] <ogra_> 15.04 isnt even usable on the old XPS13
[15:57]  * ogra_ will actually do a reinstall of trusty on the weekend 
[15:58] <ochosi> Laney: switching to legacy left me with: floppy, cd/dvd, usb and mini-sd as boot options...
[16:58] <seb128> Laney, can you retry desktop-next iso build?.
[16:58] <Laney> try using your powers?
[17:00] <seb128> Laney, where is the button?
[17:00] <seb128> I don't see any control on https://launchpad.net/~ubuntu-cdimage/+livefs/ubuntu/wily/ubuntu-desktop-next
[17:01] <ogra_> ssh nusakan :)
[17:01] <Laney> wrong and wrong
[17:01] <Laney> go to http://iso.qa.ubuntu.com and log in
[17:01] <Laney> then go to "Wily daily"
[17:01] <Laney> tick the boxes next to the things you want to rebuild
[17:01] <ogra_> pfft
[17:02] <seb128> shrug
[17:02] <seb128> login fails "The name seb128 is already taken."
[17:02] <larsu> lol
[17:02] <Laney> haha
[17:02] <Laney> did it use sso?
[17:02] <larsu> other seb :D
[17:02] <seb128> "Have you forgotten your password?" points to http://iso.qa.ubuntu.com/user/password
[17:02] <seb128> which is "The resource you tried to access doesn't exist. This can happen because of an invalid link or because of a bug."
[17:02] <seb128> great
[17:02] <seb128> quality website
[17:03] <Laney> be charitable please
[17:03] <seb128> sorry ;-)
[17:03] <Laney> probably go ask stgraber, that is his product
[17:03] <seb128> can you click for me then? ;-)
[17:04] <Laney> fine, if you go ping him so that I don't have to keep doing this forever
[17:04] <Laney> probably good to see if doing it via this button works first anyway ...
[17:04]  * Laney always did it over ssh before
[17:04] <seb128> Laney, done on -devel
[17:05] <Laney> thx
[17:05] <seb128> thank for pointing me to the website
[17:07] <Laney> didn't work anyway
[17:07] <Laney> #FAIL
[17:09] <seb128> :-/
[17:09] <seb128> stgraber is looking at my issue
[17:09] <seb128> it's better I'm older than the sso
[17:09]  * seb128 dinosaur
[17:32] <Laney> :)
[17:32] <Laney> I retried it the old way for now
[17:34] <seb128> Laney, thanks