[00:08] backlight [00:08] woops [00:17] nacc: yes [00:17] rbasak: ack, will do [00:17] Thanks! [00:19] rbasak: and actally, i'm going to delete the vivid/wily tasks for mysql-5.7, as it doesn't exist there, if that's ok with you? i've never been quite sure on that policy [00:19] Deleting tasks is a little dangerous, as they can't be recreated (Launchpad bug) [00:19] So I usually use Invalid instead, since I found out about that. [00:19] rbasak: oh i didn't realize that, as i thought i had recreated them in the past [00:19] will just leave at don't fix [00:20] *won't [00:20] Sure [00:39] rbasak: if you have the time, i would like to still chat tmrw to talk about the UOS session [01:04] nacc: sure [01:05] rbasak: thanks === JanC is now known as Guest21552 === JanC_ is now known as JanC === salem_ is now known as _salem [05:02] Good morning [05:37] good morning === cpaelzer_ is now known as cpaelzer [06:13] nacc, probably yes [06:15] coreycb, pandas sync? [06:15] not sure BTW [07:15] nacc, we could. or are you on that libvirt apparmor one already cpaelzer ? [07:15] nacc, cpaelzer for xenial the question is whether we want to push things over the current proposed to limit the number of releases [07:16] erm and morning [07:26] smb: morning :-) [07:26] smb: no deep in qemu atm - no touch on libvirt by me [07:27] * smb hands cpaelzer a snorkel [07:27] cpaelzer, ack, then I will have a look as time permits [07:27] smb: if the fix is easy and the current xenial hasn't passed proposed I'm for your suggestion of pushing over the current one [07:27] smb: the current one is only the preventive fix for the service names [07:27] smb: that one surely can take a sibling I think [07:28] smb: otherwise if bigger I'm for "one change at a time" [07:28] * cpaelzer is known as snorkler [07:28] cpaelzer, right, the not shutting down all guests is more annoying that that and the fix is rather simple [10:20] nacc, llvm-toolchain-3.7 fixed for powerpc [10:21] caribou, ^^ feel free to steal the patch for 3.6 [10:21] LocutusOfBorg: ok, will look at it [10:24] let me know if you upload please [11:23] nacc: rbasak: LocutusOfBorg has a fix for the powerpc FTBS on 3.7 that I can backport to 3.6 [11:25] nacc: rbasak: then Debian has just removed llvm usage from the package until upstream fixes things to use LLVM 3.8 [11:25] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=839850 [11:25] Debian bug 839850 in src:clamav "clamav: FTBFS with LLVM 3.8" [Serious,Fixed] [11:25] So either I upload a new LLVM 3.6, or we just disable LLVM in clamav, & forget about 3.6 [11:26] nacc: rbasak: not sure what's the best option now that we're in Final Freeze === hikiko is now known as hikiko|ln [12:06] caribou: thanks! Maybe best to ask the release team? [12:09] caribou: maybe from an SRU perspective, if it's just the binary name, it'd be better to add a workaround to clamav for that for Yakkety only (and not copy that forward to Z)? [12:09] I don't know what mechanism we might use exactly, but what I mean is the principle that we do the minimal thing for Yakkety now and then fix llvm in Z. [12:10] And that minimal thing could be in clamav so as not to risk other packages. [12:10] rbasak: imho, minimal for Y is to add LocutusOfBorg's two line patch to llvm3.6 to fix the powerpc's FTBS === _salem is now known as salem_ [12:11] clamav remains untouched [12:11] caribou: is that minimal though? It could impact every SRU as dependent packages are rebuilt. [12:11] llvm-toolchain-3.6 builds for all architectures [12:11] rbasak: according to nacc & my check, only clamav depends on llvm3.6 [12:12] Oh, OK. That's fine then I guess :) [12:12] rbasak: that being the reason to get rid of it, but now is a bit too late [12:12] Yeah. [12:12] It sounds fine to me then if the release team agree. [12:12] rbasak: I'll fix that for Z [12:12] ok, I'll check with them [12:12] Thanks again for handling this! [12:13] LocutusOfBorg's fix is already in 3.7 === hikiko|ln is now known as hikiko [12:38] pitti, hey - need some help on bug 1631328 [12:38] bug 1631328 in systemd (Ubuntu) "ceph-osd stays blocked on Yakkety: "No block devices detected using current configuration"" [Undecided,New] https://launchpad.net/bugs/1631328 [12:38] what's the right way to install a new udev rules file that relies on a user/group that the package maintainer script has not yet created [12:39] it appears the yakkety udev drops that part of the rule when loading them? [12:43] jamespage: such rules shouldn't be a problem -- we install those all the time, as long as the postinst actually addgroups them [12:43] pitti, it does [12:43] X on yakkety with two displays is giving me an area that acts like a hall of mirrors [12:44] pitti, but I'm def seeing the behaviour I think I am [12:44] pitti, udev complains about the missing user/group [12:44] about 2 seconds before they are created [12:44] and then block devices don't get the correct perms (ceph/ceph) until I do a restart or reload of udev [12:44] jamespage: yes, I figure it picks up the new rule during unpack via inotify [12:45] that was my guess [12:45] jamespage: your postinst needs to udevadm trigger those after group cretion [12:45] but it seems it already does? [12:45] * jamespage looks [12:45] I don't think so [12:46] jamespage: merely installing an udev rule also doesn't actually do anything -- in order for them to get active the device needs to get removed/added or udevadm triggered [12:46] jamespage: oh, I thought because you said so in #8 [12:46] pitti, yeah I know that - the charm does that bit once its configured with block devices etc... [12:46] the charm does a udevadm trigger to make udev pickup disks its just prepared for use [12:46] the trigger is firing, but the perms are not applied [12:47] so its kinda like an incomplete rule application [12:47] pitti, I guess a fix is to reload the udev daemon in the postinst script [12:48] jamespage: udevadm control --reload? [12:48] jamespage: could work, yes, if that's a race condition [12:48] it only auto-updates via inotify every 5 seconds or so, to avoid bursts [12:49] pitti, its def a race condition [12:50] its a shame that when udev does not detect a user/group, it just sets up the rules partially [12:50] that feels a bit odd [12:53] jamespage: it's actually odd that it runs the rules at all during package installation [12:54] pitti, its not [12:54] normally that doesn't wiggle devices [12:54] pkgs installed, charm configures block devices, then triggers the udevadm trigger [12:55] pitti, my issue is that the rule gets partially loaded due to the race with user/group setup [12:56] pitti, http://paste.ubuntu.com/23288779/ look ok? [12:56] jamespage: that's after the addgroup? [12:56] pitti, addgroup is done in ceph-common, rule is installed in ceph-osd [12:57] ceph-osd -> ceph-base -> ceph-common [12:57] jamespage: well, still needs to be "after" -- if ceph-osd depends: ceph-common that's also fine [12:57] so it should happen that way [13:15] LocutusOfBorg, looking at pandas [13:20] LocutusOfBorg, synced, thanks for the nudge [13:29] thanks === jgrimm is now known as jgrimm-out [14:42] smb: cpaelzer: yeah, i trust your judgment(s) [14:43] LocutusOfBorg: ack (re: powerpc fix) [14:43] caribou: ack on fixing powerpc for now and removing the dep in Z [14:44] nacc: I'm on it; will upload early monday morning unless the release team tells me otherwise [14:44] (or before eod if they ping me) [14:44] caribou: great, thanks for following up on it! [14:51] nacc: it is scary you trust me and I don't even remember what I wrote [14:51] * cpaelzer reading back [14:51] cpaelzer: re: libvirt fixes and whether to combine more into -proposed [14:52] nacc: ah yeah, I already discussed on this and more with smb - good for now [14:52] cpaelzer: thanks! [14:52] nacc: I debugged and triaged a bug we had, but we decided to postpone until after release [14:52] nacc: so it got to the backlog with some extra info [14:52] nacc, cpaelzer yeah ... actually its alrady uploaded now [14:52] smb: thanks [14:52] smb: thanks! [14:57] cpaelzer, Oh to clarify, nacc mentioned different messages of apparmor which I could not reproduce at all. For those I asked in the bug report for confirmation they are not gone for other reasons. The one we looked at are new to yakkety while the other ones were in Xenial [14:58] cpaelzer: it was just another libvirt bug in the server-next queue [15:41] cyphermox: do you have some time or know someone to verify bug 1592721? [15:41] bug 1592721 in network-manager (Ubuntu Xenial) "Don't write search domains to resolv.conf in the case of split DNS" [Medium,Fix committed] https://launchpad.net/bugs/1592721 [15:41] ok [15:41] it's otherwise ready to release [15:41] yep === frickler_ is now known as frickler [17:21] pitti: you mentioned a quote in the last foundations meeting (yes I read those :P) stating something like "if it [17:21] argh I hate this enter key placement! [17:21] "if it's hard, do it more often" [17:21] pitti: where did you find that? :P [17:22] pitti: or is that your quote? I was just wondering because it was in quotation marks [18:53] cyphermox: looks like a regression on your most recent update to initramfs-tools..https://bugs.launchpad.net/ubuntu/+source/initramfs-tools/+bug/1631474 [18:53] Launchpad bug 1631474 in initramfs-tools (Ubuntu) "No networking with initramfs-tools 0.122ubuntu8.3 and ip=dhcp boot option" [Undecided,New] [19:22] tsimonq2: I think it's a Google motto [19:22] tsimonq2: but I really like it === xclaesse` is now known as xclaesse === zigo is now known as Guest10079 [19:34] chiluk: LP: #1631474 : you may want to also sync with lamont on this [19:34] Launchpad bug 1631474 in initramfs-tools (Ubuntu) "No networking with initramfs-tools 0.122ubuntu8.3 and ip=dhcp boot option" [Undecided,In progress] https://launchpad.net/bugs/1631474 [19:34] slangasek ok .. at the moment, I'm just tryign to understand what the user is attempting to accomplish with ip=dhcp ... perhaps they are using a nfs root? [19:34] I don't know just yet. [19:35] slangasek I'll drive it for the time being.. it looks like a fairly straightforward bash change. [19:35] chiluk: ok [19:35] I just need to figure out what's not exactly working [19:35] bdmurray: can http://launchpadlibrarian.net/275354292/update-manager_1%3A16.10.2_1%3A16.10.3.diff.gz be changed to not pull in the whole python3-launchpadlib stack into standard? (http://people.canonical.com/~ubuntu-archive/priority-mismatches.txt) [19:36] chiluk: there's mention in the bug of AWS, which is a particularly strange place to be doing nfs or iscsi root, I would think [19:36] well, it's really broken anyway [19:36] yeah that's my thought as well. and that is correct it is in aws [19:36] bdmurray: like, dropping it to a Suggests: and gracefully not doing PPA changelogs if the module doesn't exist, or doing REST calls with urllib? [19:36] cyphermox: what's really broken? [19:36] that logic there in initramfs -- I see exactly what's wrong [19:37] we don't skip on, all, and dhcp when parsing IP [19:37] yeah the device isn't getting declared explicitly? [19:37] chiluk: well, klibc had a mess of different options differently configured [19:38] infinity: ah, perl is being dragged in by the new "rename" -- that seems awfully heavy too [19:38] do basically the if there that looks if ip= contains BOOTIF needs to not run if ip = (dhcp|all|on) [19:38] s/all/any/ [19:39] ah ok. [19:39] infinity: which is not seeded and purges without issue in a VM, and is by itself just a recommends of perl -- WTH [19:39] cyphermox: want me to punt it back to you? [19:40] chiluk: well if you want to do it, feel free, or I can fix it myself [19:40] it needs a fix in both xenial and yakkety [19:40] cyphermox you would likely end up sponsoring the upload anyways. [19:41] well, you could get one more sponsored upload I can vouch for ;) [19:41] I don't mind either way [19:41] alrighty.. I'll do the legwork then. [19:41] I really could use the uploads .. [19:46] it might take me a little longer than you to fully wrap my head around what this awesome bash script is doing.. but this is a good opportunity to spread the expertise. === Guest99384 is now known as Zic === wendar_ is now known as wendar [20:04] pitti: cool :) === giraffe is now known as Guest34241 [20:07] cyphermox, around ? [20:07] pokign aroudn with open-iscsi and such. [20:08] i launch a vm with a nick on a lxdbr0 and then run [20:08] http://paste.ubuntu.com/23290520/ === beisner- is now known as beisner [20:13] smoser: seems to me like a badly configured dhcp server? [20:15] is it expected that dhclient use 'ip' from busybox ? [20:15] yes, that's fine [20:17] well, dhcp server is configured by lxd [20:17] http://paste.ubuntu.com/23290551/ [20:17] and stgraber says it should work [20:18] are there free IPs? [20:18] what packets go through? [20:18] tbh there is nowhere near enough information there to know what is wrong, but it's not likely because of which /bin/ip we use === sunweave1 is now known as sunweaver [20:20] i'm trying to do ipv6 dhcp [20:20] so probably [20:20] and there are 2 hosts on that network [20:21] smoser: well, we do exactly that to deploy maas instances in an ipv6 only network and it works [20:21] lamont: ideas? [20:22] cyphermox, well, "works" is a bit of a strong word. :) [20:22] it *works* [20:22] lamont has been strugging with it for quite a while [20:22] yeah, there's various pieces into that, not just dhcp. [20:22] smoser: initramfs-works. [20:22] shutdown later doesn't === arlen_ is now known as arlen === xnox_ is now known as xnox [20:23] in that it shuts down the interface whjere the root disk lives === Darcy is now known as Spydar007 [20:23] which could ultimately be an initramfs-tools bug, but it's not a configuration bug === balkamos_ is now known as balkamos [20:24] having said that, bridges and ivp6 address diescovery have their moments... since they "optimized" the bridge code to not forward multicast packets taht weren't "needed", and having that on has caused great pain with ipv6 address discovery for me. [20:25] smoser: packet traces are a very good start for figuring out what is happening with dhcp [20:26] http://paste.ubuntu.com/23290593/ [20:27] smoser: . /lib/functions (or such) and run configure_networking() ? [20:28] yeah, i can try. [20:28] so fyi, i run: dhclient -6 -d -v eth0 [20:28] it errors like in [20:28] http://paste.ubuntu.com/23290520/ [20:28] second time, it actually tries to do something [20:28] but still fails [20:29] so it surely seems to me that the 'ip' that is causing garbage was bringing the interface up [20:29] which seems important. [20:31] smoser: the code that actually runs in the initramfs is: . /scripts/functions; configure_networking [20:31] and that code works [20:32] well, that fails the same way for me. [20:32] and what do the packet traces look like? [20:32] message status code NoAddrsAvail: "no addresses available" [20:32] stgraber, ^ [20:32] "NoAddrsAvail:" ? [20:32] tcpdump -ni any por5 547 [20:33] only without the typo [21:08] cyphermox: bah, your initramfs-tools update for xenial broke some of my PXE boot related tooling :P still digging into the details, not yet sure if it's fragile assumptions on my end or a bug on your end... will let you know [21:08] but the symptom is a kernel panic when trying to PXE boot [21:11] jderose: there is a bug [21:11] I bet your PXE deployments pass ip=dhcp [21:11] cyphermox: have the # or link handy? [21:11] cyphermox: yes, indeed it does === kitterma is now known as ScottK [21:12] https://bugs.launchpad.net/ubuntu/+source/initramfs-tools/+bug/1631474 [21:12] Launchpad bug 1631474 in initramfs-tools (Ubuntu) "No networking with initramfs-tools 0.122ubuntu8.3 and ip=dhcp boot option" [Undecided,In progress] [21:12] entirely my fault, I derped there adding the logic to parse BOOTIF. [21:12] chiluk is working on the fix and I'll sponsor it as soon as it's ready [21:13] cyphermox: happens to everyone, no worries :) [21:13] it's kind of crippling ;) [21:13] yeah I'm finding some other issues as well while I look through the code. [21:13] cyphermox: also, if you ever have a proposed package with PXE boot related changes... i'm always happy to test, feel free to ping me anytime :) [21:14] jderose can you add your /proc/cmdline to the bug and paste it here as well [21:14] so I know how people are using this. [21:14] thanks. [21:14] I'm trying to make sure all cases get covered. [21:14] chiluk: yup, will do [21:17] chiluk: done [21:17] cyphermox: for example if ! echo "${IP}" | grep -qc 'BOOTIF'; then will always return true regardless [21:28] jderose: would you be willing to test an updated initramfs-tools for me shortly? [21:32] chiluk: yup, absolutely! prefer a ppa, but i can manage a deb too, will just take a bit more hacking around (my tooling doesn't make debs especially easy) [21:32] sure I'll create a ppa .. [21:32] one sec.. I'm trying to test it myself and the deb is failing to install .. [21:32] which is strange. [21:32] chiluk: awesome, easy peasy then [21:36] chiluk: what? [21:36] chiluk: oh, for completeness here and not just in the bug, my current cmdline is something like: initrd=initrd.img-4.4.0-36-generic root=/dev/nfs nfsroot=10.17.76.1:/var/lib/tribble/ephemeral ip=dhcp ro nomodeset net.ifnames=0 [21:37] cyphermox: the IP variable should never contain BOOTIF [21:37] so the grep will always fail so the if will always be true [21:37] that's not true [21:38] the IP variable totally can and does contain BOOTIF sometimes [21:38] BOOTIF == whatever interface teh system used to boot with [21:39] cyphermox does dash support ;& fall-through syntax? [21:39] I do not know [21:40] it looks like a no.. which pisses me off [21:40] why aren't we just using bash.. [21:41] chiluk: not sure if it will be helpful, but i rather recently discovered https://www.shellcheck.net/ and it's kinda badass :) [21:43] chiluk: http://paste.ubuntu.com/23290876/ [21:43] should be sufficient afaik [21:44] crap cyphermox that's finding a world of bugs on just this one area. [21:44] oh, looking for on should be strict, in case we ever have devices that start in on ;) [21:45] I'm going to have to come back later, I have a dinner planned tonight [21:46] cyphermox: enjoy, thanks for the quick response! [21:46] ok cyphermox... I'll put together a patch and ppa and get that out to folks as soon as I'm happy === ochosi_ is now known as ochosi [22:02] my kingdom for full bash interpretter instead of fricking dash. [22:02] I'm trying to make things pretty here. [22:18] infinity: Kubuntu would like to get a security fix in before release: https://anonscm.debian.org/git/pkg-kde/frameworks/kcoreaddons.git/commit/?id=ab7258dd8a87668ba63c585a69f41f291254aa43 [22:19] ScottK: Security fixes welcome. [22:19] K. Thanks. === bdmurray_ is now known as bdmurray