[13:19] <uppock> Does cloud init support setting up ipv6 via dhcp6. Im having problems getting the correct network config in a ubuntu intance on openstack
[14:11] <jgrimm> thanks rbasak
[14:20] <smoser> uppock, if you use config drive on openstack and it provides network_data.json then it should work.
[14:21] <smoser> however, if you  just want to do "dhcp v6 on the first nic", then there is not currently a way to do that without modifying an image.
[14:21] <smoser> i recently just did this in bug
[14:21] <smoser>  
[14:21] <smoser>  https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1657940/comments/4
[14:22] <smoser> see the 'sudo tee' line there. thats how you can configure the system to do it for a spicific NIC by mac. i am pretty sure if you know the name the nic will get you can drop the mac
[14:49] <uppock> Oh
[14:51] <uppock> I do have network_data.json via the metadata server, but that does not help?
[14:58] <smoser> uppock, currently no it does not.
[14:58] <smoser> that is a change that can be done, but its not working now.
[15:02] <uppock> smoser: ok
[15:16] <uppock> thanks for the help
[16:40] <smoser> rharper, https://code.launchpad.net/~wesley-wiedenmeier/cloud-init/+git/cloud-init/+merge/317589
[16:40] <smoser> so...
[16:40] <smoser> NETWORK_CONFIG_V2  is a bad name i think
[16:40] <smoser> NETPLAN_PASSTHROUGH ?
[16:40] <smoser> no. its not passthrough.
[16:40] <smoser> what shoudl we call that.
[16:57] <smoser> rharper, also i had https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/319506 for you
[16:58] <smoser> which is terribly rendered there because i rebased to master
[16:58] <rharper> smoser: I don't have a better suggestion;  I had talked with magicalChicken about having maybe a list of network renderers; as well as paththrough  features
[17:00] <rharper> smoser: oh, the ntp test
[17:01] <smoser> hm.. i guess the name isnt terrible in itsel.f
[17:01] <smoser> as i think cloud-init is basically stating that it can render
[17:01] <smoser>  http://paste.ubuntu.com/24152513/
[17:01] <smoser> which is "network config v2"
[17:02] <smoser> an interesting thing though is that we cannot render the sysconfig for that yet, right?
[17:02] <rharper> not yet
[17:02] <smoser> we have a generic v2-to-v1 ?
[17:02] <rharper> yes
[17:02] <smoser> so we could v2-to-v1-to-sysconfig
[17:03] <smoser> outside of features not supported.
[17:03] <rharper> yes, v2 to network-state, and network-state to *whatever*
[17:03] <smoser> right
[17:03] <rharper> this branch adds a v2 parser -> netstate;  and then a state -> netplan
[17:04] <rharper> we have a v2 -> output skipping state
[17:04] <rharper> aka, 'passthrough'
[17:10] <powersj> smoser: test regression wrt chpasswd last night LP: #1671883
[17:10] <powersj> https://bugs.launchpad.net/cloud-init/+bug/1671883
[17:11] <smoser> powersj, should be fixed in trunk
[17:12] <powersj> smoser: oh look at that :)
[17:12] <powersj> I'll launch the tests again then. thanks!
[17:13] <smoser> yeah, horay for that integration test.
[17:13] <smoser> rharper, https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/319605
[17:14] <smoser> magicalChicken, ^
[17:14] <smoser> that is simplified and more-inert version of https://code.launchpad.net/~wesley-wiedenmeier/cloud-init/+git/cloud-init/+merge/317589
[17:31] <magicalChicken> smoser: calling it NETWORK_CONFIG_V1 might be confusing though, as there are versions of cloud-init with network config support but without the feature flags
[17:31] <powersj> smoser: https://paste.ubuntu.com/24152754/
[17:31] <smoser> "it" ?
[17:31] <rharper> smoser: would we fill that out with other input formats we support?  I sorta liked the idea of specifying the list of inputs and outputs supported
[17:31] <magicalChicken> smoser: sorry, the feature for network support
[17:32] <smoser> magicalChicken, there, i just took your suggestion, but currently if i merged this into trunk, we do not support NETWORK_CONFIG_V2
[17:32] <smoser> but we *do* support NETWORK_CONFIG_V1
[17:32] <smoser> so i listed that support as a feature
[17:32] <magicalChicken> smoser: oh, right that makes sense
[17:32] <magicalChicken> yeah, it may be good t fill out that list
[17:33] <magicalChicken> the mp i had filed had been into ~raharper/cloud-init:netconfig-v2-passthrough, once that branch lands in trunk it will support v2 though
[17:34] <smoser> magicalChicken, right.l but i was just grabbing the "add feature list"
[17:34] <magicalChicken> yeah, that makes sense
[17:34] <smoser> rharper, i dont know what to do there..
[17:35] <smoser> wrt listing "sysconfig" or "ifupdown"
[17:35] <rharper> right; those might be per distro
[17:35] <smoser> i think if we follow curtin's lead, we end up with:
[17:35] <magicalChicken> i like the features output command
[17:35] <smoser>  NETWORK_CONFIG_V1_IFUPDOWN
[17:35] <rharper> it could be a KEY into a list of distros that support those renderes
[17:35] <smoser> and
[17:36] <smoser>  NETWORK_CONFIG_V1_SYSCONFIG
[17:36] <smoser> unfortunately, that then possibly leads to
[17:36] <smoser>  NETWORK_CONFIG_V1_SYSCONFIG_WITH_BONDS
[17:36] <smoser> or something like that
[17:36] <magicalChicken> rharper: i think we had discussed this before, the per-distro features vs global features
[17:36] <rharper> RENDERERS = 'eni, sysconfig, netplan, netcfg';   distro_renderers = { 'Ubuntu': ['eni', 'netplan'], 'Fedora': ['sysconfig'] }
[17:36] <rharper> yeah
[17:42] <smoser> powersj, [embarrased] how do i run integration test for specific test ?
[17:42] <smoser> rharper, so i dont want to over complicate things.
[17:43] <powersj> As an example: python3 -m tests.cloud_tests run -v -n xenial -t tests/cloud_tests/configs/modules/set_password.yaml --deb cloud-init_daily_all.deb
[17:43] <smoser> i dont think t here is any reason to query cloud-init and say "can you render Fedora/sysconfig"
[17:43] <rharper> smoser: agreed; certainly for now, a flag or two makes sense
[17:43] <smoser> i can see a reason to say "if i gave you v2, can you render that for *THIS SYSTEM*"
[17:43] <rharper> we may want something more complex later
[17:44] <rharper> smoser: from curtin perspective, it will know what it has (v1 or v2)  and it will want to know if cloud-init can render it , and it *will* know what the target os is
[17:44] <rharper> so, I do think it's cloud-init, can you render v2 -> fedora/sysconfig
[17:45] <rharper> because, they may want to see if we can v2 -> fedora/networkd
[17:45] <rharper> I do think it's  os + networkcfg format
[17:45] <smoser> powersj, what am i doing wrong
[17:45] <smoser> http://paste.ubuntu.com/24152788/
[17:47] <powersj> smoser: what version of pylxd do you have? If I recall 2.1.3 is still the one that works best until we get magicalChicken's branch added
[17:47] <smoser> 2.2.3-0ubuntu1
[17:48] <magicalChicken> smoser: you need to run from the current branch then
[17:48] <magicalChicken> smoser: the old cloud_tests can't handle 2.2
[17:49] <magicalChicken> wait, if 2.2.3 is out that means that the fix for the leaked file handle bug is released, so I can go ahead and update the tox env in the new version
[17:49] <magicalChicken> that means we should be able to run 2.2.3 on jenkins
[17:49] <powersj> yep looks like that just got released!
[17:49] <magicalChicken> nice
[17:51] <powersj> getting that + the tox commands would make smoser's life easier too
[17:51] <smoser> yeah, do you have a example tox env ?
[17:52] <magicalChicken> smoser: 1 sec
[17:52] <magicalChicken> http://paste.ubuntu.com/24152810/
[17:52] <magicalChicken> smoser: ^
[17:53] <magicalChicken> powersj: i am going to be able to work almost full time next week, so i may go ahead and add support for distro and platform feature flags and write some of the kvm platform
[17:54] <magicalChicken> powersj: i have been talking to jgrimm about getting centos tests running smoothly so we can start integration testing there too
[17:54] <powersj> magicalChicken: no beach vacation for spring break? ;)
[17:54] <magicalChicken> powersj: haha no, probably not :)
[17:55] <powersj> magicalChicken: sounds great -- let me know what you start on. I can jump between bugs and the kvm stuff
[17:55] <magicalChicken> powersj: sure, sounds good
[17:55] <magicalChicken> i'll get all thise done in a new branch so that mp doesn't get any bigger
[17:55] <jgrimm> thanks magicalChicken
[17:56] <magicalChicken> jgrimm: no problem :)
[17:56] <smoser> powersj, was that the test that failed ? tests/cloud_tests/configs/modules/set_password.yaml
[17:56] <powersj> smoser: yes
[17:56] <smoser> ok.
[17:58] <smoser> magicalChicken, can you just put a MP for the citest toxenv right now ?
[17:58] <smoser> that is very useful on its own.
[17:59] <magicalChicken> smoser: sure i'll pull that out
[18:00] <magicalChicken> smoser: if it would make it easier to review the main one, i can split it up into a few smaller ones. they'll all still be needed to get everything working but it may make the diff easier to read
[18:02] <powersj> magicalChicken: if there are some things that we could divide out it probably would be good to do anyway
[18:03] <magicalChicken> powersj: part of the problem is that if i divide things out, the test suite as a whole won't work until almost all of it is merged in
[18:03] <smoser> magicalChicken, patch series are easier, yeah.
[18:04] <smoser> you can just work it as a patch series...
[18:04] <smoser> in a single mp, and rebase ... commit --fixup.... like that.
[18:04] <magicalChicken> smoser: the commit history is already pretty heavily modified, there aren't really any junk commits left
[18:05] <magicalChicken> i can do it a bit further though
[18:05] <smoser> ok.
[18:05] <smoser> if you want, i can just do the tox.ini change for trunk right now.
[18:06] <smoser> powersj, that wont cause harm to antying in c-i right ?
[18:06] <smoser> if i aadded 'citest' tox
[18:06] <magicalChicken> smoser: that should be fine
[18:06] <powersj> smoser: correct that is fine
[18:06]  * powersj greatly wants that change too
[18:06] <magicalChicken> smoser: it would have to just be the pylxd 2.1.3 env for now though ,the 2.2.3 env would break
[18:07] <smoser> ok
[18:11] <smoser> magicalChicken, powersj
[18:11] <smoser> http://paste.ubuntu.com/24152915/
[18:12] <smoser> that look sane ? i'll push that now.
[18:12] <smoser> w
[18:12] <magicalChicken> smoser: yeah, that looks good, thanks
[18:12] <powersj> +1
[18:12] <smoser> pushed.
[18:13] <magicalChicken> at some point we should probably update the docs again, but there's enough stuff changing for now that it can probably wait
[18:15] <smoser> and powersj trunk should hopefully pass again
[18:16] <powersj> smoser: thank you
[18:16] <smoser> rharper, you want to ack https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/319605 ? and i'll pull it
[18:19] <jgrimm> magicalChicken, fwiw i generally like to see doc changes accompany said code/test changes.
[18:19] <jgrimm> but please open a bug as reminder if known gaps
[18:20] <magicalChicken> jgrimm: sure, i'll open a bug to track integration-testing doc updates
[18:20] <jgrimm> thanks
[18:20] <magicalChicken> i'll see if i can get to some of those changes next week
[18:20] <jgrimm> +1 thanks
[18:39] <rharper> smoser: done
[18:48] <smoser> powersj, i kicked https://jenkins.ubuntu.com/server/job/cloud-init-integration-z/ and it ran successful
[18:48] <powersj> smoser: good, I had gone into torkoal after I saw the failure and blew away the offending lxc image
[18:49] <powersj> that means tot is fixed as well :) \o/
[18:49] <smoser> ah. i didnt realize there had been lxc issues
[18:49] <powersj> "Failed to create ZFS filesystem: cannot mount '/var/lib/lxd/images/(uuid of image)"
[18:50] <powersj> this is the issue cpaelzer and I keep hitting on the test systems
[18:51] <powersj> I am trying to reproduce locally, but waiting for new daily images as I think that is when it happens
[19:33] <cpaelzer> ?
[19:33] <cpaelzer> powersj: so we hit it again?
[19:33] <powersj> cpaelzer: on torkoal yeah :(
[19:33] <cpaelzer> without touching LXD versions
[19:33] <powersj> correct
[19:33] <cpaelzer> just as I promised
[19:34] <cpaelzer> are you giving stgraber access again?
[19:34] <cpaelzer> probably the best debug chance
[19:34] <powersj> well, I already blew the failed image away
[19:34] <cpaelzer> did that make it working?
[19:34] <powersj> yes
[19:34] <cpaelzer> e.g. after sync of the same image it is ok
[19:34] <cpaelzer> hmm
[19:34] <cpaelzer> interesting
[19:35] <cpaelzer> but that is a great indicator where stgraber can check next time it happens
[19:35] <powersj> yeah
[19:35] <cpaelzer> powersj: did yu update the github issue I filed`
[19:35] <cpaelzer> ?
[19:35] <powersj> no, but I can
[19:35] <cpaelzer> thanks
[19:35] <cpaelzer> uh, I just got suggested that I#M in EOD (EOW) for a while ...
[19:36]  * cpaelzer hushing back to family
[19:36] <powersj> :D
[19:40] <powersj> magicalChicken: for integration tests, the only system requirements right now are python3, lxd, and pylxd==2.1.3?
[19:41] <magicalChicken> powersj: as far as i know yeah
[19:42] <powersj> Updating the docs with how to run via tox + putting small section on dependencies
[19:42] <magicalChicken> powersj: that's all that is set up in the tox env and that works
[19:42] <magicalChicken> powersj: oh, nice
[19:42] <powersj> magicalChicken: https://paste.ubuntu.com/24153367/
[19:42] <magicalChicken> i'll pull those changes in with the pylxd 2.2 stuff and update them for that as well
[19:43] <magicalChicken> powersj: yeah, that looks good
[19:43] <powersj> ok
[19:43] <powersj> thx!
[19:43] <magicalChicken> powersj: it may be worth changing the full path to the test config to just moduules/user_groups
[19:44] <powersj> I don't need the "tests/cloud_tests/configs" part?
[19:44] <magicalChicken> it doesn't really make a difference, but i find it easier to read that way
[19:44] <magicalChicken> powersj: no, the argument normalizer will handle that for you
[19:44] <powersj> O.o didn't know this
[19:45] <magicalChicken> i built in relative path -> absolute path stuff pretty much everywhere because i'm lazy typing out commands :)
[19:45] <powersj> let me go try this then
[19:48] <smoser> rharper, http://paste.ubuntu.com/24153396/
[19:50] <rharper> ok
[19:50] <rharper> smoser: re the virtual stuff; that's just super annoying
[19:51] <powersj> magicalChicken: very nice
[19:51] <rharper> I don't think rename in lxc is an issue, no? it's created with the name it needs
[19:51] <rharper> smoser: so not sure why that matters
[19:53] <magicalChicken> powersj: :) the code that handles args in the new version is kinda cool
[19:53] <smoser> well, definitely one way to interact with lxc is to request it to name your network devices.
[19:54] <smoser> but another is to tell cloud-init via network config.
[19:55] <rharper> smoser: I updated my comment in the bug; we need to somehow detect duplicate macs and sort out the *real* interface that should be renamed
[19:56] <smoser> yeah.
[19:56] <rharper> smoser: that's for the review, replying to it is somewhat awkward
[19:56] <smoser> agree
[19:57] <smoser> why did i do it like that..
[19:57] <smoser> because it wasnt submitted for merge
[19:57] <smoser> right ?
[19:57] <smoser> https://code.launchpad.net/%7Ecurtin-dev/curtin/trunk/+activereviews
[19:58] <smoser> bah. well it woudlt be there
[19:58] <smoser> for some reason i thought you hadn't clicked submit
[19:58] <smoser> i can replay my comments onto the mp
[19:58] <smoser> https://code.launchpad.net/~raharper/cloud-init/+git/cloud-init/+merge/319259
[20:07] <smoser> rharper, updated. https://code.launchpad.net/~raharper/cloud-init/+git/cloud-init/+merge/319259
[20:14] <rharper> smoser: I sent a mail reply
[20:22] <dtp> hello.  cloud-init-0.7.8-5.fc25 is failing to write net config for me on Fedora 25.  error is [ValueError: Unknown subnet type 'loopback' found for interface 'lo'] at net/sysconfig.py:238
[20:23] <harlowja> larsks have u seen anything like that ( dtp was poking me - also works at godaddy, but not sure if u've already fixed that, lol )
[20:24] <larsks> harlowja: I haven't seen that previously, but 'loopback' is clearly not one of the supported types in NetInterface.iface_types...
[20:24] <harlowja> ya, lol
[20:24] <larsks> ...but why are they passing in configuration for lo?
[20:24] <larsks> That sounds goofy.
[20:25] <harlowja> agreed
[20:25] <dtp> my content/0000 looks like ubuntu w/ an [iface lo inet loopback] line
[20:26] <dtp> http://imgur.com/a/qEElh
[20:26] <rharper> isn't the content/XXXX path where a complete eni  is being injected rather than using network_data.json ?
[20:27] <dtp> added network_data.json image to that link, too
[20:28] <dtp> i don't see a loopback in network_data.json; so must be coming from content/0000?
[20:28] <rharper> I need to review the code path, but I thought if we got a content eni file, we'd just pass that through; but maybe we're attempting to inject the eni into network_State and then re-render something else (like sysconfig)
[20:29] <rharper> which sounds like what's happening
[20:29]  * harlowja didn't think we did that (but maybe we do, ha)
[20:29] <rharper> well
[20:29] <dtp> added most of the stack trace to that link
[20:29] <rharper> but if openstack sends a network_data.json, I'd think we prefer that to injesting eni into network_state
[20:30] <larsks> rharper: and yet, the network_data.json doesn't container any reference to lo...
[20:30] <rharper> it shouldn't
[20:30] <larsks> well, of course not.
[20:31] <larsks> So the exception dtp  is seeing suggests that we *don't* prefer network_data.json.
[20:31] <rharper> yeah
[20:31] <rharper> I wonder why they send both
[20:36] <larsks> rharper: huh, looking at the code, it seems as if network_json should take precedence: https://github.com/cloud-init/cloud-init/blob/master/cloudinit/sources/DataSourceConfigDrive.py#L142
[20:36] <larsks> dtp: do you see any of those message in your logs (on lines 145 or 150)?
[20:37] <dtp> not that i recall larsks, but i will dbl check now
[20:38] <dtp> not that i can see
[20:39] <larsks> Hmm. I need to run off, but if you haven't already, open a bug so that we don't forget (including details of cloudinit version, distro and version, cloud provider, etc)
[20:39] <dtp> ok
[20:51] <dtp> https://bugs.launchpad.net/cloud-init/+bug/1671927
[20:54] <rharper> larsks: not quite;  the code I see checks if the we have a self._network_config; if it's None, then we check if we have a network_json and convert;  so the eni file mapped in the metadata content will get used first
[20:55] <larsks> rharper: ah, good catch.
[20:56] <rharper> larsks: but I think we're meant to
[20:56] <rharper> given the network_eni
[20:59] <rharper> in distro/rhel.py we do this;  so I suspect we're getting the eni passed in here;  entries = net_util.translate_network(settings)
[21:17] <rharper> something funny is going on w.r.t the type of network config that was found and passed in;  more investigation needed;
[21:18] <rharper> dtp-afk: if you can, attaching the configdrive to the bug would be super helpful
[21:20] <smoser> rharper, sorry for the noise above with the patch comments... can you re-play on the MP ?
[21:20] <rharper> yes
[21:20] <smoser> i'ms orry i just didnt' think you'd proposed for merge
[21:20] <smoser> :-(
[21:20] <rharper> no worries
[21:21]  * rharper forklifts into web form
[21:21]  * rharper mumbles something about gerrit 
[22:37] <dtp> rharper you want the directory listing or ?
[22:52] <dtp> uploaded 0000 and meta_data.json