=== shardy is now known as shardy_mtg === shardy_mtg is now known as shardy [13:57] hm... https://jenkins.ubuntu.com/server/job/cloud-init-integration-a/91/ [14:38] rharper, [14:39] http://paste.ubuntu.com/25133128/ [14:39] yes [14:40] i'm going to put together a "simple" bond, vlan, and bridge config [14:40] and add tests for sysconfig rendering of them. [14:40] that bond shows stacktrace in current trunk and seems to render sanely with your [14:40] "Fix sysconfig rendering of virtual interfaces"... [14:41] yes [14:41] the last commit in my branch fixes an issue when you have a mac_address property on the bond, vlan or bridge [14:41] but that's easy enough to fix up when we merge that commit [14:41] I sent you a unittest [14:42] that covered bond and vlan [14:42] but not exhaustively [14:42] ah. i see yeah, you render the 'all' [14:42] we could add a bridge device into that config for the oen I posted, and it would cover all 3 virtual types [14:42] that's in the final commit [14:42] but something simpler for the current patch, http://paste.ubuntu.com/25128675/ [14:42] is there [14:42] which is just a combination.. i think having individual ones would be good [14:43] sure [14:43] oh. [14:43] ididt see .. you did that pate yesterday? [14:43] thats great. [14:44] your paste there doesnt have a 'route' on the subnets... so it doesnt excercise that code that you fixed [14:45] i'm sorry i missed your patch [14:45] there are two paths [14:45] we can add one with subnets (single subnet) and one with subnets with routes [14:46] cls._render_subnets(iface_cfg, iface_subnets) is for the subnets, and cls._render_subnet_routes(iface_cfg, route_cfg, iface_subnets) for any 'routes' keys under a subnet [14:46] hm.. [14:47] we can just add a 'routes' list if you want to hit that path as well [14:47] so 'routes' can be a sibling of subnets ? [14:47] so, we have static routes (top level type) [14:47] and then we can associate a list of routes for each subnet [14:47] which is what 'routes' is, which models network_config.json from openstack [14:50] http://paste.ubuntu.com/25133177/ ? [14:50] (that doesnt seem to work) [14:51] sorry, type: routes top level [14:51] or key under a subnet [14:51] subnets: [{type: static, routes: [list of routes]}] [14:52] config: [{type: route, dest: X, gateway: y}] [14:52] you want the former [14:52] drop lines 29 to end [15:13] larsks, i have a question you probalby know the answer too [15:14] our rpms build and get a .fc25 or .el6 [15:15] on fedora do they re-build every time ? [15:15] every release... ie, if i upgrade do i get 100% new packages ? [15:20] smoser: packages are rebuilt for every release, yes. [15:21] But if there's no change in version/release, then for an *upgrade* I guess you might not get a new package. [15:21] Just a sec, let me check. [15:24] #fedora confirms that during an *upgrade* packages won't be replaced if there isn't an actual change in the package version. [15:30] But honestly, #fedora seems uncertain. Shouldn't matter in either case, though: if the package version and release are the same there should not be a difference between the two, regardless of the .dist tag. [15:32] larsks, hm. [15:32] but how does it know that they would differ or not ? [15:32] I mean, that's what the package version and release are for. [15:32] as they could differ in build-time logic [15:32] ie, our cloud-init truunk will build an rpm on copr for fc25 and el6 [15:33] and they'll differ [15:33] If you think your build environment is generating different code, you increase the package release. [15:33] but you do that by design... you look at rpm build macros and such [15:33] You'll often see exactly this in the changelog for packages: "Rebuilt for https://fedoraproject.org/wiki/Fedora_24_Mass_Rebuild" [15:33] For example. [15:33] In that case, you increment the release on the rpm even though you're not making any changes to the rpm content itself. [15:34] ok.. so heres an explicit example. [15:34] we build the trunk rpm in copr. [15:34] it spits out a .el6 rpm and a .el7 rpm [15:34] the .el6 one has upstart files and adds upstart jobs on installation [15:35] the .el7 one has systemd and sets up systemd on installation [15:35] but they have the same version [15:35] i'm wondering how we would handle a user upgrading from 6 to 7 [15:35] it seems they'd *have* to get the .el7 version [15:36] but if they did that'd mean that an upgrade of 6 to 7 is 100% of packages even if the package *didn't* change. [15:36] I will say that for EL, major version upgrades aren't something in which I have a lot of confidence. [15:36] * smoser tweets ;) [15:36] Well, for example, there is no "live" upgrade from 6->7. [15:36] larsks, thats fine. its quite likely i'm just thinking too much. [15:37] Whereas fedora has e.g. the "fedup" tool for that. [15:37] For the most part, we treat el6 and el7 as completely separate entities. [15:37] i just assumed one could upgrade 6 to 7 or 5 to 6 to 7 [15:37] that is helpful, thank you. [15:38] I'll see if I can set up a test just for kicks sometime this week. Not next week, though: I'll be on vacation :) [15:49] smoser: hey, take a look at https://wiki.centos.org/TipsAndTricks/CentOSUpgradeTool [15:49] "Warning: use of this tool is currently BROKEN as several system-critical packages are of a higher version number in CentOS 6.7 than they are in CentOS 7 so those do not get upgraded correctly." [15:49] :) [15:49] rharper, so on your branch [15:50] TestSysConfigRendering [15:50] y [15:50] er... how is TestSysconfigRoundTrip different from TestSysConfigRendering [15:52] it differs in input and how it's called; I think the latter used an eni file as input to convert and then render a sysconfig [15:54] ok. we lost the meaning of RoundTrip along the way [15:54] you're right [15:54] the 'round trip' of eni's test was that it started with an ENI, read that, generated a network config, and then rendered that [15:54] it's just doing it one=way [15:54] yeah [15:54] I suspect we can just remove the RoundTrip part for now [15:55] yeah. and just merge it with the previous class. [15:55] I don't think we necessarity need to attempt to parse sysconfig dir back into network config [15:55] i'll adjust stuff. [15:55] right [15:55] ok [15:55] we dont have a sysconfig parser at the moment [15:55] we *did* have a eni parser [15:55] and that was why round trip made sense [16:02] y [17:36] rharper, take alook at https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/327821 [18:33] smoser: approved [18:35] rharper, i'm going to let https://jenkins.ubuntu.com/server/job/cloud-init-ci/62/console finish [18:35] before i push [18:35] ok [18:35] but then i have it ready to go [18:35] ok [18:46] ok unit testing branch is up https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/327827 (part 1 of aws dhclient support in init-local stage) [18:46] now rebasing the 2nd branch on this one [18:50] rharper, awesome. one of the tests i added exposed the bug you fixed in the next commit [18:50] \o/ [18:53] https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/327828 [19:01] smoser: here is the latest version of kvm backend: https://git.launchpad.net/~powersj/cloud-init/commit/?id=28e84d93c193d7db41f1cba3bf5ff7a0ea364fc7 [19:01] still stuck on mount-image-callback trying to run commands via '/bin/sh -c' [19:02] http://paste.ubuntu.com/25134551/ [19:04] rharper, onboot seems "global" to a device [19:04] right ? [19:04] per interface file [19:04] it's equivalent to auto $iface [19:04] right. [19:08] i think there is a bug there. [19:08] rharper, http://paste.ubuntu.com/25134578/ [19:09] I think it's a deficiency [19:09] eni has control per stanza [19:09] sysconfig does not [19:10] hrm, but the bug that is present is that you should see IPADDR0, 1, 2 3 [19:10] for each subnet [19:10] lemme see if later commits fix that [19:10] right. thats the bug i'm pointing at [19:10] we only have one addr [19:10] and rendering eni for that breaks [19:10] is 'type': 'manual' right ? [19:11] somethings not right; I had that fixed [19:12] I think it's type manual [19:12] it should be type: static, control: manual [19:13] there we go [19:13] that works [19:14] rharper, http://paste.ubuntu.com/25134607/ ? [19:14] yes [19:14] that renders [19:15] ONBOOT=yes [19:15] right, you have one interface that's control: auto [19:15] this is just a deficiency [19:15] there's no way to support mixed control [19:15] nah. [19:15] drop the second address [19:15] same thing [19:15] no [19:15] you get IPADDR1 [19:15] and NETMASK1 [19:15] http://paste.ubuntu.com/25134616/ [19:16] renders [19:16] http://paste.ubuntu.com/25134617/ [19:16] that's a bug [19:16] but different than the first [19:16] thats what i was saying :) [19:16] that is what i was saying all along [19:16] well, you changed [19:16] well, if you put 'type: manual' [19:16] then it paid attention [19:17] so thats why i tried that [19:17] not type: manual, control: manual [19:17] you have a subnet which is type: static, but you want control: manual (define but dont' up it) [19:17] that works fine; [19:18] for single subnet, with type: static, control:manual, ONBOOT should be set to what subnet.get('control') == 'manual' is [19:18] you can try to come up with something that actually does what you want [19:18] which is't not, which is the bug you've found [19:18] but i dont thikn you'll find it. [19:18] I'm not following [19:18] take one of my yaml, and make it into somethign that you think should render ONBOOT=no [19:18] and show me. [19:18] http://paste.ubuntu.com/25134632/ [19:19] that works (no bugs) [19:19] but the one you posted (single subnet control: manual: should set ONBOOT=no) and it doesn't (which is new bug) [19:19] you get ONBOOT=yes in your example [19:19] well, you could argue the first one I posted is bug too; I say it's a known definciency [19:19] because one of the two subnets is not control: manual [19:19] so youre saying please UP this subnet [19:20] but then we say, don't up the next one [19:20] and ONBOOT is a toggle [19:20] and if you drop the address, so you only have 1 thing... then you *still* get ONBOOT=yes [19:20] ie, sysconfig does not support ONBOOT per subnet (like eni does via iface [19:20] right [19:20] if you specify control: manual, it's missing the check on what control value is [19:21] for single subnet, we can get that right [19:21] which is the bug I think you originally found [19:21] well, we could revert to aliased interface files if we really wanted to support that [19:21] eth0 (ONBOOT=yes) eth0:1 (ONBOOT=no) [19:22] I suspect, they could extend ONBOOT to ONBOOT{$index} [19:22] like they've done else where [19:22] * rharper looks a scripts to see if that's so [19:22] doesn't look like it on el7 [19:25] rharper, you needed this [19:25] http://paste.ubuntu.com/25134666/ [19:25] your fix didn't change anything. because it was in fact checking subnet['type'] [19:26] well, it did something; so I need to find if we've a broken network config [19:26] huh, we have type: manual and control: manual [19:26] now I need to look at docs [19:28] it shoudl be control: manual [19:28] as it can be type: dhcp [19:28] control: manual [19:29] yes, it worked in eni because the it had both type: manual and control: manual [19:29] the subnet code ignored the type check, but had the control check like you have [19:29] I'll fix up the test-case in curtin to use type: static [19:29] but you're fix is still the right one for sysconfig [19:39] rharper, https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/327828 [19:40] it doesnt seem to want to update, but [19:40] http://paste.ubuntu.com/25134735/ [19:40] is the diff against my just-pushed upstream/master [19:41] ok, looking [19:41] then I'll rebase my branch again [19:44] urg [19:44] the last commit collides with your extra unittests in test_net [19:44] I was hoping to avoid that =( [19:45] no, something else; me fixes rebase [19:46] looks like indentation sytle changes [19:47] sysconfig: enable mtu set per subnet right ? [19:48] no, the last MAC vs HW_ADDR [19:56] almost done; fixed up the last commit's unittests (the ipv6 /64 change, and the new unittest you had for bonds needs MACADDR instead of HWADDR (which is what the last patch fixes) [19:56] ok, force pushed mine after a rebase on master [20:08] rharper, https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/327832 [20:08] that one is OK'd by c-i. you ack, i pull [20:09] ok, when will launchpad update with the correct diff [20:09] i dont know that it wil [20:09] just fetch and diff yourself :) [20:10] or click on the individual commits [20:11] i have the next one ready too [20:11] sysconfig: enable mtu set per subnet [20:11] with tests added [20:12] ok [20:13] we've a new bug in the network state: the mtu test in curtin registered a manual subnet with no address (AFAICT that's legit); we have network_state code that checks for 'address' in a subnet and blows up if 'address' key isn't present [20:13] the type: manual avoided that path since it was never looked at [20:13] http://paste.ubuntu.com/25134871/ [20:14] that's a one-liner if 'address' in subnet and ':' in subnet['address'] [20:16] currently sysconfig raises a valueerror [20:16] ValueError: No config network address keys [address] found in {'control': 'manual', 'type': 'static'} [20:18] actually, yeah. bot in my test raise a valueerror [20:18] saying its bogus [20:18] eni and sysconfig [20:18] hrm [20:18] let me try curtin.net [20:19] it's possible that I can just remove it altogether as I can't see what value it would add to the test-case at this time [20:19] http://paste.ubuntu.com/25134888/ [20:19] nope, fail in curtin.net as well; well, let's remove it and see what it looks like [20:20] yeah [20:20] oh, it was ordering iface stanzas [20:21] hrm, so how do we get an iface $name manual ? [20:21] and no ip configuration ? [20:23] rharper, https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/327832 [20:23] is now pushed with my fixes on yours squashed and laucnhpad re-rendered [20:24] no diff other than commit message versus what c-i ran at [20:24] https://jenkins.ubuntu.com/server/view/Cloud-init/job/cloud-init-ci/67/console [20:24] so i will pull if you say good. [20:24] looks good [20:24] and then i have 7 ready and i might just include 8 in with it i just straight cherry-picked it [20:24] ok [20:30] https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/327836 [20:32] rharper, so. ^ has 2 more of them. [20:32] which leaves us just aaa7101790d5 [20:33] Fix style issue with unused exeception variable in DataSourceAzure [20:33] which is not related [20:33] and i'm not sure what caught it. cause that isnt new [20:33] and then [20:33] right, it's not, but i wanted make style-check to pass [20:33] sysconfig: use MACADDR on bonds/bridges to configure mac_address [20:33] it fails here (xenial laptop) [20:35] ah. well, the answer is that we dont care. [20:36] run tox -e pyflakes [20:36] tox passes [20:36] am I missing an update? [20:36] surely we run a tox flakes under xenial ? [20:36] or is it style from $dev_release only ? [20:37] coding style is checked via running [20:37] tox -e flake8 [20:38] what does 'make style-check' run ? [20:38] that pins versions of of things to what we accept as current targets [20:38] all 'make' targets use whatever things are installed on the system. [20:38] is there a reason why it doesn't use tox ? [20:38] if that's the one-true-way ? [20:39] make is bascially "use what is on the system" [20:39] we run 'make test' during the ubuntu builds [20:39] at best it passes but most likely wrong; I'll submit a patch later; [20:39] so that we verify the test runs against system versions [20:39] right, but style-check could certainly do the tox -e flake8 instead [20:39] but we do not run style checks [20:39] right, which I'm perfectly fine with [20:40] I'm just out-dated on how to check style [20:40] if we have a partually useless make target, I'd like to fix it up [20:40] that's all [20:40] because other wise we are held to the least common denominator of styles [20:40] its not completely useless. [20:40] it passes for me [20:40] but it doesn't for me [20:40] you're on artful [20:40] (as artful has the same (or similar) versions that we *do* care about) [20:41] i'm not opposed to removing it... but it doesnt run when you just type 'make' [20:41] but I've got something else that does't like it ; anyhow, we can bikeshed in the MP (if I submit it) [20:41] and that is by design [20:41] that's fine [20:41] I don't want to change style-check to run anywher else [20:41] jsut make style-check should match what we say is the right style, which is the frozen set in tox -e flake8 [20:42] nothing in 'make' depends on the internet [20:42] thats why make doesn't run tox [20:42] so, sed 's,pep8 $(pyflakes),tox -e flake8' [20:42] nothing with plain 'make' depends on the style-check by design [20:42] so why not have the style-check actually check the style with the right versions ? [20:42] * rharper tables it for later [20:45] once https://jenkins.ubuntu.com/server/view/Cloud-init/job/cloud-init-ci/69/ finishes i'll pull [20:46] k [20:46] just about done with rebase of the last few [20:46] ok, pushed [20:50] ok... https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/327836 will pass c-i. i will pull it . that means we just have the newest commit on your branch to grab. [20:51] sysconfig: use MACADDR on bonds/bridges to configure mac_address [20:51] i'll take your tests that you added there and put them into the ones i added rather than adding the mis-named RoundTrip [20:53] you merged both mtu-per-subnet and ipv6 def-route ? [21:02] those are both in 7 [21:02] which i'll just pull now (bringing in the 2 individual commits that are there) [21:03] k [21:03] those are now there. [21:03] i'm woring on the last commit. [21:03] hora! [21:03] ack [21:04] no more rebase needed then [21:10] rharper, https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/327838 [21:10] i just cherry picked yours and then [21:10] https://git.launchpad.net/~smoser/cloud-init/commit/?id=64bd6b52a57cc0a7cc3a4c689958d0c7b2f4add6 [21:12] ack [21:12] approved [21:12] thanks! [21:12] * rharper ends smoser some beer [21:12] yeah, ... [21:12] so you dont need this in ubuntu [21:12] you just needed it in trunk for the moment [21:12] but i think i might just upload to ubuntu too [21:13] minor comment on https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/327838 [21:13] blackboxsw, ack. i'll grab that change. [21:13] there was at least one commit needed for eni rendering [21:13] +1 thx [21:13] * rharper looks for the list [21:14] i dont like using the .get() twice [21:14] (but we were doing it anwyay) [21:14] ie, i dont like: [21:14] if mydict.get('key'): [21:14] x = mydict.get('key') [21:14] smoser: https://bugs.launchpad.net/cloud-init/+bug/1701097 ,is needed back to Xenial [21:14] Ubuntu bug 1701097 in cloud-init (Ubuntu Artful) "eni rendering of ipv6 gateways fails" [Medium,Confirmed] [21:14] but oh well. [21:15] but only once we release (and sru) an curtin with passthrough enabled [21:16] we should see about getting those scheduled/started soon given our timeline and desire for MAAS testing [21:20] blackboxsw, https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/327827 looks very clean. thank you [21:20] nice commit messages. [21:21] good deal smoser, thanks [21:21] I'll get out from under this aws ipv6 in short order I hope [21:23] smoser: to confirm, how a datasource is determined to run in local mode is by the absense of .datasources sources.DEP_NETWORK in the deps listing? [21:25] right [21:43] ok. so https://jenkins.ubuntu.com/server/view/Cloud-init/job/cloud-init-ci/72/ should finish in 12 minute and then i can pull [21:43] i might get to that later tonight. [21:43] have a good one smoser [21:45] later [22:04] smoser: /me is wondering why we'd want to try searching/get_data for DataSourceEc2 in the network stage if we can run DataSourceEc2 in local [22:04] smoser: /me is wondering why we'd want to try searching/get_data for DataSourceEc2 in the network stage if we can run DataSourceEc2 in init-local [22:12] n/m I think I recall now. The reason attempting the datasource in both init-local and init-network stages would be to keep non-aws Ec2 clones running in the init-network stage so we don't impact their initialization w/ this changeset.