/srv/irclogs.ubuntu.com/2017/07/20/#cloud-init.txt

=== shardy is now known as shardy_mtg
=== shardy_mtg is now known as shardy
smoserhm... https://jenkins.ubuntu.com/server/job/cloud-init-integration-a/91/13:57
smoserrharper,14:38
smoserhttp://paste.ubuntu.com/25133128/14:39
rharperyes14:39
smoseri'm going to put together a "simple" bond, vlan, and bridge config14:40
smoserand add tests for sysconfig rendering of them.14:40
smoserthat bond shows stacktrace in current trunk and seems to render sanely with your14:40
smoser "Fix sysconfig rendering of virtual interfaces"...14:40
rharperyes14:41
rharperthe last commit in my branch fixes an issue when you have a mac_address property on the bond, vlan or bridge14:41
rharperbut that's easy enough to fix up when we merge that commit14:41
rharperI sent you a unittest14:41
rharperthat covered bond and vlan14:42
rharperbut not exhaustively14:42
smoserah. i see  yeah, you render the 'all'14:42
rharperwe could add a bridge device into that config for the oen I posted, and it would cover all 3 virtual types14:42
rharperthat's in the final commit14:42
rharperbut something simpler for the current patch, <rharper> http://paste.ubuntu.com/25128675/14:42
rharperis there14:42
smoserwhich is just a combination.. i think having individual ones would be good14:42
rharpersure14:43
smoseroh.14:43
smoserididt see .. you did that pate yesterday?14:43
smoserthats great.14:43
smoseryour paste there doesnt have a 'route' on the subnets... so it doesnt excercise that code that you fixed14:44
smoseri'm sorry i missed your patch14:45
rharperthere are two paths14:45
rharperwe can add one with subnets (single subnet) and one with subnets with routes14:45
rharpercls._render_subnets(iface_cfg, iface_subnets) is for the subnets, and cls._render_subnet_routes(iface_cfg, route_cfg, iface_subnets) for any 'routes' keys under a subnet14:46
smoserhm..14:46
rharperwe can just add a 'routes' list if you want to hit that path as well14:47
smoserso 'routes' can be a sibling of subnets ?14:47
rharperso, we have static routes (top level type)14:47
rharperand then we can associate a list of routes for each subnet14:47
rharperwhich is what 'routes' is, which models network_config.json  from openstack14:47
smoserhttp://paste.ubuntu.com/25133177/ ?14:50
smoser(that doesnt seem to work)14:50
rharpersorry, type: routes top level14:51
rharperor key under a subnet14:51
rharpersubnets: [{type: static, routes: [list of routes]}]14:51
rharperconfig: [{type: route, dest: X, gateway: y}]14:52
rharperyou want the former14:52
rharperdrop lines 29 to end14:52
smoserlarsks, i have a question you probalby know the answer too15:13
smoserour rpms build and get a .fc25 or .el615:14
smoseron fedora do they re-build every time ?15:15
smoserevery release... ie, if i upgrade do i get 100% new packages ?15:15
larskssmoser: packages are rebuilt for every release, yes.15:20
larsksBut if there's no change in version/release, then for an *upgrade*  I guess you might not get a new package.15:21
larsksJust a sec, let me check.15:21
larsks#fedora confirms that during an *upgrade* packages won't be replaced if there isn't an actual change in the package version.15:24
larsksBut honestly, #fedora seems uncertain.  Shouldn't matter in either case, though: if the package version and release are the same there should not be a difference between the two, regardless of the .dist tag.15:30
smoserlarsks, hm.15:32
smoserbut how does it know that they would differ or not ?15:32
larsksI mean, that's what the package version and release are for.15:32
smoseras they could differ in build-time logic15:32
smoserie, our cloud-init truunk will build an rpm on copr for fc25 and el615:32
smoserand they'll differ15:33
larsksIf you think your build environment is generating different code, you increase the package release.15:33
smoserbut you do that by design... you look at rpm build macros and such15:33
larsksYou'll often see exactly this in the changelog for packages: "Rebuilt for https://fedoraproject.org/wiki/Fedora_24_Mass_Rebuild"15:33
larsksFor example.15:33
larsksIn that case, you increment the release on the rpm even though you're not making any changes to the rpm content itself.15:33
smoserok.. so heres an explicit example.15:34
smoserwe build the trunk rpm in copr.15:34
smoserit spits out a .el6 rpm and a .el7 rpm15:34
smoserthe .el6 one has upstart files and adds upstart jobs on installation15:34
smoserthe .el7 one has systemd and sets up systemd on installation15:35
smoserbut they have the same version15:35
smoseri'm wondering how we would handle a user upgrading from 6 to 715:35
smoserit seems they'd *have* to get the .el7 version15:35
smoserbut if they did that'd mean that an upgrade of 6 to 7 is 100% of packages even if the package *didn't* change.15:36
larsksI will say that for EL, major version upgrades aren't something in which I have a lot of confidence.15:36
* smoser tweets ;)15:36
larsksWell, for example, there is no "live" upgrade from 6->7.15:36
smoserlarsks, thats fine. its quite likely i'm just thinking too much.15:36
larsksWhereas fedora has e.g. the "fedup" tool for that.15:37
larsksFor the most part, we treat el6 and el7 as completely separate entities.15:37
smoseri just assumed one could upgrade 6 to 7 or 5 to 6 to 715:37
smoserthat is helpful, thank you.15:37
larsksI'll see if I can set up a test just for kicks sometime this week.  Not next week, though: I'll be on vacation :)15:38
larskssmoser: hey, take a look at https://wiki.centos.org/TipsAndTricks/CentOSUpgradeTool15:49
larsks"Warning: use of this tool is currently BROKEN as several system-critical packages are of a higher version number in CentOS 6.7 than they are in CentOS 7 so those do not get upgraded correctly."15:49
larsks:)15:49
smoserrharper, so on your branch15:49
smoser TestSysConfigRendering15:50
rharpery15:50
smoserer... how is TestSysconfigRoundTrip different from TestSysConfigRendering15:50
rharperit differs in input and how it's called; I think the latter used an eni file as input to convert and then render a sysconfig15:52
smoserok. we lost the meaning of RoundTrip along the way15:54
rharperyou're right15:54
smoserthe 'round trip' of eni's test was that it started with an ENI, read that, generated a network config, and then rendered that15:54
rharperit's just doing it one=way15:54
smoseryeah15:54
rharperI suspect we can just remove the RoundTrip part for now15:54
smoseryeah. and just merge it with the previous class.15:55
rharperI don't think we necessarity need to attempt to parse sysconfig dir back into network config15:55
smoseri'll adjust stuff.15:55
smoserright15:55
rharperok15:55
smoserwe dont have a sysconfig parser at the moment15:55
smoserwe *did* have a eni parser15:55
smoserand that was why round trip made sense15:55
rharpery16:02
smoserrharper, take alook at https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/32782117:36
rharpersmoser: approved18:33
smoserrharper, i'm going to let https://jenkins.ubuntu.com/server/job/cloud-init-ci/62/console finish18:35
smoserbefore i push18:35
rharperok18:35
smoserbut then i have it ready to go18:35
rharperok18:35
blackboxswok unit testing branch is up https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/327827  (part 1 of aws dhclient support in init-local stage)18:46
blackboxswnow rebasing the 2nd branch on this one18:46
smoserrharper, awesome. one of the tests i added exposed the bug you fixed in the next commit18:50
rharper\o/18:50
smoserhttps://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/32782818:53
powersjsmoser: here is the latest version of kvm backend: https://git.launchpad.net/~powersj/cloud-init/commit/?id=28e84d93c193d7db41f1cba3bf5ff7a0ea364fc719:01
powersjstill stuck on mount-image-callback trying to run commands via '/bin/sh -c'19:01
powersjhttp://paste.ubuntu.com/25134551/19:02
smoserrharper, onboot seems "global" to a device19:04
smoserright ?19:04
rharperper interface file19:04
rharperit's equivalent to auto $iface19:04
smoserright.19:04
smoseri think there is a bug there.19:08
smoserrharper, http://paste.ubuntu.com/25134578/19:08
rharperI think it's a deficiency19:09
rharpereni has control per stanza19:09
rharpersysconfig does not19:09
rharperhrm, but the bug that is present is that you should see IPADDR0, 1, 2 319:10
rharperfor each subnet19:10
rharperlemme see if later commits fix that19:10
smoserright. thats the bug i'm pointing at19:10
smoserwe only have one addr19:10
smoserand rendering eni for that breaks19:10
smoseris 'type': 'manual' right ?19:10
rharpersomethings not right; I had that fixed19:11
rharperI think it's type manual19:12
rharperit should be type: static, control: manual19:12
rharperthere we go19:13
rharperthat works19:13
smoserrharper, http://paste.ubuntu.com/25134607/ ?19:14
rharperyes19:14
smoserthat renders19:14
smoser ONBOOT=yes19:15
rharperright, you have one interface that's control: auto19:15
rharperthis is just a deficiency19:15
rharperthere's no way to support mixed control19:15
smosernah.19:15
smoserdrop the second address19:15
smosersame thing19:15
rharperno19:15
rharperyou get IPADDR119:15
rharperand NETMASK119:15
smoserhttp://paste.ubuntu.com/25134616/19:15
smoserrenders19:16
smoser http://paste.ubuntu.com/25134617/19:16
rharperthat's a bug19:16
rharperbut different than the first19:16
smoserthats what i was saying :)19:16
smoserthat is what i was saying all along19:16
rharperwell, you changed19:16
smoserwell, if you put 'type: manual'19:16
smoserthen it paid attention19:16
smoserso thats why i tried that19:17
rharpernot type: manual, control: manual19:17
rharperyou have a subnet which is type: static, but you want control: manual (define but dont' up it)19:17
rharperthat works fine;19:17
rharperfor single subnet, with type: static, control:manual, ONBOOT should be set to what subnet.get('control') == 'manual' is19:18
smoseryou can try to come up with something that actually does what you want19:18
rharperwhich is't not, which is the bug you've found19:18
smoserbut i dont thikn you'll find it.19:18
rharperI'm not following19:18
smosertake one of my yaml, and make it into somethign that you think should render ONBOOT=no19:18
smoserand show me.19:18
rharperhttp://paste.ubuntu.com/25134632/19:18
rharperthat works (no bugs)19:19
rharperbut the one you posted (single subnet control: manual: should set ONBOOT=no) and it doesn't (which is new bug)19:19
smoseryou get ONBOOT=yes in your example19:19
rharperwell, you could argue the first one I posted is bug too; I say it's a known definciency19:19
rharperbecause one of the two subnets is not control: manual19:19
rharperso youre saying please UP this subnet19:19
rharperbut then we say, don't up the next one19:20
rharperand ONBOOT is a toggle19:20
smoserand if you drop the address, so you only have 1 thing... then you *still* get ONBOOT=yes19:20
rharperie, sysconfig does not support ONBOOT per subnet (like eni does via iface19:20
rharperright19:20
rharperif you specify control: manual, it's missing the check on what control value is19:20
rharperfor single subnet, we can get that right19:21
rharperwhich is the bug I think you originally found19:21
rharperwell, we could revert to aliased interface files if we really wanted to support that19:21
rharpereth0 (ONBOOT=yes) eth0:1 (ONBOOT=no)19:21
rharperI suspect, they could extend ONBOOT to ONBOOT{$index}19:22
rharperlike they've done else where19:22
* rharper looks a scripts to see if that's so 19:22
rharperdoesn't look like it on el719:22
smoserrharper, you needed this19:25
smoser  http://paste.ubuntu.com/25134666/19:25
smoseryour fix didn't change anything. because it was in fact checking subnet['type']19:25
rharperwell, it did something; so I need to find if we've a broken network config19:26
rharperhuh, we have type: manual and control: manual19:26
rharpernow I need to look at docs19:26
smoserit shoudl be control: manual19:28
smoseras it can be type: dhcp19:28
smosercontrol: manual19:28
rharperyes, it worked in eni because the it had both type: manual and control: manual19:29
rharperthe subnet code ignored the type check, but had the control check like you have19:29
rharperI'll fix up the test-case in curtin to use type: static19:29
rharperbut you're fix is still the right one for sysconfig19:29
smoserrharper, https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/32782819:39
smoserit doesnt seem to want to update, but19:40
smoser http://paste.ubuntu.com/25134735/19:40
smoseris the diff against my just-pushed upstream/master19:40
rharperok, looking19:41
rharperthen I'll rebase my branch again19:41
rharperurg19:44
rharperthe last commit collides with your extra unittests in test_net19:44
rharperI was hoping to avoid that =(19:44
rharperno, something else; me fixes rebase19:45
rharperlooks like indentation sytle changes19:46
smosersysconfig: enable mtu set per subnet right ?19:47
rharperno, the last MAC vs HW_ADDR19:48
rharperalmost done; fixed up the last commit's unittests (the ipv6 /64 change, and the new unittest you had for bonds needs MACADDR instead of HWADDR (which is what the last patch fixes)19:56
rharperok, force pushed mine after a rebase on master19:56
smoserrharper, https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/32783220:08
smoserthat one is OK'd by c-i. you ack, i pull20:08
rharperok, when will launchpad update with the correct diff20:09
smoseri dont know that it wil20:09
smoserjust fetch and diff yourself :)20:09
smoseror click on the individual commits20:10
smoseri have the next one ready too20:11
smoser sysconfig: enable mtu set per subnet20:11
smoserwith tests added20:11
rharperok20:12
rharperwe've a new bug in the network state:  the mtu test in curtin registered a manual subnet with no address (AFAICT that's legit);  we have network_state code that checks for 'address' in a subnet and blows up if 'address' key isn't present20:13
rharperthe type: manual avoided that path since it was never looked at20:13
rharperhttp://paste.ubuntu.com/25134871/20:13
rharperthat's a one-liner if 'address' in subnet and ':' in subnet['address']20:14
smosercurrently sysconfig raises a valueerror20:16
smoserValueError: No config network address keys [address] found in {'control': 'manual', 'type': 'static'}20:16
smoseractually, yeah. bot in my test raise a valueerror20:18
smosersaying its bogus20:18
smosereni and sysconfig20:18
rharperhrm20:18
rharperlet me try curtin.net20:18
rharperit's possible that I can just remove it altogether as I can't see what value it would add to the test-case at this time20:19
smoserhttp://paste.ubuntu.com/25134888/20:19
rharpernope, fail in curtin.net as well;  well, let's remove it and see what it looks like20:19
rharperyeah20:20
rharperoh, it was ordering iface stanzas20:20
rharperhrm, so how do we get an iface $name manual ?20:21
rharperand no ip configuration ?20:21
smoserrharper, https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/32783220:23
smoseris now pushed with my fixes on yours squashed and laucnhpad re-rendered20:23
smoserno diff other than commit message versus what c-i ran at20:24
smoser https://jenkins.ubuntu.com/server/view/Cloud-init/job/cloud-init-ci/67/console20:24
smoserso i will pull if you say good.20:24
rharperlooks good20:24
smoserand then i have 7 ready and i might just include 8 in with it i just straight cherry-picked it20:24
rharperok20:24
smoserhttps://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/32783620:30
smoserrharper, so. ^ has 2 more of them.20:32
smoserwhich leaves us just aaa7101790d520:32
smoser Fix style issue with unused exeception variable in DataSourceAzure20:33
smoserwhich is not related20:33
smoserand i'm not sure what caught it. cause that isnt new20:33
smoserand then20:33
rharperright, it's not, but i wanted make style-check to pass20:33
smoser  sysconfig: use MACADDR on bonds/bridges to configure mac_address20:33
rharperit fails here  (xenial laptop)20:33
smoserah. well, the answer is that we dont care.20:35
smoserrun tox -e pyflakes20:36
rharpertox passes20:36
rharperam I missing an update?20:36
rharpersurely we run a tox flakes under xenial ?20:36
rharperor is it style from $dev_release  only ?20:36
smosercoding style is checked via running20:37
smoser tox -e flake820:37
rharperwhat does 'make style-check' run ?20:38
smoserthat pins versions of of things to what we accept as current targets20:38
smoserall 'make' targets use whatever things are installed on the system.20:38
rharperis there a reason why it doesn't use tox ?20:38
rharperif that's the one-true-way  ?20:38
smosermake is bascially "use what is on the system"20:39
smoserwe run 'make test' during the ubuntu builds20:39
rharperat best it passes but most likely wrong; I'll submit a patch later;20:39
smoserso that we verify the test runs against system versions20:39
rharperright, but style-check could certainly do the tox -e flake8 instead20:39
smoserbut we do not run style checks20:39
rharperright, which I'm perfectly fine with20:39
rharperI'm just out-dated on how to check style20:40
rharperif we have a partually useless make target, I'd like to fix it up20:40
rharperthat's all20:40
smoserbecause other wise we are held to the least common denominator of styles20:40
smoserits not completely useless.20:40
smoserit passes for me20:40
rharperbut it doesn't for me20:40
rharperyou're on artful20:40
smoser(as artful has the same (or similar) versions that we *do* care about)20:40
smoseri'm not opposed to removing it... but it doesnt run when you just type 'make'20:41
rharperbut I've got something else that does't like it ; anyhow, we can bikeshed in the MP (if I submit it)20:41
smoserand that is by design20:41
rharperthat's fine20:41
rharperI don't want to change style-check to run anywher else20:41
rharperjsut make style-check should match what we say is the right style, which is the frozen set in tox -e flake820:41
smosernothing in 'make' depends on the internet20:42
smoserthats why make doesn't run tox20:42
rharperso, sed 's,pep8 $(pyflakes),tox -e flake8'20:42
rharpernothing with plain 'make' depends on the style-check by design20:42
rharperso why not have the style-check actually check the style with the right versions ?20:42
* rharper tables it for later20:42
smoseronce https://jenkins.ubuntu.com/server/view/Cloud-init/job/cloud-init-ci/69/ finishes i'll pull20:45
rharperk20:46
rharperjust about done with rebase of the last few20:46
rharperok, pushed20:46
smoserok... https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/327836 will pass c-i. i will pull it . that means we just have the newest commit on your branch to grab.20:50
smosersysconfig: use MACADDR on bonds/bridges to configure mac_address20:51
smoseri'll take your tests that you added there and put them into the ones i added rather than adding the mis-named RoundTrip20:51
rharperyou merged both mtu-per-subnet and ipv6 def-route ?20:53
smoserthose are both in 721:02
smoserwhich i'll just pull now (bringing in the 2 individual commits that are there)21:02
rharperk21:03
smoserthose are now there.21:03
smoseri'm woring on the last commit.21:03
smoserhora!21:03
rharperack21:03
rharperno more rebase needed then21:04
smoserrharper, https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/32783821:10
smoseri just cherry picked yours and then21:10
smoser https://git.launchpad.net/~smoser/cloud-init/commit/?id=64bd6b52a57cc0a7cc3a4c689958d0c7b2f4add621:10
rharperack21:12
rharperapproved21:12
rharperthanks!21:12
* rharper ends smoser some beer21:12
smoseryeah, ...21:12
smoserso you dont need this in ubuntu21:12
smoseryou just needed it in trunk for the moment21:12
smoserbut i think i might just upload to ubuntu too21:12
blackboxswminor comment on https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/32783821:13
smoserblackboxsw, ack. i'll grab that change.21:13
rharperthere was at least one commit needed for eni rendering21:13
blackboxsw+1 thx21:13
* rharper looks for the list21:13
smoseri dont like using the .get() twice21:14
smoser(but we were doing it anwyay)21:14
smoserie, i dont like:21:14
smoserif mydict.get('key'):21:14
smoser   x = mydict.get('key')21:14
rharpersmoser: https://bugs.launchpad.net/cloud-init/+bug/1701097  ,is needed back to Xenial21:14
ubot5Ubuntu bug 1701097 in cloud-init (Ubuntu Artful) "eni rendering of ipv6 gateways fails" [Medium,Confirmed]21:14
smoserbut oh well.21:14
rharperbut only once we release (and sru) an curtin with passthrough enabled21:15
rharperwe should see about getting those scheduled/started soon given our timeline and desire for MAAS testing21:16
smoserblackboxsw, https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/327827 looks very clean. thank you21:20
smosernice commit messages.21:20
blackboxswgood deal smoser, thanks21:21
blackboxswI'll get out from under this aws ipv6 in short order I hope21:21
blackboxswsmoser: to confirm, how a datasource is determined to run in local mode is by the absense of  <DatasourceModule>.datasources sources.DEP_NETWORK in the deps listing?21:23
smoserright21:25
smoserok. so https://jenkins.ubuntu.com/server/view/Cloud-init/job/cloud-init-ci/72/ should finish in 12 minute and then i can pull21:43
smoseri might get to that later tonight.21:43
blackboxswhave a good one smoser21:43
rharperlater21:45
blackboxswsmoser: /me is wondering why we'd want to try searching/get_data for DataSourceEc2 in the network stage if we can run DataSourceEc2 in local22:04
blackboxswsmoser: /me is wondering why we'd want to try searching/get_data for DataSourceEc2 in the network stage if we can run DataSourceEc2 in init-local22:04
blackboxswn/m I think I recall now. The reason attempting the datasource in both init-local and init-network  stages would be to keep non-aws Ec2 clones running in the init-network stage so we don't impact their initialization w/ this changeset.22:12

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!