| === shardy is now known as shardy_mtg | ||
| === shardy_mtg is now known as shardy | ||
| randomhack | Is it possible to override the default cloud-init values inside the OS image? I'm trying to have it always run the disable-ec2-metadata module. TIA! | 13:42 |
|---|---|---|
| larsks | randomhack: yes, you can pass in a cloud configuration that will override what is provided in the image. | 14:13 |
| smoser | randomhack, or just put that setting into /etc/cloud/cloud.cfg.d/ | 14:13 |
| smoser | i think thats what you're asking. | 14:13 |
| smoser | echo "disable_ec2_metadata" >> /etc/cloud/cloud.cfg.d/99-disable-ec2.cfg | 14:14 |
| smoser | er... with the value too | 14:14 |
| randomhack | Thanks! | 15:08 |
| smoser | rharper, around ? | 15:13 |
| rharper | here | 15:13 |
| smoser | https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1675576 | 15:13 |
| rharper | y | 15:13 |
| smoser | shall i attempt to know understand the 00-snapd-config.yaml | 15:13 |
| smoser | and delete the files it writes undconditionally | 15:14 |
| smoser | or just unconditionally delete those ? | 15:14 |
| rharper | I think we do it like the eni one; it has known content | 15:14 |
| rharper | it's baked into the core snap image; | 15:14 |
| rharper | so, just like the eth0.cfg in the cloud-images | 15:14 |
| smoser | ok. so if the 00-snapd-config.yaml exists with known content | 15:15 |
| smoser | then i unconditionally delete the 3 other files | 15:15 |
| rharper | I think so; let me walk through the paths | 15:18 |
| rharper | 1) if cloud-init is disabled, then the image will use the baked in config; no issue 2) if cloud-init is enabled and has no network-config; it should still generate a fallback config, like we do today; uc16 will enable the netplan renderer; the baked in config should be removed 3) cloud-init is enabled, and we have a network config; uc16 is configured to use netplan renderer, so we should delete | 15:19 |
| rharper | (2) and (3) I think cover the cases for if content is known, delete it as cloud-init will render a fallback (or specified) network config | 15:20 |
| rharper | and yes to wiping the files that netplan would have generated as the netplan renderer will call 'netplan generate' which will create those files as needed (from the cloud-init config it rendered) | 15:20 |
| jgrimm | powersj and magicalChicken, you are probably interested in reviewing this too -> https://code.launchpad.net/~raharper/cloud-init/+git/cloud-init/+merge/321397 | 16:01 |
| jgrimm | rharper, thanks for writing ^^ fantastic | 16:05 |
| powersj | rharper: that doc is incredibly helpful. thank you! | 16:22 |
| rharper | cool | 16:26 |
| smoser | rharper, https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/321465 | 16:28 |
| rharper | k | 16:28 |
| rharper | smoser: I proposed that the other night | 16:29 |
| smoser | oh. link ? | 16:29 |
| smoser | i thought i'd seen it | 16:29 |
| rharper | https://code.launchpad.net/~raharper/cloud-init/+git/cloud-init/+merge/321222 | 16:30 |
| rharper | mines better because you need to import logging and create LOG object | 16:30 |
| smoser | :) | 16:31 |
| smoser | yeah | 16:31 |
| smoser | ok | 16:31 |
| smoser | rharper, http://paste.ubuntu.com/24282363/ | 16:59 |
| smoser | are you opposed to that on top of yours ? | 16:59 |
| rharper | looking | 17:05 |
| rharper | smoser: that looks fine | 17:05 |
| smoser | rharper, thats merged. if you want to read... https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/321471 | 17:09 |
| smoser | i need to add tests | 17:09 |
| smoser | its very simplistic and targetted. i had more generic looking things, but the scaffolding is unnecessary at this time. if we need somethign smarter later, add it then. | 17:10 |
| rharper | smoser: ok, looking | 17:56 |
| smoser | *just* p ushed with teests | 17:57 |
| rharper | smoser: also, is it reasonable for ds-identify to also look in /etc/cloud/cloud.cfg.d/ ? for maas datasource ? | 17:57 |
| rharper | the debconf_setselections hook for maas IIRC, writes the maas datasource cfg to /etc/cloud/cloud.cfg.d | 17:57 |
| rharper | but ds-identify detect_maas only searchs PATH_CLOUD_CONFD/*maas*cfg ; which I believe will miss the .cfg.d dir | 17:58 |
| smoser | rharper, it does look for maas datasource there, no? | 18:00 |
| smoser | it should. let me check. | 18:00 |
| rharper | would be nice if check_config mentioned the files it checked in the debug log so we could be certain | 18:01 |
| smoser | bah. i thjin its busted. | 18:01 |
| rharper | debugging a uc16 deploy with plars and it's not finding the datasource | 18:01 |
| rharper | smoser: it does check files in .cfg.d/*maas*; it;'s looking for a top-level key 'MAAS' but I believe we specify it via datasource:\n\tMAAS: {} | 18:15 |
| rharper | so it doesn't match | 18:15 |
| smoser | no. its just not looking right. the file names input are wrong. | 18:16 |
| smoser | i dont think it checks in .cfg.d | 18:16 |
| smoser | at all | 18:16 |
| rharper | hrm | 18:16 |
| rharper | I added a debug log to check_config to print the fnames that match | 18:16 |
| rharper | [up 659.59s] ds-identify | 18:17 |
| rharper | policy loaded: mode=search report=false found=all maybe=all notfound=disabled | 18:17 |
| rharper | checking config file: /etc/cloud/cloud.cfg | 18:17 |
| rharper | checking config file: /etc/cloud/cloud.cfg.d/05_logging.cfg | 18:17 |
| rharper | checking config file: /etc/cloud/cloud.cfg.d/50-cloudconfig-maas-cloud-config.cfg | 18:17 |
| rharper | checking config file: /etc/cloud/cloud.cfg.d/90_dpkg.cfg | 18:17 |
| rharper | shows it reading it | 18:17 |
| rharper | now checking why we don't match | 18:17 |
| smoser | http://paste.ubuntu.com/24282702/ | 18:19 |
| smoser | hm.. | 18:19 |
| smoser | oh. those are just other things reading it. | 18:19 |
| smoser | it does read those for some sources | 18:19 |
| smoser | for some calls to check_config | 18:19 |
| smoser | but not for maas's | 18:20 |
| smoser | try that paste ? | 18:20 |
| rharper | ok; currently it looks like key was getting clobbered from a value in the file | 18:20 |
| rharper | ahh, I see | 18:21 |
| rharper | other methods were looking there but the dscheck_MAAS wasn't | 18:21 |
| rharper | lemme try yours | 18:21 |
| rharper | smoser: ok, that works! | 18:23 |
| rharper | checking config file: /etc/cloud/cloud.cfg for key: MAAS | 18:23 |
| rharper | checking config file: /etc/cloud/cloud.cfg.d/05_logging.cfg for key: MAAS | 18:23 |
| rharper | checking config file: /etc/cloud/cloud.cfg.d/50-cloudconfig-maas-cloud-config.cfg for key: MAAS | 18:23 |
| rharper | checking config file: /etc/cloud/cloud.cfg.d/90_dpkg.cfg for key: MAAS | 18:23 |
| rharper | check for 'MAAS' returned found | 18:23 |
| smoser | that PATH_CLOUD_CONFD | 18:23 |
| smoser | was very confusing | 18:23 |
| rharper | smoser: that said, I think we want to check for datasource: MAAS though | 18:23 |
| smoser | i need to get my unit tests for ds-identify | 18:24 |
| rharper | not just 'MAAS' does it walk indented colons ? | 18:24 |
| smoser | thers a comment about that | 18:24 |
| smoser | "currently does not respect any hierarchy in searching for key." | 18:24 |
| rharper | urg | 18:24 |
| rharper | so, how do specify the datasource: MAAS then ? | 18:24 |
| rharper | we need to emit MAAS: foo and the datasource: MAAS ? | 18:24 |
| smoser | i'm confused. | 18:25 |
| smoser | datasource: | 18:25 |
| smoser | MAAS: | 18:25 |
| smoser | stuf here | 18:25 |
| smoser | that will work | 18:25 |
| smoser | the parser doesn't know that 'MAAS' there is under datasource. | 18:25 |
| smoser | iti just finds a key named MAAS | 18:25 |
| rharper | ok; the comment about hierarchy led me to believe we'd not look at anything but the top level keys | 18:25 |
| smoser | its just broken in a different way then you imagined :) | 18:26 |
| rharper | if it was written datasource: { MAAS: { } } | 18:26 |
| smoser | basically it just flattens everything and looks at only key names. | 18:26 |
| rharper | it'd break as well | 18:26 |
| smoser | maybe, yes. | 18:26 |
| rharper | ok, well, that's good enough until we do unittests and sort out edge cases | 18:26 |
| smoser | yeah. it doesnt support reading that. | 18:26 |
| smoser | wait. maybe it does. | 18:26 |
| * rharper tries | 18:26 | |
| rharper | nope | 18:27 |
| smoser | right. it doesnt. we need a tool that can merge cloud config in the same way cloud-init does and run fast. | 18:28 |
| rharper | y | 18:29 |
| smoser | rharper, mp https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/321482 | 19:26 |
| rharper | k | 19:26 |
| smoser | rharper, fwiw, stumbled upon free speedup in cloud-init at | 19:39 |
| smoser | http://rmcgibbo.github.io/blog/2013/05/23/faster-yaml-parsing-with-libyaml/ | 19:39 |
| smoser | seemingly free | 19:39 |
| jgrimm | smoser, have you benchmarked? | 19:40 |
| smoser | it probably wont be huge for cloud-init, but it is faster per | 19:41 |
| smoser | http://stackoverflow.com/questions/16864132/is-there-an-up-to-date-fast-yaml-parser-with-python-bindings | 19:41 |
| smoser | http://stackoverflow.com/questions/27743711/can-i-speedup-yaml | 19:41 |
| jgrimm | ack, yeah that's what i was curious about, whether it actually makes a difference, tho small, but free is something | 19:42 |
| smoser | rharper, cloudinit/net/netplan.py | 19:45 |
| smoser | bah | 19:45 |
| smoser | https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/321471 | 19:45 |
| smoser | prints removed | 19:45 |
| rharper | smoser: whoa , we have libyaml ? | 19:56 |
| smoser | well, in a new container this works: | 20:00 |
| smoser | python3 -c 'import yaml; from yaml import CLoader' | 20:00 |
| smoser | so i'd think that'd trow exception otherwisd | 20:00 |
| jgrimm | i had a trusty container handy, seemed to work their too (and i can see libyaml in dpkg -l too) | 20:00 |
| jgrimm | s/their/there | 20:00 |
| smoser | # apt-cache show python3-yaml | grep Depe | 20:01 |
| smoser | Depends: python3 (<< 3.6), python3 (>= 3.5~), python3:any (>= 3.3.2-2~), libc6 (>= 2.14), libyaml-0-2 | 20:01 |
| rharper | do we run py3 on trusty c-i ? | 20:01 |
| smoser | probably python2 there. | 20:01 |
| jgrimm | i ran python2 there | 20:02 |
| smoser | and we should do a try: except ImportError anyway | 20:02 |
| jgrimm | https://github.com/msmbuilder/msmbuilder-legacy/pull/199/commits/beb21bf10892dd566fccf567f907404957db1aed | 20:02 |
| rharper | oh I suppose it should still work for py2 to call to libyaml though right ? | 20:02 |
| jgrimm | looks like easy pattern to adopt | 20:02 |
| smoser | yeah | 20:02 |
| smoser | rharper, intentionally or not, your bond rename devices test case shows that vlans have the same mac as the nic they use also | 20:48 |
| smoser | so have to ignore those | 20:48 |
| smoser | so, good that you did that . | 20:49 |
| rharper | huh | 20:49 |
| rharper | yeah, I guess that makes sense | 20:49 |
| rharper | they just need to check for the vlan header but the mac is going to be the same | 20:49 |
| rharper | w.r.t "is this packet for me" | 20:49 |
| rharper | yay unittests! | 20:49 |
| jgrimm | powersj, any strong reason to require --data-dir to the integration test 'collect'? That is, why not allow for tmpname as it gets output to stdout with 'run' (so could be inspected if left alone and/or verify'd later) | 20:59 |
| powersj | jgrimm: I don't think it is a very strong feeling, but it was to prevent writing it to /tmp so you didn't loose your collect after each reboot :) | 21:01 |
| powersj | strong reason* ugh | 21:01 |
| jgrimm | cool, i assumed so. just didn't work as i'd guessed it would (since i didn't read the docs, doh) | 21:02 |
| powersj | :) | 21:02 |
| powersj | yeah if I recall when I was writing all the initial tests it would go to tmp and then the next day I would spend the first 20mins running collect again :\ | 21:02 |
| jgrimm | fwiw, the docs could be a bit more explicit on 'use collect' if you want to actually inspect a run | 21:02 |
| jgrimm | i can certain submit a pull request if agreed | 21:02 |
| powersj | magicalChicken has a MR to keep the records around when doing a run | 21:03 |
| jgrimm | powersj, cool, that's what i was wanting to do and figured collect was the intended way of doing that | 21:03 |
| powersj | https://code.launchpad.net/~wesley-wiedenmeier/cloud-init/+git/cloud-init/+merge/320401 | 21:03 |
| jgrimm | powersj, do we run integration tests on trusty as part of automation? | 21:04 |
| powersj | that however does still need a doc update :) | 21:04 |
| powersj | not yet | 21:04 |
| powersj | A few too many of the tests initially did not work with trusty versus xenial and newer due to changes in behavior | 21:05 |
| jgrimm | heh, ok i want to play with the tests and better understand.. randomly chose a module.. blows up on trusty, not elsewhere. :) | 21:05 |
| jgrimm | cool. known issue(s). :) | 21:05 |
| powersj | haha yeah magicalChicken has yet another MR for distro flags, to be able to specify what tests can run on what distros | 21:05 |
| powersj | https://code.launchpad.net/~wesley-wiedenmeier/cloud-init/+git/cloud-init/+merge/321029 | 21:06 |
| jgrimm | indeed, looking forward to that landing, really nice addition | 21:06 |
| * jgrimm checks whether this one is fixed already. | 21:07 | |
| jgrimm | powersj, but overall I think the integration test infrastructure is really nice.. easy to add to it | 21:08 |
| powersj | jgrimm: thanks! magicalChicken: ^^^ :) | 21:08 |
| jgrimm | \o/ | 21:09 |
| magicalChicken | powersj, jgrimm: nice thanks :) | 21:11 |
| magicalChicken | jgrimm: you mentioned getting everything going on trusty, i still have a couple more feature flags to mess with there i think | 21:12 |
| magicalChicken | jgrimm: part of the issue is out of date cloud-init, so i need to build a py2 deb with current master for trusty and see how many tests fail just because of the old version | 21:13 |
| jgrimm | magicalChicken, very cool | 21:18 |
| jgrimm | indeed, i'm guessing this is an issue with being out of date | 21:19 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!