cxdeviitHow can I update cloud-init in AWS?06:42
meenai've started a patch here: https://github.com/canonical/cloud-init/pull/621 hope i can get it into the next releaseā€¦ along with my /other patch/11:23
meenathis needs a better title than the title of the first commit11:26
Odd_Blokecxdeviit: cloud-init generally comes from your distro vendor as part of the images you launch, and is generally updated via standard upgrade mechanisms or via new distro images being generated.  What distro are you using, and what's the motivation for needing cloud-init upgraded?13:22
cxdeviitOne of my AWS instance has Amazon Linux 1 and Cloud-init version 0.7.6 which doesn't seem to support devel analyze etc , that's where I am stuck. If there was a way to patch it .14:17
Odd_Blokecxdeviit: Aha, right.  I would be hesitant to recommend upgrading cloud-init in any way other than distro packages: that could break your machine in ways that are difficult to recover from.  So I don't know if you have a good path to a newer cloud-init.  That said, if you're looking to use `analyze`, it has a `--infile` parameter, so you could pull your log files to a newer cloud-init installation14:23
Odd_Bloke(locally or in a different distro instance in the cloud) and perform your analysis that way.14:23
cxdeviitAha, will try that. Another issue is regarding requests dependency of urllib3 to be 1.21.1 , not sure if pip/python is messing here14:25
minimalOdd_Bloke: Hi. Re: Mac Addreses, as Netplan files are YAML also then they should also have all-numeric MAC addresses quoted, right?14:33
Odd_Blokeminimal: Presumably, yeah.14:41
minimalso then I've found a bug in the netplan renderer - when I fixed an all-numeric MAC address in a testcase then it failed due to the macaddress in the generated netconf file having no quotes14:42
Odd_BlokeCan you pastebin the error?14:43
minimalOdd_Bloke: https://gist.github.com/dermotbradley/bb7353c00bf505bbf2f1ce0eb2b56c1714:53
minimalits the 5th MAC address14:54
Odd_Blokeminimal: That's test_render_output_supports_both_grat_arp_spelling?  (I'm heading into a meeting, will look more after.)15:13
minimalOdd_Bloke: yupe, that's the one15:14
meenasmoser:  that sure is one way to say, focus on  the other PR15:54
smosermeena: ? sorry... didnt' mean to be a pita.15:54
smoseroh. funny.16:00
smoserwow. total fail16:00
smosermeena: updated that comment https://aci-github.cisco.com/atom/atomix/pull/126916:01
meenasmoser, nah, it's cool. what you're doing is useful and important16:04
meenasmoser, I'm getting address not found for that link16:15
smoserman. i totally cannot paste16:15
smoseryeah, that other is my iternal github16:16
Odd_Blokeminimal: So I think that MAC address _doesn't_ need quoting because it has values that exclude it from being a timestamp (99 > 59, in the seconds and hour fields): https://paste.ubuntu.com/p/tddYNXWrbk/16:32
minimalOdd_Bloke: hmm, the YAML "timestamp" is actually whether something is treated as a base 60 number and according to https://yaml.org/type/int.html the definition for this is |[-+]?[1-9][0-9_]*(:[0-5]?[0-9])+16:37
minimalso you're right, I'll have to rewrite my PR docs change to be even more specific about when quoting is required ;-)16:40
Odd_Blokeminimal: OK, right, I was trying to figure out why I couldn't find reference to it in the spec; didn't occur to me that it was a base 60 number!  Good find, and I agree that more specificity is needed.16:57
Odd_Bloke(At least we know that anyone running cloud-init in ancient Sumeria will experience no problems. :p)16:58
minimalOdd_Bloke: search YAML docs to find that ref gave me a headache, lol17:05
momoustaHi, we would like to land this PR (https://github.com/canonical/cloud-init/pull/614) before the code cutoff for the next SRU, Can someone help taking a look at it?17:25
minimalOdd_Bloke: PR created for the MAC address docs. Any chance you could review it and PR 618 as well?17:34
minimalfor PR 618 the question of how to test it is still open - /etc/shadow is normally only readable by root user, so guess it would need to be mocked17:35
meenawait, wait wait18:48
meenaYAML, which is a superset of JSON knows integers, but JSON does not18:48
meenathat's hilarious, i didn't realise how lossy the conversion between the two could be18:49
minimalmeena: with JSON strings are quoted so there's no room for confusion, with YAML you don't have to quote (most) strings and magically the app reading the YAML works out if a value is a number or a string...18:56
minimalexcept in this case in YAML a MAC value of 11:22:33:44:55:66 (without quotes) will be treated as a Base 60 number whereas a value of 11:22:33:44:55:6a (without quotes) will be treated as a string18:58
minimaldoes anyone on here from Canonical look after cloud-utils? Am confused that https://launchpad.net/cloud-utils has 0.31 dated 10/12/2018 as latest release but that https://github.com/canonical/cloud-utils/ has no 0.31 and has 0.30 dated 10/07/2019 (i.e. dated far newer than 0.31 on the other site)19:07
Odd_Blokeminimal: It's getting even better: the YAML 1.2 spec doesn't include base 60 (which is why I couldn't find it!) but PyYAML only supports YAML 1.1 which _does_.19:09
minimalminimal: any cloud-init's network config uses with YAML schema? (Also didn't notice mention of YAML version on netplan's website)19:10
Odd_BlokeWe (implicitly) use YAML 1.1, because we use PyYAML.19:11
minimalOdd_Bloke: any idea on what I pointed out about cloud-utils above?19:12
meenaminimal, yeah, we've seen this issue in puppet / hiera19:13
Odd_Blokeminimal: I believe the tag in GitHub was created on that much later date, but if you look at the commit it points to it's older.19:13
Odd_Bloke(In fact, there are a whole bunch of version tags that were created on that same day, so that was presumably an import to GH?)19:14
minimalok. I maintain the Alpine package for cloud-utils and 0.31 is downloaded from Launchpad........just wondered if I was using out of date code. Any idea if a 0.32 is going to happen any time soon? The last release has been out a while19:15
Odd_BlokeI know some cloud-utils things have been happening, but I haven't kept on top of them.19:17
minimalfor example in "master" there a growpart LVM commit from July that's obviously significant for cloud-init users19:19
Odd_Blokeminimal: I'll look into it.19:23
powersjheh smoser released 0.31 end of 2018 when he moved it to git19:24
minimalpowersj: yeah found the 0.31 tag hidden on the 2nd (or 3rd?) Releases page in git. Have just downloaded from both Launchpad and Github and compared to verify both 0.31 files are identical19:25
* meena quotes herself quoting someone else she heard saying it: release early release often19:26
meenaalso: why y'all afraid of 1.019:26
meenawhat's up? why the 0.?19:26
minimalmeena: as its a prime number? They're scary!19:26
meenayou can skip primes19:35
Odd_BlokeEvery number is prime if you define division the right way: https://paste.ubuntu.com/p/rcHw2M5pVf/19:40
meenaOdd_Bloke: na na na. that means we're then skipping all numbers, which brings us back to where we started19:43
minimalOdd_Bloke: for the Ansible/Jinja2 templates I'm using to create network-config-v2.yaml adding "| to_yaml" to the macaddress variables sorts the automagic YAML string quoting for me19:47
Odd_BlokeOK, good, that's what I'm hoping will be the case for most generated YAML: the libraries will DTRT.  And most networking YAML will be generated in some way, because it's handled by the platform (which launches many instances) rather than the user (who may only launch a few and not bother investing in automation).19:57
Odd_BlokeThat said, I wonder if YAML 1.2 libraries do the correct quoting for values which are strings in 1.2 but ints in 1.1.19:57
minimalTo be honest I think YAML is trying to be too smart - they should have just required strings to always be quoted and been done with it19:58
meenawait_for_physdevs_netcfg() is a terrible name for a function that's no longer in that class20:02
meenabut it looks generally useful20:02
minimalOdd_Bloke: your command about ".. include:", does this support including from another part of the same document? or only from a separate file?20:05
Odd_Blokeminimal: I think we'd need a separate file with just that snippet.20:06
Odd_BlokeWhich is perhaps overkill, but I also wouldn't be surprised if we find this info needs updating again in future.20:06
Odd_BlokeSo I could go either way.20:06
Odd_BlokeI come bearing bad news: ruamel.yaml supports YAML 1.2, and `yaml.load(ruamel.yaml.dump({"x": "11:22:33:44:55:59"}))` gives `{'x': 8846009759}` (whereas dumping with PyYAML round-trips correctly).20:10
minimalwhere is ruamel.yaml used? A quick grep doesn't show in anywhere in cloud-init20:14
Odd_BlokeIt isn't, but it's a library that could be used to generate the networking YAML that is fed to cloud-init.20:14
Odd_BlokeAnd as it is a YAML 1.2 library, it does not quote base60 ints properly (because in YAML 1.2 base60 ints aren't a thing, so their unquoted form is unambiguously a string).20:15
Odd_BlokeThis is in reference to "That said, I wonder if YAML 1.2 libraries do the correct quoting for values which are strings in 1.2 but ints in 1.1."20:16
minimalIt describes itself as a YAML 1.2 loader/dumper, whereas you'd indicated earlier that network-config-v1.yaml and network-config-v2.yaml are YAML 1.120:16
minimalthough I guess that's just because c-i uses PyYAML and not stated anywhere in the c-i "specification" for those YAML files20:17
Odd_BlokeYeah, that's the gap I'm worried about.20:18
Odd_BlokeUntil today, I hadn't really thought too much about YAML 1.1 vs 1.2, and I strongly suspect I'm not alone in that.20:18
minimalsame applies to Netplan - did'nt see any YAML version mentioned there either20:18
minimalnow you'll have the fun of determining what each of the DataSources vendors (AWS, GCE, etc) do and try and get them to agree on YAML 1.1, doh!20:20
Odd_BlokePractically speaking, if any of the large platforms could generate incorrect YAML in this way, we'd have heard about it, I think: this isn't a recently introduced "bug", we've always used PyYAML and therefore YAML 1.1.20:22
Odd_Bloke(And they obviously get through a lot of MAC addresses. :p)20:22
minimalTime to hurriedly insert "These files are in YAML 1.1 format" into the docs in big flashing letters eh? ;-) (damn, Mozilla decommissioned the HTML blink tag)20:24
Odd_BlokeI'm sure we can bring it back with some custom JS.20:24
Odd_BlokeBut yeah, I think we do want to document this.20:25
minimalI'm guessing a all-numeric MAC address where all of the component part is 00-59 is statistically unusual20:25
Odd_BlokeYeah, and it is possible they might be using a prefix (that's a MAC address thing, right, vendor prefixes?) that excludes all of their MACs from triggering this.20:26
Odd_Blokeminimal: https://bugs.launchpad.net/netplan/+bug/190108420:32
ubot5Ubuntu bug 1901084 in netplan "netplan should specify the version of YAML that it takes as input" [Undecided,New]20:32
minimalI'd already noticed that the page https://netplan.io/reference/ has at least 1 unquoted MAC address that needs to be quoted20:35
minimalOdd_Bloke: If you search online for OUI each hardware vendor that registers get a prefix to use for network cards they manufacture. Likewise a MAC beginning "52:54" is KVM (from memory). There's a OUI.txt file online managed by IETF that is regularily updated20:36
minimalin my case as I'm testing on KVM I'm using made up MACs within the KVM prefix and happened to use all numeric addresses.20:39
minimalThe other PR I raised a few days ago about user-data YAML for user settings - I'd noticed that expirydate was being misinterpreted without quotes and what was being put into the field in /etc/shadow was incorrect20:40
minimalso potentially the impact is on more than just MAC addresses20:41
meenahttps://github.com/canonical/cloud-init/pull/617#issuecomment-714758289 ds-identify is a trash fire20:58
meenamaybe reading disk labels doesn't work for bsd21:08
minimalOdd_Bloke: FYI https://en.wikipedia.org/wiki/Sexagesimal#YAML22:05

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!