| CabbageMan69 | Hello all. I'm having some problems turning on debugging info for cloud-init on an instance in EC2. Is it as simple as having "verbose:\n debug: true" in the cloud-init config? | 05:23 |
|---|---|---|
| CabbageMan69 | I'm getting the following two lines of output in my cloud-init-output.log file with that config present, and I see the following two lines ... not sure what is happening between those two steps that causes it to take 2 minutes | 05:25 |
| CabbageMan69 | Cloud-init v. 0.7.5 running 'modules:config' at Mon, 23 May 2016 05:02:58 +0000. Up 35.99 seconds. Cloud-init v. 0.7.5 running 'modules:final' at Mon, 23 May 2016 05:05:08 +0000. Up 166.10 seconds. | 05:25 |
| CabbageMan69 | For context, I'm using Ubuntu 14.04 and I'm running v0.7.5 of cloud-init (is this ancient?) | 05:26 |
| cpaelzer | smoser: Hi, I extended the MP at https://code.launchpad.net/~paelzer/cloud-init/test-apt-source/+merge/294521 | 15:03 |
| cpaelzer | it has now the dictionary based handling and all kind of tests around it | 15:04 |
| cpaelzer | smoser: I'll check tomorrow if I need to revise it a lot / a bit / not - and will start to poke at the curtin portion of it then | 15:04 |
| cpaelzer | smoser: I'll expect to have some questions tomorrow when you come online, but will let you know then | 15:05 |
| harlowja | rharper ya that's def where the complexity comes from ;) | 17:25 |
| harlowja | and/or figuring out sysconfig for myself, and its weirdness :-P | 17:25 |
| rharper | harlowja: hehe | 17:27 |
| harlowja | rharper although i think https://gist.github.com/harlowja/d63a36de0b405d83be9bd3222a5454a7/ is mostly there, just eyes on that would be cool | 17:27 |
| harlowja | let me know if it looks dumb :-P | 17:27 |
| harlowja | https://gist.github.com/harlowja/d63a36de0b405d83be9bd3222a5454a7/#file-sysconfig-py-L320 is the main stuffs | 17:27 |
| harlowja | oddly i don't think redhat has a way to configure routes without an interface | 17:28 |
| harlowja | or at least i'm not sure if there is one | 17:28 |
| rharper | harlowja: I gave it a once over the other day; it looks quite reasonable (cleaner than my initial cut); looking to steal a lot of that; we may find with the two implementations (and a networkd coming) that we have a base set of hooks that new config backends would implement | 17:31 |
| harlowja | cool | 17:31 |
| rharper | the networkd does change things up quite a bit as there is no single file to write, rather multiple files (.link for name mapping) (.network for networks which may or maynot be device specific) and (.netdev for virtual devices like bridges and bonds); so the in-order processing that happens for eni and sysconfig don't have to apply but certainly can. | 17:33 |
| harlowja | ya, sysconfig also doesn't have single files | 17:34 |
| harlowja | only eni i think has a single file | 17:34 |
| rharper | ok | 17:35 |
| rharper | bbiab | 17:35 |
| harlowja | smoser rharper can we (or shall i?) merge in https://code.launchpad.net/~harlowja/cloud-init/cloud-init-net-refactor | 17:46 |
| harlowja | that way this sysconfig stuff (which that uses as a base) ; can start to get reviewed more offically :-P | 17:46 |
| * harlowja doesn't know to well how to stack things in bzr (against a branch that hasn't merged yet) | 17:47 | |
| smoser | harlowja, can you write at very least a combined commit message and description there ? | 17:51 |
| smoser | is it all just "refactor" did you clean up the N different internal states ? | 17:51 |
| harlowja | sure boss | 17:51 |
| harlowja | ummm, let me check | 17:51 |
| harlowja | mostly refactor :-P | 17:51 |
| === nacc_ is now known as nacc | ||
| harlowja | ok, updated https://gist.github.com/harlowja/d63a36de0b405d83be9bd3222a5454a7 rharper if u want to see | 21:05 |
| rharper | looking | 21:09 |
| rharper | harlowja: nice, lines 34 -> 70ish look like cloudinit.net common stuff that all backends could use | 21:14 |
| harlowja | def | 21:15 |
| rharper | harlowja: nit: L82 in the header you say on %(now)s ; would 'at' or '@' be more natural ? L324 render_dns; you have it as a pass, but I'd think the signature would need to take a list object to append something if/when it ever did support updating dns; | 21:24 |
| harlowja | sure | 21:24 |
| harlowja | rharper yup yup, dns not done yet ;) | 21:24 |
| harlowja | let me get that in a more offical place, ha | 21:26 |
| harlowja | ok smoser added nice handy commit message in https://code.launchpad.net/~harlowja/cloud-init/cloud-init-net-refactor/+merge/293957 :-P | 21:31 |
| rharper | and at the risk of exposing my ignorance; a few of the render methods are @classmethod ; but it doesn't appear that they're used outside of other methods within the class; what's that doing ? | 21:32 |
| harlowja | ya, more or less just a thing i've gotten used to, no need to pass self around if not using it for those | 21:32 |
| harlowja | and if don't need any class attributes, then just doing staticmethod | 21:33 |
| harlowja | can make them all instance methods, but got in the habit of not doing that and using the right decorator | 21:33 |
| rharper | I see, you didn't explicitly need to reference anything in the instance s? | 21:33 |
| harlowja | nah | 21:33 |
| rharper | gotcha | 21:34 |
| rharper | w.r.t quoting, my reading on ConfigParser and INI is that no quoting should be needed; anything after the Variable= is included in the value ... | 21:34 |
| harlowja | hmmmm, i thought most of these sysconfig files were just 'sourced' into scripts | 21:34 |
| rharper | oh | 21:35 |
| rharper | hehe | 21:35 |
| rharper | I forgot | 21:35 |
| rharper | the bash scripts | 21:35 |
| rharper | you're right for sysconfig | 21:35 |
| harlowja | ya, i didn't think sysconfig stuff was very smart, lol | 21:36 |
| harlowja | and it all eventually gets parsed by some shell scripts, lol | 21:36 |
| harlowja | so without doing quoting things go bad (especially with whitespace) | 21:37 |
| harlowja | ie like https://gist.github.com/harlowja/5f9b3008d20d2668e8bcfc28eb491955#file-gistfile1-txt-L102 | 21:38 |
| harlowja | why it was done like this, who knows | 21:39 |
| rharper | yeah | 21:40 |
| rharper | I was just confused | 21:40 |
| rharper | sysconfig is huge legacy | 21:40 |
| harlowja | def | 21:40 |
| rharper | at the same time, the guts of ifupdown are ... harder to read than the in-line awk in sysconfig so no leg to stand on there | 21:41 |
| harlowja | lol | 21:45 |
| harlowja | ya, it's all one big mess imho | 21:46 |
| harlowja | guess thats why systemd :-P | 21:46 |
| rharper | one mess + second mess => systemd ? not sure what we expect then | 21:51 |
| rharper | magic ? | 21:51 |
| harlowja | :) | 21:51 |
| * rharper sprinkles some systemd magic around | 21:51 | |
| rharper | I do know that we'll now be told there _is_ one TRUE way | 21:51 |
| rharper | whether that's right or not; at least you have an answer | 21:51 |
| harlowja | ha | 21:51 |
| harlowja | ya | 21:51 |
| harlowja | mgagne if u get a chance can u look over https://gist.github.com/harlowja/d63a36de0b405d83be9bd3222a5454a7 | 22:10 |
| harlowja | its my creation, based off in part https://raw.githubusercontent.com/mgagne/cloud-init-fedora-pkg/epel7/cloud-init-0.7.5-network-info-support.patch + the refactored stuffs | 22:10 |
| harlowja | pretty sure its right, and u can just feed it a openstack json file and see what is output | 22:10 |
| harlowja | $ python sysconfig.py test.json | 22:10 |
| harlowja | for example | 22:10 |
| dmsimard | harlowja: neat, I'll look too. | 22:22 |
| harlowja | thx | 22:22 |
| harlowja | https://gist.github.com/harlowja/d63a36de0b405d83be9bd3222a5454a7#file-sysconfig-py-L406 == the main function that thinks get started from | 22:22 |
| dmsimard | harlowja: just to be sure, what version of cloud init am I supposed to be running ? | 22:26 |
| harlowja | a unmerged branch ;) | 22:27 |
| dmsimard | (for the imports) | 22:27 |
| dmsimard | ehhh, how do I set that up ? | 22:27 |
| harlowja | https://code.launchpad.net/~harlowja/cloud-init/cloud-init-net-refactor/+merge/293957 | 22:27 |
| harlowja | that one | 22:27 |
| dmsimard | can you zip it somewhere or something ? Afraid my bzr skills are pretty awesome (not) | 22:28 |
| harlowja | sure | 22:28 |
| harlowja | dmsimard actually u should just be able to run | 22:29 |
| harlowja | bzr branch lp:~harlowja/cloud-init/cloud-init-net-refactor cloud-init | 22:29 |
| harlowja | that will get u the branch | 22:29 |
| harlowja | (into a folder called 'cloud-init') | 22:30 |
| dmsimard | ok | 22:30 |
| dmsimard | let me see.. | 22:30 |
| * dmsimard installs bzr ... and shivers | 22:31 | |
| harlowja | lol | 22:32 |
| harlowja | try a few json files (from openstack); let me know if anything looks really crazy (it might be crazy) | 22:33 |
| harlowja | since things like https://git.fedorahosted.org/cgit/initscripts.git/plain/sysconfig/network-scripts/ifup-eth are crazy | 22:34 |
| dmsimard | harlowja: it's not crazy, it's empty :) | 22:34 |
| dmsimard | sec | 22:34 |
| harlowja | lol | 22:34 |
| harlowja | k | 22:34 |
| harlowja | running stuff like $ python sysconfig.py test.json will just output to stdout btw | 22:35 |
| harlowja | it doesn't write anywhere yet | 22:35 |
| dmsimard | harlowja: https://gist.github.com/dmsimard/d6eaa8b61f3c601d833b29ef430637da this is what I'm testing against | 22:35 |
| harlowja | k let me try | 22:35 |
| dmsimard | and this is my output: http://paste.openstack.org/show/498378/ | 22:35 |
| harlowja | yup, looks like that to me also, ha | 22:36 |
| harlowja | ok, let me see where things went here | 22:36 |
| dmsimard | I have the full config drive if you want, it may or may not be the one I think I may have provided you last time | 22:36 |
| harlowja | oh i know | 22:36 |
| harlowja | let me see, def seems like it should have more :-P | 22:37 |
| harlowja | one of these other format layers killing the interfaces there | 22:37 |
| harlowja | will figure that out | 22:37 |
| dmsimard | fwiw that's a mitaka config drive, if it's different or anything, no clue | 22:38 |
| harlowja | k | 22:38 |
| harlowja | u can try the example in https://bugs.launchpad.net/nova/+bug/1514082 | 22:39 |
| harlowja | i was trying that one also | 22:39 |
| harlowja | as i figure out which layer dropped the info in your file | 22:39 |
| dmsimard | harlowja: ah, yup, that one works fine | 22:40 |
| dmsimard | harlowja: the whole config drive is @ https://dmsimard.com/disk.config if you want to peek at it. | 22:41 |
| harlowja | np, not needed i think, just that one file is what matters here | 22:41 |
| dmsimard | you should add my network_data.json to your unit tests in test_network_config_conversions after you figure it out :) | 22:43 |
| harlowja | ya | 22:43 |
| harlowja | ya, ok, sooo might have to poke rharper on this one | 22:44 |
| harlowja | http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/cloudinit/net/network_state.py#L239 is puking on your date | 22:45 |
| harlowja | primarily because http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/cloudinit/net/network_state.py#L266 keys don't exist | 22:45 |
| harlowja | and in the openstack version, they aren't getting translated ... | 22:46 |
| dmsimard | I updated my gist to be pretty printed :p https://gist.github.com/dmsimard/d6eaa8b61f3c601d833b29ef430637da | 22:46 |
| harlowja | and i'm not quite sure what they should translate to/from | 22:46 |
| harlowja | it seems like that code is looking for bridge_interfaces and then interfaces (two sets of interfaces) | 22:46 |
| harlowja | where your example only has one :-P | 22:46 |
| harlowja | so what does it bridge to :-P | 22:46 |
| dmsimard | it's a vm on a linuxbridge environment | 22:47 |
| harlowja | right | 22:47 |
| dmsimard | the vm doesn't have to configure any bridge | 22:47 |
| dmsimard | it's done on the compute node | 22:47 |
| harlowja | sure | 22:48 |
| harlowja | but from my understanding of that code, its looking for a interface definitions of what to bridge to what else | 22:48 |
| harlowja | so i'm not sure the mapping of http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/cloudinit/net/network_state.py#L239 and what openstack provides (is data missing?) | 22:49 |
| harlowja | what happens is that your json file first goes into a intermediarty format that is '[{'subnets': [{u'routes': [{u'netmask': u'0.0.0.0', u'network': u'0.0.0.0', u'gateway': u'172.19.3.254'}], u'netmask': u'255.255.252.0', u'type': 'static', 'ipv4': True, 'address': u'172.19.1.34'}], 'name': u'tap1a81968a-79', 'mac_address': u'fa:16:3e:ed:9a:59', u'type': 'bridge', 'mtu': None}, {u'type': 'nameserver', u'address': u'172.19.0.12'}]' | 22:50 |
| dmsimard | So you have a network_data.json that's linuxbridge (mine) and the one you provided from the bug is from an ovs setup. It's a pretty good test to ensure both work well. | 22:50 |
| dmsimard | The OVS one you linked also has the notion of links | 22:50 |
| harlowja | which then gets processed by that code into another intermediary format and then that format is what the sysconfig.py stuff works on | 22:51 |
| harlowja | right, something is off/missing in the handle_bridge stuff here | 22:51 |
| dmsimard | The links are probably just interfaces on the compute node (bridge tap) and the networks are bridged to these links on the compute node | 22:51 |
| harlowja | ya, i'm gonna delegate to rharper on this one, unsure what the expected openstack translation is for that to correctly work | 22:53 |
| === natorious_ is now known as natorious | ||
| dmsimard | I'm off for the day, let me know if you find out and I'll poke at it some more .. I could even pitch a few VMs at it with the real thing :) | 22:53 |
| harlowja | ya, eventually we'll get there ;) | 22:54 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!