| === natorious is now known as zz_natorious | ||
| === zz_natorious is now known as natorious | ||
| === natorious is now known as zz_natorious | ||
| === zz_natorious is now known as natorious | ||
| === natorious is now known as zz_natorious | ||
| minfrin | Quick question - is it possible to run an additional cloud-init script at a later point during or after boot? The reason I ask is that Azure supports cloud-init, but has severe restrictions on template sizes that make cloud init impractical to use. Azure does support a thing called a "CustomScriptForLinux" that allows us to run a script downloaded from somewhere else, and what I want that... | 16:11 |
|---|---|---|
| minfrin | ...script to be is an additional cloud-init script. Is it possible to run additional cloud-init scripts after the initial one is run via some kind of command line option? | 16:11 |
| smoser | minfrin, you're limited by size i gather ? | 16:14 |
| smoser | the CustomData on azure can be gzip compressed inside there. that will buy you some. | 16:14 |
| smoser | also, you can use '#include' style to consume data from elsewhere. | 16:14 |
| smoser | see https://help.ubuntu.com/community/CloudInit | 16:15 |
| smoser | then your user-data can effectively be anything you can fit inside the content of a url | 16:15 |
| minfrin | Gzip is no good for us unfortunately, the thing that is blowing our size limit are the private keys for the machines which aren't compressible in any meaningful way that makes a difference to us. | 16:19 |
| minfrin | How do you secure data coming from an URL? | 16:19 |
| smoser | minfrin, as in put secret data there? or as in secure against MIM | 16:35 |
| smoser | it can read over https so that secures transmission | 16:35 |
| smoser | for secret data you can use '#include-once' with a url to a long one time hash | 16:35 |
| smoser | but if you want to run stuff after boot, you can | 16:35 |
| smoser | cloud-init single --frequency=always --name=<that-module-name> | 16:36 |
| minfrin | I tried out cloud-init single, but it forces us to choose a --name option, and I've interpreted that as meaning we are only able to run one single module in our additional cloud-init script. What I need to do is run all modules, is this possible? | 16:40 |
| minfrin | We're trying to find a solution that doesn't involve too many hacks given the number of hacks we've had to apply to work around Azure's size limits. :( | 16:40 |
| cbolt | smoser: is there a way to make cloud-init reformat a drive on every boot? | 16:48 |
| cbolt | currently getting an error stating the filesystem is already mounted | 16:48 |
| smoser | minfrin, you can run the stage again 'cloud-init init' or 'cloud-init config' | 16:48 |
| cbolt | first boot, it takes the unformatted drive and formats it properly. when i reboot the box it attempts to reformat it but mkfs.ext4 fails stating the drive is already mounted | 16:48 |
| smoser | cbolt, probably want/need to take it out of /etc/fstab . at least that could work. | 16:49 |
| smoser | you can probably configure 'mounts' to not automatically mount it. | 16:50 |
| smoser | minfrin, do you have sensitive data in the user-data ? | 16:50 |
| smoser | is that why you're adverse to urls ? | 16:50 |
| smoser | you could put the sensitive data (assuming it fits) into the body of the CustomData and #include large code hunks | 16:50 |
| cbolt | ok thanks, ill come up with a workaround for this | 16:51 |
| smoser | cbolt, http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/doc/examples/cloud-config.txt#L174 | 16:51 |
| smoser | i think if you feed it user-data that says 'mounts: [ephemeral0, null]' | 16:51 |
| smoser | then it wont getm ounted. and that shoudl fix your reboot issue | 16:51 |
| cbolt | yea but then ill need to come up with a solution for mounting it | 16:52 |
| smoser | oh. yeah. ok. | 16:53 |
| cbolt | really i shouldnt be having to force cloud-init to reformat the drive on every reboot | 16:53 |
| minfrin | Yes on the sensitive data - it's all private keys for the machines. | 16:53 |
| minfrin | If I was to add an additional cloud-config script, what command would I run? In other words: [additional-cloud-init-command] cloud-init-with-keys | 16:53 |
| cbolt | i need to fix the underlying issue | 16:53 |
| smoser | minfrin, what is the size limit ? | 16:54 |
| smoser | just really curious. i didnt' remember there was one. (initially there wasnt, thats how we implemented custom-data there without azure's knowledge :) | 16:54 |
| minfrin | To explain the background, Azure supports customData of up to 64kb in size. Their templates are however constrained to approx 320k in size before failing on deployment. We have 21 machines to deploy as a unit (and climbing), and a customData script of anything bigger than 7k causes the deployment to crash. | 16:55 |
| smoser | hm. | 16:55 |
| smoser | have you raised that to them ? | 16:56 |
| smoser | for "custom cloud-config script" above, what did you mean ? | 16:56 |
| smoser | you want to add a config module and run it ? | 16:56 |
| minfrin | Yes. Took them a while to figure out the cause was a size limit, we were getting "internal server error" for a while which told us nothing of what was wrong. | 16:56 |
| smoser | the 64kb sucks as a limit. | 16:58 |
| smoser | on ec2 its 64k | 16:58 |
| smoser | but its binary | 16:59 |
| smoser | is theirs 64kb of mime-encoded (so it can be shoved safely into xml body) ? | 16:59 |
| minfrin | We want to run cloud-init twice - once on boot using their normal process, and a second time at some point after boot using their "CustomScriptForLinux" feature, which while being URL based can be secured. "CustomScriptForLinux" allows us to specify a file to download (which is the cloud-init script in theory) and a command to run (which is in theory "cloud-init [options]"). | 16:59 |
| smoser | ok. i think maybe i udnerstand. | 17:00 |
| smoser | if you can run stuff, then | 17:00 |
| smoser | let azure boot, | 17:00 |
| smoser | run command taht does: rm -Rf /var/lib/cloud/ /etc/cloud/cloud.cfg.d/*azure* | 17:00 |
| smoser | then populates /var/lib/cloud/instance/nocloud/seed/ | 17:01 |
| smoser | or... possibly simpler. | 17:01 |
| smoser | let azure boot | 17:01 |
| smoser | hm.. | 17:01 |
| smoser | well, ^ above shoudl work. | 17:02 |
| minfrin | We're getting nowhere near the 64kb size limit, the limit we're hitting is the total size of their deployment template. In theory we're told it is 1MB - itself very small, but we're then told that a UTF8 to UTF16 halves the effective size of the template to 512k, and there are further unspecified limits we hit, bringing the size where we hit the wall to about 320k-ish. | 17:02 |
| smoser | ah.ok. | 17:02 |
| smoser | what is the user-data (what format) that you'd normally send to cloud-init ? | 17:02 |
| smoser | is it cloud-config ? | 17:02 |
| minfrin | Normally #cloud-init. It was all working great up to the point where we added the private key. | 17:03 |
| minfrin | Sorry, #cloud-config. | 17:03 |
| smoser | ok. if its all cloud-config i think you should be able to: | 17:05 |
| smoser | * let azure boot as it would | 17:06 |
| smoser | * run some program that does: | 17:06 |
| smoser | * rm -Rf /var/lib/cloud/instance/ | 17:06 |
| smoser | * put your #cloud-config data into /etc/cloud/cloud.cfg.d/your-data.cfg | 17:06 |
| smoser | * reboot or run the stages of cloud-init manually | 17:07 |
| minfrin | Let me give this a play and see what I am limited to - I still need to see exactly how the CustomScriptForLinux works and what restrictions are imposed on us. Thank you for confirming the cloud-init steps, I appreciate it. | 17:08 |
| smoser | yeah, i dont know how that works at all | 17:08 |
| smoser | Odd_Bloke, around ? | 17:32 |
| Odd_Bloke | smoser: In and out. | 17:33 |
| smoser | http://paste.ubuntu.com/11954733/ | 17:34 |
| smoser | does that make sense on top of https://review.openstack.org/203533 | 17:35 |
| openstackgerrit | Merged stackforge/cloud-init: Implement a DictRegistry. https://review.openstack.org/203094 | 17:40 |
| openstackgerrit | Merged stackforge/cloud-init: Use a registry to configure reporting handlers. https://review.openstack.org/203093 | 17:44 |
| openstackgerrit | Merged stackforge/cloud-init: Make reporting handlers configurable. https://review.openstack.org/203533 | 17:53 |
| Odd_Bloke | smoser: Without context, it looks like a pointless change... | 17:56 |
| smoser | ah. acutally never mind. you're right | 17:59 |
| smoser | i didnt'realize you were using event. in the getLogger call | 18:00 |
| smoser | so its not pointless its broken | 18:00 |
| Odd_Bloke | Oh, also true. :p | 18:00 |
| Odd_Bloke | Great review from me there. >.< | 18:00 |
| smoser | i was trying to avoid getLogger and join on every publishEvent | 18:00 |
| Odd_Bloke | Ah, I see. | 18:10 |
| === zz_natorious is now known as natorious | ||
| === natorious is now known as zz_natorious | ||
| smoser | Odd_Bloke, http://paste.ubuntu.com/11955467/ | 19:46 |
| smoser | any reason you use unittest and cloudinit.tests.TestCase ? | 19:47 |
| Odd_Bloke | smoser: Force of habit? (No. :p) | 19:48 |
| smoser | thats fine. was just looking at cloud-init 0.7 for this | 19:48 |
| * harlowja prefers 'cloudinit.tests.TestCase' | 20:08 | |
| harlowja | makes it easier to add new base functionality if needed | 20:08 |
| harlowja | and/or compat stuff | 20:08 |
| openstackgerrit | Scott Moser proposed stackforge/cloud-init: tests: use cloudinit.tests.TestCase everywhere https://review.openstack.org/206688 | 20:09 |
| smoser | harlowja, ^ | 20:09 |
| harlowja | cool | 20:09 |
| smoser | hey, you want to take over my "main" ? | 20:09 |
| harlowja | hmmmmmmmmm | 20:10 |
| smoser | https://review.openstack.org/#/c/202743/ | 20:10 |
| harlowja | one does not take over main, lol | 20:10 |
| harlowja | one does not just takeover main, lol | 20:10 |
| openstackgerrit | Scott Moser proposed stackforge/cloud-init: add cloud-init main https://review.openstack.org/202743 | 20:13 |
| smoser | why would flake8 blow up | 20:16 |
| smoser | not like my ordering ? | 20:16 |
| harlowja | smoser http://docs.openstack.org/developer/hacking/#imports | 20:22 |
| harlowja | 'Do not import objects, only modules' | 20:22 |
| harlowja | might complain, might not | 20:22 |
| harlowja | maybe we turned that check off, not sure | 20:22 |
| smoser | oh. | 20:22 |
| smoser | well, i couldnt get H306 to fail | 20:22 |
| smoser | harlowja, http://paste.ubuntu.com/11955830/ | 20:55 |
| smoser | thoughts ? | 20:55 |
| smoser | Odd_Bloke, ^ | 20:55 |
| smoser | i dont want to pas a handle around everywhere | 20:58 |
| Odd_Bloke | smoser: We could do a stack, but that's pretty brittle. | 21:10 |
| harlowja | hmmm | 21:11 |
| Odd_Bloke | smoser: I don't think there's going to be a good way to do this that won't (a) break unexpectedly, or (b) fail to expand to the next use case we think of. | 21:12 |
| harlowja | probably better to use a stack, put it in a thread-local variaible, then have most of this reporting be decorators and not explict calls imho, at least hide the calls to start/end all over | 21:12 |
| harlowja | but this is the cost of that level of tracing :-P | 21:12 |
| harlowja | *event emission* | 21:12 |
| Odd_Bloke | smoser: harlowja: Immediate example of a stack failing: parallel data-source discovery. | 21:12 |
| harlowja | yup | 21:12 |
| harlowja | thread-local stacks | 21:12 |
| Odd_Bloke | harlowja: If that's how we implement parallel discovery, maybe. | 21:13 |
| harlowja | depends how u want to do it, without all the async stuff, paralleism probably just easier via spawning threads imho | 21:13 |
| Odd_Bloke | But I don't know that I want event reporting to dictate how we parallelise the code which does the actual work. :p | 21:13 |
| harlowja | agreed | 21:14 |
| Odd_Bloke | Also, given that we expect the output of this to be computer-readable, we need it to be super-hard to break. | 21:14 |
| Odd_Bloke | Humans can cope with the occasional weird log message; computers, not so much. | 21:15 |
| harlowja | ya, got me, if u want a stack, u probably need a stack structure somewhere :-P | 21:16 |
| harlowja | *stack like output | 21:16 |
| harlowja | or u have to use the call-stack as the stack, not many other options, ha | 21:16 |
| Odd_Bloke | This also feels like we might actually only use it in one or two places, and it might be easier for those places to manage it. | 21:17 |
| Odd_Bloke | (e.g. all of those instances will _know_ that they are doing ds-search, so can just use that as their event name) | 21:17 |
| Odd_Bloke | s/instances/examples of calls/ | 21:18 |
| harlowja | sure | 21:18 |
| Odd_Bloke | So I also wonder if this is a YAGNI situation. | 21:18 |
| harlowja | possibly, lol | 21:18 |
| harlowja | https://github.com/jek/blinker doesn't seem to support stacks either | 21:18 |
| harlowja | so my guess maybe we overthinking it | 21:18 |
| harlowja | either does https://github.com/openstack/taskflow/blob/master/taskflow/types/notifier.py (my event like thing) | 21:20 |
| === zz_natorious is now known as natorious | ||
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!