[01:57] <patcable> question -- if I want to start a particular service before cloud-init in upstart, is there one good service to pick for that purpose?
[01:57] <patcable> there's lots of options. should I just or them all?
[07:34] <Odd_Bloke> patcable: cloud-init-local should do the trick, I think.
[08:39] <NerdyBiker> Hey guys, getting an issue when using the cloud-init stuff on EC2, finding that the user running the script doens't have knowledge of their HOME dir... not sure if that's expected or I'm missing somehting.
[08:46] <Odd_Bloke> NerdyBiker: cloud-init has a few different ways of running a script; could you paste the user-data you're passing?
[08:48] <NerdyBiker> I'll get it now. The main bit is a chef run and that's the bit failing. somewhere in that process it requires the "HOME" environment varibale but that gets output as blank.
[08:51] <Odd_Bloke> chef itself requires it, or your chef recipes?
[08:51] <Odd_Bloke> NerdyBiker: (A copy of the error you're seeing would be good too :)
[08:56] <NerdyBiker> Odd_Bloke one of the chef recipes requires the HOME to be set, here's a gist with the parts and brief explaination that should help https://gist.github.com/MattDevUK/7d4a7a52775a30706719
[08:59] <Odd_Bloke> NerdyBiker: So my first reaction is that this is a bug in your Chef scripts rather than in cloud-init; but that might just be because then I don't have to do anything to fix it. ;)
[09:01] <NerdyBiker> Odd_Bloke that might be the case, but this is the first time using cloud-init but that recipe is used on fresh boxes using the same AMI as this machine all the time. the echo at the beginning of the user-data outputs "home is set to ''" :(
[09:02] <Odd_Bloke> NerdyBiker: I'm not especially familiar with this part of cloud-init, let me have a dig through the source and see what I find. :)
[09:03] <NerdyBiker> it works fine if you log into the machine manually and run the chef-client. but then I'd expect if it was a cloud-init issue, it would've cropped up all the time before as I'd expect a lot of people to require the HOME dir when performing script sons tart up :P
[09:03] <NerdyBiker> scripts on startup*
[09:05] <Odd_Bloke> NerdyBiker: So it looks to me like cloud-init just runs scripts with the environment that it starts in itself.
[09:05] <Odd_Bloke> NerdyBiker: So the problem you're hitting is that (upstart|systemd|...) doesn't set HOME in the environment of things that it runs.
[09:05] <Odd_Bloke> s/problem/problem I think/
[09:06] <Odd_Bloke> NerdyBiker: See http://askubuntu.com/a/394330/141343 for a bit more info.
[09:08] <NerdyBiker> Odd_Bloke so it seems the issue is the user that executes the cloud-init hasn't tehcnically "Logged in" so their home would not be set? If I'm reading that right
[09:08] <Odd_Bloke> NerdyBiker: Yep, precisely.
[09:09] <Odd_Bloke> NerdyBiker: So I think you'll have to set HOME yourself; changing cloud-init to set it is probably opening a bag of worms.
[09:09] <Odd_Bloke> Or perhaps I mean a bag of bugs. :p
[09:10] <NerdyBiker> Odd_Bloke That's as I thought. so either we code around that, or possibly ask the aws guys how/why it's not being logged in properly.
[09:10] <NerdyBiker> :P
[09:10] <Odd_Bloke> NerdyBiker: It shouldn't be logged-in; it's run at startup by the init system.
[09:10] <NerdyBiker> Ah yes,
[09:11] <Odd_Bloke> (I assume; obviously I don't know what AMI you're using :p)
[09:11] <NerdyBiker> yeah that sounds right, I thought it did some from of log in tuil I asked in ##aws today and they said it's just a stratup script that pulls the user-data in.
[09:12] <NerdyBiker> Ah well, we'll work around it then :D Thank you for your help.
[09:12] <Odd_Bloke> No worries, happy to help.
[09:13] <Odd_Bloke> NerdyBiker: It is worth noting that you might be able to do what you want with the more complex form of user-data.
[09:13] <NerdyBiker> Odd_Bloke oh? I'm listening :D
[09:15] <Odd_Bloke> NerdyBiker: So I don't know a great deal about the Chef side of it, but http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/doc/examples/cloud-config-chef.txt is an example user-data which does cheffy things.
[09:16] <NerdyBiker> Odd_Bloke new to cloud-init so not familair with it all yet, will take a look. Thanks!
[09:16] <Odd_Bloke> NerdyBiker: And a bit more info at http://cloudinit.readthedocs.org/en/latest/topics/modules.html#chef
[09:18] <NerdyBiker> Odd_Bloke Ah, saw the first one before, but thought it was a config file due to the yaml. Didn't realise I could pass a yaml file as user-data.
[09:21] <Odd_Bloke> NerdyBiker: Yeah, the key is the #cloud-config at the start; that tells cloud-init to process it as config rather than a shell script.
[09:21] <Odd_Bloke> You can also do multi-part data, which would allow you to do the YAML config for some stuff and still pass in a script to be executed.
[09:24] <NerdyBiker> Odd_Bloke Awesome, think I've got enough to go on from now, you've been super helpful :D
[09:33] <Odd_Bloke> :)
[09:50] <NerdyBiker> Odd_Bloke looking at the cc_chef module, would you know if it's possible to define which version of Chef to install? I can't seem to find anything referencing a version number anywhere
[09:51] <NerdyBiker> well, apart from the gem method
[09:53] <Odd_Bloke> NerdyBiker: I don't think it is; which method were you looking to use?
[09:53] <NerdyBiker> omnibus ideally
[09:55] <Odd_Bloke> NerdyBiker: https://bugs.launchpad.net/cloud-init/+bug/1462693
[09:57] <NerdyBiker> Odd_Bloke awesome :D Should've looked there first I guess
[09:59] <Odd_Bloke> NerdyBiker: That looks like it'd be fairly easy to fix, if you wanted to have a go. ;)
[10:04] <NerdyBiker> Odd_Bloke Would be a way to get my head around how some it works and stop asking silly questions for sure. :P
[10:06] <Odd_Bloke> NerdyBiker: You haven't asked a single silly question thus far!
[10:16] <NerdyBiker> Odd_Bloke that's good
[11:25] <patcable> Odd_Bloke: hmm, thanks. I did try to stick it before cloud-init-local but something else was going on. I may need to make sure I have the same start condition (I started on filesystem and cloud-init-local starting, but maybe i need to do runlevel & cloud-init-local)
[11:26] <Odd_Bloke> patcable: My head is too much in systemd-land for me to be much more help.
[11:26] <Odd_Bloke> patcable: smoser might be able to help you out, he's good like that.
[11:27] <patcable> Odd_Bloke: that makes sense; a demo of something I'm doing is using 14.10 right now. just bumping from upstart to systemd at least wont be as bad
[11:28] <Odd_Bloke> patcable: Utopic's EOL is today (https://wiki.ubuntu.com/Releases), so that's probably worth doing. :p
[11:28] <patcable> ha
[11:28] <patcable> we basically wrote a way to send encrypted data to cloud init so you could put private data into a user script, but it relies on a key being present somewhere before cloud-init is invoked
[11:30] <Odd_Bloke> Oh, cool.
[11:31] <Odd_Bloke> patcable: So how do you get the key?
[11:31] <NerdyBiker> Odd_Bloke just to let you know, changed our user-data to use the cc_chef module, its cleaner but still stops at the same spot with the HOME dir, so going to have to manually set that variable as thought before :P
[11:32] <Odd_Bloke> NerdyBiker: Great!  (Except for the HOME bit :p)
[13:01] <patcable> Odd_Bloke: it's complicated. there's a process that is able to pull half the key from a verifier (that ensures the state of a trusted platform module is a okay) and the other half from openstack. The crypto folk I work with have a much better explanation, heh
[13:04] <Odd_Bloke> patcable: Fair enough. :)
[14:04] <smoser> patcable, running upstart 'start on starting cloud-init-local' should get you run before cloud-init's first job
[14:04] <smoser> but you're basically guaranteed to not have network then
[14:04] <smoser> which is probably not what you want
[14:05] <smoser> start on starting cloud-init
[14:05] <smoser> would guarantee you network, and in ec2 is what actually does things (as there is no local datasource for ec2, theirs requires metadta network accesss)
[14:05] <patcable> ahh ok
[14:06] <smoser> and above, you're not "basically guaranteed", you shoudl be guaranteed to have no network.
[14:06] <patcable> gotcha
[16:03] <patcable> smoser: though, we're using configdrive since we're in an openstack environment
[16:03] <patcable> smoser: so I imagine we could still do what we want before cloud-init starts, though we'd have to work through the details later if we're going to port this to an EC2 like environment