| === harlowja_ is now known as harlowja_away | ||
| fish_ | hi | 10:44 |
|---|---|---|
| fish_ | I'm building a ubuntu AMI from scratch and use cloud-init. when finished building the AMI, /var/lib/upstart is there but after I boot it's gone and I don't have any upstat logs. wondering if this might be related to cloud-init | 10:45 |
| Odd_Bloke | fish_: Do you mean /var/log? | 10:46 |
| fish_ | Odd_Bloke: ehh sorry, yes sure /var/log/upstart | 10:48 |
| Odd_Bloke | fish_: I would be surprised if cloud-init were doing anything to /var/log/upstart, but smoser could probably tell you more. | 10:49 |
| fish_ | Odd_Bloke: hrm okay, any ideas what might cause it beside cloud-init? | 10:50 |
| fish_ | I found this: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/990102 - unfortunately (and a great example why important) it doesn't mention the commit this was fixed | 10:50 |
| Odd_Bloke | fish_: Well, that wasn't a cloud-init bug, it was a problem with the cloud images. | 10:52 |
| Odd_Bloke | fish_: Which was fixed ~3 years ago. :) | 10:53 |
| fish_ | Odd_Bloke: I know, I just ask here because cloud-init is the only thing I *expect* to change stuff on the system on first boot :) | 10:53 |
| fish_ | but yes, there is probably something else tampering with /var/log on first? boot | 10:53 |
| Odd_Bloke | fish_: Are you sure /var/log/upstart is actually included in your image? | 10:54 |
| fish_ | Odd_Bloke: I know the bug is old, but I suspect it's the same root cause as in my cause (they tried the same - building AMIs) | 10:54 |
| fish_ | Odd_Bloke: yes, I'm I double checked that | 10:54 |
| Odd_Bloke | fish_: Out of interest, why not build your image based on the images at cloud-images.ubuntu.com? | 10:55 |
| fish_ | Odd_Bloke: well, I want to actually build the images (vs starting a instance and snapshotting) - I actually wrote a few words about why (+how) here: http://5pi.de/2015/03/13/building-aws-amis-from-scratch/ but tl;dr | 10:56 |
| fish_ | I want the AMIs small and a clean separation between run and built-time | 10:57 |
| Odd_Bloke | fish_: Right, but why not take the images from there and modify them by mounting them? | 10:57 |
| fish_ | Odd_Bloke: that's an option.. deboostrap seemed more straight forward. do you think there are advantages in using the cloud-images directly? also worried about security updates because I want the amis to be pretty much immutable (well, at least I don't want to run config management etc on top to keep things up to date) | 10:59 |
| fish_ | but I just realize that I'm wrong, the resulting image indeed has no /var/log/upstart.. now I'm confused since I added a 'mkdir /var/log/upstart' to my build process which failed because it was there already.. looks like something during the build removes it | 11:00 |
| Odd_Bloke | fish_: Well, using the Ubuntu images would (probably) have saved you from this bug, for example. | 11:01 |
| Odd_Bloke | And you presumably have the same problem with security updates whoever is building the images? | 11:01 |
| Odd_Bloke | In your modification process, an "apt-get -y update; apt-get -y upgrade" would pull in any security updates the same as installing from scratch. | 11:02 |
| fish_ | hrm yes that's true.. well, I simply used debootstrap because it seemed like the right tool to use. but yeah, it's definitely a good option to use the cloud-images, will consider that | 11:03 |
| smoser | fish_, generally speaking, i really dont think you should build your own images. | 12:57 |
| smoser | any more than i think you should build your own kernel, or your own python or elibc. | 12:58 |
| smoser | you're certainly welcome to do it, and tools are availalbe to do so. but doing so means you get to re-discover bugs which are fixed. | 12:58 |
| smoser | the process that I would recommend, is | 12:59 |
| smoser | * download cloud image | 12:59 |
| smoser | * mount-image-callback --system-mounts --system-resolvconf $IMAGE -- chroot _MOUNTPOINT bash -s < your-update-script | 13:00 |
| smoser | and 'your-update-script' does things like: | 13:00 |
| smoser | apt-get update | 13:00 |
| smoser | apt-get install foo | 13:00 |
| smoser | apt-get clean | 13:00 |
| smoser | the above does require root, which is less than ideal, but | 13:01 |
| smoser | a.) if you don't trust the ubuntu images, you're kind of SOL anyway | 13:01 |
| smoser | b.) you can just run that in a VM to alleviate potential mount based attacks. | 13:01 |
| fish_ | smoser: yes, the more I think about that the more it makes sense. for now, things are working and I'm about to roll that out, but in the next iteration I'll definitely look into that | 13:15 |
| fish_ | I use a intermediate "base" image to avoid running a fresh deboostrap for every update anyways, so it should be fairely easy to use a cloud-image instead | 13:16 |
| smoser | fwiw, the maas-images build process does essentially the above. | 13:20 |
| === zz_gondoi is now known as gondoi | ||
| === gondoi is now known as zz_gondoi | ||
| === harlowja_away is now known as harlowja_ | ||
| harlowja_ | claudiupopa i think we might have to remove the channel from https://review.openstack.org/#/c/169293/ | 16:46 |
| harlowja_ | its gonna be hard to get an operator in this channel without kicking everyone out (which nobody has the permission to do) | 16:46 |
| harlowja_ | so might have to just skip that part | 16:46 |
| harlowja_ | years ago i think we all forgot to setup this channel with an operator and its pretty hard to do it post-creation | 16:47 |
| harlowja_ | *afaik* | 16:49 |
| smoser | harlowja_, why didn't you ever add 'tar' to the write_files. and http:// | 16:52 |
| smoser | s/you/me/ | 16:52 |
| harlowja_ | hmmm | 16:52 |
| harlowja_ | or was that me? | 16:52 |
| harlowja_ | idk | 16:52 |
| harlowja_ | haha | 16:52 |
| smoser | ie, would be nice to have write_files either read content from a url | 16:52 |
| smoser | or read a tarball from url and extract it into a target dir. | 16:53 |
| * harlowja_ looking | 16:53 | |
| smoser | http://paste.ubuntu.com/10691441/ | 16:53 |
| smoser | that is what i have, but having those 2 big blobs as 'path: http://' would have been nice. | 16:54 |
| smoser | and then also: | 16:54 |
| smoser | path: http:// | 16:54 |
| smoser | format: tar | 16:54 |
| smoser | extract-dir: /writable/user-data/cloud-init | 16:54 |
| smoser | or something | 16:54 |
| harlowja_ | hmmm, ya why did we do that, ha | 16:54 |
| harlowja_ | msg: ":::::: Hi Mom :::::::" | 16:56 |
| harlowja_ | lol | 16:56 |
| harlowja_ | hmmmm | 16:56 |
| harlowja_ | seems like we should just do that... (allow url stufF) | 16:57 |
| harlowja_ | and tar | 16:57 |
| harlowja_ | guess maybe we just didn't think of it ? | 16:57 |
| smoser | yeah, i think we just didnt. | 17:09 |
| smoser | the inline makes sense. | 17:09 |
| smoser | as you may not have networking at that point. | 17:09 |
| smoser | but if you do have networking, then http:// makes good sense. | 17:09 |
| smoser | and tar is just a nice archive format :) | 17:09 |
| harlowja_ | :-P | 17:17 |
| harlowja_ | shall i code that up or u want to? | 17:17 |
| harlowja_ | pop out the codes | 17:17 |
| harlowja_ | lol | 17:17 |
| harlowja_ | smoser ^ | 17:37 |
| harlowja_ | smoser https://code.launchpad.net/~harlowja/cloud-init/write-files-fetch-from-somewhere/+merge/254816 | 19:12 |
| harlowja_ | ok thats part 1 (no tar) | 19:12 |
| harlowja_ | should be mostly ok, ha | 19:12 |
| smoser | python3... | 19:13 |
| harlowja_ | ya, durn it | 19:13 |
| smoser | pre_content comes as binary... pretty sure (or we ant to make sure it is) | 19:13 |
| smoser | and then we want to write content without conversion | 19:13 |
| harlowja_ | will make sure | 19:14 |
| harlowja_ | load_tfile_or_url does that i guess | 19:14 |
| smoser | tfile loads text | 19:14 |
| smoser | ew want blobs of unadulterated raw data | 19:14 |
| harlowja_ | kk, let me check here | 19:14 |
| harlowja_ | stupid stff | 19:14 |
| harlowja_ | lol | 19:14 |
| harlowja_ | *stuff | 19:14 |
| * harlowja_ goes and builds up my 3.4 venv | 19:15 | |
| smoser | harlowja_, see why i said i hate pypi | 19:19 |
| harlowja_ | lol | 19:19 |
| harlowja_ | ya | 19:19 |
| smoser | oh darn, some dude's cable modem is out | 19:19 |
| smoser | pypi fail | 19:19 |
| harlowja_ | lol | 19:19 |
| harlowja_ | ok dokie; added some tests and stuff; seems to work as expected | 19:47 |
| harlowja_ | at least under basic tests | 19:48 |
| harlowja_ | smoser check that out if u want | 19:51 |
| smoser | harlowja_, thanks. | 19:57 |
| harlowja_ | np | 19:58 |
| Odd_Bloke | smoser: Am I right in thinking that vendor-data simply isn't a thing in the version of cloud-init in precise? | 20:19 |
| smoser | right . not at all. | 20:28 |
| smoser | Odd_Bloke, i'd sponsor an sru | 20:28 |
| harlowja_ | precise is 12.04 right? | 20:28 |
| Odd_Bloke | harlowja_: Yeah. | 20:28 |
| harlowja_ | k, can't ever remember the codenames, lol | 20:28 |
| === harlowja_ is now known as harlowja_away | ||
| tmclaugh[work] | I want to set hostname to the aws instance ID. Is there a var available for doing that ot do I have to use some commands during bootcmd to handle this? | 22:07 |
| === harlowja_away is now known as harlowja_ | ||
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!