| rharper | smoser: https://launchpad.net/~cloud-init-dev/+archive/ubuntu/daily/+packages is this the daily ppa? last publish was 8/18 -- was that the git switch over? | 13:45 |
|---|---|---|
| smoser | :-( | 13:47 |
| smoser | it has definitely built since git. i'll look. | 13:47 |
| smoser | :-( | 13:47 |
| rharper | I was going to get an updated core snap that points to cloud-init daily PPA and have it build core snap daily against that so we always have a fresh core.snap | 13:48 |
| smoser | https://code.launchpad.net/~cloud-init-dev/+recipe/cloud-init-daily | 13:49 |
| smoser | it sucks... that contents sometimes gets changed like randomly | 13:50 |
| smoser | 'bzr-builder' i think is what is killing it | 13:50 |
| smoser | there are no changes to bzr 'lp:cloud-init' | 13:50 |
| smoser | will look at what its supposed to have | 13:50 |
| rharper | huh | 13:51 |
| smoser | yeah, i'm not making it up. just now changed it to http://paste.ubuntu.com/23379003/ | 13:54 |
| smoser | click the green thing and it goes to bzr-builder | 13:54 |
| smoser | i'm going to delete it and hope recreating makes better | 13:55 |
| rharper | whoa | 13:55 |
| smoser | thought i'd seen a bug at some point | 13:55 |
| rharper | we should get powersj to have a jenkins job to boot lxc container, add this ppa and install it | 13:56 |
| * powersj likes that idea | 13:56 | |
| jgrimm | powersj, good morning! | 13:56 |
| powersj | Can do something similar to my daily ISO test and compare when a new one comes out too | 13:56 |
| powersj | morning :) | 13:56 |
| smoser | well, *not* boot an lxc container and install the thing | 13:57 |
| smoser | thats absically the first proof of integration test | 13:57 |
| smoser | wgrant in launchpad pointed me at https://bugs.launchpad.net/launchpad/+bug/1623924 with work around . so looking. | 13:57 |
| powersj | right, so we could have a daily integration test based on the daily ppa to run a subset of tests. | 13:58 |
| smoser | alright. building now. | 14:02 |
| smoser | rharper, thanks for noticing. | 14:02 |
| rharper | np | 14:06 |
| NerdyBiker | Hey all, does anyone know of any issues using Cloud-init chef function on a HVM type instance in AWS? As soon as we use a HVM instance, we get "Running chef (<module 'cloudinit.config.cc_chef' from '/usr/lib/python2.6/site-packages/cloudinit/config/cc_chef.pyc'>) failed" in the logs | 15:20 |
| smoser | NerdyBiker, more info ? | 15:20 |
| smoser | it works on non-hvm ? | 15:20 |
| NerdyBiker | yup works on PV types | 15:21 |
| smoser | distro ? cloud-init version ? | 15:21 |
| smoser | that is really odd | 15:21 |
| NerdyBiker | Oh sorry :D CentOS 6.5 | 15:21 |
| smoser | can you paste the cloud-init.log ? | 15:21 |
| NerdyBiker | version 0.7.5 cloud-init | 15:21 |
| smoser | or at least the stack trace | 15:21 |
| NerdyBiker | Perferred method of pasting? | 15:22 |
| smoser | pastebin other than pastebin.com :) | 15:23 |
| NerdyBiker | http://pastebin.com/kGLPkXZf | 15:23 |
| smoser | that was just rude :) | 15:23 |
| NerdyBiker | damn | 15:23 |
| NerdyBiker | haha! | 15:24 |
| NerdyBiker | terrible timing | 15:24 |
| smoser | thats fine. | 15:24 |
| smoser | its just ad-everywhere | 15:24 |
| smoser | /var/lib/cloud/instance/scripts/runcmd: line 3: chef-client: command not found | 15:24 |
| smoser | well that'd be it. | 15:24 |
| smoser | i guess your ami that yul're running for vhm does not have that package installed and the pvm does | 15:25 |
| NerdyBiker | the user data is set to install chef | 15:25 |
| smoser | user-data runs after package install | 15:25 |
| smoser | but cc_chef.py might install it. | 15:26 |
| smoser | looking | 15:26 |
| smoser | hm.. | 15:27 |
| NerdyBiker | https://gist.github.com/MattDevUK/064ece1a309616176695b9b67a866aa6 | 15:27 |
| smoser | actually, its your runcmd that is failing. | 15:27 |
| NerdyBiker | thats the user data we pass in | 15:27 |
| smoser | you have /var/log/cloud-init-output.log ? | 15:27 |
| NerdyBiker | that was the first pass | 15:27 |
| NerdyBiker | paste* | 15:27 |
| smoser | oh. yeah. | 15:28 |
| smoser | do you ahve a /var/log/cloud-init.log ? | 15:28 |
| NerdyBiker | in every other image we've used it on, this installs chef automatically using the params set in the chef block, then runs chef-client. | 15:28 |
| smoser | that one will have the stack trace and generally more info | 15:28 |
| NerdyBiker | sure :D | 15:28 |
| NerdyBiker | it's empty :( | 15:29 |
| NerdyBiker | any log levels we can change to try and get any content? | 15:34 |
| smoser | NerdyBiker, its a bug in c loud-init that leaves it empty. | 15:37 |
| smoser | can you get into that instance ? | 15:37 |
| NerdyBiker | yup, I'm on it | 15:37 |
| smoser | vi /etc/cloud/cloud.cfg.d/05* | 15:37 |
| smoser | the logging file | 15:37 |
| smoser | comment out | 15:37 |
| smoser | 15:37 | |
| smoser | - [ *log_base, *log_syslog ] | 15:37 |
| smoser | then reboot | 15:37 |
| smoser | maybe rm -Rf /var/lib/cloud /var/log/cloud* | 15:38 |
| smoser | and reboot | 15:38 |
| smoser | that should make it think its a first instance | 15:38 |
| NerdyBiker | ok so just leave &log_file in there? | 15:38 |
| NerdyBiker | ooo | 15:39 |
| NerdyBiker | no i see it now :D | 15:39 |
| NerdyBiker | booting now | 15:40 |
| NerdyBiker | so... it apparently worked that time | 15:43 |
| smoser | hm.. | 15:43 |
| smoser | maybe the omnibus installer is doing something int he background ? | 15:43 |
| smoser | and we're just racing it | 15:43 |
| NerdyBiker | the output after the reboot; https://gist.github.com/MattDevUK/9ad23f20e3570e82a1b2912e21d57ef0 | 15:46 |
| NerdyBiker | you can see where the "running chef failed" line was previously, now says installing Chef | 15:46 |
| NerdyBiker | on a positive note, that's the first time we've managed to get that to happen on a HVM image, out of the 10 or so we've tried. | 15:48 |
| smoser | follow bill's advice. | 15:49 |
| smoser | reboot! | 15:49 |
| smoser | and reinstall | 15:49 |
| NerdyBiker | so uninstallec chef, remvoed cloud files, rebooted and success again. wonder if it's timing when starting a fresh instance | 15:54 |
| NerdyBiker | @smoser just launched 2 fresh instances, both of them failed the cc_chef part on first run. | 16:01 |
| smoser | first boot io is very slow | 16:03 |
| smoser | but if the omnibus installer is backgrounding... thats a pita | 16:03 |
| smoser | and it should not do that | 16:03 |
| NerdyBiker | hm, just tried a basic restart, and it failed again | 16:04 |
| NerdyBiker | seems to be somehting timing related. cleared the /var/lib/cloud, rebooted and it worked. :( | 16:08 |
| NerdyBiker | damn that sucks | 16:08 |
| NerdyBiker | I'll tets out some other scenarios and see what comes up | 16:15 |
| NerdyBiker | test* | 16:16 |
| NerdyBiker | @smoser so just got the logging change into a base AMI, started an instance and it failed, this is in the cloud-init.log; https://gist.github.com/MattDevUK/9f58b9716a6914406536839a77cadcf0 | 16:30 |
| smoser | hm.. | 16:31 |
| NerdyBiker | there is also a stack for the cc_locale run, but don't think that impacts the chef issue as that fails een when chef succeeds | 16:32 |
| NerdyBiker | even* | 16:32 |
| smoser | right. | 16:33 |
| NerdyBiker | is there any more info I can give? Is this a valid problem? | 16:44 |
| smoser | NerdyBiker, i think probably valid | 18:27 |
| smoser | it seems likely to me to be related to omnibus | 18:27 |
| smoser | woudl be nice to try with newer cloud-init. | 18:28 |
| smoser | magicalChicken, https://code.launchpad.net/~wesley-wiedenmeier/cloud-init/+git/cloud-init/+merge/306731 fails rebase for me | 20:34 |
| magicalChicken | smoser: I'll look into it, I hadn't noticed issues before | 22:19 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!