[00:38] <flaccid> smoser: http://dpaste.org/Se9H/ only a couple of things - Couldn't get a file descriptor referring to the console; error: unexpectedly disconnected from boot status daemon; FATAL: Could not load /lib/modules/2.6.32-305-ec2/modules.dep: No such file or directory (need to work out best way to install these on squeeze/debian);
[00:38] <flaccid> and Waiting for /dev to be fully populated...udevd-work[258]: kernel-provided name 'evtchn' and NAME= 'xen/evtchn' disagree, please use SYMLINK+= or change the kernel to provide the proper name
[00:38] <flaccid> and Setting kernel variables ...error: "net.ipv6.bindv6only" is an unknown key
[00:39] <flaccid> most of the above is kernel related. smoser i just need to address/fix the above and all should be good. if you could also advise what the latest actual kernel is that would be great
[00:39] <smoser> "actual kernel"
[00:40] <smoser> all the ubuntu published items conform to https://wiki.ubuntu.com/UEC/Images/NamingConvention
[00:40] <flaccid> yes
[00:40] <smoser> (or close to it)
[00:40] <smoser> for released kernels, they'll have ubuntu-images/<...> in their name (per describe-images)
[00:40] <smoser> er.. for kernels it will be
[00:40] <flaccid> yeah so what is the latest kernel id ?
[00:41] <smoser> well, for us-east-1, and lucid (released) that would be:
[00:41] <smoser> 099720109477/ubuntu-kernels/ubuntu-lucid-amd64-linux-image-2.6.32-305-ec2-v-2.6.32-305.9-kernel
[00:41] <smoser> aki-0b4aa462
[00:42] <flaccid> is there someone that lists these ?
[00:42] <smoser> i have considered putting kernels in the data at http://uec-images.ubuntu.com/query/
[00:42] <smoser> but largely, we don't consider kernels aki to be a product as much as a side effect of the image
[00:43] <smoser> that said, http://uec-images.ubuntu.com/query/lucid/server/released.current.txt
[00:43] <smoser> will have a list of the latest ami
[00:43] <smoser> and their assicoated aki
[00:43] <flaccid> thanks that helps a lot
[00:44] <smoser> there is 'daily.current.txt' there also
[00:44] <flaccid> now back to the errors if you don't mind helping?
[00:44] <smoser> and maverick ...
[00:44] <smoser> so lots to chose from
[00:44] <smoser> :)
[00:45] <smoser> i'vve seen the "Couldn't get a file descriptor" error before, but dont really recall what that was about.
[00:45] <smoser> fwiw, jjohansen is the kernel team member for this ec2
[00:49] <smoser> i'm looking at some logs
[00:49] <smoser> we see the "Could'nt get a file descriptor" on karmic currently
[00:49] <flaccid> cool thanks
[00:49] <smoser> and on lucid prior to alpha2
[00:49] <smoser> err.. prior to alpha3
[00:50] <flaccid> and would also like your advise on installing/enabling the kernel mods from the ubuntu package in debian and if this is an ok practice etc.
[00:50] <smoser> where did you get the ramdisk ?
[00:50] <smoser> or am i mistaken:
[00:50] <smoser> Begin: Loading essential drivers... ...
[00:50] <smoser> Done.
[00:50] <smoser> Begin: Running /scripts/init-premount ...
[00:50] <flaccid> its ubuntu lucid i386 linux image 2 6 32 301 ec2 v 2 6 32 301 4 ramdisk 20100114 1
[00:50] <flaccid> ari-3f2d7c7a
[00:50] <smoser> yeah, so we dont use ramdisk anymore
[00:51] <flaccid> right so set to default
[00:51] <smoser> set to default ?
[00:51] <flaccid> i'm assuming default=none now
[00:52] <flaccid> i'll relaunch with that
[00:52] <smoser> yeah, it probably wont hurt or affect it.
[00:52] <smoser> that has been my experience.  the ramdisk doesn't hurt anything. just doesn't help.
[00:53] <smoser> regarding installing ubuntu packages, are you asking from licensing perspective ?
[00:53] <smoser> those modules come from the packages... our ec2 kernels are normal kernle builds and have normal packages.
[00:54] <smoser> https://launchpad.net/ubuntu/+source/linux-ec2
[00:54] <smoser> i dont know how well those would install onto a debian
[00:54] <flaccid> more from an operational point of view. this is to address FATAL: Could not load /lib/modules/2.6.32-305-ec2/modules.dep: No such file or directory
[00:54] <flaccid> so yes the question is regarding installing them on debian
[00:55] <flaccid> see because the amazon kernels are not working with debian, i am having to use the ubuntu kernels
[00:56] <smoser> yeah. so i dont know whether or not the packages there will "just work" (ie install) on debian or not.
[00:56] <smoser> but almost certainly a dpkg -x linux-image-*.deb would get you the /lib/modules/ tha tyou need
[00:56] <flaccid> well i would either have to use the lucid repos briefly and install via apt or manually install the package and its deps via dpkg
[00:57] <flaccid> well the linux-image-* pkgs have deps..
[00:59] <flaccid> ok well no ramdisk fixed that
[00:59] <flaccid> zcat: invalid magic - another small one to address
[00:59] <flaccid> i mean no ramdisk fixed the console descriptor errors
[01:10] <flaccid> well it still defaulted to ubuntu lucid i386 linux image 2 6 32 301 ec2 v 2 6 32 301 4 ramdisk 20100108 smoser0
[01:10] <flaccid> that must be tied to the kernel
[01:13] <flaccid> wb
[01:43] <flaccid> smoser: if i do a reboot and i have installed these modules, this should remove the FATAL: Could not load /lib/modules/2.6.32-305-ec2/modules.dep: No such file or directory message right?
[01:44] <smoser> well maybe
[01:44] <smoser> you may be getting that from the ramdisk
[01:44] <smoser> at least i'm fairly certain that ramdisks do 'modprobe' which will result in a message like that.
[01:45] <flaccid> but if the file is there, it should not return No such file or directory right?
[01:45] <smoser> are you using a ramdisk ?
[01:45] <flaccid> this kernel appears to default to ubuntu lucid i386 linux image 2 6 32 301 ec2 v 2 6 32 301 4 ramdisk 20100108 smoser0
[01:46] <smoser> if you are using a ramdisk,and it was built with a differnet kernel then it won't have that /lib/modules/<kernel-ver> and it will try to modprobe
[01:46] <flaccid> aki-296f3e6c
[01:46] <smoser> kernels dont default to a ramdisk
[01:46] <smoser> this is something that sucks.
[01:47] <flaccid> oh but an image defaults to a ramdisk?
[01:47] <smoser> if you registered an image with a ramdisk and kernel, then you are not able (that I know of) to boot without a ramdisk or kernel
[01:47] <smoser> and, actually, it goes back step, since you can give ramdisk and kernel to the bundle command
[01:47] <flaccid> ok so ramdisk is bogus, no need to build image with a ramdisk value?
[01:47] <smoser> if you've done that, then there is no way to specify "None'
[01:47] <smoser> right
[01:48] <smoser> then, you can (if you wanted) specify --ramdisk <ari-abcdefg>
[01:48] <smoser> but there is no --ramdisk <none>
[01:48] <flaccid> sweet then i'll omit ramdisk
[01:48] <smoser> right. thats a superset of the functionality, but then, if you ever wanted a ramdisk, the user has to specify one, which just isn't very friendly
[01:49] <flaccid> if ec2-bundle-vol rcvs --kernel --ramdisk --generate-fstab will it simply ignore the no param values
[01:49] <smoser> just do not pass --ramdisk
[01:49] <smoser> or --kernel
[01:49] <smoser> on bundle or on register
[01:50] <smoser> fwiw, there are scripts 'uec-publish-image' that you could use that, in my opinion are more friendly than ec2-bundle, ec2-upload, ec2-register
[01:50] <flaccid> well i still want a default kernel
[01:51] <smoser> http://manpages.ubuntu.com/manpages/lucid/man1/uec-publish-image.1.html
[01:51] <smoser> right.
[01:51] <smoser> i *think* (you'd have to test this)
[01:51] <smoser> that you can bundle without anything
[01:51] <smoser> and then add --kernel when you register the image
[01:51] <smoser> which, in this case, would allow you to avoid re-bundling
[01:52] <flaccid> thats cool i'll test that
[01:54] <flaccid> hmm it doesn't specify params http://docs.amazonwebservices.com/AmazonEC2/dg/2007-01-03/CLTRG-register.html
[02:23] <smoser> see the date in that url
[02:24] <smoser> http://docs.amazonwebservices.com/AWSEC2/latest/CommandLineReference/index.html?ApiReference-cmd-RegisterImage.html
[02:24] <smoser> is what you want
[02:26] <flaccid> oh yeah thanks, eek on google!
[02:27] <flaccid> thanks muchly :)
[02:27] <flaccid> standby, i'll have some results soon
[02:27] <flaccid> 1 is building now
[02:30] <flaccid> the good thing i've noticed so far is that these AMIs i'm creating work fine with the lucid kernel and boot up really fast
[02:35] <flaccid> hmm that doc doesn't mention -K and -C options yet i am using them currently..
[03:01] <flaccid> Required option '-K, --private-key KEY' missing (-h for usage)
[03:01] <flaccid> heh its not in their doc
[03:51] <flaccid> ah must be in http://docs.amazonwebservices.com/AWSEC2/latest/CommandLineReference/index.html?ApiReference-cmd-RegisterImage.html
[03:51] <flaccid> err http://docs.amazonwebservices.com/AWSEC2/latest/CommandLineReference/index.html?CLTRG-common-args-api.html
[04:11] <flaccid> Unable to read instance meta-data for ramdisk-id
[04:11] <flaccid> Unable to read instance meta-data for product-codes
[04:12] <flaccid> ^^ from ec2-bundle-vol. what should i do about this ?
[04:41] <flaccid> smoser: built and registered with no ramdisk, a launch with default assigns ubuntu lucid i386 linux image 2 6 32 301 ec2 v 2 6 32 301 4 ramdisk 20100108 smoser0
[04:41] <flaccid> the default kernel being assigned is ubuntu-kernels-testing/ubuntu-lucid-i386-linux-image-2.6.32-306-ec2-v-2.6.32-306.11-kernel
[04:42] <smoser> those errors don't surprise me.
[04:42] <flaccid> i guess we'll see in console output if it is wanting 306 or 301 modules that will determine if the ramdisk is asking for it or not
[04:42] <flaccid> yes but i want to get rid of those errors :)
[04:42] <smoser> no, the "unable to read instance metadat" errors from bundle-vol
[04:43] <smoser> they dont surprise me
[04:43] <flaccid> yeah i want to fix those..
[04:43] <smoser> bundle-vol is a yucky hack. you shouldn't rely on it.
[04:43] <flaccid> well thats what aws publishes..
[04:43] <smoser> bundle-vol basically has to "un-boot" a system.
[04:44] <smoser> which is just simply going to require a bunch of distribution/OS specific HACKs
[04:45] <smoser> its much cleaner to use debootstrap and bundle-img
[04:45] <flaccid> well FATAL: Could not load /lib/modules/2.6.32-306-ec2/modules.dep: No such file or directory. so its the kernel that wants it. i'll extra the mods in this build
[04:45] <flaccid> where is bundle-img ?
[04:45] <flaccid> i am using debootstrap
[04:46] <smoser> ec2-ami-tools
[04:47] <flaccid> oh you mean ec2-bundle-img
[04:47] <smoser> yeah
[04:47] <smoser> can you give me output of describe-istances for the image you booted ?
[04:47] <smoser> i think you booted with a ramdisk.
[04:47] <smoser> or, you dont 'have those modules in the root filesystem
[04:47] <smoser> ec2-describe-instances i-abcdefg
[04:49] <flaccid> for some reason that returns Client.InvalidInstanceID.NotFound: The instance ID 'i-ba897dfe' does not exist
[04:49] <flaccid> yet its running
[04:50] <flaccid> i didn't include the modules in that build however
[04:50] <flaccid> rightscale dashboard is indicating use of ubuntu lucid i386 linux image 2 6 32 301 ec2 v 2 6 32 301 4 ramdisk 20100108 smoser0
[04:51] <smoser> it maybe that rightscale is buggy
[04:51] <smoser> if you did not bundle with '--ramdisk'
[04:51] <smoser> and did not register with '--ramdisk'
[04:52] <smoser> then you should not have a kernel
[04:52] <smoser> err... you should not have a ramdisk
[04:52] <smoser> it is possible that rightscale is being nice, and trying to use that one, in which case, that really sucks.
[04:52] <smoser> but describe-instances certinaly should be able to give you output
[04:52] <smoser> what about describe-images of the ami
[04:54] <flaccid> that returns does not exist as well. no idea why
[04:54] <flaccid> hmm are these region specific?
[04:55] <flaccid> i think its only returning us-east when i need us-west
[05:00] <smoser> yes. all of it is region specific
[05:00] <smoser> each region is its own "cloud". the only data they share is your account id
[05:00] <flaccid> nps
[05:02] <flaccid> this is the image
[05:02] <flaccid> IMAGE	ami-b5b8e9f0	041819229125/RightImage-Debian-squeeze-20100618033019-i386-v5.4.6	041819229125	available	private		i386	machine	aki-3197c674			instance-store
[05:03] <flaccid> this is the instance
[05:03] <flaccid> INSTANCE	i-ba897dfe	ami-e7b6e7a2	ec2-204-236-179-7.us-west-1.compute.amazonaws.com	ip-10-162-143-68.us-west-1.compute.internal	running	CJF: AWS US-West (testing)	0		m1.small	2010-06-18T03:39:43+0000	us-west-1b	aki-c18edf84	ari-8f2978ca		monitoring-disabled	204.236.179.7	10.162.143.68			instance-store	
[05:04] <flaccid> so i guess either rightscale is allocating ari-8f2978ca or ec2 (likely rightscale)
[05:06] <flaccid> can you help confirm that there is no such thing as a 'default' ramdisk when registering or launching?
[05:07] <flaccid> we seem to be using 'default' with a reserved ari then..
[05:07] <smoser> yep
[05:08] <smoser> its rightscale is being "nice"
[05:08] <smoser> well, when you register or bundle and pass '--ramdisk' that gives you a "default ramdisk"
[05:08] <smoser> and if you do that, you can't ever run an instance without a ramdisk.
[05:09] <smoser> but, the describe-images info above shows that the image has no ramdisk associated with it.
[05:09] <smoser> which means either
[05:09] <smoser> a.) the user must provide one in run-instances
[05:09] <smoser> b.) there will not be one used
[05:09] <flaccid> that would of came up in the describe-images but right? i took this out of the build process
[05:09] <flaccid> yep we are on same level
[05:09] <smoser> right. it would be shown in describe-images output
[05:09] <flaccid> i will raise a bug for the rightscale dashboard issue
[05:09] <smoser> look at any other one, you'll see an ari
[05:10] <smoser> fwiw, that 'smoser0' is a debug ramdisk :)
[05:10] <smoser> (with some interesting characteristics, it can process user data)
[05:10] <flaccid> hehe, looks like we chose your ramdisk image for the default... :)
[05:15] <smoser> our choice to not have ramdisks has thrown errors much furrther than i thought it would have
[05:16] <flaccid> there is issues with no ramdisk?
[05:19] <flaccid> is there any way to remove a debootstrap target dir? i.e. you can't remove /proc and /dev
[05:36] <smoser> you have to unmount proc and dev first
[05:36] <smoser> then remove
[05:36] <smoser> there are no issues with no ramdisk other than tools that expected a ramdisk
[05:37] <smoser> UEC had fits (Eucalyptus)
[05:37] <smoser> righscale apparently had fits
[05:37] <smoser> the expectation of an ari was just hard coded many places
[05:37] <smoser> oh yeah, other places.
[05:37] <smoser> landscape
[05:37] <smoser> and the canonical "image store" (part of UEC)
[05:38] <flaccid> i see
[05:39] <flaccid> ok now launching with a new image
[06:14] <flaccid> smoser: extracting from the .deb for those modules doesn't give the desired files
[06:14] <smoser> no?
[06:15] <smoser> you're right.sorry.
[06:15] <flaccid> i'll cp them across from the worker which is lucid
[06:16] <smoser> just the wrong package
[06:16] <smoser> the linux-ec2 is a metapackage
[06:16] <smoser> that depends on the right linux-image-2.6.32-305-ec2
[06:17] <flaccid> i'm extracting linux-image-2.6.32-305-ec2_2.6.32-305.9_i386.deb
[06:17] <flaccid> i tell ya what but it boots so fast
[06:17] <smoser> http://packages.ubuntu.com/hu/lucid/linux-image-2.6.32-305-ec2
[06:17] <smoser> yeah, thats what you want
[06:22] <flaccid> hopefully cp -R /lib/modules/2.6.32-305-ec2 "$DEBOOTSTRAP_TARGET_DIR"/lib/modules/ is good enough..
[06:28] <smoser> i'd think so
[06:29] <flaccid> i'm getting 1.2min startups
[06:30] <flaccid> 14.5mins to create and build an AMI
[07:00] <flaccid> smoser: only a few things left to address now, http://dpaste.org/5fAp/
[07:01] <smoser> start up is all over the map
[07:01] <smoser> what do you think is left ?
[07:01] <smoser> it looks pretty good
[07:03] <smoser> what do you mjean by "1.2min startups" ?
[07:03] <smoser> total time to login/ssh from what
[07:04] <smoser> flaccid,
[07:04] <flaccid> i think thats the boot time, not sure how its configured
[07:05] <flaccid> need to address the ' WARNING: Currently emulating unsupported memory accesses'
[07:05] <flaccid> there is no /lib/tls on that instance
[07:05] <flaccid> also Waiting for /dev to be fully populated...udevd-work[86]: kernel-provided name 'evtchn' and NAME= 'xen/evtchn' disagree, please use SYMLINK+= or change the kernel to provide the proper name
[07:06] <flaccid> and the few under cannot unset powersave mode
[07:06] <flaccid> and Setting kernel variables ...error: "net.ipv6.bindv6only" is an unknown key
[07:06] <flaccid> and maybe /build/buildd/linux-ec2-2.6.32/drivers/rtc/hctosys.c: unable to open rtc device (rtc0)
[07:06] <flaccid> also Setting kernel variables ...error: "net.ipv6.bindv6only" is an unknown key
[07:13] <flaccid> i'll build a lenny now
[07:37] <flaccid> lenny works but not with rightlink
[07:39] <smoser> i'm going to bed. good luck.
[07:39] <flaccid> thanks; cya later