#ubuntu-ec2 2009-08-10
<uvirtbot`> New bug: #411297 in ubuntu-on-ec2 "System locks up after a period of time" [Undecided,New] https://launchpad.net/bugs/411297
#ubuntu-ec2 2009-08-11
<agussman> In this post http://tinyurl.com/n3l9jz they reference an ALestic start up script that's used to fetch user data from Amazon.  Anyone know which startup script does that?
<rick_h_> agussman: I've seen some references to it like here: https://lists.ubuntu.com/archives/ec2/2009-April/001020.html
<rick_h_> I've not tested it, but seems there's something added to the default image init.d to pull it
<rick_h_> I tend to just scp a .sh script after the instance is up
<agussman> rick_h_:thanks for the info, i'll check it oout
<erichammond> agussman: http://ec2-run-user-data.notlong.com
<erichammond> agussman: This is the startup script used to run the user-data in the images on http://alestic.com
<erichammond> http://alestic-user-data.notlong.com/
<erichammond> And that is an article which describes its use.
<agussman> erichammond: thanks for the info!  Where does the ec2-run-user-data on the image?
<erichammond> agussman: /etc/init.d/ with appropriate links from /etc/rc?.d/
<erichammond> The Ubuntu images built by Canonical have a similar startup script.
<agussman> Hmm... I'm not seeing it in /etc/init.d.  Is it called ec2-run-user-data?
<erichammond> What AMI id are you running?
<agussman> It was an AMI that I downloaded and am running locally as a Xen filesystem image
<agussman> well, not running locally, I'm running it under Nimbus, but I don't think that matters
<agussman> It's based off of the Aug3rd or Aug4th AMI for 9.04, 32bit
<erichammond> If you're using an AMI built by Canonical, the startup scripts were completely rewritten and have different names and locations.
<erichammond> Start in /etc/init.d/ec2-init
<agussman> erichammond: Thanks, I found it.  It was where you said it was originally, I didn't realize I was logged into the wrong machine :/
#ubuntu-ec2 2009-08-12
<LyricHartley> stupid question, but I just don't see it....what is the syntax for the AWS_CREDENTIAL_FILE
<LyricHartley> AWSAccessKeyId works for one
<LyricHartley> but can't figure out the other
<LyricHartley> tried versions of AWSSecretAccessKey
<LyricHartley> don't see in docs
<LyricHartley> ?
<LyricHartley> appears to be AWSSecretKey
<uvirtbot`> New bug: #412143 in ubuntu-on-ec2 "ec2-init init script has bad Required-Start header" [Low,Confirmed] https://launchpad.net/bugs/412143
<ahasenack> guys, is the wiki the official and up-to-date source for the official ubuntu AMI names?
<ahasenack> https://help.ubuntu.com/community/EC2StartersGuide#Getting%20the%20images specifically
<ahasenack> I'm still running into the network bug during startup. The bug is in "fix released" state, but there is no AMI mentioned in the ticket
<ahasenack> https://bugs.launchpad.net/vmbuilder/+bug/308530 is the bug
<uvirtbot`> Launchpad bug 308530 in ubuntu-on-ec2 "Wait for network before downloading ssh credentials or user-data" [Medium,Fix released]
<agussman> Would anyone care to hazard a guess as to why this is causing a segfault?  #  du -hc / | grep -P '\t/[^/]+$'
<ahasenack> agussman: check dmesg
<agussman> ahasenack: thanks.  After playing around with it, I realized it was the grep that was causing problems but I could avoid using it by providing the --max-depth 1 option to du
<ahasenack> agussman: the dmesg would be to check if there was any kernel panic or oops happening
<ahasenack> bar that, yeah, it's just a user space bug in grep or du
<walski> Hi there
<walski> I'm looking for an AMI to build a high load web server for static pages.
<walski> It's just to test a load generator and must only run for about an hour
<walski> so it would be nice to have something that comes pretty much pre configured
<walski> high load means something about 100 req/sec
<walski> can you give me a hint what would be a good image?
<erichammond> walski: Since you're presumably using Ubuntu, it's easy to start with a clean server AMI and install Apache.
<walski> is there a clean ubuntu image?
<walski> looked at the comunity rep[ository
<walski> but found only a lot of preconfigured stuff which i just does not need
<erichammond> walski: What release (version) would you like to use?
<walski> Doesn't matter ;)
<walski> As I said it's more a fire and forget thing
<walski> would take what comes as the easiest solution
<erichammond> walski: I publish a number of Ubuntu releases on http://alestic.com which are pretty popular for production use.
<erichammond> I'd recommmend picking a "server 32-bit" Hardy or Jaunty.
<erichammond> I also list the AMIs published by Canonical.
<erichammond> Then: apt-get update && apt-get upgrade -y && apt-get install -y apache2
<erichammond> Drop your static files in /var/www and you should be set.
<walski> thanks!
<erichammond> If you find it can't perform to your expectations with m1.small, then upgrade to c1.medium which gives 5x the CPU for 2x the price.
<erichammond> Then step up to a c1.xlarge (which requires 64-bit AMI)
<walski> works great, thank you a lot!
#ubuntu-ec2 2009-08-13
<elmoro> hello
<elmoro> i have a problem with elastic fox and the command line tools - can anyone help me?
<elmoro> I try to make an instance of image ami-7a86ae0e but i cant find it with the command line tools?
<erichammond> elmoro: That image is in Europe.  When using the command line tools, specify "--region eu-west-1"
<elmoro> oh the master of the images
<elmoro> thx for the hint - i am a newbie..
<elmoro> thx for your great work with ec2.
<elmoro> when u are in cologne germany u get a big cup of beer
<erichammond> I live in Los Angeles and am not a fan of traveling, but please express your gratitude by helping somebody else.
<erichammond> Welcome to EC2 :)
<elmoro> thx and i will do my best ;)
<elmoro> a good hint for set up the networking?
<erichammond> elmoro: I'm off to http://ThursdayLunch.com but you  might want to be more specific in what you're looking for so folks can help out.  There is also a ##aws channel with a slightly different mix of experts.  Both channels can take hours for a response to show up.
<elmoro> ok bye
<erikd> if i base an ami "ami-a" on another ami "ami-b" and ami-b gets removed from s3, do I lose access to ami-a?
<erichammond> erikd: Once you have an instance running, you no longer have a dependency on the original AMI, so "ami-b" is safe.
<erikd> the reason i ask is because i'm basing an ami for myself on one of the ubuntu amis and bundling that one to my s3 account.  i was hoping to continue working on the ami i bundled and base more amis off that (for updates to software and everything). i can't really afford to keep all those amis between the ubuntu public ami and my most current one in s3, though
<erikd> i was looking through the manifest for one of the test amis i made and it included a reference to the original ami, is that just for historical purposes or is my bundle actually a diff on the original?
#ubuntu-ec2 2009-08-14
<erichammond> erikd: You might find that AMI storage is pretty cheap (cents per image per month).  I have over 500 AMIs listed.
<erichammond> erikd: The entire image is uploaded to S3 in the bundle.
<erichammond> The only dependencies it might have are the kernel (AKI) and ramdisk (ARI).
<erichammond> If the kernel or ramdisk are deleted by their owners, then you would need to specify different ones when running your image.
<cagdas> hi all
<cagdas> how to we bundle images downloaded from ec2-images.ubuntu.com
<cagdas> i didn't figure it out
<cagdas> in other words how do we convert these images to AMIs
<soren> cagdas: You use ec2-bundle-image.
<soren> cagdas: Let me see if I can find some instructions somewhere.
<cagdas> thanks soren
<cagdas> it would be good if this is added to the topic part of this channel
<soren> cagdas: I'm not sure we have any good docs on the subject. I recommend you look at the documentation for ec2-bundle-image and just ask me if you have specific questions.
<cagdas> thanks soren
<cagdas> i think i handled after your comment
<cagdas> i used ec2-bundle-image
<soren> Oh, fantastic.
<cagdas> then used ec2-upload-bundle
<cagdas> now i have an AMI
<cagdas> ans I just launched a new instance after registering my AMI
<cagdas> however, I'm waiting for ssh to be available on there
<cagdas> it says operation timeout
<cagdas> is it possible that ubuntu images do not have sshd working by default?
<cagdas> (i have opened the ports already on amazon ec2 security group)
<cagdas> it looks like i failed to define kernel-id and ramdisk-id
<cagdas> where do i define them?
<cagdas> while bundling the image?
<cagdas> or while launching an instance?
<elmoro> Hello to ALL
<elmoro> i have a problem with  ec2-running-instances (eu-west-1) with the error keypair not exist. I have a working key pair but it will not be found
<elmoro> also in the aws console i have only one existing keypair - when i use the ec2-describe-keypairs then i get my 3 generated keypairs.
<soren> elmoro: Keypairs are local to regions. Could that explain it?
<soren> elmoro: I.e. a keypair generated in us-east-1 does not exist in eu-west-1.
<elmoro> ok thy
<elmoro> i think i generated it in US and then later i want to use it in Eu region. Thx
<deadprogrammer> anyone have trouble with apt-get upgrade of module-init-tools (3.3-pre11-4ubuntu5.8.04.1)?
<deadprogrammer> sudo apt-get upgrade has me hungup forever on module-init-tools... and my poor server is in quite a sad state
<elmoro> anyone has experience with real time kernel for Gameserver ?  The ping performance is bad and the cpu is 99% idle?
<deadprogrammer> sudo apt-get upgrade has me hungup forever on module-init-tools... and my poor server is in quite a sad state :(
<jeremydei> elmoro:  what gameserver are you running?
<elmoro> hl2 dedicated server
<elmoro> and in the future cod4 and tf2/battlefield
<elmoro> but with the small instance - the ping performance is not the best. At the weekend i will try the bigger machines
<jeremydei> elmoro: interesting.  I haven't tried it (running source server) yet, but I would like to.  Are you using Elastic IP?  you may want to try that also, maybe there's some magic there that can help the ping
<elmoro> the perfomance seem to be okay (tick 66)
<elmoro> but the ping looks bad
<elmoro> yes i use elastic ip..
<deadprogrammer> anyone using ebs with xfs?
<erichammond> deadprogrammer: Yes, I use EBS with XFS and MySQL on production servers.
<deadprogrammer> Hi erichammond
<deadprogrammer> yes I am running mysql well... this time it is postgresql
<deadprogrammer> I am running an somewhat older hardy that is based on one of your older amis
<deadprogrammer> it has been running in production for a while
<deadprogrammer> until earlier todau when and apt-get upgrade when bad
<deadprogrammer> module-init-tools (3.3-pre11-4ubuntu5.8.04.1) hung, and then I have been unable to get the instance back to 'normal'
<deadprogrammer> so then I decided to just launch a new instance from the AMI
<deadprogrammer> I noticed the snapshot did not appear to have my actual database... it was as if no data had ever been saved to the EBS volume
<deadprogrammer> hence my original question... anyone running Postgresql on EBS using XFS should have been the specific question
<erichammond> deadprogrammer: I don't know what might have happened to cause the upgrade to hang.  I am not able to reproduce it myself.
<erichammond> The "empty EBS volume" is a commonly reported problem.
<deadprogrammer> urg... first time I have run into it
<erichammond> However, it always seems to turn out to be user error.
<erichammond> Common causes include:
<erichammond> - The volume was not mounted where you thought it was
<erichammond> - The snapshot was of the wrong volume
<erichammond> - On mounting the new volume, you ran "mkfs" and overwrote the files
<deadprogrammer> this server has been running since march... and did not re-run mkfs on this restore attempt
<erichammond> It can be very tricky to get everything in the right place all the time with EBS volumes and snapshots, especially since there the ids are opaque.
<deadprogrammer> yes, I keep thinking I must have made an error somewhere
<erichammond> Is the original volume still attached to the instance?
<deadprogrammer> yes
<deadprogrammer> but the files are not there on that volume
<erichammond> You can connect to the instance?
<deadprogrammer> there are the original databases
<deadprogrammer> I am still connected to it
<deadprogrammer> postgresql was removed as part of the apt-get upgrade, so I cannot just start the server to see if there database is "there"... but postgres is fairly transoprent about certain things
<deadprogrammer> like there is an actual file on disk that show what databases exist for that server
<erichammond> I'm confused.  You have the original EBS volume attached to the original EC2 instance, and you have mounted the file system on that volume, but it is empty?
<deadprogrammer> not empty... it does not have the database I expect to be there. I can tell by examing the file system on that volume
<deadprogrammer> likewise, a new instance with an EBS volume created from any snapshot, has only those same databases... but not my production database
<erichammond> Ah, I see.  You had a database in Postgres, but now you can't find where the files for that database reside.
<deadprogrammer> exactly
<deadprogrammer> as if postgres never actually wrote any updates to the file system
<deadprogrammer> despite the facts that it has been running for months
<deadprogrammer> and has been updated at least once without event
<erichammond> I don't know Postgres, but if this were MySQL, I would suspect that the configuration was pointing to database files which were not on the EBS volume or that the EBS volume was not mounted at the time the database was running.
<erichammond> Have you looked "under" the EBS file system mount point? I.e., umount and look at the mount point directory/
<erichammond> ?
<deadprogrammer> how do I look at the mount point directory?
<erichammond> Where is the EBS volume mounted?
<deadprogrammer> I just umount /vol
<deadprogrammer> it is /dev/sdh
<erichammond> sudo umount /vol
<erichammond> ls /vol
<deadprogrammer> device dev/sdh mapped to /vol
<deadprogrammer> after I umount I see the correct data!
<deadprogrammer> so do not umount?
<deadprogrammer> well not sure if "correct data" but at least I see entry in file that indicates that database existing
<deadprogrammer> I think you have just saved me from myself... thanks erichammond for your time, I sincerely appreciate your expertise
<deadprogrammer> one last question however. since I seem to have messed up, but having a directory on /dev/sda1 named /vol
<deadprogrammer> how can I get my EBS volume mounted
<deadprogrammer> ?
<deadprogrammer> just correct fstab and mount -a?
<erichammond> I can't provide specifics since I don't have access to all the details of your configuration, but you probably need to move the existing (good) contents of /vol aside, mount the EBS volume, copy the good contents onto the volume, snapshot the volume, create a new volume, and test with a new instance, or something to that effect.
<erichammond> Be very careful as your production database is on ephemeral storage and is not backed up at all.
<erichammond> If that instance goes down, you've lost it.
<deadprogrammer> yes I realize that!
<deadprogrammer> I just added this to my fstab
<deadprogrammer> hmm
<erichammond> I've updated http://ec2ebs-mysql.notlong.com/ so that the mount point is created mode "000" to help prevent accidents like this.
<erichammond> I do that in my own systems, but didn't include it in the original tutorial.
<deadprogrammer> that is a good idea
#ubuntu-ec2 2009-08-15
<elmoro> hello
<elmoro> working on saturday is ...
<elmoro> but i have a question - i try to use the following --> ami-5486ae20 --instance-type m1.large -k profiit --kernel aki-550d2521
<elmoro> is this kernel working with this image?  anyone has this combination already tested?
<elmoro> i need this 1000hz kernel ! what kind of image is working with this kernel.
<elmoro> I cannot connect to this running instances via ssh
<elmoro> the console output is stopping by creating /dev
<elmoro> how long does it take to startup a m1.large instance?
#ubuntu-ec2 2009-08-16
<erichammond> elmoro: When you use a different kernel (aki) than the default, you also need to specify the corresponding ramdisk (ari) for that kernel.
<erichammond> In the case of kernel aki-550d2521, the corresponding ramdisk is ari-f4193180
<erichammond> So, the full start command you want (tested to work) is:
<erichammond> ec2-run-instances --region eu-west-1 --instance-type m1.large -k KEYPAIR --kernel aki-550d2521 --ramdisk ari-f4193180 ami-5486ae20
