#ubuntu-cloud 2011-02-14
<smoser> flaccid, i would think you must have an old version of the module.. .possibly you upgraded?
<smoser> and you're using a cached version of the object and new code... but i'm not sure.
<smoser> i have not seen it.
<flaccid> smoser: well this is actually natty package which must not be ready as the attribute is not in the module
<flaccid> i do nearly have the maverick package working on squeeze though
<smoser> well, it is there.  look in /usr/share/pyshared/cloudinit/DataSource.py
<flaccid> the attributes are not in the module
<flaccid> the module is there yes
<smoser> i'd really like for you to chase the trunk working... i can help, but not really right now
<smoser> well, you should in that file see :
<smoser> DEP_FILESYSTEM = "FILESYSTEM"
<smoser> DEP_NETWORK = "NETWORK"
<flaccid> must not be in the natty pkg yet. is there an easy way to build the trunk src into packages ?
<smoser> flaccid, yes, actually, but the natty pkg should have it... anyway
<smoser> (otherwise this would be completely busted, but alpha-2 did work)
<smoser> flaccid, run ./tools/bddeb is a short cut "build me a deb" of the trunk
<flaccid> danke. most likely cloud-init is going in these debian images. there isn't too many things left to do before release
<flaccid> smoser: not sure why, i added both your gpg keys, but am still getting gpg: skipped "Scott Moser <smoser@ubuntu.com>": secret key not available
<flaccid> oh its secure, i'd need your secret key. i see. guess i have to change that
<flaccid> ok cool i learnt about that
<flaccid> so cloud-init uses initctl thoroughly however does not have a dep, upstart. this is making it difficult for debian
<koolhead11> join #lopsa
<smoser> flaccid, it does use initctl, you're right, but i don tthink there is many dependencies on it any more
<smoser> yeah, it doesn't really use it, externally something coudl take the 'cloud-config' event, but internally the upstart jobs don't use it anymore
<lazyshot> my ubuntu ec2 instance just seems to refuse incoming connections. like a firewall is configured, but the security group has all proper rules. and no errors appear in the console output. what do i do?
<lazyshot> i have httpd and sshd running, both refuse connections. no issues for a week until today.
#ubuntu-cloud 2011-02-15
<elro> does it matter that ebs root image amis are not available in ubuntu-images-eu?
<kim0> elro: what do you mean by "does it matter"
<elro> kim0: I assumed it was there so that it used repositories within the ec2 region
<kim0> elro: the instance gets the correct ec2 repos configured for it, as it boots
<elro> ah, ok fine. I guess ubuntu-images-eu is only there for historical reasons now
<smoser> elro, i'm confused.
<smoser> "does it matter that ebs root image amis are not available in ubuntu-images-eu?"
<smoser> what does that mean
<smoser> there are most certainly ubuntu ebs root images in the eu-west-1 region
<elro> smoser: when launching an instance from the aws management console, if I filter AMIs by Instance-Store Images and "ubuntu-images-eu" I get results, but not when I filter by EBS Images and "ubuntu-images-eu"
<smoser> ah.
<smoser> i would not filter by that
<smoser> :)
<smoser> i'm not exactly sure how the filtering works.
<smoser> but released images for the eu-west-1 region
<smoser> will all be named with "ubuntu-images/"
<smoser> instance-store images have a "manifest path" (that is just not something relevant to ebs root instances).
<smoser> and that manifest path will be: ubuntu-images-eu
<smoser> 1 of 2 things is happening so that your instance-store search works
<smoser> a.) it searches manifest path
<smoser> b.) it searches only 'name', but if there is no name, uses 'manifest-path'.
<smoser> if 'b' is the reason, then you're only seeing older images.
<smoser> easiest way to get which image you want is to go to http://uec-images.ubuntu.com/release or http://cloud.ubuntu.com/ami/
<smoser> kim0, do you load uec-images.ubuntu.com/query on every page load of http://cloud.ubuntu.com/ami/ ?
<kim0> smoser: nope
<kim0> smoser: it's updated every hour
<smoser> why is it slow to load.
<smoser> theres a good 3 seconds on my system of "processing" before it shows something
<kim0> hmm
<smoser> elro, ^
<elro> thanks smoser
<smoser> elro, if you're interested in doing this programatically, then there is data at http://uec-images.ubuntu.com/query that you can use.
<smoser> that is what kim0's page reads from
<kim0> smoser: Here's the code that generates the json table with release data http://bazaar.launchpad.net/~kim0/+junk/cloudubuntu/view/head:/amilocator/views.py
<kim0> smoser: The releasesTable method.
<kim0> I think @cache_page(60 * 60 * 1) is not doing what I thought it would
<smoser> so you see that slow load also
<kim0> I thought it would cache the functions output server side for fast responses
<kim0> smoser: to me it's sometimes fast (0.5) sometimes take 3s
<kim0> when I put a wget in a shell for loop .. it's consistently fast (0.5)
<smoser> http://code.djangoproject.com/ticket/1015
<smoser> that looks old though
<kim0> hmm I think I have it
<kim0> http://cloud.ubuntu.com/wp-content/themes/uecv2/ami-locator.php >> slow
<kim0> http://cloud.ubuntu.com/ami-locator/releasesTable >> fast
<elro> Anyone know what the DEVPATHS line of /etc/ebsmount.conf should be for it to work on ec2?
<kim0> in development, we had written ami-locator.php (server side proxy), it's still being used, although it doesn't have to
<kim0> I'll try to switch to direct connections .. should hopefully be faster
#ubuntu-cloud 2011-02-16
<Kiall> anyone know if its possible to convince cloudinit to set an EC2 instance's hostname to its external name, rather than internal?
<flaccid> edit the module or create a new one i think
<Kiall> flaccid, yea - im looking at the source now .. I would have thought it a common request and figured it was something simple ;) bah!
<flaccid> feel free to contribute
<Kiall> flaccid, if i could get my head around bzr / quilt and their friends .. i would ;)  .. git I can handle .. bzr and quilt make me sore ;)
<flaccid> i agree
<Kiall> infact .. its not even quilt .. i can use that ;) .. its just bzr! ;)
<flaccid> bzr is pretty straight forward. i'm sure many ubuntu peeps can help you if you have a problem
<Kiall> I'm sure its just that i'm way too used to git to allow myself to get used to it!
<flaccid> hey im' like that too for sure
<Kiall> yup.. launchpad doesnt exactly help either .. never have the slightest clue which branch and repo to use! anyway .. guess I'll figure another way around it as I need a fix pretty quickly .. i think preserve_hostname: true and setting it in runcmd's should work
<flaccid> yeah its confusing as. i don't know much about cloud-init yet. hoping someone else here can assist you
<superxgl> hi all, is cloud-init only for ubuntu? is there any RPM package for CentOS installing ?
<soren> It was written in the hope that other distros would pick it up.
<soren> ...whether they have or not... No idea.
<superxgl> soren: thank you. i want to install cloud-init on my CentOS, but did not find any rpm package
<superxgl> i don't know if i can convert the cloud-init deb package into rpm package...
<kim0> superxgl: I think a program named "alien" might help convert it .. doesn't mean it'll work though :)
<superxgl> kim0: thank you. i will have a try :)
<TREllis> superxgl: well, amazon use it in their "official" linux ami, which looks relatively centos based
<TREllis> superxgl: infact, if you launch one of their official linux ami's you can grab the src.rpm using 'get_reference_source -p cloud-init'
<superxgl> TREllis: Thank you very much :)
<TREllis> not sure why they make you use that script to get to the source packages, but I guess it's something to do with all their repos & mirrors only being accessible from AWS
<superxgl> TREllis: hmm... i don't have an account for amazon
<flaccid> superxgl: get the rpm from the amazon linux ami.
<flaccid> they have a version of cloud-init and it works
<superxgl> flaccid :  i don't use EC2 so i can not access the linux ami :(
<flaccid> oh crap i some already said sorry
<flaccid> start using it.
<flaccid> err *someone said
<superxgl> hmm....
<flaccid> i'd launch an instance and get it but i cbf :)
<superxgl> flaccid: thx :)
<TREllis> superxgl: I'm feeling helpful today...  http://trellisnet.co.uk/junk/cloud-init-0.5.14-22.amzn1.src.rpm
<flaccid> thats the source rpm
<flaccid> you can get the real rpm out of the ami or the repos
<ubuntucloud304> where do i start am ubuntu/linux newbie
<TREllis> or rpmbuild -ba that ;-)
<superxgl> TREllis : oh, cool....thank you very much for ur kindly help  :)
<flaccid> ubuntucloud304: reading the docs
<superxgl> and thank you flaccid, thank you all of you here ..
<ubuntucloud304> -=flaccid specifics
<flaccid> whichever you can find
<flaccid> its like getting a car and saying 'where do i start'
<flaccid> well how long is a piece of string..
<ubuntucloud304> from a drivin school first
<flaccid> anyway i'm a bit too sarcastic tonight, so i will go watch a movie
<ubuntucloud304> cool
<TREllis> superxgl: http://cloud.ubuntu.com/
<TREllis> that's a good starting point, really depends on what you aim to do
<kim0> ubuntucloud304: are you aiming to start with "cloud" stuff, or generally "Ubuntu" itself
<ubuntucloud304> i'l now hit the road
<ubuntucloud304> cloud
<superxgl> TREllis: thank you very much ! :)
<kim0> ubuntucloud304: ah ok .. ping me when u're back
<kim0> will try to help
<TREllis> superxgl: woops that was directed to ubuntucloud304 ;-)
<superxgl>  TREllis: aha, it is also helpful to me :)
<madafu> kim0 ubuntucloud304 help
<TeTeT> madafu: what's the problem?
<TeTeT> anyone ever got an 'unable to update user' warning when changing/deleting a user in the web interface of 10.04 UE
<madafu> TeTeT: New in ubuntu cloud
<kim0> madafu: hey .. welcome :)
<kim0> madafu: so what got you interested
<kim0> what are you trying to do
<madafu> cloud
<kim0> so you wanna run virtual machines and stuff
<madafu> downloaded the ubuntu cloud server what next?
<madafu> yeah
<kim0> madafu: This is the main installation page https://help.ubuntu.com/community/UEC/CDInstall
<madafu> thank you
<kim0> madafu: for some extra reading, this following page is a good jumping page http://cloud.ubuntu.com/docs/uec-book/
<superxgl> TREllis: i could not rebuild the cloudinit-src.rpm, it said that :  error: Failed build dependencies:  python-devel-abi is needed by cloud-init-0.5.14-22.noarch
<TREllis> superxgl: never mind, you'll need to grab the binary from the amazon linux ami then
<superxgl> i have installed something like python-devel, python-abi, and it did not works. and i googled a lot , did not find the python-devel-abi
<TREllis> superxgl: and/or try python26-devel since Centos5 uses python2.4 by default
<kim0> superxgl: yum whatprovides python-devel-abi
<kim0> might help
<superxgl> i did yum python-devel-abi, but it saids not have this one...
<kim0> just try that command
<superxgl> i am trying install python26 now
<TREllis> it's probably easier just to spin up an instance and user yumdownloader to grab the binary
<TREllis> -r
<superxgl> i did yum whatprovides python-devel-abi , still not found...
<TREllis> superxgl: 2 secs, I'll pull it out of one of my instances
<superxgl> TREllis : oh...thank you very much, u help me a lot today :)
<TREllis> superxgl: http://trellisnet.co.uk/junk/cloud-init-0.5.14-23.amzn1.noarch.rpm interestingly the one they provide the source of is version -22 not -23
<superxgl> TREllis:  oh, ya , it did is -22... thank you for ur help , i am really appreciate for all of your help here :)
<TREllis> superxgl: sure, np
<superxgl> TREllis : cool...the rpm works pretty well :) tnx:)
<TREllis> superxgl: you are using it for spinning up centos emi's in UEC?
<superxgl> TREllis: i am using it for execute some scripts once the instance boots, for i want to run hadoop on the eucalyptus .and  i am using centos EMI now
<TREllis> superxgl: right ok
<superxgl> TREllis : hopes it works :)
<TREllis> superxgl: I've done a bit of hadoop in UEC in the past
<TREllis> superxgl: but generally with the Ubuntu EMI's :-)
<superxgl> TREllis : oh ,really ?
<superxgl> it is cool..
<superxgl> so how did u do that ?
<superxgl> i just followed the guide : http://wiki.apache.org/hadoop/AmazonEC2
<TREllis> superxgl: there is a number of ways, there are a few puppet modules for hadoop floating around on the web, I used cloud-init with some already configured hadoop configs since it was only for some tests
<superxgl> and i modified the scripts to make it works  on eucalyptus
<superxgl> TREllis : hmm...cool..
<superxgl> puppet modules ? what are they?
<superxgl> hmm...looks like the coming release 0.4.0 of apache whirr will support eucalyptus ..
<elro> Is anyone using ebsmount on ec2? I'm seeing an "error: devpath not of expected structure, or failed lookup" when I try sudo ebsmount-manual /dev/sdf
<RobertLaptop> Question,  Under EC2 once can define a puppet script to customize an image in it startup process.  Is there a way to do it in a Private eucalyptus cloud?
<TREllis> RobertLaptop: cloud-init can hook into puppet
<RobertLaptop> TREllis, clound-init any documentations?  I guess in my week long research on running a cloud I missed that.
<TREllis> RobertLaptop: see https://help.ubuntu.com/community/CloudInit & http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/doc/examples/cloud-config.txt for a fully commented cloud-config file for cloud-init
<TREllis> RobertLaptop: actually, this file has the puppet cloud-init docs: http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/doc/examples/cloud-config-puppet.txt
<RobertLaptop> TREllis, thanks will read
<TREllis> you can't specify a specific puppet class in cloud-init, but you can get it to do the bootstrap,
<TREllis> you'll have to have a node definition that knows that ec2 node name or hostname glob and match that to classes or use some examples that mathias produced a while back: http://ubuntumathiaz.wordpress.com/ check out all his blog posts on puppet+ec2/uec
<RobertLaptop> Reading
<RobertLaptop> Yea that was my one big question was how to pass that info to puppet for something that will get created on the fly.
<RoAkSoAx> TREllis: I already have a tesst RA for the CLC, and works like a charm
<jmgalloway> I have a question about security groups
<daker> !ask | jmgalloway
<daker> !ask
<jmgalloway> I have a lot of users that are using my cloud
<jmgalloway> when they create instances, they have trouble getting ssh to work
<jmgalloway> I tried to create a security group for them to use with port 22 open, but I think I did it wrong
<jmgalloway> I am the only admin, the others a normal users
<jmgalloway> GROUP	admin	cs691	cs691
<jmgalloway> PERMISSION	admin	cs691	ALLOWS	tcp	22	22	FROM	CIDR	0.0.0.0/0
<jmgalloway> PERMISSION	admin	cs691	ALLOWS	tcp	21	21	FROM	CIDR	0.0.0.0/0
<jmgalloway> PERMISSION	admin	cs691	ALLOWS	tcp	80	80	FROM	CIDR	0.0.0.0/0
<jmgalloway> is this not right?
<jmgalloway> anyone?
<jmgalloway> anyone here?
<daker> jmgalloway, just wait ã
<jmgalloway> I just have a few questions about getting ports open
<kim0> jmgalloway: hey
<kim0> jmgalloway: when you launch instances using your account, under "admin cs691" security group, I suppose it works as it should
<kim0> How are those other users allowed to create instances ? or you allowing them with IAM ?
<kim0> try to leave some more detail .. and someone will try helping
<mathiaz> smoser: hi!
<mathiaz> smoser: is there anything in the LTS ami that would automatically lock the ubuntu account?
#ubuntu-cloud 2011-02-17
<smoser> mathiaz, i dont think so
<smoser> anything in console output ?
<mathiaz> smoser: well - I'm trying to log with the ubuntu account
<mathiaz> smoser: once I've configured the system with puppet
<smoser> and aparently you cannot. :)
<mathiaz> smoser: the ubuntu account is then locked
<mathiaz> smoser: not sure what does this though
<smoser> hmm..
<smoser> i really dont know.
<mathiaz> smoser: me neither
<mathiaz> smoser: I've worked around it for now
<mathiaz> smoser: might investigate it later
<mathiaz> smoser: keep up the good work! the ec2 images are great base to build upon!
<smoser> well the account *is* locked initially
<smoser> as in no password set
<smoser> (usermod -L)
<smoser> but obviously you can get in
<mathiaz> smoser: yeah - using my public key
<mathiaz> smoser: however there is a password set for ubuntu
<mathiaz> smoser: in /etc/shadow
<mathiaz> smoser: the string starts with !
<smoser> there should not be
<smoser> oh
<smoser> yeah
<mathiaz> smoser: but there is a long string there
<elro> Is there a way to specify that my rc.local should be run after mounting all filesystems? Putting a mount in rc.local shows that my ebs volume /dev/sdf has not yet been mounted.
<elro> I guess it is because of the nobootwait
<smoser> mathiaz, just for kicks i tried to see if just doing a 'usermod -U' would end up locking you out, but i can still get in
<mathiaz> smoser: I think it may be related to the sshd_config file I put in place
<mathiaz> smoser: because I can log in before I install the instance via puppet
<mathiaz> smoser: using the ubuntu account
<mathiaz> smoser: once puppet has run, the ubuntu account cannot be used anymore
<smoser> mathiaz, i'm sure you're as capable as me to debug.
<smoser> but i'd start by copying off /etc/ before and comparing to after (assuming you can still get in)
<smoser> RoAkSoAx, i have another ec2 scripts task for you at some point :)
<RoAkSoAx> smoser: ok cool :)
<smoser> right now the publish is all serial. first publishes us-east-1 , then us-west-1 , then eu-west-1, then ap-southest-1 ...
<smoser> it'd be nice if we just did those in parrallel
<smoser> a full publish takes like 6+ hours at the moment
<smoser> probably could cut that in half
<smoser> and there are probably other things we could do to speed it up.
<smoser> well, i've got to run. later.
<RoAkSoAx> alrighty
<RoAkSoAx> have a good one
<superxgl> hi all, is there anyone who have ever written a webservice like this ?   http://open.eucalyptus.com/participate/wiki/autoscaling-behalf-monitoring-virtual-machines
<superxgl> hmm... i am a newbie , i haven't written a webservice before, can anyone here point me somewhere how to write a webservice ?
<flaccid> google.com
<superxgl> k, i am seaching now..
<superxgl> hmm...using axis or xfire..
<kim0> superxgl: I suppose if you're not a developer .. hire someone who can implement this cleanly. I also don't think it has to be implemented that way
<kim0> superxgl: potentially just a shell script that ssh'es .. checks load .. and decides to launch more instances might do huh
<flaccid> superxgl: what are you trying to achieve.? i'll let you know what to do
<superxgl> kim0,flaccid: thanks very much :)  i want to achieve the load of instances , then i can decide if it is need to start a new instance (i.e a hadoop slave node)
<flaccid> load of instances?
<superxgl> yes
<flaccid> what is that?
<superxgl> the system metrics
<superxgl> like the cpu load
<TREllis> system load I'd imagine
<superxgl> yes, i want to get the load average of the cluster
<kim0> superxgl: for s in servers ; do ssh $s uptime ; done
<kim0> there you have it :)
<kim0> decide to launch more instances ..
<superxgl> kim0: hmm. yeah, seems like just a shell script is ok
<kim0> yeah .. I'm saying it can be .. ofc if it gets more complex .. perhaps some proper code needs to be written
<superxgl> hmm, maybe i also need cron
<superxgl> i found a good example :   http://bash.cyberciti.biz/monitoring/get-system-information-in-html-format/
<flaccid> superxgl:  i use RightScale which uses collectd and RRD
<superxgl> flaccid: what is collectd and RRD ?
<flaccid> google.com
<superxgl> finding now :)
<superxgl> http://support.rightscale.com/12-Guides/RightScale_Methodologies/Monitoring_System/Writing_custom_collectd_plugins/Custom_Collectd_Plug-ins_for_Linux
<superxgl> collectd is very cool ..
<flaccid> indeed it is
<superxgl> yeah, i want to have a try :)
<flaccid> go for it. if you want to use rightscale, its free
<flaccid> you get free monitoring for the first week of each instance/server
<superxgl> great . i want to apply collectd and rrd into my private cloud
<flaccid> can do that too. its just a package..
<superxgl> but i am still a little confused...
<superxgl> that is : how graphs  be drawn  ?
<flaccid> checkout rrdtool
<superxgl> ok,tkx
<superxgl> i have a question, how about ganglia ? which one is better ?
<superxgl> why RightScale not use ganglia instead of rrdtool?
<RobertLaptop> Does eucalyptus support more then 1 Public interface Nic?  In other words I have 2 diff ISP with there own IP range.  So I would want Public IP range A = br0, Public IP range B = br1, and Private IP range C = br2
<SpamapS> smoser: hey.. so I've been thinking more about cloud-init and the cloud type setting.. I am curious.. could we check the install media type to auto detect it?
<smoser> i'm not sure i follow the question
<smoser> but its a runtime decision
<smoser> it has to be
<smoser> the idea is that we're making one image that runs on multiple clouds (or, non-clouds)
<SpamapS> smoser: right, but if the install media is CD, the cloud type should be local file
<SpamapS> smoser: as happy as I am that the boot only slows by 30 seconds now instead of 20 minutes .. I'd really prefer it to not slow things at all :)
<smoser> it should not stop you for 30 seconds
<SpamapS> it definitely does
<smoser> the reason looking  at install media is not valid is that you should be able to do an install in kvm (or bare metal for that matter) and then publish that on EC2 (ideally)
<smoser> hm.. do you know why you'r ebeing slowed down for 30 seconds ?
<smoser> you really shouldt be. i suspect the ovf datasource is what is giving you fits
<SpamapS> well 1 thing is I'm gettng a DataSourceNotFoundException ...
<superxgl> oh..ganglia is based on rrdtool...
<smoser> and there may be a fix for htat in trunk already
<SpamapS> cloud-init start-local running: Thu, 17 Feb 2011 16:00:42 +0000. up 16.88 seconds
<SpamapS> I get that right after root is mounted..
<SpamapS> then after a bi
<SpamapS> t
<SpamapS> no instance data foudn in start-local
<SpamapS> init: cloud-init-local main process (269) terminated with status 1
<SpamapS> cloud-init start running: Thu, 17 Feb 2011 16:01:03 +0000. up 36.45 seconds
<smoser> yeah.. i'm pretty sure its the ovf
<smoser> it tries to mount cds. i had expected mount to timeout quicker
<smoser> in trunk now it does a read of the device before trying to mount, the read (in my tests in kvm) returns immediately with failure, where mount was taking quite a long time.
<smoser> if its -local thats taking a long time, its definitely not EC2.
<SpamapS> ahhh
<SpamapS> Makes sense
<SpamapS> smoser: well then thank you for indulging my crazy idea section for the day. :)
<hazmobile> niemeyer, heading back from doc.. eta 30m
<niemeyer> hazmobile: How are you?
<hazmobile> stitched and anesthesized
<niemeyer> hazmobile: Awww..
<hazmobile> on the bright side found aideable Android irc client
<hazmobile> useable
<niemeyer> hazmobile: Clearly ;-)
<niemeyer> hazmobile: Which finger was it?
<Edulix> hi people
<Edulix> I want to do a cpu intesive task that can be divided into small chunks that can be sent to different computers, using some kind of clustering
<Edulix> is ubuntu cloud recommended for cluster apps? I want to be able to plug in a new computer to my farm, then boot a thin client on it that runs a node of the cluster..
<RobertLaptop> Edulix, can it yes.
<Edulix> RobertLaptop: I think I might use MPI for this (I need to patch my app to make parallelizable)
<RobertLaptop> Although it isn't designed to do cluster work so interconnections between the VM will likely be slower then in something designed to be a cluster.
<Edulix> RobertLaptop: I'm new ubuntu cluster. ubuntu cloud always run images in an VM?
<RobertLaptop> Edulix, Yes.
<Edulix> ok then it's definitely not what I'm looking for
<Edulix> thanks RobertLaptop
<Edulix> RobertLaptop: isn't there anything like ubuntu cloud but that runs the images non virtualized? :P
<RobertLaptop> Edulix, not that I know of not saying it doesn't exist just not something I know about.
<Edulix> it'd be nice if there was a distro specialized in cluster support, so that you just install & configure your mpi-enabled software in an image, then start the server and connect new nodes
<Edulix> and then the new nodes boot via ethernet etc
<Edulix> RobertLaptop: how does ubuntu server deal with hardware?
<Edulix> RobertLaptop: how do I configure a new node, for example. does it boot over network, or something?
<RobertLaptop> Edulix, just to be clear I don't work for Canorical just a user.
<Edulix> RobertLaptop: me too :P
<RobertLaptop> Edulix, It deal with hardware like most Linux Server.
<Edulix> RobertLaptop: so for each node, you need to install ubuntu cloud server etc
<RobertLaptop> The cloud stuff uses either Xen or KVM to handle visualization and if you you setup your images just right the VM will get pretty close to bare metal ussage.
#ubuntu-cloud 2011-02-18
<wolfric> are there any videos around of a controller assigning images to nodes and making them work?
<wolfric> Also - any more elaborated documentation on the matter other than the community pages?
<RobertLaptop> I have a questions regarding https://help.ubuntu.com/community/UEC/Topologies it shows breaking systems out with some systems label just NC.  The question is does the system running other services not showing NC run VM's or is it just for managemnt stuff?
<willkessler> hello, i'm having a cpu load problem under medium network traffic on 10.04LTS, large instances, can anybody help me?
<flaccid> what is using the cpu?
<willkessler> apache
<flaccid> then that is your problem
<willkessler> it would make sense, except that traffic to the app does not significantly change
<flaccid> how does that conclude anything
<flaccid> you could have 0 traffice and apache do 100%
<willkessler> true; however, the application responds pretty much the same way to everybody that uses it. it's a social game, and tens of 1000's of players play it all day
<willkessler> however, at certain times of the afternoon load will surge in apache withotu the app doing anything different per se
<flaccid> sounds normal to me
<flaccid> its your problem to put it simply
<willkessler> not serving more users, different game play, etc. the same at 10am as at 4pm. but between 3 and 6, the normal load for its traffic (3) spikes to 50
<flaccid> you can't expect the open source community to debug your app and shit
<willkessler> if it were a ton more players, i would think apache would become more allocated (looking at /server-status)
<flaccid> yeah bad assumptions once again
<willkessler> i don't expect that, certainly
<willkessler> and why is that bad assumptions?
<flaccid> because you don' need a load of users to have a mem and/or cpu leak or other indirect issue
<flaccid> it does not look like the problem with your app is anything to do with aws or ubuntu
<willkessler> i am merely hoping to hear that either a) this release of ubuntu has issues with high network load and to use something different or b) load on ec2 instances can vary even without your instance doing any work at all, since i've done a ton of debugging on the app and do not see an issue except when the app is in the cloud
<flaccid> keen to hear any evidence to suggest otherwise
<willkessler> for example, the app has been heavily loaded outside the cloud by browsermob, without issues
<flaccid> yeah sorry, not a good hope.
<willkessler> flaccid: hmm, ok, i'll take your feedback as a valid single data point
<flaccid> i do hope that other users here come out with other responses..
<willkessler> a hope i share as well :-)
<willkessler> flaccid: as regards your suggestions, certainly there could be a ram leak, but i don't see evidence of it using the usual tools (top, iostat, free, etc)
<willkessler> flaccid: swap is never allocated, e.g.
<willkessler> flaccid: as far as CPU leak, also possible, i don't expect the community here to debug anything like that. however, again, if code doesn't change hour over hour yet system behavior does, it leads one to suspect OS or metal
<flaccid> can you show stats at least?
<willkessler> sure
<willkessler> flaccid: which stats would you want to see
<willkessler> flaccid: i have some output from iostat, for example
<flaccid> well
<flaccid> whats the actual issue first
<flaccid> cpu usage?
<willkessler> yes...
<willkessler> well, load overall, causes tasks that take say 300ms to return to take 1500ms, etc
<flaccid> is top basically correlating to the load reported in w
<willkessler> yes, it does exactly
<flaccid> well what can i say, apache is using that cpu
<flaccid> it doesn't look like a load accounting bug
<flaccid> how about you try maverick?
<willkessler> it definitely is the app but what i don't understand is, why the app doing the same thing it does all day, should suddenly demand so much more cpu
<flaccid> nobody here can answer that question
<willkessler> is maverick production ready?
<flaccid> sure
<flaccid> no ubuntu is production ready
<flaccid> thats the nature of it
<flaccid> but use it if you like
<willkessler> ok, i wil give it a tryout then, thank yo
<flaccid> i'd be curious to see if the same thing occurs on debian 6
<willkessler> i might just switch to centos after reading this posting: http://alestic.com/2010/10/ec2-ubuntu-maverick#comment-484
<flaccid> willkessler:  we offer alternate builds @ RightScale if you want to try them out
<flaccid> that post seems a little speculativ
<flaccid> well it is. you don't report bugs on blog comments for a start.
<willkessler> as was pointed out
<willkessler> where do those rightscale builds live?
<flaccid> willkessler: http://support.rightscale.com/18-Release_Notes/OS_and_Software_Package_Support
<flaccid> im an engineer at rightscale, so please feel free to join us in #rightscale if you want
<willkessler> do i need to be a customer?
<flaccid> no
<willkessler> certainly, rightscale has gotten high reco's from my colleagues, and if we could afford it , i'd probably talk to you via the front door... but not quite yet
<flaccid> rightscale is free unless you would like a subscription
<willkessler> where can i read more about rightscale/free? that same site?
<willkessler> i'll just look at the links, nvm
<willkessler> thanks and good night
<flaccid> willkessler: http://support.rightscale.com/03-Tutorials/01-RightScale/1._Signing_Up_for_RightScale/Sign-up_for_a_Free_RightScale_Account
<flaccid> good night then.
<RobertLaptop> Question I selected the wrong options during install is there a way to re-run the install script?
<flaccid> what install?
<RobertLaptop> I selected CLC, Walrus, on the wrong box.
<superxgl> hi all, i wanna ask a question, now in every nodes i use Wget to get the ip of nodes, and i want this IPs be sent to cc, how should i do ?
<superxgl> in nodes, i use cron to execute the a shell script to run Wget every minutes
<superxgl> and i want it to report to the cc
<TeTeT> how would I specify --addressing private for uec-run-instances? When I run 'uec-run-instances --attach-volume=$vol -- --addressing private $emi' I get an error. I use the TREllis modified cloud-utils packages for support of attaching volumes
<superxgl> if the cc can not receive the ip of nodes execeed 3 mins, i will think the node dies and start a new one...
<superxgl> TeTeT:  hmm.. i also use --addressing private
<TeTeT> superxgl: do you use euca-run-instances or uec-run-instances? euca-run-instances works fine
<superxgl> TeTeT: i use euca-run-instances, hmm.. don't know then ..
<TREllis> TeTeT: --run-args=RUNARGS    pass option through to run-instances
<TeTeT> TREllis: doesn't seem to work either, http://pastebin.ubuntu.com/568724/
<TREllis> TeTeT: hmm I see the same on trunk
<TeTeT> TREllis: ok, this is the syntax: uec-run-instances -v --attach-volume=$vol $emi --run-args="--addressing private"
<TeTeT> note the quotes around the run-args, tried it without the -- and the quotes first
<TREllis> I normally put the emi at the end, but testing here that doesn't work either
<TeTeT> superxgl: hmm, you try to detect if the nodes are still up and running?
<TeTeT> TREllis: request for improvement: check if the volume is in an attachable state ;) if it's already been attached to a running instance, print an error or warning
<TREllis> TeTeT: hmmm I tried with a different runargs and it worked -k mykey
<TeTeT> TREllis: but otherwise, works very nicely! Thanks for providing that option, will introduce it in my next class :)
<superxgl> TeTeT : yes..i call it heartbeat, if the cc can not receive the heartbeat execeed 3 mins, i would think this node dies...
<TeTeT> superxgl: why don't you use an existing hearbeat system then?
<superxgl> TeTeT: Is there have one ?
<TREllis> TeTeT: did you get the run-args working then?
<TREllis> TeTeT: thanks for the suggestion, thats one on the list :)
<TeTeT> TREllis: yes, got it working with the quotes and the --
<TeTeT> superxgl: please have a look at 'apt-cache show pacemaker'
<TeTeT> superxgl: there's also https://wiki.ubuntu.com/UbuntuHighAvailabilityTeam/PacemakerHeartbeat for more doc on how to get going
<TREllis> infact, RoAkSoAx has been working on a resource agent for the CLC too
<superxgl> TeTeT: tnx:) go and checking now ..hmm.. i am really a newbie to this..
<TeTeT> superxgl: no worries, we've all been newbies once
<superxgl> TeTeT : :)
<superxgl> TeTeTï¼hmm...PacemakerHeartbeat is very cool :)  but since it is my first time to do something like this, now i want to do it myself. my though is the nodes do a Wget like http://cc.com/?thisismyip=192.168.1.2 , that is the Ip be embedded into the GET parameter, so that the cc can get it's ip address..
<superxgl> anyone here have any idea?
<RoAkSoAx> TeTeT: that documentation is outdated
<RoAkSoAx> superxgl: you might be interested in this: http://www.roaksoax.com/2010/10/high-availability-uec-clc-howto
<RoAkSoAx> TREllis: and as a matter of fact, the RA for the CLC is ready. I'll publish it later today
<superxgl> RoAkSoAx : tnx very much :) go and checking now:)
<TREllis> RoAkSoAx: cool :D nice work
<RoAkSoAx> TREllis: the hardest thing was to test it lol!
<TREllis> RoAkSoAx: heh
<TeTeT> RoAkSoAx: great, thanks for the update!
<RoAkSoAx> TeTeT: wiki.ubuntu.com/ClusterStack/LucidTesting is the most updated, which still stands for natty
<TeTeT> RoAkSoAx: ok, will bookmark it, superxgl might be interested in it as well
<RoAkSoAx> TeTeT: you might also be interested in http://www.roaksoax.com/2010/10/high-availability-uec-clc-howto (I'll update it later today with a better way to monitor the CLC though)
<superxgl> i am really interested in part: Load Balancing with Pacemaker/ldirectord :)
<RoAkSoAx> superxgl: ah!! I have one how-to for that I just haven't make it public yet
<superxgl> RoAkSoAx: i'll wait for it then :)
<smoser> SpamapS, awake ?
<SpamapS> smoser: only have a couple minutes.. wassup?
<smoser> i had some upstart questions.
<smoser> hold on. let me get a pastebin
<smoser> SpamapS, http://paste.ubuntu.com/568842/
<SpamapS> smoser: ok.. whats the ?
<smoser> so i'm trying to force the order of those 3 things
<smoser> cloud-init-local -> cloud-init-nonet -> cloud-init
<SpamapS> ah ok
<SpamapS> since they're all tasks..
<SpamapS> stopped and started are equivilent
<SpamapS> start on stopped cloud-init-local  ... start on stopped cloud-init-nonet ...
<SpamapS> also since they're tasks..
<SpamapS> you can do this (I think this is evil but it may be necessary for fine grained control... have a task that just does
<SpamapS> start cloud-init-local ; start cloud-init-nonet ; start cloud-init
<SpamapS> note that && would maybe be better ;)
<smoser> so do you not think what i have there would work ?
<SpamapS> it should actually
<SpamapS> the mounted is tricky tho
<SpamapS> I'd only do it on the first one
<SpamapS> unless you MUST block mounting / on all 3?
<SpamapS> or rather, mounting anything except mount
<SpamapS> mounted blocks mountall
<smoser> well, we want to block.
<SpamapS> smoser: just make sure those tasks *always* finish or the system won't boot
<SpamapS> mostly because filesystem won't fire .. once / is mounted a lot is already done :)
<smoser> not really.
<smoser> not much is done on MOUNTED /
<smoser> most stuff requires filesystems
<SpamapS> true.. like rc-sysinit
<smoser> and ssh, just about everything. and really you should'nt start before that
<smoser> you're not guaranteed that tmp files in /tmp wont get deleted while you write them
<jmgalloway> can someone tell me where the uec website is located on the cloud controller?
<smoser> SpamapS, shoot. my plan doesnt work
<smoser> i think i know why
<SpamapS> smoser: ?
<smoser> http://paste.ubuntu.com/568842/ doesnt work. i get
<smoser>   cloud-init-nonet waiting for a network device.
<smoser> cloud-init-nonet gave up waiting for a network device.
<smoser> even though there was a netowrk device
<smoser> i think the problem was that eth0 came up before cloud-init-nonet was running
<smoser> (ie, while cloud-init-local was running)
<smoser> SpamapS, make sense ?
<TeTeT> jmgalloway: /etc/eucalyptus/cloud.d/www defines admin.xml, which refers to /var/lib/eucalyptus/webapps/root.war I think
<jmgalloway> oh thanks, I'll take a look there.  I'm running apache and another website on my cloud controller..I just wanted to know if I could customize the web gui uec already has
<SpamapS> smoser: right so you also need to create another task that emits an event on net-device-up or a timeout
<smoser> getting messy :-(
<SpamapS> network detection is messy
<SpamapS> wait is nonet supposed to figure out if there is or isn't a network?
<smoser> yeah.
<SpamapS> so make a task that sends it a SIGUSR1 every time net-device-up fires
<SpamapS> and handle that event by checking again
<SpamapS> s/event/signal/
<SpamapS> thats how mountall does it
<smoser> the problem is that network comes up before it runs.
<smoser> i think
<SpamapS> so its failing to detect that fact.. sounds like it should do/while rather than while .. ;)
<smoser> ?
<smoser> i was trying to force order... cloud-init-local -> cloud-init-nonet -> cloud-init
<smoser> but (i think) the network comes up while cloud-init-local is running
<smoser> so cloud-init-nonet does not get the 'net-device-up' to stop it
<smoser> and so it waits until timeout
<SpamapS> Right so you need to check if its up *first*
<SpamapS> while (!netup) sleep();
<SpamapS> while (!netup()) sleep();
<smoser> who do i check if netup?
<smoser> SpamapS,
<smoser> suggestions?
<SpamapS> smoser: ifconfig.. /sys .. /proc .. not sure actually ;)
<smoser> yeah.
<SpamapS> there may be a single command that does it
<smoser> SpamapS, looks like ifquery might hold some information ... and also that /var/run/network/ifstate might just be suffiicent
<SpamapS> smoser: yeah that sounds like you're on the right path.
<SpamapS> gotta go afk.. bbl
<RobertLaptop> Question how do you define more then 1 DNS server?  VNET_DNS="192.168.100.213" to you add 2 lines or 1 line with a comma or a space?
<smoser> hggdh, ping
<hggdh> smoser: pong
<smoser> i think i don tneed you now, hggdh
<hggdh> smoser: k
<smoser> and i'm done with rig
<hggdh> smoser: thank you, I will reset it for Daviey
#ubuntu-cloud 2011-02-19
<Daviey> hggdh, appreciated
<Daviey> have a good weekend
<Daviey> :)
<hggdh> Daviey: mabolo, marula, santol, sapodilla & soncoya are ready for you
<hggdh> Daviey: and you have a nice weekend also :-)
<RoAkSoAx> Daviey: dude!! don't you sleep?
<Daviey> RoAkSoAx, seems not.. i am a machine! :)
<RoAkSoAx> Daviey: lol :P
<RoAkSoAx> Daviey: you should instlal PowerNap lol!
#ubuntu-cloud 2011-02-20
<mwhudson> how long do the images on http://uec-images.ubuntu.com/maverick/current/ remain available for?
#ubuntu-cloud 2012-02-13
<ubuntucloud181> And, I am new to uibuntu oneiric, and i had installed the desktop version, and I am planning to install Openstack using Denstack
<ubuntucloud181> will that work...in some manuals , I had read about installing in Ubuntu 11.10 server only. Will DEvstack scripted installation work in the Ubuntu desktop version?
<roaksoax> win 2
<daniel_____> hello everybody
<daniel_____> anyone willing to help ?
<daniel_____> when i juju status the enviroment, i get invalid ssh_key, though i RSAed the zookeeper
<daniel_____> come on guys
<daniel_____> guess no one knows what i m talking about, still with the UEC
#ubuntu-cloud 2012-02-14
<lool> Hey
<lool> A Linaro coworker says there are issues with an EC2 Ubuntu mirror
<lool> we're running our vms in us-east-1
<lool> a) is there a place to ticket these issues
<lool> b) is there some other alternate mirror we should use (S3 maybe?)
#ubuntu-cloud 2012-02-15
<danwea> hello
<ubuntucloud060> hi
#ubuntu-cloud 2012-02-16
<whitenhiemer> hey guys, i'm new to ec2 instances
<whitenhiemer> is there anything specific that I need to do to make port 80 publicly accessible
<whitenhiemer> I'm not running iptables rules yet
<TeTeT> whitenhiemer: probably ec2-authorize to open the port
<whitenhiemer> is that in the control panel?
<TeTeT> whitenhiemer: let me check
<whitenhiemer> i disassociated my elastic IP and lost my shared ip, i think its just time to relaunch and start over, fun toy, maybe have some use for this
<erichammond> Which Ubuntu releases on EC2 use a root disk label of "cloudimg-rootfs" and which use a rood disk label of "uec-rootfs"?
#ubuntu-cloud 2012-02-17
<gholms> smoser: Is cloud-init start-local supposed to fail on EC2?
<smoser> you want it to exit success
<gholms> Hmm...
<smoser> i thoguth there was  bug on that.
<gholms> I'm looking at the new comment on bug 861943.
<uvirtbot`> Launchpad bug 861943 in cloud-init "Add systemd support" [Medium,Fix committed] https://launchpad.net/bugs/861943
<gholms> I suspect I'm doing something wrong, but I'm not sure what.
<smoser> gholms,  it does exit false if it fonds nothing.
<smoser> it probably shoudl exit success.
<gholms> Sounds like that should be a separate bug, then.
<gholms> cloud-init start should later find the EC2 source and make everything work, right?
<smoser> gholms, no.
<smoser> cloud-init-local should start earlier and find nothing
<smoser> cloud-init should then come and find a metadata service from the network
<smoser> cloud-init would be bettter named cloud-init-net
<gholms> I was under the impression that start-local was just supposed to clean up the cache after previous runs.  Why is it looking for a data source?
 * gholms goes through the code again
<smoser> gholms, because there are local data sources
<smoser> (DataSourceNoCloud)
<smoser> and i just added one for openstack "ConfigDrive" yesterday.
<gholms> Sure, but can't those be done during start, not start-local?
<smoser> gholms, well they run earlier.
<smoser> they don't depend on network.
<smoser> so they can influence more of boot.
<smoser> the configdrive one can write /etc/network/interfaces
<gholms> Ah, that's why.
<gholms> So should start-local *and* start exit successfully if there is no data source, or just start-local?
<smoser> i really can't htink of a good reason to not make them exit 0 on non-failure but no datasource found.
<smoser> initially, i'm sure i considered it failure if there was no data source
<smoser> i'm opne to input here.
<gholms> start-local certainly shouldn't fail due to that.
<gholms> I could go either way with start.
<gholms> Both use the same code path at that point, right?  It would probably be easier to just change them both.
<smoser> yeah, the exit is the same place.
<gholms> worksforme
<smoser> i just pushed that change.
<gholms> Thanks
<gholms> So... as long as you're here... :)
<gholms> util.runparts runs ``run-parts --regex .* foo''.
<gholms> Can we kill off the "--regex .*"?
<gholms> Fedora's run-parts script is somewhat braindead.
<gholms> I suppose I should just file a bug.
<ganimede> hello. Is there a way/program to run a partition in a virtual machine, please?
<smoser> gholms, sorry... regarding regex..
<smoser> i tihnk i needed that for some reason.
<smoser> and there is a bug.
<smoser> by default, run-parts skips some files. man page says
<smoser>  ames must consist entirely of ASCII upper- and lower-case letters, ASCII digits, ASCII underscores, and ASCII minus-hyphens.
<smoser> which would seem sufficient, but i sweare i did this for a reason
<smoser> gholms, yeah.. if i search through logs i find
<smoser> http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/revision/99
<gholms> Ouch.
<gholms> smoser: Would a condition of some kind make sense or would you rather I carry a patch?
<smoser> if you can think of a way to do some condition, i'll take that patch for sure.
<gholms> I guess one test could simply be seeing if the file starts with #!/bin/bash.
<gholms> The Debian one is a compiled executable.
<smoser> yuck, thogh.
<gholms> Yup
<smoser> my othe rscary thought was to just implement run-parts in python
<smoser> but i'm almost certain i'll do it incorrectly in some sense
<smoser> at least at first.
<gholms> "for file in sorted(os.listdir(dirp)): subprocess.check_call([file])" ?
<gholms> Since the goal is to run *everything* and all
<gholms> But yeah, there are bound to be corner cases.
#ubuntu-cloud 2012-02-19
<liminal> hello
<liminal> are an ubuntu .ami available for the AWS EC2 free teir?
