SuperMarioRockswhy won't channel #ubuntu let me send messages00:09
SuperMarioRockswhat is this room for00:14
=== msnsachin12 is now known as msnsachin
=== medberry is now known as med_out
=== hugo is now known as Guest44721
=== daker_ is now known as daker
StathisVHello! Three weeks before I was installed the Ati Radeon HD 6670 in my computer under  Ubuntu 10.04.  I had a lot of problems with ati drivers and usualy my system in booting,  message me  "checking battery state" and then died . The problems were solved when I made upgraded to 11.04 with "radeon" vga driver. But suddenly yesterday the same message had rise. What should I do to fix13:10
StathisVit? I was searching the internet, but nothing useful found.13:10
head_victimStathisV: this is not a general support channel, try #ubuntu13:10
ubot2The Ubuntu Classroom is a project which aims to tutor users about Ubuntu, Kubuntu and Xubuntu through biweekly sessions in #ubuntu-classroom - For more information visit https://wiki.ubuntu.com/Classroom13:10
StathisVhead_victim ok :) thnx :)13:12
=== AndroUser is now known as CloudAche84_droi
=== med_out is now known as medberry
* soren taps the microphone15:50
* soren taps the microphone15:50
sorenHello, everyone. Thanks for stopping by.16:00
=== ChanServ changed the topic of #ubuntu-classroom to: Welcome to the Ubuntu Classroom - https://wiki.ubuntu.com/Classroom || Support in #ubuntu || Upcoming Schedule: http://is.gd/8rtIi || Questions in #ubuntu-classroom-chat || Event: Ubuntu Cloud Days - Current Session: Getting started with OpenStack Compute - Instructors: soren
ClassBotLogs for this session will be available at http://irclogs.ubuntu.com/2011/07/26/%23ubuntu-classroom.html following the conclusion of the session.16:00
sorenI should mention I'm currently at OSCON, on the conference wifi, so I *might* disappear.16:00
soren...but so far the wifi has been ok, so here's hoping.16:00
sorenUsually, I start with a run-down of the history of OpenStack and such, but seeing as there's a more general OpenStack session at 1900 UTC, I'll skip that this time.16:01
sorenOk, so "Getting started with Openstack Compute".16:01
sorenI'm probably going to call it "Nova" at times, but it's the same thing.16:01
sorenusually, i start with a run-down of the history of OpenStack and such, but seeing as there's a more general OpenStack session at 1900 UTC, I'll skip that this time.16:02
soren(Also, my shift key seems to be acting up.)16:02
sorenHopefully at the end of this session, you'll have a basic understanding of the components involved in an OpenStack Compute cloud, have a cloud running on your laptop and you'll be able to start and stop instances.16:03
sorenSimple stuff, since we only have an hour :)16:03
sorenLet's start by downloading an image. It'll probably take a while, so we'll save a bunch of time later on if you start downloading it now.16:03
sorenIf you're running 64 bit Ubuntu, grab this:16:03
sorenI.e. just fire up a terminal and run:16:04
sorenwget http://cloud-images.ubuntu.com/releases/11.04/release/ubuntu-11.04-server-uec-amd64.tar.gz16:04
sorenIf you're runing 32 bit ubuntu, grab this instead:16:04
sorenIf you hit any problem along the way, let me know, maybe we can solve them straight away, or at least we can keep track of the problems so that we can find solutions later.16:04
sorenSo,  I'll just assume you're all downloading this now and move on.16:05
sorenOpenStack Compute consists of a number of "internal" and "external" components.16:05
soren"External" components (a term I just came up with, so you probably won't find it in any docs or anything) are things that aren't part of Nova itself, but we need anyway. For our needs today this is just rabbitmq.16:06
sorenRabbitMQ is a message queueing system. It's a core component of a Nova deployment.16:06
sorenIt passes messages between the different components.16:06
sorenIt's the only control communication mehcnaism we have.16:07
sorenmechanism, even.16:07
sorenNova itself has 5 components that we'll work with today.16:07
sorennova-api is the fronten server.16:07
sorenWhenever you -- as an enduser -- want to interact with Nova, this is the component you talk to.16:08
sorenIt exposes the native OpenStack API (which is derived from the Rackspace Cloud Servers API) as well as the EC2 aPI.16:08
sorenWell, a subset of the EC2 API.16:08
sorenIt receives your request over HTTP, authenticates and validates the request.16:09
soren...and turns it into a message and puts it on the message queue for someone else to consume.16:09
sorenYou can have any number of these servers.16:10
sorenThey all act the same: Verify the request and sends it on to some other component for processing.16:10
=== Hugo is now known as Guest95592
sorenNo failover or anything is involved, all components are "hot".16:10
sorennova-compute is the component that actually runs your virtual machines.16:11
sorenIt receives a request from the scheduler, which I apparently should have talked about first, but here we are :)16:12
sorenOk, so this request has information about which image to run, how much memory and disk and whatnot to assign to the virtual machine.16:12
sorennova-compute makes this happen by creating virtual disk images, shoving the AMI image into it, setting up networking and running the virtual machine.16:13
sorenSo, on the bulk of your servers in your cloud, this is the thing you run.16:14
ClassBotkim0 asked: Does rabbitmq choose a node to deliver the message to? if that node fails, does it reroute to some other node ?16:15
sorenIt depends on the type of message.16:15
sorenHm... Let me explain a bit more first, and get back to this.16:16
sorenThe scheduler.16:16
sorenAs you might have guessed, it schedules stuff.16:16
sorenWhen you ask the API server to run a virtual machine it sends the request to one of the schedulers.16:16
sorenThe scheduler is (meant to be) smart.16:17
sorenIt makes the decision about which compute node is meant to run the virtual machine.16:17
sorenWhen it has decided, it sends the message on to the chose compute node.16:18
sorenOk, so rabbit.16:18
sorenThe API server sort of broadcasts the request to all the schedulers.16:18
sorenHere, rabbitmq makes the decision about which scheduler gets the request.16:18
soren...and applies its usual logic about sending the request somewhere else if it doesn't get acked.16:18
sorenfor the scheduler->compute message, it's different, of course.16:19
soren...since it's being sent directly to a specific node.16:19
sorenI'm not sure what the failure modes are there.16:19
sorenThe scheulder doesn't just schedule VM's, though.16:20
sorenIt's supposed to make decisions about which storage node is supposed to host your EBS-like volumes and so on.16:20
sorennova-network is... the network component.16:20
sorenIt hands out IP addressses and in certain network modes it acts as the gateway.16:21
sorenIt also does NAT'ing for elastic IP's and a few other things.16:21
sorenIt doesn't do firewalling. The compute nodes do that (in order to decentralise the firewall and spread the filtering load).16:21
sorenFinally, there's the nova-objecstore.16:22
sorenIt's a simple ple implementation of the S3 API.16:22
sorenYou can use OpenStack without using it, but in Ubuntu we still use it because that's how EC2 and Eucalyptus work, so we can reuse a bunch of documentation an tools and whatnot this way.16:23
ClassBotkim0 asked: Is there any way to specify locality, like launch this VM besides that VM as close as possible to that EBS volume ?16:24
=== Wilczek is now known as multivitamin
sorenI'm not completely sure.16:24
sorenThere's certainly been talk about supporting it, but I don't remember seeing code to do it.16:25
sorenThat said, I've been on holiday and at conferences for a while, so maybe it landed recently.16:25
sorenIt's definitely on the roadmap.16:25
ClassBotkim0 asked: Why does nova-objecstore exist? Why not just swift or mini-swift ?16:25
sorenIf I didn't answer this already, can you rephrase this?16:26
ClassBotkim0 asked: Has openstack dropped the centralized DBs ? wasn't mysql used before?16:26
sorenWe have not.16:26
=== bambee_ is now known as bambee
sorenSo, if you're setting up a multi-machine cloud, you'll need a shared database. E.g. MySQL, PostgreSQL or whatever floats your boat.16:27
sorenWhatever sqlalchemy supports should be fine.16:27
soren"should" being the operative word.16:27
sorenData access works fine for all the different backends, but schema changes have only been tested with mysql, postgresql and sqlite.16:28
sorenThis is a common theme, by th e way.16:28
sorenWe can use a bunch of different DB backends.16:28
sorenWe can also use a bunch of different hypervisors (kvm, Xen, LXC, UML, Xen Server, VMWare, Hyper-V).16:29
sorenDifferent storage backends (HP SANs, iSCSI, AOE)16:29
sorenAdding more drivers is meant to be reasonable easy, so if you want to use something else, that should be possible.16:30
sorenLet us know if that's the case.16:30
ClassBotkim0 asked: Is there code to accelerate certain operations via SAN (like clone volume, snapshotting ..etc)16:30
=== multivitamin is now known as Wilczek
sorenI forget which operations are exposed in abstraciton layer, to be honest. I do believe the backend call to create a snapshot is just "create a snapshot", so if a particular backend has nifty shortcuts to do that, it should totally be possible.16:31
sorenWhether the drivers actually *do*... I don't know.16:31
sorenAlright then.16:32
soren16:32 < Guest95592> QUESTION : is there any docs talk about openstack with Hyper-V ? Or real use case ?16:32
sorenI'm not actually sure of the state of the Hyper-V support. We don't really have the means to test it.16:33
sorenGuest95592: I'll make a note to dig up the docs and send it to you. I'm not sure where they are.16:34
sorenOk, so let's get practical.16:34
sorenI assume you've all downloaded the image.16:34
sorenFor those who just joined:16:34
soren16:03 <+soren> Let's start by downloading an image. It'll probably take a while, so we'll save a bunch of time later on if you start downloading it now.16:35
soren16:03 <+soren> If you're running 64 bit Ubuntu, grab this:16:35
soren16:03 <+soren> http://cloud-images.ubuntu.com/releases/11.04/release/ubuntu-11.04-server-uec-amd64.tar.gz16:35
soren16:04 <+soren> I.e. just fire up a terminal and run:16:35
soren16:04 <+soren> wget http://cloud-images.ubuntu.com/releases/11.04/release/ubuntu-11.04-server-uec-amd64.tar.gz16:35
soren16:04 <+soren> If you're runing 32 bit ubuntu, grab this instead:16:35
soren16:04 <+soren> http://cloud-images.ubuntu.com/releases/11.04/release/ubuntu-11.04-server-uec-i386.tar.gz16:35
sorenI'm assuming you're running Natty.16:35
sorenIs anyone not running Natty?16:35
ClassBotkim0 asked: Would you explain how a large scale (1000?) server deployment may look like. Guidelines would be great. Is it also an inverted tree of a bunch of clusters like eucalyptus?16:35
sorenkim0: A lot of it will be dictated by the network structure you want.16:36
sorenThis is a good question, really. I just write the software, I don't actually run it at scale.16:37
sorenIt's not a strict hierarchy.16:37
sorenAt all.16:37
sorenYou have the message queue, which everything shares.16:38
soren...the database that everything shares.16:38
soren"a number" of API servers. :)16:38
sorenI don't really know how many I'd set up.16:38
sorenIdeally, they'd run as instances on your cloud so that you could scale them as appropriate.16:38
sorenI'd be surprised if one or two of them wouldn't be sufficient, but if you find out that you need a bunch of them, just add more and point them at the same message queue and db and you should be golden.16:39
sorenNumber of storage servers depends on how much storage you want to expose, really.16:40
soren...and how much you expect your users to use them.16:40
sorenNova doesn't really add any overhead there.16:40
sorenIf you expect to have a lot of I/O-intensive stuff going on on your cloud, you'd probably have a lot of servers with a lot of bandwidth.16:40
sorenIf you don't expect a lot of I/O-intensive stuff you might just spring for fewer servers with more disks in each.16:41
sorenI don't think there really are any good, general answers.16:41
sorenThe architecture is really flexible, though, so if you discover that you need more store or bandwidth, you just add more.16:41
sorenSince the architecture is so flat.16:42
sorenOk, so back to our demo.16:42
sorenEveryone runs natty. Great.16:42
sorensudo apt-get install rabbitmq-server unzip cloud-utils16:42
sorenThis install rabbitmq and a few utilities that we'll need in a few minutes.16:43
sorenWhen that is done, do this:16:43
sorensudo apt-get install nova-api nova-compute nova-scheduler nova-network nova-objectstore16:43
sorenI'm not compltely sure this still needs to be two separate steps, but just to be safe, we do it this way.16:43
sorenOk, so next up, we create a user.16:43
sorenIf your name is not "soren" you can specify something else where it says "soren" :)16:44
sorensudo nova-manage user admin soren16:44
sorenThis creates an admin user called "soren"16:44
sorensudo nova-manage project create soren soren16:44
sorenThis creates a "project" called "soren" with the user "soren" as the admin.16:44
sorenNext, we need to create a network.16:45
sorenIf you can't use (beacuse you already use it), make something else up.16:45
sorensudo nova-manage network create 2 2416:45
sorenThis creates two networks of 24 IP's each in the subnet.16:45
sorensudo nova-manage project zipfile soren soren16:46
sorenThis fetches a zipfile with from credentials in it for the user soren for the project soren.16:46
sorenUnpack it:16:46
sorenunzip nova.zip16:46
sorenSource the novarc file:16:46
soren. novarc16:46
sorenNow we upload the image we downloaded earlier.16:46
sorenuec-publish-tarball ubuntu-11.04-server-uec-amd64.tar.gz ubuntu16:47
sorenubuntu-11.04-server-uec-amd64.tar.gz is the filename, "ubuntu" is the name of the S3 bucket you want to use.16:47
sorenNext, we create a key:16:47
soreneuca-add-keypair mykey > mykey.priv16:47
sorenchmod 600 mykey.priv16:48
sorenuec-publish-tarball may take a while.16:48
sorenThe last thing it outputs might look like:16:48
sorenemi="ami-3a0c3765"; eri="none"; eki="aki-279dfe6a";16:49
sorenWe need the first ID there (ami-3a0c3765).16:49
sorenYour id will be different.16:49
sorenWell, probably.16:49
sorenthere's a 1 in 2^32 chance it'll be the same :)16:49
sorenOk, next:16:50
sorenecho '#!/bin/sh' >> myuserdata16:50
ClassBotThere are 10 minutes remaining in the current session.16:50
sorenecho "wget http://f.linux2go.dk:8080/$HOSTNAME/"  >> myuserdata16:50
sorenAnd finally:16:50
soreneuca-run-instances -k mykey -t m1.tiny -f myuserdata "the ami ID we found a bit further up"16:50
sorenIf this works, we'll be able to see your hostname on  http://f.linux2go.dk:8080/16:51
soren...and you'll also be able to ssh into the instance.16:51
sorenIs this working for everyone?16:52
ClassBotThere are 5 minutes remaining in the current session.16:55
sorenI don't really anything more, I didn't know how long this demo thing would take.16:55
sorenAlright, thanks everyone for stopping by.16:57
=== ChanServ changed the topic of #ubuntu-classroom to: Welcome to the Ubuntu Classroom - https://wiki.ubuntu.com/Classroom || Support in #ubuntu || Upcoming Schedule: http://is.gd/8rtIi || Questions in #ubuntu-classroom-chat || Event: Ubuntu Cloud Days - Current Session: UEC on Ubuntu 10.04 LTS - Instructors: TeTeT
ClassBotLogs for this session will be available at http://irclogs.ubuntu.com/2011/07/26/%23ubuntu-classroom.html following the conclusion of the session.17:00
TeTeThello there, glad to have you aboard for a session on UEC on Ubuntu 10.04 LTS17:00
TeTeTthanks to soren for providing us with info on openstack, a very interesting and recent development in cloud space17:01
TeTeTin the next hour I will write about UEC, Ubuntu Enterprise Cloud, as we find it in Ubuntu 10.04 LTS17:01
TeTeTmost of you will already know that the LTS (Long Term Support) releases are quite important to the Ubuntu universe17:01
TeTeTwith 5 years of support for servers, it is a good choice for deploying it in any datacenter17:02
TeTeTdrawback is of course, that you don't get the latest and greatest software on it, you pretty much have to live with what was there in April 2010 for Ubuntu 10.04 LTS17:03
TeTeTso with regards to UEC, we find that the base is eucalyptus 1.6.217:03
TeTeTsome of you might have attended nurmi's talk yesterday on euca 317:03
TeTeTthere's of course quite a lot of features missing in 1.6.2, but if you are in an LTS environment, or plan to set one up, you still can get started with UEC17:04
TeTeTso the most useful document for getting started with it, is on the wiki:17:04
TeTeTit contains various info that to the best of my knowledge mostly still applies to 10.04 LTS17:05
TeTeThowever, due to some problems with service discovery, one might need to manually set up a cloud when doing a packaged install17:06
TeTeTI've detailed the needed steps here, as I need them often when teaching the UEC class:17:06
TeTeTtypically it takes 10-20 minutes to set up a two node cloud with UEC17:07
TeTeTone node acting as the front-end, the other as the node controller17:08
TeTeTwhile this is hardly sufficient for any serious deployment, it's good to get started and you can add more node controllers later on17:08
TeTeTonce the cloud is up and running, you need to either download credentials for accessing i from the web ui or via the euca_conf command on the front-endt17:09
TeTeTI prefer the later, so a $ sudo euca_conf --get-credentials admin.zip17:09
TeTeTwill store the credentials for the admin user in the admin.zip file17:09
TeTeTnext I save these credentials on a client system into ~/.euca-admin/17:10
TeTeTnote that the wiki recommends saving it in ~/.euca, but if you have multiple users on one client system in one account, it's better to have several directories17:10
TeTeTe.g. I then usually create a user 'spindler' that has non-admin privileges and store the credentials in ~/.euca-spindler17:11
TeTeTI think using multiple users for multi-tenancy in the cloud is a very nice feature, so I like to use it17:11
TeTeTonce the credentials are available, I source the accompanying 'eucarc' file and we're ready to go17:12
TeTeTwith euca-describe-availability-zones verbose I do a quick check if the cloud is operational17:12
TeTeT euca-describe-availability-zones verbose17:12
TeTeTAVAILABILITYZONE|- vm typesfree / max   cpu   ram  disk17:12
TeTeTAVAILABILITYZONE|- m1.small0023 / 0024   1    192     217:12
TeTeTAVAILABILITYZONE|- c1.medium0022 / 0022   1    256     517:12
TeTeTAVAILABILITYZONE|- m1.large0011 / 0011   2    512    1017:12
TeTeTAVAILABILITYZONE|- m1.xlarge0005 / 0005   2   1024    2017:12
TeTeTAVAILABILITYZONE|- c1.xlarge0002 / 0002   4   2048    2017:12
TeTeThope I don't get kicked for flooding17:12
TeTeTso here you see a basic UEC in operation. The name of the cluster is 'torstenCluster', you see the private IP of the cluster controller17:13
TeTeTthe lines below the first show how many instances of each instance type you can run17:13
TeTeTthe instance types will look familiar to those having AWS background17:13
TeTeTbasically the command tells me that there's a cluster controller operational and a node controller has been found as well17:14
TeTeTat least one node controller17:14
TeTeTif you only see 0 for free and max, registering the node controller didn't work and you probably need to repeat that step17:14
TeTeTin theory a new node controller in the same subnet as my cluser controller would get picked up automatically17:15
TeTeTin practice it always worked for me, but I've read in bug reports that at times it didn't for other users17:15
TeTeTtalking about the auto registration - there is one caveat17:16
TeTeTdon't setup more than one UEC cloud in the same LAN17:16
TeTeTotherwise the auto registration will bring strange fruits17:16
TeTeTyou can always turn off the auto registration agents in their upstart / init jobs, though, in case you need to17:17
TeTeTe.g. you want to setup a classroom with multiple clouds and one LAN17:17
TeTeTback to our cloud, the next thing to do is getting an image and uploading it to the cloud17:17
TeTeTthe easiest way to do so is using the web interface17:17
TeTeThowever, almost as easy is downloading a tarball from uec-images.ubuntu.com and provisioning that17:18
TeTeTthanks to the developers there are scripts that make this really easy17:18
TeTeTuec-publish-tarball is the one for publishing a tarball from uec-images in the cloud17:18
TeTeTonce this is done, the image resides in /var/lib/eucalyptus/bukkits/<bucket name>/ on the front-end17:19
TeTeTcloud tech speaking we have stored a machine image in a bucket on S317:19
TeTeTwe need to store it in a way the node controller can later download and execute the image file17:20
TeTeTand S3 is one such storage method in UEC17:20
TeTeTnext step is to create a ssh keypair usually17:20
TeTeTthis can be done with euca-add-keypair <keyname>17:21
TeTeTbest is to first check with euca-describe-keypairs which ones already exist17:21
TeTeTwhile the command nowadays blocks creation of another key with the same name as an existing one, this was not always the case in the past ...17:22
TeTeTonce we have an image and a keypair, we can start an instance17:23
TeTeTwhen doing all of this on the command line, rather than using hybridfox or landscape or any other mgmt tool17:23
TeTeTI find it useful to save the identifiers in variables17:23
TeTeTfor example17:23
TeTeTthis is a maverick server image for 64bit from back in January17:24
TeTeTI store the emi identifier in a variable emi17:24
TeTeTso when I want to run an instance of that type, I state17:25
TeTeTeuca-run-instances $emi -k <key name> -t <instance type>17:25
TeTeTI usually neglect the instance type and make do with m1.small, but at times bigger instances are worth it17:25
TeTeTthe euca-run-instances command returns another identifier, this time for the instance17:26
TeTeTI save that in the variable 'inst' or 'server-<name>' or whatever else I like17:26
TeTeTso I can quickly check the status of the instance with $ euca-describe-instances $inst17:26
TeTeTif you're out on your own with a single user and a small cloud, it might not make much sense for restricting the output of euca-describe-instances17:27
TeTeTbut if you have a class full of people, everybody starting and stopping instances, it is useful to only see info on the wanted instance17:27
TeTeTonce the instance is in state running, one can ssh to it17:28
TeTeTssh -i <identity file> ubuntu@<public or private ip>17:28
TeTeTthe -i option we need to use the right private key file, the one that was returned by euca-add-keypair earlier17:28
TeTeTif we need the public or private ip depends on where our client sits17:29
TeTeTif we use the front-end as client, which a lot of people will initially do, the private ip is as good as any17:29
TeTeTif the client is on your laptop or any other system not hooked up to the clouds infrastructure, you need to use the public ip17:30
TeTeTwhen you have done all the work needed by the instance, you can terminate it with 'euca-terminate-instances'17:30
TeTeTso that covered the basic operation of a UEC cloud17:31
TeTeTfirst install the systems, either by CD or by package17:31
TeTeTsecond get the credentials17:31
TeTeTthird check if the cloud is operational17:31
TeTeTnext make an image available17:31
TeTeTstart an instance of that image17:32
TeTeTtest via ssh or whatever other service the instance may provide17:32
TeTeTbut you don't need to stop there, there's plenty to still explore17:32
TeTeTlike attaching persistent storage devices to the instance17:32
TeTeTallocating public ip addresses to the instance17:32
TeTeTbefore we cover the persistent storage, let's see why we have the need for that17:33
TeTeTwhat happens when an instance is started?17:33
TeTeTa node controller sooner or later gets tasked by a cluster controller to run an instance17:34
TeTeTthat instance is of a specific emi, eucalyptus machine image17:34
TeTeTthe node controller checks a local cache if the emi is there, and if not, the emi is transferred to the node controller via S317:34
TeTeTthe node controller then copies the image to an instance directory17:35
TeTeTinside the instance directory the image is configured according to the start parameters of euca-run-instances17:35
TeTeTe.g. an ssh key is injected17:35
TeTeTe.g. a cloud init file is added to the boot sequence of the instance17:35
TeTeTsome more magic happens and the once xen like image is now a kvm image17:36
TeTeTand kvm is used to boot the instance17:36
TeTeTso in a way eucalyptus and UEC is like kvm on steroids17:36
TeTeTwhen you want to get rid of an instance, you stop it with euca-terminate-instances17:37
TeTeTand what happens on the node controller is that the kvm process stops _and_ the runtime directory containing the image is deleted17:37
TeTeTthis means that any state in the instance is gone forever17:37
TeTeTof course most services require to save some state somewhere. This could be an external database server, or a file server.17:38
TeTeTor, as we look at now, EBS, the elastic block storage service17:38
TeTeTin EBS a device appears on an instance that is mapped to a file on a storage server, the EBS server17:39
TeTeTthere are two methods by which EBS is transported, ata over ethernt (AoE) or iscsi. In Ubuntu 10.04 LTS we only have AoE17:39
TeTeTthis implies a restriction when using 10.04 LTS, as the EBS server has to reside on the same LAN as the node controllers it serves17:40
TeTeTas AoE does not route17:40
TeTeTwith euca-create-volume we can create a new volume for storage on the EBS storage server17:41
TeTeTonce this has finished, it can be assigned to an instance with euca-attach-volume17:41
TeTeTeuca-attach-volume <vol id> -i $inst17:41
TeTeToops, there's -d <device> missing17:42
TeTeTdevice would be sdb or sdc or anything17:42
TeTeTbut to my finding the instance will just pick the next device name anyhow17:42
TeTeTwith the device attached to the instance, one can create a partition table, filesystem and whatever on the device17:43
TeTeTjust as with a regular device17:43
TeTeTthe nice thing is, that you can snapshot these devices17:44
TeTeTe.g. you attach the device to an instance, put the needed data there, detach it and snapshot it17:44
TeTeTbased on that snapshot you can create new volumes that are clones of the snapshot17:45
TeTeTmight be nice if you have read only data that you need on all instances of your application17:45
TeTeTthat's it pretty much for EBS17:46
TeTeTlastly I want to cover elastic IPs17:46
TeTeTwith help of euca-describe-addresses you see which IPs are public in your UEC17:46
TeTeTyou can either randomly get one or pick one with euca-allocate-address17:46
TeTeTthen this ip can be assigned to a specific instance with euca-associate-address17:47
TeTeTin this way you can make sure that a certain service is always reachable by the same IP, minus the time it takes to do the associate dance17:47
TeTeTwell, that was all I wanted to cover. Thanks for following. If you have any questions, either use the classbot or you can reach me as TeTeT on #ubuntu-cloud usually17:48
ClassBotThere are 10 minutes remaining in the current session.17:50
ClassBotThere are 5 minutes remaining in the current session.17:55
m_3can I type in here yet?17:59
nigelbyes :)17:59
m_3ok, I'll go ahead and get started17:59
m_3Hi, I'm Mark M Mims (hence the m_3)18:00
=== ChanServ changed the topic of #ubuntu-classroom to: Welcome to the Ubuntu Classroom - https://wiki.ubuntu.com/Classroom || Support in #ubuntu || Upcoming Schedule: http://is.gd/8rtIi || Questions in #ubuntu-classroom-chat || Event: Ubuntu Cloud Days - Current Session: Node.js/Mongo with Ensemble - Instructors: m_3
ClassBotLogs for this session will be available at http://irclogs.ubuntu.com/2011/07/26/%23ubuntu-classroom.html following the conclusion of the session.18:00
m_3On the Ensemble team working to build out the base set of formulas we have to work with in ensemble18:00
m_3I'd like to go into a little detail today about the different kinds of formulas ensemble can work with18:01
m_3at this point I hope that you've seen the earlier presentation on 'getting started with ensemble'18:01
m_3I'll be demoing stuff in byobu-classroom, where you can watch along18:02
m_3you can either open http://ec2-67-202-23-199.compute-1.amazonaws.com/ up in a browser18:02
m_3or ssh there directly as "guest" with password "guest"18:02
m_3So, how do you use ensemble to deploy a node.js app?18:03
m_3the short version (we'll go repeat in more detail)18:03
m_3$ ensemble bootstrap18:04
m_3$ ensemble deploy --repository .. mynodeapp18:04
m_3ensemble deploy --repository .. mongodb18:04
m_3ensemble deploy --repository .. haproxy18:04
m_3to deploy all the services as we expect18:04
=== basso_ is now known as basso
m_3here, we have a simple (exposed) mongodb service18:05
m_3a node.js app that sits above the mongodb18:05
m_3and haproxy that faces the outside world18:06
m_3we deploy the services, then relate them:18:06
m_3ensemble add-relation mongodb mynodeapp18:06
m_3ensemble add-relation mynodeapp haproxy18:06
m_3then wait for the relation hooks to complete18:06
m_3the outcome of that is shown in ssh18:07
m_3The point of this talk is that there are two different kinds of services involved here:18:08
m_31.) "canned" services... like haproxy and mongodb18:08
m_3these are services you typically just configure and use18:09
m_3they're tailored to your infrastructure through configuration parameters18:09
m_32.) what I like to call "framework" formulas18:09
m_3these are formulas for services that you write18:10
m_3i.e., mynodeapp is an application that I keep under my own revision control system18:10
m_3a framework formula is a formula that helps you deploy such a service alongside other canned services18:11
m_3let's look at an example18:11
m_3mynodeapp is a _super_ simplistic node.js app18:12
m_3there's a basic http server listening on
m_3that tracks hits to the page18:13
m_3and sticks them in a mongo database18:13
m_3easy peasy18:13
m_3it reads some information from a config file...18:13
m_3that's just simple json18:14
m_3I've written a basic ensemble formula to wrap the installation and deployment of that node.js application18:15
m_3the metadata.yaml shows where it fits within my overall infrastructure18:15
m_3this consumes a db, and provides a http interface18:15
m_3I can of course attach monitoring services, remote logging services, etc18:16
m_3there's a little cruft in here about formula config18:16
m_3that's a new feature that recently landed18:17
m_3so all of that should be replaced with 'config-get <param>'18:17
m_3the installation of node itself and npm are pretty straightforward18:17
m_3try to use packages when you can18:17
m_3when you're on HEAD or living closer to the edge, you can pull and install from source if you want18:18
m_3that's worth mentioning again...18:18
m_3packages are great for stability18:18
m_3and often what the ops guys want18:18
m_3developers want the bleeding edge all the time18:19
m_3ensemble formulas support either18:19
m_3so it becomes a policy decision within your group18:19
m_3but anyway... I'm pulling my node.js from github18:19
m_3you can use any number of other options18:20
m_3the key is to install the packages first for the VCS you use18:20
m_3note there's a need for some idempotency guards18:21
m_3this is most important for long lifecycle services18:21
m_3let me pause for a sec for questions...18:22
m_3ok, so to summarize the first part of what we're looking at...18:22
m_3this is a formula to deploy my node.js app right alongside all the 'canned' service formulas18:23
m_3this is the 'install' hook that gets called upon service unit deployment18:23
m_3this install hook:18:23
m_3loads some configuration18:24
m_3then goes about installing18:24
m_3and then my application (pulled from github)18:24
m_3and then delays any further startup/config until we have the right parameters to fill in from the mongodb server18:25
m_3so that's simple enough18:25
m_3now, the real magic of ensemble18:25
m_3are relations18:26
m_3this sets ensemble apart18:26
m_3the lines of demarcation between what's specific to a service18:26
m_3and what's really specific to the relations between the services18:27
m_3here, we see the mongodb-relation-changed hook18:27
m_3this isn't called until the mongodb service is related to our mynodeapp service18:28
m_3start at the bottom...18:28
m_3note that we don't actually start the "mynodeapp" service18:28
m_3until we've set the relation parameters18:28
m_3this makes sense18:28
m_3if we look at the config.js again for the app18:28
m_3the default paramters the node.js app is using expect mongodb to be local18:29
m_3ok, if you noticed during the install hook, I _do_ have it installed locally18:29
m_3but that was just for testing18:29
m_3in general, the mongodb service is external18:30
m_3so the node.js app would barf if we started it with this inf18:30
m_3so, the code before starting the service is just to get the right connection information from the db18:30
m_3when a relation is created18:31
m_3ensemble opens a two-way comms channel between the services through the use of 'relation-get' 'relation-set'18:31
m_3the particular mongodb service we're using is pretty simplistic18:31
m_3let's look at the other end of this relation18:32
m_3that's it!18:32
m_3when a service connects to the mongodb service, mongodb just sends it's connection point18:33
m_3a more mature version of this would have access control, ports specified, etc18:33
m_3but that's enough info for a first pass at understanding ensemble relations18:34
m_3the mynodeapp service just grabs the relation information (here just the host)18:34
m_3and crams it in the config file18:35
m_3we can go out to the live service and see what the config file looks like18:35
m_3first notice the relation status between the mongodb and the mynodeapp services18:36
m_3I can ssh directly to one unit18:36
m_3and see that the relation hook filled in the ec2 internal address for the mongodb service18:37
m_3and the port's just default18:37
m_3cool... questions so far?18:37
m_3ha, ok... I continue18:38
m_3next, let's look at what happens when we fire up additional units18:38
m_3ec2's been quite slow today, so we'll wait a bit18:39
m_3notice as this comes up that the services 'mynodeapp' and 'mongodb' are already related18:40
m_3all we did with ensemble add-unit was to add an additional service unit for the mynodeapp service18:40
m_3(horiz scaling)18:40
m_3so ensemble spins up a new instance18:41
m_3runs the install hook we saw before18:41
m_3let's look again for a sec...18:41
m_3notice there's nothing in here that would depend on another service unit18:41
m_3we piulled some config from the formula18:42
m_3we installed stuff that's going to be in common with all instances of our node.js app18:42
m_3the relation is the only real information that the node.js app "nodes" or units would share18:43
m_3ensemble just calls the relation hooks after installing the new service unit18:43
m_3so then this new service unit should get the same relation-specific configuration that the first one had18:44
m_3(and cram it in config.js for the app)18:44
m_3ugh... I thought for sure I rambled enough for ec2 to catch up18:45
m_3well anyway, while we're waiting, we can test a couple of things18:45
m_3note that you can hit the first mynodeapp service unit18:46
m_3on machine 218:46
m_3or ec2-72-44-35-213.compute-1.amazonaws.com18:46
m_3and see the node.js app in action18:46
m_3(remember we're on port 8000)18:46
m_3you can try http://ec2-72-44-35-213.compute-1.amazonaws.com:8000/18:47
m_3as well as http://ec2-72-44-35-213.compute-1.amazonaws.com:8000/hits18:47
m_3should give us some additional traffic in the db18:47
m_3this is open in this case18:48
m_3it wouldn't be in real life18:48
m_3ensemble has features to control ec2 security groups18:48
m_3so we'd have this webservice exposed on port 8000 only on the internal ec2 interface18:48
m_3you'd have to go through haproxy for public access18:49
m_3hey, ok, our new mynodeapp service unit is up18:49
m_3mynodeapp/1 on machine 418:49
m_3note that you can hit that one independently too18:49
ClassBotThere are 10 minutes remaining in the current session.18:50
m_3SpamapS: asks "how do we get to the port if I haven't exposed that port"18:50
m_3answer is it's all open at the moment for the class18:50
m_3ok, so let's check out the config file in the new node and make sure that the relation added the right info18:51
m_3so when we hit either node, they're writing the hits into the common external mongodb datastore18:52
m_3sorry, I keep using bash aliases...18:52
m_3es='ensemble status'18:52
m_3ok, so what's missing here?18:52
m_3notice that we have a haproxy service up18:53
m_3but no relations18:53
m_3this will call hooks to relate the webservice http interface provided by each mynodeapp to the haproxy balancer18:54
m_3really it just tells haproxy its hostname and port (8000 here)18:54
m_3the haproxy service is written so that when a webservice joins, it adds it to the roundrobin queue18:55
ClassBotThere are 5 minutes remaining in the current session.18:55
m_3ok, almost done18:55
m_3now, we can hit ec2-174-129-107-151.compute-1.amazonaws.com on port 8018:55
m_3and it's balancing between the node.js nodes18:56
m_3our nodes18:56
m_3ok, so to wrap up18:56
m_3two types of formulas18:56
m_31. canned formulas18:56
m_32. "framework" formulas18:56
m_3mongodb and haproxy are examples of canned ones18:57
m_3mynodeapp is a framework one18:57
m_3the intended use is to fork an example and tailor to your needs18:57
m_3thanks all...18:57
=== ChanServ changed the topic of #ubuntu-classroom to: Welcome to the Ubuntu Classroom - https://wiki.ubuntu.com/Classroom || Support in #ubuntu || Upcoming Schedule: http://is.gd/8rtIi || Questions in #ubuntu-classroom-chat || Event: Ubuntu Cloud Days - Current Session: OpenStack: An open cloud infrastructure project - Instructors: ttx
ClassBotLogs for this session will be available at http://irclogs.ubuntu.com/2011/07/26/%23ubuntu-classroom.html following the conclusion of the session.19:00
ttxHi everyone19:00
ttxMy name is Thierry Carrez, I'm the release manager for the OpenStack project19:01
ttxRaise your hand in #ubuntu-classroom-chat if you're here for the OpenStack Project intro session !19:01
ttxI'm also an Ubuntu core developer, working on Ubuntu Server, time permitting19:01
ttxIn this session I'd like to introduce what OpenStack is and how you can participate19:02
ttxYou can ask questions in #ubuntu-classroom-chat, like this:19:02
ttxQUESTION: what is OpenStack ?19:02
ttxI'll review them periodically.19:02
ttxSo, what is OpenStack ?19:03
ttxIt's an open source project that creates software to run a cloud infrastructure.19:03
ttxIt allows you to be an IaaS (infrastructure as a service) provider19:03
ttxIn short, it's software you can run to create a competitor to Amazon Web Services or the Rackspace Cloud19:03
ttxBut it also scales down so that you can use it to manage your private SMB computing resources in a cloudish way.19:04
ttxThe project is built around 4 "open":19:04
ttxOpen source: The software is Apache-licensed, and the whole project is open source19:04
ttxSo no "enterprise edition" that keeps the tasty features19:04
ttxOpen design: We have open design summits, much like UDS, every 6 months19:04
ttx(The next one is in Boston, October 3-5 !)19:05
ttxAlso everyone can propose a feature, we use Launchpad for blueprints... but be ready to implement it if you do :)19:05
ttxOpen development: We use DVCS (bzr and git) with merge proposals and public comments, so you know why a particular piece of code is accepted or not19:05
ttxThe code is written in Python, which makes it easier for everyone to read the code, understand it, investigate why it fails and propose patches19:06
ttxOpen community: We have community-elected project leaders and seats on the project policy board, we do meet weekly on IRC19:06
ttx(next meeting in 2 hours in #openstack-meeting !)19:06
ttxWe have about 100 different developers that have committed code, from more than 20 different companies19:06
ttxQuestions so far ?19:06
ttxNo questions, so everyone knows OpenStack already, good good19:07
ttxOpenStack is made of several subprojects19:07
ttxWe have "core projects", that follow all our rules and are part of the official OpenStack release19:07
ttxWe also have "Incubated" projects, that learn the process and should become core projects one day19:08
ttxFinally we have "Related" projects, that are created in the OpenStack community but should not become core projects for one reason or another19:08
ttxWe used to have a 3-month release cycle, but at the last design summit we decided to switch to a 6-month release schedule, with monthly milestones19:09
ttxMilestones can be used to evaluate new features as they land in the projects.19:09
ttx(For example we'll release the diablo-3 milestone Thursday !)19:09
ttxThe 6-month release is aligned with the Ubuntu release cycle, so that you can get the latest OpenStack release in the latest Ubuntu release.19:09
ttxTherefore OpenStack 2011.3 (also known as "Diablo") should be released on September 22, and included in 11.10.19:10
ttxQuestions ?19:10
ClassBotrwh asked: does Canonical have plans to launch an AWS competitor based on this?19:10
ttxI'm not part of Canonical (anymore) but I don't think so.19:10
ttxIf there are no other questions on the project itself, let's go into more details in the 3 current core subprojects19:11
ttxThe first one (and most mature) is Swift (OpenStack Object storage)19:12
ttxYou can use it to run a raw cloud storage provider, very much like Amazon S3.19:12
ttxIt allows your users to PUT and GET small or very large binary files, and get them properly replicated and available.19:12
ttxIt's very mature. It's actually the code that runs Rackspace "Cloud files", but also Internap or Nephoscale "Cloud storage".19:12
ttxIt scales horizontally using a share-nothing architecture, so you can add up hosts to cope up with the load19:13
ttxYou deploy it using commodity hardware, and Swift replication takes care of the redundancy of your data19:13
ttxSo you don't need expensive SANs or RAID setups.19:13
ttxQuestions ?19:13
ClassBotkoolhead17 asked: any duration for support cycle? like ubuntu?19:13
ttxOpenStack is being distributed through downstream distributions, like Ubuntu19:14
ttxThey all have their own support cycles, we don't have a particular one19:15
ttxAt this point we don'"t backport anything but critical issues and security issues into past releases19:15
ttxfor for long term support I advise that you opt for one of those distributions of OpenStack.19:15
ttxIf there are no more questions, we'll switch to Nova19:16
ttxNova (Cloud Compute) is the biggest of the three current core projects.19:17
ttxA few hours ago soren gave you the keys to it, I'll repeat the essentials now19:17
ttxYou can use it to run a cloud compute provider, like Amazon EC2 or Rackspace Cloud Servers19:17
ttxIt allows your users to request a raw virtual machine (based on a disk image from a catalog)19:17
ttxSo for example users can request a VM with Ubuntu Server 11.04 on it, then access it using SSH, customize and run things on it.19:18
ttxThe architecture is a bit complex. We have several types of nodes, and you can run any number of each of them to cope with load19:18
ttxWe have API nodes that receive requests, Scheduler nodes that assign workers...19:18
ttxNetwork nodes that handle networking needs, Compute and Storage workers than handle VMs and block storage.19:18
ttxEverything communicates using a RabbitMQ message queue19:19
ttxIt's *very* modular, so you have to choose how you want to deploy it.19:19
ttxFor example it supports 8 different virtualization technologies right now:19:19
ttxQEMU, KVM, UML, LXC, Xen, Citrix XenServer, M$ HyperV and VMWare vSphere19:19
ttxUbuntu by default should concentrate on two of those: KVM and LXC19:19
ttxNova is still fast-moving, but it's used in production at NASA Nebula cloud19:20
ttxand it will be deployed this year to replace current Rackspace Cloud Servers software19:20
ttxQuestions ?19:20
ClassBotkoolhead17 asked: how feasible it is to use openstack in production environment due to the nature of rapid change in architecture/feature every new release?19:20
ttxDepends on the component.19:21
ttxSwift is very mature and slow moving now, you can certainly easily deploy it in production19:21
ttxNova is still changing a lot, though that will calm down after Diablo. At this point running it in production requires a good effort to keep up19:22
ttxSome deployment options are more tested, and therefore more stable than others19:22
ttxSo it is feasible... and will become more feasible as time passes19:22
ttxAny other question on Nova before we switch to Glance ?19:23
ttxok then19:24
ttxGlance (OpenStack Image service) is the latest addition to the core projects family19:24
ttxIt's a relatively-simple project that handles VM image registration and delivery19:24
ttxSo your users can use it to upload new disk images for use within Nova19:24
ttxIt supports multiple disk formats and multiple storage backends19:24
ttxSo disk images end up being stored in Swift or S319:25
ttx(or locally if you can't afford that)19:25
ttxAs far as stability is concerned, I'd say it's on par with Nova, and maturing fast.19:25
ttxQuestions on Glance ?19:25
ttxhah! crystal clear, I see19:26
ttxSo, what can *you*, Ubuntu user, do with it ?19:26
ttxYou can try it. We provide several distribution channels...19:26
ttxStarting with 11.04, the latest OpenStack version is released in Ubuntu universe19:26
ttx(And the "Diablo" release should be in main for 11.10)19:27
ttxIf you need something fresher (or running on LTS), you can use our PPAs:19:27
ttxWe have release PPAs (with 2011.2 "Cactus") for 10.04 LTS, 10.10, 11.04 and Oneiric19:27
ttxWe also have "last milestone" PPAs and "trunk" PPAs for the same Ubuntu releases19:28
ttx(The "trunk" PPA contains packages built from the latest commit in the trunk branch !)19:28
ttxSee http://wiki.openstack.org/PPAs for more details19:28
ttxYou don't need a lot of hardware to test it19:28
ttxYou can actually deploy Nova + Glance on a single machine19:29
ttxIt's easier to use a real physical machine to be able to test virtualization correctly19:29
ttxSwift would prefer a minimum of 5 servers (to handle its redundancy)...19:29
ttxbut you can fake it to use the same machine, or you can (ab)use virtual machines to test it19:29
ttxThat's about all I had in mind for this presentation, so we can switch to general Q&A19:30
ttxFeel free to join our community: test, report bugs, propose branches fixing known issues19:30
ttxThe best way to ensure it's working for *your* use case is actually to try it and report bugs if it doesn't :)19:30
ClassBot_et asked: I am confused by the flow of control in the architecture diagram here http://ur1.ca/4sni7. can you shed more light on how each component interacts with each other and how comm to and from outside flows?19:31
* ttx looks19:31
ttxok, let's take an example19:31
ttxA call to create a server19:32
ttxrun_instance in EC2-talk19:32
ttxYou use a client library or CLI that will talk the EC2 or the OpenStack APi19:32
ttxyour request will be received by a nova-api node19:32
ttxYou can run any number of those, in order to cope with your load19:33
ttxnova-api will place your request in the queue19:33
ttxnova-scheduler pick up requests in the queue19:33
ttxagain, you can run any number of them in order to cope with your load19:33
ttxscheduler looks at your request and determines where it should be handled19:34
ttxhere, it determines which nova-compute host will actually run your VM19:34
ttxscheduler places back on the queue a message that says "compute node X should run this query19:34
ttxnova-compute node X picks message in the queue19:35
ttxit retrieves disk image from Glance, starts your VM19:35
ttxupdated DB so that if you run describ_instances you can see it's running, etc19:36
ttxall the other arrows are for other types of requests19:36
ClassBot_et asked: scheduler decides based on wat the load balancer says?19:36
ttxno, scheduler decides based on the scheduler algorithm you use19:36
ttxthere are multiple scheduler algorithms available, and you can easily plug your own19:37
ttxthe default ChanceScheduler is just a round robin19:37
ClassBotBuZZ-T asked: are there any possibilities to automatically scale instances (e.g. by load)19:37
ttxThat would be the role of tools above the IaaS layer19:38
ttxScalR, Rightscale, etc all provide tools that monitor your IaaS and react accordingly19:38
ttxbut then you tread into PaaS territory19:38
ttxand OpenStack is already very busy trying to cover the IaaS space19:39
ttxhttp://wiki.openstack.org/ is a good starting point for all things19:39
ttxWe also hang out in #openstack (support) and #openstack-dev (development) on Freenode !19:40
ttxAny other question ? Or let me know if any of my answers was incomplete or confusing19:40
ClassBot_et asked: how are IPs assigned? does compute do it thru scheduler? or is it the scheduler  all by itself? ( if u ve time )19:41
ttxYou define networks with scores of available IP addresses, and the network node just picks one that is available19:41
ttxThere is an ambitious project to separate the network components into a set of projects19:42
ttxto achieve complete and complex network virtualization19:42
ttxCheck out Quantum, Melange and Donabe in Launchpad19:42
ttxIf there are no other questions, I'll close this online event, since this is the last session19:43
ttxkim0 is not around, but asked me to thank all attendees19:44
ttxFor further questions, feel free to hit #ubuntu-cloud19:44
ttxwhere most of the people that presented hang out anyway :)19:44
ttxThank you all for making Ubuntu the best platform for the cloud, and on the cloud19:45
ClassBotThere are 10 minutes remaining in the current session.19:50
ClassBotThere are 5 minutes remaining in the current session.19:55
ClassBotLogs for this session will be available at http://irclogs.ubuntu.com/2011/07/26/%23ubuntu-classroom.html20:00
=== ChanServ changed the topic of #ubuntu-classroom to: Welcome to the Ubuntu Classroom - https://wiki.ubuntu.com/Classroom || Support in #ubuntu || Upcoming Schedule: http://is.gd/8rtIi || Questions in #ubuntu-classroom-chat ||
=== koolhead17 is now known as koolhead17|afk
=== yofel_ is now known as yofel
=== MO is now known as Guest83118
=== medberry is now known as med_out

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!