=== bootri is now known as Omega === niemeyer_dinner is now known as niemeyer === mintos is now known as mintos_afk === mintos_afk is now known as mintos_out === mintos is now known as mintos_out === mintos_out is now known as mintos === _LibertyZero is now known as LibertyZero === msnsachin12 is now known as msnsachin === _LibertyZero is now known as LibertyZero === shrewduh_ is now known as shrewduh === daker_ is now known as daker === Harry is now known as Guest36736 === ziviani is now known as JRBeer === nigelbabu is now known as nigelb === daker_ is now known as daker === yofel_ is now known as yofel [13:44] #ubuntu-classroom-chat === goom is now known as GooOm === daker_ is now known as daker === niemeyer is now known as niemeyer_lunch [15:42] hello all [15:48] #ubuntu-classroom-chat [15:53] * koolhead17 kicks nigelb [15:54] Hi folks .. Ubuntu Cloud Days starting in around 5mins here in #ubuntu-classroom .. You can discuss and ask questions in #ubuntu-classroom-chat .. Please feel free to tweet and share this info with your friends [15:56] Those unfamiliar with IRC can simply use this web page http://webchat.freenode.net/?channels=ubuntu-classroom%2Cubuntu-classroom-chat [15:59] hi all [16:00] ok... [16:00] so shall i start, mr kim0 ? [16:00] You might indeed [16:01] Hi, my name is Scott Moser. I'm a member of the Ubuntu Server team. [16:01] For the past 18 months or so, I've been tasked with preparing and managing the "Official Ubuntu Images" that can be used on Ubuntu Enterprise Cloud (UEC) or on Amazon EC2. === ChanServ changed the topic of #ubuntu-classroom to: Welcome to the Ubuntu Classroom - https://wiki.ubuntu.com/Classroom || Support in #ubuntu || Upcoming Schedule: http://is.gd/8rtIi || Questions in #ubuntu-classroom-chat || Event: Ubuntu Cloud Days - Current Session: Rebundling/re-using Ubuntu's UEC images - Instructors: smoser [16:01] Hi, my name is Scott Moser. I'm a member of the Ubuntu Server team. [16:01] For the past 18 months or so, I've been tasked with preparing and managing the "Official Ubuntu Images" that can be used on Ubuntu Enterprise Cloud (UEC) or on Amazon EC2. [16:01] Logs for this session will be available at http://irclogs.ubuntu.com/2011/03/24/%23ubuntu-classroom.html following the conclusion of the session. [16:02] Some links, for reference: [16:02] - https://help.ubuntu.com/community/UEC/Images [16:02] - http://uec-images.ubuntu.com/releases/ [16:03] The first gives some general information about the images, the second gives access to download the images for use in UEC or Eucalyptus and AMI ids on Amazon. [16:03] The subject of my discussion here is "rebundling/re-using Ubuntu's UEC images". [16:03] "Rebundling" is the term used for taking an existing image (or instance), making some modifications to it, and creating a new image from it. [16:03] With the Ubuntu images on UEC or EC2, There are generally 3 ways to rebundle an image. [16:03] * use a "bundle-vol" command from either euca2ools (euca-bundle-vol) or ec2-ami-tools (ec2-bundle-vol) [16:03] * use the EC2 'CreateImage' interface. [16:03] * make modifications to a pristine (unbooted) image via loop-back mounts [16:04] I'll talk a little bit about each one of these, and then open the floor to questions. [16:04] == General Re-Bundling Discussion == [16:04] The "Official Ubuntu Images" are generally stock Ubuntu server installs. As with any default install, they're not much use out of the box. [16:05] There are 2 basic ways to use Ubuntu images on EC2. [16:05] * "rebundle" one and have your own private (or public) AMI. (as we're discussing here) [16:05] * use the stock images, and customize the instance on first boot via scripted or manual methods. [16:05] We've gone to a fair amount of trouble to make them generally malleable so that you can use them without the need to rebundle. [16:06] Cloud-init (https://help.ubuntu.com/community/CloudInit) was developed to reduce the need for users to need to maintain their own AMIs. Instead, the images are easily customizable on first boot. [16:06] kim0, has put together several blog posts about how to get cloud-init to do your bidding so its ready to use once you attach to it. [16:07] There is some work in maintaining a rebundled image, and, it you can remove that effort by using stock images. [16:07] s/it you/it / [16:07] bah. you understand what i was saying. you can remove the effort of maintaining a rebundled image by using a stock image. [16:08] http://foss-boss.blogspot.com/ is kim0's blog. [16:08] you should all have that bookmarked and available in your RSS reader of choice [16:08] :) [16:09] So, while there are some costs involved in rebundling, there are reasons to rebundle. If you have a large stack that you're installing on top of the stock image, it may take some time for you to do so. [16:09] Rebundling generally allows you to have a more "ready-to-go", "custom" image. [16:09] By rebundling an image, you can add your stack and then reduce the bringup time of your custom AMI. [16:10] any questions ? [16:10] == Bundle Volume == [16:10] Using 'ec2-bundle-vol' was the first way that I'm aware of to rebundle. The euca2ools also provide a work-alike command 'euca-bundle-vol'. [16:11] The way most people use this tool is to [16:11] * boot an instance that they want to start from [16:11] * add some packages to it, make some changes ... [16:11] * issue the rebundle command [16:12] [sorry, issue the 'ec2-bundle-vol' or 'euca-bundle-vol' command] [16:12] what this does is basically copy the contents of the root filesystem into a disk image [16:12] and then package that disk image up for uploading [16:13] as you can imagine, simply doing something like "cp -a / /mnt" (ignoring the recursive copy of /mnt) is not the most "clean" thing in the world. [16:14] the euca-bundle-vol command and ec2-bundle-vol command both include some OS specific hacks , so they dont copy certain files. [16:14] and, inside the images themselves, we've made some changes to make this "just work". [16:15] once you've set up the euca2ools, you might rebundle with something like: [16:15] sudo euca-bundle-vol --config /mnt/cred/credentials --size 2048 --destination /mnt/target-directory [16:16] !question [16:16] hm.. [16:17] there was a question as to whether this applied to openstack [16:17] largely, openstack's EC2 compatibility should make this work. [16:18] i've not tried rebundling an image in openstack exactly, but i do know that the euca2ools interact with openstack fine. and copying a filesystem is not really cloud specific [16:19] koolhead17 asked: Is this class limited to eucalyptus image bundling or openstack as well ? [16:19] right. so my previous comments attempted to address that question [16:19] kim0 asked: is euca-bundle-vol only for running instances, can't I poweroff an instance and bundle its disk while powered off [16:20] euca-bundle-vol only runs in instances. [16:20] euca-bundle-image (and ec2-bundle-image) take a filesystem image as input. [16:21] after using euca-bundle-image (or ec2-bundle-image) you then have to upload and register the output [16:21] i generally would suggest using 'uec-publish-image' instead, which is a wrapper that does those three things. The most recent version of this tool in natty allows you to use either the ec2- tools or euca2ools under the covers. [16:22] koolhead17 asked: but open-stack doesnot use the ramdisk part of image [16:22] i might be missing something. [16:22] it is my understanding that the issue with the ubuntu images and openstack was that openstack was hard coded to expect a ramdisk [16:23] where as the 10.04 and beyond images from Ubuntu do not use a ramdisk, so none was available in the tarball that you download. [16:23] i believe that a.) that bug is fixed [16:24] b.) there *is* in openstack a way to boot a instance with an internal kernel, ie not having a separate kernel/ramdisk at all, but relying on the bootloader installed in the disk image [16:24] boy... i'm getting loads of questions, and i'd like to kind of get back to my over all plan, and then i can take questions. [16:24] rather than sitting in interupt mode for the whole hour [16:25] we will *definitely* have time to take questions, so please queue them up in #ubuntu-classroom-chat [16:25] now where was i... [16:26] so, after bundling, then you have to use euca-upload-bundle or ec2-upload-bundle and -register to register your image. [16:26] I should have noted above, that this "bundle-vol" really is only for instance-store images. [16:26] Eucalyptus (in 2.0.X) only supports instance store images. [16:27] i believe that they plan to have EBS root images in the future. [16:27] So, that brings us to the second type of bundling [16:27] == CreateImage == [16:28] When amazon began offering EBS root instances, they added an API call called 'CreateImage' [16:28] CreateImage is an AWS API call that basically does the following: [16:28] * stop the instance if it is not stopped [16:28] * snapshot it's root volume [16:28] * register an AMI based on that snapshot [16:28] * start the instance back up. [16:28] The CreateImage api is exposed via a command line tool (http://docs.amazonwebservices.com/AWSEC2/latest/CommandLineReference/ApiReference-cmd-CreateImage.html) and also via the EC2 Web Console. [16:29] This feature makes it dramatically easier for anyone to create a custom AMI. There is literally one button that you push in the EC2 Console, and then type in a name and description. [16:29] I would generally recommend using CreateImage if you're using an EBS based image. It is an extremely useful wrapper, and will get your a consistent filesystem snapshot. [16:30] Once you have a snapshot id of a filesystem, you could actually fairly easily upload an instance-store image based on that snapshot. [16:30] this is left as an excercise to the reader. [16:31] So, the final way of rebundling an image [16:31] == modify pristine download images == [16:31] Few other image producers on EC2 make their images available as filesystem images for download. [16:32] Ubuntu does this so you can easily grab the image, make some changes to it, and then upload and register your modified image [16:33] This might be the most involved way of creating an image, but it is also the one that lends it self best to automation [16:34] For a simple example, say I wanted to add a user to an ubuntu image so that I could log in as that user on initial boot. [16:34] What I would do is: [16:35] * launch a utility instance in EC2 [16:35] I'd pick a lucid 64 bit image, possibly even use an EBS root image and a t1.micro size. The size would largely depend on what I wanted to do. [16:35] once that image was up, I'd ssh to it. [16:36] then, download a image tarball that I found a link to from https://uec-images.ubuntu.com/releases/lucid/ [16:36] $ wget https://uec-images.ubuntu.com/releases/lucid/release/ubuntu-10.04-server-uec-amd64.tar.gz [16:36] then, extract the image, mount it loopback and make my modifications [16:37] $ tar -Sxvzf ubuntu-10.04-server-uec-amd64.tar.gz [16:37] $ sudo mkdir /target [16:37] $ sudo mount -o loop *.img /target [16:37] $ sudo chroot /target adduser foobar [16:37] ... follow some prompts ... [16:37] maybe make some other changes here [16:38] $ sudo umount /target [16:38] assuming you've also set up your credentials so that you can use euca-* or ec2-* tools, then you can do: [16:39] uec-publish-image x86_64 *.img my-s3-bucket [16:39] and out will pop a AMI-XXXXX id that you can then launch. === niemeyer_lunch is now known as niemeyer [16:40] This process lends itself *very* well to scripting. You can launch the instance, connect to it, and do all the modifications via a program and revision control them. [16:40] so you'll know exactly what you have [16:40] Also, we make machine consumable information about how to download the images available at https://uec-images.ubuntu.com/query/ [16:41] For some things I was working on, I put together a script that does much of the above [16:41] It assumes the instance is launched, and you're on it with credentials, but then does the rest [16:41] http://bazaar.launchpad.net/~smoser/+junk/kexec-loader/view/head:/misc/publish-uec-image [16:41] so... [16:42] sorry for pushign through all that without taking interupts, but I wanted to get through it. [16:42] obino asked: is there a "preferred" file system for instances? [16:42] 10.04 images I believe use ext3 filesystem. [16:43] that should have been ext4, as the images are really intended to be "stock ubuntu installs", and ext4 was the default filesystem in 10.04 [16:43] the 10.10 images use ext4, and so does natty. [16:44] it is possible that Ubuntu will move to BRTFS as the default in 11.10. if thats the case, I'd like to follow that in the images (brtfs is has some *really* nice features). [16:44] so as to "preferred".... [16:44] I know people use xfs, as it offers snapshotting functionality not available in ext4 [16:45] and Eric Hammond's "create-consistent-snapshot" is a popular tool that sits atop using xfs for your data partitions. [16:45] I wrote a blog entry on how you can rebundle the Ubuntu images into an xfs based image at [16:45] http://ubuntu-smoser.blogspot.com/2010/11/create-image-with-xfs-root-filesystem.html [16:47] navanjr, regarding "are you suggesting we should use the CreateImage method on EC2 to create an image for my UEC Private Cloud?" [16:47] i might have somehwat covered that. === sre-su_ is now known as sre-su [16:47] but you really cannot use CreateImage on EC2 to create an image for UEC [16:47] one general approach that would work would be to get your instance into a state that you're happy with it [16:47] then stop the instance [16:47] snapshot its root volume [16:48] start the instance [16:48] attach that snapshot as another disk [16:48] then copy the filesystem contents of the second disk to a disk image. [16:48] that disk image then could be brought to UEC. [16:49] i'd have to think through that a bit more, but i believe the general path is correct. [16:49] semiosis pointed out: CreateImage will also snapshot any other (non-root) EBS volumes attached to the instance, and those snapshots are automatically restored & attached to new instances made from the AMI. [16:49] CreateImage is *really* a handy wrapper. [16:49] obino asked: thanks for cloud-init! Is there any plan to make cloud-init available for other distro? [16:50] well, amazon has taken cloud-init to their CentOS derived "Amazon Linux". [16:50] and I believe that they intend to continue doing so. [16:50] i'm definitely interested in helping them, and have worked with some of their engineers. [16:51] I'd also like to get cloud-init into debian. I know of one person who was trying to do that, and one person who was interested in getting it into fedora. [16:51] There are 10 minutes remaining in the current session. [16:52] So, yes, I would like to see that. I think consistent bootstrapping code accross linuxes would be a general win. [16:53] navanjr, asked 'so there is no "createImage" similar for use on a running UEC instance?' [16:54] there is no createImage like functionality in UEC. It relies upon EBS and block level snapshotting. Eucalyptus does not have EBS root functionality in any released version that i'm aware of [16:55] however, navanjr you might be able to get some more information out of obino [16:55] (sorry, obino) [16:55] I suspect htat question might have been planted. [16:55] it leads into the next session very well [16:56] TeTeT will talk about "UEC Persistency", which offers a way to get EBS root-like function on UEC, and even LTS 10.04. [16:56] i wont take his thunder, though. [16:56] There are 5 minutes remaining in the current session. === ChanServ changed the topic of #ubuntu-classroom to: Welcome to the Ubuntu Classroom - https://wiki.ubuntu.com/Classroom || Support in #ubuntu || Upcoming Schedule: http://is.gd/8rtIi || Questions in #ubuntu-classroom-chat || Event: Ubuntu Cloud Days - Current Session: UEC persistency - Instructors: tetet [17:01] Logs for this session will be available at http://irclogs.ubuntu.com/2011/03/24/%23ubuntu-classroom.html following the conclusion of the session. [17:01] Hello! Nice to have you in class today. [17:02] It's a bit weird, but I'm as nervous as before giving a live class :) [17:02] If you have any questions, ask them in #ubuntu-classroom-chat with the prefix [17:02] QUESTION [17:02] I am not very familiar with the classbot, but I'll do my best to check if there are any questions in the queue. [17:02] A brief introduction: I'm Torsten Spindler, been working for Canonical since December 2006 and I am part of the corporate services team, so unlike most other presenters here, I'm on the commercial side. [17:03] I bring this up as one of my responsibilities is to maintain and deliver the Ubuntu Enterprise Cloud classes. [17:03] This is directly related to this session, as I present one of the case study exercises found in the UEC class, it's the latest addition to the course material and brand new [17:03] So what do I want to present here? [17:04] During the UEC class I often got the question: 'How can I have an instance on UEC that stores all of its information on an EBS volume, so I can simply use it like a regular virtualized server?' [17:04] Why is that of interest? The data on an instance is volatile, e.g. if the instance dies, all data of it is gone. [17:04] Unless you store the data on a persistent cloud datastore; in UEC we have two of them: Walrus S3 and EBS volumes. [17:05] An EBS volume is a device in an instance that serves as a disk, very much like a USB stick you insert in your system. [17:05] you can see a graphic for this at http://people.canonical.com/~tspindler/UEC/attach-volume.png [17:05] keep in mind that the disk actually attaches to an instance running on the node controller, not on the node controller itself [17:06] the technology used in UEC is 'ATA over Ethernet - AoE', which means that the EBS volume is exported via network from a server, the EBS storage controller. [17:06] With Amazon Web Services (AWS) it is possible to boot from an EBS volume in the public cloud, with a technology named 'EBS root'. For more info on this, please see http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/index.html?Concepts_BootFromEBS.html [17:06] With UEC for the private cloud we use Eucalyptus as upstream technology. With Eucalyptus the concept of booting an instance straight from an EBS volume is not given. [17:07] So the situation looks a bit like this: http://people.canonical.com/~tspindler/UEC/01-instance-volume.png [17:07] forgive my lack of design skills ;) [17:07] in UEC people usually have this: http://people.canonical.com/~tspindler/UEC/02-instance-volume-standard.png [17:08] an instance holds the kernel and /, while the data is saved on a persistent EBS volume [17:08] UEC persistency is about moving the kernel and / to the EBS volume [17:08] depicted in http://people.canonical.com/~tspindler/UEC/03-instance-volume-ebs-based-instance.png [17:08] that is, the kernel of the instance launches a kernel stored on an EBS volume [17:09] the kernel on the EBS volume will use / from the ebs volume and run completely from there [17:09] I asked the Ubuntu server team on advice for realizing such a service back in January 2011. It was motivated by the questions I received during the UEC class. [17:10] My initial thought was to have a specialized initrd that calls pivot_root on a root filesystem stored in an EBS volume. [17:10] But Scott Moser (smoser) had a much better idea: Why not use kexec to load the kernel and root from the EBS volume. [17:10] More information on kexec-loader can be found at http://www.solemnwarning.net/kexec-loader/ [17:10] Scott went ahead and implemented this and my colleague Tom Ellis tested it. [17:10] Then I used the docs from Scott, tested them and created an exercise for the UEC class out of it. [17:11] We decided to publish this work in Launchpad and you can find the result in https://launchpad.net/uec-persistency. [17:11] The branch under lp:uec-persistency contains the needed code and the exercise in the docs directory. [17:12] smoser just told me that we use something like kexec-loader, not exactly the same tech [17:12] see the chat channel for more background info ;) [17:12] If you don't have bazaar installed right now, you can also take a look at the exercise PDF, found at http://people.canonical.com/~tspindler/UEC/ebs-based-instance.pdf [17:12] the odt file is also freely available [17:12] I will now cover this exercise step by step. [17:12] No questions so far on what the aim is? [17:13] to repeat, we want to use the kernel and root filesystem found on an EBS volume, not that in the instance itself [17:13] The steps during the exercise have to be conducted from two different system, one I named the [cloud control host], the other the [utility instance]. [17:13] it would be useful to have the PDF or ODT open for the exercise now [17:14] With cloud control host I mean any system that holds your UEC credentials, so you can issue commands to the front end. [17:14] The utility instance is created during the exercise and is used to populate an EBS volume with an Ubuntu image. [17:15] The first two steps are preparing your cloud with a loader emi. This loader emi will later be used to kexec the kernel on your EBS volume. [17:15] Step three and four setup the utility instance. This is a regular UEC instance that is large enough to hold an Ubuntu image and store it on the attached EBS volume. [17:15] The EBS volume created and attached in step three will be the base for your EBS based instances later on. [17:16] All the steps five to nine are needed to copy an Ubuntu image to the EBS volume and ensure it boots fine later on. [17:16] In step 11 a snapshot of the pristine Ubuntu EBS volume is made. While one could use the EBS volume right away, it's much nicer to clone it via the snapshot mechanism of EBS. [17:17] Just in case you later want another server based on that image. [17:17] Steps 12 and 13 are there to launch an instance based on an EBS volume. [17:17] The final step 14 describes how to ssh to the instance and check if it is really based on the EBS volume, e.g. /dev/sdb1. [17:18] That's really all there is to it, thanks to smosers work. Perfectly doable by yourself within 2 - 4 hours, starting with two bare metal servers [17:18] Once you've been through all the steps and you want more EBS based instances of the same image, simply repeat from step 11 boot_vol. [17:19] With this you should have virtualized servers in your UEC within a few minutes, quite a nice time for provisioning. [17:19] Especially useful might be to assign permanent public addresses to those instances. [17:19] This can be done with help of euca-allocate-address and euca-associate-address. [17:20] any questions so far? Everything crystal clear? [17:21] Well, it was a very short session then I fear... [17:22] Closing words, we're looking into automating the storage of the Ubuntu image on the EBS volume to make this step less work intense. So keep an eye on the Launchpad project. [17:22] In the end you should be able to use any Ubuntu image within UEC on Ubuntu 10.04 LTS on an EBS volume in a few minutes, rather than hours [17:23] if there's anyone interested in contributing to the project or UEC in general, please get in touch with kim0 [17:23] we're looking for people with any skills, from coding to writing documentation [17:24] we'd also love to hear from you if you try UEC persistency and it works or doesn't work for you [17:24] I tested the exercise a few times, but you'll never know [17:25] you can touch base with us in #ubuntu-cloud and myself also in #ubuntu-training [17:25] obino asked: have you even RAIDed the EBS volumes? Is there any advantage in doing so? [17:26] nope, I've never RAIDed the EBS volumes, but would think there might be a bit of a performance hit due to the ATA over Ethernet protocol [17:26] also keep in mind that while served via network, the EBS volumes are likely to come from the same Storage Controller (SC) [17:27] so not sure if this is a good approach [17:28] might be interesting to use drbd though, and maybe use the EBS volume as well as the ephemeral storage and see how that goes [17:28] I guess there's quite a bit of room for experimentation for EBS based instances [17:30] guess you have no 30 minutes left in the session, so enough time to actually do the exercise if you have a UEC ready :) [17:30] no=now [17:32] thanks for attending, catch me in #ubuntu-training if you run into problems with the exercise, bye now [17:33] So we finished a bit early on this session [17:33] Daviey starts in less than 30 mins with a puppet session [17:34] Time for a coffee break :) [17:51] There are 10 minutes remaining in the current session. [17:56] There are 5 minutes remaining in the current session. [18:01] kim0, Are you managing the session? [18:01] It's pretty much self managing .. topic will be changed in a few seconds === ChanServ changed the topic of #ubuntu-classroom to: Welcome to the Ubuntu Classroom - https://wiki.ubuntu.com/Classroom || Support in #ubuntu || Upcoming Schedule: http://is.gd/8rtIi || Questions in #ubuntu-classroom-chat || Event: Ubuntu Cloud Days - Current Session: Puppet Configuration Management - Instructors: Daviey [18:01] Logs for this session will be available at http://irclogs.ubuntu.com/2011/03/24/%23ubuntu-classroom.html following the conclusion of the session. [18:02] My name is Dave Walker (Daviey), and I am a member of the Ubuntu Server Team. [18:02] Welcome to the puppet classroom session. This session is mainly targeted at those that have had minimal or no exposure to the puppet. [18:03] It allows reproducible, consistent deployments, which is good for horizontal scaling and replacing machines which have malfunctioned. [18:03] A good reference for more details about the project is at: [18:03] http://projects.puppetlabs.com/projects/puppet/wiki/About_Puppet [18:03] Please take a few moment to grok the content of that page, there is little point in my reproducing the content here. [18:03] * Daviey waits a few minutes. [18:04] Now, some of that might sound little complicated but it really is simple when you get started. [18:05] * Daviey continues. [18:05] Puppet focuses on the 'configuration' management. The initial operating system deployment is usually done with either, preseeding the installer, cobbler, FAI or simply spawning a cloud machine, such as EC2. [18:05] In regards to EC2.. people tend to use user-scripts or increasingly cloud-init. [18:06] Once the base operating is installed, there is always some changes that need to be made to make the server usable for production. This varies from performance tweaks, application configuration and even custom versions of packages. This could all be handled with scripts and such, but this is less than clean and near impossible to maintain. This is where puppet provides a clean solution. [18:06] Once the base operating is installed, there is always some changes that need to be made to make the server usable for production. This varies from performance tweaks, application configuration and even custom versions of packages. This could all be handled with scripts and such, but this is less than clean and near impossible to maintain. This is where puppet provides a clean solution. [18:06] Puppet generally acts on a client/server method, to manage multiple nodes. However, it is also possible to use puppet on a single host. For simplicity, this session will demonstrate a single host deployment example and some of the features of puppet via their configuration format - called a manifest. [18:07] In this session, we will do the following: [18:07] • Connect to an instance in the cloud [18:07] • Install puppet [18:07] • Initial configuration [18:07] • Configure the same node to install and create a basic apache virtual host [18:07] Firstly, i hope everyone will be able to look at a console window, and this IRC session concurrently. [18:08] I'm going to invite everyone to connect via ssh to a cloud instance: [18:08] $ ssh demo@demo.daviey.com [18:08] You'll need to accept the host key [18:08] I don't think it really requires verification in this instance. [18:08] (Although, it's generally good pratice to compare the fingerprint) [18:09] The password is 'demo' [18:09] (Secure huh?) [18:10] * Daviey waits for a confirmation. === pvo_ is now known as pvo [18:10] I will type in the IRC channel comments, so please multi-task by looking at both.. Thanks :) [18:11] So, i just checked to see if we have apache2 installed... we do not! [18:11] (You can check there is nothing running as a httpd on port 80, by visiting http://demo.daviey.com [18:12] (You should get a failure) [18:12] I'm running sudo apt-get update, to make sure our indexes are updated [18:12] The observant amongst you, will notice i'm running Natty [18:12] The current development version [18:12] (I must be crazy doing a demo on this! :) [18:13] So, i just, sudo apt-get install puppet [18:13] This installs the puppet application and it's dependencies. [18:14] This stage, would normally be done automatically during installation [18:14] (if you preseed it such) [18:14] You'll notice the output here: [18:14] puppet not configured to start, please edit /etc/default/puppet to enable [18:14] Did you all see the START=no, option [18:15] This means that the puppet client agent will not run automatically [18:15] My intention is to invoke puppet manually.. so i do not need the client to be running [18:16] (one moment please) [18:17] (slight technical issue, please hold) [18:21] Okay! [18:21] we are back [18:21] okay, this is the directory structure we should see [18:21] on a fresh installation [18:22] Okay, i have just copied a manifest to /etc/puppet/manifest [18:22] I hope everyone can see this [18:22] It's quite a quick one i have thrown together [18:22] It should: [18:22] Install apache2 [18:22] add a virtual host, called demo.daviey.com [18:23] and enable it [18:23] (I'll make it avaliable afterwards via a pastebin) [18:23] The stanza towards the bottom mentions, ip-10-117-82-138 [18:23] (for the observant, you'll notice this is the hostname of the machine) [18:24] I could equally, have put 'default' here... which would mean that it would do it for every machine connected [18:24] (in this instance, i am only using one machine) [18:24] Now, the actual virtual host needs a template... [18:24] lets create it. === daker is now known as daker_ [18:25] puppet uses Ruby's ERB template system: [18:25] You'll notice that there are parts which can be expanded. [18:25] So, this is a generic apache virtual host template, that could be used for other virtualhosts [18:25] other than demo.daviey.com [18:26] Now... lets make puppet do it's thing! [18:27] I love it when a plan comes together. [18:28] Essentially, i did a dry run with this configs before the session.. and didn't clean up properly! [18:28] This is why i should have used puppet to clean up, as it would have done it better than me! [18:29] So, puppet install apache2 and enabled the virtual host [18:29] puppet knows which package hander to use [18:29] ie, apt, yum etc [18:30] Now... if we check to see if apache started.. we'll see it failed... one moment [18:31] So... [18:31] (2)No such file or directory: apache2: could not open error log file /var/log/apache/demo.daviey.com-error.log. [18:31] Unable to open logs [18:31] This means i made a typo in my template... suggestions on how i should fix this? [18:31] kim0, Is quite correct with: [18:31] Daviey: should be "apache2" there [18:31] But... How should i *fix* it? [18:32] We edit the template of course! [18:33] Now, we should be able to go to http://demo.daviey.com/ [18:33] (My simple Task didn't try to start apache if it wasn't already running!) [18:33] notice: /Stage[main]//Node[ip-10-117-82-138]/Apache2::Simple-vhost[demo.daviey.com]/File[/etc/apache2/sites-available/demo.daviey.com]/content: content changed '{md5}5047b9f9a688c04e2697d9fd961960ed' to '{md5}2c32102fd06543c85967276eeee797e2' [18:34] ^^ Puppet knew it should create a new virtual host, based on the template changing! [18:34] How neat is that?! [18:35] Now, in a real life example - puppet would also manage pulling in the website.. [18:35] puppet provides a FileBucket interface.. [18:35] This is similar to rsync, and allows files to be retrieved from there. [18:35] However, for large files - people often use an external application which is configured via puppet. [18:36] This could be anything from rsyncd, nfs or event torrent! [18:36] facter is a really useful tool. This is where the variables used in the templates are expanded from... I think of it as lsb_release supercharged. [18:37] Here is an example of the output, just generated: [18:37] http://paste.ubuntu.com/584952/ [18:37] This is a list of 'facts' about the system [18:38] One of the really nice things about the manifests... is that they can be conditional === Mike is now known as Guest49828 [18:38] So, i could do a different task based on they virtual type (or lack of) for example. [18:39] There is no point trying to use this machine as a virtual machine server, if it doesn't fit the requirements [18:39] Usually bare metal - and amount of memory free [18:39] The configuration files are largely inheritance based, which fully supports overriding of configurations from the base class. [18:40] When puppet is installed on a client / server basis... it uses SSL for secure communciation between the elements [18:41] The server runs on port 8140. so make sure firewall is opened (or ec2 security group allows communication!) [18:41] Client (Agent) - puppetd [18:41] Server - puppetmasterd [18:41] ^^ This is the name of the applications [18:41] The puppetd runs on all the clients, and polls the Server with the default value of every 30 minutes looking for changes [18:42] It defaults to looking for the dns hostname of 'puppet' [18:42] So, it's a good idea for the puppet master to have that dns entry set for a local network [18:42] Equally, i could have set puppet.mydomain.com [18:43] This is probably a good place to stop the demo. I will make my puppet configuration avaiable for others to experiment with. [18:43] It really is not as complicated as it seems to get started. [18:43] When i first tried puppet, i found the 'getting started' docs to be somewhat complicated. [18:44] I would recommend people start with a minimal example like this.. and then build from there. [18:44] The puppet website has some excellent recipies to use as an example... but probably a good idea to start simple. [18:44] I will now take questions, and answer them as best as i can === drc is now known as Guest51125 [18:45] Annnnd. classbot, i hate you [18:45] classbot isn't +v [18:45] sveiss asked: how large is 'large'? is there a rule of thumb as to how much data a FileBucket can cope with? -- There is 1 additional question in the queue. [18:46] sveiss, that is a good question.. I seem to remember reading that since 2.6... massive improvements have gone into increasing it's efficiency [18:46] However, it is still believed to be the likely bottlekneck [18:47] I haven't found the later versions to suffer to badly from this bottlekneck [18:47] but others have commented. [18:47] I think it depends on load.. [18:48] I would ask that if you do try the filebucket that you report back to the ubuntu server team with your success. [18:48] (We often don't get enough feedback) [18:48] sveiss asked: how large is 'large'? is there a rule of thumb as to how much data a FileBucket can cope with? [18:49] kim0 asked: Wouldn't clients looking for dns name "puppet" and blindly following it .. be a secruity risk [18:49] Well yes.. this is true.. This is one of the reasons SSL is used. [18:49] Essentially, the pupper master (usually has a self signed key) [18:49] but the client needs to accept it. [18:49] This would normally happen as part of the installation, or bootstrapping [18:50] Which is an area before puppet works. [18:50] kim0 asked: Wouldn't clients looking for dns name "puppet" and blindly following it .. be a secruity risk [18:51] Wow.. i now understand ClassBot [18:51] There are 10 minutes remaining in the current session. [18:51] kim0 asked: Do you reuse ready made recipies [18:52] You would be foolish not to! [18:52] There is a true gem of samples on the puppet wiki, and other locations. [18:52] Additionally, there are additional modules [18:52] Which allow you to reduce the burden of what you need to do [18:53] shttp://forge.puppetlabs.com/ [18:53] http://forge.puppetlabs.com/ , rather [18:53] If there are no more questions, i will end my session. [18:54] I would like to thank everyone for coming [18:54] Please do experiment with puppet, and report back to us. [18:54]