| === niemeyer__ is now known as niemeyer | ||
| erichammond | http://cloud.ubuntu.com/ami/ is broken on my browser (chromium): "DataTables warning: JSON data from server failed to load or be parsed. This is most likely to be caused by a JSON formatting error." | 09:42 |
|---|---|---|
| kim0 | erichammond: It's not browser specific .. something seems to have went wrong .. thanks for the heads up | 09:56 |
| kim0 | Can anyone help me answer that: Will Eucalyptus be 2.0.1 (upstream released Oct 20) be in Natty ? | 15:26 |
| smoser | Daviey, ? | 15:34 |
| Daviey | kim0: can't say for certain, but 'probably'... depending if there are any difficult regressions when one of us has a smoke test. | 15:38 |
| Daviey | kim0: certainly will not have 2.1 | 15:38 |
| kim0 | Daviey: thanks :) | 15:48 |
| kim0 | Daviey: Any luck thinking of some community contribution venues | 15:54 |
| kim0 | smoser: your input is much appreciated as well :) | 15:55 |
| Daviey | kim0: smoser has an AMAZING idea. | 16:06 |
| smoser | https://blueprints.launchpad.net/ubuntu/+spec/cloud-server-n-image-rebundle | 16:07 |
| smoser | :) | 16:07 |
| smoser | mostly seriously on that. rebundling is a very important area of our images that isn't addressed in documentation or tools terribly well. | 16:07 |
| TeTeT | smoser: we have an exercise on it in the UEC training | 16:09 |
| * kim0 in a call | 16:10 | |
| kim0 | but I would like to understand exactly what would contributors do ? | 16:10 |
| kim0 | is it documentation | 16:10 |
| smoser | some is documentation, yes. but some is development of utilities for "cleaning" or making that process easier. | 16:11 |
| kazade | hi smoser, I've just been asking some questions in #ubuntu-uk about Amazon EC2 and Daviey mentioned you might be able to help me (they aren't Ubuntu related, just general EC2 questions). you free? | 16:19 |
| smoser | sure. | 16:20 |
| kazade | basically, I have this running EC2 instance and I'd like to create a duplicate | 16:20 |
| kazade | the EC2 instance has a single EBS root device.. | 16:20 |
| smoser | kim0, this is the type of thing that i'd like better covered | 16:21 |
| smoser | (it almost smells like a setup) | 16:21 |
| smoser | kazade, what release ? 10.10 ? 10.04 ? | 16:21 |
| kazade | 10.04 | 16:21 |
| kazade | basically, if I click "Create Image" will it freeze/pause the running instance? and will that create me an image I can create a new EC2 vm with? | 16:21 |
| * kazade is confused generally | 16:22 | |
| kim0 | smoser: I'll ping ya in a 20 mins or so .. thanks a lot though! :) | 16:22 |
| kazade | also, if I create a snapshot of the EBS device... would restoring that snapshot essentiall "rollback" the EC2 vm? | 16:23 |
| smoser | if you hit 'Create Image', that makes the 'CreateImage' (i think thats the api name) call , which can also be done via 'ec2-create-image'. | 16:24 |
| smoser | it will stop the instance (by issuing 'shutdown'), and then snapshot the volume and create a new ami from that snapshot, then start the instance back up. | 16:24 |
| smoser | this is probably what i would recommend you do. it is very easy and very effective. | 16:25 |
| kazade | ok cool that sounds like exactly what I needed to know. How long would the image creation take normally? | 16:25 |
| smoser | couple minutes | 16:29 |
| smoser | i might advise that you shut down your instance (/sbin/poweroff) yourself | 16:29 |
| smoser | it just seems cleaner to me, but i believe it does work the other way. | 16:29 |
| kazade | smoser, thanks just one final question: is taking a snapshot of the EBS device exactly equivalent of taking a snapshot in virtualbox (for example) | 16:30 |
| smoser | i dont know what it means in virtualbox, but its pretty much a volume snapshot | 16:30 |
| smoser | ie, it saves all data at the block device level | 16:30 |
| kazade | what I mean is, if I reverted the snapshot and booted the EC2 vm, would it essentially rollback? (I mean, the EC2 wouldn't break or anything) | 16:31 |
| smoser | did i answer your question ? | 16:31 |
| smoser | the running instance is not affected by any snapshots of its root device | 16:31 |
| kazade | right.. | 16:32 |
| smoser | you can even do them to a mounted filesystem | 16:32 |
| kazade | if I restored a snapshot of the device though, what would that do to the running machine? (or is that not possible?) | 16:32 |
| smoser | you cannot reset an existing volume's content to a snapshot you can only create new volumes from the snapshot. | 16:32 |
| kazade | smoser, right ok, that makes sense | 16:32 |
| kazade | Can an EC2 machine change it's root device? | 16:33 |
| smoser | kazade, sort of. | 17:00 |
| smoser | what do you want to do ? | 17:00 |
| kazade | I just wondered if I could snapshot the EBS device, and then if I screwed up the VM, restore the EBS device | 17:01 |
| kazade | (e.g. by pointing the EC2 machine at a newly restored snapshot) | 17:01 |
| smoser | Daviey, if you see kazade, you can tell him that you can easily attach a volume of a snapshot to runnign instance, mount that volume, diff it against / anda sync contents that way. | 17:37 |
| kim0 | Hi everyone, anyone new around here ? say Hi please | 18:02 |
| daker | poor kim0 | 18:07 |
| kim0 | daker: It takes some effort man :) | 18:08 |
| kim0 | However I am presistent hehe | 18:09 |
| daker | yes | 18:09 |
| kim0 | Daviey: must the CLC be configured to NAT packets manually ? i.e. is that not part of setup ? | 18:17 |
| kim0 | Daviey: I'm trying to answer http://ubuntuforums.org/showthread.php?t=1646048 .. where basically NC doesn't have internet connectivity | 18:17 |
| kim0 | kirkland: ^ | 18:28 |
| kim0 | smoser: Hi! so you're basically proposing people attack https://blueprints.launchpad.net/ubuntu/+spec/cloud-server-n-image-rebundle | 18:34 |
| smoser | sure. | 18:34 |
| kim0 | smoser: did you ever blog or write something explaining what needs to be done | 18:34 |
| kim0 | manually | 18:34 |
| smoser | well if you have a way they can do it automatically (as opposed to manually) i'm happy to hear it! | 18:35 |
| kim0 | what I'm saying this .. did you ever explain the manual steps, so that I have a better understanding of what needs to be done | 18:36 |
| kim0 | s/this/is/ | 18:36 |
| * kim0 reading https://wiki.ubuntu.com/CloudServerNattyCloudUtils | 18:42 | |
| DigitalFlux | kim0: Interesting | 18:44 |
| DigitalFlux | smoser was talking about it i think today at #openstack | 18:44 |
| kim0 | yeah interesting indeed :) | 18:45 |
| smoser | kim0, no. not really. theres a bunch of things covered there. | 18:45 |
| kim0 | smoser: by first impression .. that's a whole lot of things :) can u pick an important one or two scenarioes to tackle | 18:45 |
| kim0 | I guess downloading ec2 images .. boot in kvm .. modify .. republish | 18:46 |
| smoser | i've probably covered many of them before on posts to ec2ubuntu and such | 18:46 |
| kim0 | smoser: what about if this "cleaning" utility records files at ~ while booting .. and at cleaning stage presents the user with all newly created files to delete or keep | 18:47 |
| kim0 | we could have defaults based on regex matches .. like for /var/log/* -> delete | 18:48 |
| kim0 | for /etc/ -> keep | 18:48 |
| kim0 | it may monitor the whole system .. not just ~ | 18:48 |
| kim0 | don't know .. comments | 18:48 |
| smoser | yeah, its really crappy and heuristic | 18:48 |
| smoser | so it needs config options | 18:48 |
| smoser | and its going to be wrong in many cases in both ways | 18:49 |
| kim0 | smoser: what's the main purpose of uncloud-init | 18:51 |
| kim0 | can't find any proper page talking about that | 18:51 |
| smoser | to hackily boot one of our images without a ec2 metadata service | 18:51 |
| kim0 | without cloud-init blocking for ages | 18:52 |
| kim0 | that once bit me :) | 18:52 |
| smoser | that problem sucks in general | 18:53 |
| smoser | there is no way to tell if i'm on EC2, and thus should wait for the metadata service | 18:53 |
| smoser | but on EC2 (or on UEC), i *have* to wait for the metadata service, indefinitely. | 18:53 |
| kim0 | yeah! | 18:54 |
| raymdt | hey | 19:17 |
| kim0 | raymdt: hey | 19:32 |
| === dendro-afk is now known as dendrobates | ||
| soren | smoser: This is my last working day of the year. If you want to chat about stuff, this is a good time :) | 19:43 |
| smoser | hm.. | 19:45 |
| smoser | give me a couple minutes ? | 19:45 |
| soren | smoser: Sure, I'm not going anywhere. | 19:49 |
| soren | yet :) | 19:49 |
| smoser | soren, ok | 19:59 |
| smoser | so | 19:59 |
| smoser | our cloud images, i *want* them to be bit for bit identical on different clouds. | 20:01 |
| smoser | or as close to it as possible | 20:01 |
| smoser | i want the download on uec-images.ubuntu.com to not need a bunch of "fixing" for before its useable on EC2, UEC, openstack, and ideally "raw kvm/libvirt" | 20:01 |
| smoser | but we have this metadata service thing, that i want/have-to wait around for in some cases (uec / ec2) and not in others ('raw kvm/libvirt') | 20:04 |
| smoser | right now in the .tar.gz files, there is a -floppy file, which is a bootable floppy, and documentation, then that says how to boot it (boot with floppy bootable). | 20:05 |
| soren | smoser: Right. | 20:05 |
| smoser | the floppy then boots the kernel in the instance with special args telling it "hey, theres no cloud here" | 20:05 |
| soren | smoser: What is the canonical use case that makes us have to wait for the meta-data service (rather than just handling it in the background once it turns up). | 20:06 |
| soren | ? | 20:06 |
| smoser | but even that is yucky. ideally, it would boot up, find that there was no metadata service or any other method of letting a user log in, and "uncloud" itself. | 20:06 |
| smoser | well, righ tnow it actually blocks boot. | 20:07 |
| smoser | which allows the user to do things (via user data) when it runs | 20:07 |
| smoser | some of these things might include modifying the system in ways that are difficult to do once it is further up | 20:08 |
| smoser | ie, at that point you can still mount another volume over the top of /home or modify config of a server before it has started. | 20:08 |
| smoser | so those things are valuable, and backgrounding would make them impossible (or require restart/reboot) | 20:09 |
| soren | right, ok. | 20:09 |
| soren | Do we have any data on how long it takes before the meta-data service turns up? | 20:10 |
| smoser | we have annoyances | 20:11 |
| soren | min/max/mean time sort of stuff? | 20:11 |
| smoser | i would have marked this as not an issue sometime about 6 months ago on EC2. | 20:11 |
| smoser | i have never really seen a "waiting for metadata service" message there. | 20:11 |
| smoser | and i had even turned it down to something small, 20 seconds or something, maybe less. | 20:11 |
| smoser | but UEC started failing, and we were told "you have to wait longer" | 20:12 |
| smoser | so, wait longer we did. | 20:12 |
| soren | :( | 20:12 |
| smoser | in the vast majority of the cases, waiting more than 30 seconds was not that useful, i thikn, but dont have a lot of data on it. although we did at the time. | 20:12 |
| soren | smoser: So the difficult case is UEC. | 20:15 |
| soren | smoser: Luckily, that's the one we control. | 20:15 |
| smoser | mostly, yes, but then it came about because of EC2 | 20:15 |
| smoser | so UEC is feature compatible :) | 20:15 |
| soren | Sure, sure. | 20:16 |
| soren | How about we change UEC to pass a "Hi, I'm UEC. Please bear with me while I get off my a*** and get the meta-data service up and running for you" sort of kernel parameter? | 20:16 |
| smoser | soren, well, i can somewhat control the kernel parameters that are passed | 20:17 |
| smoser | however, the control i have over that is via /boot/grub/grub.cfg | 20:18 |
| soren | Eh? | 20:18 |
| smoser | which would ideally be read by kvm or openstack "full disk" or ... | 20:18 |
| soren | UEC never boots an out-of-image kernel anymore? | 20:18 |
| smoser | remember my 'loader' hack/feature in eucalyptus | 20:18 |
| === dendrobates is now known as dendro-afk | ||
| soren | I didn't think that was used all the time. | 20:19 |
| smoser | it does, and in that case, sure, you could hard code a "you're running on UEC" parameter. | 20:19 |
| soren | That's a start, at least. | 20:19 |
| smoser | its not used all the time, but that is the "right way" going forward (or some other mechanism that is more 'normal' to allow guest to control its own kernel loading) | 20:19 |
| smoser | so that solution is really only for "legacy" | 20:19 |
| === dendro-afk is now known as dendrobates | ||
| soren | Can't we ask the BIOS how it booted? | 20:20 |
| soren | If it's from floppy, there's a 99.999% chance it's UEC. | 20:20 |
| smoser | well, the way i have to run kvm without cloud for those right now is via a floppy also | 20:21 |
| soren | Doh. | 20:21 |
| smoser | that was the easiest way to inject kernel parameters | 20:21 |
| soren | Wait, what? | 20:21 |
| smoser | (the -floppy disk distributed is hacky | 20:21 |
| smoser | but it basically does: boot (hd0,1)/vmlinuz some-kernel-args NOCLOUD | 20:21 |
| smoser | so its quite less than ideal. | 20:22 |
| smoser | in natty cycle, i hope to have full disk images, and just say "kvm -hda disk.img" | 20:22 |
| smoser | btw, i'm thankful for your input, please dont' think i'm just shooting down ideas | 20:23 |
| smoser | one thing i've considered is putting something into the image, which could be written to without understanding the filesystem | 20:23 |
| smoser | and the providing a tool that could muck that bit | 20:23 |
| kim0 | smoser: what about providing different kernels, such that the user would choose a different kernel for a direct kvm boot | 20:30 |
| smoser | well that would incur the overhead of maintaining 2 kernels. | 20:31 |
| smoser | we recently got a *major* feature add when we dropped the -ec2 kernel | 20:32 |
| soren | smoser: You've lost me. I thought you said you used grub.cfg for something. | 20:32 |
| kim0 | smoser: not really different kernels .. just 2 kernel entries .. booting same kernel with different params | 20:32 |
| soren | smoser: (/me is catching up on the conversation... Many things going on at the same time) | 20:33 |
| soren | smoser: re: "btw, i'm thankful for your input, please dont' think i'm just shooting down ideas | 20:33 |
| soren | " | 20:33 |
| soren | smoser: Not a problem at all. | 20:33 |
| smoser | kim0, in which case i'd still have to get the user (or cloud) to select the kernel to boot. | 20:34 |
| smoser | soren, currently we jsut ship a partition image | 20:34 |
| kim0 | smoser: defaults to cloud kernel .. and kvm user chooses a different one | 20:34 |
| smoser | that partition image has no bootloader in it | 20:34 |
| smoser | but, it has both /boot/grub/grub.cfg and /boot/grub/menu.lst in it | 20:34 |
| smoser | grub.cfg is used if you boot with 'loader support' on UEC | 20:35 |
| smoser | menu.lst is used if you boot with pv-grub on EC2 | 20:35 |
| smoser | then, there is a floppy disk in the .tar.gz file. | 20:35 |
| smoser | when you boot off of that, it does a dirty hack and says "boot /vmlinuz from inside disk 1 partition 1, and pass 'nocloud' as a kernel parameter" | 20:36 |
| kim0 | what about a count down .. like Press N for no-cloud 5 4 3 2 1 | 20:37 |
| smoser | kim0, i'd like to avoid manual action on every reboot (even the first) | 20:37 |
| kim0 | so cloud kernel works ok for ec2 and uec .. only problem is, direct kvm case | 20:38 |
| kim0 | well u just have to detect run time env I guess then .. somehow | 20:39 |
| kim0 | no detectable info on ec2 xen store ? | 20:39 |
| smoser | there is definitely stuff in the xen store, but not something published that i could rely on being there. | 20:40 |
| smoser | the thing that is documented is the metadata service :) | 20:40 |
| smoser | and even then... ideally.... the image is easily used as xen instance. | 20:41 |
| smoser | i know that i'm asking a lot, but thas what the goal is... this disk image "just works" | 20:41 |
| smoser | so far, your timeout on selection of non-EC2 cloud kernel entry is about the best i have. | 20:42 |
| smoser | the issue with that is that | 20:42 |
| smoser | a.) then you penalize the UEC boot by 5 seconds | 20:43 |
| smoser | b.) that default isn't easily modifyable without mounting the image -- this is what i really want to stay away from. | 20:44 |
| kim0 | what about a no-penalty .. press shift while booting kinda thing ? | 20:45 |
| euca | hi guys! I'm trying to run a private cloud with UEC and I want all my machines to have a static IP on the network. The VMs get an IP, but then they don't boot. There is a script that tries to obtain metadata from a server it can't reach, and so it just keeps looping. | 20:45 |
| euca | DataSourceEc2.py[WARNING]: waiting for metadata service at http://169.254.169.254/2009-04-04/meta-data/instance-id | 20:45 |
| euca | DataSourceEc2.py[WARNING]: 16:06:10 [ 1/100]: url error [timed out] | 20:46 |
| kim0 | lol :) | 20:46 |
| kim0 | haahaahh .. speak of the devil | 20:46 |
| kim0 | euca: how long did you wait for them to boot ? | 20:48 |
| euca | I've read that this happens when the private IP is the same as the public IP.. but I can't seem to make them be different. Has anyone dealt with this? I'm using the cloud controller runs Ubuntu Server 10.10, the node runs CentOS, both with eucalyptus 2.0.0 | 20:48 |
| euca | kim0: I let it stay overnight | 20:48 |
| kim0 | ew! | 20:49 |
| kim0 | why aren't you running one software stack in all ur cloud | 20:49 |
| euca | kim0: because Ubuntu Server failed to install on the node | 20:49 |
| kim0 | :) | 20:49 |
| euca | and CentOS didn't | 20:49 |
| kim0 | euca: how did it fail | 20:49 |
| euca | it didn't detect any storage | 20:50 |
| euca | devices | 20:50 |
| kim0 | can you post a bug report please | 20:50 |
| euca | well, I can, but this is sort of urgent for me | 20:50 |
| kim0 | hmm | 20:50 |
| euca | I'm sure the underlying OS is not the issue | 20:51 |
| kim0 | try asking in #eucalyptus maybe | 20:51 |
| smoser | euca, make sure you're not running a dhcp server on your network | 20:52 |
| euca | smoser: I don't have control over the network | 20:52 |
| euca | but there is a dhcp server | 20:52 |
| euca | on that network | 20:52 |
| smoser | then you really can't run eucalyptus | 20:52 |
| smoser | the guests will dhcp, which gets put onto the network of the host | 20:53 |
| smoser | there are 2 dhcp servers | 20:53 |
| smoser | it will take the lease from whichever runs first | 20:53 |
| smoser | returns first | 20:53 |
| kim0 | wild west competition | 20:53 |
| euca | the guest get the correct IP | 20:53 |
| euca | I configured it to be STATIC | 20:53 |
| euca | I can traceroute and ping the machine. I even checked the MAC and the IP. They are both correct. | 20:54 |
| euca | when I try to ssh, I get a connection was reset error. and the guest is trying to fetch metadata from that URL | 20:55 |
| euca | I tried the URL manually, and it returns 404 Not Found | 20:55 |
| euca | I've read that this used to be a bug in lucid, when both the public and private IPs are the same | 20:55 |
| smoser | euca, that url will only be good from inside the guest | 20:55 |
| euca | smoser: I suspected as much, but I thought I'd try ;) | 20:56 |
| smoser | i wonder how you configured the IP to be static, though, and which IP you chose. | 20:56 |
| euca | smoser: hold on, I'll post the eucalyptus.conf somewhere | 20:56 |
| smoser | because the instance should be configured with the private ip | 20:56 |
| smoser | (ie, ifconfig inside it will show the private IP) | 20:57 |
| smoser | and routing magic on the node lets it through | 20:57 |
| smoser | euca, for debugging, you can try running my ttylinux images, which do not depend on the metadata service | 20:58 |
| smoser | http://smoser.brickies.net/ubuntu/ttylinux-uec/ | 20:58 |
| kim0 | smoser: btw for UEC, does CLC perform NATing as part of default setup, or is the user expected to do that manually | 20:58 |
| smoser | use the latest there, use uec-publish-tarball to publish it, run an instance, and you can ssh in. of course you have to somehow set your IP address (those expect that they get dhcp) | 20:58 |
| smoser | they will also, after a time, start dumping debug info on their view of the world. | 20:59 |
| smoser | kim0, i dont belive it does any natting, i'm not really certain on the config of that stuff. | 20:59 |
| kim0 | so out of the box, NCs doesn't have internet connectivity | 21:00 |
| euca | smoser: http://codepad.org/uwgop8kl (eucalyptus.conf on the CC) http://codepad.org/BoFMV18z (part of eucalyptus.conf from the NC) | 21:01 |
| euca | smoser: euca-describe-instances output http://codepad.org/OUozYObx | 21:03 |
| smoser | so how do you think that you made the instances use a static IP address ? | 21:03 |
| smoser | i could be incorrect, but i believe that everything is based on dhcp | 21:03 |
| euca | I thought so too.. but then I ran nmap and saw the MAC associated to that IP | 21:04 |
| smoser | well, it could very well get that. | 21:05 |
| smoser | but its a race | 21:05 |
| smoser | run one of those ttylinux images | 21:05 |
| euca | I'll try doing that in a sec | 21:05 |
| smoser | and see what happens. | 21:05 |
| smoser | they'll open up a ssh server on 22 with a password | 21:06 |
| euca | what's the username and password? | 21:08 |
| * euca 30k/sec | 21:08 | |
| smoser | really ? | 21:08 |
| euca | yes | 21:09 |
| smoser | i think its ubuntu:passw0rd | 21:09 |
| smoser | hm.. | 21:09 |
| euca | and it's stuck now | 21:09 |
| euca | hm | 21:09 |
| euca | 40k now ;) | 21:09 |
| smoser | hm.. that stinks. i just pulled to here *michigann) at 600k, but my ec2 instance is only getting it at ~ 30 k | 21:10 |
| euca | I'll wait | 21:11 |
| smoser | no idea whats wrong there, maybe i should get a mirror for those | 21:13 |
| euca | smoser: can I run it in a t1.small ? | 21:28 |
| smoser | m1.small you mean ? | 21:29 |
| smoser | yes, it should fit inside your pocket | 21:29 |
| euca | yes, m1.small :) | 21:29 |
| smoser | (it has a 24M root filesystem i think) | 21:29 |
| euca | pending.. | 21:31 |
| euca | interesting | 21:37 |
| euca | I can't run it | 21:37 |
| euca | the machine doesn't show up in 'xen list' | 21:38 |
| euca | and there are no errors in the error log | 21:38 |
| euca | and it's not in /usr/local/eucalyptus/admin/ | 21:38 |
| euca | i.e the folder is there, but there are no files | 21:38 |
| euca | and on the second try, only kernel and kernel-digest were copied to the NC | 21:39 |
| euca | hm, apparently it failed to delete /usr/local/eucalyptus//eucalyptus/cache/eki-457115F8/kernel-staging | 21:42 |
| euca | i'm deleting the cache and retrying | 21:43 |
| euca | nope | 21:47 |
| euca | it's not running it | 21:47 |
| euca | and I can't find an error in nc.log | 21:47 |
| euca | nothing in xend.log | 21:49 |
| euca | :/ | 21:50 |
| === dendrobates is now known as dendro-afk | ||
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!