[00:55] Hey any one can help a cloud noob with some questions? :P [00:57] no [00:57] how long is a piece of string? [01:57] that wasn't nice flaccid [01:57] buuo: ask your question [01:58] :) === Evet_ is now known as Evet [15:29] smoser: hi! [15:29] smoser: where are the new releases of Ubuntu EC2 images usually announced? [15:29] smoser: ie which blog/mailing list should I follow? [15:29] ubuntu-cloud and ubunt-server i think [15:30] smoser: great - thanks! [15:30] but programattically, you should just check at http://uec-images.ubuntu.com/query [15:31] smoser: is ubuntu-ec2@lists.ubuntu.com still active? [15:32] no. folded into -cloud [15:32] smoser: hm - nope [15:36] smoser: can the lucid images from http://uec-images.ubuntu.com/releases/10.04/release/ be used with kvm/libvirt? [15:36] not terribly easily. [15:37] https://code.launchpad.net/~smoser/+junk/boothooks is what i used in that timeframe to do local testing. [15:39] smoser: so a script in user-data ... where does that log to? [15:40] asac, console [15:40] ie /dev/console, in ec2/uec that should be seeable in euca-get-console-output [15:41] smoser: nice... thx. will try!! [15:41] asac, you can also very easily write it wherever you want [15:41] if its shell, just do [15:41] { [15:41] my script here [15:42] } 2>&1 | tee some-other-file.txt [15:42] good idea ... now i would have to figure how to make wiki show my #!/bin/sh if i place it at beginning of line ;) [15:50] smoser: https://wiki.linaro.org/Platform/Android/Specs/LinaroAndroidBuildInfrastructure#Experimental%20Node%20Scripts ... is that about the right approach to do something like this? [15:52] asac, yeah, that should work. [15:52] 2 things, though [15:52] a.) "cloud-config" can do similar things (the update an package install) [15:52] b.) see https://help.ubuntu.com/community/CloudInit for multipart information on user data. [15:53] multipart becomes useful if you've got separate sections of code that want to add things to cloud-config or cloud-init . then you can keep them separated in input parts [15:54] right [15:54] looks interesting [15:54] for now its prototyping ... more important for me is to remember the internal ip of the controller so i can copy the results and logs back ;) [15:55] i thought about encoding that in user-data ... but guess [15:55] i will read through that page and see how i can make things cleaner with cloud-init/config [15:56] smoser: user-data is constrained to 16k, right? [15:56] yes, but it can be compressed. [15:56] ok ... just wonder how to best ensure that i dont grow more than that ... but seems its better to put everything in bzr like i try now [15:57] and just have the update, apt, etc. stuff and then run that [15:57] "runit" in http://bazaar.launchpad.net/%7Eawstrial-dev/awstrial/trunk/annotate/head%3A/awstrial/trial/ec2_helper.py is an example of appending parts [15:58] smoser: i know its lame, but is it safe to just copy a script like i have there in the user data field when launching instances on the webUI? [15:58] or better use ec2- commands with --user-data-file? [15:58] (well better is wrong question ...) [15:58] yes, it should work. [15:59] cool ;) [15:59] regarding 'safe' no one makes promises on sensitive data you put in there though. [15:59] asac, please let me know if pasting into that box does not work when running from command line does (for plain text) [15:59] i dont ever launch from that ui [15:59] sure ... safe == script is properly run ;) [15:59] smoser: i am still not sure how to copy stuff back ... :/ [16:00] copy back ? [16:00] as in phone data home ? [16:00] smoser: is hte idea to create a keypair for one run ... copy back to controller and then invalidate keypair? [16:00] smoser: yeah [16:00] like i want the logs and the image produced come back to my controller [16:00] for persistence [16:00] right. [16:00] but i dont know what best to do ... and how to ensure that that is not completely insecure [16:01] so it really just depends on how secure you want to be. [16:01] like ... putting private id_rsa in user-data ;) [16:01] what are the options? [16:01] well, no one makes promises on secure. but realistically, nothing outside your instance is going to get at that. [16:02] yeah ... but what are the options? [16:02] so, you could: [16:02] - create a set of iam credentials, share that with the instance, let hte instance write to s3, then invalidate the credentials [16:03] (you can even make a set of credentials that only work from a single IP address) [16:04] - share a secret in user data or in an snapshot/ebs-volume or s3 to the instance, then have it use that to post back to the controller [16:04] (ie, in a header of http post) [16:05] probably lots of other ways... [16:05] yeah [16:05] i think i like the idea to http post a notification back ... and then have the controller pull stuff over [16:05] rather than a push from the instance [16:06] but i dont want to muss the notification because then i guess the shutdown would have to be done by controller as well [16:06] and i hate having logic to shut down stuff after a certain amount of time if nothing happened ... though maybe i hav eto do that anyway [16:06] i will think a bit more about it ;) [16:06] so you say i can create a bucket for each instance and create a special key for just that bucket? [16:06] in s3? [16:07] hmm. i think there was a constrained of number of buckets [16:07] smoser: last thing for today ... in user data web field ... base64 encoded ... should i check that? [16:08] or does it mean that i pasted base64 encoded stuff? [16:08] (guess the latter) [16:09] ok fired off without base64 [16:09] there is a limit to the number of buckets you can have (i think) [16:09] but you may be able to limit to s3 bucket path [16:10] yeah, i would think it means you're pasting base64. [16:10] ok ... will check that out... also look into messaging stuff etc. [16:10] i didn't know you coudl do that. [16:10] let me see if my job runs ;) [16:11] asac, http://docs.amazonwebservices.com/IAM/latest/UserGuide/UsingWithS3.html [16:11] Example 5: Allow a partner to drop files into a specific portion of the corporate bucket [16:11] so you could have a bucketk "results" [16:11] smoser: one question wrt to money ... if i shutdown after 5 minutes do i pay for full hour? [16:11] asac, yeah. [16:11] damn ;) [16:12] the ceiling function kinda sucks. [16:12] my script failed now i cannot shut it down [16:12] guess i will try on micro next time ;) [16:12] there are ways to play faking user data post-first boot [16:12] so the user data script is not run in /root/ home directory it seems [16:12] in maverick its fairly easy [16:12] cannot find where it did the checkout :/ [16:13] hm... i dont actually know what the path woudl be. [16:13] etting up bzrtools (2.2.0-2) ... [16:13] Processing triggers for python-central ... [16:13] You have not informed bzr of your Launchpad ID, and you must do this to [16:13] write to Launchpad or access private data. See "bzr help launchpad-login". [16:13] Branched 2 revision(s). [16:13] nothing changes cwd so, whatever upstart would run something at [16:13] /var/lib/cloud/data/scripts/part-000: 8: ./ec2-tools/node/setup-android-build-node: not found [16:14] https://groups.google.com/group/ec2ubuntu/browse_thread/thread/d4d51238a2afb55b/bca69f3e89c88d0b?lnk=gst&q=user+data#bca69f3e89c88d0b discusses how you can test user data after initial launch [16:15] thx [16:19] guess i will use a spot instance next time i try it ;) [16:19] asac, its only $0.10 [16:19] let me check [16:20] smoser: 0,34 [16:20] smoser: problem is i need 64-bit for android [16:20] so either micro with slow EBS ... or large :/ [16:20] true. [16:20] not really a big problem either ;) ... will just expense it [16:21] i do use spots a lot for playing. the problem with them for development is that they're slower to start [16:21] ie, total time till up and reachable will be probably 8 minutes rather than < 3. [16:21] damn [16:21] my auto java license accept didnt work :((( [16:21] guess i have to set seen as well [16:22] yeah [16:22] smoser: do spot instances get killed earliest after 1h? e.g. if i pay in 1h rates? [16:23] you always pay in 1 hour in crements. [16:24] i was always under the impression that spot instances would be killed on 1 hour units. but i'm not sure. [16:24] ie, it would seem somewhat unfair for you got get a spot instance, and then them kill you after 10 minutes and charge you for 55 [16:24] er... for 60 === dendrobates is now known as dendro-afk === dendro-afk is now known as dendrobates [17:59] soren, you around ? [19:48] Why don't UEC images have devpts in fstab? Is it unnecessary because of the kernel? I'm customizing a Centos image (that will use an Ubuntu kernel) and was wondering about this difference. [20:12] smoser: http://paste.ubuntu.com/542014/ ... m2.xlarge with latest instance store 64-bit maverick :/ [20:12] doesnt boot :/ [20:12] known issue [20:13] smoser: oh ... so just for that instance type? [20:13] hold on. getting bug [20:13] can i select a different kernel? [20:15] bug 651370 [20:15] Launchpad bug 651370 in linux "ec2 kernel crash invalid opcode 0000 [#1]" [Medium,Fix committed] https://launchpad.net/bugs/651370 [20:15] you can't specify a different kernel [20:16] asac, hopefully updates next week maybe, with that through -proposed [20:16] hmm [20:17] i can select kernels when creating an instance ... whats that? [20:17] its what you think. [20:17] sort of [20:17] :) [20:17] why cant proposed kernels be made available there? [20:17] you can select a kernel, and prior to maverick, we would have published that kernel to ec2 that you could select [20:17] ah [20:17] but with maverick the kernel is "pv-grub" [20:17] maybe "proposed" AMIs ? [20:17] and pv-grub loads the kernel from inside the image [20:18] oh [20:18] ok [20:18] proposed amis woudl be somewhat useful for this sort of thing [20:18] yeah [20:18] definitly ;) [20:18] i would use it!! [20:18] and right now, lucid is building with --proposed [20:18] because i wanted to test some things, but in general, the idea is that the dailies get turned into the rleeases [20:18] bit for bit [20:19] maybe i could start doing a weekly -proposed, which would never get promoted [20:19] but i tihnk in general we have too much selection up there (with the daily builds) and that confuses people. [20:19] hmm [20:19] selection where? you mean on uec-images? ... maybe [20:20] (ie, you wouldn't get or expect an ISO to have -proposed enabled or have been built from there) [20:20] as in selection, i really mean in number of "ubuntu" images on ec2 [20:21] so this bug ... is that juts for my instance type? or just more frequent for large memory instances? [20:22] the reporter talksa bout t1.micro [20:22] but my previous instances worked well [20:22] like m1.large etc. [20:23] just m2.xlarge is causing pain now [20:24] its really only the larger memory instances. [20:24] damn ;) [20:24] those i want to test now :-P [20:24] t1.micro is what the opener (me) opened it with [20:24] ah [20:24] i just launch a t1.micro and run 'ubuntu-bug' [20:25] especially if you can't seem to get a instance type that you want to open to start :) [20:25] alrighty [20:25] the issue is really with getting a certain processor [20:25] so is lucid perfect? [20:25] ;) [20:25] lucid has some issues too. [20:26] booting big memroy? [20:28] lucid's big issue is https://bugs.launchpad.net/ubuntu/+source/linux-ec2/+bug/614853 [20:28] Launchpad bug 614853 in linux-ec2 "kernel panic divide error: 0000 [#1] SMP" [Medium,Confirmed] [20:28] there is a patch, but you can look at it (http://launchpadlibrarian.net/58956370/lp614853.patch) its not that pretty [21:47] smoser: Yes. [21:49] soren, i'll probaly try to ping you monday [21:49] smoser: I'm intrigued :) [21:51] soren, well, it was regarding the ec2 metadata swrevi ce [21:51] andthe fun that that that delivers [21:52] basically, i'm trying to make "full disk images" of the uec images, that ccan then just be booted under kvm. [21:52] in maverick i did this, but i did so via a floppy and bootinga partition image. the floppy loaded the kernel from inside the image but passes some custom kernel parameters that disable the waiting for the ec2 metadata service [21:52] smoser: How does that relate to metadata? The block mapping? [21:53] but if i just have one disk image, i'd like for its contents to boot correctly and not wait some stupid long time for the metadata service to come about. [21:55] i was wondering if you had any good ideas for how i could decide when to wait for it. but i can't use kernel parameters, because on uec, the grub.cfg is used for passing kernel params (so i can't just have that disable ec2 wiating) [22:02] I remember having this discussion a couple of times. :) [22:02] With you, even :) [22:03] I think our conclusion was that there's no really good way to tell the difference. [22:03] smoser: Ok, let's attack this from a slightly different angle. [22:04] soren, no, ther isn't. [22:04] smoser: What if we try to minimise the impact of this wait time? Why do we wait for it to turn up? [22:04] and i said i should have waited till monday as i have to run righ tnow. [22:04] Instead of just letting it wait in the background. [22:04] smoser: Ah. [22:04] :) [22:04] we wait for it to turn up because its possibly important [22:04] Alright, Monday it is. [22:04] smoser: Use cases? [22:05] really its only most important on first boot [22:05] but the idea is that the user can do things on first boot to an instance that will get done [22:05] such as setting mount points or importing ssh keys... [22:06] some of the stuff might be difficult to do "later" [22:06] anyway, i do have to run. [22:07] Sure. [22:07] I'd be happy to talk on Monday. [22:07] Have fun. === dendrobates is now known as dendro-afk === dendro-afk is now known as dendrobates === dendrobates is now known as dendro-afk === dendro-afk is now known as dendrobates