=== dendro-afk is now known as dendrobates === dendrobates is now known as dendro-afk === dendro-afk is now known as dendrobates [03:02] holy f [03:02] part-handler [03:02] there's some serious power here [03:02] i can write my own mime-type handler to handle chef's json attributes [03:05] if you include a part hanlder, does that ruin the pre-existing cloud-init support for various mime types? [04:16] hm having problems with user-data on the ubuntu 10.10 ami [04:16] http://ubuntu-cloud.pastebin.com/LwgRX710 [04:16] doesn't seem to set the date or anything really [04:19] cloud-init.log http://ubuntu-cloud.pastebin.com/n9rRgww5 === dendrobates is now known as dendro-afk [05:19] crazed, how did you make the multipart file ? [05:26] crazed, i'm wondering if it might not like your dos line feeds. [05:30] crazed, so, i downloaded your pastebin, then manually separated into 4 files (using vim), then [05:30] write-mime-multipart ./part-handler.py:text/part-handler ./node.json:text/chef-attributes ./solo.rb:text/chef-solo ./cloud-config.txt:text/cloud-config > ../user-data.txt.new [05:30] and launched with that user-data.txt.new as user-data-file, and i think it had the affect you wanted. [05:33] resultant user-data at http://paste.ubuntu.com/574292/ [05:34] crazed, but realize that 'output: [05:34] all: "> /tmp/cloud-init.log"' is only in natty === dendro-afk is now known as dendrobates === daker_ is now known as daker === daker_ is now known as daker === daker_ is now known as daker [14:14] oh really? [14:14] thanks for the tips [14:14] smoser: i used write-mime-multipart the same way you did there [14:16] crazed, well, it did "work for me" when i tested it. [14:16] that's actually awesome [14:16] what AMI did you use? [14:16] or were you on euca [14:17] http://groups.google.com/group/ec2ubuntu/browse_thread/thread/8eb2f732e03b0334 [14:17] has more details of what i did [14:17] i did it on a private cloud. but used maverick [14:17] hmm [14:18] i'm going to give it another try [14:22] also i did notice that write-mime-multipart wasn't in the bzr repo under tools anymore? [14:23] i wound up running it from a machine that had cloud-init [14:24] 0.5.15-0ubuntu3 is the version of cloud-init i'm running [14:26] its in cloud-utils now [14:26] crazed, [14:26] and fwiw, do not write log to /tmp [14:26] or the tmp cleaner will clean it [14:27] hm alright, i was just seeing if it would work [14:27] ie, you'll start writing there early in boot, and then aprocess that does an 'rm -Rf /tmp/*' (effectively) will remove your file [14:27] that wont work for 10.10 [14:27] timezone, final_message, and output are all 11.04 only. [14:27] ah [14:28] where is the output of the parthandler print messages supposed to go? /var/log/cloud-init.log? [14:28] in 10.10 that is [14:29] hm i've copied your multi part file, still don't get the part-handler to run it appears [14:32] is there a verbose way of initiating the cloud-init run, without relaunching instances? === RoAkSoAx is now known as andreserl [14:50] smoser: i've been using cloud-config to bootstrap machines in EC2 for the last couple months, and just ran into an issue yesterday for which i'd like to offer some feedback. it seems the security.ubuntu.com repository is overwhelmed with traffic (since at least yesterday morning) and this causes cloudinit to die while waiting to connect (for cloud-config's apt_update or apt_upgrade). Using the EC2 mirror for the [14:50] security repository seems to work just fine, but apt_mirror has no effect on the security repository location. [14:52] besides that one small issue cloudinit & cloud-config are great and I'm excited about the new features coming in natty, thank you! [14:52] they do seem pretty powerful, just wish i could get my part-handler to work :) [14:52] crazed: I had written this on multipart handlers, might be helpful http://foss-boss.blogspot.com/2011/01/advanced-cloud-init-custom-handlers.html [14:53] kim0: that's where i got most of my info from [14:53] very good stuff there [14:53] hehe, ok glad then :) [14:53] i see the part handler getting output to /var/lib/cloud/data/plugin [14:53] but it doesn't seem to actually run [14:53] i see nothing in the logs, so i'm not sure what's goin gon [14:54] OH [14:54] it's in teh console output [14:55] ah my python syntax is off lol [14:55] * crazed has been using too much ruby lately [14:56] * kim0 likes snakes not gems [14:56] i like chef for config management [14:56] python is pretty nice though [15:02] kim0: just wanted to add i've also learned a lot from your blog, thank you [15:02] oh wow you're in egypt? [15:03] crazed, i often do 'rm -Rf /var/lib/cloud && reboot' [15:03] if you want to change your user data, then you have to use the 'seed' stuff [15:03] hmm [15:03] https://groups.google.com/group/ec2ubuntu/browse_thread/thread/d4d51238a2afb55b [15:04] oh i did see that thread too [15:04] should have read it :) [15:07] smoser: in 10.10 is it possible to get script output elsewhere besides the console? or is that natty only at this point [15:07] just takes amazon more time to display the output than it does for me to login and check files [15:09] yeah, its annoying. [15:09] in 10.10, any script or program you run, just have it redirect its own output to a file elsewhere rather that writing to stdout [15:09] ie [15:09] #!/bin/sh [15:09] ah i know what you mena, that's what i was doing originally [15:09] exec >/root/my.log 2>&1 [15:10] then i had this idea to separate the node attributes file from the script [15:10] to make things more portable [15:11] photerran, hm.. i don't really knwo what to say about that. [15:11] yay part-handler is now working for me [15:12] yesterday the security mirrors were *really* slammed [15:16] smoser: indeed they were. i was thinking it would be nice to be able to set the security repository location in cloud-config (apt_mirror_security?) or just using the EC2 mirror as is done for the other repository locations... [15:17] smoser: is there an order in which things are ran? part-handler, user-scripts, etc [15:17] apt_mirror_security might be ok, with an explicit warning on it. [15:17] photerran, the reason they don't do that is that they want security fixes out as fast as possible [15:17] and mirrors lag [15:17] crazed, part-hander is "very early" . actually while cloud-init is running and consuming metadata [15:18] user-scripts are "rc-local" like [15:19] awesome [15:19] that should work out how i want then [15:22] imho, using the EC2 mirror would get the updates out faster than this [15:24] part-handlers can block boot entirely [15:24] just fyi. [15:24] maybe all the EC2 machines out there are what is causing the extraordinary congestion on security.ubuntu.com [15:24] smoser: how so? [15:24] photerran: oh thanks! [15:24] photerran, right, but it takes time to propogate to the mirrors. [15:24] its originally published to security, then the mirrors sync from that [15:30] hmm damn it [15:30] missing ppa [15:31] ah i see. do you think there's any chance for getting the updates published to the EC2 mirrors when they're published to security? know where I might go to suggest that to the right people? [15:40] crazed, they can hang block boot, ie by 'sleep 3600' [15:40] but i fyou stack trace in one, it shouldn't screw up anything else in cloud-init [15:40] did it ? [15:40] ie, if cloud-init did not catch the exception, please file a bug [15:44] oh no it didn't screw up anything else [15:45] i just wasn't seeing my syntax errors [15:45] since they were in the console output, i didn't think to check there [15:59] * kim0 rings a bell, Community meeting starting in 2 hours. All welcome to hangout and chat [15:59] what's that? [16:03] crazed: A chance for everyone to get to know everyone else [16:03] chat about how they're using ubuntu/cloud [16:03] where, here? [16:03] share info, have a few laughs :) [16:03] right here [16:03] in 2 hours [16:03] oh [16:03] i'll be here [16:04] awesome [16:04] photerran: You're welcome to attend as well [16:04] Everyone is of course :) [16:04] * kim0 jumps back to finishing his virtio blog entry [16:05] tyvm! i intend to. [16:05] awesome :) [16:10] virtio! [16:10] what are you writing about it === dendrobates is now known as dendro-afk === dendro-afk is now known as dendrobates [16:19] crazed: just getting Windows7 installed over KVM with optimized virtio disk and network drivers [16:19] ah alright [16:32] http://ec2-72-44-58-95.compute-1.amazonaws.com/ [16:32] :) fully configured using cloud-init, chef-solo, and drush [16:32] rock n roll :) === andreserl is now known as RoAkSoAx [17:16] seriously impressed with this [17:16] now to try and scale it out using cloud formation [17:21] crazed: would love to hear your thoughts about the tools and process to get this running, if you have the time and inclination to write something down. [17:22] i'll be writing a blog post [17:22] probably tonight [17:33] Anyone know of problems installing sun jdk (6u24, 64bit, rpm format) in a chrooted loop device? I've been trying to get this installed but it hangs the loopback and I can't umount... [17:35] crazed: awesome, I'll look forward to that. Can you point me to your blog so I can look for it? [17:37] mrjazzcat: http://allanfeid.com [17:37] crazed: thanks! [18:00] patrickw_: are you installing right from the partner repo? [18:01] * kim0 rings a bell .. Cloud community meeting starting [18:01] :) [18:01] any new faces around [18:02] let's all get to know each other [18:02] now everyone is silent hehe [18:02] * photerran is new here [18:02] photerran: hey o/ [18:03] nice to meet you all [18:03] let's get to know each other better [18:03] can you introduce yourself please [18:04] well you can call me photerran, i'm a sysadmin and lately have been working on building a hosting platform in EC2 using ubuntu [18:04] building a hosting platform on top of ec2 ? [18:05] indeed [18:05] interesting stuff :) [18:05] photerran: do you offer VMs directly to your customers [18:06] no we provide a web service to ad agencies and i'm working on a new platform to run our custom application === daker is now known as Ghost_of_daker [18:07] hehe [18:07] awesome :) [18:07] crazed: howdy o/ [18:08] i'm using cloudinit to bootstrap puppet to manages services & apps, and using glusterfs (on top of EBS and instance stores) to provide shared storage for different parts of the app [18:08] yeah it is awesome [18:08] really beats having to roll my own AMIs :) [18:08] indeed it does hehe [18:08] hmm that stuff sounds interesting [18:09] photerran: have you heard about cloud days [18:09] yeah about two hours ago hehe [18:09] hehe [18:09] ummm [18:09] photerran: what about giving a session [18:10] sharing the knowledge you've gained building that system basically [18:10] it's nothing hard or complicated [18:10] would this be over IRC? [18:10] Yep [18:10] just like this ;) [18:11] hmm [18:11] interesting [18:11] photerran: actually, if you join #ubuntu-classroom [18:11] there's a session going on right now [18:11] just to get a feel [18:11] it's part of Ubuntu Developer Week [18:11] cloud days is very similar to that [18:11] crazed: hey .. what's the interesting part :) [18:11] thanks i'll have a look [18:12] using gluster on ec2 [18:12] how's the performance with that? === dendrobates is now known as dendro-afk [18:13] the first thing that came to my mind as well :) [18:13] hehe [18:13] do you want the long or the short answer? lol [18:13] both haha [18:13] LOL [18:13] short answer is "not too bad" [18:13] long answer is... [18:14] it's latency sensitive, so performance increases as file size increases [18:14] hmm [18:14] if your files are 100kB+ then performance should be OK, but I'm using front-end caching in apache & varnish to make it better [18:15] what excatly are you storing on gluster? [18:15] static content or? [18:16] also, the speed seems to be limited by the EC2 LAN, not the instance size, so spreading the gluster volume out over more machines (and using the native gluster client) can get better aggregate throughput even tho single transfer rate may not be all that great [18:16] this is all very preliminary btw [18:16] YMMV of course [18:17] storing web content, HTML, JS, CSS, JPG, etc... the usual stuff [18:19] photerran: wonder if you can aggregate many little files into one bigger file to use the increased throughput [18:19] any reason you're not using s3 for that? [18:19] can't think of a way though [18:20] crazed: i'm a sysadmin not a developer. thats a requirement of the app [18:21] photerran: did you play with ceph as an alternative [18:22] i read about it but it seems too new to bet the farm on [18:22] my conclusion at the time was "maybe when btrfs is ready for production" [18:22] kim0, nice work: http://cloud.ubuntu.com/ami/ [18:23] the ap-northeast-1 images are there. [18:23] also i'm not too excited about having a metadata server... gluster is fully distributed [18:23] you beat alestic.com to it :) [18:23] hehe [18:23] smoser: oh northeast .. that's the Tokyo one [18:23] * smoser gives erichammond a rasberry [18:23] cool! [18:23] photerran: fair enough, same here actually. we deal with managing applications in the cloud, and typically will steer away from any shared filesystem usage as s3 gives you the added benefit of faster image loads (using asset host) and the ability to use cloudfront as a basic CDN. usually we will help teh devs understand what needs to be done an help migrate [18:24] kim0, yes. all our released images are there (not milestones though) [18:24] awesome :) [18:24] it sure seems to me like you're not caching though [18:26] smoser: you mean performance wise ? [18:26] yeah [18:26] just seems like "processing" [18:26] crazed: yeah we'd like to move in that direction, but we decided to migrate the legacy app first & develop those new features later [18:26] well .. it's a chance to have you look at the code [18:27] momento :) [18:27] * kim0 prepares to be embarrased [18:27] * smoser apologizes for interupting a conversation [18:27] lol [18:28] no appology needed [18:28] either way, it's cool to see people doing things with puppet [18:28] which i see is supported in cloudinit [18:28] personally i'm a fan of chef, so i've been working on getting a solid solution using the part-handler parts of cloudinit [18:29] only cool thing i have so far is the ability to launch a full dev environment of drupal 7 with any number of modules installed [18:29] well thats a start [18:29] it is, i have a decent chunk of chef cookbooks which i should be able to tie into relatively easily === dendro-afk is now known as dendrobates [18:30] then put that together with cloudformation to bring up larger environments, but i'm not sure how to handle making sure the various pieces exist [18:30] for example, there's a race condition, what if my database isn't up before i try to connect to it [18:31] i suppose i could just add some retrying in there [18:33] i have yet to try cloudformation, but it looks nice. i'm hoping for better documentation before I dive in [18:35] crazed: I'm interested in what you describe as a race condition [18:35] so basically .. chef configs a bunch of machines [18:35] and starts a bunch of services [18:35] yes [18:35] but you have no way to ensure dependencies right [18:35] exactly [18:35] for example in a cloud formation stack, i have an RDS instance and an ec2 instance [18:36] nor pass info between machinse (DB passwords ..etc) [18:36] the ec2 instance is going to try and populate the rds instance database [18:36] but if it doesn't exist, that will fail [18:36] i added basic cloud formation support to fog recently, so i've been playing around with it [18:37] seems cool, but would really like to see some sort of control on how to make sure resources of the stack are available [18:37] hmm the main cloudformation web page suggests there is a way to do just that [18:38] under the "Easy to Use" section [18:39] really? [18:39] hm [18:40] i've only got some basic cloud formation templates [18:40] there's a lot of syntax there [18:41] btw, the cloudformation main page i mentioned is http://aws.amazon.com/cloudformation/ [18:42] haven't checked that out yet .. do you have to write that template yourself [18:43] yes but there's a few of pre-written ones from amazon [18:44] it's brand new, announced on Friday, barely a week old [18:45] * kim0 tweets the new Tokyo Ubuntu images → http://twitter.com/#!/ubuntucloud .. Thanks smoser [18:45] crazed: photerran if you guys are interested .. follow that twitter account [18:47] I havent yet jumped on the twitter bandwagon, but when i do, i will follow [18:47] until then i just check in on the web every once in a while [18:48] https://s3.amazonaws.com/cloudformation-templates-us-east-1/AutoScaledPHPWebServerWithMultiAZRDS-1.0.0.template [18:48] photerran: are you on the facebook bandwaggon ? hehe https://www.facebook.com/ubuntucloud [18:49] nope [18:49] * kim0 hugs photerran :) a clean sole [18:49] haha [18:49] still looking for this ability to depend on other resources.. bha [18:49] the whole shoe is clean [18:49] j/k [18:50] https://s3.amazonaws.com/cloudformation-templates-us-east-1/Drupal-NoSSH-1.0.0.template [18:51] crazed: do either of those templates show how to declare an EC2 -> RDS dependency? [18:51] not that i can see [18:51] photerran: I'm actually installing into a centos image, thus the rpm [18:52] hm it might be handled by some scripts on the AMI they're using [18:53] because i don't see any user data scripts to install drupal either [18:53] just basic userdata here [18:57] hopefully AWS will publish better docs soon, i suspect there's more to cloudformation than these templates show. you might want to ask the question on the AWS forums in the mean time though [19:01] crazed: man, consider giving a session on chef automation as well .. sounds damn interesting :) [19:03] kim0: how often do you do these community meetings? [19:05] photerran: weekly same time [19:05] it's just a time for everyone to get together and talk [19:05] photerran: and crazed .. it would be great to mark your calendars [19:06] and try to make it every week [19:06] I'm trying to build some momentum around ubuntu/cloud which is for everyone's benefit really [19:08] ok cool. i like it. i've been using ubuntu a long time and am just now starting to get involved with the community. [19:09] until recently i've been using ubuntu personally & centos/rhel professionally, but that's changing as I mentioned earlier [19:10] * kim0 was a centos junkie for about 5 years ago [19:10] but I've been building some consistent respect for ubuntu server! [19:10] lost of innovative stuff really .. [19:11] and since we're still a small community .. every contribution counts :D if you're thinking about registering a session, "just do it" hehe [19:12] yeah its exciting. my experience so far with ubuntu on EC2 has been very positive until yesterday when the congestion on security.ubuntu.com got in the way of spinning up new instances [19:12] * kim0 nods [19:13] smoser: any idea why do those mirrors melt .. it seems it has happened a few times already [19:13] can't we balance on a few [19:14] $ host us-east-1.ec2.archive.ubuntu.com [19:14] us-east-1.ec2.archive.ubuntu.com has address 10.210.205.172 [19:14] us-east-1.ec2.archive.ubuntu.com has address 10.250.142.223 [19:14] us-east-1.ec2.archive.ubuntu.com has address 10.252.111.96 [19:14] us-east-1.ec2.archive.ubuntu.com has address 10.202.26.15 [19:14] i have a ticket open. they should not (as you say) melt. but yes, they have. [19:14] it was the default location for the security repo (in sources.list) that got me, as i mentioned to smoser earlier [19:15] changing it to the EC2 mirror works great, but that's not easily automated when cloudinit it bootstrapping an instance for the first time [19:15] is bootstrapping* [19:19] fwiw, in my case i'll be removing the apt_update & apt_upgrade from my cloud-config, leaving only the puppet agent stuff in there, and doing the apt_update & apt_upgrade later on in the process with puppet, after puppet has updated the sources.list [19:19] photerran, the easiest way to do it would be to do dns trickery [19:20] you can write security.ubuntu.com into /etc/hosts before apt comes through and updates. do that with a 'boothook' on maverick [19:20] this is not tested. but i think it would work. [19:21] ooh good idea, thank you [19:21] and if you try to intsall a package (puppet), then it will do an update. [19:22] ah yes that would have bitten me soon enough [19:23] kim0: i don't mind doing a session on chef automation, though i'm by no means 100% expert at chef [19:23] there's a lot to know about it [19:23] i use chef-solo, mostly because we don't have a subscription ot the opscode platform and the opensource chef-server doesn't support multiple clients that well [19:24] crazed: a pretty basic session would be more than enough [19:24] crazed: that idea is to introduce the concept and answer questions [19:24] i could handle that [19:24] the time doesn't really allow for anything close to advanced :) [19:24] crazed: awesome! [19:24] do you have a wiki account [19:24] i do [19:24] actually [19:25] crazed: edit https://wiki.ubuntu.com/UbuntuCloudDays/Timetable [19:25] now i have to remember my wiki account password :) [19:25] hehe lol [19:25] oh this is launchpad [19:26] hm i think i know that login [19:26] :) [19:26] the wonders of the human memory [19:26] oh yeah that reminds me, what are the requirements to be a presenter? wiki account... anything else? [19:26] photerran: not even that :) [19:26] I could edit the wiki for you [19:27] crazed: could you email me kim0 AT ubuntu.com [19:27] oh nice [19:27] photerran: you too please if you'll be giving a session [19:27] this is just so that I can remind you and coordinate [19:27] thanks guys [19:28] kim0: sure what kind of email? [19:28] crazed: just "hi" :) wanna get your email [19:28] photerran: so do you have a wiki account? have you decided on the sesion thing yet ? [19:29] photerran: scalable cloud web app would be great :) [19:29] no i dont have an account, and yeah i'd like to do a session. [19:29] awesome [19:29] photerran: what's the session title [19:29] * kim0 will edit [19:30] not sure how much detail i can go into, i'll talk it over with my team leader & let you know in my email what I'll cover [19:30] give me an hour or two to get it sorted out [19:30] great .. again, there's usually not much time for lots of details .. it's mostly laying down the basics and answering qquestions ..etc [19:30] so nor worries [19:30] no* [19:31] photerran: thanks for the contribution [19:31] ok great, it's my pleasure [19:32] all mine :) [19:33] now i have to figure out UTC [19:33] woo [19:33] lol [19:33] crazed: time now in UTC is 7:33pm [19:33] date -d "14:00 EST" -u [19:34] your tz skills are l33t :) [19:34] I end up googling hehe [19:34] wolframalpha is good for that too [19:35] http://www.wolframalpha.com/input/?i=time+utc === mrjazzcat is now known as mrjazzcat-lunch [19:38] kim0: how long are these sessions typically [19:39] crazed: just one hour [19:39] seems all the times on there are during my work hours [19:39] hmm [19:39] woah actually [19:39] i'm 80% sure i'll be in seattle those two days lol [19:39] going to amazon hq [19:39] hmm [19:40] does that mean you may not be able to present ? [19:40] just email me once you know [19:40] possibly not this time, is this monthly/weekly? [19:40] mm .. it's usually of less frequency, although there's zero problem doing this more often if people are willing to share :) [19:41] crazed: let me know once you're sure whether or not you can make it [19:42] kim0: will do [19:43] thanks [19:50] just my luck... signed up for launchpad to get into ubuntu wiki, now ubuntu wiki gives 500 error after the SSO gateway [19:51] i was going to just do this once i logged in, but maybe i should mention it here first anyway [19:52] kim0: the session details link (to Sessions page) on the cloud days page is broken, maybe you meant it to be the Timetable page instead? === dendrobates is now known as dendro-afk === Ghost_of_daker is now known as angeldaker === dendro-afk is now known as dendrobates === photerran is now known as lz0 === mrjazzcat-lunch is now known as mrjazzcat === dendrobates is now known as dendro-afk === dendro-afk is now known as dendrobates === dendrobates is now known as dendro-afk === dendro-afk is now known as dendrobates === lz238 is now known as semiotic === dendrobates is now known as dendro-afk [21:31] kim0: I sent you an email about the cloud days session. i'm all cleared but had to change nick (used to be photerran) as I explained in the email === dendro-afk is now known as dendrobates === dendrobates is now known as dendro-afk [21:53] semiosis: just read your email [21:53] that's awesome [21:53] what you mentioned is totally fine [21:54] semiosis: puppet or gluster .. hmm what about two sessions :) [21:54] semiosis: still unable to edit the wiki ? should I do it [21:55] let me try again, i was getting 500 errors earlier [21:56] semiosis: that sometimes happen, but the "action" usually completes fine [21:58] ah yes thats the case === dendro-afk is now known as dendrobates [22:02] now getting 500 errors trying to open the edit page... i'll try again later. in regards to two sessions... i wouldnt want to take up two slots just yet, hopefully someone else will see an opening and want to contribute as well [22:03] semiosis: then I'd take gluster and more generally .. storage best practices for scalable web apps [22:03] semiosis: should I write that title ? [22:03] let me know what to write === dendrobates is now known as dendro-afk [22:07] how about "Scaling legacy web apps in the cloud with GlusterFS" ? [22:08] btw since we're going in the direction of gluster i should note that I dont use the ubuntu packages, but the latest version from gluster [22:13] kim0: the wiki is working for me now, i added the topic to the timetable. thanks for your support! === dendro-afk is now known as dendrobates === mrjazzcat is now known as mrjazzcat-afk