[00:16] argh! WARNING: Invalid keys at [https://launchpad.net/~clint-fewbar/+sshkeys] [00:18] lol [00:18] all the above seems about the usual standard for ubuntu [00:19] oh you are a canonical/ubuntu guy? === dendro-afk is now known as dendrobates [00:49] hello everyone [00:51] it is a requirement to download the server version 10.10 of ubuntu to use landscape and set up the private cloud? or it is the same kernel I have the alternate version 64bit 10.10 should that work? [00:56] basically what I am asking do I need to download the version 10.10 server of ubuntu or in reality the version 10.10 desktop 64bit will have the same packages and capabilities? [01:15] 3mins, uber patient :) === dendrobates is now known as dendro-afk === dendro-afk is now known as dendrobates === dendrobates is now known as dendro-afk === dendro-afk is now known as dendrobates === dendrobates is now known as dendro-afk [09:27] hi all, i have running an instance : euca-run-instances $AMI_IMAGE -k $KEY_NAME --addressing private -n 1 -g $CLUSTER_MASTER -f "$bin"/$USER_DATA_FILE -t $INSTANCE_TYPE $KERNEL_ARG --ramdisk eri-A535178D [09:27] and i have installed cloud-init package [09:27] but i don't why "-f "$bin"/$USER_DATA_FILE" do not execute [09:28] is there something wrong ? [09:29] the file "$bin"/$USER_DATA_FILE" did *NOT* be executed [09:36] you might wanna show the stdeer and exit code [09:37] yeah, is $USER_DATA_FILE a regular shellscript? Can you get it from the instance via wget? [09:42] this is the shell script : http://pastebin.com/mijcWEmc [09:45] captcha, lame [09:46] ok so where is the output? [09:47] this shell script starts the hadoop daemon, no output [09:49] you might wanna run it under -ex so you can see what is occurring with the commands. [09:51] i have copy this file to my instance , and it can run correclty [09:52] sorry, what do u mean run it under -ex ? [09:52] they are bash parameters [09:52] going to dinner; bye [09:53] flaccid : ok ,tkx.bye [10:02] hmm...because it can running in the instance correclty, so i think the shell script is ok [10:02] so i think maybe the cloud-init has problems [10:03] http://pastebin.com/VTneuzuE [10:13] TeTeT: how can i get it from the instance via wget ? [10:14] wget from the instance meta-data ? [10:18] superxgl: hmm, don't know from the top of my head, something like wget http://169.254.169.254/current/userdata [10:19] superxgl: http://docs.amazonwebservices.com/AmazonEC2/dg/2007-01-03/AESDG-chapter-instancedata.html [12:26] TeTeT: thnx. i will have a try [12:27] TeTeT: http://pastebin.com/5vRdxGy1 [12:28] Is there a good opensource project for providing a layer ontop of ec2 or uec for scalable websites or webapps? [12:28] something like scalr.net [12:28] scalr is open source. [12:29] superxgl: sorry, out of ideas then [12:30] TeTeT: tkx. i will try finding it out [12:31] soren: yeah but it isn't perfect ;) I just wanted to ask for alternatives :D [12:33] michael__: If you want better answers, ask better questions. [12:33] michael__: Like, say, explain how scalr isn't meeting your needs. [12:34] scalr isn't stable yet, the current version has a lot of bugs [12:34] no automatic caching (no needed, but would be a great feature) [12:35] a Paas layer would be really cool. something like google app engine for uec [12:36] Cloud Foundry perhaps? :) [12:41] where is the cloud-init's message output ? [12:43] superxgl: on the console, via euca-get-console-output [12:45] [root@CLC bin]# euca-get-console-output i-464C07BC [12:45] i-464C07BC [12:45] 2011-04-13T11:59:22.263Z [12:45] NOT SUPPORTED [12:46] hmm...eucalyptus not supported [12:46] i found /var/log/cloud-init.log ,but the file is blank [12:53] maybe /var/log/syslog [13:00] kim0: i find all the file under /var/log, could not find. [13:02] superxgl: cd /var/log ; grep -r cloud-init * ? [13:04] hereis the cloud-init file : http://pastebin.com/LYyrxxJP [13:05] -bash-3.2# cd /var/log ; grep -r cloud-init * [13:05] rpmpkgs:cloud-init-0.5.14-23.amzn1.noarch.rpm [13:08] maybe the cloud-init is not running ? === dendro-afk is now known as dendrobates [13:14] maybe the grepping is just not right too [13:14] that's an amazon linux instance right .. never used that before [13:16] no, it's not an amazon linux instance, it's an eucalyptus instance [13:16] but the cloud-init package is downloaded from the amazon linux instance [13:17] maybe that's the problem ? [13:17] maybe i have to change something ? [13:18] just use an ubuntu instance ? :) [13:19] emm..but now i need to use centos instance [13:21] you'd have to dig deep then how the package starts with the system? rpm -ql cloud-init [13:22] is there a service, is it started, open the script, check where it logs ..etc [13:24] kim0 :　oh, tkx. it is a really useful command. http://pastebin.com/rfUS8Zjv === chuck_ is now known as zul [14:37] http://open.eucalyptus.com/forum/mysql-server-eucalyptus-image-how "This only works currently with Ubuntu guests and CentOS AMI guests." [14:37] maybe not CentOS EMI guests [14:39] smoser: I'm checking the latest cloud-init in natty... I want to use it on a baremetal install, I see the preseed values but I'm missing how I specify meta-data/user-data urls? can I do that as a kernel option? [14:40] yes... [14:40] ie. If I seed it to 'NoCloud', How'd I seed /var/lib/cloud/seed...? [14:40] let me look. [14:40] theres kind of two ways i think [14:40] I'm just poking around the code, but I thought asking you might be quicker :) [14:41] i would really like your help on getting this documented... [14:41] i think the 2 ways are something like: [14:42] a.) drop files ('user-data' and 'meta-data') in /var/lib/cloud/seed/nocloud-net [14:42] Sure, I'm kinda documenting this as I go anyway but for a different project, should be easy to pull out the relevant bits [14:43] user-data can be empty, meta-data should have a value 'seedfrom: http://tinyurl.com/sm-' [14:43] ah-ha :) [14:43] what will then heppen is it will pull from http://tinyurl.com/sm-meta-data and http://tinyurl.com/sm-user-data [14:44] the second way you can do this is via kernel command line. you can put cloud-config values in [14:44] ok, so I'm thinking in a late_command I'll pull the meta-data and user-data files down then... or earlier in the boot grokking /proc/cmdline for urls [14:44] ah so there is that already [14:45] curl http://202.115.132.66:8773/latest/user-data [14:45] i can get the user-data [14:46] superxgl: try doing that if you aren't actually on a cloud :-) [14:46] ie. on baremetal [14:49] TREllis: sorry, i could not follow u. what do u mean " on baremetal" ? [14:50] TREllis, right. that will work. [14:50] or should [14:51] smoser: but you say 2nd way, via kernel command line, what's the option 'user-data=url.to/user-data' ? ;-) [14:51] TREllis, ./doc/examples/kernel-cmdline.txt has info on how you feed the kernel command line [14:52] you can just set cloud-config values. and one of those values (i think) you can set is 'meta-data' [14:52] superxgl: ie, you are installing a bunch of physical systems and you have zero meta-data service on your network [14:52] err... 'seed-from' [14:53] smoser: ah-ha, found that file thanks, I'll write up a small post on it. [14:55] TREllis, please let me know what you find. i really apologize for not having done this myself. [14:55] but i would like to get the doc into cloud-init [14:55] smoser: no worries, it's bleeding edge new stuff :) [14:55] and fix anything that doesnt work. [14:56] TREllis ： sorry for my poor englist. so what should i do to get the cloud-init running ? i am still a little confused.. [14:57] on a related note, I notice that cobbler doesn't respect --kopts-post for ubuntu [14:57] superxgl: sorry, I don't follow you, I was talking about a different topic [14:58] TREllis : hmm...seems that you were talking about cloud-init , but i could not follow ;-) [14:59] superxgl: we were discussing ways you can provide the same meta-data and user-data like ec2/eucalyptus for cloud-init but without using ec2 or eucalyptus [15:01] TREllis : oic...it's exciting.. [15:04] TREllis, bummer regading cobbler [15:11] smoser: actually... it might just be me. I notice it only uses them in a post snippit which I'm not using, so as long as our grub2 works with 'grubby' it should be ok [15:16] maybe i can use this way on my instances [15:16] i will put the scripts on my instances [15:17] and once the instance start , make the scripts running === salgado is now known as salgado-lunch === dendrobates is now known as dendro-afk [15:34] -bash-3.2# python /usr/bin/cloud-init [15:34] File "/usr/bin/cloud-init", line 48 [15:34] except IOError as e: [15:34] ^ [15:34] SyntaxError: invalid syntax [15:34] looks like the script has some syntax errors ? [15:46] ? looks like cloud-init uses python 2.6 === dendro-afk is now known as dendrobates [15:52] i have installed python 2.6, but my default version still is python 2.4 [15:54] but not this problem [15:57] -bash-3.2# python /usr/bin/cloud-init [15:57] Traceback (most recent call last): [15:57] File "/usr/bin/cloud-init", line 23, in ? [15:57] import cloudinit [15:57] ImportError: No module named cloudinit [16:27] well, superxgl, i wonder how you installed, but you're not here. === niemeyer is now known as niemeyer_lunch === salgado-lunch is now known as salgado === niemeyer_lunch is now known as niemeyer [17:19] smoser: from what I've seen it seems he copied the scripts to whatever os and hoped it would run [18:20] nwl yeah cloud foundry looks cool missing the php support :D === daker is now known as daker_ [18:21] any other opensource layer for using uec as a scalable iaas or as a paas (like scalr.net or google app engine) thanks for suggestions [18:21] michael__: none that are very good or mature [18:22] yeah that'S my problem atm [18:22] scalr isn'T bad so [18:22] but paas would be awesome, but it should support php node.js at least [18:30] nwl openfoundry looks great they will add php support, but when :D === hallyn is now known as hallyn_afk [19:01] Hi everyone [19:01] Ensemble community weekly meeting starting [19:02] basically this meeting should help keep community members aware of ensemble development [19:02] I'll be directing any interested people to attend [19:02] also summaries of this meeting will be posted to planet ubuntu ..etc [19:02] This week and last week in ensemble.. we've had a few things go in and a few things in progress. the overall status board for this milestone development (ends at UDS-O) is at http://people.canonical.com/~niemeyer/budapest.html [19:03] ideally, any future community contributions would be discussed as well [19:03] Since this is the very first time [19:03] I think it might be appropriate [19:03] We'll soon be moving that to https://ensemble.canonical.com, btw.. we're just getting the web site organized. [19:03] Well, s/organized/setup [19:04] to introduce what works "today" in ensemble .. and what is being worked on [19:04] niemeyer: Great [19:04] Last week the ensemble-team landed some changes to formula authorship, specifically the relation-changed-hook was broken out into three separate hooks the relation-joined, relation-changed, relation-depart. [19:05] hazmat care to explain why that split was needed [19:05] Currently the team is working on a way for the admin to configure services, formula upgrades and fixes to recipes for the the hook change and bashifying our examples. [19:06] kim0, in the course of three formula authors writing hooks, it became clear that none of them where getting the sequencing write on the exchange between services. Ensemble was calling the relation-changed hook with one three events join, changed, depart. [19:06] I think it makes sense to touch a bit deep on what each change would mean to a user or formula developer [19:07] it should be noted that the example recipes for wordpress and mysql have been updated to the new hook semantics seen with -relation-joined, -relation-changed, and -relation-departed [19:07] join denotes that a unit of the related service came online [19:07] change denotes the unit of a related service changed its settings [19:07] depart denotes the unit of a related service went offline [19:07] hazmat: perfect [19:08] hazmat: can you comment on the stuff you're doing for next week as well [19:08] There are additional descriptions of what a service/service unit/ and relation in our online docs. http://j.mp/ensemble-docs [19:08] Thanks for noting that [19:09] hazmat: How is formula upgrades planned to work [19:10] The problem was that formula authors where coding things to relation-change, but that created a subtle timing issue for the sequence, in a distributed system, the change may have already happened by the time the join event was recieved, such that there would be no changed.. The change event is generated/observed against the last time the unit saw the remote unit settings. [19:11] Since it was clear this was a common and subtle issue, we decided to change the hook invocation, such that 1. the hooks where split out into relation-joined, relation-changed, and relation-broken [19:11] 2. That relation-changed would always be invoked after relation-joined [19:11] Yeah makes sense [19:11] I think that makes sense from a configuration management point of view [19:11] There's still some work to be do to upgrade the principia repository with these latest semantics [19:12] So moving on to formula upgrades [19:12] Great way for anyone wanting to wet hands with ensemble .. would be fixing principia to the new hook semantics :) [19:12] (based on my experience, the upgrade to these new hook semantics is quite painless) [19:12] jimbaker: is there some pattern ? [19:13] kim0, basically what you see is that the handshaking becomes explicit [19:13] prior to the hook semantic changes, formula authors, including myself, were using the accidental ordering of hooks to do the handshaking [19:14] setup -> joined , tear-down -> departed right [19:14] now what you do in your script is you get your relation settings if you need to consume them. if not there, wait until they become available [19:14] The scope of upgrades is pretty large when talking about an arbitrary services, for now to we're tackling an initial step of upgrading just the formula itself, after some back and forth, we'll be restricting formula upgrades to only those service units that are currently running, if its a broken state, it won't be upgraded. We're separately adding a notion of resolving workflow errors, with optional hook retry. [19:14] in your script it is as simple as doing an immediate exit 0 at the point of getting the setting [19:15] jimbaker: thanks for exaplining that .. guess reading the built-in examples is also a great way to read an example [19:15] (end of story on painless changes for new hook semantics, it's that simple) [19:15] cool [19:16] so back to upgrades, end result there will be two new ensemble subcommands, ensemble upgrade-formula --repository=examples mydbservice to upgrade a service formula. [19:16] and ensemble resolved [--retry] unit_name [19:16] hazmat: why isn't upgrading a formula as simple as departing an old service unit .. joining a new one .. or is it ? [19:16] kim0, so we're doing formula upgrades, not service upgrades here [19:17] ideally the semantic allows for seamless transition without bringing down units [19:17] hazmat: assuming our perfect example of a wordpres SU running, with a backend mysql SU running [19:17] ie. no events are lost, all queued hook executions and observed changes would continue with the new formula [19:17] could you explain the steps for an upgrade please [19:18] how it's planned to work rather [19:18] so say we've got three units of a wordpress service called myblog running [19:18] * kim0 nods [19:18] i've got a new formula which enables some cool plugin in wordpress [19:19] to upgrade the service, i'd run ensemble upgrade-formula --repository=$REPO_DIR myblog [19:19] That will look into the repo directory for a formula matching the formula of myblog, verify its a newer revision, upload and it mark the units for upgrade. [19:20] On the unit side, they'll detect they need upgrades, download the formula, ... and most importantly, execute the upgrade-hook from the new formula before any other hooks from the new formula are executed. [19:21] As an additional restriction, to ease authorship of a upgrade hooks, we'll only be upgrading service units that are currently running. [19:21] aha .. a new hook [19:21] so if the service unit had failed to install or start correctly, it won't be upgradeable [19:22] otherwise it becomes a very open-ended question of what the upgrade hook needs to consider [19:22] if they can't make any assumptions about the environment their running in [19:22] Question: Is there some "good-bye" hook for the old version of the formula? [19:23] Question: Also, does "upgrade-hook" know which version it is upgrading from ? For instance some wordpress upgrades can have (or not) DB schema changes based on the version you're coming from [19:23] kim0, there are two good-bye hooks depending on context, in the relation context, if a service is related to other services, if a service unit goes away, those units will get a -relation-depart hook execution, if the service is removed or the relation between services is removed, they'll get a -relation-broken hook execution. [19:24] in the context of a service unit, if the unit is removed or shutdown (including if the service is removed), the 'stop' hook will be executed. [19:24] hazmat: yes but do any of depart or broken get fired in case of upgrade [19:25] kim0, no.. they don't, the to mysql it looks like the service stays online the whole time [19:25] cool [19:25] wrt question2 .. what version the upgrade is starting from [19:26] is that known to upgrade hook [19:26] the mysql units don't see any changes to the state of the wordpress unit... optionally we're looking in the future of allowing an upgrade hook to iterate and modify the relations of the unit, if a hook uses that api, then the remote side would see changes [19:26] kim0, not yet, but that sounds like a good idea :-) [19:26] hehe cool [19:27] Ironically for "Wordpress" what version you're coming from matters a lot [19:27] Should I file a bug :) [19:27] kim0, i'll add it the formula upgrade bug 750483 [19:27] Launchpad bug 750483 in ensemble "unit agent needs to support upgrades" [High,In progress] https://launchpad.net/bugs/750483 [19:27] awesome [19:28] that sounds well covered [19:28] Any other points [19:29] hazmat: anything else ? [19:30] nothing else comes to mind for me that's landing this week, bcsaller is doing service-config work which is very cool, but is going to be another week, jim's doing some work on exposing/securing services using ec2 security groups. [19:30] jimbaker: Wanna talk about anything [19:30] I hope i'm not interrupting, but As far as i remember, i did read somewhere that ensemble can depend on shell scripts or puppet to apply the configuration .. [19:31] currently ensemble deploys machines such that the firewall is open [19:31] DigitalFlux, all hook are is effectively just an executable, those executable can be in any language and use any tools they'd like [19:31] DigitalFlux: Yeah it's a high level orchestration tool .. the "hooks" can be written in any language [19:31] I don't think that's in the docs, right ? http://people.canonical.com/~niemeyer/ensemble/index.html [19:31] DigitalFlux: two answers for the price of one ;) [19:31] Docs are very basic right now [19:32] DigitalFlux: Help make it better :) bzr branch lp:ensemble [19:32] we will be addressing such that the firewall is configurable by the ensemble user (ensemble expose, ensemble unexpose), with corresponding hooks (exposed, unexposed), and hook commands (open-port, close-port) [19:32] kim0, DigitalFlux: Well, even suggesting improvements would already be welcome, if you don't have the time to push the changes yourself [19:32] jimbaker: wow that's cool stuff [19:33] DigitalFlux, it's described to a certain extent in the docs. i actually copy edited some text in the formulas that make it more explicit [19:33] formulas *section* [19:33] kim0: It is indeed! Effectively an Ensemble user can just do "ensemble expose myblog" and that works [19:33] niemeyer, do the draft/specs get published on the doc site? [19:33] hazmat: Yeah [19:33] ah. there it is [19:33] What is open-port close-port ? why are those needed [19:33] the expose work is described in further detail at http://people.canonical.com/~niemeyer/ensemble/drafts/expose-services.html [19:34] * kim0 clicks [19:34] kim0, we need to tell ensemble what ports a service needs opened [19:34] and http://people.canonical.com/~niemeyer/ensemble/drafts/expose-services.html#service-exposing-for-formula-authors [19:34] there was some debate on whether to describe this in the metadata, but we decided that it was too restrictive to make it static [19:34] That's great !, i wil just have to read more about the hooks then .. [19:34] also, a service can open and close ports for a given service unit when that unit is in fact ready [19:35] kim0, there's a notion that a service isn't nesc. running on a fixed port, that over time we'll need to do some negotiation for service ports, so that we can do multi-tenant on a machine [19:35] ah that's cool as well [19:36] ie. run three units of different wordpress services on a single machine, each connected to a load balancer for the service.. even with lxc network namespace isolation, we'll be doing a nat against the host machine interface to the container network device. [19:36] does {open,close}-port operate on top of ec2 security groups ? [19:36] DigitalFlux, this copy editing is in a branch right now, see lp:~jimbaker/ensemble/new-hook-semantics-5-docs [19:36] kim0, the mechanism is generic, in future we'll be utilizing machine level firewalls, but for now it will use ec2 security groups [19:36] but such a change won't effect the formula-author or user workflow [19:37] * DigitalFlux pulling .. [19:37] this was another important reason - we wanted it provider independent [19:37] hazmat: so different backends .. i.e. in the future things like cisco firewall APIs would be useable right [19:37] without even formulas needing change! freaking awesome :) [19:37] alonswartz: How does it feel so far.. looks reasonable? Anything we can help with? [19:38] kim0: Exactly! [19:38] jimbaker: Great stuff, anything else you want to comment on [19:38] Overwhelming :) [19:38] Indeed it is [19:38] to be fair the implementation change won't effect them, but the introduction of port-negotiation/allocation might. [19:38] Only because it's the first time I suppose :) [19:39] indeed [19:39] kim0, i think that pretty much covers it for me. i have some other work to smooth out rough edges and inconsistencies, as might be expected :) [19:39] Any special handling to RPC style services .. a la NFS [19:39] I guess it's a corner case [19:40] but why not bring that up :) [19:40] kim0, we have only considered a very simple mode of port opening, but potentially we could extend it [19:40] definitely makes sense [19:40] Great stuff indeed [19:40] we do cover tcp and udp, but not how it would interact with stateful firewalls for example [19:40] jimbaker: guess ec2 security groups cover those too as well [19:41] kim0, indeed, it was an easy mapping for now :) [19:41] um .. icmp ? ipsec ? future versions right [19:41] those not having a notion of port .. might be a bit problematic [19:41] i do believe the mechanism we have in place should readily support other protocols in the future, it's not restrictive on what open-port means for example [19:42] yeah let's leave it till then [19:42] cool [19:42] niemeyer: sorry, I had to handle a quick emergency... [19:42] alonswartz: Oh, no problem [19:42] alonswartz: hey [19:42] Hey kim0 :) [19:42] alonswartz: Was just wondering if we could help with anything you might be pondering about, since you already had some background on the project previously [19:42] o/ [19:42] I didn't get to follow the whole conversation, but I do have a question [19:43] Is the project ready for outside collaboration yet? [19:43] It is! [19:43] * kim0 looks around [19:44] bzr branch lp:ensemble [19:44] alonswartz: Absolutely [19:44] alonswartz: The whole development happens in the open [19:44] alonswartz: https://lists.ubuntu.com/mailman/listinfo/Ensemble [19:44] alonswartz: Open IRC channel, mailing list, source code, etc [19:44] I have to admit I haven't looked into ensemble since my brush with it at UDS-M [19:45] alonswartz: We've come a long way.. it actually works now :-) [19:45] hehe [19:45] but it seems you have made great head way [19:45] I'll read up the docs, take it for a spin, and then I'll be more clued up as to the design, and hopefully have ideas on how I can help [19:46] bcsaller: o/ [19:46] hmm? [19:47] OT, I saw you chose to use txaws. out of curiosity, why not boto? does ensemble use async? [19:47] It's the first time [19:47] bcsaller: So you're free to comment on anything you like :) [19:47] I think I'll have more new stuff to show next week [19:47] alonswartz, we use twisted completely [19:48] (this works better with the underlying zookeeper implementation) [19:48] bcsaller: sure thing .. we're not limited to last week though for this time .. [19:48] before I ask stupid questions that are answered in the docs, I'll read them first... [19:49] Alrighty [19:49] we debated a while ago about twisted vs. eventlet/gevent.. but its done at this point. [19:49] but I can't help myself, does/will (or how does) ensemble handle scaling ? [19:49] short answer: yes! [19:49] as in add service units in runtime [19:49] alonswartz, at the moment, you can scale via adding removing units to any service in the environment [19:49] I think it already does huh [19:50] so the user needs to do it manually, right? there isn't a master ensemble server that handles it transparently... [19:50] alonswartz, again i'd refer you to our docs, many of the hooks deal with scaling up and down [19:50] there isn't any autoscaling exposed at the moment, just the ensemble command line to do it. [19:51] alonswartz, but its a bit more automated than manual.. if i have mediawiki connected to mysql, memcached, and haproxy.. adding a unit of mediawiki, auto configures it with all those other services. [19:51] fair enough, i'll read the docs and come back with better questions :) [19:52] Open topic .. any comments, questions [19:52] Thanks all! [19:52] the one thing we don't do that's part of the autoscaling is automatically determine how many service units you need. that would seem to depend on your specific stack needs [19:52] Can't believe we almost consumed the full hour [19:52] this was a blast [19:52] jimbaker: That's up for debate [19:53] but we do handle the orchestration, and that's a differentiating factor compared to say puppet [19:53] jimbaker: We haven't covered that topic yet, so let's not provide the impression we're not doing it when in fact we really have to think through the use cases. [19:53] I would want to hook auto-scaling to some performance-measuring-hook [19:53] niemeyer, agreed [19:53] kim0: Exactly [19:53] lovely [19:53] * kim0 hugs Ensemble [19:54] Thanks folks! [19:54] Thanks niemeyer jimbaker bcsaller hazmat alonswartz DigitalFlux [19:54] Meeting end [19:54] kim0, thanks for organizing this [19:54] kim0: thanks [19:54] Thank you folks .. This is really exciting stuff [19:55] Yeah, Thanks kim0 [19:55] Lovely Ubuntu communities hehe :) [19:56] autoscaling with upper limits for the resources would be AWESOME ! [19:56] ec2 limits you to 20 instances any way :P [19:56] kim0: Oh, didn't know that [19:57] you have to explictly ask them to raise that limit [19:57] But still, if that can be done based on some metrics that i specify, and machines will scale up and down .. [19:57] Well, i will have to quit my job then :-) [19:58] DigitalFlux: ec2 has some form of that [19:58] they can monitor load average and auto-scale up/down [19:58] TEH cloud :) [19:58] BUT [19:58] So what form of orchestration do you point to here ? [19:58] kim0: ec2 has a 20 instance limit? [19:58] ec2 would "scale" by launching you an instance and you're on your own [19:58] jimbaker: So what form of orchestration do you point to here ? [19:59] kim0: ofc [19:59] alonswartz: yeah afaik .. you can raise it [19:59] you'd just have to contact them [19:59] or use a different account .. [20:00] DigitalFlux: with ensemble .. you're not on your own [20:00] an instance is fired, hooks are triggered to configure it and join it to neighbour instances [20:00] kim0: good to know. I didn't know they had a limit. [20:00] kim0: That's the new slogan for the project ? :-) [20:00] alonswartz: you probably wanna check that too :) [20:01] dont depend on my leaky memory .. but I found traces of that on google .. so I'm probably not making it up [20:01] i was planning on spinning up a 100 instance cluster (for a batch job), so I'll need to look into that [20:01] hehe perfect timing [20:02] i'll report back :) [20:02] * kim0 is interested what would need 100 instances [20:03] kim0: batch build of all turnkey appliances, all target formats, all amazon regions, all... [20:04] the list goes on and on .. [20:04] a ha .. got it [20:04] one thing I disliked about ec2 .. is their biggest instance is 64G ram [20:04] add 64bit to the equation and it spirals out of control... [20:05] a friend of mine doing scientific numerical analysis needs a bigger rig .. any suggestions === hallyn_afk is now known as hallyn === dendrobates is now known as dendro-afk [20:22] DigitalFlux, sorry was in another meeting [20:23] jimbaker: no probs, i'm in another one right now :) [20:23] ensemble ensures that all hooks are fired on all service units, in the correct order [20:24] jimbaker: you mentioned orchestration [20:24] so this is what i would describe as orchestration [20:24] hmm, and puppet doesn't do something similar ? [20:24] a given service unit only needs to take care of itself, responding as its hooks fire [20:25] puppet/chef is very different. it's top down. and because there's a need to worry about the layout, it's hard to write generic recipes [20:26] yeah, there have been some efforts done to enhance this, but i guess the model is still different than ensemble [20:26] I need to work on how can i get both of the worlds i guess [20:26] * DigitalFlux needs to do some lab [20:27] DigitalFlux, it's definitely a different model. one thing i was challenged on is that puppet better accommodates a model of sysadmins actually logging into a machine and messing around [20:28] this is defended on the basis that critical fixes must be done and now ;) [20:28] i think the cloud model where we don't care about specific machines is inherently better for this [20:29] hmm, considering puppet+mcollective, wouldn't that be already solved using this combination ? [20:29] as i recall mcollective, we still have a better foundation in zookeeper, which gives us total ordered events and a consistent, durable view of the world [20:30] jimbaker: I would love to capture that in plain English in a faq page :) [20:30] but i can really talk best about ensemble [20:30] did niemeyer check mcollective as well [20:30] and how ensemble would compare [20:31] kim0, it really comes straight from a zookeeper faq, but there are some ensemble considerations that need to be mixed in === dendro-afk is now known as dendrobates [20:32]