imbrandonto do that it would have to be not just tecnical but they would need to be aware of all the reporcusions and how to debug repotly but be certain its not in vein etc00:00
imbrandonand well past what can / should be encapsulated in a first draft charm if at all00:01
SpamapSMan, I really really really want a "pretend to re-negotiate the relations" command00:01
imbrandonoh SpamapS btw, i'm at feature parity with jjc.c now on jujube00:02
SpamapSSo many times where I just need to tweak something, and run upgrade-charm ..00:02
SpamapSimbrandon: lol.. of course you are00:02
imbrandonbut i did a few thiings that i decided are a little beeter another way so i'm refactoring a little bit of it00:02
imbrandonbut even that i should be done tonight too00:02
imbrandonoh and i said that wrong tooo, i'm not at fp with the whole domain , i broke it up into parts in my head and the README but none else know that LOL ... i intended to say i'm there with all of the charm browser aspects of jjc.c00:08
imbrandonincludiing both the html output and a .json out out on the same routes if /api/v1/ is prefixed00:09
imbrandonlike /charms/precise lists all precise charms and /charms/precise/appflower is details for appflower , then /api/v1/charms/precise and same for appflower are json output and also take a optional ?callback=anything returning jsonp so CORS will work as well00:11
imbrandonwith cross origin domain calls, and later other sites could use JS to just make remote api calls for the data thet want ( or what i'm more excuted about is that juju-jitu could as well should it want to maybe it will spark something cool )00:12
SpamapShttp://ec2-54-245-5-73.us-west-2.compute.amazonaws.com/nagios3/  (user:nagiosadmin pass:ohpuipeera)00:16
SpamapSSo there's my nagios magic00:16
SpamapSstill needs to add host/service groups00:17
SpamapSand I *think* I can do some clever parent/child stuff by inspecting the relationships a bit (provides should be the parent of requires.. perhaps)00:18
negronjlSpamapS: nice re: nagios00:41
m_3SpamapS: nagios is compelling... can I use this as part of a "mature" stack for Thursday?01:15
hazmatSpamapS, there is no clean up re subs01:26
hazmatSpamapS, thats a significant feature thats needed across the board01:26
hazmatorderly cleanup and instead of parent tree kill01:26
hazmatimbrandon, incidentally i got the oks to push jc.c public01:27
hazmatSpamapS, goju should hopefully have that from the start, it was discussed  as part of the mongo storage work01:29
hazmatimbrandon, yes charms are copied on disk for each unit... pls file a bug if its of concern01:29
hazmatfor ec2 there's not much to do, it must be present on each machine01:34
hazmatfor local..01:34
hazmatSpamapS, how'd you get juju with ostack working on hpcloud?01:35
hazmatah.. auth-mode keyapir02:27
imbrandonhazmat: I had to use auth-mode: userpass ... see PM02:48
hazmatimbrandon, i'm making some progress with keypair02:48
imbrandonhazmat: nice, if you do get keypair to work later that would be cool, much less complex than currently i think for the env.y03:17
imbrandonalthough thats a set and forget thing so ...03:17
SpamapSm_3: this is pretty solid, I'll chat w/ you tomorrow to get you up to speed on deploying it. I think it should be widely consumable.07:02
hitexltHi, i am trying to deploy openstack on VM's. I have deployed all required charms, but for most machines in MAAS it shows agent-state: not started, though they are running. Here is output of juju status http://pastebin.com/vAb1mb3r Any ideas why juju is not registering running nodes?07:06
SpamapShitexlt: the agent state is running on machine 207:08
SpamapShitexlt: and machine 2's rabbitmq seems to be started actually07:09
SpamapShitexlt: can you look at the console of machine 1 and see if something failed on first boot?07:10
=== philipballew__ is now known as philballew
=== almaisan-away is now known as al-maisan
=== mrevell_ is now known as mrevell
jamespagejml, sad today?10:34
jmloh yeah10:34
jmlI just had to kill off juju processes that were chewing up all my CPU10:34
jamespagejml, local provider?10:34
jmljamespage: yeah. I pretty much never use ec210:35
mgzjml: canonistack is actually pretty reasonable currently10:36
jmlmgz: I can run precise stuff on that, right?10:36
jamespagemgz, hows the openstack provider work coming on?10:39
mgzit's is a working state, but there are a few required features left and some cleanup to do.10:40
=== al-maisan is now known as almaisan-away
hazmatmgz, i've spent some time yesterday working it across several providers (rspace, hp, cstack)and doing some cleanups. rspace is very different behavior with v2 api11:27
hazmatno floating ips11:27
hazmatauto public ip11:27
mgzright, hp have also switched to enabling the auto-assign of a public ip11:28
mgzthere's an easy enough work around (see if there's a public ip before allocating a floating ip)11:28
mgzbut that doesn't get me round the sleep unless I have a way of getting the config state out of a particular deployment11:28
hazmatmgz it would be nice to avoid the wait entirely11:28
mgzwhat I'd *really* *really* like is juju to move the public ip requirement to the ports handling11:29
hazmatmgz, there's nothing to run that code for the bootstrap node11:30
mgzit wouldn't be that hard to juju ssh to ensure the instance is accessible11:30
hazmatthat's a workaround not a solution11:30
mgzwell, it's moving the requirement of being able to talk to the machine from instantation to when it's actually needed11:31
mgzany code that currently expects to access public_ip could basically do that lookup with a check that will allocate one on demand11:33
hazmatmgz ah.. so your saying automatically instead of manually. that's reasonable.11:33
hazmatmgz, even distinguishing what's the public ip isn't well defined11:33
hazmatmgz, also for ec2, it no longer waits for the security group shutdown/delete dance, it just reuses the groups11:34
hazmaton trunk11:34
hazmatfor ostack this removes the delete dance at shutdown11:34
hazmatall the providers seem to support just killing the machine11:34
mgzthat's an improvement, is there any cleanup for the groups?11:36
mgzhazmat: do you have a branch for your changes up anywhere? and did you find my hp tweaks branch? there are a few other differences they have that need properly integrating into the provider11:41
hazmatmgz, it looked like the hp tweaks where already mainlined11:48
hazmatmgz, i'm still in progress on my changes.. but here's the crack in progress ~hazmat/juju/openstack_provider/11:48
hazmati need to run an errand, bbiab11:49
mgzright, I merged Clint's, but last week an HP guy did some more testing as they want to use juju in a demo11:49
mgzand turned up some more things11:49
hazmatmgz, rackspace has their own auth form derived from v2 but with different fields.11:49
hazmatmgz, where's that branch?11:49
mgzsee lp:~gz/juju/openstack_provider_hp_tweak11:49
hazmatcool, i'll check in when i get back11:50
mgzI'll take a look at your current state too on the rackspace auth things.11:50
hazmatmgz, i still have to parameterize all the security group work there, since all of those apis are 404s on rspace11:51
mgzhm, all of them? they don't support security groups at all?11:53
hazmatmgz, at the moment no11:56
hazmatnor floating ips11:56
hazmatmgz none of those apis is in v211:57
mgzokay, those auth changes make sense, shoule be able to integrate that nicely11:57
hazmatthere in quantum11:57
mgzwell, there are various things moving from nova to other projects11:57
mgzin not very backwards compatible ways11:57
mgzprovided detecting can be done sensibly, doing both shouldn't be too hard, just a little annoying11:58
=== almaisan-away is now known as al-maisan
=== zyga is now known as zyga-afk
hazmatmgz, there's some minor capability though inconsistent to detect via service catalog version info13:35
hazmatrspace actually publishes multiple compute endpoints of different versions13:35
mgzhm, probably need to change the endpoint picking logic again then13:35
surgemcgeeHmmm, so the quotes are interpeded as a string now in the yaml. Is there a different type which will evaluate the quotes?13:55
mgzyou mean will include the quote marks in the string?13:56
surgemcgeeYes, are we just using the pipe now?13:57
surgemcgeeWell, I mean, will evaluate the quotes as a special character (escape sequence)14:00
surgemcgeeWith the default type of string, the quotes are included. I think this was changed.14:00
surgemcgeeAlso, referring to the single quote14:01
mgzhm, was about to say it works for me, but turns out I don't use any quotes in my config14:03
surgemcgeeI am trial/error'ing it now. I will post my results.14:06
smosercan someone take a look at http://askubuntu.com/questions/162255/does-juju-really-need-two-nodes ?14:25
mgzI looked upon it an despaired14:26
mgzhe wants to do an openstack deployment with only one box, the maas demo used 9.14:26
hazmatjitsu deploy-to can lesson it14:27
smoserit seems to me that the JuJu bootstrap node failed to come up.14:27
hazmatsmoser, agreed14:28
hazmatit looks like the node is up but not running zk14:29
mgzI've had similar symptoms from cloud-init steps failing14:30
mgzso, sshing to the box and looking at the logs is what I'd do next14:30
mgzI just don't see it's really helping his end goal14:31
smosermgz, cloud init does not fail.14:31
smoseri'm done talking to you14:31
mgzsorry, 'apt-get fails and cloud-init plows on regardless' :P14:31
smosermgz, yeah, that is what i woudl have suggested, but i dont know which logs to get.14:31
smosermgz, thats better.14:32
smoserhazmat, or mgz, could you request posting of some logs that might give information?14:32
mgz/var/log/cloud-init-output.log and the ones under /var/log/juju generally14:32
=== al-maisan is now known as almaisan-away
surgemcgeeIs it appropiate for me to start bug reports on juju? E.g. Boolean in config uses python interpratation with capital first letter --> "True". But bash uses lowercase --> "true".14:36
surgemcgeeAhhh crap. Well, if anyone was curious about the quote thing, careful where you put you delimiter settings. IFS="$(echo -e "\n\r")" is not normal for commands.15:01
hazmatsurgemcgee, yes re bug reports, that particular one is addressed by some recent changes, but requires the charm to explicitly specify format 215:04
=== zyga-afk is now known as zyga
hazmatwhich makes hook cli api output normalize to yaml15:05
SpamapShazmat: kind of weird though... format 2 isn't supported by precise...15:13
SpamapShazmat: I suggest that people just use --format=json15:13
hazmatmgz it looks like the fixes re hp tweaks are to support an older form of sec groups extant there15:50
hazmatit seems like the version identifiers here have no meaning re 1.1 nova compute, as they entail quite a bit differences between extant entities15:51
=== salgado is now known as salgado-lunch
marcoceppiHas openstack support landed yet?15:52
hazmatmarcoceppi, almost, just trying to kick the tires across a few providers15:53
marcoceppiAwesome, good to hear15:53
mgzright, the api versioning isn't... er... robust15:53
hazmatconsidering the size of the branch, i'm sorely tempted to have it just land, and fix per provider incrementally..15:54
hazmatmgz, does the server respond back with version headers in any meaningful way?15:54
hazmatmgz, the keystone svc catalog doesn't seem to mandate versions15:54
hazmatwhich makes it hard to utilize that15:55
hazmatmgz, visualizing what v2 compatibility is going to look like with a provider that supports separate network, compute, volume api.. while keeping compatibility with 1.1 feels quite different15:56
mgznope, though versions should generally be in the catalog... main issue seems to be they're largely ignored15:57
hazmatthe encapsulation needs to flow from abstractions inspecting the svc catalog, but i digress.. its probably better to fix up some of the window dressing15:57
mgzso, stuff get broken and no one notices15:57
hazmatmgz, ignored by the current code.. or by the svc provider?15:58
hazmatwell the former is true.. just curious about the latter15:58
* hazmat openstack's api innovation makes me sad15:58
hazmathmm.. s/me/15:58
mgzfor instance, nova-client handles 1.1 and 2 with the same code under v1_1 module16:00
mgzthere's pushback now it's starting to get use, but the history so far is messy16:00
hazmatmgz, thats going to fail as soon as people start deploying quantum.. its  already a fail using the cli against v2 impl16:06
hazmatwith v2 we're going to have construct our facade if we want to support both nova-network and quantum and dispatch by version id16:07
hazmati'm not as concerned about that at the moment though re getting this into trunk, since those are extant16:07
mgzright, and the volumes code has the reverse problem, I have to use an old version of python-novaclient because the api was moved out of 'compute' into 'volume' and current deployments don't support the new way16:08
mgzI think these problems aren't too hard from the juju side though, given the api usage it pretty light and we should alwayse be able to pick a method that works16:09
hazmatsame problem fundamentally.. two versions address the same core api in different ways16:09
hazmatmgz, is volume going to be exposed at its own endpoint in the svc catalog as well?16:09
mgzand in fact is, but... canonistack gives an unresolvable internal url, and hp gives an endpoint which uses the old naming :)16:10
hazmatargh.. snowflakes16:10
SpamapSdoesn't keystone hand back a map.. "volume -> this url, network -> this url" ?16:11
hazmatSpamapS, in future in theory yes16:11
hazmati believe rspace has both of those in private beta16:11
hazmatrspace is on v2 of the api.. hp is effectively on a forked version of the 1.1 api from diablo time frame which is different than any  other impl.. canoistack is on v1.1 but doesn't advertise versions for its svc catalog endpoints (which keystone hands back).16:12
SpamapSI'm surprised HP hasn't spun up an Essex region16:18
hazmatmgz, SpamapS have you guys noticed any oddity about the bootstrap node getting a stringified id in hpcloud stored in zk, and thus not useable for queries to hpcloud about it?16:23
SpamapShazmat: yes16:24
SpamapShazmat: the bootstrap node just is wonky w/ hpcloud16:24
SpamapSI haven't looked into it very much16:24
hazmatthe provider needs to materialize it back to an int i think before passing it off to the api serialization16:24
mgzhazmat: yeah, that's an issue, I hacked around it in the hp tweak branch I linked earlier16:24
hazmatmgz, it looked like the branch was trying to serialize all the ids to strs16:25
hazmatoh.. this the curl against the main id16:26
hazmater bootstrap id against fstorage16:26
mgzthere's a neater way of doing it in StatusCommand._process_machine but I was trying to avoid changes to other bits of juju16:26
hazmatmgz, i'm thinking just sniff the type in the provider16:27
_mup_Bug #966577 was filed: add explicit egress 'owner' rule on non-bootstrapping nodes to require root access to zookeeper <rls-p-tracking> <juju:New> <juju (Ubuntu):Triaged> <juju (Ubuntu Precise):Triaged> < https://launchpad.net/bugs/966577 >16:27
hazmatyup.. that does the trick16:29
SpamapShazmat: ^ the bug above is my hack for preventing non-root users from poking at ZK..16:34
SpamapShazmat: No diff yet.. just the idea. But I think its worthwhile even after ACL's land16:34
SpamapShazmat: which, btw, are we going to land ACLs ?16:34
SpamapShazmat: or is libzookeeper still an unknown weird fail-generator ?16:34
imbrandonzk dosent handle acl's at all? it expects the client to 100% ?16:38
imbrandonhazmat: btw i found if you use local placement on the bootstrap node the ssh 0 is still failwhale but you can juju ssh service/0 to it fine and juju status us the same at the top but its correct down at the server part with ip and all16:40
imbrandonhazmat: personally i've just been doing juju ( the charm ) deployment or something small just for convience but if that help with the debug path in the code maybe heh16:41
imbrandonhpcloud ^^ btw16:41
negronjl'morning all16:43
imbrandonugh, where are unity's global hotkeys stored anyone know right off to save me 15 minutes of looking , please dont say gconfeditor ...16:44
imbrandonheya negronjl16:44
negronjlhi imbrandon16:44
hazmatimbrandon, just fixed that fwiw16:44
imbrandonoh sweet, i was aobut to get lunch bvut after mind if i snag ur branch :)16:44
hazmati guess clint rolled out16:44
imbrandonoh cool cool, i'll catch up when i get back i've been head down in code and only half paying attn to irc today16:45
imbrandonok back in a bit, foooooood16:45
hazmatthe ssh fingerprint problem is more pronounced with ostack providers i'm noticing16:47
hazmati guess thats the habit of reusing the floating ip16:47
hazmatto a different machine16:47
hazmatSpamapS, yes stunnel si worthwhile16:55
hazmatSpamapS, i haven't dug into libzk issues16:56
SpamapSstunnel would be a bigger deal16:56
SpamapSjust using the iptables rule we at least limit all traffic to the bootstrap node to root16:56
SpamapSimbrandon: not sure what happened while I was split off. ZK has ACL's, but we don't use the (yet)16:57
hazmatimbrandon, just hold down 'windows'/cmd key for a while and the shortcuts appear16:58
hazmata listing that is16:59
SpamapSooo somebody is learning unity17:00
SpamapShazmat: hey, the charm versions in import/export are pretty bogus..17:01
hazmatSpamapS, ?17:01
hazmatSpamapS, how so?17:01
hazmatSpamapS, pastebin pls17:01
SpamapShazmat: with local charms at least.. I no longer have "version25" of mysql.. I"m on 31.. but it deployed 31 as local:mysql-2517:01
SpamapShazmat: causes no harm, but its confusing17:02
hazmatSpamapS, ah.. i actually added explicit support for that17:02
hazmatSpamapS, otherwise it would fail to deploy at all if the version in a local repo wasn't an exact match17:02
SpamapSIt works fantastically, but its a bit weird. ;)17:02
SpamapShazmat: I'd rather see the actual versions (and warnings). But.. we can do that in the go version, right? ;)17:03
hazmatSpamapS, basically for local charms, the version id isn't a hard requirement, it will look for the specified version and then fallback to namematch17:03
marcoceppiInteresting question, can I change the region during deployment? to have nodes across regions?17:03
hazmatmarcoceppi, no.. that's not supported in constraints atm.. twas a long discussion..17:03
hazmatwell.. you can specify az17:03
marcoceppihazmat: thanks, I figured it would be pretty involved17:03
hazmatfor a constraint at a service level17:03
hazmatbut if you want a multi region service, the result is you need a service in each that are connected17:04
hazmatthat way if you need to add capacity (or a node disappears) replacing it is explicit17:04
SpamapSI'd love to have a service constraint of ec2-availability-zone=balanced17:04
SpamapSmeaning, put these in as many different zones as possible17:04
* hazmat nods17:05
hazmatwe went back and forth on that one a few times at uds orlando17:05
hazmatnot treating individual nodes as special was the concern for supporting it explicitly within a service17:05
hazmatif  i have three nodes in one az and one node in another az, and it dies, what does add-unit do to correct properly17:06
hazmata juju policy level  identifier for zones would work though17:06
hazmatbut at the moment the recommendation is to have separate services in each zone and relate them17:07
dpb___is it possible to reboot an instance inside an install hook (for example)?17:08
hazmatdpb___, not at the moment, the address change doesn't properly propogate if there is one, if the address is stable then yes17:09
=== salgado-lunch is now known as salgado
dpb___hazmat: ok, you mean the ip address?17:10
hazmatdpb___, yes, both public and private would have to be stable for it to work reliabily17:10
dpb___hazmat: ah, ok.  I guess the private one can change between boots on aws?17:11
hazmatdpb___, both change on reboot in aws17:11
dpb___hazmat: ok, thx, I did not know that.17:11
hazmatdpb___, there's an unloved branch for it, if your interested, that updates address at boot, and propogates to relations.. with it doing things like suspend/restore an ec2 env become possible17:12
dpb___hazmat: well, first let me see if I can do some magic to make the reboot uncessary.  that would be speed up the process anyway.17:13
hazmatdpb___, absolutely that would be the best way17:13
dpb___hazmat: thx17:13
SpamapSdpb___: I would defer reboot to config-changed17:13
dpb___SpamapS: How do you mean?  you mean that config_changed can handle a reboot in the middle?17:16
SpamapSdpb___: config-changed is what will be run after the reboot17:18
jcastroSpamapS: heya18:10
jcastroSpamapS: mims is on review this week but I didn't account for oscon18:10
jcastroSpamapS: if it's possible, what do you think about reviewing the queue this week and have mims grab next week, when it was your turn to go.18:10
jcastrobasically, wanna swap?18:10
SpamapSYeah I'm going to take his shift18:11
SpamapSjcastro: I saw that coming yesterday. :)18:11
jcastrothanks dude18:11
SpamapSjcastro: which is unfortunate because I was hoping to have Mims review my massive Nagios refactor :)18:11
SpamapSbut he probably will anyway as he might use it in his demo :)18:11
jcastroalso we have decided we want you writing more subordinates. :p18:13
SpamapSME TOO :)18:18
SpamapSwell I already took over james_w's nrpe18:18
SpamapSjcastro: http://ec2-54-245-5-73.us-west-2.compute.amazonaws.com/nagios3/18:18
SpamapSjcastro: user:nagiosadmin pass:ohpuipeera18:19
SpamapSand no I don't mind if you all pound the crap out of it ;)18:19
imbrandonhazmat: yea i knew that actually, cuz its the ones i keep hitting ( cmd ) and  so i want to swap enmase` cmd<->ctl so my hotkeys for osx and ubuntu should line up almost identically, as it is i keep bringing up the hud every time i goto save a file or copy/paste etc etc, finaly had enough and gonna find it even if i got to recompiler unity :)18:49
jcastroSpamapS: can you leave that instance up for a few hours?18:51
jcastrothat'd be nice to show during our talk18:51
SpamapSjcastro: when are you guys speaking?18:53
SpamapSjcastro: I can give you an export to recreate it. :)18:53
jcastroin 1.5 hours.18:53
jcastrowell, we didn't want to go too deep into it, actually, I'll just screenshot it18:53
SpamapSjcastro: are you guys demo'ing at all or just talking?18:53
jcastrodemoing a bunch of stuff18:53
jcastroso we didn't want to add another demo18:54
SpamapShah yeah ok18:54
SpamapSjcastro: let me add thinkup back in and then you can screenshot it18:54
jcastrook, lmk when18:55
SpamapSjcastro: its there now, just PENDING18:55
SpamapSjcastro: more fun if its all green :)18:55
jcastroI like it with the pending though18:57
jcastroshows that it all just works, and you don't care, you set the relationship, it WILL work, etc.18:57
jcastroexcept when it doesn't, but whatever. :p18:58
SpamapSjcastro: got a shot yet? I want to refresh one of the relations18:58
jcastrowe're good18:58
SpamapSjcastro: Ok re do your snap of services now. MUCH more compelling :)19:06
SpamapSjcastro: and I can give you some reds if you want. :)19:07
jcastrook one sec19:08
jcastrothanks dude, got it19:09
imbrandonSpamapS: haha ok know what i just realized, ok so like none of this info is new to me ( or you either i dont think ) but it JUST all clicked a few seconds ago19:11
SpamapSimbrandon: you forgot one thing. You forgot, to hookup, the doll19:11
imbrandonso ... Google had another announcement like 2 days ago that they just finished laying the 200th mile of FttH in Kansas City19:12
imbrandondoll ?19:12
imbrandonso ok the fiber rollout is almost complete, its double subsidised once by the city/county to get ppl on it and once by google, so total cost to residents at rollout is $30 per month out of pocket and thats for 100GB SYNC connection , not just to the street but all the way to the net AND 16 static IP .....19:14
imbrandonon the residential connection ... ok so add that ... pluss i start rounding up some HP blades and Sun NAS boxes  heheh19:14
imbrandontoss open stack on it , and start a lil mini Linode in my basemnent19:15
imbrandonlikely rivaling if not exceding the quality of many mom and pop operations round the country19:15
imbrandonanyhow i likely will do all that cept the resell part, i dont want that headache :)19:16
SpamapSimbrandon: re doll.. http://www.imdb.com/title/tt0090305/quotes .. look for the word 'doll'19:16
SpamapSI'm like the only one who remembers that quote.. but I use it in my head constantly :p19:16
imbrandonheh, thats ok i had the audio playing ( very loud too ) for the samuel jakson seen at the start of pulp fiction when my brother came by and dident reconise it19:18
imbrandoni told him i disowned him19:18
SpamapSimbrandon: your power setup will be crap w/o a diesel generator, but otherwise yeah you will have quite a bit of advantage. :)19:22
SpamapSperhaps KC will become us-east-2 ;)19:22
imbrandonhrm yea but that would not be too bad :)19:22
imbrandoncant wait for real, i'm just happy as hell that not only its got 16ips static standard but that are quoted to say something very very close to "KC is the pilot city here is some cool shit now go make the next gen web"19:24
imbrandonPLUS sync both ways :)19:24
imbrandonyou should see all the new Tech startup that are already downtown that are newly funded and starting to setup shop now in prep19:25
imbrandongonna be a wild ride the first year i bet19:25
imbrandonSpamapS: ok, i'm on Time Warner now ... kc.rr.com , and check this pic out http://www.businessinsider.com/google-fiber-speed-2011-819:27
hazmatit looks like hpcloud is blocking private traffic by default now19:27
imbrandonthey block all by default i thought19:27
imbrandonand you had to add pub and priv to the zones19:28
imbrandoni forgot what they call their security groups19:28
hazmati couldn't deploy services because the agents couldn't connect back to the bootsstrap node19:28
hazmatsecurity groups..19:28
imbrandonright, juju dont do that on diablo19:28
imbrandonSpamapS: had me do it by hand so i just set it to
imbrandonand never destroyed it19:29
SpamapSOh right I totally forgot about that19:29
imbrandoni did too19:29
imbrandonuntill hazmat said it19:29
imbrandonsince destroy never took the groups away19:30
imbrandonand i used the same name19:30
hazmatthis is going to break most of the charms unless we netmask all internal port traffic19:30
imbrandonto redeploy , never had to mess with it again19:30
hazmatto the env sec group19:30
hazmati had the same issue on azure although lacking any notion of a private network there19:31
imbrandonright , but you can only do that by name and not ip with the python-novaclient :(19:31
SpamapShazmat: there's no concept of private traffic being allowed between security group members?19:31
imbrandonSpamapS: they is but its off and docs say its not in the webui19:31
hazmatSpamapS, there should be but it was pretty broken on diablo19:31
* hazmat grumbles about snowflakes19:32
imbrandonsnow ? its over 90f here19:32
imbrandonno effin way19:32
hazmatimbrandon, just referencing the individual snowflake implementations of openstack ;-) its a 100+ here19:33
SpamapSahh diablo seems to be just making everything ridiculous19:33
imbrandonohhh ohh ok, i've interntionally stayed clear of the OS guts for now, until it evens out a little19:33
imbrandonand i dont waste a ton of time on something that will be bad before i really "get" it19:34
hazmatSpamapS, the differences between essexes is just as bad.. ie the delta for rackspace and canonistack is huge19:35
imbrandoni figured by the time RAX and that other one rolls em public will be a good time to hunker down and really "get" all the bits that i would need19:35
SpamapShm, a wise man once told me that the browser wars were the only thing that saved the web19:35
imbrandonto know about , or maybe not but i still like to understand the full stack eventually, kinda sucks lately intentionally staying out of the juju and OS code just for that19:36
SpamapSperhaps we're right at the point where netscape has implemented a billion little extensions, and IE has done the same...19:36
imbrandoni dunno about saved it, but they sure as hell are what drove it and still do19:36
imbrandonSpamapS: we are, even software devlopemnt as a whole is an insanely young disapline let along generations inside of that19:37
SpamapSimbrandon: the theory being that while HTTP RFC's and w3c specs *helped*.. what really kept things honest was users demanding the Netscape/Firefox/etc. interoperate with sites that blindly used IE stuff..19:37
SpamapSTho I could make a compelling argument that what really put a nail in the IE-special-snowflake was the iphone19:37
imbrandonwho's got money on hp will give in to the OSapi and join the ranks and goog will be the wild card like IE19:38
imbrandonwith "alomst" but "better" stuff :)19:38
SpamapSsubordinates should have a guarantee that their primary relations will have fired before any others19:39
imbrandonnah, it was way before the iphone, thats just when everyone realzed it but by that time its tooo late, itunes is what sealed everything19:39
SpamapSnrpe killing me here by relating to nagios before it can talk to its local service19:39
imbrandonSpamapS: no, bad juju , dont force serialization, lets make the hooks smarter to deal with async19:39
imbrandonyea iu know easier said than done19:39
SpamapSimbrandon: I do that19:39
SpamapSbut.. we need a 'defer me until later' to make that easy19:40
SpamapSotherwise every hook ends up calling every other hook19:40
imbrandoni know, you do, it was really me just "going by the book" but in reality your right19:40
imbrandonheh , have you seen my hooks, they do call every other one ;)19:40
imbrandonfor realz19:41
SpamapSyeah I'm doing that too and its just making me sick how crap thats going to be for some use cases19:42
imbrandonso i want to charm http://c9.io ... but its got a "magic" build system i dont under stand and cant make work on 12.04 but same git checkout works on OSX ... got time / wanna help on it :) heh19:42
SpamapSlike nagios.. where I fully expect thousands of units related19:42
imbrandonwhat thousands ? maybe on the top 3% but the vast majority of deployments will be 20 or less i bet, Penton only had about 10019:43
SpamapSthere's a really big network effect though19:43
SpamapSso even 100 units will be painful to do 4 times every time 1 comes and goes19:44
imbrandonyea but having not looked at it at all i know that the "nagios" way of dealing with that is not the typical setup, we ( read you , i aint dont shit but bitch ) might be rtying to bend nagios into a traditional scale out form factor its not gonna work at ... this is problably going to be the most complex charm i bet , including the OS ones19:45
imbrandonnah that shouldent be19:46
imbrandonthose checks are cheap19:46
imbrandonthink about how many hit apache a second19:46
imbrandonit just SEEMS like it would19:46
SpamapSits the constant chatter back and forth between juju and the hooks and execing relation-get for every relation-list unit for every relation-id relation19:46
imbrandonthat and thats one reason they should not all report to the one19:47
SpamapSits not about *nagios*19:47
SpamapSits about the charm eating up all the CPU cycles just dealing with the churn19:47
SpamapSI can scale out nagios really easily19:47
SpamapSseveral ways19:47
imbrandonoh well if the charm cant scale thats not nagios's fault19:47
SpamapSmy point exactly19:48
imbrandonthen we need to look at mq19:48
imbrandonbut i thought zk did that for us19:48
SpamapSits not that either19:48
SpamapSits calling all the hooks all the time19:48
imbrandonsure you can slow the queue back19:48
SpamapSnagios is pretty critical.. I'd argue slowing the queue is a major fail19:48
imbrandonright , so how do we make the hooks cheaper ...19:49
SpamapSif I have 60 minute guaranteed response time, getting the check 5 minutes late could cost me money19:49
SpamapSwrite them in go!19:49
imbrandonnagios will slows its queue too, i'm talking ms not minutes19:49
imbrandonjust enought to give the cpu time to slice19:49
imbrandonok i do see what your saying now tho, and actually19:50
imbrandonthinking aobut it 30 whole seconds heheh19:50
imbrandonits a design problem19:50
imbrandonhooks need to be fat19:50
imbrandonbut in that cane they cant19:50
SpamapSIts much simpler to write hooks that just regenerate everything every time19:51
imbrandonwell its not onlythat19:51
SpamapSBUT that will cost a lot of efficiency19:51
imbrandonyou would need to say ok, you only get 250ms of runtime each fire19:51
SpamapSand juju has an event based model that should allow us not to do that19:51
imbrandondo what you need19:51
imbrandonbut then19:51
imbrandonyou cant garentee19:51
imbrandonso yea, its like conflicting goals19:51
SpamapSI don't think we can guarantee..19:51
marcoceppiIIRC you can do "push" notifications to Nagios. So nagios doesn't have to poll every time19:52
imbrandonbut ...19:52
imbrandonyea we HAVE to19:52
SpamapSbut a best effort would be to try and keep hook execution down19:52
imbrandonyok SpamapS i got the solution19:52
SpamapSmarcoceppi: indeed, thats a better model I think19:52
imbrandonconfig mangement, stop going it in bash19:52
SpamapSbut.. there are some things you can't push.. like making sure the thing is actually reachable19:52
SpamapSimbrandon: mine are in python :)19:52
imbrandonlet the bash fire async off to the nagios boz anddo it19:52
SpamapSat least, a huge portion19:52
imbrandonsure, but you ssee what i mean19:53
SpamapSbash is for the simple stuff19:53
marcoceppiSpamapS: right, we had a similar problem at InMotion hosting. Monitoring 2,500 services in Nagios19:53
imbrandonright i was telling him about that at GSI was the same19:54
imbrandonbut its really that 2500 hooks fire not19:54
SpamapSmarcoceppi: mod_gearman for nagios can help with scaling out the polling for things that still have to be polled19:54
imbrandonthe nagios19:54
SpamapSits not 2500 hooks, lets be clear19:54
SpamapSlets say you have 100 units total19:54
SpamapSall of which are related to nagios19:54
imbrandoni was using marcoceppi example silly19:54
imbrandonbut the checks arent a problem19:55
SpamapSand you add or remove at least one every 3 minutes...19:55
imbrandonits that python hook on the juju relations that is the bottle neck19:55
SpamapSdeparted, joined, and changed all then *must* finish within 3 minutes, and cannot eat all of the CPU for that 3 minutes19:55
imbrandonok wait a min, we/i'm going in circles, why are the checks and the hooks coupled , i was assuming they wernt19:56
imbrandonuntill just now19:56
SpamapSbecause the checks run on the same box as the hooks19:56
SpamapSwhere else would they run? another box proxied to?19:57
imbrandonyea only 319:57
imbrandonnot all of tghenm19:57
imbrandonbut ... ok let me find a way to get what i'm thinking right the first time19:57
SpamapSSo my point is that the joins, changes, departs, brokens, have to be moderately efficient19:57
imbrandonyou got nagios svr19:57
SpamapSbut if they loop over 100 units running relation-* 5 times for each unit..19:58
imbrandonthat solved nothing really tho19:58
SpamapSthats not going to take 3 minutes, but it might take 30 seconds. And thats 30 seconds of CPU backlog19:58
imbrandonbecause their is nothing stopping another charm form beong ineffecianrt19:58
imbrandonand yea if we're talking the main server running 300 check every few minutes then i think we might be oiptimizing before there is an issue, each nagios check should only take milizeond or 2 so 500 to 1000 milizeonds20:00
imbrandonthen again on the callbackl20:00
imbrandonso 2 seocnds max at 100 services20:00
SpamapSthe hooks alone are the problem20:00
imbrandonoh jesus i just asked why you switched on my and they was coupled above20:01
SpamapSThey're on the same box, but they're not the problem20:01
imbrandonso the hook runs 3 times ... and the checks run 300 to 50020:01
SpamapSthe problem I'm facing is keeping the nagios changed/departed/broken hooks efficient, thats all.20:01
imbrandonthats still very cheap20:01
imbrandonSpamapS: hadcop did this20:01
imbrandonif its just the hooks then its the same problem20:02
imbrandonthe 3000 nodes or /we20:02
SpamapSwell anyway, I'm already seeing the hooks take 3 - 5 seconds with 7 units20:03
SpamapSspending their time waiting on relation-*20:03
imbrandonohhh and wait, for real, cant you put the join/depart ips and stuff your redoing the config for optionally into a mysql/pgsql20:03
SpamapSor just a sqlite :)20:04
SpamapSyes I do some of that20:04
imbrandonSpamapS: that could be zk + python spinup time too20:04
imbrandonabout half of the 3 to 720:04
SpamapSREST API, I can has, plrease20:04
imbrandonmaybe only ship the .pyc files in the /hooks20:04
=== zyga is now known as zyga-afk
SpamapSimbrandon: the pyc's for juju.* are already compiled20:05
imbrandonSpamapS: sure http://jujube.websitedevops.com/api/v1/20:05
imbrandonand i was saying the hooks not juju20:05
imbrandonyour hooks are in python20:05
SpamapSthe hooks aren't really slow20:05
SpamapSmeasuring them w/ strace -c ..20:06
SpamapSwait() leads the pack20:07
imbrandonmarcoceppi: oh btw i did get a full CharmStore API ReST interface for json and jsonp done20:07
marcoceppiimbrandon: cool, saw the link - got a 40420:07
imbrandonjust need to push it up but i need to split out some commits, i'll email ya with the details20:07
imbrandonthat route plus whatever the normal url is will give ya json , and add ?callback=your_callback20:08
imbrandonwill give ya jsonp20:08
imbrandonbut like i said there are some other little stuff to , i got the readme docs about 3/4 i should be able to get it out for ppl to poke sometime tonight20:08
imbrandoncuz i added a php5-yaml dep as well too and some other things20:09
imbrandonthat would be a PITA if you just had to debug why its broke if missing :)20:09
imbrandonSpamapS: what would be calling wait() disk io ?20:10
imbrandonthat blocks i assume20:10
imbrandonunless its just a thrad wait20:10
imbrandonor something20:10
SpamapSimbrandon: wait() is waiting on forked processes20:11
SpamapSimbrandon: as in, relation-*20:11
imbrandonoh ... so yea it is the scripts then waiting on the pid to finish20:11
imbrandonerr parent waiting on the pid of the scipt etc20:12
SpamapSI could, as you say, keep all the relation data cached locally20:12
SpamapSand only update it when the hook fires20:12
SpamapSperhaps juju should do that20:12
imbrandonso it should be doing something in the script then durring that but its child ... hrm dtrace ?20:12
imbrandonSpamapS: something this complex i think that might be the most correct way without getting it into productions some and looking again from 30kft20:13
SpamapSimbrandon: the relation-*'s are largely stuck waiting for ZK20:14
SpamapSbecause the way we interrogate ZK is I think a bit verbose20:14
imbrandone.g. might comes up with something not occuring to you the last week or whatever after you see it live some afternoon :)20:14
imbrandonSpamapS: imho zk is very slow but i thought that was just the java being slow20:15
imbrandoni was hoping that we'd move to mongo :)20:15
SpamapSmongo will make queries faster20:15
SpamapSbut I am not convinced we will be able to observe it as easily20:15
imbrandonwe use zk like mongos's gridfs implmentation20:15
imbrandonso it would be identical20:15
imbrandoninfact moreso because distributed mongodb is MUCH easier20:16
imbrandonand this dist gridfs20:16
imbrandona part could be on all nodes reoplication to all the others20:17
imbrandonwith like 3 hours of design20:17
SpamapShm, sounds somewhat like my 0mq idea20:17
SpamapS(have relation data live on nodes that own it, not in the central database)20:17
SpamapSI'm about to say screw it and just regenerate the whole config every time..20:18
=== medberry is now known as med_
imbrandonand monogo has driveers and gridfs interfaces for everything and its fast and it has auth built in ( something zk also lacks i think )20:18
SpamapStoo many weird cases where I miss one thing and then there's no way to ask juju to re-fire a hook and regenerate the configs20:18
imbrandonheh yea , working is better than correct , we're going to be going back and fixing alot of these charms20:18
imbrandontook me a long time to actually come to grips with that being OK and we're also still so stringent on other things20:19
SpamapSI'm also doing something extremely weird.. maybe even wrong.. by relating the primary *and* nrpe to nagios20:19
SpamapSI think I can actually simplify by just relating nrpe to nagios20:19
SpamapSbut ugh20:19
SpamapSI've already spent so much damn time on this20:19
* SpamapS goes to eat and think20:19
imbrandonheheh iterate man iterate, do it wrong once or 40 times20:20
imbrandonno one is gonna get this stuff  right on the first go and definately not the ones doing it first .... :)20:20
imbrandonbut if it works but is just slow at 100 nodes, thats ok :)20:21
imbrandonfo now20:21
* imbrandon says that like any of the above you did not already know20:22
imbrandonheh ugh ok i need to ... there was something i neeeded to do before getting back to that charm ....20:22
imbrandonugh, memory of a dog and the attn span of a gnat ... fml20:23
imbrandonoh hazmat btw , dug this out for ya to poke at later just cuz ... http://caucho.com/products/whitepapers/quercus.pdf20:25
imbrandonbut hazmat you konw when i was grabbing that tho, i ran into this and about seriously fell on the floor almost out of breath ... http://morepypy.blogspot.com/ SpamapS marcoceppi  m_3 jcastro , yall check that too20:26
imbrandonthe second one20:26
ejathi .. sorry to interrupt .. just want to ask about this email .. is it applicable to all ? https://lists.launchpad.net/lubuntu-qa/msg00787.html20:41
imbrandonejat: its already began but you can get in touch with jcastro as he is the liason for that promo iirc20:43
ejatimbrandon : thanks20:43
imbrandonejat: see http://wiki.ubuntu.com/HPCloud iirc ( typed from mem, may be off )20:44
ejatDEADLINE FOR THE FIRST ROUND OF APPLICATIONS IS 3 JUNE - Sorry this is so short notice, we'll be doing applications in batches. :( i just notice today :(20:47
dpb___Is there a hook that is called when a service is destroyed (destroy-service)20:51
dpb___I guess maybe I shouldn't be calling destroy-service, but I'm not sure.20:53
ejatimbrandon : is this the latest from askubuntu http://goo.gl/a0mMX20:55
ejatjuju with hpcloud or now already supported ?20:55
imbrandondefined supported ? hehe its somewhat functioning with some known issues but there is a spcific LP branch that can be used and myself , hazmat , SpamapS , mgz and i'm sure a few others have been trying it out and some fixes have landed but its no where near offical yet20:59
imbrandonlet me find you the url ...21:00
imbrandonejat: here is the banch lp:~gz/juju/openstack_provider21:05
imbrandonSpamapS: thats not back in trunk yet correct ? ^21:05
SpamapSdpb___: no there's no cleanup currently.. you just have to terminate the machines21:08
SpamapSimbrandon: correct, but hazmat is working on it21:08
ejatimbrandon : thanks21:10
dpb___SpamapS: thx21:24
SpamapSThis bit where subs never depart or break really screws me. :-/21:31
SpamapShazmat: ^^ is that bug documented somewhere?21:32
imbrandonthe hour of irclogs to convince me of it ? lol21:32
SpamapSNo this even screws me if I do it the easy way21:33
SpamapSbasically I'm trying to simplify21:33
SpamapSI should be able to relate nagios<->nrpe<->*  and have * get monitored21:34
imbrandonyea i was trying to be a smrt az21:34
SpamapSbut if nrpe units never depart ..21:34
imbrandonno workie tho21:34
SpamapSthen I don't know when to tell nagios to regen the configs21:34
imbrandonconfig-change fire after when it shold have ?21:34
SpamapSSo make the admin poke it?21:34
SpamapSthat sounds like "the suck"21:35
imbrandonyea but it sounds like that or delay until a patch is pushed21:35
SpamapSthere's no patch for this IIRC21:35
SpamapSits fundamentally broken21:35
SpamapSIMO subordinates are kind of just clunky and should be rethought21:35
SpamapSnice first try..21:35
imbrandonahh ok i was thinking it was still lot further out21:35
SpamapSbut I think they should just be regular services like anything else, just directed to spin up inside the container21:36
SpamapSmaking them special is making them broken21:36
* imbrandon feels that way about all the plugins and hooks too, thet work but feel tied we cant enhance them21:36
imbrandonthat sounds rihgt21:37
SpamapSI can flip it around and make it work21:37
imbrandonyour takergeting the container for the hooks relating and the plane ments should not matter21:37
SpamapSrelating nrpe<->*<->nagios21:37
SpamapSand just having nrpe tell * to inject stuff into the monitors relationship21:37
SpamapSbut that is *lame*21:37
SpamapSbecause now I can't have generic nrpe/nagios relations21:37
imbrandonso in theory you could have a subordiate that lives on its own single node but relates to many tings that way21:38
SpamapSI'm going to step away from nagios now21:38
SpamapSIts pissing me off too much. ;)21:38
imbrandonSpamapS: yup i know the feeling, thus how many times i have done the same thing on nginx and others ... maybe not that complex but i get jaded that i cant do something in a way i reason in my head is rweally the correct one21:39
imbrandonso yea ... i feel yea. and now as you pointed out the other day i think lots of mediocre charms just need to get pushed, these are ALL frakiin wrong and bad21:40
imbrandonbut they are out base and we just iterate them when each thing lands or is fixed21:40
imbrandonvs not having nginx for a month :)21:40
imbrandoni hate that, i hatre working like that but i dont see another way right nowwith how things are setup, too much stuff is waiting on go, wel really only probably 2 things are but then it cascades and nothing can go, because to "fix" some of this its really implment the feature/provider21:42
* imbrandon preaches to the chior more21:42
* imbrandon is sadened by his ~/Projects directory ... 21:43
imbrandonway tooo many things in to ( as i tend to only keep in progress projects there and once they hit a certain point they live online or a nother perm place on the hss/server .... but SpamapS check this ....21:46
imbrandonbholtsclaw@mini:~$ ls -lR ~/Projects|wc -l21:46
SpamapSimbrandon: randomly delete half of them? ;)21:47
imbrandonwell i got 2 big things that are adding to that ... a charm/getall symlink and a OMG full checkout21:48
imbrandonminus those two and it would be a bit more reasonable ... but still21:48
imbrandonits way way past where my OCD/Retardation normaly keep it21:49
imbrandonbholtsclaw@mini:~$ du -sh ~/Projects/21:49
imbrandonand crap i just realized that the hostname on this is wrong ... hrm suprised more things are not broken for me ... that i noticed yet least21:50
imbrandonoh yea synergy is broke ... wth , how would my hostname revert ...21:51
imbrandonahh terminal was left open from prior to it changing ... heh21:52
hazmatSpamapS, generally speaking it is22:00
hazmati answered that last night.. but you might have not been here.. but basically its the lack of coordination around stop. ie. parent supervision kills children22:00
hazmatin this context (subordinates) its actually a bit worse22:01
hazmatSpamapS, pls file a bug if you have a specific behavior around that you'd like to see22:01
hazmatdata jitsu ;-) http://oreilly.com/data/radarreports/data-jujitsu.csp?cmp=tw-strata-books-data-products22:14
imbrandonheh wow22:16
imbrandonhazmat: u seen @http://jit.su ? the nodejs api tool22:17
imbrandonerr the nodejitsu api too22:17
hazmatimbrandon, yeah.. i've got an account there22:17
hazmatone of the few paas's to support websockets22:18
imbrandonheh me too and appfog and phpfog and vmc and phpcloud and heroku and about a half dozen others i shiuld use more22:18
imbrandonyea i *think* nodester does22:18
imbrandontoo, iirc22:18
hazmatSpamapS, imbrandon so to expose services on hpcloud.. did you do it by hand via the control panel?22:19
imbrandonugh i need to rethink/redo my emails again and unzub from some things ... i hate that gotta do  it about once every 2 years22:20
imbrandonhazmat: yea i'm lazy and just set it to on bootstrap for the default one22:20
imbrandonand not care about the others22:20
imbrandoncuz not prod services were there yet22:20
imbrandone.g wide open22:21
imbrandonerr not default but the service name without a number22:21
imbrandoncuz all of them had it22:21
hazmatactually it looks like the expose functionality works with the tweak branch22:21
hazmatimbrandon, i just setup for internal traffic limited to
imbrandoncool, yea i have it pulled but noy launced a new env yet22:22
hazmatie. private network addrs22:22
imbrandonand the pubs are always 15.18522:22
imbrandonin all 3 az22:22
imbrandonjust fyi if you wanna limit that for someting later22:22
hazmatits unfortunate but not horrible i guess22:22
hazmatits manual though which is lame and needs docs22:23
imbrandonyea i've seen 15.185.100 .99 .98 and .10222:23
imbrandonmanual ?22:23
hazmatimbrandon, the rule management for the internal traffic22:23
hazmatsince hpcloud doesnt' support a rule for a self referential group (ie open group traffic)22:24
imbrandonthey have cli tools too , and juju is doing something with the IPs too funky as they dont automaticly get floating ones22:24
imbrandonyea it does22:24
imbrandonyou just have to use the nova client to make it22:24
imbrandonyou cant make it with the webui22:24
imbrandonhalf sec22:24
hazmatimbrandon, you mean the hpcloud tool?22:25
imbrandondamn internet is going slow, but its in the docs, and no i mean python-novaclient22:25
imbrandoneg "nova" onthe cli22:25
imbrandoni'll toss ya my env vars to get it setup22:26
imbrandonthis crap was all trial an error22:26
imbrandonits all piecedmealed and fraken stein and wrong22:26
SpamapShazmat: bug 1025478 btw.. filed yesterday :)22:28
_mup_Bug #1025478: destroying a primary service does not cause subordinate to depart <juju:New> < https://launchpad.net/bugs/1025478 >22:28
hazmatthere nova is some forked bespoke version22:28
imbrandonyup so is their fog22:28
imbrandonaka hpfog22:28
hazmatyeah.. but at least that one works pretty easily22:29
hazmatthere nova fork never did auth succesfully for me22:29
SpamapSso they just went off in the weeds22:29
* hazmat nods22:29
hazmatSpamapS, it looks like their going down the path of supporting all their own client libs22:30
hazmatand some what amusingly python isn't one of those per se22:30
SpamapSthats just awesome22:30
SpamapSwhy isn't anyone shaking fists at these people?22:30
SpamapSprobably because nobody is actually trying to use them yet22:30
hazmatSpamapS, because everyone is busy congratulating them on validating openstack?22:30
SpamapSI would think the libcloud guys in particular would be a champion for this22:31
imbrandonexport NOVA_USERNAME=me@brandonholtsclaw.com22:31
imbrandonexport NOVA_PASSWORD=22:31
imbrandonexport NOVA_URL=https://region-a.geo-1.identity.hpcloudsvc.com:35357/v2.0/22:31
imbrandonexport NOVA_PROJECT_ID=me@brandonholtscaw.com-default-tenant22:31
imbrandonexport NOVA_VERSION=1.122:31
imbrandonexport NOVA_REGION_NAME=az-3.region-a.geo-122:31
Davieyimbrandon: what is the NOVA_PASSWORD ?22:31
hazmatimbrandon, pastebin would have been nicer, but thanks22:31
imbrandonhazmat: that will make their nova work + your webui pass22:31
hazmatimbrandon, tried that didn't work for me though22:32
imbrandonbah was lazy, ohhh crap for got that one Davie, its PinkPoniez2322:32
hazmati tried just about every permutation of key/username/password/token auth22:32
hazmatthe ruby cli tool works but is limited22:32
hazmatand frankly given how poorly nova worked against rspace..22:32
imbrandonalso with the swift cli tool you need to drop the auth to v1 as well22:32
SpamapSI was able to use stock precise nova manage22:32
hazmati'm not expecting to much22:33
imbrandonyea i can use the nova22:33
hazmatSpamapS, oh.. interesting.. i'll have to try that ..22:33
hazmati tried their version of nova in a sandbox22:33
imbrandonnova i installed from repo, the hpfoghpcloud i setup the first time, then it screwed my rvm/rails install22:34
imbrandonand nevew reinstalled it22:34
hazmatmost of nova cli doesn't work against rackspace.. volume/secgroups, etc.. because that's all split out into separate services that aren't publicly available atm22:34
imbrandonbut did send em a patch so it would work on 1.8.7 :)22:34
hazmatits really a mystery to find out what impl of nova actually supports as far as api22:34
hazmatmost of these have some sort of java front end proxy doing request load balancing against the api services and rate limiting it appears (rspace and hp)22:35
imbrandonhp has a knolge base one just for nova cli that lists what does/cont22:35
SpamapSjava front say what?22:38
imbrandoni think they had apache traffic server on the GUI console when i first looked22:46
imbrandonbut their strange anyhow without much direction i dont think, they have yui and jquery and a few other double duties things i've noticed22:47
imbrandonthey tend to grab whatever tool from where ever to do that they need for the moment and customize it a bit then not really fully intergrate any of it22:48
imbrandoncept the php-bindings22:48
imbrandonthat is the only exception :)22:48
imbrandonanyhow none is a big deal until after a month or 6 you add them all up22:48
imbrandonand its like wow this really sucks22:49
SpamapSI hate bash22:51
SpamapSwhy am I doing things in bash?22:51
imbrandonbwhahaha <?php require('/hooks/lib/common');22:52
imbrandonyou know you wanna :)22:52
SpamapSWhat I want is for my charms to be 90% declarative22:52
imbrandonugh i need to push the new stuff for that too ....22:52
SpamapSI wonder if salt's states can be easily used w/o salt's agents22:53
imbrandonnah i like the flexability , i just want first class config mangemwent and raw access to the cloud-init even if hidden normally22:53
imbrandonchefsolo or capistrano or fabric ... ? /me figured ytou'd use fabfiles.py22:54
imbrandonSpamapS: you know since we require cloud-init on the machines , it bakes capistrano and puppet as dependencies already so22:55
imbrandonthey are their to be used :)22:56
imbrandonwe just cant use them in cloud-init but could still use them :P22:56
SpamapShow does one break out of puppet into a real language when it is needed?22:56
imbrandonno idea, i hate puppet personally its a DSL not ruby22:56
imbrandonbut i know others like em22:57
imbrandononly really seriously used capistrano or homecrown config mgmt for any length of time22:57
imbrandonbut i seriously do thing you CAN22:57
imbrandonjust not sure HOW22:57
SpamapSYes I know its a DSL22:58
SpamapSand there are these tiny moments where you need to do something that the DSL can't do22:58
SpamapSI want to know how do I add those capabilities22:58
SpamapSI suspect "with ruby"22:58
imbrandonSpamapS: you might check mozillas github repo for their config stuff they use it alot iirc22:59
imbrandonwell all of them22:59
imbrandonno thats what i'm saying you dont have to add them they are already installed you just need to like call "puppet somefile.pp"23:00
imbrandonin your shell23:00
SpamapSOk here's another problem with subs..23:00
SpamapSso my nrpe charm had a bug in the joined hook for the subordinate relation23:00
SpamapSI fixed it23:00
SpamapSupgraded charm23:00
SpamapSbut oh noes.. I can't actually apply the fix23:00
imbrandoncloud-init depends on them , thats why i was irritated when we was restricted from them i'm like ummmmm that sux23:00
SpamapSI have to now rewrite that joined charm to be "relation agnostic"23:00
SpamapSjoined hook23:00
SpamapSand then I have to call that hook in upgrade-charm23:01
imbrandonSpamapS: yea when that happeens to me i ssh to the box and edit the files in /var/lib/juju/blahvblhblag23:01
SpamapSexcept, the error was in the way it grabs JUJU_REMOTE_UNIT23:01
imbrandonthen --resolved -retry23:01
imbrandonor whatever23:01
SpamapSwhich is not available ever again23:01
SpamapSso.. doh .. kill the primary service and start over23:02
imbrandonwell i do it local imediately but the alternative is to destroy it23:02
SpamapSimbrandon: it did not error23:02
SpamapSso there's nothing to retry23:02
imbrandonnot debuu=g hooks23:02
imbrandonok lets step back23:02
imbrandoni know there is nothing .. i'm telling you my hack23:02
imbrandonthat works BUT now will get fixed23:02
imbrandonok soooooo23:02
SpamapSthat fix won't even work here23:03
imbrandonwhen it gets to that point that you only have the choice to destroy it23:03
imbrandoni dont debug-hook23:03
imbrandoni ssh TO it23:03
imbrandonreal ssh23:03
imbrandonend edit the cached charm23:03
SpamapSthere's a value that I can't get23:03
SpamapSits gone23:03
imbrandonthen when i run retry --resiolved23:03
imbrandonetc it refiles23:03
SpamapSthere is nothing to retry23:04
SpamapSstate is up23:04
imbrandoni know there isnt . but it still does it23:04
imbrandonand fixes my problem23:04
imbrandonyea i think that unintentionally that "juju resolved --retry service/2"23:05
imbrandonis out "force a hook"23:05
SpamapS2012-07-17 16:05:22,323 WARNING Matched relations are all running23:05
imbrandonsure but watch the log23:05
SpamapSnothing running23:05
SpamapSwhich hook would it even run?23:06
imbrandonit always tells me that it dident need to sdo it and act like it dont then i watch the debug log and hooks start going to town23:06
SpamapSI think you been smokin something son23:06
SpamapSthat does nothing23:06
imbrandonyou could also edit the upgrade temp23:06
SpamapSI could do 100 things23:07
SpamapSbut I know juju in and out23:07
SpamapSusers might not want to learn that23:07
imbrandonyea they do if there was a way23:07
imbrandonno seriously they will tho23:07
imbrandonthink about how well the chef users know that23:08
SpamapSIf you brought me something and said "well sometimes you just have to ssh in and edit random files that you don't understand on disk".. I'd bring you a rather large blunt object to beat me to death with23:08
SpamapSwe are not targetting chef users23:08
imbrandonits part of their profession that like most ggeks will take pride in knowing more than the boss23:08
SpamapSchef is a big beast23:08
SpamapSso is puppet23:08
SpamapSnot './configure && make && make install' for the cloud.. *apt-get* for the cloud23:08
imbrandonsure if thats all we needed we have aws-tools in the repo23:09
SpamapSas in, install it, configure it, and get it to the point where you can use it.23:09
imbrandonyou all keep saying shit like that and are contradiucting your selfs23:09
SpamapSI know how *I* feel23:09
SpamapSI think I disagree with quite a few on this23:10
imbrandondude we just had tis exact convo23:10
imbrandonlike a week ago23:10
SpamapSand I said we want puppet for the cloud?23:10
SpamapScause, I was drinking a lot last week...23:10
imbrandonwell i explained untill shit lands we are puppet23:10
SpamapSoh lol23:10
SpamapSuntil shit lands we are a ship w/o a rudder23:10
imbrandonyea that was the jest of it23:11
SpamapSso lets not talk about our problems, but our intentions23:11
SpamapSintentions == simpler way to get stuff into the cloud23:11
imbrandonno for real tho we went though all this, and you kept telling me how we wernt but we could do this and that and this yet  and had to ignbore dry and blah , and i'm like ummm ok so we're not ...... yet :)23:11
SpamapSAnd a way to share the deployment "formulas"..errr.. charms.. so you don't have to sit down and open your editor and refactor half of the thing to get it tow ork in your infra23:12
imbrandonoh yea i totally see where we're headed , or i wouldent be here23:12
SpamapSWhat we aren't, and what we want to be, are the same thing. :)23:12
SpamapSSo yes, we can ssh in and dick with files23:12
SpamapSbut I am kind of tired of throwing bugs on the pile. ;)23:12
imbrandonbut i was and am being a realist about right now, we do pretty much aboutr 60% of things worse with plans that are solid to be 200% better23:12
imbrandonthe same realist in me says tho once it does all pan out its gonna be sweet tho !23:14
imbrandonheh and much better than if you bolted this onyo chef or salt stacks23:14
imbrandonor made anseble not use the /etc/hosts file for 7000 nodes23:14
SpamapSI see you tried it out? :p23:15
imbrandonyou rwealize that btw, ok so here is the 5min explination of anseble == juju s#zookeeper#/etc/hosts+hash in memory no matter how many nodes of ALLLLLL meta data and databags#g23:16
imbrandonyea i try everything , some things more than i should some less ;), thats why i end up with accounts on all services , because unless you try or learn about every part that you might at some point need to know i dunno to me you just should know the options available as much of a pita it si, like your CentOS thing :)23:17
SpamapSyeah I try to keep it limited to the biggest alternatives, not all alts23:18
imbrandonbut yea. try to use anseible with 700 nodes launched and another 300ish  off but in the host file ... I saw the results and laughed as i unsubscribed from the ML23:19
imbrandonits perfect for 5 machines, does somethings alot better , but ... yea $6 on fuckit23:19

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!