=== c0s is now known as cos1 [00:17] * arosales reads back scroll [00:19] its not very complex [00:19] arrange meeting with c0s [00:22] c0s we can install the payload from big top and something we should think of for power8 [00:22] c0s the mechanism to easily change the payload is resources [00:24] The main thing is to ensure the target install source is always available or the user gets and install hook error and thinks it was juju not the target install host [00:24] c0s we can also scale services with add unit and remove unit we just need to model that correctly in the charm [00:24] But good questions [00:25] he's also very much left :P [00:25] I would be interested in know more about your use cases [00:25] magicaltrout: no wonder auto complete wasn't finding him [00:25] :-) [00:26] hehe [00:26] rick_h_: I'll catch up tomorrow when cos is back in channel [00:26] arosales: rgr [00:26] arosales: ty [00:27] you should leverage his friendship with Roman to get pivotal onboard :) [00:39] mostly because it would get me out of writing a fat bunch of charms that I don't have time to write but really want :) === menn0 is now known as menn0-afk [08:00] cory_fu: https://github.com/juju-solutions/layer-basic/pull/49 is broken I think, the apt_install doesn't silently fail on trusty [08:00] lazyPower: let me know when the kibana stuff gets merged so I can test beats again! :) === dpm_ is now known as dpm [10:44] gnuoy, quite a big one - https://review.openstack.org/#/c/295714/ [10:45] glad to see it though [10:56] jamespage, hey when you can I'd like to investigate the issue from yesterday with the maas multi-nic containers [12:00] gnuoy, and its buddy - https://review.openstack.org/#/c/295745 === zz_CyberJacob is now known as CyberJacob === CyberJacob is now known as zz_CyberJacob [12:16] I've got a production environment running 1.25.4, proposed stream. Future upgrades should hopefully only be to official releases. Can I set the agent-stream back now, or do I need to wait until I upgrade? === zz_CyberJacob is now known as CyberJacob [13:38] I have an environment where a charm radically changed and when we ran upgrade-charm I don't think it cleaned up all the old state from the previous charm [13:39] what's the best way to get rid of that service in order to install it from scratch? [13:39] which destroy command should I call? do I need to call multiple ones or will one of them cascade down them for me? [13:40] it's gunicorn and is a subordinate [13:41] destroy-service seems intuitive, but I'd like a sanity check [13:54] skay: yes, that's it [13:54] tvansteenburgh: thanks :thumbs: [13:54] (:thumbs: is actually bd but people mistake it for batman's mask) === cos1 is now known as c0s [14:32] magicaltrout: ack, lemme give it a little longer for manjo [14:45] aisrael - the guy we met from treasure data was Eduardo right? [14:45] lazyPower: Yep [14:45] * lazyPower grins [14:46] sam just put me back in touch with him today [14:46] Excellent! [15:02] aye [15:31] mbruzek - if you have a minute, i just sent you a doc with our status update [15:31] can you proof that before i ping the list with it? === HaloSponge is now known as FreeSponge === FreeSponge is now known as LibreSponge [15:46] lazyPower: sure [15:57] mbruzek - my mail got rejected due to the attachments, so i'm converting this into ablog post [15:58] lazyPower: I removed some contractions and added a few comments. [15:58] Was my review too late? [15:58] ack [15:58] not at all [15:58] i'll get htem folded in, igotta head to a meeting with weave, but i'll incorp those before publishing [15:58] thanks for taking a look! [15:58] yar [16:22] bdx: heya, mind PR'ing your slides or a link to this: https://github.com/juju/presentations [16:22] I'm going to start pushing more presentations/talks into that repo [16:25] jcastro - are you exporting our docs slides to like ODP and uploading as well or? [16:27] lazyPower: so I have a place to put html slides, and a place for PDFs and other outputs [16:28] I figure we can use the readme for links to slides/vids, etc. [16:28] ah that or we can pdf upload [16:28] and I am about to make a folder for things like "talk titles and submissions" [16:28] but im' not really a fan of keeping pdf's in git as they are blob and just bloat the repo [16:28] so nvm carrry on sir [16:28] instead of "hey chuck mail me the last 10 submissions for devops days you sent in === dpm is now known as dpm-afk [16:28] omg i love this idea [16:28] who's teh genius i need to hug? [16:28] I will put tips and tricks there too [16:28] like terms to use, terms to avoid, etc. === urulama is now known as urulama|afk [17:46] jamespage: sometime back you were working ovs charm with DPDK? will you please point me pointers so that i can give a try with JOID> [17:46] narindergupta, I've not started on that yet [17:47] jamespage: oh ok [17:47] jamespage: do you know anyone else did? [17:47] no one else has done that yet... [17:47] narindergupta, its on the list of things todo still, just not got to it yet... [17:48] jamespage: ok thanks for information [18:19] aside from 'kill-contoller' any folks have any hints on how to reclaim and juju 2.0 environment? [18:19] * arosales stuck in this loop http://paste.ubuntu.com/15473933/ [18:20] * arosales may need to check in #juju-dev [18:26] fyi fix was to rm ~/.local/share/juju/models/cache.yaml _if_ you only have the one controller you care about. If you have other controllers you care about then you need to remove the offending lines [18:26] may need to clean up clean up ~/.local/share/juju/controllers.yaml as well [18:27] but I am not able to reboot strap again [18:27] thanks to cherylj [18:27] thats lazyPower 's favourite fix [18:27] lazyPower: has a pretty good toolbox [18:27] hi [18:27] what did i do? [18:27] i'm actually working on a python script to nuke a leftover controller, but i dont want to publish it [18:28] all the things [18:28] because hand editing cache.yaml is frightening [18:28] and i dont want to advocate anyone do this [18:28] lazyPower: I think it is a bug they should address, if not we'll have the juju clean up script again [18:28] bugs open [18:28] 1 sec let me find the link [18:28] cherylj: ping me if you need me to open a bug on it. [18:28] lazyPower: oh is a bug open already on it? [18:28] lazyPower: I basically hit http://paste.ubuntu.com/15473933/ [18:28] https://bugs.launchpad.net/juju-core/+bug/1560191 [18:28] Bug #1560191: kill-controller is hinky without a model-controller behind it [18:28] several, I'm sure [18:29] that's technically a different issue [18:29] this is the source of arosales' problem: https://bugs.launchpad.net/juju-core/+bug/1543223 [18:29] Bug #1543223: kill-controller fails on missing volume

=== cos1 is now known as c0s [18:29] see also, bug #1555744 [18:29] Bug #1555744: kill-controller / destroy-controller prevents reuse of controller name

[18:30] lazyPower: thanks for filling the bug, I added a comment [18:30] cherylj - nice, thats the other side of it that i've hit [18:31] thanks :D [18:31] * cherylj cries seeing arosales' suggestion to rm cache.yaml in a bug [18:31] heh [18:31] ok I posted in both bugs [18:32] but not 1555744 cause I didn't want to spam all the bug reports :-) [18:32] cherylj: hey it worked [18:32] lol :-) [18:32] you may cry [18:32] cherylj: I did state "given I only had 1 controller I cared about" [18:33] i usually remove the whole .juju folder :P [18:33] but I guess I should have noted that was the nuclear option [18:33] * cherylj weeps uncontrollably [18:33] ;) [18:33] ah well I don't feel as bad now magicaltrout :-) [18:33] i happens often enough [18:33] normally user error :P [18:34] its commonly pilot error for me an not a bug [18:35] although i have one production system stuck on 2.0alpha1 [18:35] which i'm not allowed to break [18:35] because I have no way of upgrading it \o/ [18:36] those are always fun, snowflakes that we create in alpha/beta land [18:36] i've been fighting with myself to not setup any 2.0 beta controllers for running systems due to that very reason [18:36] yeah its like the bastard child, its 2.0 with 1.2 configuration setup [18:36] but with all the goodness thats in here, its really hard to do it [18:36] *to not [18:41] you know when its beer o'clock and you can't drink cause you have a conference call in 20 minutes.......... [18:41] plus you need to write a proposal to build a SQL over JSON interface [18:42] grr [18:45] magicaltrout: that's the best time to have a beer [18:45] hehe [18:45] it wouldn't be the first time === alexisb is now known as alexisb-afk [19:10] cory_fu: which is better? [19:10] @when('nginx.available', 'charm-svg.running') [19:10] @when('nginx.available') [19:10] @when('charm-svg.running') [19:11] seperation or as *args ? [19:11] They are equivalent and it's up to your personal aesthetic. [19:11] cool beans, thanks [19:11] cory_fu: same for @when_not? [19:11] marcoceppi: The main difference is that the order of args from @when decorators is a little confusing when split. It basically goes bottom to top, left to right [19:12] cory_fu: yeah, I've encountered that from patch library [19:12] marcoceppi: All decorators are ANDed together. The only decorators that do not also AND their args are @when_any and @when_not_all [19:12] cool [19:13] I think I'll leave them split for now, it's easier to talk to [19:14] cory_fu: I' [19:15] I tend to group the ones that are either related or the same for two similar blocks, and split ones that vary between two blocks. So, if two blocks have shared preconditions, but one is @when('foo') and the other is @when_not('foo'), I will group the shared preconditions @when('bar', 'qux') but split the changing state @when('foo') [19:15] If that makes any sense [19:15] cory_fu: Ive been thinking about the config stuff, how they're basically using states for events [19:16] cory_fu: would it be better to standardize that as charms.reactive.emit ? so you could emit('state') instead of set_state and removing it? [19:16] (where emit would basically say, at end of state execution, remove this state) [19:17] There was a discussion about that on one of the issues or PRs [19:18] So, emit would be shorthand for "set_state(event); hookenv.atexit(remove_state, event)" [19:18] cory_fu: right now I'm wrestling with update-status hook. I'd like to just make a @when('update-status') or something similar which would be triggered during the update-state hook, but also something I could poke at from methods by just emitting that state [19:18] cory_fu: basically [19:19] marcoceppi: I tend not to collect status reporting into a single handler, though sometimes it is more useful to do so [19:19] bcsaller: Thoughts on ^ [19:19] ? [19:22] marcoceppi: Here was the other comment touching on events vs states: https://github.com/juju-solutions/charms.reactive/issues/44#issuecomment-176278218 [19:23] remove at end of hook is only one possible semantic, sometime that same spelling might be intended to mean, remove once processed. If the hook fails either the trigger condition needs to regenerate or the cleanup event was wrong, IMO it's better to detect state (de)activation and decouple it from hook context [19:26] bcsaller: If the hook fails, it is likely that the states will not be flushed and things will re-run from the initial state on hook retry, though that's not guaranteed [19:26] atexit can make that tricky [19:27] How so? [19:28] atexit isn't run on error [19:31] cory_fu: not on any sys.exit? [19:32] * TheMue listens trying to learn a bit [19:32] hi Cory, hi Benjamin [19:34] hi [19:35] bcsaller: even after leaving Canonical Juju fascinates me and I advertise it *bg* [19:36] excellent [19:36] o/ TheMue [19:37] bcsaller: https://github.com/juju-solutions/charms.reactive/blob/master/charms/reactive/__init__.py#L74 [19:37] Only if there are no errors and it exits 0 [19:39] cory_fu: ahh, thanks, didn't recall we were using our own impl of that [19:39] marcoceppi: people around me are always fascinated about its possibilities. my talks in the past have been more about the technology (just core *g*), but now maybe I should place talks about charms [19:39] cory_fu: I still tend to think that trying to couple states to hook invocations isn't needed [19:42] bcsaller: In general, I agree and would go even further to say that it's a bad idea. But there are also cases where we want to ensure that a given state can be handled by every handler that wants to and is cleaned up afterward. [19:42] That can be tricky to do, because no single handler should remove the state in that case since it might block other handlers from responding. [19:43] The end of the hook context becomes a work-around in that case. Really what it means is that "all handlers that are going to trigger given the current overall state have done so" [19:43] cory_fu: which should be something we can detect, record and clear automatically [19:44] The only way we can detect it automatically is when the dispatch loop terminates, and the very next thing that is run then is atexit [19:48] bcsaller: Merlijn had another interesting take on it with the idea of having an "event" that handlers only got one "bite at the apple", as it were. So if their other conditions didn't match when it was emitted, they didn't see it later if their preconditions did change. That's something we can't really model now at all [19:50] cory_fu: do you have time for a hangout at some point? [19:50] Sure [19:52] hi, how can we specify juju to deploy 16.04 on MAAS instead of the default 14.04? [20:13] falanx: that depends on the charm, most of the charms are targeted at 14.04 (trusty) - what are you trying to deploy? [20:14] stokachu: got a min for a quick review? https://github.com/battlemidget/juju-layer-nginx/pull/4 [20:15] marcoceppi: we are trying to deploy the controller [20:15] falanx: are you using 1.25 or 2.0? [20:15] 2.0 [20:19] marcoceppi: merged, thanks! [20:20] falanx: you should just be able to do `juju bootstrap --bootstrap-series=xenial` [20:20] marcoceppi: what do you think about me moving those layers under the juju-solutions org? [20:21] stokachu: the nginx one? [20:21] and nodejs [20:21] to start with [20:21] I don't have a problem, but they're fine in your namespace [20:21] ok thats cool [20:21] stokachu: I do have some plans to spruce up the nginx layer, but it breaks compat so I'm not sure how to handle this gracefully [20:21] cory_fu: ^ [20:22] i thought there were some talk of versioned layers? [20:22] falanx, marcoceppi: adding bootstrap-series results in "ERROR cmd supercommand.go:448 failed to bootstrap model: no matching tools available [20:22] " [20:22] LiftedKilt: --upload-tools [20:22] ninja'd [20:23] stokachu: results in same error [20:23] LiftedKilt: weird. [20:23] I'm running: juju bootstrap juju2 dr --upload-tools --bootstrap-series=xenial --debug [20:24] LiftedKilt: what does `juju version` say? [20:24] marcoceppi: 2.0-beta2-wily-amd64 [20:25] LiftedKilt: can you pastebin `juju list-clouds` ? [20:25] ugh i can reproduce [20:25] huzzah [20:25] cherylj: is it possible to bootstrap a xenial controller? [20:26] marcoceppi: sure is. What cloud? [20:26] cherylj: MAAS [20:26] a private cloud [20:27] MAAS version 1.9.1+bzr4543-0ubuntu1 (wily1), for what it's worth [20:27] ah, if you can't get to the streams for tools, you'll need to --upload-tools [20:28] cherylj: so, even with upload-tools the bootstrap fails [20:28] LiftedKilt: are you bootstrapping from an ubuntu machine? [20:28] you may not have an up-to-date distro info [20:28] cherylj: from a 15.10 machine, yes [20:28] which would cause that error [20:29] LiftedKilt: do you have Xenial in /usr/share/distro-info/ubuntu.csv? [20:30] 16.04 LTS,Xenial Xerus,xenial,2015-10-22,2016-04-21,2021-04-21 [20:30] LiftedKilt: can you send me a paste of the bootstrap --upload-tools --debug? [20:31] marcoceppi, cherylj: actually it fails with the same error for bootstrap-series=wily as well [20:31] cherylj: sure [20:31] ah, there's some weird behavior with bootstrap-series [20:31] LiftedKilt: try default-series=xenial [20:32] http://pastebin.com/L7ziCLJx [20:32] cherylj: I thought default series was for charms? [20:32] --config default-series=xenial [20:32] LiftedKilt: it will also be used when adding machines [20:33] cherylj: perfect - it's bootstrapping now [20:33] yay! [20:34] LiftedKilt: there are already bugs open about the bootstrap-series not working. I can find them if you'd like [20:35] cherylj: no that's fine - as long as I can get around it I'm happy [20:36] cherylj, marcoceppi thanks for the assistance! [20:36] thanks cherylj [20:36] anytime! [20:37] marcoceppi: Sorry, was on a call. Were you asking me about how we deal with breaking changes in layers? You were one of the opponents to versioned layers, so you tell me. ;) [20:37] cory_fu: I was just supporting your rederict ;) [20:38] TBH, I'm still on the fence wrt versioning for base or interface layers. [20:39] tbh, I think it's up to the charm author to deal with it, but maybe supporting "major" revision, or backward incompat versions, might be a compromise === alexisb-afk is now known as alexisb [20:39] where each version is basically an epoch [20:40] can I specify the lxc profile to use when using the juju 2 lxd provider? [20:42] marcoceppi: For interface layers, there's also the difficulty that we now have two things that could potentially be versioned: the interface protocol and the interface layer's API. Now that we have interface layers, it's actually less of a concern for the protocol to change, as long as the layer manages the complexity of maintaining backwards compat. but it's a huge deal if the layer API changes in a breaking way [20:43] cory_fu: right, given the simplicity of the interface (key val comm) backwards compat isn't nearly as complex as a code change breakage [20:43] If we're talking about doing "epoch" versions, then each epoch version change is something like a fork [20:43] cory_fu: more or less, yes [20:52] cory_fu: with all that you just said wrt protocols, interfaces, and versions - how you envision to communicate underlying components' APIs to the client software [20:52] say, if juju bundle include component_7 (where's 7 is next revision) then how would I know that my software will work with it? [20:53] it == the component that component_7 charm represents [20:53] am I making any sense? === menn0-afk is now known as menn === menn is now known as menn0 [21:24] c0s: So, I'm not sure I understand your question, but in particular, bundles don't contain layers (what I assume you mean by components), they just contain charms. Charms are built from layers, and some of those layers (interface layers) are responsible for manging the communication protocol of interfaces and providing a defined, documented API to charms. [21:24] c0s: But, also, layers are not combined or updated at deploy time, only at build time (which could be thought of as the "compile" phase for the charm) [21:25] I guess I am looking a bit deeper into the relation between the layer (sorry for mis-using the terminology) and the actual software the layer is deploying/managing [21:25] So, regardless of whether those layers are versioned or just updated at every build, it's the job of the charm author to verify that the newly built charm works as expected, and is why we want bundles (and sometimes charms) to include tests [21:26] right, that's how you produce the stack [21:27] now, when I (as a user) deploy your stack (or bundle in the juju-speak) - how do I know that my hdfs app relying on 2.2.0 API version will work. Does the version of underlying component (software) get exposed somehow? [21:28] c0s: You would know that it works because it should be tested before it is published to the charm store [21:29] say, if I deploy using packages I can do apt show package-name and get some info [21:29] wait, this is my client application. You can not possibly claim that you test _all_ applications out there [21:30] I am not questioning the integrity of the bundle [21:30] charms that are in the recommended namespace have been tested and validated [21:30] so if you install a charm not in the recommended namespace YMMV [21:31] just a note to all people using direct publishing. charm store on production was updated and you'll need new charm command that marcoceppi will make ready soon [21:31] I think we are still talking past each other. I guess that's my English as a second languge [21:31] c0s: I think there's some disconnect on what you and I mean by API === urulama|afk is now known as urulama [21:31] yeha [21:31] yeah, I am sure. By API I mean the interfaces of the software you're deploying with charms. I am not talking about juju interfaces [21:32] say, HDFS open() API [21:32] There is the API that the layers use to work together (basically, the states that they set using set_state and watch using @when, etc) [21:32] or better yet - truncate, which exists in some versions of HDFS and not others. [21:32] and if I write software to work with an HDFS cluster it will fail if I am calling truncate which isn't there [21:33] Ok, so in that case, the HDFS charm would be responsible for making sure it only installs a version of HDFS that it knows how to work with. And the big data charms report the version of Hadoop they install to ensure they all have the same version [21:33] just for the sake of argument: I know this call exists in HDFS x.y.z. If I deploy apache-core-batching bundle - is it easy for me to figure out what version of HDFS is coming with it? [21:34] ok, good - now we are on the same page. [21:34] So, charms have a way to communicate the versions of underlying software (components). Good [21:35] let's get back to the original discussion of how charms should be versioned, shall we? ;) [21:35] c0s: Well, they can communicate the version to other charms that connect to them, but it's up to the charm author (well, actually, the interface author) to make that a part of the interface protocol [21:35] but what about the user? Will he has any way of finding out the version? [21:36] That's all I am trying to figure out [21:36] in other words: is there an analog of apt show packagename ? [21:36] The version might be a config option on the charm, or it might be hard-coded in the charm and documented in the README [21:36] ok, so I need to look into a particular charm README to find what versions it packs, right? [21:37] Generally, yes. [21:37] ok, got it. [21:37] now the last one, I promise - currently, bundles aren't versioned, per se, right? [21:37] although layers (or charms) are. [21:38] Most charms are hard coded to install a specific version of the software, though many do allow you to specify either a version or a source URL for the software so that you can control the version to some degree [21:38] ok, got it [21:38] Bundles and charms have revisions, and bundles can specify what revision of each charm they deploy (which is required for recommended bundles) [21:38] thanks [21:39] yet again ;) - revisions aren't directly visible to the user, right? [21:39] Bundles can also specify config for charms, so if the software version is configurable in a charm, the bundle can specify that [21:39] ok, thanks [21:40] Revisions are visible to the user, yes. https://jujucharms.com/u/bigdata-dev/apache-hadoop-namenode/trusty/4 is revision 4 of that charm [21:40] aka cs:~bigdata-dev/trusty/apache-hadoop-namenode-4 [21:40] You can also leave the revision off to get the latest [21:41] cool, thanks [21:48] I guess you guys are going to hate me soon [21:48] ;) [21:50] In the current design if I want to take an advantage of existing 3rd party deployment code (eg. a bigtop puppet recipe), I would have to wrap it into Python reactive script, right? [21:55] c0s: lazypower did a cool talk about juju leveraging ansible [21:55] which could be repurposed for puppet or chef [21:56] https://www.youtube.com/watch?v=0eymk93lY8k [21:56] as in "I can write ansible scripts which Juju be able to reuse for the deployment" or something else? [21:56] yeah, as in, I have ansible already but want to leverage juju's deployment capabilities so make a few changes and deploy my ansible code via juju [21:56] or something like that [21:57] make sense, thanks for the link [21:59] c0s: You can also do reactive handlers in bash, but you still do end up having to call out to your existing cfg mgmt tool from a handler, yes [22:00] yup, that's what I thought. Thanks! [22:03] Why does the lxd charm only have options for lvm and btrfs? Wasn't zfs also supported? [22:09] jamespage, falanx: What would it take to enable zfs support in the lxd charm for xenial? === urulama is now known as urulama|afk [22:42] tvansteenburgh: I've done some long needed triage in the charm-tools repo, cleaned up the milestones, and make sure things were assigned: https://github.com/juju/charm-tools/milestones unless you're going to be working anymore on these tonight I'm going to cut a 2.0.0 since the new charm command is ready [22:45] marcoceppi - yeah man you filled me up with notices :D [22:45] but look at all that progress! === redir_ is now known as redir [22:59] Hi all; any charmers able to look at http://juju-ci.vapour.ws:8080/job/charm-bundle-test-aws/3277/ and http://juju-ci.vapour.ws:8080/job/charm-bundle-test-aws/3278/ and confirm that this is just broken test infrastructure? I can't see anything in the logs which indicates it's a problem with my MP. [22:59] *MPs [23:20] blahdeblah - yeah looks like the security group cleanup at the testrun start is what caused that [23:21] the output looks fine on 3278 otherwise [23:21] boto.exception.EC2ResponseError: EC2ResponseError: 400 Bad Request [23:21] lazyPower: thanks - anything I need to do to make sure the MPs don't end up in limbo? [23:21] blahdeblah - i'll poke someone to take a look at this tomorrow and follow up - but it LGTM otherwise, i see your bundle stood up and got 1 test passed. [23:22] is it listed on review.juju.solutions? [23:22] yep [23:22] i dont see anything from you in there :/ [23:22] 2nd & 3rd from the bottom [23:22] oh probably not sharing irc handle w/ launchpad [23:23] derp [23:23] Yeah, so long as they are in there it'll get reviewed. Typically we kick the ci runs off the morning of our review time to get fresh results [23:23] cool [23:23] thanks [23:23] so i wouldn't stress over that initial result :( i can kick it again if you like, maybe its been resolved [23:24] marcoceppi: cool, i won't be getting any more done tonight, release away [23:25] tvansteenburgh: cool, thanks [23:31] kwmonroe: cory_fu: I am still not able to deploy realtime-syslog-analytics with juju 2.0 beta2 in aws-east1 [23:31] * arosales is going to try a different region [23:31] http://paste.ubuntu.com/15475911/ is what I see --- stuck in waiting to agent init to finish . . . [23:35] well arosales if the machine doesn't come up you wont get the agent init to finish [23:35] arosales - have you done this? [23:35] you sure you're not getting something like the AWS instance upper limit errors? [23:35] arosales 'juju retry-provisioning #' [23:36] default upper limit is 15 iirc, bumpable to 25 [23:44] sorry I was looking at my aws account [23:44] magicaltrout: ya it feels like an aws limit, but I have bootstrapped this before [23:44] and my amazon instance doesn't show any limit issues [23:44] pehraps my sec groups limit is getting close . . . [23:45] lazyPower: I haven't tried that yet, I was going to see if I had better luck in us-west-2 [23:46] 500 sec group limit, I am at 162 so ok there [23:46] and 20 instance limit and I have 0 current, so I am ok there too [23:47] surely juju debug-log gives some clue as to the failure cause? [23:48] not really [23:48] excellent! ;) [23:49] arosales i had some issues with local charms earlier this week but i was hard to reliably reproduce [23:50] none of those charms are local right? [23:50] i realize this was at the infra provider level, but, just crossing off the list [23:50] me still investigating though [23:50] * arosales still investigating though [23:50] lazyPower: all charm store charms [23:50] mfw using juju 1.25 where juju dhx -s still works [23:50] semi off topic but there's 2 tools ubuntu is missing by default. mosh for those of us moving around and pastebinit to dump charm logs to pastebin without messing around [23:51] * lazyPower falls in love with dhx all over again, after all these weeks [23:52] magicaltrout: it would be nice to pass to cloud-init via juju a metadata file that includes user tools they like seeing in their environments like the ones you mention [23:52] yeah arosales thats a cool idea