[01:51] <waigani> axw: quick question?
[01:51] <waigani> why do you do this: var _ state.Policy = environStatePolicy{}
[01:52] <axw> waigani: it's a convention in juju to ensure that the implementation continues to implement the interface
[01:52] <axw> also, documentation as code
[01:55] <waigani> ah right. I saw that before. The type assertion will fail if the struct's methods change.
[01:59] <axw> the code won't actually compile, because the assertion can be proven at compile time
[02:00] <axw> it's a static assertion
[03:44] <waigani> axw: here is my first pass. https://codereview.appspot.com/63790045
[03:44] <waigani> axw: No tests yet and I have not implemented ValidateEnvironConfig in any provider yet. I took a look at adding it to the manual provider, but I wanted to talk over how to rewire SetConfig with you first.
[03:44]  * axw looks
[03:45] <axw> waigani: you shouldn't need to implement it anywhere, just use the one on EnvironProvider
[03:45] <axw> you can get the provider from the config
[04:50] <davecheney> axw: http://paste.ubuntu.com/6969152/
[04:50] <davecheney> ^ this error looks pretty terminal
[04:51] <davecheney> any suggestions ?
[04:51] <axw> wtf mate
[04:51] <axw> umm
[04:51] <axw> no
[04:51] <axw> are you packages up to date?
[04:52] <axw> wallyworld was having some problems with lxc on trusty
[04:52] <davecheney> axw: can you tell me the commands that get exectued under the hood
[04:52] <wallyworld> axw: the latest update an d areboot fixed it
[04:52] <davecheney> i could try to do them manually to smoke test it
[04:52]  * davecheney shrugs
[04:52] <davecheney> i'll give it a go
[04:53] <davecheney> oooh, i see lxc-templates
[04:53] <davecheney> fingers crossed
[04:53] <wallyworld> yeah you need lxc-templates installed as well as lxc
[04:53] <wallyworld> lxc rc4 works for me now
[04:53] <wallyworld> rc3 seemed to barf
[04:55] <davecheney> yeah, i have the templates
[04:55] <davecheney> just i guess not hte latest ones
[04:55] <axw> davecheney: I'm not sure what the command is, but you want to create a container with the ubuntu-cloud template
[04:55] <davecheney> mmm kay
[04:55] <davecheney> which series does that assume ?
[04:56] <davecheney> lots of shit just expects precise
[04:56] <axw> you tell it which series
[04:56] <axw> looks like you pass "-r trusty" to lxc-create
[04:56] <davecheney> whoop whoop!
[05:42] <davecheney> axw: wallyworld nope, no joy
[05:42] <davecheney> still horribly broken
[05:43] <wallyworld> hmmm
[05:43] <wallyworld> it was broken for me until i updated everything yesterday
[05:43] <davecheney> ubuntu@winton-02:~$ sudo lxc-create -t ubuntu-cloud -n dave
[05:43] <davecheney> /usr/share/lxc/templates/lxc-ubuntu-cloud: line 276: type: ubuntu-cloudimg-query: not found
[05:43] <davecheney> lxc_container: container creation template for dave failed
[05:43] <wallyworld> and got lxc rc4 and rebooted
[05:43] <davecheney> lxc_container: Error creating container dave
[05:43] <davecheney> ^ this ?
[05:44] <davecheney> ii  lxc                                    1.0.0~rc4-0ubuntu1            ppc64el      Linux Containers userspace tools
[05:44] <wallyworld> i was getting a cgroups error
[05:44] <axw> davecheney: do you have a "ubuntu-cloudimg-query" command on your system?
[05:44] <davecheney> ii  lxc-templates                          1.0.0~rc4-0ubuntu1            ppc64el      Linux Containers userspace tools (templates)
[05:44] <davecheney> axw: i guess not
[05:44] <davecheney> does juju enforce this ?
[05:44] <davecheney> ie, is there an install dep ?
[05:44] <axw> hmm
[05:44] <axw> not sure
[05:44] <davecheney> ubuntu@winton-02:~$ sudo apt-get install ubuntu-cloudimg-query
[05:44] <davecheney> Reading package lists... Done
[05:44] <davecheney> Building dependency tree
[05:44] <davecheney> Reading state information... Done
[05:44] <davecheney> E: Unable to locate package ubuntu-cloudimg-query
[05:45] <axw> not a package
[05:45] <axw> just run the command
[05:45] <axw> should tell you what you need...
[05:45] <davecheney> not found
[05:45] <davecheney> and no sugestion
[05:45] <davecheney> do you have it ?
[05:46] <axw> yeah, how do I figure out what provides it again..
[05:46] <davecheney> dpkg -S $(which ubuntu-cloudimage-query)
[05:46] <wallyworld> davecheney: i just thought - i'm running a trusty host but am deploying precise charms and that worked for me as of late yesterday
[05:47] <davecheney> wallyworld: same
[05:47] <davecheney> this is a trusty host
[05:47] <wallyworld> ok, i was thinking you may have been trying trusy charms
[05:47] <axw> davecheney: cloud-image-utils
[05:47] <davecheney> well, i have to do that
[05:47] <wallyworld> yeah my host is trusty also
[05:47] <davecheney> but that is problem 2^17
[05:47] <davecheney> axw: ta
[05:48] <axw> and no, no check in the local provider
[05:48] <davecheney> axw: cock http://paste.ubuntu.com/6969306/
[05:53] <axw> bugger
[05:55] <davecheney> ok, the tasty-taco's are on it
[07:46] <axw> wallyworld: I think there may still be a problem with local upgrades, but not sure. Looks like a container in my local env is looking for downloaded-tools.txt in the host machine path
[07:47] <wallyworld> data dir is used for the place to look
[07:47] <axw> unit-axwtest-0: 2014-02-21 07:46:44 ERROR juju runner.go:220 worker: exited "upgrader": cannot read tools metadata in tools directory: open /home/andrew/.juju/local/tools/1.17.3.1-trusty-amd64/downloaded-tools.txt: no such file or directory
[07:47] <axw> unit-axwtest-0: 2014-02-21 07:46:44 INFO juju runner.go:254 worker: restarting "upgrader" in 3s
[07:47] <wallyworld> it comes from agent config file
[07:47] <wallyworld> that location looks ok
[07:47] <axw> no
[07:47] <axw> that's on machine 0
[07:47] <axw> the unit is in a container
[07:48] <axw> I mean, that path would be ok if it were on machine 0
[07:48] <wallyworld> what does the container agent conf file have
[07:48] <wallyworld> for data dir
[07:49] <axw> in upstart it has --data-dir /var/lib/juju as expected
[07:49] <wallyworld> hmmm.
[07:49] <wallyworld> that value is what is passed to the unit upgrader to use
[07:49] <wallyworld> ah
[07:49] <wallyworld> i mean machine agent
[07:50] <wallyworld> um
[07:50] <wallyworld> ffs
[07:50] <axw> this is a unit agent if it matters
[07:50] <wallyworld> there's a UnitUpgraderAPI
[07:50] <wallyworld> which is paased the data dir
[07:51] <wallyworld> which comes from the agent conf
[07:51] <wallyworld> so off hand i'm not sure what is going wrong
[07:51] <hazmat> g'morning
[07:51] <wallyworld> but i have to run away now to go to soccer
[07:51] <axw> nps, later
[07:51] <axw> hey hazmat
[07:52] <wallyworld> fwereade_: i have to go to soccer now and will miss stand up but want to talk about my latest machine agent upgrade work later, ok?
[07:52] <wallyworld> tests pass but i'm unhappy with 2 bits of the implementation
[07:54] <hazmat> axw, welcome to the world of picky certs :-) thanks for diving into that.
[07:54]  * hazmat has been having the same sort of cert fun with mysql ssl enablement
[07:54] <axw> no worries
[07:54] <axw> it's kind of interesting, but I wish rsyslog had better logging and less horrible configuration format :(
[07:55] <axw> it's all working now, just need to think about upgrades a bit
[07:55] <hazmat> axw, there's a major version change of rsyslog from precise to trusty 5.8 to 7.6
[07:56] <hazmat> might be worth verifying on both
[07:56] <axw> I've been testing with a trusty container and saucy host, local provider
[07:56] <axw> saucy is still 5.x
[07:56] <axw> but thanks for hte heads up
[07:58] <hazmat> axw, the notion of a worker seems like a sane path forward re syslog manage & upgrade. it also seems like it would be good on the road towards ha if there currently immutable.
[07:59] <axw> yeah, I think I will finish up the basics and then do that. there's no decent way of upgrading this stuff at the moment
[09:13] <wrtp> TheMue: ping
[09:17] <TheMue> wrtp: one moment, people at the door
[09:21] <TheMue> wrtp: so, back, we've got some troubles with our wastewater
[09:22] <wrtp> TheMue: my laptop has just broken, so i need to revert back to using my old mac, and i know you use some kind of VM - i wondered what you use, and if you recommend it
[09:24] <TheMue> wrtp: I'm using VMware Fusion and it works fine, only needs some RAM. At least 8 Gigs. Advantage is that you can do snapshot, e.g. before upgrading to new releases like trusty.
[09:25] <TheMue> wrtp: But it costs. I got it cheaper when it has been released the first time.
[09:27] <TheMue> wrtp: There are also Parallels and VirtualBox.
[09:28] <dimitern> axw, hey are you around still?
[09:28] <wrtp> TheMue: ok. i did have an old vmware version, but it suggested i upgrade it, but the upgraded version wasn't compatible with the old licence keys
[09:29] <wrtp> TheMue: so i've just bought a new version
[09:29] <wrtp> TheMue: and now i'm wondering how to get ubuntu running on it. ideally i'd like to be able to bootstrap from my laptop's hard drive, but i'm not sure that's possible
[09:30] <axw> dimitern: yup?
[09:31] <dimitern> axw, I saw your comments on my ssh fix
[09:31] <axw> oh yeah
[09:31] <dimitern> axw, if you have 10m I wanted to ask you to bring me up to speed with all the stuff around ssh
[09:31] <axw> sure
[09:31] <dimitern> axw, so we're using 2 different clients? why's that?
[09:31] <axw> for Windows
[09:31] <dimitern> ah
[09:32] <axw> so, because bootstrap now requires ssh, we *need* an ssh client
[09:32] <dimitern> right
[09:32] <axw> whereas before we could just say meh you can't have juju ssh
[09:32] <dimitern> how do you suggest i should test the fix to make sure i didn't introduce a regression?
[09:32] <axw> so now if we don't find "ssh" in $PATH, then we fall back to an embedded go.crypt/ssh client
[09:33]  * axw looks at the change again
[09:33] <dimitern> i suppose: ssh + scp commands, manual bootstrap + manual add-machine ssh, possibly debug-hooks.. windows?
[09:33] <axw> if ssh/scp aren't in your path, it'll behave like it would on Windows
[09:33] <dimitern> i found out scp -- <args> does not work btw
[09:33] <dimitern> ah, good point, I can fake it on ubuntu then
[09:34] <TheMue> wrtp: afaik it's not possible, you have to create a new image. but then you should be able to mount the old drive
[09:34] <axw> ah, so if we pass through -- then the embedded client will break
[09:34] <dimitern> wrtp, why the change of nick? going clandestine? ;)
[09:35] <dimitern> right!
[09:35] <dimitern> so basically I should test both client paths everywhere :)
[09:35] <axw> yeah, it's a pain in the bum but that's probably safest
[09:35] <dimitern> ok, thanks a bunch, will do
[09:36] <axw> so you'd probably want to compare what happens when you do "juju ssh 0 -- command" and "juju ssh 0 command"
[09:37] <dimitern> when is the -- jobbie supposed to work?
[09:37] <dimitern> with the embedded or the openssh client?
[09:37] <axw> openssh
[09:38] <dimitern> ok, i think that's all i need then
[09:38] <axw> that's just to say that the args won't go to ssh, but to the command it execs
[09:38] <dimitern> right
[09:40] <TheMue> wrtp: sorry for the breaks between my answers, have also to answer the craftsmen some questions around our problems
[09:40] <TheMue> s/wrtp/rogpeppe
[09:40] <rogpeppe> dimitern: wrtp used to be my nick, and i'm on my mac laptop which has my old configuration set up
[09:41] <dimitern> rogpeppe, i see
[09:43] <rogpeppe> hmm, i've managed to mount my laptop drive in my NAS box, but i only see a grub partition
[09:44] <rogpeppe> i wonder how i can get access to the main stuff
[09:46]  * TheMue is reminded to do a backup of his VMs after rogs troubles
[09:47] <TheMue> rogpeppe: the usage of VMs allows me to also have pure testing VMs, with no influence of the development VM
[10:46] <dimitern> rogpeppe, standup?
[10:46] <rogpeppe> dimitern: could you paste a link please?
[10:46] <dimitern> https://plus.google.com/hangouts/_/calendar/am9obi5tZWluZWxAY2Fub25pY2FsLmNvbQ.mf0d8r5pfb44m16v9b2n5i29ig
[10:47] <rogpeppe> "you are not allowed to join this video call"
[10:47] <rogpeppe> one mo while i get my auth key
[10:47] <rogpeppe> dimitern:  ^
[10:47] <dimitern> you need to login to the canonical account
[10:48] <rogpeppe> yeah
[11:51] <wallyworld> fwereade_: hi, do you have a minute to talk about my upgrade branch? I think i really want roger but he isn't logged on
[11:52] <fwereade_> wallyworld, I'm making an effort to program in the dark today but I'm actually already distracted, so let's
[11:52] <wallyworld> sorry :-(
[11:52] <wallyworld> https://plus.google.com/hangouts/_/calendar/am9obi5tZWluZWxAY2Fub25pY2FsLmNvbQ.mf0d8r5pfb44m16v9b2n5i29ig?authuser=1
[13:30] <wrtp> mgz: ping
[13:34] <wrtp> at long frickin' last!
[13:34] <wrtp> i have finally managed to get my laptop drive mounted in a VM
[13:34] <natefinch> wrtp: nice!
[13:35] <wrtp> this was the final key: http://www.tuxradar.com/answers/296
[13:35] <wrtp> but, honestly!
[13:47] <dimitern> natefinch, how can I build a windows installer from trunk?
[13:54] <natefinch> dimitern: you need innosetup (either in a windows VM or possibly Wine, I haven't tried Wine).  Build cmd/juju using cross compilation to make a windows binary, copy juju.exe to scripts/win-installer, then load scripts/win-installer/setup.iss into innosetup and build it
[13:54] <natefinch> dimitern: I should put that into a doc somewhjere
[13:55] <dimitern> natefinch, right, how to do the cross compilation?
[13:56] <natefinch> dimitern: http://dave.cheney.net/2013/07/09/an-introduction-to-cross-compilation-with-go-1-1
[13:56] <natefinch> dimitern: unfortunately, step 1 is: "install go from source"
[13:58] <dimitern> natefinch, ok, tyvm
[13:58] <natefinch> dimitern: welcom
[14:26] <mgz> wrtp: yeay!
[14:27] <mgz> can you work okay from the mac then?
[14:27] <wrtp> mgz: we'll see. it looks ok so far.
[14:28] <wrtp> mgz: much slower, but that's probably mostly the disk drive being spinning rust
[14:28] <mgz> do you have a date idea on when the repair will be done? I'm thinking about london...
[14:52] <wrtp> mgz: they said 6 working days
[14:52] <wrtp> mgz: so i'm crossing my fingers
[15:24] <wrtp> anyone remember how to get the required mongod when running under precise?
[15:24] <wrtp> mgz: ^
[15:24] <mgz> add the cloud-tools pocket and install from there
[15:25] <mgz> you can probably use the new juju-mongodb even
[15:25] <wrtp> mgz: how do you add the cloud-tools pocket?
[15:25] <wrtp> mgz: apt-add-repository cloud-tools-pocket ? :-)
[15:25] <mgz> `sudo add-apt-repository cloud-archive:tools`
[15:26] <mgz> the apt-get update and install juju-mongodb
[15:26] <wrtp> mgz: thanks
[15:27] <wrtp> mgz: (i have the binaries, but of course i'm running precise and the binaries are trusty 'cos they're on my laptop drive)
[15:28] <wrtp> mgz: i tried fiddling with LD_LIBRARY_PATH but no joy. i love shared libraries
[15:30] <wrtp> mgz: hmm; http://paste.ubuntu.com/6971425/
[15:31] <wrtp> and /etc/apt/sources.list doesn't even have 60 lines :-\
[15:31] <mgz> well, that's amusing.
[15:31] <mgz> look in /etc/apt/sources.liust
[15:32] <mgz> *look in /etc/apt/sources.list.d/
[15:32] <wrtp> cloudarchive-tools.list is there
[15:32] <mgz> can you pastebin that?
[15:32] <wrtp> mgz: does that mean it's actually worked, despite the error message?
[15:33] <wrtp> deb http://ubuntu-cloud.archive.canonical.com/ubuntu precise-updates/cloud-tools main
[15:33] <wrtp> deb-src http://ubuntu-cloud.archive.canonical.com/ubuntu precise-updates/cloud-tools main
[15:33] <mgz> it does if apt-get update works, if it doesn't then presumable it borked
[15:33] <mgz> that seems fine
[15:34] <wrtp> well, apt-get update works, so i guess it worked :-)
[15:34] <wrtp> mgz: except it doesn't seem to know about juju-mongodb
[15:34] <mgz> bug 789859 it seems
[15:34] <_mup_> Bug #789859: add-apt-repository adds invalid deb line if PPA contains a - <amd64> <apport-bug> <natty> <software-properties (Ubuntu):Fix Released> <software-properties (Ubuntu Precise):Triaged> <software-properties (Debian):Invalid> <https://launchpad.net/bugs/789859>
[15:35] <sinzui> mgz, do you have a few minutes to review https://codereview.appspot.com/66780046
[15:35] <mgz> sinzui: sure thing, but natefinch might be a better bet if he's around
[15:36] <mgz> ..actually, that's trivial
[15:36] <mgz> sinzui: lgtm
[15:36] <mgz> I presume the version bump in the go code already happened?
[15:36] <sinzui> wrtp, when this bug is fixed, we will release a new juju that is juju-mongodb aware: https://bugs.launchpad.net/juju-core/+bug/1271937
[15:36] <_mup_> Bug #1271937: Use juju-mongodb when the package is available <arm64> <ci> <mongodb> <ppc64el> <trusty> <juju-core:In Progress by natefinch> <https://launchpad.net/bugs/1271937>
[15:39] <mgz> wrtp: does `grep -r "^in" /etc/apt/sources.list*` give anything?
[15:39] <wrtp> mgz: nope
[15:39] <mgz> well, lets assume it was transient weirdness then
[15:39] <wrtp> mgz: and the only entry in sources.list.d is the cloudarchives-tools.list file
[15:40] <wrtp> mgz: except that... i can't apt-get juju-mongodb
[15:40] <wrtp> mgz: apt-cache search juju doesn't print anything that looks reminiscent
[15:41] <mgz> yeah, it seems it only has the normal mongdb packages at present
[15:41] <mgz> so, just install mongodb (or just -server)
[15:41] <wrtp> mgz: so if i apt-get install mongodb, will i get the cloud-tools mongodb
[15:41] <wrtp> ?
[15:42] <mgz> yup
[15:42] <wrtp> mgz: ok, i'll try that
[15:42] <mgz> it will also upgrade if you have the precise one
[15:42] <wrtp> mgz: yeah, it looks like it's fetching from the right place, ta
[15:42] <mgz> should be 1:2.4.6-0ubuntu5~ctools1
[15:44] <wrtp> mgz: i see that
[15:48] <wrtp> mgz: at last! i have juju tests passing under my VM
[15:48] <wrtp> mgz: thanks for the help
[15:48] <mgz> ace!
[15:54] <Beret> sinzui, when do you expect .3?
[15:54] <Beret> I can't remember
[15:54] <sinzui> hmm, between 2 and 20 hours.
[15:54] <sinzui> Beret, everything in in the Lp builders as of 5 minutes ago
[15:56] <Beret> sinzui, awesome, thanks :)
[15:57]  * wrtp uses chroot(1) for real for the first time ever
[15:59] <mgz> rogpeppe: I've used it a fair bit from inside chromeos recently when I needed to restore my ubuntu partition :)
[16:00] <rogpeppe> mgz: aw dammit, "no ptys"
[16:00] <rogpeppe> mgz: i can't run a terminal emulator in it
[16:02] <mgz> really? what's the underlying terminal?
[16:03] <natefinch> sinzui: btw I am trying to land my "use juju-mongod" branch, but hit some spurious test failures.  Going to try again in a little bit, have to run to the store real quick.
[16:03] <sinzui> thank you natefinch
[16:12] <dimitern> sinzui, sorry, it seems I won't be able to land the fix for bug 1281577 in time for 1.17.3, as I said in the xteam doc
[16:12] <_mup_> Bug #1281577: juju 1.17.2 doesn't pass command line options to ssh <docs> <landscape> <regression> <ui> <juju-core:In Progress by dimitern> <https://launchpad.net/bugs/1281577>
[16:14] <mgz> sinzui: have you picked a trunk rev already, or will you go with what we have in a few hours?
[16:14] <sinzui> dimitern, mgz, 1.17.3 tarball is released.
[16:15] <sinzui> dimitern, I will release 1.17.4 when I see natefinch's branch land and CI confirms it works with the new packaging rules
[16:15] <sinzui> dimitern, So I may release 1.17.4 tomorrow, certainly next week
[16:15] <dimitern> sinzui, ok, great
[17:47] <rogpeppe> oh bugger, my disk enclosure seems to have stopped working
[17:47] <rogpeppe> i hope it hasn't killed the disk
[17:50] <mgz> >_<
[18:02] <rogpeppe> mgz: ah, it seems that the VMWare USB stuff is extremely sensitive to unplugging. when i unconfigured the usb, shut everything down, and started it all up in just the right order, it's working again
[18:02] <rogpeppe> phew
[18:02] <rogpeppe> but time to start a backup going, i think
[18:20] <sinzui> natefinch, thank you. I will push the new packaging rules
[18:33] <natefinch> rogpeppe: sounds like one of those things where, once you get it working you don't. touch. anything.
[18:34] <rogpeppe> natefinch: yeah. the connectors seem dodgy... both s/w and h/w
[18:55] <rogpeppe> right, i'm done
[18:55] <rogpeppe> happy weekends all
[18:57] <rogpeppe> question to ponder: anyone know why StatePort is no longer stored in the agent config?
[18:58] <mgz> hm... nope
[18:58] <mgz> is it just a constant now?
[19:05] <natefinch> rogpeppe: I think stateport was something I added with the new machine agent  stuff and that it wasn't there before.... and since that code isn't merged yet, it's not in trunk
[19:16] <natefinch> mgz: have time for a quick review?  https://codereview.appspot.com/67080043/
[19:17] <mgz> natefinch: will do in a bit
[19:54] <rick_h__> anyone able to give me hints on getting the error "ERROR empty image-stream in environment configuration" during a deploy command?
[19:55] <rick_h__> this is on ec2 during a test of our charm and everything I see around image-stream seems based on azure
[19:55] <rick_h__> my env config has no image-stream config in it for ec2 env
[19:55] <rick_h__> http://paste.ubuntu.com/6972735/ is the command run and error returned
[20:03] <rick_h__> trying it by hand I get "ERROR juju.cmd supercommand.go:294 empty image-stream in environment configuration"
[20:11] <dpb1`> rick_h__: check .juju/environments/ec2.jenv
[20:11] <rick_h__> dpb1`: ah, good call. Regenerating that and trying again
[20:12] <dpb1`> dpb1`: k
[20:12] <dpb1`> blah
[20:12] <dpb1`> rick_h__: ok. :)
[20:22] <rick_h__> dpb1`: ok, so I get that error on the juju deploy command. There's no image-stream set in the  environments.yaml or ec2.jenv
[20:22] <rick_h__> if I try to set the image-stream to : or : "" I get errors on bootstrap about it being a bad value
[20:23] <dpb1`> rick_h__: is it possible for you to paste your environments.yaml (sanitized)?
[20:23] <dpb1`> just that section
[20:24] <natefinch> rick_h__: setting it to "released" should work, though not set should be equivalent to "released"
[20:24] <rick_h__> dpb1`: sure thing, sec
[20:24] <rick_h__> natefinch: yea, that failed for me as well. I saw that in the azure stuff
[20:24] <rick_h__> I'm on 1.17.2-trusty-amd64
[20:25] <rick_h__> http://paste.ubuntu.com/6972856/ dpb1`
[20:32] <rogpeppe> natefinch: stateport is in 1.12 serialization but not 1.16
[20:37] <natefinch> rogpeppe: ok, I just know I added stateport to something in my changes, couldn't remember what.
[20:40] <natefinch> rick_h__: add in a --show-log to the deploy command and it'll spit back a bigger log that may be helpful
[20:46] <rick_h__> natefinch: http://paste.ubuntu.com/6973005/
[20:49] <natefinch> rick_h__: well that's less than useful, huh? :/
[20:50] <rick_h__> :)
[22:17] <rick_h__> sinzui: is .3 in trusty or on the way? to test for #1277636
[22:17] <_mup_> Bug #1277636: juju 1.17.x cannot destroy a 1.16.5 azure environment <azure-provider> <ci> <destroy-environment> <regression> <juju-core:Fix Released by axwalk> <https://launchpad.net/bugs/1277636>
[22:17] <rick_h__> all of us hitting the bug are trusty users on .2 currently
[22:29] <sinzui> rick_h__, It in the the public ppa. Ubuntu are responsible for putting it in trusty
[22:38] <sinzui> rick_h__, I am sorry for trusty users being forced test unstable software. My plan was to never let it devel releases go into an LTS. You can find the new release in https://launchpad.net/~juju/+archive/devel/+packages for now
[22:38] <rick_h__> sinzui: rgr, thanks
[22:58] <sinzui> rick_h__, you may already know about this http://manage.jujucharms.com/heartbeat
[22:58] <sinzui> jujucharms.com is not happy, looks like manage...is at fault
[22:58] <rick_h__> sinzui: working with webops on it and running out of time to get a fix :/
[22:59] <rick_h__> ES voted out all the shards again but the fix from last time is not working
[22:59] <sinzui> rick_h__, The fix was to put the other values back...the 3 mintues later all was good
[22:59] <sinzui> Is that the fix tried?
[23:00] <rick_h__> sinzui: "values back"?
[23:00] <rick_h__> sinzui: last time we removed /var/lib/elasticsearch and restarted it and then restarted charmworld and reran es-update and it rebuilt itself
[23:01] <rick_h__> I'm trying to figure out how/what command to send to ES via the JSON api to get it to allocate a shard
[23:01] <sinzui> rick_h__, the first time it happen, it was caused when the charm config value was changed. It didn't need to be, if you know what the old value was, setting the config back restored it
[23:02] <rick_h__> sinzui: oh hmm, no config changed I'm aware of. Will see if we can get what's set and compare.
[23:50] <rick_h__> sinzui: should be back now
[23:50] <sinzui> thank you rick_h__
[23:51] <rick_h__> sinzui: turns out they run mojo which runs CI and part of that brings up a new ES server. Which any other ES servers on the network try to join with and caused things to go badly.
[23:51] <rick_h__> loves ES and it's auto scaling stuff
[23:51] <sinzui> rick_h__, ha ha
[23:51] <rick_h__> kind of funny that CI passes while production goes down
[23:52] <sinzui> rick_h__, that was the reason we added the shard and names parts to charmworld, to ensure the names are always unique
[23:52] <rick_h__> yep, the names in CI are getting updated now
[23:52] <sinzui> Their CI, has to use different names or run on different networks per ES
[23:52] <rick_h__> yep
[23:52] <rick_h__> that's the great discovery of the day