[00:00] <anastasiamac> axw: sure, i'll look \o/
[00:10] <axw> anastasiamac: if you liked that PR, you'll love this one: https://github.com/juju/juju/pull/7216 ;)  if you have any time... it's kinda big
[00:11] <anastasiamac> axw: i saw and got excited but it may need to wait until after lunch... if it's k
[00:11] <axw> anastasiamac: no worries. finally going to look at 1.8 test failures this morning
[00:12] <anastasiamac> axw: \o/
[00:13] <anastasiamac> axw: i was going to say "unless u want to figure out why multiwatcher is not seeing changes"... but m not even convinced that this si an issue m looking at now... is there any way that transaction integrity could b violated in some way?
[00:13] <anastasiamac> axw:  may i need more coffee too..
[00:13] <anastasiamac> maybe*
[00:14] <axw> anastasiamac: I don't know
[00:22] <axw> veebers: are you aware that qa.jujucharms.com is unavailable?
[00:23] <anastasiamac> axw: yes :) veebers is working on resurecting it.. apparently 1.25 is involved :D
[00:24] <axw> anastasiamac: okey dokey
[00:26] <veebers> axw: aye, looking. (as anastasiamac says)
[00:31] <anastasiamac> axw: reviewed (could not resist) :D
[00:31] <axw> anastasiamac: thanks :)
[02:36] <axw> anastasiamac: interestingly, I have no applications in the modelEntityRefs collection for the model that's looping
[02:37] <axw> anastasiamac: nor machines, volumes, filesystems
[02:37] <anastasiamac> axw: i do occasionally... yeah no machines/volumes/filesystem but "ubuntu' as an app makes an appearance
[02:39] <anastasiamac> axw: so the best i can say that application get removed at/before/after watcher is created... ?
[02:39] <anastasiamac> axw: no changes to app collection come thru, but the first check to see if model is empty still believes that there is an app in entityrefs tables
[02:39] <axw> anastasiamac: seems that restarting the controller agent resolved it in this case
[02:39] <anastasiamac> axw: right
[02:40] <anastasiamac> axw: because the chemckmodelempty so that there is nothing for apps in modelentityrefs coz "lingering" app was removed
[02:47] <anastasiamac> axw: m reb-bootstrapping with status update to display the actual erro that we get when we r infitely waiting for changes :)
[02:48] <anastasiamac> axw: it's https://github.com/juju/juju/blob/develop/state/model.go#L1047
[02:49] <anastasiamac> axw: and becuase we only re-check if model is ready to die if/when we get changes, we'll never re-check agains coz we have not seen any changes on application and machine collections
[02:53] <axw> anastasiamac: because they're separate collections, it is theoretically possible that we'll observe the app removal before the ref removal
[02:54] <axw> anastasiamac: I would've thought it'd happen too quickly, but it is theoretically possible
[04:20] <mup> Bug #1615986 changed: Agents failing, blocked by upgrade but no upgrade performed <canonical-is> <juju:Invalid> <juju-core:Expired> <juju-core 1.25:Expired> <https://launchpad.net/bugs/1615986>
[04:20] <mup> Bug #1661681 changed: Broken agent complaints about tomb: dying <juju-core:Expired> <https://launchpad.net/bugs/1661681>
[05:40] <axw> anastasiamac: teeny weeny review? https://github.com/juju/juju/pull/7217
[05:42] <anastasiamac> axw: will look in a sec... m just about to propose my changes and will need u to return a favor :D
[05:43] <axw> anastasiamac: okey dokey
[05:54] <veebers> Quick Q, how can I tell if a filesystem on an AWS instance is persistent? I need to backup the juju db so I can safely wipe it (to get replication started again)
[05:55] <jam> axw: reviewed 7217
[05:55] <veebers> I have limited '/' space, but see "/dev/xvdb       3.9G  8.1M  3.7G   1% /mnt" which would be enough for the back up. Can I be sure that's not going to disappear on me?
[05:55] <axw> jam: thanks
[05:55] <anastasiamac> jam: tyvm!
[05:55] <jam> axw: I'm concerned that the proxy changes indicate a real problem that we need to understand
[05:56] <veebers> axw: would you know? ^^
[05:58] <axw> veebers: sorry I don't understand your first question about persistence
[05:59] <veebers> axw: ah right, I'm concerned that if I move the backup to that filesystem (under /mnt) if the machine restarts will that file system be there again (or is it some semi-permanent FS that gets lost on reboot
[05:59] <veebers> axw (it's an aws question, not juju :-))
[06:00] <veebers> I only ask because I saw this: https://forums.aws.amazon.com/thread.jspa?messageID=562254 where they say they rebooted and lost their /dev/xvdb
[06:02] <axw> veebers: I see. IIRC the cloud images mount the ephemerical block device at /mnt. you should be able to confirm by looking in the ec2 console, and looking at the volumes associated with the instance
[06:02] <axw> veebers: IOW, probably not persistent, but you can check that way
[06:02] <veebers> axw: ack will check. Thanks
[06:10] <axw> jam: I've responded to your comments
[06:12] <axw> anastasiamac: and I've responded to your comments on the vsphere PR, if you can please see my replies before I type $$billz$$
[06:12] <anastasiamac> axw: sure thing, gimme a sec - almost there :D
[06:21] <anastasiamac> axw: jam: PTAL https://github.com/juju/juju/pull/7218
[06:23] <axw> anastasiamac: looking
[06:24] <axw> anastasiamac: presumably you've tested, and you're not seeing failures anymore?
[06:24] <anastasiamac> axw: thnx :) m happy with 7216
[06:25] <anastasiamac> axw: yes, tested :) we also have CI tests that exercise different scenarios than the one i've used.. i cannot fail it any more... but if u r keen, feel free to run it too :D
[06:28] <jam> axw: I have comments, I feel like we should probably just try to chat about them directly.
[06:28] <jam> I have to start up a long-running test here, once I get it kicked off, I think a hangout/IRC chat is appropriate
[06:29] <axw> jam: sure
[06:32] <axw> anastasiamac: a few little things, main thing is to move the settings changes out
[06:33] <anastasiamac> axw: thnx, i'll look :D
[06:50] <jam> anastasiamac: i had a couple comments as well
[06:52] <anastasiamac> jam: \o/
[08:12] <anastasiamac> jam: axw: PR 7218 updated, manual tests pass... happy for me to land?
[08:12] <axw> anastasiamac: yup, it's accepted
[08:12] <anastasiamac> axw: awesome \o/
[08:26] <anastasiamac> axw: jam: the bit that was a drive-by
[08:26] <anastasiamac> https://github.com/juju/juju/pull/7219
[08:57] <axw> jam: ready to chat yet?
[08:57] <jam> axw: ah sorry, yeah, let's go ahead and do it, I'll interrupt my other interruptions :)
[08:58] <jam> axw: https://hangouts.google.com/hangouts/_/canonical.com/jam-axw?authuser=1
[08:58] <axw> jam: IRC is fine for me, then you can multi task
[08:58] <axw> or that
[12:44] <rogpeppe> anyone care for a review of some changes to some of the API testing code? https://github.com/juju/juju/pull/7222
[18:28] <perrito666> hey annybody here?
[18:29] <rick_h> perrito666: party
[18:29] <perrito666> rick_h: hehey, do you happen to know if curtis or aaron are on?
[18:30] <rick_h> perrito666: not sure, /me looks for abentley_ and sinzui (not here)
[18:31] <abentley_> perrito666: Hi.
[18:32] <perrito666> abentley_: hello :) I had not seen you with that underscore
[18:32] <perrito666> abentley_: I am looking with someone with privileges to remove me from canonical organization in github, juju is quite spammy of my email  :)
[18:35] <abentley_> perrito666: Are you sure that will have the desired effect?  AIUI, Juju is under the Juju team, not Canonical.
[18:36] <rick_h> perrito666: let me know which org and I can pull it
[18:37] <perrito666> rick_h: I did not know you had that kind of power ;)
[18:37] <rick_h> perrito666: :)
[18:38] <rick_h> perrito666: removed from both
[18:39] <perrito666> rick_h: ah cool, now I can actually see notifications from my own projects :) cheers
[18:39] <rick_h> perrito666: party on
[23:44] <menn0> anastasiamac: while testing some stuff I kept running into the same issue you've been looking at (can't destroy model when there's an app with no units)
[23:45] <menn0> anastasiamac: restarting the controller is an inconvenient workaround
[23:45] <anastasiamac> menn0: on tip?
[23:45] <menn0> thumper: ^^^
[23:45] <anastasiamac> menn0: coz i've landed a fix yesterday
[23:46] <menn0> anastasiamac: hmmm
[23:46] <menn0> anastasiamac: not sure if this was with your fix or not
[23:46] <menn0> anastasiamac: i'll try again
[23:49]  * anastasiamac holding breath and looks over menn0's shoulder...
[23:52]  * menn0 has also managed to get the juju client into a state where "juju models" segfaults
[23:52] <menn0> will look into that soon
[23:53] <anastasiamac> thumper: menn0: wallyworld_:axw: here is an interesting question: should retry -provisioing work on containers?
[23:53] <menn0> anastasiamac: I would have thought so
[23:53] <axw> anastasiamac: I don't see why not
[23:53] <anastasiamac> menn0: I was going to look into juju models in general.. we have a lot of different failure scenarios for it.. but feel free to lend a hand :D
[23:53]  * menn0 has no time for any of this
[23:53] <anastasiamac> menn0: axw: k, I'll triage it as a wishlist
[23:54] <axw> anastasiamac: seems like a bug to me.
[23:54] <anastasiamac> menn0: don't worry about list models then :D i'll address (at some stage)... but really really want to know if u cannot destory model still
[23:54] <anastasiamac> axw: does it? do we have code that allows it to cater for containers already?
[23:55] <wallyworld_> it wasn't designed to work with containers
[23:55] <wallyworld_> but it could
[23:55] <axw> anastasiamac: I don't know, but how is that relevant? lack of code does not mean it's not a bug
[23:55] <wallyworld_> it was designed to handle transient cloud failures
[23:56] <wallyworld_> containers should not have the same class of transient errors
[23:57] <axw> wallyworld_: why? the host/container system can fail in transient ways too
[23:57] <wallyworld_> "the same class of"
[23:57] <wallyworld_> but yeah
[23:57] <wallyworld_> there's no reason why the code can't be extended
[23:58] <wallyworld_> i think there's an IsTransient method or something on an error?
[23:58] <anastasiamac> axw: lack of code indicates an RFC - feature improvement: not a bug but wishlist: "fill this functional gap for me" :D
[23:58] <wallyworld_> i can't recall if that's what is checked
[23:58] <axw> anastasiamac: a design bug is still a bug.
[23:58] <anastasiamac> axw: and when i say wishlist, it does not mean we should not address :)