[06:43] <stub> skay_: I don't think the snap layer should support private snaps. It would seem to be a major security problem, where if any machine running the private snap got compromised then it could be escalated to all machines running the private snap (if you can download a private snap, you can push updates to it)
[06:46] <stub> skay_: I don't know about unpublished snaps. How do you install them normally? Do you just provide an explicit revision? I'd been thinking for that sort of use case people would use the Juju resource functionality. But I don't know your particular use case :)
[06:51] <stub> skay_: The code is all at https://code.launchpad.net/layer-snap ( git+ssh://git.launchpad.net/layer-snap ), and a mirror on github at https://github.com/stub42/layer-snap
[07:02] <stub> (please correct me if I'm wrong on private snaps, as we have avoided snaps for some projects because of the issue)
[09:13] <kjackal_> Good morning Juju world!
[09:14] <kjackal_> is https://jujucharms.com/store available on your side or is it only me?
[09:14] <kjackal> Good morining Juju World!
[09:15] <kjackal_> nm somethis was wrong with my vpn
[13:21] <SimonKLB> is it possible to run the upgrade-charm hook when a relation-joined hook have failed?
[13:21] <SimonKLB> or is it required that the relation-joined hook is fixed before it can continue?
[13:21] <SimonKLB> and if so, how are you supposed to fix it without upgrade-charm?
[13:22] <SimonKLB> (i've tried upgrade-charm --force-units)
[13:25] <SimonKLB> forgot to add, the error occurs because of a lib in the wheelhouse, and therefore the upgrade-charm hook is required to install the new lib which includes the fix
[13:27] <stub> SimonKLB: No hooks will run while the unit is in an error state. I think you are stuck with 'juju resolved' to clear the error state, repeating as necessary. Then dropping the relation and doing the upgrade (or doing the upgrade and then dropping the relation).
[13:28] <stub> SimonKLB: You might be able to repair the relation, but you are probably better off dropping and recreating it
[13:36] <SimonKLB> stub: yea, i fixed it by removing the relation, upgrading and adding it again - but is there a reason that the upgrade operation is part of the normal flow? is it due to race conditions between hooks and upgrades or something else?
[13:37] <stub> I don't know the official reasons, but I would think that if a hook is in an error state then the system is in an indeterminate state and Juju isn't going to risk doing anything.
[13:40] <SimonKLB> stub: okok, it's just frustrating that you have a fix that you want to apply but you cant upgrade the charm because of the error you're trying to solve :D
[13:41] <SimonKLB> stub: btw, have you had any time recently to look at relative imports in reactive?
[13:41] <stub> SimonKLB: My PR is still in the charms.reactive queue
[13:42] <stub> https://github.com/juju-solutions/charms.reactive/pull/51
[13:42] <SimonKLB> stub: yea, does it simply not have enough prio or is it something that is blocking it?
[13:43] <stub> SimonKLB: I think it is in Corey and Ben's hands now. I havn't been nagging about that one, as I have a work around and other PRs I need landing first ;)
[13:45] <SimonKLB> stub: haha yea, i really want to get rid of that hotfix in lib/reactive... :)
[13:48] <stub> But yeah, six months without comments.... not exactly encouraging contributions :-(
[14:26] <marcoceppi> SimonKLB: you can do `juju upgrade-charm --force-units` which will unpack the charm payload regardless of hook state
[14:26] <marcoceppi> so if you patch a relation hook, and want to test with the failed state, just run that then a juju resolved
[14:26] <marcoceppi> stub: ^^ fyi
[14:29] <SimonKLB> marcoceppi: in case the fix is in the charm code yes, but if it's in a lib in the wheelhouse that doesn't seem to do it since the wheelhouse is only updated in the upgrade-charm hook
[14:30] <marcoceppi> SimonKLB: that's a good point, you can however, if you have a debug-hooks session open
[14:30] <marcoceppi> just run the upgrade-charm hook despite being in the relation context
[14:30] <SimonKLB> marcoceppi: how do i execute a specific hook when im in the debug mode? ive only used it to catch a hook when it's firing
[14:31] <marcoceppi> SimonKLB: right, so you breakpoint the hook execution ,after an upgrade-charmm --force-units
[14:31] <marcoceppi> SimonKLB: you're now in the relation hook even
[14:31] <marcoceppi> event*
[14:31] <marcoceppi> SimonKLB: but you can just run `hooks/upgrade-charm` despite not being in that hook context
[14:45] <skay> stub: I decided to forgo the concept of a private snap and will just upload the file as a release download on launchpad
[15:00] <BlackDex> Hello there, can i prevent an lxd container from being connected to a specific network space?
[15:01] <BlackDex> it doesn't seem to matter if i use --bind or not
[15:01] <BlackDex> it always connects to all available networks
[15:10] <SimonKLB> marcoceppi: so now i have a relation-joined error, i enter `juju debug-hooks` and execute `juju upgrade-charm`, and then `juju resolved`
[15:10] <SimonKLB> marcoceppi: still i get into the relation-joined hook, not the upgrade-charm hook
[15:10] <marcoceppi> SimonKLB: that's okay
[15:11] <marcoceppi> did you do upgrade-charm with --force-units?
[15:11] <SimonKLB> marcoceppi: yep
[15:11] <marcoceppi> SimonKLB: so, if you now run `hooks/upgrade-charm` in your debug window
[15:11] <SimonKLB> marcoceppi: how do i get from here to have the wheelhouse install again?
[15:11] <marcoceppi> your new deps will be unpacked
[15:11] <SimonKLB> aaah, perfect!
[15:11] <SimonKLB> thanks
[17:02] <petevg> cory_fu, bcsaller: filed the "failed status after reboot" bug here: https://bugs.launchpad.net/juju-core/+bug/1649637
[17:02] <mup> Bug #1649637: Juju agent in a "failed" state after machine reboot on some charms <juju-core:New> <https://launchpad.net/bugs/1649637>
[17:55] <petevg> cory_fu: https://github.com/juju-solutions/matrix/pull/43 LGTM
[17:55] <petevg> cory_fu: do you want to keep your commits unsquashed?
[17:57] <cory_fu> petevg: Oh, no.  I'll squash them.  thanks for catching that
[17:58] <petevg> cory_fu: cool. I'll let you do the squash and merge on github, since you know which comments you want to keep :-)
[18:08] <kwmonroe> hey cory_fu, i have a curiousity.  for a subordinate with use_venv=true and include_system_packages=false, how is python able to import sys prior to activate_venv?  https://github.com/juju-solutions/layer-basic/pull/80/files#diff-0afefcaa00589c20f3d956fa58ddbbe7R97
[18:20] <cory_fu> kwmonroe: It gets run with the system Python, and the activate_venv re-execs it using the venv's python
[18:21] <cory_fu> kwmonroe: So, you can import the standard modules but anything that's only installed in the venv will fail to import until after the activate_venv()
[18:22] <kwmonroe> cory_fu: if i import stuff before reload_interpreter, will those imports still be known by vpy?
[18:22] <cory_fu> kwmonroe: Also, anything installed in the system python (such as by the principal charm, if it's not also using venv) will be available before activate_venv, which is why hadoop-plugin is working by accident right now
[18:23] <kwmonroe> ack cory_fu.  i'm kinda surprised at that one given hadoop-client (i thought) used a different major version of jujubigdata.
[18:24] <cory_fu> kwmonroe: What actually happens, is that it runs through the lines up to reload_interpreter under the system python interpreter, which then reinvokes the program using the venv python interpreter.  That new interpreter then re-does everything from the start, but activate_venv includes a check to skip reload_interpreter if it's already using the venv interpreter
[18:25] <kwmonroe> ah, neat
[18:25] <cory_fu> kwmonroe: Another option would be to change the shebang lines on the actions to point to the venv: #!../.venv/bin/python
[18:27] <cory_fu> kwmonroe: Then we could leave out the call to activate_venv entirely
[18:28] <cory_fu> petevg: Oh, poop.  bcsaller went and merged before I could squash those commits
[18:28] <bcsaller>  oh, sorry
[18:29] <cory_fu> bcsaller: No worries.  The "wip" commits are yours anyway.  ;)
[18:30] <petevg> Oh well. A few "wip" comments don't destroy the git log or anything :-)
[19:36] <kjackal> kwmonroe: did you test the change you merged here: https://github.com/juju-solutions/layer-cwr/pull/11/files#diff-07490436a8da97cc3ca6df82ae11568eR31
[19:36] <kjackal> kwmonroe: ^ ?
[19:38] <kjackal> kwmonroe: from what I've seen https://github.com/juju-solutions/layer-cwr/pull/11/files#diff-794ba32b8d472d3963159bcfcf069f8dR19 needs a string without the "cs:"
[19:39] <kwmonroe> kjackal: i did -- this works for me:  >>> yaml = cs.files('cs:~bigdata-charmers/hadoop-processing', filename='bundle.yaml', read_file=True)
[19:41] <kjackal> kwmonroe: strange, but I trust you!
[19:41] <kwmonroe> kjackal: i was wondering why you wanted to strip 'cs:' so badly.  i'll keep poking on it today with different bundle strings, but like i said, it works for me with a 'cs:' prefix.
[19:42] <kjackal> kwmonroe: https://api.jujucharms.com/charmstore/v5/cs:~kos.tsakalozos/bundle/juju-ci-bundle-3/meta/manifest gives out a Message": "not found: URL without charm or bundle name: \"cs:~kos.tsakalozos\"",
[19:42] <kjackal> but https://api.jujucharms.com/charmstore/v5/~kos.tsakalozos/bundle/juju-ci-bundle-3/meta/manifest is fine
[19:43] <kjackal> kwmonroe: Thats with using postman, it might be that the charmstore lib is doing some cleanup
[19:44] <tvansteenburgh> like url-encoding the : for example
[19:47] <kjackal> I was trying the url format we initialy had on the actions.yaml (bundle:....), that did not work so I just went to see what the rest API of the charmstore
[19:47] <kjackal> that says id/whatever
[19:49] <kwmonroe> oh neat kjackal -- i can get your failure with the 'bundle:' syntax
[19:58] <kwmonroe> kjackal: let's go with "~<user>/<bundle>" as the documented syntax, and strip cs: or bundle:.  that will keep theblues happy, and bundletester already handles ~bundle syntax, so we don't need to re-add a prefix there.
[20:09] <kjackal> kwmonroe: if it works it is fine with me
[20:34] <BlackDex> what does the * mean after some of the units in juju 2.0?
[20:39] <jrwren> BlackDex: it is the elected leader.
[20:39] <BlackDex> Ah :)
[21:00] <BlackDex> If i want to upgrade juju i get the following
[21:00] <BlackDex> juju upgrade-juju
[21:00] <BlackDex> ERROR a hosted model cannot have a higher version than the server model: 2.0.2 > 2.0.1
[21:03] <rick_h> BlackDex: the controller has to be the highest one
[21:03] <rick_h> BlackDex: so have to upgrade the controller and then the model you're on?
[21:04] <rick_h> BlackDex: a model can't use features the controller api doesn't know how to deal with
[21:05] <BlackDex> show how do i upgrade the controller?
[21:31] <anastasiamac> petevg: kjackal: kwmonroe: cory_fu: I think we have a zookeeper bug - https://bugs.launchpad.net/juju-core/+bug/1649637 ... any advice?
[21:31] <mup> Bug #1649637: Juju agent in a "failed" state after machine reboot on some charms <juju-core:Incomplete> <https://launchpad.net/bugs/1649637>
[21:32] <cory_fu> anastasiamac: You've got that backward.  It *doesn't* happen for ZK.  It does happen consistently for both mysql and mediawiki, and those were just the three we tested
[21:32] <petevg> anastasiamac: zookeeper works. wiki and MySQL don't.
[21:33] <cory_fu> anastasiamac: Also, I don't see any way that the payload could affect the agent, so I'm actually surprised that ZK did work
[21:33] <petevg> Yeah. It's puzzling ...
[21:33] <anastasiamac> cory_fu: interesting :) so if it works for latest zookeeper and not other charms, would it not mean that the other charms ned to be updated?...
[21:33] <anastasiamac> need*
[21:34] <cory_fu> anastasiamac: Updated how?  How are the charms supposed to influence the juju agent?
[21:34] <cory_fu> anastasiamac: One difference is that both mysql and mediawiki are trusty.  Perhaps this is a trusty vs xenial issue?
[21:35] <anastasiamac> cory_fu: ah... that is an **interesting** thought.. do they not have xenial versions? would b interesting to see if they can work
[21:36] <cory_fu> anastasiamac: They do not
[21:39] <anastasiamac> cory_fu: petevg: thank you :) i'll update the bug \o/
[21:39] <cory_fu> anastasiamac: I'm testing it now with cs:ubuntu-8 on both trusty and xenial to confirm if the same charm behaves differently by series
[21:40] <anastasiamac> cory_fu: u r amazing! tyvm :D
[21:41] <cory_fu> It did.  Trusty failed and Xenial succeeded.  I also got an "action terminated" response from the `juju run ubuntu-xenial/0 'sudo reboot'` command but nothing from the one on trusty
[21:42] <anastasiamac> cory_fu: it's great that we have a confirmation \o/ it's sad that it's happening.. I'll add all this to the bug unless u r keen to do it urself
[21:42] <cory_fu> anastasiamac: Already typing it up
[21:43] <anastasiamac> cory_fu: my hero \o/
[21:43] <cory_fu> anastasiamac: :)  Posted
[21:43] <anastasiamac> cory_fu: tyvm
[21:44] <cory_fu> anastasiamac: np.
[21:44] <petevg> Thx, cory_fu (also, thx anastasiamac, for looking at the bug quickly)
[21:45] <cory_fu> petevg: I'm glad we have a reasonable explanation for the different behavior.  Also, the fact that I got "action terminated" from the xenial unit makes me think that the libjuju call wouldn't hang for those either
[21:46] <petevg> Yeah. Explanations are nice.
[21:47] <petevg> cory_fu, tvansteenburgh: are we trying to keep python-libjuju relatively dependency free? I'm working on fixing an issue with timestamps inside of unit.py, and I'm very tempted to just add python-dateutil as a dependency ...
[21:47] <tvansteenburgh> petevg: the fewer the better imo
[21:49] <cory_fu> petevg: Any idea why we're getting inconsistent formats for the status timestamps?  Does that depend on series as well, or is it perhaps a juju version thing?
[21:49] <petevg> tvansteenburgh: The tricky thing is that I'm being tempted to hack in cross platform support for getting a timezone aware datetime object from a datetime string, and I'm thinking that the dateutils folks have already done it, and I can point at them rather than at my own foolishness if there are bugs ...
[21:49] <cory_fu> petevg: Do you have a VM where you could test it under the latest beta?
[21:50] <petevg> cory_fu: I think that what's happening is that you were chopping off the "Z" as part of chopping down the nanosecond, but sometimes we get a string with no nanoseconds, possibly just because it's exactly on the millisecond or something.
[21:50] <petevg> cory_fu: and then the Z shows back up.
[21:50] <petevg> cory_fu: I think that it unearths the fact that we're taking a UTC string and stripping the timezone info off of it.
[21:51] <petevg> ... which isn't too horrible, because the other side can just assume UTC.
[21:51] <petevg> But it makes me feel bad.
[21:51] <petevg> *nanoseconds
[21:51] <cory_fu> petevg: Odd.  I don't recall ever seeing the Z, but maybe I just blocked it out.
[21:51] <petevg> cory_fu: It was too horrifying to contemplate :-p
[21:51] <cory_fu> anastasiamac: Can we get confirmation if the times reported by juju status will *always* be in UTC?
[21:52] <anastasiamac> cory_fu: u can :)
[21:52] <cory_fu> petevg: I say we just chop it off and assume UTC, then.  :)
[21:52] <petevg> cory_fu: the actual problem is the Python datetime object that gets created. Python doesn't have native support for timezones when it creates them with strptime.
[21:53] <petevg> cory_fu: so the resulting datetime object has a timezone of None.
[21:53] <cory_fu> petevg: But we're only using it for a delta anyway, right?
[21:53] <cory_fu> As long as we compare it to utcnow() (which is also timezoneless, IIRC), are worry about the delta, we should be fine, right?
[21:53] <petevg> cory_fu: we're returning it to anyone who asks for agent_status_since or workload_status_since in python-libjuju.
[21:54] <petevg> cory_fu: so we're fine, but someone else using the API might not be.
[21:54] <cory_fu> OIC
[21:54] <petevg> ... except that things default to UTC anyway, I think.
[21:54] <petevg> Bleh.
[21:54] <cory_fu> petevg: Can we take the output from the strptime and re-package it with a UTC TZ?
[21:54] <cory_fu> So that we return a TZ aware DT even though we're internally just assuming it to be UTC?
[21:55] <petevg> cory_fu: that's where the hacky bit comes in. You can't just tell datetime that the timezone is UTC. You have to pass in a thing with an offset and other things that it expects.
[21:55] <tvansteenburgh> i would just leave it naive
[21:55] <tvansteenburgh> if a consumer wants to make it timezone aware, fine
[21:55] <petevg> tvansteenburgh: that's def. the easiest thing to do.
[21:55] <cory_fu> Yeah, just document that it will be UTC but naive and call it a day
[21:56] <petevg> kk. I will discard the "Z" without a care.
[21:56] <tvansteenburgh> but i wouldn't bother in the lib unless it's actually necessary
[21:56] <petevg> tvansteenburgh: cool. If someone really wants the data, they can extract it themselves from data['since'], and do dateutils things to it.
[21:58] <tvansteenburgh> petevg: yeah, and we may find down the road that we want/need to add pytz or python-dateutils. but i'm in favor of holding out for as long as possible ;)
[21:58] <anastasiamac> cory_fu: so status does not convert times.. it just displays whatever it finds in db
[21:58] <anastasiamac> cory_fu: mongo stores times in local times and we have to manually convert times to UTC
[21:59] <anastasiamac> cory_fu: we try to do that consistently on reads, but there mayb places where it does not happen (altho we try really hard, I am sure)
[22:00] <cory_fu> anastasiamac: OTOH, does the cloud image use UTC as the system time anyway?
[22:00] <cory_fu> Though I suppose the charm could do something crazy and change the TZ
[22:00] <petevg> cory_fu, anastasiamac: I guess the good news is that the "Right Thing" that I was going to do wasn't actually going to address some times possibly being local times. So the simpler code isn't more broken than the more complex code :-)
[22:01] <anastasiamac> cory_fu: :D
[22:02] <cory_fu> petevg: Well, if the TZ on the unit was changed, presumably the str representation would encode the new TZ, so the dateutils approach would pick that up, right?
[22:02] <petevg> cory_fu: If the repr actually had the timezone encoded, yes.
[22:03] <cory_fu> petevg: Isn't Z the encoded version of UTC?
[22:03] <petevg> cory_fu: yes.
[22:03] <anastasiamac> cory_fu: apparently, our cloud images use utc, but clouds are allowed to change them, for eg. gce does this... (thank you juju qa for confirmation!)
[22:04] <cory_fu> anastasiamac: You're making my life more difficult by the (UTC) minute.  ;)
[22:05] <anastasiamac> cory_fu: the feeling is mutual ;D
[22:05] <cory_fu> ha
[22:05] <anastasiamac> cory_fu: but m glad we r talking \o/
[22:05] <cory_fu> anastasiamac: Actually, I guess I should say that you're making petevg's life more difficult.  >=D
[22:06] <petevg> All I have to do is install a lib. It will just make tvansteenburgh sad :-)
[22:06] <cory_fu> petevg: It sounds like we can't rely on the value to be UTC after all, so show tvansteenburgh's who's boss and add dateutils to the requirements.txt
[22:06] <cory_fu> ha
[22:53] <anastasiamac> cory_fu: petevg: had a question from the group, updated in the bug :D
[22:54] <petevg> anastasiamac: recreating the issue. Will drop in a reply once I'm done.
[22:54] <anastasiamac> petevg: brilliant \o/ thnx :)
[22:54] <petevg> np
[22:55] <cory_fu> petevg: I have it available already
[22:55] <petevg> That's easier, then :-)
[22:57] <cory_fu> anastasiamac: Updated
[22:58] <anastasiamac> cory_fu: petevg: tyvm!!