#smooth-operator 2020-07-20
<Chipaca> ï½ï½ï½ï½ï½ï½ï½ï½  ï½ï½ï½ï½ï½ï½ï½ï¼
<jam> morning chipaca-of-the-early-morning
<Chipaca> good morning, again :-)
 * Chipaca brb
<jam> Chipaca, I still need reviews on operator:359
<jam> operator#359
<mup> PR #359: ops/testing.py: Harness.add_resource <Created by jameinel> <https://github.com/canonical/operator/pull/359>
<Chipaca> aaand network's back
<Chipaca> jam: oops. Will look in 5.
<Chipaca> ooh! i'm an arctic code contributor!
<Chipaca> arctic code vault*
<facubatista> Â¡Muy buenos dÃ­as a todos!
<Chipaca> facubatista: buen dÃ­a!
<Chipaca> flake8-bugbear is interesting
<facubatista> hola Chipaca
<facubatista> Chipaca, didn't review the list super thouroughly but all stuff I saw makes sense
<facubatista> jam, your operator#359 PR... are we "forcing" people to call addCleanup(harness.cleanup) when using resources? can't that be more automatic somehow? people surely will forgot to add that
<mup> PR #359: ops/testing.py: Harness.add_resource <Created by jameinel> <https://github.com/canonical/operator/pull/359>
<jam> facubatista, we can pass in the test suite to the Harness instance, or have them call cleanup(). Fixtures having a cleanup isn't an uncommon pattern.
<jam> That said, if they don't call cleanup, it actually does still cleanup but gives a warning (TemporaryDirectory has that property)
<jam>  morning facubatista
<Chipaca> facubatista: btw that bug you had with github itself sounded suspiciously like the issue i had where one of the two test runs was merging something weird
<Chipaca> facubatista, jam, https://github.com/chipaca/ops-lib-k8s/pull/2 is a thing
<mup> PR chipaca/ops-lib-k8s#2: lots of small tweaks and cleanup <Created by chipaca> <https://github.com/chipaca/ops-lib-k8s/pull/2>
<facubatista> Chipaca, they are aware of the "running actions on mistakenly merged commits", they just say it's very rare
<facubatista> Chipaca, ack
<Chipaca> trying out the juju-style 'copyright' header
<Chipaca> must say i like it :)
<facubatista> jam, yes, it's eventually removed (which is a good thing!), but I think there's no warning involved
<jam> facubatista, there is, because I was trying it before calling cleanup()
<facubatista> jam, I'm not being able to it to emit a warning in any combination of using TemporaryDirectory, and the docs doesn't mention any either
<facubatista> jam, which are the steps you make to see one?
<jam> facubatista, it may depend on Python version, if I do this edit:
<jam> +++ b/test/test_testing.py
<jam> @@ -749,7 +749,6 @@ class TestHarness(unittest.TestCase):
<jam>                  type: oci-image
<jam>                  description: "Another image."
<jam>              ''')
<jam> -        self.addCleanup(harness.cleanup)
<jam>          harness.populate_oci_resources()
<jam>          path = harness.model.resources.fetch('image')
<jam>          self.assertTrue(str(path).endswith('/image/contents.yaml'))
<jam> I get this warning:
<jam> $ ./run_tests test/test_testing.py
<jam> ................................./usr/lib/python3.8/tempfile.py:958: ResourceWarning: Implicitly cleaning up <TemporaryDirectory '/tmp/tmp-ops-test-resource-jn_sy0yg'>
<jam>   _warnings.warn(warn_message, ResourceWarning)
<jam> .............
<jam> ----------------------------------------------------------------------
<facubatista> TIL: https://docs.python.org/3/library/weakref.html#weakref.finalize
<facubatista> Return a callable finalizer object which will be called when obj is garbage collected. Unlike an ordinary weak reference, a finalizer will always survive until the reference object is collected, greatly simplifying lifecycle management.
<facubatista> Much nicer than adding __del__
<facubatista> oh, I love the lifecycle management of TemporaryDirectory!
<facubatista> (reading it's code)
<facubatista> Chipaca, I also like that copyright style!
<facubatista> will adapt it for my projects
<facubatista> maybe will add a link to the project, though
<facubatista> Chipaca, this "version magic management" can be separated in a different module/project? I love it, but don't want to start copying that code everywhere (maybe it's tricky because it's quite entangled with setup.py)
<facubatista> Chipaca, +1ed with comments
<Chipaca> yeah, ive been pondering that
<facubatista> Chipaca, let's make it the new standard for version management across the world
<Chipaca> facubatista: let's add a new keyword to the language
<Chipaca> we'll call it â¦ $$VERSION$$
<Chipaca> or maybe $v$
<jam> Chipaca, :)
<Chipaca> i have forgotten cvs tags! woo hoo
 * Chipaca likes when his head makes space with the _right_ stuff
<Chipaca> a pleasent change
<Chipaca> pleasant*
<Chipaca> sigh
<facubatista> jajaj
<Chipaca> does juju have a 'trace' log level?
<jam> Chipaca, yes, but Python doesn't IIRC
<Chipaca> it's easy to add it to python :-) (we did that in syncdaemon iirc)
<jam> Chipaca, so definitely juju internally uses TRACE and I think you can pass it to 'juju-log'
<mup> PR operator#359 closed: ops/testing.py: Harness.add_resource <Created by jameinel> <Merged by jameinel> <https://github.com/canonical/operator/pull/359>
<Chipaca> names are hard (tm)
<facubatista> Chipaca, we had TRACE in syncdaemon, yes, we used it to log all messages between client and server at the byte level
<Chipaca> ikr
<crodriguez> if I want to edit the charmcraft tool and test my changes, do I need to package it somehow ? There's no makefile and I am not very familiar with packaging
<Chipaca> crodriguez: edit the charmcraft tool itself, or your charm?
<crodriguez> the tool itself. It's not including "templates" directory in the build so my charm fails to deploy correctly
<crodriguez> Chipaca ^
<Chipaca> crodriguez: you should be able to just 'run' the charmcraft directory with python
<Chipaca> crodriguez: i.e. python3 ./charmcraft (or whatever)
<Chipaca> you might need to tweak the pythonpath, depending on how you've set it up :)
<crodriguez> ok I'll try that thank you
<Chipaca> e.g. PYTHONPATH=. python3 ./charmcraft
<crodriguez> hum that does not really  work https://pastebin.canonical.com/p/wrBzTnS5g4/
<crodriguez> tried this too.. https://pastebin.canonical.com/p/4JWS3Mjmsf/ . It would be nice if the charmcraft README would explain the correct steps :)
<crodriguez> Chipaca: I might just go back to using the submodule..
<mup> PR operator#360 opened: make debug logs all along the ops.lib.use dance <Created by chipaca> <https://github.com/canonical/operator/pull/360>
<Chipaca> crodriguez: so close!
<Chipaca> crodriguez: drop the 'export' from that last one and it should work
<Chipaca> crodriguez: bah, and then the tabulate thing, because you're not running it from the virtual env where you installed the requirements
<Chipaca> crodriguez: agreed about having this in the README
<Chipaca> facubatista: ^
<facubatista> crodriguez, did you create a virtualenv for charmcraft?
<facubatista> Chipaca, https://github.com/canonical/charmcraft/issues/88
<Chipaca> facubatista: ð
<Chipaca> facubatista: question WRT the k8s component '_dir': if instead of e.g.  (self._dir / "wobble").open(...)  the coded did  self._path("wobble").open(...)  would that be clear?
<jam> Chipaca, facubatista ?
<facubatista> Chipaca, I don't understand why I can not merge https://github.com/canonical/charmcraft/pull/86
<mup> PR charmcraft#86: The "list revisions from the Store" command <Created by facundobatista> <https://github.com/canonical/charmcraft/pull/86>
<facubatista> Chipaca, it says it's waiting for Travis checks, but if I go to travis all is done and green :/
<Chipaca> facubatista: because travis <-> github is complicated
<Chipaca> facubatista: restart one of the workers on travis
<facubatista> Chipaca, I don't find a "restart" anywhere
<Chipaca> facubatista: log back in to travis
<facubatista> https://travis-ci.org/github/canonical/charmcraft/jobs/710017112 for example
<Chipaca> facubatista: are you seeing 'no repositories found' in the left column, also?
<Chipaca> facubatista: i didn't have the 'restart job' either, so i logged out and back in and now it's there
<facubatista> wtf
<Chipaca> facubatista: something in auth is expiring wrong Â¯\_(ã)_/Â¯
<facubatista> well, that worked
<Chipaca> microservices
<facubatista> Chipaca, there, restarted
<facubatista> the hope is that will end up travis telling github that all is fine?
 * facubatista bb~30'
<Chipaca> facubatista: yup
<Chipaca> jam: are you using actual venv, or is that short for virtualenv?
<jam> Chipaca, pretty sure it is python venv
<jam> Chipaca, I'll set it up a second time to make sure
<jam> Chipaca, if I do: https://paste.ubuntu.com/p/2TbvPX7Wj8/
<jam> I get charmcraft/__main__.py can't find module named charmcraft
<Chipaca> yeah, repro'd it, so you're not 100% deranged
<Chipaca> but, anyway, must do this other thing :)
<facubatista> Chipaca, jam, store release: https://github.com/canonical/charmcraft/pull/89
<mup> PR charmcraft#89: Store release <Created by facundobatista> <https://github.com/canonical/charmcraft/pull/89>
 * facubatista brb, reboot
<crodriguez> Does the new framework handle resources differently, compared to reactive?
<facubatista> Chipaca, did we get to a conclusion about what we'll do about the "--system only in bionic" problem?
 * facubatista eods
<Chipaca> facubatista: ð¥ð¥ð¥?
<Chipaca> facubatista: detect bionic (or the patch) and work around it
<Chipaca> facubatista: probably the only thing we can do
<Chipaca> facubatista: where was it that --system did _not_ do the right thing?
<Chipaca> ah, when used with a non-ubuntu pip maybe?
 * Chipaca tries
<Chipaca> facubatista: so, one (ugly) way would be to detect ubuntu-patched pip by doing
<Chipaca> from pip._internal.commands.install import InstallCommand
<Chipaca> InstallCommand('i', 'i').cmd_opts.get_option('--system') is not None
<Chipaca> not the pretties approach, but I'm not sure there are alternatives
<Chipaca> --system is not mentioned in usage
<Chipaca> I guess you could run 'pip3 install --help', and look for --system in that
<facubatista> maybe, yes
<facubatista> ugly, all ugly
<Chipaca> facubatista: both of those options take a whole second to run /o\
#smooth-operator 2020-07-21
<jam> morning all
<Chipaca> ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½  ï½ï½ï½ï½ï½ï½ï½ï¼
<jam> morning Chipaca . it seems Guido was against main scripts inside packages
<jam> I'm not sure why they created __main__.py then.
<Chipaca> jam: is this related to the 'cannot import commands' thing?
<jam> Chipaca, yes. You can see my investigation on mattermost, but basically when running the primary script
<jam> your __name__ is __main__ which means you aren't part of a __package__
<Chipaca> facubatista: I just opened latest/beta, and moved latest/edge to it, so people can use that and we can open master to edge in due course
<Chipaca> jam: lol
<Chipaca> jam: another LOL is that jujuignores aren't gitignores
<jam> Chipaca, probably closer to bzr ignores I would guess
<Chipaca> and the only gitignore python implementations seem to be terrible
<Chipaca> e.g. https://github.com/mherrmann/gitignore_parser
<Chipaca> 'A spec-compliant gitignore parser for Python'
<jam> Chipaca, also, something to consider, one reason git ignores work well, is because you can create an exception with an explicit 'git add' but you can't do that for charms unless we have some sort of manifest
<Chipaca> has an open issue about 'foo' matching 'barfoo'
<Chipaca> and the author says "I have no idea LOL"
<jam> Chipaca, I have no idea... always a good response :)
<Chipaca> and it has no tests
<Chipaca> even though it's based on a different project that does has tests, but hasn't seen work in years
<Chipaca> (to the point that it's only been tested on 3.5)
<Chipaca> and that summarises how far i got yesterday before hating the world and going to sleep :)
<jam> Chipaca, don't hate the world, hate the... players ?
<Chipaca> jam: I shall hate the world, by the power of synecdoche!
<Chipaca> going back to jujuignore, it's a SMOP to translate https://github.com/juju/charm's jujuignore.go (and jujuignore_test.go) to python for our use
<Chipaca> but I don't think I can make it for release
<Chipaca> so I propose we look for a .jujuignore and bail out if it exists
<jam> Chipaca, so what about .gitignore ?
<jam> Switch and use 'git ls' ?
<jam> or just do .jujuignore but otherwise add everything since that matches juju deploy ?
<Chipaca> jam: well, that's the other approach
<Chipaca> i was thinking of getting .gitignore parsing to work, which should be a day of work on top of one of the se projects
<Chipaca> these*
<jam> Chipaca, I'd go for 'juju deploy' over most other things, because that is setting the precedence
 * jam goes to make coffee
<Chipaca> fair
<Chipaca> ok, let's do that
<Chipaca> i should be able to do that :)
<Chipaca> i don't think i can do that and 'charmcraft init' in four days though :)
<jam> Chipaca, can you split off pieces that I can help with?
<jam> eg, the jujuignore parser I could do while you focus on other things
<Chipaca> jam: you also have plenty on your plate already :)
<jam> Chipaca, indeed, but charmcraft release happens frist
<jam> first
<Chipaca> jam: ok let's do it
<jam> Chipaca, ok, I'll put together a jujuignore.py for charmcraft
<Chipaca> meanwhile i'll poke the init beast
<Chipaca> facubatista: WRT --system, it's *probably* easier to try with --system and if it fails try again without
<Chipaca> jam, facubatista: meeting
<Chipaca> jam: in setup.py, set zip_safe=False
<Chipaca> we can sort out the why and the how later :)
<jam> Chipaca, to make it do what? It still doesn't solve the problem that you can't run "python ./charmcraft' because the import paths are wrong
<jam> You *can* run python -m charmcraft for whatever reason
<Chipaca> jam: ah, for 'python setup.py install' to work, i meant
<jam> Chipaca, does zip_safe=false actually fix that? It seems like you still wouldn't have a 'charmcraft' you could drop in your path
<Chipaca> work as in result in a working 'charmcraft' script
<Chipaca> it does fix that, yes
<Chipaca> that's the only difference between what 'setup.py install' and 'pip install' does
<Chipaca> jam: python ./charmcraft  (and just 'python charmcraft') work as long as your PYTHONPATH includes .
<Chipaca> jam: (which it shouldn't by default)
<jam> Chipaca, but isn't that bad behavior?
<jam> Chipaca, so my goal was to 'build something' and then symlink a 'charmcraft' into my 'bin' directory so I can just run it
<jam> that feels like something that should work
<jam> if zip_safe=false gets us there, that seems ek
<jam> ok
<Chipaca> jam: zip_safe=False gets you a charmcraft script in the venv's bin, which you should be able to symlink
<Chipaca> i haven't tried that
<jam> Chipaca, I just did and it works
<jam> well, with venv active, let me try without
<jam> yep
<Chipaca> terrible news then
<Chipaca> :)
<Chipaca> jam: getting __main__.py to dtrt when it can find itself but not the package is a matter of tweaking sys.path, which is ugly but ok
<Chipaca> and if we do that we might be able to move the entry point to that? dunno :)
<Chipaca> at which point we get eggs back
<Chipaca> but, not for now
<Chipaca> now, for init
<jam> zip_safe does what I wanted
<jam> and is small
<facubatista> Â¡Muy buenos dÃ­as a todos!
<facubatista> Chipaca, ok with edge->beta, thanks
<Chipaca> oh drat, lunch
<Chipaca> facubatista: buen dÃ­a seor!
<Chipaca> but also, lunch
<Chipaca> siiigh
<facubatista> Chipaca, ?
<Chipaca> facubatista: sigh
<Chipaca> facubatista: :)
<Chipaca> little frustrations with jinja
<facubatista> :)
<jam> Chipaca, what's wrong with jinja ?
<jam> facubatista, reviewed your 'release' pr. I had a couple questions about divergence from snapcraft's CLI.
<jam> If it is intentional, then go with it.
<Chipaca> jam: it prunes the last \n when generating a file, and i hadn't spotted the option to turn that off :)
<jam> Chipaca, https://stackoverflow.com/questions/36870953/jinja2-how-to-remove-trailing-newline
<jam> it seems '-%' the '-' is relevant
<jam> Chipaca, and/or 'trim_blocks=True' 'lstrip_blocks=True' (or false) in Python
<Chipaca> jam: nah, it was the keep_trailing_newline option on the environment i was missing
<Chipaca> not in th eblocks
<Chipaca> 's all sorted now :)
<jam> Chipaca, as long as there is yet-another-way to do it
<jam> :)
 * jam aways
<Chipaca> those options you mentio are about trimming the whitespace around {%s, as opposed to trimming the whitespace at the bottom of the file
<Chipaca> anyway, yeah, all sorted
<Chipaca> also i can't type
<Chipaca> jam: the problem about "No module named 'charmcraft.commands'" kept on bugging me, but i figured it out: eggs don't like new-style namespace packages
<Chipaca> jam: easiest fix for our woes: drop an __init__.pu in charmcraft/commands/
<Chipaca> facubatista: is there a reason for not having that __init__ beyond "wasn't needed"?
<Chipaca> jam: an __init__.py probably works better
<Chipaca> than a .pu
<Chipaca> what even is a pu
<Chipaca> no don't answer
<facubatista> Chipaca, I used that __init__.py so you can do this elegant thing in main.py:
<facubatista> from charmcraft.commands import version, build, store
<facubatista>         store.ReleaseCommand, store.StatusCommand,
<Chipaca> facubatista: which __init__.py?
 * Chipaca puzzled
<facubatista> charmcraft.commands.store
<facubatista> oh, wait
<facubatista> you mean charmcraft/commands/__init__.py ?
<Chipaca> yes
<facubatista> there's not such thing
<Chipaca> exactly
<Chipaca> and that breaks things :)
<Chipaca> nothing serious i don't think, but it does
<Chipaca> facubatista: that's why 'python setup.py install' doesn't work
<facubatista> Chipaca, it's not needed theorically
<Chipaca> facubatista: because that builds an egg, and eggs don't like new-style namespace packages
<facubatista> stupid rotten eggs
<Chipaca> bah, it's supposed to be fixed in https://bugs.python.org/issue17633 but something's still off
<Chipaca> because just dropping __init__.py in there fixes it :)
<Chipaca> facubatista: the other alternative is zip_safe=False on setup.py/setup, but that's less nice i think
<facubatista> Chipaca, I just did ./setup.py install --prefix=/tmp/foo
<facubatista> which left me a lib/python3.8/site-packages/charmcraft-0.2.0-py3.8.egg in that dir
<facubatista> I unzipped that egg
<facubatista> and it has all the files ok in it
<Chipaca> facubatista: yes. But try running the script that you get from that install.
<Chipaca> you get
<Chipaca> ModuleNotFoundError: No module named 'charmcraft.commands'
<Chipaca> from ...charmcraft.egg/charmcraft/main.py
<Chipaca> line 25
<Chipaca> which was puzzling because that comes right after 'from charmcraft import __version__'
<Chipaca> and commands is right there in the egg
<facubatista> Chipaca, actually, https://paste.ubuntu.com/p/xzPhxjXgQN/
<Chipaca> facubatista: i'm assuming prefix=/tmp/foo means you're not getting the egg added to your import path
<Chipaca> facubatista: so try again with PYTHONPATH=/the/charmcraft.egg
<facubatista> 11:53:38|facundo@blackfx:/tmp/foo$ PYTHONPATH=lib/python3.8/site-packages/ bin/charmcraft
<facubatista>   File "/tmp/foo/lib/python3.8/site-packages/charmcraft-0.2.0-py3.8.egg/charmcraft/main.py", line 25, in <module>
<facubatista> ModuleNotFoundError: No module named 'charmcraft.commands'
<Chipaca> k
<Chipaca> facubatista: so that's what jam spotted yesterday, and is fixed by making commands not be a namespace package
<facubatista> Chipaca, ok
<Chipaca> facubatista: (please confirm) :)
<facubatista> Chipaca, confirm what? that it works? or that the theory is solid? I don't know enough of that corner, don't want to spend a couple of hours reading about it unless necessary
<Chipaca> facubatista: that dropping an __init__.py into commands makes it work
<Chipaca> facubatista: but, eh, we can dig next week or the other
<Chipaca> or i can propose a pr, innit
<Chipaca> just a silly thing really, it was bugging me
<facubatista> Chipaca, I don't care about it, people should be using snapcraft or the branch from github :p
<facubatista> s/snapcraft/the snap/
<facubatista> Chipaca, and IMO nobody should never ever do "sudo pip install"
<Chipaca> pip install would've worked because it doesn't make eggs :)
<facubatista> I wonder why :)
<Chipaca> facubatista: i agree, but i also don't want things to break in strange ways
<facubatista> Chipaca, more reasons! is adding quirks to our code good for the corner case of somebody installing running setup.py?
<facubatista> Chipaca, as you wish
<Chipaca> facubatista: we can't control how peple use our stuff :-)
<Chipaca> we can only control what happens when they do
<facubatista> Chipaca, but we have to prioritize where we invest our scarce time
 * Chipaca invests it in diophantine equations
 * facubatista invests in elephants equestrianism
<alejdg> Hello everyone, I'd like to know if there's an alternative for unitdata.kv in the operator framework. If not what is the best way to save information pertinent to units
<Chipaca> alejdg: I'm afraid I don't know what unitdata.kv is
<Chipaca> alejdg: how would you do the thing in the absence of the operator framework?
<alejdg> Chipaca: hey, this is what I'm referring to: https://charm-helpers.readthedocs.io/en/latest/api/charmhelpers.core.unitdata.html
<Chipaca> alejdg: that sounds like something you'd do using  _stored = StoredState()  on the charm class
<Chipaca> alejdg: not sure doing deltas of config is a good pattern, but if you need to do it that's how you'd go about it
<Chipaca> that'll give you per unit stored state. In the class's __init__ you'd do e.g. self._stored.set_default("some-kv", {}), and then you'd use self._stored["some-kv"] in the hook handlers
<alejdg> nice I'll try that
<alejdg> I'm doing that with some config data that comes from a relation.
<alejdg> maybe there's a better practice for it?
<Chipaca> alejdg: and you definitely need to react to _changes_ to that data, as opposed to the data itself?
<alejdg> Chipaca: That data doesn't change once the relation has sent it
<Chipaca> alejdg: ah so you're not looking for the "delta these things" feature, just the storing the data for later use?
<Chipaca> then _stored is definitely what you want :-)
<alejdg> gotcha, thanks a lot!
<Chipaca> alejdg: the 'set_default' thing is possibly the only non-obvious bit of it, where if you forget to do that and instead do the more obvious self._stored["some-kv"] = {}  in the __init__, then you're overwriting it every time
<crodriguez> How do I add an event? I added in init `self.framework.observe(self.on.restart_iscsi_services, self.on_restart_iscsi_services)`, I have a function defined `def on_restart_iscsi_services(self, event):`, and I added an actions.yaml file with `restart-iscsi-services` in it. But I get the error that `  File "./src/charm.py", line 47, in __init__
<crodriguez>     self.framework.observe(self.on.restart_iscsi_services, self.on_restart_iscsi_services)
<crodriguez> AttributeError: 'CharmEvents' object has no attribute 'restart_iscsi_services'`
<crodriguez> I was looking at the example from https://github.com/openstack-charmers/charm-ceph-iscsi and they didn't add anything else than that for their actions
<crodriguez> I want a user to be able to do : juju run-action restart-iscsi-services
<facubatista> crodriguez, let me see
<crodriguez> facubatista, jam told me it is because I'm missing _action at the end
<facubatista> crodriguez, oh, right, I didn't remember that
<crodriguez> there's not really an easy way to know that lol
<facubatista> crodriguez, our documentation status makes me sad
<crodriguez> me too :/
<crodriguez> another question :) is there an easy way to set some configuration parameters as mandatory? Wondering if there is a reusable component somewhere or if I should code it myself
<facubatista> I don't think there's anything in the operator for that
<crodriguez> ok. Is there a way to stop the code execution after setting "self.unit.status = BlockedStatus" ?
<crodriguez> the charm keeps on executing so the status gets cleared up really quickly
<facubatista> "return"?
<facubatista> crodriguez, I mean, if you set to blocked, and right away you do something and it gets unblocked it's fine, right?
<facubatista> or if you go to blocked and want to stop doing things until another event arrives, just return to exit the method?
<crodriguez> ah yeah I guess I could true/false on the check_mandatory_config function, and then exit on_install() with return depending on the result
<crodriguez> So in reactive I would do something like event A triggers B, when B completed -> trigger C, etc. and that way I would avoid having a really long function doing B+C. Can I do the same with the framework.observe?
<crodriguez> currently I have on.config_changed -> render_config . I would like to trigger something when render_config is completed
<facubatista> crodriguez, have a function that does B, then another function that does C; then create a function that is hooked to event A, which calls B() and then C()
<facubatista> I mean, it's something that can be solved simply enough at a Python level, doesn't need to have infrastructure for it
<crodriguez> yeah it can be. I just got used to set states in reactive and triggers things with these states, I was wondering if there is an equivalent here
<crodriguez> facubatista, I see options to set active, maintenance, blocked, but no option to set a charm in error state ? https://github.com/canonical/operator/blob/master/ops/model.py#L770
<Chipaca> crodriguez: exiting with failure does that
<Chipaca> i don't think you can 'set' error state otherwise
<Chipaca> but i'm no expert :)
<facubatista> jam, ^ :)
<facubatista> crodriguez, sorry, I was off already
<facubatista> (didn't see your message before)
<crodriguez> oh hey :) no problem, notifications aren't great on my side neither. I think it would be good to be able to set an error state sometimes. It allows to set a message more explicit than "config-changed failed"
#smooth-operator 2020-07-22
<jam> crodriguez, error should be used sparingly, because it means the charm can't do anything else other than retry the hook that failed
<jam> so Blocked is much more what you want than Error
<jam> morning ChanServ
<jam> Chipaca, I mean
<stub> Last I looked, eggs seemed deprecated for everything, replaced by wheels. https://packaging.python.org/discussions/wheel-vs-egg/
<Chipaca> jam: morning :-)
<Chipaca> i guess the question then is: is there a way to make 'setup.py install' not go eggy?
<jam> Chipaca, isn't that zip_safe=False ?
<Chipaca> jam: that'll install a decompressed egg, which isn't pretty (even though it works)
<Chipaca> anyway, back to init for me :)
 * Chipaca writes systemd in python
<Chipaca> wait no not again
<jam> Chipaca, https://stackoverflow.com/questions/48550546/how-to-install-a-wheel-style-package-using-setup-py
<jam> seems to say you have to "setup.py wheel; then pip install .whl"
<jam> that's pretty ugly user experience
<Chipaca> jam: just 'pip install .' works fine :)
<Chipaca> jam: otoh, 'python setup.py develop' might also work
<jam> Chipaca, indeed. I can certainly switch to that, just is unfortunate that the ingrained method wasn't improved
<Chipaca> i still think dropping an __init__.py in commands is the right thing, for least surprise
<jam> Chipaca, should charmcraft master be calling itself 0.2 ? isn't it 0.3 pre?
<Chipaca> jam: 0.2.0+59.gab2e71e.dirty
<jam> Chipaca, ah, because it is many commits past 0.2
<Chipaca> yep
<Chipaca> 59, in this case
<jam> good/bad of using git version
<jam> it doesn't do 'pre' well :)
<Chipaca> correct :)
<jam> I guess you could tag 0.3beta ...
<jam> Chipaca, I think it should know how many more commits you need for 0.3 and then you'd have a nice count down
<jam> and you'd
<jam> know when you're done coding. :)
<Chipaca> jam: I like the way reality looks from where you're standing
<jam> btw, any comments on charmcraft#90 ?
<mup> PR charmcraft#90: charmcraft/jujuignore.py: ability to parse .jujuignore <Created by jameinel> <https://github.com/canonical/charmcraft/pull/90>
<Chipaca> jam: i haven't even peeked yet
<Chipaca> i'll do that now
<jam> Chipaca, bind charm chatc?
<Chipaca> d'oh, omw
<jam> barryprice, https://juju.is/docs/k8s-charms-tutorial mentions having a ports section
<jam> which states "protocol: TCP"
<jam> I would guess you change that to protocol: UDP
<jam> I couldn't say off hand if you need both TCP and UDP whether that is 2 separate port declarations, 2 separate protocol declarations, or something like "protocol: TCP,UDP"
<barryprice> jam: yup, that's actually what I have here, I will see how that goes - if it's that simple then great
<barryprice> my assumption was two declarations but I'll play around with it
<jam> barryprice, Ports        []ContainerPort `json:"ports,omitempty" yaml:"ports,omitempty"`
<jam> ports is a list, but protocol is just a string.
<jam> still not sure
<jam> barryprice, the k8s docs seem to say that UDP support depends on the cloud, and nginx as an ingress controller has some confusing discusison
<jam> https://kubernetes.github.io/ingress-nginx/user-guide/exposing-tcp-udp-services/
<jam> I think you noted that it was part of a ConfigMap of the ingress service, which seems a bit surprising
<barryprice> jam: yup, that's the page that sent me looking into how to set a ConfigMap
<barryprice> the second example on that page is for UDP/53
<jam> barryprice, if I understand that page correctly, the main problem is that you need to change the ConfigMap of some other service in response to you deploying *your* service.
<jam> eg, that is having NGINX running as the ingress controller listen on 53 so that it can proxy for the actual application behind it
<jam> barryprice, I would certainly just start with a port declaration for protocol: UDP and then see where that gets you
<barryprice> jam: will do, hopefully get something working in the next hour before I EOD, I'll let you know
<barryprice> keep finding silly bugs in the image, and have to rebuild each time - hopefully this round
<barryprice> jam: ok that works for me locally with microk8s
<barryprice> App   Version  Status  Scale  Charm     Store  Rev  OS          Address         Notes
<barryprice> bind           active      1  bind-k8s  local    8  kubernetes  10.152.183.197
<barryprice> Unit     Workload  Agent  Address     Ports          Message
<barryprice> bind/9*  active    idle   10.1.68.21  53/TCP,53/UDP  Pod configured
<barryprice> and from a terminal on my laptop, outside the pods, "dig @10.152.183.197 ubuntu.com" gives me a correct result
<Chipaca> jam, facubatista, should we skip the bug revue today?
<jam> Chipaca, either way for me
<Chipaca> huh, charmcraft tests don't run here now
<Chipaca> jam: do they work for you, on charmcraft master?
<jam> Chipaca, they pass on my branch, I'll check master
<facubatista> Â¡Muy buenos dÃ­as a todos!
<Chipaca> facubatista: ð
<jam> morning facubatista
<jam> Chipaca, so I had to update my dependencies for master
<jam> but once I did that, the tests passed
<Chipaca> facubatista: charmcraft tests on master don't work for me (E   ModuleNotFoundError: No module named 'tests.factory') and was wondering if i needed to do anything
<facubatista> do you have a tests/factory.py file?
<jam> I d
<jam> o
<Chipaca> i do
<Chipaca> and i can import it
<Chipaca> by hand from a python running from the same place i run the tests
<jam> Chipaca, want to come by the bug review hangout and we'll chat on it?
<Chipaca> sure
<Chipaca> jam, facubatista, https://docs.pytest.org/en/latest/pythonpath.html#pytest-vs-python-m-pytest
<jam> Chipaca, which as I read it says "put an __init__.py and things will work" :)
<Chipaca> there's also a strong recommendation of using src/
<Chipaca> but that doesn't fix my issue :-/
<Chipaca> --import-mode=importlib fixes all my woes \o/
<jam> Chipaca, https://blog.ionelmc.ro/2014/05/25/python-packaging/#the-structure seems to say the advantage is that you have to install in order to do tests
<jam> I find that extra step to be a bit onerous to a code test code test code test lifecycel
<jam> build in place is pretty good
<Chipaca> yeah
<Chipaca> also 2014 is SO six years ago
<jam> Chipaca, it also explicitly breaks your version.py cheats
<Chipaca> unfortunately --import-mode=importlib requires pytest 6
<jam> Chipaca, and they say "use tox" but my explicit problem with tox is the 10+s it adds to each test iteration
<Chipaca> and this is all because the init templates include a 'tests' directory, btw
<Chipaca> also, there's a test in charmcraft that looks at the flags that are used to run the test instead of .. something else? but maybe that's a 6.0 breakage, looking
<Chipaca> no, this is test breakage
<Chipaca> facubatista: :-(
<Chipaca> facubatista: if you run pytest with any flags at all, they're passed straight in to e.g. test_main_no_args
<Chipaca> and fail
<Chipaca> facubatista: e.g., python -m pytest --log-auto-indent=on
 * facubatista is trying to understand these lines
<Chipaca> facubatista: ââ¼ââ¾ââ¼ââ¾ââ¼ââ¾ââ¼ââ¾â
 * Chipaca helps
<facubatista> Chipaca, I don't see how is possible to happen what my eyes are seeing; if you still need my help let's HO
<Chipaca> facubatista: osm ho now tho
<Chipaca> davigar15_: trying to 'charm push' a .charm doesn't immediately tell me to do one so maybe it'll work? i don't have the right credentials to try :)
<Chipaca> facubatista: can we have a meet
<Chipaca> facubatista: (standup)
<Chipaca> facubatista: "tan"
<facubatista> :)
 * Chipaca enquires about lunch
<crodriguez> jam: I understand that, but there are some cases where you need the charm to be in error. It's also a lot easier to troubleshoot because error gives you access to the debug-hooks and resolve actions.
<jam> crodriguez, sure. if you need error, by all means use it, just realize the implications
<crodriguez> jam: yeah it's for a specific case. So I think a class "ErrorStatus" should be added to ops.model
<jam> crodriguez, you can always raise an exception.
<jam> Juju doesn't let us set Error status
<jam> it only happens as a result of exit nonzero
<crodriguez> yes it does.. well it was an option in the reactive framework at least
<jam> I'll double check
<crodriguez> I'll dig to see how it was done in reactive
<Chipaca> jam: ah! (?s:<regexp>) is new in 3.6
<Chipaca> jam: you want (?s)<regexp>
<jam> Chipaca, or I can just set the flag on the compile
<jam> Chipaca, do you prefer having it in the regex?
<Chipaca> i prefer having it in a flag :)
<jam> Chipaca, also adding a test for paths with '\n' in them if we want DOTALL to be set
<jam> (which fnmatch did, so I'm following suit)
<jam> crodriguez, $ juju run --unit ubuntu-lite/0 'status-set error flobar'
<jam> ERROR invalid status "error", expected one of [maintenance blocked waiting active]
<jam> I'm not sure how reactive was handling error, but it isn't allowed by status-set
<jam> It may be that we can call 'status-set blocked message' and then raise an exception to return nonzero
<jam> but we'd need a reasonable mechanism
<crodriguez> jam: alright yeah maybe they did a workaround for that. I can settle for a blocked status for now to do what I need to do, thanks for double-checking
<jam> looking here: https://github.com/juju/charm-helpers/blob/master/charmhelpers/core/hookenv.py#L62
<jam> and here:https://github.com/juju/charm-helpers/blob/master/charmhelpers/core/hookenv.py#L1123
<jam> it seems that using error would just raise a ValueError
<jam> unless I'm reading it wrong
<jam> (or the code has changed in the meantime)
<facubatista> jam, .jujuignore rules are pretty much the same ones than .gitignore, right? https://git-scm.com/docs/gitignore#_pattern_format
<jam> facubatista, they were modeled after them, yes
 * Chipaca runs the tests against the 'charmcraft init'-generated code and wonders if it's enough meta-programming
<Chipaca> why don't pathlib.Path objects have a chdir() that's a context manager?
<Chipaca> as in: with tmp_path.chdir(): â¦
<facubatista> Chipaca, meeting?
<Chipaca> oops
<Chipaca> facubatista: for the sys.argv thing, are you fixing that or should I?
<Chipaca> facubatista: I need to pass arguments to pytest now, how would you prefer I did that?
<facubatista> Chipaca, if you fix the sys.argv thing, you can pass args to pytest, right?
<Chipaca> facubatista: yes, from run_tests
<facubatista> Chipaca, the fix is just removing the sys.argv default from the function definition, right?
<Chipaca> facubatista: yes
<facubatista> you can include that in your branch
<Chipaca> facubatista: i can push a pr to do just that
<Chipaca> or i can include it, indeed
<Chipaca> ok
<facubatista> thanks
<Chipaca> brb (gpg agent died, easiest way to get it back is restarting the session)
<Chipaca> facubatista: https://github.com/canonical/charmcraft/pull/91 fwiw
<mup> PR charmcraft#91: First pass at 'charmcraft init' <Created by chipaca> <https://github.com/canonical/charmcraft/pull/91>
<facubatista> wee
<Chipaca> and i'm off to make dinner & etc
<Chipaca> ttfn
<facubatista> ack
<facubatista> Chipaca, please remember https://github.com/canonical/charmcraft/pull/89
<mup> PR charmcraft#89: Store release <Created by facundobatista> <https://github.com/canonical/charmcraft/pull/89>
<facubatista> (and I think there's a problem with Travis... that PR is not listed at all in https://travis-ci.org/github/canonical/charmcraft/builds )
 * facubatista eods
#smooth-operator 2020-07-23
 * Chipaca EODs
<Chipaca> ï½ï½ï½ï½ï½ï½ï½ï½  ï½ï½ï½ï½ï½ï½ï½ï¼
<facubatista> Â¡Muy buenos dÃ­as a todos!
<facubatista> Chipaca, please remember https://github.com/canonical/charmcraft/pull/89
<mup> PR charmcraft#89: Store release <Created by facundobatista> <https://github.com/canonical/charmcraft/pull/89>
<facubatista> Chipaca, jam, is there a way to "re-send the work to Travis from GH"?
<facubatista> I need to do that here: https://github.com/canonical/charmcraft/pull/89
<mup> PR charmcraft#89: Store release <Created by facundobatista> <https://github.com/canonical/charmcraft/pull/89>
<Chipaca> facubatista: let me look
<jam> if you log into travis you can ask it to rebuild, IIRC
<facubatista> jam, I don't see the work in travis
<facubatista> s/work/job/whatever
<Chipaca> facubatista: https://travis-ci.org/github/canonical/charmcraft/builds/710773290 ?
<Chipaca> or is that the wrong commit
<facubatista> Chipaca, that it's not listed here: https://travis-ci.org/github/canonical/charmcraft/builds
<Chipaca> facubatista: https://travis-ci.org/github/canonical/charmcraft/pull_requests
<facubatista> oh
<Chipaca> the commit hashes don't match so i dunno
<Chipaca> but it's got two builds 22 hours ago, which matches the two commits from 22 hours ago :)
<facubatista> if I tell it to "see this commit in github" I see a diff like looks like my PR, but with a message "This commit does not belong to any branch on this repository."
<facubatista> so it looks it's a travis commit?
<facubatista> anyway, I'll trigger a re-build of one of the jobs to see if that pings GH correctly
<jam> did you rebase something?
<facubatista> jam, nop
<facubatista> meanwhile, this: https://github.com/canonical/charmcraft/pull/93
<mup> PR charmcraft#93: Store status <Created by facundobatista> <https://github.com/canonical/charmcraft/pull/93>
<facubatista> (but note that it depends on 89
<facubatista> )
<jam> facubatista, is there anything blocking 89 other than @Chipaca ?
<facubatista> nop
 * Chipaca blocks all the things
<facubatista> Chipaca, if you answer "ehhhh... maybe? :-)" in a review conversation it makes me try to find the new commit changes to see if you changed or not that particular line
<facubatista> Chipaca, it's better IMO if you just say "no" (and continue talking) or "yes" (and I can assume you changed as suggested)
<Chipaca> facubatista: my apologies. You are right.
<facubatista> Chipaca, no problemo :)
<Chipaca> facubatista: I changed it to a warning, but I don't know how much sense it makes, because I am not able to create the conditions that would cause that line of code to be exercised
<facubatista> Chipaca, ack
<Chipaca> facubatista: there are a couple of 'ifs' there that aren't covered because the docs say they can happen but not how :)
<Chipaca> heh, jam, good idea about filtering, why didn't i look at that
<jam> Chipaca, for the OSError that we want to log?
<Chipaca> ye
<mup> PR operator#361 opened: ops/testing.py: Basic support for checking pod spec <Created by jameinel> <https://github.com/canonical/operator/pull/361>
<Chipaca> facubatista, jam, we standupping?
<jam> Chipaca, I'm sitdowning
<facubatista> going
<Chipaca> we're going to grow a shared lib between ops and charmcraft aren't we
<jam> facubatista, reading through your outline I think it flows fairly well.
<jam> obviously needs fleshing out, but the basic flow seems good
<facubatista> jam, thanks!
<Chipaca> facubatista: venv is not always there fwiw
<Chipaca> why am i replying to reviews on irc
 * Chipaca chastises himself
<jam> Chipaca, because you want immediate feedback ?
<facubatista> we can talk here or in a HO about some specifics from the review (higher bandwidth), but I'd need more context
<Chipaca> facubatista, jam, do neither of you have your name in gecos?
<facubatista> it worked in my machine
<Chipaca> that's: getent passwd $USER | cut -d: -f5 | cut -d, -f1
<Chipaca> fwiw
<facubatista> Chipaca, jam, store-status is formally ready for review (store-release landed and merged into it)
<Chipaca> facubatista: WDYT of https://paste.ubuntu.com/p/8NqQx9mp9P/ ?
<facubatista> Chipaca, I'm not sure... we do need to add a "long help", but not sure if the docstring is the proper place to put it.
<facubatista> we should dedent that at least
<Chipaca> the default formatter does that already
<facubatista> which we're replacing
<Chipaca> we are?
<facubatista> we are
<Chipaca> ah
<facubatista> there's no way to conform to Canonical specifities using argparse help constructor
<Chipaca> facubatista: the one we have does dedent though :)
<Chipaca> facubatista, jam, virtualenv in the template README: env, or .env?
<facubatista> env
<Chipaca> k
<facubatista> I don't think we should create hidden stuff if possible
<jam> Chipaca, I've always done venv
<jam> I think that is the default if you just "python3 -m venv"
<Chipaca> jam: ENV_DIR is required for venv as well
<Chipaca> OTOH 'env' might confuse people into thinking it's environment, not virtualenv-ish
<facubatista> I'm fine with it just being visible
<Chipaca> visible is a terrible name for a virtual environment directory
<facubatista> jajaj
<Chipaca> jam: i just hit the 'flake8 checks all the files in the venv' thing
<Chipaca> adding a .flake8 :)
<facubatista> Virtual Included Super Important Basic Large Environment
<Chipaca> \o/
<Chipaca> facubatista: what's the pytest equivalent of assertRaises?
<facubatista> Chipaca, pytest.raises()
<facubatista> Chipaca, it have a `match` arg where you can pass a regex to check for the exception message
<facubatista> I've been comparing the exact text in a lot of places by doing `str(cm.value)`, though
<facubatista> (cm is the context manager)
<facubatista> with pytest.raises(ValueError) as cm:
<facubatista> Chipaca, for the `init` command, what about a "versions" file?
<Chipaca> facubatista: ?
<Chipaca> facubatista: what do you mean a versions file?
<facubatista> Chipaca, a file named 'version'
<Chipaca> facubatista: that sounds like a bad idea to me :)
<Chipaca> facubatista: because i'd expect 'charmcraft build' to generate that
<Chipaca> facubatista: as per charmcraft#37
<mup> Issue charmcraft#37: Populate the 'version' file as part of 'charmcraft build' <Created by jameinel> <https://github.com/canonical/charmcraft/issues/37>
<facubatista> Chipaca, ok :)
<facubatista> Chipaca, mmm... however, what we'll put in the 'automatic version file'? we only have the git hash, users will want the version to be 1.0, 1.1
<facubatista> I'm +1 for build to produce something if not there, but a good formed project probably should provide its own information there
<Chipaca> facubatista: sounds like something we should figure out soon :) maybe comment on that bug?
<facubatista> done
<Chipaca> facubatista: got a sec?
<facubatista> Chipaca, got two!
<Chipaca> facubatista: standup meet plz
<facubatista> Chipaca, "series" in the metadata.yaml file is mandatory for the Store
<facubatista> Chipaca, jam, updated https://gist.github.com/facundobatista/e279a6e5adac93bb6b048cbf5a9fe96f
<facubatista> Chipaca, jam, unless "English corrections" or any specific comment from you, it would be done
<facubatista> Chipaca, also proposed https://github.com/canonical/charmcraft/pull/94
<mup> PR charmcraft#94: Store commands autocomplete <Created by facundobatista> <https://github.com/canonical/charmcraft/pull/94>
<Chipaca> facubatista: is the store really not answering anything other than 'rejected' when it doesn't like an upload?
<Chipaca> facubatista: charmcraft writes everything to stederr :-(
 * Chipaca files a bug
<facubatista> Chipaca, they landed an improvement these days
<facubatista> Chipaca, we can fix the issue I opened for when they provide more info
<Chipaca> k
<facubatista> Chipaca, regarding stderr, we need to decide what we'll send to stdout and stderr, check the "messages to the user" doc
<Chipaca> facubatista: the expected output of a command is never stderr
<facubatista> Chipaca, what about errors?
<facubatista> anyway, the doc
 * facubatista is not really here
 * Chipaca EODs
#smooth-operator 2020-07-24
<Chipaca> morning all!
<facubatista> Â¡Muy buenos dÃ­as a todos!
<facubatista> Chipaca, https://github.com/canonical/operator/pull/360 is ready to land, right?
<mup> PR #360: make debug logs all along the ops.lib.use dance <Created by chipaca> <https://github.com/canonical/operator/pull/360>
<Chipaca> facubatista: almost
<Chipaca> facubatista: morning :)
 * Chipaca was having lunch
<Chipaca> facubatista: almost because there was a typo in a comment i just spotted, fixed and pushed now
<mup> Issue operator#342 closed: ops.lib.autoload does not log what it is doing <Created by stub42> <Closed by chipaca> <https://github.com/canonical/operator/issues/342>
<mup> PR operator#360 closed: make debug logs all along the ops.lib.use dance <Created by chipaca> <Merged by chipaca> <https://github.com/canonical/operator/pull/360>
<facubatista> Chipaca, I just pushed my two PRs after merging with master (and doing the fixes in the second one)
 * facubatista brb
<crodriguez__> hi guys, any update on the "including every file in the build" for charmcraft ?
<crodriguez__> https://github.com/canonical/charmcraft/issues/80 or/and  https://github.com/canonical/charmcraft/issues/39
<facubatista> crodriguez__, we're progressing in a dependency of that, which is to support .jujuignore semantics
<facubatista> as we do not want to include EVERY file
<facubatista> (like, your .git directory)
<facubatista> there's a PR for that, in a good shape, should be landed ~Monday
<facubatista> crodriguez__, after that, doing the extra mile should be easy
<crodriguez__> okay facubatista , I'll check back early next week !
<Chipaca> facubatista: gah, one thing _was_ missing from 'init': using tabulate for the TODOs
<facubatista> ah, right
<Chipaca> facubatista: ooh, i thought of a way to make the 'output to the user' table more interesting
<Chipaca> s/thought/remembered/
<Chipaca> --output={text,json,yaml}
 * Chipaca runs away
<facubatista> Chipaca, that can be interesting, yes, if we think charmcraft will be used from scripts
<Chipaca> my experience from over in snapdland is that it's not an if but a when
<facubatista> Chipaca, I just improved what you mentioned in charmcraft#93
<mup> PR charmcraft#93: Store status <Created by facundobatista> <https://github.com/canonical/charmcraft/pull/93>
<Chipaca> facubatista: my approval stands :)
<facubatista> ah, I need jam's, right
<facubatista> thanks
<facubatista> Chipaca, ah, I need yours in charmcraft#94
<mup> PR charmcraft#94: Store commands autocomplete <Created by facundobatista> <https://github.com/canonical/charmcraft/pull/94>
<facubatista> thanks
<facubatista> jam, Chipaca, https://github.com/canonical/charmcraft/pull/96
<mup> PR charmcraft#96: Updated snapcraft.yaml for new Py deps <Created by facundobatista> <https://github.com/canonical/charmcraft/pull/96>
<Chipaca> facubatista: Ã³som
<Chipaca> facubatista: I'm going to EOD
<Chipaca> facubatista: have a great weekend, and enjoy the week ahead!
<facubatista> thanks! have a nice vacations!
 * facubatista eods and eows, see you all on Monday
#smooth-operator 2020-07-26
<mup> Issue operator#261 closed: Harness should support pod-spec-set <Created by jameinel> <Closed by jameinel> <https://github.com/canonical/operator/issues/261>
<mup> PR operator#361 closed: ops/testing.py: Basic support for checking pod spec <Created by jameinel> <Merged by jameinel> <https://github.com/canonical/operator/pull/361>
