#smooth-operator 2020-04-13
<niemeyer> Good morning all
<mup__> Issue # closed: operator#98, operator#113, operator#122, operator#123, operator#156, operator#158, operator#161, operator#165, operator#166, operator#169, operator#171, operator#172, operator#174, operator#175, operator#178, operator#179, operator#180, operator#185, operator#186, operator#187,
<mup__> operator#188, operator#189, operator#198, operator#200, operator#205, operator#206, operator#208, operator#211, operator#214, operator#215, operator#219, operator#220
<mup__> PR # closed: operator#135, operator#190, operator#196, operator#203, operator#209, operator#210, operator#212, operator#216, operator#218, operator#221
<mup__> Issue # opened: operator#98, operator#113, operator#122, operator#123, operator#156, operator#158, operator#161, operator#165, operator#166, operator#169, operator#171, operator#172, operator#174, operator#175, operator#178, operator#179, operator#180, operator#185, operator#186, operator#187,
<mup__> operator#188, operator#189, operator#198, operator#200, operator#205, operator#206, operator#208, operator#211, operator#214, operator#215, operator#219, operator#220
<mup__> PR # opened: operator#135, operator#190, operator#196, operator#203, operator#209, operator#210, operator#212, operator#216, operator#218, operator#221
<niemeyer> Dmitrii-Sh: The change you added to Chipaca's #203 doesn't make much sense given the PR purpose
<mup_> PR #203: ops, test: revert the EventSet change, make EventsBase ObjectEvents <Created by chipaca> <https://github.com/canonical/operator/pull/203>
<Dmitrii-Sh> niemeyer: that's for a different repo & PR
<niemeyer> Dmitrii-Sh: I don't understand the comment given what I just said
<niemeyer> Dmitrii-Sh: I'm talking about *that* change in *that* PR
<Dmitrii-Sh> https://github.com/canonical/operator/pull/203/commits - there's only one commit there, I didn't make any changes
<mup_> PR #203: ops, test: revert the EventSet change, make EventsBase ObjectEvents <Created by chipaca> <https://github.com/canonical/operator/pull/203>
<niemeyer> Dmitrii-Sh: I see, thanks
<niemeyer> Dmitrii-Sh: Indeed I misread the GitHub comments as being a change in that PR.. confusing
<Dmitrii-Sh> np, references are not very explicit, I agree
<niemeyer> Yeah, it clearly says "<nick> added commit ab3b to <branch>"
<niemeyer> So we need to go see which branch it is
<niemeyer> Dmitrii-Sh: Is #210 about a new interface or an existing interface that existing real world charms are using?
<mup_> PR #210: tls-certificates interface (requires side, existing interface) <ð interface> <Created by dshcherb> <https://github.com/canonical/operator/pull/210>
<niemeyer> Dmitrii-Sh: The summary and the labels disagree
<Dmitrii-Sh> tls-certificates is about the existing one
<niemeyer> Not sure if the idea of the label was to flag things being added to the repo or things being created anew
<niemeyer> We'll need to catch up tomorrow about that once everyone is back
<Dmitrii-Sh> right, haven't noticed that the "interface" label has a description saying "A new interface!"
<facubatista> Muy buenos dÃ­as a todos!
<niemeyer> facubatista: Heya!
<facubatista> hola niemeyer!
<niemeyer> Lunchero
<facubatista> jam, hola! I just approved #209 but please see a comment in the test part, thanks!
<mup_> PR #209: Enable stderr logging if JUJU_DEBUG is set <Created by jameinel> <https://github.com/canonical/operator/pull/209>
<facubatista> mup_, thanks :)
<jam> facubatista, hiya, thanks! I thought you were off today
<facubatista> jam, nop, thu & fri last week; Chipaca is off today
<jam> ah, gotcha
<jam> facubatista, actually, the main asserts are outside of the patch, but I went ahead and removed the extra assert that was inside, because it functionally is redundant with the rest.
<facubatista> perfect
<mup_> PR operator#209 closed: Enable stderr logging if JUJU_DEBUG is set <Created by jameinel> <Merged by jameinel> <https://github.com/canonical/operator/pull/209>
<mup_> Issue operator#200 closed: Install stderr log handler when running under debug-hooks <Created by stub42> <Closed by jameinel> <https://github.com/canonical/operator/issue/200>
<jam> nice to see mup working
<jam> but why mup_ ?
<jam> I guess only one mup on freenode at a time
<niemeyer> No, if works across channels, networks, and even protocols..
<niemeyer> It's the same mup here and in Signal, for example
<niemeyer> But I've been hacking on it recently as mentioned.. this is the one running on my laptop.. soon the other one will go away and this will be moved to a proper place
<jam> sorry, I meant one bot named 'mup' and running the new one will use a different name until you have it worked out.
<niemeyer> Ah, yeah, that's it
<niemeyer> Dmitrii-Sh: Are you off today?
<Dmitrii-Sh> niemeyer: no, but I got transferred to a different team
<niemeyer> Dmitrii-Sh: Are you coming to the standup?
<Dmitrii-Sh> no, I declined the invite for today
<mup_> Issue operator#222 opened: Test harness: network_get support <enhancement> <Created by dshcherb> <https://github.com/canonical/operator/issue/222>
<mup_> PR operator#210 closed: tls-certificates interface (requires side, existing interface) <ð interface> <Created by dshcherb> <Closed by niemeyer> <https://github.com/canonical/operator/pull/210>
<mup_> PR operator#216 closed: tcp-load-balancer interface (provides and requires sides, completely new interface) <ð interface> <Created by dshcherb> <Closed by niemeyer> <https://github.com/canonical/operator/pull/216>
<mup_> PR operator#135 closed: Operator Framework charm writing guide <Created by VariableDeclared> <Closed by niemeyer> <https://github.com/canonical/operator/pull/135>
 * facubatista -> lunch
 * facubatista is back
 * facubatista brb
<facubatista> Docs for the debugger: https://discourse.juju.is/t/live-debugging-at-python-level/2914
 * facubatista eods
<niemeyer> facubatista: Have a good evening
<facubatista> niemeyer, thanks!
<mup_> Issue operator#223 opened: Operator fails on Juju 2.7.0 and below <Created by chris-sanders> <https://github.com/canonical/operator/issue/223>
#smooth-operator 2020-04-14
<jam> morning all
<niemeyer> Hello charmers
<Chipaca> goood morning!
<niemeyer> Chipaca: Hiya!
<Chipaca> my laptop is back at being stuck at 800MHz :-(
<Chipaca> might have to spend some time talking with tech support
<niemeyer> Might be a safety mechanism so you're not going too fast
<niemeyer> stub: Good question about the place to put the "lib".. once they go out, the idea is to have a module directory under ops.lib
<niemeyer> stub: My original idea when we discussed was to have it under the charm itself, so that there would be a local "lib" namespace.. but there's an issue with that plan
<niemeyer> We need to be able to have multiple charms of the same author using that same library, precisely so that it becomes easier to reuse and to move into ops.lib once it evolves enough
<stub> And 'from lib import pgsql' would be importing lib/lib/pgsql/__init__.py ?
<stub> I'm mostly interested in getting the namespace right at my end.
<niemeyer> stub: Right, it was a good question.. my original idea was src/lib/pgsql, but that's not good enough
<niemeyer> stub: lib/lib isn't great either
<niemeyer> stub: Not naming spacing pgsql isn't good either, as the top-level namespace is busy and we'll often be implementing libraries to integrate existing things in charms
<niemeyer> *namespacing
<stub> Ideally without turning hooks/install into a mess of PYTHONPATH manipulation hacks
<niemeyer> Yeah, we shouldn't need any further manipulation..
<niemeyer> We already have src for local, and lib for external, so we should be good
<Chipaca> jam: you froze
<Chipaca> or my network died
<jam> Chipaca, its probably me
<niemeyer> Lunch, biab
<facubatista> Muy buenos dÃ­as a todos!
<jam> morning facubatista
<facubatista> hola jam
<Chipaca> whoops, laptop lost power while i was having lunch
<jam> Chipaca, I was wondering where you went :)
 * Chipaca heads for the hills
<facubatista> Chipaca, jam, we need to decide *how* we will support a dispatch entrypoint; this doc talks about it as "a new thing" but does not describes any migration: https://docs.google.com/document/d/1RHrOMbqsXz9qwk_3sHJ7AJOdlh-Mb-TkFUAtbXb37Og/edit#heading=h.df280vskamcl
<facubatista> Chipaca, jam, so, what happens if both dispatch and classic hooks coexist?
<jam> facubatista, yeah, I'm planning on adding a bit to that doc. I think it evolved and covers what Juju needed to do, but part of why that plan was ok was because of the plan for the framework. I'm working on that now.
<jam> I think we had a good plan, just need to get it written down.
<facubatista> jam, ack
<facubatista> jam, please let me know when that's done, then, thanks!
<facubatista> Chipaca, we need this ^ before continuing with #221, right?
<mup_> PR #221: osp, test: support dispatch <Created by chipaca> <https://github.com/canonical/operator/pull/221>
<Chipaca> facubatista: yep
<Chipaca> well, more or less :)
<Chipaca> i already know the gist of it so i can move forward
<Chipaca> but i've got plenty other things on my plate so i'll leave it be for now
<jam> facubatista, is it worth talking about all the ways and why we chose not to do the alternatives, or just focus on the expected path?
<facubatista> jam, it totally worths it, otherwise you may keep answering that to everybody who read the docs -- better to write it just once
<jam> Chipaca, I just remembered one of the other concerns we should keep in mind.
<jam> namely, hooks/install => src/charm.py for compatibility, but also dispatch => src/charm.py. We don't want dispatch getting called to cause install to get called and double-handle every event (or worse, infinitely recurse. :)
<facubatista> jam, we can see that hooks/install is present *and not* a symlink to ourselves
<facubatista> s/see/check/
<jam> facubatista, indeed. just something we need to remember to do
<facubatista> jam, don't trust our memory! that's the kind of detail that should be written down
<jam> yeah, I'll be adding it to the doc.
<facubatista> juju deploys my charm in an unit; which directory should I work with, from the charm, to put "local files"? /var/lib?
<facubatista> where is the best way to ask these questions? ^
<facubatista> s/way/place/
<jam> facubatista, local to the charm, local to the app? many people reuse $PWD (which is /var/lib/juju/agents/<unit-name>/charm) but that is usually considered non-ideal (it leads to dirty charm directories and upgrades/etc also touch that dire)
<facubatista> jam, local to the app
<facubatista> jam, I want to put a file there that will be served later
 * facubatista tries to os.mkdir("/var/lib/mything")
<jam> facubatista, local to the app depends on the app itself, right? eg Apache likes '/www'
<facubatista> jam, but my charm doesn't run in the same unit than apache, right?
<facubatista> jam, I'm *exactly* blocked in that point
<jam> facubatista, it depends how you deploy it.
<facubatista> jam, do you know how to do it?
<facubatista> this needs to be a subordinate charm somehow?
<jam> facubatista, if you need to be on every unit of another app, the usual recommended way is to be a 'subordinate'
<jam> exactly
<jam> you can always have a bundle that colocates 2 different applications on the same machines, but if you're saying "I want to populate data for Apache to serve", that sounds like a subordinate.
<jam> subordinates explicitly don't have their own 'number of units', they are strictly a 'place a unit of this everywhere you place a unit of that'
<facubatista> mmmm... actually... let me see something else first
<facubatista> jam, thanks!
<facubatista> jam, Chipaca, niemeyer, meeting, https://meet.google.com/fqw-mdqc-dsf
<Chipaca> ok, need to reboot
<niemeyer> Me too.. that is, *I* need to reboot, not the laptop
 * facubatista -> lunch
<Chipaca> sneaking in for a bit while writing yet another usb image :-/
 * facubatista is back
 * Chipaca booted into windows, was terrified, and ran away
<facubatista> Chipaca, are you ok for the meeting, or shall we pospone it?
<Chipaca> facubatista: i'll be there in 15 seconds
<Chipaca> came back for it
<niemeyer> Module namespaces are not my friends
<niemeyer> At least not as I was trying it out
<niemeyer> Can't get ops.beta to be a namespace while we have content in ops
<Chipaca> ok, off to do the Lesser Scary Thing
<niemeyer> # XXX Is this the right thing for subpackages like zope.app?
<niemeyer> Not a great comment to find in the implementation of namespaces.. :)
<facubatista> jejeje
<niemeyer> Okay, got something that starts to look promising...
<mup_> Issue # closed: operator#98, operator#113, operator#122, operator#123, operator#156, operator#158, operator#161, operator#165, operator#166, operator#169, operator#171, operator#172, operator#174, operator#175, operator#178, operator#179, operator#180, operator#185, operator#186, operator#187,
<mup_> operator#188, operator#189, operator#198, operator#205, operator#206, operator#208, operator#211, operator#214, operator#215, operator#219, operator#220, operator#222, operator#223
<mup_> PR # closed: operator#190, operator#196, operator#203, operator#212, operator#218, operator#221
<mup_> Issue # opened: operator#98, operator#113, operator#122, operator#123, operator#156, operator#158, operator#161, operator#165, operator#166, operator#169, operator#171, operator#172, operator#174, operator#175, operator#178, operator#179, operator#180, operator#185, operator#186, operator#187,
<mup_> operator#188, operator#189, operator#198, operator#205, operator#206, operator#208, operator#211, operator#214, operator#215, operator#219, operator#220, operator#222, operator#223
<mup_> PR # opened: operator#190, operator#196, operator#203, operator#212, operator#218, operator#221
<niemeyer> Hmm.. mup seems to be taking a network error as a valid document
<niemeyer> Need to look into it
<niemeyer> facubatista: When you have a moment, would like to have a quick chat about this:
<niemeyer> https://paste.ubuntu.com/p/cHP2gq5MkK/
<facubatista> niemeyer, I was about to get a cup of tea; let's do it in 10'?
<niemeyer> +1
<niemeyer> I mean.. yes, 10' :)
<facubatista> ok, in 11' :p
<facubatista> yes, jajaaj
<facubatista> niemeyer, https://meet.google.com/veq-yfqm-kdk ?
<niemeyer> facubatista: omw
 * facubatista eods
#smooth-operator 2020-04-15
<jam> morning all
<Chipaca> moin moin moin
<mup_> Issue operator#224 opened: [Idea] provide some form of scaffolding <Created by woutervb> <https://github.com/canonical/operator/issue/224>
<jam> morning Chipaca
<Chipaca> jam: how's things?
<jam> Pretty good. how are things with you?
<Chipaca> jam: not bad. I'm not in hospital, so that's winning, right?
<jam> Chipaca, yeah, not sick, stuck at home, but healthy and well fed. :)
<Chipaca> jam: on a less serious note I managed to install windows on my work laptop without nuking the linux installation (not even grub) so that's also a plus
<Chipaca> so that should help dell figure out what's going on with it without having to come out
<jam> Chipaca, ah, dell requires Windows to do support?
<Chipaca> normally i would've gone "i bought this with linux off of you, do your <grawlixes> job", but that would've meant sending someone out just to figure it out
<jam> I'm surprised you didn't have to reinstall grub
<Chipaca> jam: remote diagnostics, yes, apparently it's a windows-only thing
<Chipaca> jam: me too! apparently uefi means windows is not as aggressive to the bootloader as it used to be
 * Chipaca wonders what happened to issue 224
<mup_> Issue #224: [Idea] provide some form of scaffolding <Created by woutervb> <https://github.com/canonical/operator/issue/224>
<Chipaca> yep, 404ing for me
<jam> ran away ?
<Chipaca> but mup seems to find it
<jam> found it from https://github.com/canonical/operator/issues/224
<jam> Chipaca, mup has a bad url
<jam> 'issue/224' vs 'issues/224'
<Chipaca> ah
<Chipaca> niemeyer: ^ that one's on you probably
<Chipaca> the mup issue i mean :)
<jam> Chipaca, the email from Sohini "Weekly Update". Elements we will be working to prioritize with the charmtech team
<Chipaca> mhmm
<jam> "Options" seems particularly misplaced, as options only exists because of reactive (AIUI)
<jam> (they had an options section as a way to configure a layer, but since we use instantiation of a class as the way to 'bring in a component' it doesn't feel like you need YAML config.)
<Chipaca> jam: I have no idea what that is, so I'll believe you =)
<jam> unless I'm misunderstanding what they were trying to do.
<jam> Chipaca, it sounds like we need to be having some form of discussion with whoever is asking for these things ):
<jam> :)
<Chipaca> I missed sohini yesterday with my hardware shenanigans, but will catch up today
<niemeyer> Sorry, I'll get that mup issue fixed today.. waited to fix it in that batch of changes but missed it
<jam> Debug as an action is something I'm curious about.
<Chipaca> wait, tomorrow (it's moved)
<Chipaca> jam: debug-as-an-action i imagine is "have an action that's just for using via debug-hook" but irunno
<Chipaca> i'm good at imagining stuff :-|
<jam> Chipaca, maybe. Or maybe it's 'juju run-action --wait debug' which dumps out a databag of stuff you might want to figure out what the charm is doing
<jam> but as a line-item its a bit ill defined.
<jam> Chipaca, I've been trying to think about how we could handle 'ensuring enough packages are installed by the time you get to your charm code'.
<Chipaca> jam: any conclusions so far?
<jam> The problem being you can't just put "import external_lib" at the top of your src/charm.py because when the install hook runs nothing is installed yet
<jam> Chipaca, so I think you can do things like lazy importing (wait to import until you need it), and put it in install, but it all feels a bit cludgy.
<Chipaca> jam: if it's just an import yes, the problem is if you need a system package, isn't it?
<jam> I was considering being evil and saying that 'hooks/install' runs before Charm.on.install, and you can put your dependency handling in there.
<jam> but you still have the problem of src/charm.py struggling to start up if you do it the 'natural' way
<jam> Chipaca, right, so if you have "on_start(): import package"; but "on_install(): import ops.apt; ops.apt.install(python-package)"
<jam> you could make it work
<jam> but it does feel kludgy
<Chipaca> jam: wait, why a python package?
<Chipaca> 'system package' i meant something like postgres
<jam> Chipaca, so for system packages generally, you can just apt install it during on_install so no problem
<Chipaca> right
<jam> the problem, IMO, is python packages needing to be installed before your charm can import them.
<Chipaca> and python packages should be in the charm already
<Chipaca> via the build step
<Chipaca> (of course this is me living in the future where the build tool is done and platform is a thing)
<jam> Chipaca, how does the PYTHONPATH get setup to find them?
<Chipaca> jam: properly
<Chipaca> :)
<Chipaca> jam: but more seriously, via having the charm.py shepang point at something that sets up the pythonpath
<jam> So *a* step needs to rewrite something for '/var/lib/juju/agents/unit-<foo-X>/charm/???' to be a place that we can satisfy imports from, right?
<Chipaca> shebang*
<Chipaca> jam: whether that something is a virtualenv python, or just a wrapper, i don't know yet (nor particularly care), but both would work
<jam> Chipaca, but what rewrites that path?
<jam> and what installs the venv?
<Chipaca> jam: the build step
<jam> Chipaca, how does that handle the fact that it installs to a different path-on-disk during 'juju deploy' ?
<jam> relative paths?
<jam> eg, what sets up the venv on the target system
<jam> we package a python binary in the charm?
<Chipaca> relative paths should work, we need to test that
<Chipaca> otherwise things get even more interesting
<Chipaca> i'd rather not ship a python binary in the charm but i'm not ideologically opposed to it
<Chipaca> all it _should_ take in my mind is a wrapper to set up the environ before calling system python
<Chipaca> but i have yet to carve out time to play with this
<jam> Chipaca, does that wrapper end up being 'dispatch' ?
<Chipaca> jam: that could work also
<niemeyer> Isn't all that what we already have?
<Chipaca> niemeyer: what do you mean?
<niemeyer> I'm just trying to understand what the discussion was about.. it sounds like we have what was being described:
<niemeyer> A wrapper that sets up Python paths and calls the charm, including known locations inside the charm in sys.path
<Chipaca> niemeyer: are you still writing?
<niemeyer> No :)
<niemeyer> We need a mup plugin..
<Chipaca> niemeyer: where do we have that wrapper?
<niemeyer> Chipaca: dispatch, per jam note
<Chipaca> niemeyer: where is it
<Chipaca> the wrapper, where, on github, is it
<niemeyer> Chipaca: Wasn't support for dispatch in jam's agenda?
<Chipaca> niemeyer: you said we already have what was being described; where do we have it?
<niemeyer> Chipaca: It's not implemented.. it's agreed, in jam's agenda, in juju development code as well.. I was simply asking if that's what the discussion was about.. not sure why we're now arguing about where in GitHub it is?
<jam> I think the confusion is that we were discussing how it should hang together, and you seemed to claim that while we have plans for it, we haven't actually implemented those yet. And I was asking about concretely how it would/should function.
<Chipaca> niemeyer: because "Isn't that what we already have?" shuts down the discussion we were having
<jam> Do we bundle what would otherwise be system libs, so that it is all served from a not-touched-by-the-system venv.
<Chipaca> niemeyer: so, as you shut down the discussion, I'm calling you out so that you explain yourself
<niemeyer> Oh my.. You know you can say "no, because...".. right?
<niemeyer> Chipaca: I literally said "I'm just trying to understand"..
<Chipaca> niemeyer: yes, now that I know what you meant, I can say that, but the discussion is still dead and needs resurrecting to continue it
<Chipaca> yeah
<niemeyer> Sorry, I'm stepping out.. I'm not in a good vibe to have this kind of discussion right now
<stub> Might want to look at charms.reactive bootstrap process. It setup a virtualenv (so subordinates are insulated from version conflicts with primaries), installs python dependencies and deb packages, then reinvokes itself with the virtualenv interpreter.
<jam> stub, indeed. I have looked at it a bit, though actually finding where the bits are is a bit convoluted, and hidden behind the various yaml definitions, etc.
<jam> Trying to determine if the only way is to go via wheelhouses with full depedencies and isolated venvs
<stub> I think it was mostly in layer:base
<niemeyer> It also built stuff in production, which seems unwise
<stub> wheelhouses were a pain because we want platform independent, and wheels are not always
<niemeyer> But stub is right, it's some incarnation of that
<jam> incarceration of that?
<jam> :)
<niemeyer> And we should definitely learn from it
<niemeyer> jam: to much covid
<niemeyer> too much
<niemeyer> stub: Is there a way to avoid the platform dependency, though?
<niemeyer> stub: I would love to justify having pure python charms, but the conversations so far make me believe we can't do without
<stub> I think my problems were due to having old versions of setuptools/distutils etc. in the chain. And python dependencies that built C extensions even when they were told not too (looking at you cassandra client)
<niemeyer> stub: Wheelhouses also feels like an insufficient solution in the sense we need to carry C lib dependencies, etc
<niemeyer> stub: Building *in deployment* right?  Yeah, we need to kill that
<stub> I think I hit both cases. Things building in deployment, and things building at build time and embedding platform dependent shared libraries.
<stub> Yes, we want to avoid both.
<niemeyer> stub: You mean shared libraries that have unsatisfied symbols at the target?
<jam> stub, so why not use the platform libs for interacting with the platform app that you're installing?
 * Chipaca steps out for a bit
<stub> amd64 shared libraries getting embedded in the charm, which isn't going to work if someone tries to deploy on z-series
<stub> jam: There are not always platform libraries, and offline deployments make pulling them in from PPAs a pain.
<jam> (it feels like venvs reinvent the distribution problem, which is the point of distribution platforms)
<stub> The cassandra charm needed to connect to cassandra, which has a really fat client with c extensions and a complex build and not in Ubuntu main or universe.
<niemeyer> jam: Yeah, and I don't know how to get charms forward without us diving into that for the hundredth time
<stub> The original charm pulled it in from a PPA. The reactive rewrite managed to get it from the wheelhouse.
<niemeyer> I would love to just be able to say "charms need to be pure Python"
<jam> stub, but still had to compile it for the arch?
<niemeyer> But it sounds unrealistic
<stub> jam: I think I managed to get it to avoid building the C extensions ('charm build' tries to enforce that, but it didn't work). But I would not be surprised to find amd64 specific shared libraries in the wheels.
<stub> niemeyer: 'charms need to be pure Python' may well be the best approach. Bootstrap can inject platform specific binaries before the main charm.py is imported. Bootstrap installing deb packages would solve 90% of the use cases.
<niemeyer> stub: The issue with recommending that is that we force charmers to become deb packagers as well.. and not only that.. we now need alternative binary deployment strategies for every platform
<jam> stub, certainly the Juju model that charms are architecture independent was premised on arch specific things would be handled at the apt layer
<jam> bbiab, making coffe
<stub> Other 10% would be weirder stuff, like pulling a tarball from a Juju resource (say, a JVM). Or installing wheels from a wheelhouse.
<niemeyer> stub: That's why I'm finding difficult to workaround the need for charms to become yet another binary package
<stub> Yes, 'learn to package debs' is a high barrier to entry :-/
<niemeyer> and distribute!
<niemeyer> ... and even then not have a cross-platform strategy
<stub> And it would be sad if it put a heavy burden on charmers dealing with the more common situation, where pure python and maybe a deb package from main.
<niemeyer> Yeah, this case should remain simple
<stub> Maybe it is just a case of declaring arch: ['amd64','arm64'] in metadata.yaml, and having arch/amd64 or arch/arm64 directories added to $PATH and $PYTHONPATH. Fat charms.
<niemeyer> stub: Yeah, that sounds unavoidable as soon as we agree to do binaries
<niemeyer> stub: Otherwise we have no control of the outcome whatsoever
<niemeyer> stub: The new charm hub will also deal with multiple architectures gracefully
<stub> The fat could move to resources, which could be intelligently pulled down (or attached at deploy time for offline deployments). But that is more complex.
<niemeyer> So again we get back to the issue of building and organization of both the source and the result of that operation
<niemeyer> facubatista has that on his agenda, but we don't have something clear yet
<jam> stub, I think resources make a strong case for binary deps
<niemeyer> stub: Agreed about resources
<Chipaca> niemeyer: fwiw facu is waiting for feedback on the build tool email before continuing that work (because he isn't blocked on other fronts)
<Chipaca> brb, rebooting into windows (eugh)
<facubatista> Muy buenos dÃ­as a todos!
 * Chipaca now has to decide whether to be without his work laptop for at least a week, or be with an extremly slow laptop until the big Q is over
<jam> morning facubatista
<facubatista> hola jam
<jam> Chipaca, they just extended ours 'indefinitely' (until further notice) :(
<Chipaca> yeah :-(
<Chipaca> ours isn't going away anytime soon
<Chipaca> facubatista: ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½ï½  ï½ï½ï½ï½ï½ï½ï½ï¼
<facubatista> Chipaca, hola!
 * facubatista is reading previous conversation, why are we discussing AGAIN if we'll freeze a Python binary inside the charm, etc?
<facubatista> do not discuss this again in the air, let's comment on the proto-spec I created, otherwise we're always in t0
<Chipaca> facubatista: agreed
<facubatista> "the problem, IMO, is python packages needing to be installed before your charm can import them." <-- this is already solved, I'm waiting feedback on that
<Chipaca> facubatista: my intention is to play with this stuff based on your build tool thing, and get back to it
<jam> niemeyer, shall we just cancel the weekly 1:1? We can always schedule time to chat when we have a topic instead.
<jam> facubatista, I updated https://docs.google.com/document/d/1RHrOMbqsXz9qwk_3sHJ7AJOdlh-Mb-TkFUAtbXb37Og/edit# with a discussion of the plans for the Operator framework. Feel free to leave comments that we can use to help clarify anything.
<facubatista> jam, thanks!
<facubatista> jam, what do you mean with " The intent is that each of those handlers is maintaining its own state engine, "?
<facubatista> jam, each method registered in my charm should maintain one state engine??
<jam> facubatista, generally handlers are going to be maintaining some sort of state in response to events. And while they ultimately share some of that state, the intent is that you shouldn't ever stop sending them lifecycle events just because something else is also responding to it.
<jam> facubatista, for your charm "have I written out the blog's static files" "do they need updating" feels like the state engine that you're using.
<jam> the apache interaction of "have I configured apache to serve my files" is separate logic.
<facubatista> mmm
<facubatista> I wasn't planning to maintain that state, but maybe I'm too naive?
<jam> I don't think it is usually explicitly model as a state machine, but functionally it is.
<jam> facubatista, for simple charms, I feel it is perfectly fine to determine your state by querying the model, rather than storing it separately.
<facubatista> if I get the install event it means that I have not written out the static files yet; if I get the upgrade event it means that the install ended correctly, and files are there, etc
<jam> facubatista, but like the pgsql interface needs to know what you've requested of pg, and whether it has told you that it has done its preparations for you (what roles you need, what you want to call your database, etc.)
<facubatista> jam, sorry, we're moving away from the static files example, right?
<jam> facubatista, so Juju generally guarantees "at least once" events, and otherwise hooks should be idempotent. If something fails, it may call a hook >1 time.
<jam> facubatista, yeah, I was giving a clearer example of a state machine. For static files, your plan seems fine.
<jam> I don't know if there will be config, which might indicate you need to write new files, etc.
<facubatista> I understand that idempotency, it's orthogonal to needing to have a state machine
<facubatista> jam, I'm trying to parse you're sentence about pgsql, 1'
<facubatista> jam, I'd really to "see" that pgsql example; in my mind when I instantiate the PG interface I will tell it whatever I need, then the interface when receives the relation event will configure the DB engine, and trigger the "all ready" event for me
<jam> facubatista, correct. but that is the state engine that the DB component keeps track of.
<jam> 'what did you ask for, has the DB told me that it is ready, ok, ready to fire the DB is ready event"
<facubatista> my point is: no state machine is needed for that
<facubatista> "what did you ask for" -> this is stored in the interface
<facubatista> "has the DB told me that it is ready" -> this is what "receiving the relation event" means
<facubatista> etc
<facubatista> (that's why I need to "see" the code, maybe I'm missing parts and/or complexity here)
<jam> facubatista, so that is still a state machine, even if you are storing the state in the interface
<jam> Chipaca, did we forget to add a meet link to the bug revue calendar entry?
<jam> just use standup?
<Chipaca> jam: yeah let's just use standup
<facubatista> jam, I'm not storing state for a state machine, I'm storing the asked DB engine config
<mup_> Issue operator#161 closed: self.model.pod.set_spec should only be run on leader <Created by mthaddon> <Closed by niemeyer> <https://github.com/canonical/operator/issues/161>
<mup_> Issue operator#225 opened: BoundSharedState changed events is misnamed (or misleading) <Created by addyess> <https://github.com/canonical/operator/issues/225>
 * facubatista brb
 * facubatista -> lunch
 * facubatista is back
<niemeyer> Alright.. off from the last call of the day, and calling it a day
<niemeyer> Have a good eod folks
<facubatista> niemeyer, have a nice rest
<Chipaca> facubatista: i'm off as well. enjoy the evening solitude!
<Chipaca> facubatista: or, don't. You do you.
 * facubatista is alone in the office, turns the music volume on to the max
<facubatista> niemeyer, ended trying to collect our thoughts here: https://docs.google.com/document/d/1270Fl9c6NzfMMA71xkqTwyO5nUIyBO6GQnvy34wtH5w/edit#
 * facubatista eods
#smooth-operator 2020-04-16
<jam> seems quiet... too quiet
<niemeyer> Must be because everyone is isolated in their homes..
<niemeyer> Wait..
<niemeyer> ;)
<niemeyer> Morning
<jam> morning
<mup_> PR operator#226 opened: ops/framework.py: StoredStateChanged includes the attribute <Created by jameinel> <https://github.com/canonical/operator/pull/226>
<niemeyer> jam: I wonder if we should do that ^ or if we should hide the event altogether
<niemeyer> jam: This was part of the original implementation.. back then everything would fire changed
<niemeyer> jam: Then we moved to only snapshoting once for performance
<jam> niemeyer, I could see going either way. We've generally thought that _stored should be private, which means you shouldn't be listening for changes on it.
<jam> if we are going to have changes, then they should be useful
<jam> and right now it only emits if the top level attribute changes
<jam> so either remove it entirely, or make it correct.
<jam> the above makes it correct, but if we want to say 'don't do it' then we can do that easily enough. :)
<niemeyer> jam: Agreed
<niemeyer> jam: Is that attribute only the tip if there are multiple nested dicts?
<jam> niemeyer, so I considered a couple different things, but went with top level attribute to start
<jam> otherwise you end up with some sort of path, which might include lists, dicts, etc.
<jam> and the attribute of the dict might be a string or an int or
<niemeyer> jam: Agreed.. we'd have to build some sort of dotted notation
<niemeyer> foo[3].bar
<jam> yeah.
<jam> I didn't want to deal with paths yet, when the top level attribute is at least a good enough hint.
<niemeyer> I suggest just killing it for now, making it private if we still require it for the internal implementation to some  degree
<niemeyer> We can ask to see what's the use case before we axe it
<niemeyer> Gut feeling is that this shouldn't be important, and might even lead to brittle code as this isn't precise enough
<jam> niemeyer, just ripping out the code no tests fail...
<niemeyer> +1 to that
<niemeyer> I'll copy & paste this conversation into the issue
<Chipaca> mo'in all!
<jam> morning Chipaca
<mup_> PR operator#227 opened: ops/framework.py: remove StoredStateChanged <Created by jameinel> <https://github.com/canonical/operator/pull/227>
<jam> team policy decision on issue #225, do we fix it (PR #226) or get rid of it (PR #227)
<mup_> Issue #225: BoundSharedState changed events is misnamed (or misleading) <Created by addyess> <https://github.com/canonical/operator/issues/225>
<mup_> PR #226: ops/framework.py: StoredStateChanged includes the attribute <Created by jameinel> <https://github.com/canonical/operator/pull/226>
<mup_> PR #227: ops/framework.py: remove StoredStateChanged <Created by jameinel> <https://github.com/canonical/operator/pull/227>
<niemeyer> jam: Given that theory, couldn't we get rid of all the sub-type magic we have there?
<niemeyer> jam: I mean, we introduced these StoredLists, etc, precisely to be able to handle the subevents
<jam> niemeyer, we still need dirty
<niemeyer> Ah, indeed
<niemeyer> Otherwise we'll always snapshot, okay
<niemeyer> jam: Here is my vote: https://github.com/canonical/operator/issues/225#issuecomment-614496681
<jam> thx
 * Chipaca runs tests â wonders about coverage â looks into coverage â weeps â drags self back to work at hand â runs tests
<Chipaca> to be fair, it's just main.py that has dreadful coverage, the others aren't too bad
<Chipaca> but i was working on main.py
<niemeyer> I don't think the coverage tests will handle that one correctly
<niemeyer> By definition the best testing for it needs to fire it externally
<niemeyer> That external process won't be accounted for coverage
<Chipaca> yes, and things like _get_charm_dir should be covered by unit tests
<Chipaca> it has two branches, one of them simple, one of them suspicious
<Chipaca> i replaced the suspicious one with "raise RuntimeError" and everything was happy
<Chipaca> i mean, main.py's main, sure, hard to unit-test, no problem. but that file is almost all helpers, almost all uncovered (without looking beyond the 20% coverage it has)
<Chipaca> ANYhow
 * Chipaca drags self back to work at hand again
<mup_> Issue operator#228 opened: Adopt a standard and well integrated templating module <Created by niemeyer> <https://github.com/canonical/operator/issues/228>
<mup_> Issue operator#229 opened: The pod spec API must be improved <Created by niemeyer> <https://github.com/canonical/operator/issues/229>
<mup_> Issue operator#113 closed: Charms as YAML <question> <Created by knkski> <Closed by niemeyer> <https://github.com/canonical/operator/issues/113>
<niemeyer> Chipaca: If you raise an exception and it does nothing, that's obviously uncovered.. but the "20%" number means nothing if our tests are exercising logic in this file via an independent process
<niemeyer> Which I think they are
<niemeyer> Sorry if that was clear before and I'm just repeating myself.. it just felt like a coverage number was still being taken as correct
<niemeyer> We have that charms/test_main charm that is trying to make the check as realistic as reasonable.. when we do that externally, none of the lines touched by the test will get into the report
<Chipaca> I'll look into this after
 * Chipaca steps out for a bit
<niemeyer> Lunch
<facubatista> Muy buenos dÃ­as a todos!
<facubatista> jam, if you can land #227, that means #226 could be just closed, right?
<mup_> PR #227: ops/framework.py: remove StoredStateChanged <Created by jameinel> <https://github.com/canonical/operator/pull/227>
<mup_> PR #226: ops/framework.py: StoredStateChanged includes the attribute <Created by jameinel> <https://github.com/canonical/operator/pull/226>
<jam> If we decide to do 227 then it replaces 226, correct
<mup_> PR operator#227 closed: ops/framework.py: remove StoredStateChanged <Created by jameinel> <Merged by jameinel> <https://github.com/canonical/operator/pull/227>
<mup_> PR operator#226 closed: ops/framework.py: StoredStateChanged includes the attribute <Created by jameinel> <Closed by jameinel> <https://github.com/canonical/operator/pull/226>
<mup_> Issue operator#225 closed: BoundSharedState changed events is misnamed (or misleading) <Created by addyess> <Closed by addyess> <https://github.com/canonical/operator/issues/225>
<mup_> PR operator#230 opened: Testing harness no events for own data <Created by jameinel> <https://github.com/canonical/operator/pull/230>
<Chipaca> facubatista: buen dÃ­a seÃ±or!
<facubatista> hola Chipaca
<jam>  Chipaca I'm assuming its ok if I miss the framework sync (in 45 min). or would you like me to be there?
<Chipaca> jam: indeed
<Chipaca> jam: if they wanted you in it it'd be at a reasonable time for you :-)
 * facubatista had(has) a car issue but is back
<niemeyer> Ouch :(
<Chipaca> brb, reboot
<Chipaca> wooo! hello from my zippy laptop again!
<Chipaca> looking frankenstein-y right now so requires another reboot, and here's hoping it's still zippy after that
 * Chipaca brb rebooting again
<facubatista> Chipaca, how did you fix it?
<Chipaca> I don't think it's fixed, but it works for now
<facubatista> Chipaca, just improved suddenly or you did something?
<Chipaca> facubatista: unplugged the battery
<Chipaca> and turned it on from just ac
<Chipaca> then power it off again, plug the battery back in
<Chipaca> getting at the battery is ~20 screws but hey, it worked
<Chipaca> facubatista: but i'm not going to do this every 20 days :-( so it's not "fixed", but i might do this every 20 days until the Q is over
<facubatista> Chipaca, if it serves of consolation, battery is also involved in my issue :p
<Chipaca> facubatista: hehe
<Chipaca> facubatista: batteries are the worst, except for not having batteries
<facubatista> wife took the car and went to the greengrocer's, got back, parked outside to clean some leaves from the garage, and then the car just didn't start anymore
<niemeyer> GOod place to get stuck
<facubatista> lucky us that it didn't die at the greengrocer's (which is ~10 blocks away, not much, but would complicate a lot the operational part of waiting some service or replacement)
<facubatista> niemeyer, exactly
<facubatista> of course, I can not just go and by a new battery, this kind of shop is closed
<Chipaca> facubatista: something something condensation
<facubatista> Chipaca, what?
<Chipaca> facubatista: sorry, that was a bit obscure of me. A few years ago somebody asked in a forum somewhere, something like (from memory), "dear $forum, I don't know what to do! I nipped out to the grocer's this morning and came back, and then when I went to go to work my car didn't start, so I came back into the house and my partner was in bed with another person!" and they answered "probably condensation in your $engine_part, try blowing it dry"
<facubatista> jejej
<Chipaca> and the discussion was about how doing the short prior trip lead to the condensation
 * Chipaca steps away for a break before the last meeting of the day
<facubatista> niemeyer, https://www.pagina12.com.ar/259910-las-claves-del-modelo-portugues-frente-al-coronavirus
<niemeyer> Yeah, the story is pretty good so far.. now to see how the second half of the story goes
<niemeyer> It's not true that under 15 haven't returned to school, btw... at least not universally true.. our son has been doing doing remote schooling for a couple of weeks now
<niemeyer> This week they started doing video calls with the whole class
 * facubatista -> car support
<facubatista> niemeyer, my kids are also with virtual classrooms and group video calls
<mup_> Issue operator#231 opened: Harness: Has errors on charms which define action handlers <Created by Vultaire> <https://github.com/canonical/operator/issues/231>
 * facubatista is back
<facubatista> new battery, ~USD 120
<Chipaca> brb, rebooting
<mup_> PR operator#232 opened: ops.testing.Harness actions metadata support.  (#231) <Created by Vultaire> <https://github.com/canonical/operator/pull/232>
<mup_> PR operator#233 opened: Execute the registered methods to events under PDB if proper envvar is set <Created by facundobatista> <https://github.com/canonical/operator/pull/233>
 * facubatista opened ^, but we need to talk around it
<facubatista> which will happen tomorrow
 * facubatista eods
#smooth-operator 2020-04-17
<Chipaca> ââáµ¢â ââáµ¢â
<niemeyer> o/
 * Chipaca doing dispatch like https://boingboing.net/2020/04/16/watch-a-plane-gracefully-land.html
<Chipaca> grah, fake_script_* depends on $0 being the script and not a symlink to it
<niemeyer> Symlinks don't work very well on windows
<niemeyer> With Python, specifically
<niemeyer> There's code in the interpreter that hides away the symlink name
<Chipaca> i think windows doesn't have symlinks of non-directories, but i might be wrong
<Chipaca> need to improve the test suite so it supports non-symlink things
<Chipaca> but that can be a followup
 * Chipaca â runch
<niemeyer> Good idea
<facubatista> Muy buenos dÃ­as a todos!
<facubatista> Chipaca, I'm not sure if #221 is ready to review or wip (does github have a way to signal this?)
<mup_> PR #221: ops/main.py: support dispatch <Created by chipaca> <https://github.com/canonical/operator/pull/221>
<facubatista> oh, I've found a "Still in progress? Convert to draft" thing
 * facubatista -> errands, will be back for the standup
<Chipaca> niemeyer: to start using spread, should i reuse the snapd credentials?
<niemeyer> Chipaca: To play with it you can set it up with your own credentials as you did in snapd.. once this is ready for merging we'll need to set up the repository as well
<Chipaca> ok
 * facubatista is back
<niemeyer> facubatista, Chipaca: I'll need a few more minutes..
<Chipaca> ooh, #232 is nice
<mup_> PR #232: ops.testing.Harness actions metadata support.  (#231) <Created by Vultaire> <https://github.com/canonical/operator/pull/232>
<Chipaca> might need changes but as a followup
 * facubatista -> lunch
<niemeyer> Chipaca: Indeed
<Chipaca> niemeyer: ah, missed your comment, i could fix that as well :)
 * Chipaca does so
<niemeyer> Thanks!
<Chipaca> bah
<Chipaca> niemeyer: all the assertEqual in that file seem to be (expected, got)
<Chipaca> boo
<Chipaca> no, i lie
<Chipaca> half of them?
<Chipaca> grmbl
<Chipaca> the ones in the odd-numbered tests
<niemeyer> Chipaca: :P
<Chipaca> definitely not something to burden this pr with
<niemeyer> Chipaca: Agreed, as mentioned there
<Chipaca> niemeyer: yeah, i thought it was just one test or two to change, related to the pr... but no
<niemeyer> Chipaca: I know for sure that the original tests had expected on right hand side, consistently
<niemeyer> Chipaca: (because I wrote them.. ;)
<niemeyer> Debating about the right way would be harder, though :)
<Chipaca> :-D
<Chipaca> the docs are very unhelpful
<Chipaca> assertEqual(first, second)
<Chipaca> psh
<Chipaca> anyhoo, i'm going to go buy bread
<niemeyer> Yeah.. let's just keep expected on right hand side..
<niemeyer> I think this is the most common convention for tests in general
<niemeyer> Including if statements
<niemeyer> There are certainly variations, but I think it's fair to say the constant tends to be after the variable more often than not
 * Chipaca nods
<Chipaca> i might make a pass to do that change later, if i need the distraction
<Chipaca> (ie if dispatch gets too gnarly)
<Chipaca> ok now yes i go out for a bit
<facubatista> Chipaca, we need to create the project in GH... `charm`? `charm-tool`?
<facubatista> `charm-cli`?
<niemeyer> charm-tool sounds like the better of these.. not sure if there's a better option
<facubatista> niemeyer, I wonder how this collides with *the current* charm tool
<niemeyer> facubatista: This is the next version of the current tool
<facubatista> niemeyer, yeap, but where is the old one hosted? shall we have a big sign in the README saying "don't confuse this `charm` with the other `charm`", etc
<facubatista> I mean, I expect some confusion about both, like I have a bug will not know in which project to open the issue, that kind of detail
<mup_> Issue operator#234 opened: Debugger "at" reserved words <Created by facundobatista> <https://github.com/canonical/operator/issues/234>
 * facubatista eods
<facubatista> see you all on Monday
