#smooth-operator 2020-07-27
<facubatista> Â¡Muy buenos dÃ­as a todos!
<facubatista> jam, hello! in which situations you're running flake8 yourself, in charmcraft?
<jam> morning facubatista
<jam> when I have a typo, I'd generally run flake8 again to make sure it is correct (eg, just that check)
<jam> that, and autopep8 runs whenever I save a file, which needs flake8 rules, though it only runs on the file that is modified
<facubatista> jam, which editor do you use?
<facubatista> it doesn't have integrated flake8?
<facubatista> jam, in any case, I +1ed your PR
<jam> Pycharm, it integrates via FileWatchers. It also gives warnings, etc built in. I'd still interact via the command line from time to time
<jam> thanks
<facubatista> jam, please remember my https://github.com/canonical/charmcraft/pull/93
<mup> PR charmcraft#93: Store status <Created by facundobatista> <https://github.com/canonical/charmcraft/pull/93>
<jam> will do
 * facubatista brb
 * facubatista is back
<alejdg> Hey folks, I got my jenkins-agent charm working fine when I deploy only 1 unit, but if I scale up, it starts to deploy multiple replicasets and more units than requested. I don't know why that's happening and I'd like to know if anyone has an idea.
<alejdg> Another thing is that I'm using StoredState to save information, and that information is pertinent for a single unit. Because of that, I believe I need to set a different spec for each unit or maybe use secrets instead, but I'm not sure how to that.
#smooth-operator 2020-07-28
<facubatista> power is back!
<facubatista> Â¡Muy buenos dÃ­as a todos!
 * facubatista got the power! https://www.youtube.com/watch?v=j1BNcSBApOU
<facubatista> jam, hello! did you see alejdg question from yesterday?
<jam> alejdg, I'd have to see how you are defining your pod spec, my guess is that you should be setting a single container pod spec, and that Juju would handle the 'count' portion. I'm not sure how you're trying to distinguish specifics for each unit
<jam> thanks facubatista I had missed it.
<facubatista> jam, my plan is to review all issues, delay current ones marked with milestone 0.3.0 to 0.4.0, except anything that we want to include this week
<facubatista> jam, I'll check out your PR first, see if we can land it, and then maybe change build to use it, so we start including everything in the project?
<jam> sgtm
<facubatista> jam, travis is not happy with your branch...
<jam> facubatista, thanks for the heads up, looking
<jam> looks like a 3.5ism
<jam> pathlib.Path *sigh*
<facubatista> yes, 3.5
<facubatista> jam, after you deal with that, let's have a HO to sync? just ping me, thanks!
<crodriguez__> I've heard about harness being the new way to do unit tests with the ops framework. Is there any documentation out there about it ?
<facubatista> crodriguez__, don't think so, but maybe we can provide you with examples?
<crodriguez__> facubatista, yeah, anything that could help
<ballot> Hello !
<facubatista> hello ballot
<crodriguez__> any link you could share facubatista ?
<facubatista> crodriguez__, 5'
<facubatista> crodriguez__, https://paste.ubuntu.com/p/67Hcd5yPT3/
<facubatista> jam, let me know if we can talk at some point today so I organize myself
<crodriguez__> thank you facubatista
<alejdg> jam:  How would I set specific secrets for each unit? Is there a way to do that?
<crodriguez_> I don't understand harness... I thought it was supposed to mock the charm? But it looks like it is executing functions of the charm directly on my system instead. For example, I am testing harness.update_config , which triggers render_config in my charm. render_config, amongts other things, tries to create a directory in /etc/, which is denied because of lack of permissions
<facubatista> crodriguez_, we'll need help from jam for that :/
<crodriguez_> facubatista, is he on PTO?
<facubatista> crodriguez_, nop, but we'll need to wait for tomorrow
<crodriguez_> Ok facubatista, thanks for the info. I might schedule a meeting/work session for that
<facubatista> crodriguez_, nice
<vultaire2> @crodriguez_: Re: your questions about operator framework - if you don't mind, I'm going to paraphrase a few of the questions here
<vultaire2> I missed backscroll in this channel as I just joined
<vultaire2> Your main question was: operator framework is supposed to mock the charm, thus why is it running stuff directly on your box?  ...Is that a fair way of rephrasing, or did I miss something?
<drewn3ss> "For example, I am testing harness.update_config , which triggers render_config in my charm. render_config, amongts other things, tries to create a directory in /etc/, which is denied because of lack of permissions"
<crodriguez_> pretty much yeah vultaire2
<drewn3ss> yeah, you've got it, vultaire2
 * drewn3ss is wondering iif you still need to mock things like the render as we do in reactive unit tests
 * crodriguez_ wondering the same thing :)
<vultaire2> Sorry for the delay here, jumping around a bit...
<vultaire2> So, basically: it is mocking a lot of the stuff.  You could kind of say it lets you test functions of your charm code without juju running underneath it
<vultaire2> that being said, it's not any sort of virtualization or anything like that, it's not spinning up containers or anything - all code _is_ running locally on your machine
<vultaire2> with normal unit testing, you're typically running single functions directly on your development system, perhaps mocking out areas where the code does things you don't want it to (like talking to a DB, for example)
<vultaire2> basically, with unit testing you're trying to test one function, or a small area of code, in isolation from the rest of the code
<vultaire2> Now, this is normally pretty difficult with charms, especially pre-operator charms, as there's a lot of stuff running "under" them
<vultaire2> all the juju binaries and such which get called behind the scenes
<vultaire2> if you're just running unit tests and not doing a full end-to-end integration test, you typically don't want that, and thus you end up having to mock out stuff.  Lots of stuff.
<vultaire2> With operator framework, a large amount of that is effectively mocked out for you
<vultaire2> Just like with non-juju unit tests, your tests run on your machine.  So if your charm does things that you don't want modifying your system environment (like installing packages), you'll still need to mock those bits out
<vultaire2> ...Let me stop here.  Does this make sense somewhat?
<drewn3ss> so, harness isn't providing a mocked virtual environment in total, but rather mocking out the juju layer in general?
<crodriguez_> ok, thanks for the explanation. Yes that makes sense, I'll need to mock out some functions so it doesn't mess up my system haha..
<vultaire2> yes
<crodriguez_> your video is helpful too with the examples!
<crodriguez_> I'll give it another shot and might ask more precise questions when I hit blockers
<drewn3ss> crodriguez_: you might look to the unit tests I was just poking at for mock_render  https://git.launchpad.net/charm-prometheus-ceph-exporter/tree/tests/unit/test_reactive_ceph_exporter.py#n95
<vultaire2> Sounds good, please do and best of luck
<drewn3ss> it's not operator, but maybe some of that code will help you mock the render
<crodriguez_> thanks drewn3ss!
<facubatista> vultaire, drewn3ss, thanks!
 * facubatista eods
#smooth-operator 2020-07-29
<jam> thanks for the explanation vultaire. You're right that Harness takes out the Juju side but doesn't change anything else about the environment
<facubatista> Â¡Muy buenos dÃ­as a todos!
 * facubatista is back for a while
<facubatista> jam, I put the router on the UPS and myself in the laptop... I may have "systems running" for a while
<facubatista> jam, let's do a HO to sync?
<jam> facubatista, sure, bug HO or standup?
<facubatista> jam, whatever, but please DM me the link
<crodriguez_> vultaire (or jam) so to mock an action, you need to create a FakeActionEvent class ? Would this class be created in the same file as the unit tests?
<crodriguez_> I tried to just use action_event = Mock() from this example https://paste.ubuntu.com/p/67Hcd5yPT3/ but I get `TypeError: 'Mock' object is not subscriptable`
<jam> crodriguez_, so ceph-iscsi went with https://github.com/openstack-charmers/charm-ceph-iscsi/blob/master/unit_tests/test_ceph_iscsi_charm.py#L192
<jam> They use a MagicMock so they can easily populate the params that are returned by the action event
<jam> The interface of ActionEvent is that it has a 'params' that is a dict containing the content from 'action-egt'
<jam> If you don't want to create your own, you can just use ActionEvent itself, but the backend would complain if you called log or set_results in your charm.
<crodriguez_> so in MagicMock, should I set results (result of the event) as a param ?
<crodriguez_> the FakeActionEvent class that vultaire shared sets set_result at least, so that doesn't fail. I can add log() as well
<vultaire> you set return values on magic mock functions via setting the .return_value property, in the general case
<vultaire> let me look at the slide deck, been awhile since I did this...
<jam> crodriguez_, params is the content that the Human passed when they did "juju run-action"A
<jam> set_results is what *your action* tells the human once it completes
<vultaire> ok, now I'm a little better contexted :)
<crodriguez_> mhm... well it works if I do it like that https://paste.ubuntu.com/p/csDbCRNpDH/ , it's just heavy with a whole class to replicate the event
<jam> generally there isn't a return value from set_results because you are setting them for someone else to read
<jam> crodriguez_, what you have seems reasonable and easier to understand than Mock (IMO).
<crodriguez_> jam: I don't understand that last statement. I should just not set any set_result in my charm ?
<jam> Ideally we'd have something like Harness.run_action(action_name, params) -> (results, logs) possibly raising an exception on Fail
<vultaire> crodriguez_: I tend to agree with jam.  Mock and MagicMock definitely have their place, but in this case it seems like you simply need something to take the place of an action event object - having a class which mimics the interface at a very basic level (FakeActionEvent) isn't necessarily a bad pattern
<crodriguez_> okay thanks, i'll keep it like that then
<vultaire> although I think it's fair to perhaps suggest that the operator framework perhaps offer some primitives to make this type of testing easier?
<crodriguez_> honestly.. I don't even see the point really in testing something like that. I'm basically overwriting the subprocess call with mock, forcing it to return true, and assessing it reported true.  I'm forcing a positive result
<crodriguez_> maybe I need to take a class on the benefits of unit tests lol
<vultaire> :)  unit testing is something that takes practice to get good value out of
<jam> crodriguez_, you *are* testing that you don't have typos in your code, and you have a way to test that if the subprocess succeeds the code reports success (doesn't call fail, etc)
<vultaire> the point is that you exercise your code and make sure that it flows the way you expect and gives the results (or exceptions) that you expect
<jam> I don't see a huge value of lots of unit tests here, if you aren't actually setting content
 * jam eods
<crodriguez_> true true :)
<vultaire> the value is certainly more obvious when you don't have to mock anything :)
<vultaire> for example, if you're doing a unit test of a factorial function and making sure that factorial(3) returns 6...  that's a very obvious test with clear value
<crodriguez_> it's not pointless. It's just hard to know what I should do unit tests against or not in a new charm
<vultaire> because you're testing that your function gives the correct result
<vultaire> so...  regarding that
<vultaire> well, granted, there's a number of different schools of thought here :)
<vultaire> one school of thought is test-driven (or test-first) development
<vultaire> which in a nutshell is where you write unit tests first (which will fail since you haven't written any code), and then you write the implementation which will make those tests pass
<vultaire> ...it's kind of an odd method for first-timers, but some people swear by it
<vultaire> I do use it, at least sometimes
<vultaire> in that case, you write unit tests just as a side-effect of developing
<crodriguez_> yeah I've heard about test-driven dev, haven't done it yet in my work though
<vultaire> ...if you write tests after the fact, then you could take a few different tacks... unit testing particular use cases, or unit testing to exercise certain branches of code
<vultaire> ...and in some cases, if you're just mocking too much, it may be better to do a more end-to-end test.  I'd say experience is a good guide for that.
<vultaire> as in the slide deck, while I believe most testing should be done at the unit test level (because the tests are small and *very* fast, typically), there's still absolutely value in e.g. deploying a model and interacting with actual units to make sure they do what you want
<vultaire> ...I hope this helps :)
<crodriguez_> yeah well I already have functional tests ready for this charm. I just though I'm *supposed* to do unit tests too to make it a good charm :P
<crodriguez_> I agree that func tests takes a lot of time though
<vultaire> well, one last bit is: have someone do a code review of your charm.  Get feedback from others, and let that guide you as you get more familiarity with all of this
<crodriguez_> so maybe I'll leave out "render_config" out of the unit tests.. It is the main thing tested by the func tests
<crodriguez_> ah yes of course
 * facubatista eods
#smooth-operator 2020-07-30
<ballot> jam: following our discussion yesterday about podspec "spec" documentation : https://bugs.launchpad.net/juju/+bug/1889519
<jam> thanks ballot
<barryprice> jam: oh one other thing, I noticed that on deploy, my units get Workload set to 'active' in juju status, but after a config change, new units are spawned and their Workload becomes 'unknown'
<barryprice> not very high priority but I thought I was handling that
<jam> barryprice, let me look
<mthaddon> barryprice, jam: I think that's because in configure_pod you're only setting status on the leader?
<barryprice> mthaddon: it affects the leader too though
<barryprice> hmm or maybe the old leader sets spec which causes new pods to come up and until config-changed next fires, there's no status set?
<barryprice> I'm a bit fuzzy about the order in which things happen there, do the new pods come up during config-changed or after it's fired?
<jam> barryprice, so none of the other units have a status set (as tom noted). For the leader itself, I would think that Juju would look at the readiness probe to determine when the pod has transitioned to fully ready
<jam> after being restarted
<jam> barryprice, IIRC, you should set the status to active, and then Juju will handle the fact that it isn't actually active while the pod is restarting.
<barryprice> ha, ok - thanks. this is low pri, it was just bugging me
<mthaddon> I'd expect the pods are rescheduled if there's a config change by "self.model.pod.set_spec(spec)"
<mthaddon> jam: https://paste.ubuntu.com/p/7n6t8kbq6M/ does that look like a reasonable .jujuignore for the jenkins-agent charm?
<jam> mthaddon, looks reasonable to me. Juju has some default ignores (like build), but I think it is fine to be explicit in your rules
<mthaddon> ack - I wasn't sure if there were defaults, but sounds good, thx
<barryprice> mthaddon: fyi https://github.com/canonical/charmcraft/blob/master/charmcraft/jujuignore.py#L208
<mthaddon> ah thx
<mthaddon> looks like I need to include trailing / for directories, will update
<facubatista> :hlo:
<facubatista> Â¡Muy buenos dÃ­as a todos!
<facubatista> jam, did you see my mail about release notes? I'd try to do the release now (in case my electricity provider decides to ruin my day later)
<jam> facubatista, morning. I did see them, I'll look over them once more and confirm. They seemed ok
<facubatista> jam, thanks!
<jam> facubatista, meeting?
<facubatista> oops
<mup> Issue operator#249 closed: Let's instrument all registered methods' calls <bug> <Created by facundobatista> <Closed by jameinel> <https://github.com/canonical/operator/issues/249>
<ballot> Hello there. Is there a way to retrieve the config value set for juju-external-hostname config item ? I'm logging self.model.config and the key doesn't seem to be here even when set.
 * facubatista -> errands, bb~1h
 * facubatista eods
#smooth-operator 2020-07-31
<facubatista> Â¡Muy buenos dÃ­as a todos!
<gnuoy> Hi, I'm converting a charm over to use charmcraft to build it and somewhere along the way I've lost a number of hooks files from the hooks dir. Should charmcraft fully populate the hooks dir at build time or are they supposed to be automagically created at deploy time ?
<gnuoy> I seem mention in charmcraft that a handle of hooks are created for old juju version, so I assume the full set are supposed to appear at deploy time ?
<gnuoy> After deploying I still only have those created by charmbuild
<gnuoy> I see https://github.com/canonical/operator/issues/257 talks about the hook files being missing if the install hook fails but from what I can tell the install hook ran fine.
<gnuoy> The charm log mentions that it is running the legacy install hook if that's relevant
<studentz> Advantages  of smooth-operator over reactive charms? Thanks
<gnuoy> ok, so my charm is dispatch aware, which appears to mean no symlinks for me https://github.com/canonical/operator/blob/master/ops/main.py#L178
<gnuoy> That being the case, when I am in a debug-hooks session for update-status for example, what should I run ?
<gnuoy> Looks like upgrading juju did the trick
<mup> Issue operator#362 opened: Block charm if Juju version is too low <Created by gnuoy> <https://github.com/canonical/operator/issues/362>
<facubatista> gnuoy, hello!
<facubatista> gnuoy, if you don't have any special needs, you can just avoid having a hooks directory: charmcraft will create a "dispatch" script (for new jujus) and a hooks directory with a couple of basic hooks (for older jujus), and the operator framework will do the rest
<gnuoy> facubatista, ack. ooi in a debug-hooks session what do I run ?
<gnuoy> If I'm trying to debug a charm issue
<facubatista> gnuoy, you can call the dispatch script setting the JUJU_DISPATCH_HOOK env... but you may want to use debug-code, which lets you debug directly inside the charm code
<gnuoy> facubatista, ah! ok thanks
<facubatista> gnuoy, sorry, JUJU_DISPATCH_PATH, not *HOOK
<facubatista> gnuoy, with debug-code you can tell it to debug a specific hook (and will leave you with a PDB session inside the running charm, at the beginning of your hook execution)
<facubatista> gnuoy, or you may even add breakpoints() inside the code (that will make the code interrupt ONLY if you're in a debug session)
<facubatista> gnuoy, you can see all that here: https://discourse.juju.is/t/live-debugging-at-python-level/2914
 * facubatista brb
<gnuoy> that looks great facubatista, thanks
<mup> Issue operator#362 closed: Block charm if Juju version is too low <Created by gnuoy> <Closed by gnuoy> <https://github.com/canonical/operator/issues/362>
<jesseleo> Hey guys I'm trying to test the install of my charm, but I'm having trouble  adding the resource in the test harness
<jesseleo> https://paste.ubuntu.com/p/gqnJXgvGyK/
<jesseleo> I see that add resource function was a recent addition so I'm installing ops repo from master because `add_resource` doesn't show up when you pip install ops
<jesseleo> I'm just not really sure what I'm doing wrong because it says that `add_resource` isn't an attribute of Harness
<jesseleo> but "begin" is an attribute of harness and that executes fine
<jesseleo> there is a nefarious print statement in there. ignore that^^
<facubatista> jesseleo, sorry, I'm eod, but will take a look at this on Monday
<jesseleo> no worries, enjoy your weekend
<jesseleo> facubatista
#smooth-operator 2020-08-02
<mup> PR operator#363 opened: 0.8 test charm no pickle <Created by jameinel> <https://github.com/canonical/operator/pull/363>
