[00:18] <wallyworld> babbageclunk: we mis you
[00:18] <babbageclunk> oops! omw
[01:21] <babbageclunk> wallyworld: any idea why actions would just get queued as pending and never be run?
[01:21] <babbageclunk> worse - one did run and then others never have.
[01:21] <babbageclunk> (even on different units)
[01:24] <wallyworld> babbageclunk: the unit agent watches actions and reacts - it marks them as running when it executes them. so if an action is not being run, you need to look at the unit agent to see why
[01:24] <babbageclunk> I can't see anything in the debug-logs, I'll have a look in the files.
[01:25] <babbageclunk> thanks wallyworld
[01:46] <wallyworld> babbageclunk: i had a look at the action resolver in the uniter and there's no filtering - it really does just react directly to an action watcher that flicks across any queued/pending actions
[01:46] <thumper> babbageclunk: actions require the hook-execution lock
[01:47] <thumper> babbageclunk: look at the machine-lock file on the machine
[01:47] <thumper> and the juju_machine_lock command
[01:48] <babbageclunk> nothing holding the lock - I can see the first run releasing the lock at the end, and I've had hooks run after that. Just not getting anything for the new pending actions I've added.
[01:51] <babbageclunk> so that means that the watcher isn't seeing them for some reason?
[01:51] <wallyworld> maybe - you'd need logging in remotestate/watcher etc
[01:52] <thumper> babbageclunk: if you specify juju.apiserver to TRACE you'll see the details of all the api calls
[01:52] <thumper> that may help
[01:53] <babbageclunk> ok, thanks
[01:55] <babbageclunk> there's logging in remotestate/watcher for when the action watcher triggers, I can see it for the first one but not subsequent actions...
[02:02] <babbageclunk> it's almost like the actions watcher has stopped.
[02:06] <wallyworld> kelvinliu: the issue is that the secret labels in 2.7 include model name. that's not needed because the secrets for an app are created in the app's namespace and we know the model. if we remove that label it will work as 2.6.9 correctly uses the juju-app=foo label
[02:06] <wallyworld> configmap also uses model label
[02:07] <wallyworld> we should only use model lable for global resources
[02:07] <kelvinliu> ic. we should only have model labels for global resources
[02:07] <wallyworld> so it will be a quick fix
[02:08] <kelvinliu> i will fix it after current pr landed.
[02:21] <kelvinliu> wallyworld: so I just tested on gke, aks working fine, microk8s failed due to this issue https://github.com/ubuntu/microk8s/issues/757  I will test CDK on aws after lunch.
[02:24] <wallyworld> sgtm, ty
[02:24] <wallyworld> kelvinliu: is latest change pushed?
[02:25] <babbageclunk> thumper: can I pick your brains about this?
[02:26] <kelvinliu> the credential schema change,  yes, but I need rename the func name and to see if other clean up/rebase etc need to be done.
[02:26] <wallyworld> that's ok, i might try and test too
[02:31] <thumper> babbageclunk: sure
[02:31] <babbageclunk> in 1:1?
[02:31] <thumper> omw
[03:23] <wallyworld> kelvinliu: testing with CDK on AWS, the LoadBalancer service is not getting an external IP so bootstrap fails. not sure if that's related. might be an issue with integrator
[03:28] <kelvinliu> wallyworld: my cdk is deploying, i will see if it's reproducable
[03:30] <wallyworld> kelvinliu: also, the nodes are missing the juju.io/cloud labels so i am raising a bug
[03:31] <kelvinliu> yeah, it's missing https://pastebin.ubuntu.com/p/xZMPxCwJYn/
[03:58] <kelvinliu> wallyworld: i tested on cdk/aws, all good for me. I didn't have the LB provision issue.
[03:59] <wallyworld> must be my account or something
[03:59] <kelvinliu> missing -- trust?
[03:59] <wallyworld> juju deploy charmed-kubernetes --overlay /home/ian/aws-overlay.yaml --trust
[03:59] <wallyworld> that's supposed to work
[04:00] <kelvinliu> i use the same cmd and copied overlay from the doc https://pastebin.ubuntu.com/p/WMgVhNQyTF/
[04:05] <wallyworld> yup, i did too
[04:12] <wallyworld> babbageclunk: did you sort out the actions thing?
[04:17] <babbageclunk> wallyworld: nope
[04:17] <babbageclunk> it's really weird
[04:18] <wallyworld> what charm?
[04:19] <babbageclunk> wallyworld: it's my charm - but it doesn't seem like the watcher internals are passing along the update
[04:20] <anastasiamac> wallyworld: babbageclunk : PTAL https://github.com/juju/juju/pull/10801 - corrected creation of cloud call context
[04:21] <wallyworld> babbageclunk: i'll try deploying a stock charm and see if actions work
[04:24] <wallyworld> babbageclunk: hmmm, works for ubuntu-lite with a simple action i added
[04:24] <babbageclunk> wallyworld: and if you queue another action?
[04:29] <wallyworld> babbageclunk: works
[04:29] <wallyworld> this is on develop
[04:30] <babbageclunk> ok, I think it's to do with jam's changes. Can I talk through with you a bit? I'm pretty unfamiliar with the hubwatcher code
[04:31] <babbageclunk> wallyworld: in standup?
[04:31] <wallyworld> sure
[04:44] <wallyworld> anastasiamac: i have to duck out for a bit, a couple of quick initial questions have been left. i may be missing something
[04:47] <anastasiamac> k
[04:56] <anastasiamac> wallyworld: updated
[06:04] <wallyworld> looking
[06:05] <wallyworld> anastasiamac: while i look? https://github.com/juju/juju/pull/10800 small
[06:05] <wallyworld> hpidcock: did you need me to look at any wip?
[06:06] <anastasiamac> wallyworld: k
[06:09] <hpidcock> probably monday morning, I'm pretty tired going to finish up a bit earlier
[06:10] <wallyworld> anastasiamac: with the network facade, that is a model facade right? so technically it is ok as is i would have thought?
[06:11] <wallyworld> as it always gets the right state when it is constrcuted
[06:11] <wallyworld> it's only controller facades that get controller state
[06:12] <wallyworld> and the same st is being passed in anyway
[06:12] <anastasiamac> wallyworld: with network ine becaue we r using the code in common, we could potentially have a controller facade using it (dunno for what but possible ... since nothing stops us)... I did not want it to be a possibility
[06:12] <wallyworld> func NewNetworkConfigAPI(st *state.State ...)
[06:12] <anastasiamac> wallyworld: i think with this experience, every time we need call context, we should construct it
[06:12] <wallyworld> so the change does not prevent that st above being passed in
[06:13] <wallyworld> i can see passing in the one st is useful though
[06:13] <wallyworld> and making the context off that
[06:13] <wallyworld> just trying to understand the change
[06:15] <anastasiamac> wallyworld: with respect to networking facade, i.e. NewNetworkConfigAPI(...) like i said above - i don't think we should construct call context in constructor any longer... we should only construct it as needed in the individual call
[06:16] <wallyworld> yeah
[06:18] <anastasiamac> wallyworld: hence i've changed the signature and removed call context creation from constructor...
[06:24] <wallyworld> anastasiamac: func (s statePoolShim) GetModelCallContext(modelUUID string).... is there an issue there? we Release() modelState inside the func but then pass modelState.State as a return value expecting it can be used outside the func call.
[06:29] <anastasiamac> wallyworld: i don't think so.... the new implementation juts moves things around for convenience but is the same a old where we did call release in apiserver... and old one worked... so m sure the new one works too
[06:29] <anastasiamac> wallyworld: i tested live :D
[06:30] <wallyworld> ok, that reference counting code ia a bit hairy and i need to re-learn it
[06:30] <wallyworld> statepool, modelstatepool, systemstate etc etc
[06:31] <anastasiamac> the pr fixes the issue in the bug :)
[06:31] <anastasiamac> and makes sure that all similar calls are not wrong
[06:33] <wallyworld> indeed, i just wanted to make sure there wasn't a st leak
[06:33] <wallyworld> or a premature release of state from the pool
[06:33] <wallyworld> i'm still not 100% but if it works....
[06:33] <anastasiamac> ur concern is appreciate and all the vali questions :D
[06:33] <anastasiamac> valid*
[06:33] <wallyworld> i don't know the code well enough anymore
[06:33] <anastasiamac> it works in practice :)
[06:35] <anastasiamac> yeah.. that m.UUID() gave me hell... the model we get above is is not a full model... it's interfaced... but i think u r right, it's cleaner to add a method to Interface than to do the convulated dance
[06:43] <wallyworld> yeah, got to add it to mocks etc but cleaner prod code
[07:01] <manadart> wallyworld: Should I look at https://bugs.launchpad.net/juju/+bug/1849744 today?
[07:01] <mup> Bug #1849744: k8s upgrade to 2.7 results in broken apps <juju:Triaged> <https://launchpad.net/bugs/1849744>
[07:03] <wallyworld> manadart: nope, i already know the fix
[07:03] <wallyworld> just got to do a pr
[07:03] <wallyworld> it's about 3 lines of code and tests
[07:04] <wallyworld> i assume it's not blocking you?
[07:08] <manadart> wallyworld: OK. No not block me.
[07:09] <manadart> *blocking.
[08:23] <achilleasa> manadart: thanks for the bug link. I have approved 10779
[08:24] <manadart> achilleasa: Thanks.
[09:36] <stickupkid> manadart, the branch stuff is still behind a feature flag right?
[09:37] <manadart> stickupkid: Yeah. "export JUJU_DEV_FEATURE_FLAGS=generations".
[09:37] <stickupkid> manadart, nice nice
[09:52] <stickupkid> manadart,  are we calling the code internally to juju, branches or generations
[09:54] <manadart> stickupkid: It was initially just generations before we started using the "branch" name. So the packages, state collection and some types still use that nomenclature.
[09:54] <manadart> stickupkid: "branch" is generally the term for in-flight, not-committed generations.
[09:55] <manadart> stickupkid: One committed, the generation seq is bumped and the model has a new "generation", a high-water mark if you will.
[09:56] <manadart> There is a notion to make all changes use a branch implicitly, so each change will increment the model generation and you will be able to see a history of changes.
[09:56] <manadart> ^^ Once committed...
[09:57] <stickupkid> righto
[10:09] <nammn_de> manadart: if you want something small in the evening, a small PR regarding branches update  https://github.com/juju/juju/pull/10796 :D
[10:09] <nammn_de> manadart: was not 100% sure whats the way right now to work on status is, so I just opened the PR so that we can discuss in case we do it differently
[10:10] <manadart> nammn_de: OK, I'll take a look.
[10:39] <manadart> achilleasa: https://github.com/juju/juju/pull/10802
[11:39] <manadart> Oops. Looks like we have never handled space info in address migrations.
[11:46] <achilleasa> manadart: approved PR; one mock seems to have escaped the renaming
[12:57] <stickupkid> manadart, is there a way to see all branches ?
[13:03] <manadart> stickupkid: We are going to need such a command.
[13:04] <stickupkid> :-p
[13:39] <nammn_de> if someone wants to take a look at this small one. Changed the track err msg and update mockgen manadart ^ https://github.com/juju/juju/pull/10804
[13:40] <nammn_de> and the one we were just talking about how we read charm, simon already added nice comments https://github.com/juju/charm/pull/297 . I added my opnion why this also can be bad what i added
[14:36] <stickupkid> manadart, before I add some qa steps, anychance of looking at the WIP https://github.com/juju/juju/pull/10803
[14:37] <manadart> stickupkid: Yep.
[14:47] <N3tw0rK> noob question: how can i set the release of openstack, looks like it defaults to queens. Id like to set it to a more recent release.
[15:05] <pmatulis> N3tw0rK, if you're on an LTS the short answer is "UCA"
[15:07] <pmatulis> https://wiki.ubuntu.com/OpenStack/CloudArchive
[15:09] <N3tw0rK> pmatulis, ok is there anything juju specific that needs to be done once the UCA repo is added to the nodes?
[15:12] <pmatulis> N3tw0rK, are you installing O/S charm by charm or using a bundle?
[15:12] <N3tw0rK> pmatulis, charms
[15:13] <pmatulis> then this is the key syntax you need:
[15:13] <pmatulis> juju deploy <openstack-charm> --config openstack-origin=cloud:bionic-train
[15:13] <N3tw0rK> pmatulis, Thanks!
[15:15] <pmatulis> some non-openstack charms (e.g. ceph) will use a different option:
[15:15] <pmatulis> juju deploy ceph-osd --config source=cloud:bionic-train
[15:16] <hml> achilleasa:  we pass the space names as tags in some places… and _default is not a valid space tag currently
[15:17] <achilleasa> like constraints?
[15:17] <hml> :-)
[15:17] <hml> achilleasa:  “juju spaces”
[15:17] <hml> output
[15:17] <achilleasa> I sense a names.v3 PR coming soon...
[15:18] <hml> hahahaha
[15:19] <hml> achilleasa:  it would help with the Config validation as well.
[15:26] <hml> achilleasa:  it’s a bit ugly really… i don’t think we want to allow just an name with understore in front
[15:26] <achilleasa> can we have the underscore in the end instead?
[15:33] <stickupkid> manadart, it doesn't look like we validate the branch name in terms of uniqueness
[15:39] <manadart> stickupkid: I think we do for active branches, but you can use the name of a committed one.
[15:40] <stickupkid> manadart, yeah, ok, so let's say if it's not committed you can't pick that branch name
[15:40] <stickupkid> manadart, i'll add that validation
[16:48] <nammn_de> manadart: looks like my pr was not problem free. I need to address this and come back to you, probably I will remove the additional api call
[16:49] <nammn_de> manadart: seems like init cannot access the api yet
[17:37] <webstrand> I'm having trouble building juju 2.6.9; 2.1 builds fine. Any idea what I'm missing? http://dpaste.com/20WC576
[17:41] <manadart> webstrand: Juju switch to dep since 2.1. Have you run "go make dep"?
[17:41] <manadart> *switche
[17:41] <manadart> *switched dammit.
[17:42] <webstrand> Yep, I rebuilt all dependencies from Gopkg.lock
[18:10] <mbeierl> anyone else having charm build failures due to https://git.launchpad.net/layer-apt/ giving 503?