[00:32] ping, anyone OCR today ? http://reviews.vapour.ws/r/4040/ [00:34] thumper: i'm currently seeing 1 in 3 landing attempts fail because mongo crashes with the usual bad mac error [00:35] davecheney: OCRs today according to Juju Team calendar r Andrew and Eric [00:39] cmars: thanks [00:50] davecheney: I'm hoping that RSN we will just have mongo 3 where they have fixed that [00:53] Bug #1552469 opened: Credentials are not utilised when creating a hosted model

[01:06] thumper: menn0 https://github.com/juju/juju/pull/4611 [01:06] as we talked about yesterday [01:06] this is part 1 of 2 [01:07] to make sure that the apiserver knows all the ways we say that we're in upgrade state [01:07] and can convert all of them to the right ErrorCode [01:09] done [01:19] davecheney: your PR makes me cringe [01:19] davecheney: can we not have a well defined "upgrade in progress" error in state [01:20] davecheney: which the apiserver knows how to convert to it's "upgrade in progress" error? [01:23] davecheney: the reason for state returning "upgrade in progress" are quite different from the error returned by the apiserver. [01:24] davecheney: the upgrade in progress error from state is about an attempt to start an upgrade failing because an upgrade is already in progress [01:25] davecheney: maybe it should just be a different error? [01:32] menn0: i agree [01:32] it's a disaster [01:32] to be clear, I'm not going to pass state.errUpgradeInProgreess up [01:32] but there are a bunch of "helper" (using that term very loosely) methods in apiserver/common/errors.go that translate one to the other [01:36] Uh big +1 to that === bogdanteleaga_ is now known as bogdanteleaga === natefinch_ is now known as natefinch === _stowa_ is now known as _stowa === bcsaller_ is now known as bcsaller [03:27] wallyworld: hey === meetingology` is now known as meetingology [03:27] hi === mup_ is now known as mup [03:27] wallyworld: got 5 min for a quick chat? [03:27] sure [03:27] 1:1 === marlinc_ is now known as marlinc [03:41] Bug #1552523 opened: Unable to deploy a wily or precise service with MAAS

=== bradm_ is now known as bradm [04:32] anastasiamac: you'll hate me, but i'd love a review when you get a moment http://reviews.vapour.ws/r/4052/ [04:33] wallyworld: i won't hate u fo asking so nicely :D [04:33] wait till you see the code :-) [07:03] anastasiamac: sorry :-( another one if you have time http://reviews.vapour.ws/r/4053/ [07:03] wallyworld: sure, m hald way thru the 1st... then will look at this one \o/ [07:03] yay, ty [07:54] Bug #1552589 opened: TestProvisionerObservesMachineJobs fails intermittently

[08:39] anyone free to talk me through the space-discovery login synchronisation a bit? [08:40] frobware, dooferlad_? [08:41] actually brb but ping me if you know it [08:50] quick review: https://github.com/juju/juju/pull/4614 [08:52] jam, ship it -- the notification is yucky in the first place but it's not worth a rewrite to fix [09:03] frobware, dooferlad_ : in particular, ISTM that: it runs per-model but limits logins for everything; but works around that by only limiting logins until the first one that runs reports completion; but subverts *that* by always marking completion even when there's an error [09:03] frobware, dooferlad_ : any insights? [09:08] frobware, dooferlad_ : in particular my understanding is that this *is* only critical for the controller model (especially given that it's ineffective for later ones): can you think of a reason for me not to move it to the per-controller bit rather than the per-model bit? [09:09] frobware, dooferlad_ : ah, I'll ask voidspace [09:12] fwereade: ask me what... :-) [09:13] voidspace, huh, thought I'd PMed you to avoid spam [09:13] ah, you have I just didn't notice === dooferlad_ is now known as dooferlad [10:01] voidspace, fwereade, jam: standup? === Odd_Blok1 is now known as Odd_Bloke [10:02] dooferlad: frobware: just finishing a chat with fwereade - be right there [10:11] voidspace: dooferlad: http://reviews.vapour.ws/r/4055/ [11:03] morning all [12:06] frobware: you still need that review? [12:06] voidspace: yes please [12:10] frobware: can you explain to me the change in provider/maas/environ.go [12:10] frobware: it's relatively esoteric [12:11] voidspace: when the container starts it has no gateway, so it can start but can't do anything real. [12:11] frobware: doesn't setting juju_bridge_all_interfaces=0 make the code immediately following unroutable [12:12] frobware: yeah, that's the bug - I mean what do the changes do specifically? [12:12] voidspace: we know the bridge-device the container is connected to (the param in the call) so we find the IP address for that specific device and make it the gateway [12:12] (I may have interrupted - sorry if so) [12:12] what does this do? uju_networking_preferred_python_binary %[1]q --bridge-prefix=%[2]q --one-time-backup --activate %[4]q [12:13] and: [12:13] juju_ipv4_interface_to_bridge=$(ip -4 route list exact default | head -n1 | cut -d' ' -f5) [12:13] dammit [12:13] voidspace: ah, sorry. I was talking about the wrong file. :) [12:13] voidspace: first bit, bridges all interfaces with the specified bridge prefix, in our case 'br-' [12:14] frobware: line 1046 in provider/maas/environ.go [12:14] frobware: won't juju_bridge_all_interfaces always be 0? [12:15] voidspace: yes, I did that deliberately because if we merge this back from master into maas-spaces2 we should make that 1 in our branch. So, rather than continuously futzing with the changes we can choose the behaviour we want by specifiying 0 or 1. [12:15] frobware: ah... cool [12:16] voidspace: the 0 behaviour is what we want on 1.25 too [12:16] voidspace: so the if .. else ... serves as an example of how to drive the script for bridging all interfaces or just a specific interface. [12:18] voidspace: I can add a comment for this behaviour. [12:18] frobware: yeah, that would be cool [12:21] frobware: not possible to test? [12:25] voidspace: in terms of bridging all or a specific one? [12:25] frobware: the change has no test [12:25] voidspace: there are existing tests cases that exercise the all or one behaviour. [12:26] voidspace: this is a runtime thing... which I think is difficult to test in any meaningful way. Where & how would we bring interfaes up/down? [12:26] frobware: well, the finding of the ipv4 address should be testable with some mocking (the discoveryIPv4InterfaceAddress path) [12:27] voidspace: ah, that one. [12:27] voidspace: true. [12:27] voidspace: it's a tradeoff though. this would disappear once we merge maas-spaces2. [12:28] frobware: you want to land it untested? [12:28] voidspace: it's been live tested [12:28] frobware: heh, ok [12:28] voidspace: let me think about it a bit... [12:29] frobware: it will live on in 1.25 [12:29] voidspace: nope, master only. [12:29] ah, ok [12:29] voidspace: and, again, only for as long as we don't merge maas-spaces2 [12:29] frobware: you have my LGTM with a comment about the missing test [12:29] voidspace: ironically, the unit tests did not highlight that there would be a live failure. [12:30] voidspace: https://bugs.launchpad.net/juju-core/+bug/1549545/comments/8 [12:30] Bug #1549545: Bundle deploys fail at lxc-start when bridge br-eth1 is created

[12:32] yeah, I saw [12:32] :-/ [12:32] basically impossible to unit test all possible networking scenarios [12:34] frobware: I found out why my tests were failing in a weird way [12:34] frobware: the maas provider caches the result of checking if spaces are supported on creation [12:34] frobware: so setting the capabilities after creating the provider has no effect - you have to do it first [12:35] down to 3 failures now [12:35] heh === lazypower_ is now known as lazyPower [14:50] voidspace, hmm, how does discoverspaces work in an HA environment? [14:51] voidspace, it's in the singular runner, so... will logins ever be unblocked for non-master controllers? === rcj` is now known as rcj [14:57] voidspace, ah, ok, it will, but only because it's not blocked until we start the worker... which surely implies that logging in after that point is a matter of luck? [15:11] voidspace, I'm starting to think it's impossible without a persistent flag..? and I was really hoping not to have to change the apiserver. ah well === icey_ is now known as icey [15:17] fwereade: :-( [15:18] back shortly, picking up the daughter from school [15:21] fwereade: (not a matter of luck as you bootstrap one machine - which does discovery and is blocked - and then you go HA.) [15:34] voidspace, what ensures that the client won't connect after the apiserver starts but before the discoverspaces worker starts? [15:45] dooferlad: ping [15:46] frobware: pong [15:46] dooferlad: can you junp into sapphire HO? [15:46] frobware: sure [16:19] fwereade: so, nothing stops login before the discoverspaces worker has startedc [16:19] fwereade: the intention of blocking logins was that bootstrap be synchronous so that spaces are available immediately after bootstrap completes [16:21] fwereade: non-master HA controllers never unblocking sounds scary [16:22] fwereade: although it shouldn't happen - as it's in a singular runner the worker shouldn't be started on non-master [16:22] fwereade: and login is only blocked if the worker is started (the channel is set the first time the worker is started) [16:25] voidspace, right; so, how do we prevent the bootstrapping client from logging in before the discoverspaces worker is started? [16:26] fwereade: hmm... that might explain a bug that frobware has seen that I haven't (bootstrap completing and *then* login being blocked) [16:26] fwereade: I've never seen it, but it's obviously timing related [16:26] voidspace, yeah, there's nothing stopping that afaict [16:26] there isn't, setting the channel before the worker is started would be problematic for non-master HA [16:29] fwereade: are you sure you don't want me to own fixing this? [16:29] voidspace, yeah; and I'm not entirely willing to be certain that we can't have problems if we manage to bring up extra controllers without waiting [16:30] we need a different way of doing this [16:30] voidspace, still think it's my problem really -- I'm trying to get my dependency-engine update of startEnvWorkers in [16:30] fwereade: maybe remove the blocking altogether until menno's work is done and then re-introduce [16:31] fwereade: that's only nearly as bad as just doing it for the controller and not per-model [16:31] voidspace, ...that's terribly tempting actually [16:31] fwereade: if it has to be fixed anyway [16:34] voidspace, ok, I will try just dropping the notification for now and see where it takes me [16:34] voidspace, thanks [16:34] voidspace, I'll let you know where I end up [16:35] fwereade: cool [16:39] bbl [17:01] natefinch: hey can you join us in moonstone rq? [17:01] Bug #1552804 opened: worker/metrics: intermittent data race

[17:04] cherylj: is bootstrapping xenial a gating issue for 1.25.4? [17:04] frobware: yes, I would say so. It came up in the cross team call today [17:04] frobware: what are you finding? [17:05] cherylj: that seems odd to me as you have to use daily images (i think) and there's a lot of shifting sand if you do that. [17:06] cherylj: I've possibly spent a large part of the day looking at a non problem but last comment in bug #1550306 shows a correlation of xenial cloud-init failing and bootstrap failing [17:06] Bug #1550306: 1.25.3 can't bootstrap xenial environments

[17:07] :( [17:07] frobware: I'm still trying to get xenial images on my vmaas to test. The controller ran out of disk space [17:07] cherylj: the issues I saw with /e/n/i yesterday have definitely gone with today's images as /e/n/i is rendered differently. bad timing perhaps. or the curse of shifting sand. [17:08] cherylj: I'm still surprised we would consider 1.25.4 x a-daily-build-of-xenial something that is gating. [17:08] cherylj: as I've seen in the last 24 hours, daily images change... [17:09] frobware: If we can confirm that it is a problem with xenial, I think we could release and point to whatever bug we've opened against xenial cloud images [17:09] how do I get rid of all the params redacted thingie in logs? [17:09] bogdanteleaga: to see the redacted params? [17:09] yeah [17:09] bogdanteleaga: set logging level to trace [17:10] it will show you the contents [17:10] cherylj: I haven't gone beyond bootstrapping today, but with the bridge script fix for xenial (1550306) I can still bootstrap with precise, trusty and wily. [17:10] frobware: can you deploy? [17:10] cherylj: kinda run out of time now... :( [17:11] cherylj, thanks that did it :) [17:13] rick_h__: I assume the charmstore should have the same contract about push-resource matching the extension? [17:13] natefinch: ping? [17:13] katco: yo [17:13] natefinch: must have missed my previous ping ;p [17:13] natefinch: join us in moonstone? [17:13] katco: oh, crap, sorry [17:13] natefinch: lol nw [17:13] katco: new IRC client is totally screwing me up [17:25] natefinch: yes [17:31] Bug #1552815 opened: ability to apply profile/config modifications to lxd containers [17:45] katco, ping? [17:48] mattyw: pong [17:50] katco, hey hey, pinging you as I thought you might have good intuition on this: when I come to destroy controllers I normally forget the name I've given them. I reckon juju status should tell you the name of the controller (and also maybe the model) it's displaying the status of. What do you think? [17:50] mattyw: hm [17:50] mattyw: that seems reasonable to me... maybe like a header at the beginning of status? [17:51] mattyw: especially in a world where you might be flipping between models very often [17:51] katco, yeah, what more or less exactly what I was thinking [17:51] mattyw: the name of the controller may not carry it's weight since there will likely only ever be 1? what do you think? [17:53] katco, I don't know, I think it would be useful to display it, even if it's only one, since the user has named it [17:53] mattyw: i mean i'd be fine with it. it's not like it would take up a lot of space [17:53] katco, I'll do it, and see how it looks [17:53] mattyw: cool [17:54] katco, wanted to get some initial feedback to check it wasn't a totally insane idea [17:54] katco, so thanks [17:54] mattyw: hth [18:38] back [18:38] ericsnow: wondering if ListResources in the charmstore should be ResourceMeta, to match Meta about charms [18:38] natefinch: we need all the info about the resources, not just the meta portion [18:39] ericsnow: it's all meta, except the bytes [18:39] natefinch: "meta" means the resource info from the charm metadata [18:39] natefinch: a la resource.Resource vs. resource.Meta [18:41] ericsnow: in the charm package, sure. It could have a different meaning in the charmstore... [18:41] ericsnow: we are calling id/meta/resources after all [18:42] natefinch: naming conflicts aside, ListResources() needs to return the full resource.Resource from the charm store [18:42] ericsnow: resource.Resource is all the metadata about the resource :) (metadata -> everything except the data-data, i.e. the bytes) [18:43] natefinch: okay [18:44] ericsnow: anyway, I was just trying to keep with the conventions already there. It's fine by me if we use ListResources [18:44] natefinch: oh, this is not a match to charm Meta [18:45] natefinch: it is the resource "meta" plus the other things that resource.Resource adds [18:45] ericsnow: I'm just talking about the function name to wrap the call to id/meta/resources [18:45] natefinch: so more like the charm store's "entity" [18:46] ericsnow: as an aside, I think the charm/resource.Meta name was a mistake, since it's all metadata... that's possible more accurately called the resource definition (as defined in the metadata.yaml)... [18:46] natefinch: there isn't an equivalent ListCharms on the client; I'd say let's stick with ListResources() [18:46] ericsnow: that's fine [18:47] natefinch: hmm, I believe I originally called it Definition and someone pushed for Meta instead [18:47] natefinch: (maybe remembering something else) [18:47] ericsnow: I remember something about definition too... not sure if that was this or payloads or what [19:58] jam: you're currently working on lxd/juju right? [20:01] jcastro, he is but is out for the day [20:01] ok so maybe you can help [20:02] the new lxd landed in ubuntu, so now I need to update the instructions: https://jujucharms.com/docs/master/config-LXD [20:02] since lxd-images is being deprecated [20:02] the nice bit is that lxd now includes the remotes for the images built in, so doing like `lxc launch ubuntu:14.04 my-test-container` Just works ootb. [20:03] my question is, will we need this step at all in our instructions or if the lxd provider will just snag the image and pipe the output to the user [20:04] jcastro: hey [20:04] jcastro: so the provider could certainly do that; however, one nice side-effect is that the provider will use any image named ubuntu-trusty [20:05] jcastro: so it's kind of nice that you can pre-build an image and the provider will just use it [20:05] jcastro: or use alternative images (i.e. not trusty) [20:05] ok so I should just change the import command to making an alias [20:05] oh wait [20:05] I just thought about what you said [20:05] so I can premake an image [20:05] jcastro: yep :) [20:06] and as long as it's aliased as ubuntu-trusty that just works? [20:06] jcastro: i expect we'll refine this if jam and tych0 haven't begun doing so already [20:06] jcastro: you may have to twiddle with where tools are pulled from [20:06] jcastro: kindly open an informative docs bug and i'll jump on it [20:06] wait a minute [20:06] did you guys just give us image based workflows? [20:06] jcastro: but what i've done is bootstrapped a xenial host (i think), snapshotted it [20:06] jcastro: for local only, sure [20:06] jcastro: and then just alias that, and standup time is super quick [20:07] jcastro: shhhhhh [20:07] pmatulis: I think I can submit a PR, quick fix once I sort the command [20:07] jcastro: don't use the i word! [20:07] heh [20:07] marcoceppi: ^^ look what just happened! [20:07] no! nothing to see here. ignore the images behind the alias [20:07] right, I can see how that would be bad and unsupported [20:08] jcastro: more like off-brand. i'm sure solid workflows could be built off it [20:08] jcastro: oh, yeah, I knew about that ;) [20:10] hmm ok, so really the only thing we need to update is figure out how to make the ubuntu:14.04 remote image aliased to ubuntu-trusty on my local machine [20:11] jcastro: yeah [20:15] so jcastro I was chatting with jam about this this morning [20:15] we will be making some updates to the user experience on this [20:15] and I would love your input [20:16] I might schedule some time to chat [20:16] yeah, I kind of feel like, since LXD has remote images available already [20:16] pmatulis, the workflow for using lxd provider is in the release notes [20:16] and I do juju bootstrap, just spawn it, one less step for us [20:16] jcastro, yep [20:16] jcastro, we agree [20:16] but I dont want to loose the ability to tag an image and use it [20:16] alexisb: keep in mind mark's requirement that we be able to specify the image to use [20:16] alexisb: yep [20:17] alexisb: jcastro is all over it. i'm on standby [20:17] katco, yep [20:17] alexisb: (goes into muppet mode) yep yep yep yep [20:17] uhhuh uhhhuh [20:17] :) [20:17] https://www.youtube.com/watch?v=vh3tuL_DVsE === thumper is now known as thumper-afk [21:23] Bug #1552523 changed: Unable to deploy a wily or precise service with MAAS

[21:33] ericsnow: standup time! === thumper-afk is now known as thumper [22:54] help! juju init doesn't work in beta1 [22:59] marcoceppi: there's nothing to init [22:59] rick_h__: dude [22:59] read the help output [23:00] now I hav to create the XDG_DATA path and stuff [23:00] marcoceppi: it should be there with install for the cloud list [23:00] marcoceppi: and the add-credential command is coming to esit that file [23:17] Bug #1456717 opened: TestUpgradeStepsStateServer fails

[23:17] Bug #1552948 opened: metricsmanager DB collection shouldn't be "global" [23:18] rick_h__: I need help, azure 2.0-beta1 [23:18] demo in 45 mins [23:18] it's asking for an auth-type [23:19] userpass [23:19] marcoceppi, LP:#1544890 [23:19] Bug #1544890: "ERROR the name of the model must be specified" when 'juju init' required

[23:20] i opened this before i realized juju init was going away, but still [23:24] marcoceppi: heading toncomputer onw sec [23:24] rick_h__: I got it sorted [23:24] it's bootstrapping [23:24] marcoceppi: k [23:25] was just getting a little tight, since it takes 20 mins to deploy and I couldn't get 2.0 to bootstrap [23:25] <3 the error messages, they were helpful [23:25] marcoceppi: k, email sent with all the details I sent to another perosn as well [23:25] marcoceppi: has sample file in it and such [23:25] marcoceppi: in case it's giving you any grief [23:26] rick_h__: I used beta1 release notes to find what I needed to copy from environments.yaml [23:26] marcoceppi: k [23:26] I already had ARM setup in alpha3 when it landed, thankfully [23:26] tried to use 1.25 though, and realized my mistake [23:26] ah cool [23:26] so have to plow through with 2.0 for the demo [23:27] you have the alpha gui? [23:27] https://jujucharms.com/u/juju/juju-gui [23:27] marcoceppi: ^ is the beta to work on top of juju 2.0 [23:27] marcoceppi: some bugs, but supposed to work [23:27] rick_h__: ack, no more ~yellow? [23:27] I just need it to show the model, everything else is cabs cabs cabs [23:28] thanks for the mail, looking forward to beta2! [23:28] marcoceppi: it's still in ~yellow but wanted to move to a more 'public' space if we're going ot hand out a beta widely [23:29] marcoceppi: so we'll put betas under ~juju from now on as it's more meaningful than yellow [23:29] rick_h__: cool beans [23:32] i need to parse a user-facing string into an apiserver/params type in a couple of different api client packages. where is the best place to put such a ParseXYZ() function? [23:33] wallyworld ^^ any suggestion? [23:34] wallyworld, I want you to know that marcoceppi reads release nots: [23:34] rick_h__: I used beta1 release notes to find what I needed to copy from environments.yaml [23:34] cmars: one sec, just finishing standup [23:34] marcoceppi, I am sure that line just made wallyworld's day [23:36] alexisb: read the release notes? I live by them. esp for the 2.0 releases [23:36] it's like a little gift from core every few weeks [23:36] lol [23:41] fuck, I can't get rid of an errored unit [23:48] rick_h__: I found a fun gui bug [23:49] alexisb: marcoceppi: you did just make my day :-) we put a lot of effort into those notes [23:49] cmars: what sort of parsing? [23:50] hatch: ^ [23:50] marcoceppi: what's up? [23:51] rick_h__: in the ~juju/juju-gui, if I got to remove units for a service, it just creates a new machine instead of removing the unit [23:51] marcoceppi: i think they're on that one [23:51] cool [23:52] i just realized this whole demo is borked because cabs can't speak 2.0 api [23:52] oh no! [23:56] oh uh