[01:10] abentley: don't suppose you're working? CI seems to have died... merge job has been running for 2 hours [01:11] I'll just kill it and hope for the best [02:14] * thumper heading out for a bit, taking daughter to BJJ === natefinch-afk is now known as natefinch [03:40] google: ps human readable memory [03:40] response: [03:41] nobody's put a -h on ps, huh? === Makyo is now known as Guest78890 === meetingology` is now known as meetingology === JoseeAntonioR is now known as jose [05:42] nobody's put a -h on ps, huh? [05:42] oops wrong window [05:52] oops wrong window [06:31] Bug #1516891 opened: juju 1.25 misconfigures juju-br0 when using MAAS 1.9 bonded interface [06:34] Bug #1516891 changed: juju 1.25 misconfigures juju-br0 when using MAAS 1.9 bonded interface [06:43] Bug #1516891 opened: juju 1.25 misconfigures juju-br0 when using MAAS 1.9 bonded interface === mup_ is now known as mup [09:04] dimitern, ping 1:1? [09:05] frobware, sorry, omw [09:54] wallyworld: http://reports.vapour.ws/releases/3318 -- azure-arm-provider is blessed. what's the process to merge into master? [09:54] axw: yay. when master unblocked, just create a branch, merge in tip of feature branch, and propose [09:55] wallyworld: cool. no need for review on that I take it? [09:55] axw: judgement call, usually no [09:55] IMHO [10:01] voidspace, frobware, jam, fwereade, standup? [10:14] dimitern: bugger, bugger, bugger [10:14] got distracted by work [10:14] sorry [10:15] dimitern: frobware: dooferlad: I guess you've finished [10:15] voidspace: yep [10:16] dimitern: frobware: dooferlad: my standup then - I made the changes to add a machine.Principals method for my "address picking algorithm change" [10:16] voidspace, :) hey, you didn't miss much [10:16] dimitern: frobware: dooferlad: that made tests pass on CI - it's landed on 1.25 but master is still blocked [10:16] voidspace, nice! yeah - too bad for master :/ I have 3 PRs blocked as well [10:17] I thought I'd finished the implementation of Subnets using the new MAAS API and am waiting on the test harness - will need to sync up with dooferlad for that [10:17] so I'd started on the ListSpaces implementation [10:17] voidspace, please, call it Spaces() [10:17] in the process of that this morning I discovered a flaw in the new Subnets code (wasn't filtering by subnet ID) [10:17] dimitern: ok - I thought ListSpaces was your idea [10:17] but fine [10:17] voidspace, it was an early idea, but since we have Subnets() now, let's keep the consistency in names [10:18] and it was fixing the subnet ID filtering that distracted me [10:18] done now [10:18] dooferlad: have you made a start on the maas test server update? [10:18] voidspace: yes [10:19] voidspace, cheers [10:19] voidspace: working on it now. Will ping you when I have something worth sharing [10:19] dooferlad: cool, let me know if you have any questions [10:19] dooferlad: great! [10:19] dooferlad: I'll carry on with Spaces then - can land it as a single branch [10:23] dimitern: environ.Spaces() can just return a map of string -> []SubnetInfo - right? [10:23] dimitern: and should we do anything with the maas "Default Space" (named like that) [10:23] dimitern: we want to make sure our default space is the same as the maas default space [10:24] otherwise nothing works [10:26] voidspace, I think Spaces() needs to return []network.SpaceInfo [10:27] dimitern: ok [10:27] dimitern: all SpaceInfo will have is a name and a []SubnetInfo [10:27] voidspace, and that in turn should have a Name, ProviderId, Subnets []network.Id [10:27] there's no provider id beyond name [10:27] dimitern: you want network.Id for subnets rather than SubnetInfo ? [10:27] voidspace, I don't think we should care about the "Default space" [10:28] dimitern: what do you mean by "not care" - not include it? [10:28] voidspace, well, if we require Spaces() to return the full subnets info, wouldn't that be too restrictive on providers? [10:28] voidspace, no, include it - we just won't use it for openstack [10:28] the only provider we have that can list spaces is maas [10:28] voidspace, for now [10:29] and I suspect that for any provider we implement spaces for, the only way we'll get a slice of subnet IDs is by getting full subnet info anyway [10:29] voidspace, will it be easier to do []SubnetInfo instead? [10:29] voidspace, I mean, SpaceInfo with Subnets []SubnetInfo [10:29] it would mean throwing away less information away [10:29] so marginally less work [10:30] voidspace, ok, let's do it so Spaces() return the full subnets info [10:30] and include default space [10:31] we need to be careful when we populate the model that we don't end up with a "juju default space" and a "maas default space" that are actually the same thing but we treat them differently [10:33] voidspace, well, since a juju space name cannot contain a space, "Default space" will be the ProviderId, the juju name will need to be something like "default-space" I guess [10:33] dimitern: so I need to transform all maas names into juju compatible ones [10:34] dimitern: so ironically the "Provider Id" will be much more human readable than the "Space Name" [10:34] dimitern: why do we have that restriction? [10:34] voidspace, because of constraints and bindings [10:34] ah [10:34] fair enough [10:35] voidspace, the transformation should be just lowercase it and replace spaces with - [10:35] dimitern: sooo... when I fetch the space name from the maas subnet result I need to transform there too [10:35] SubnetInfo.SpaceName has to be the juju space name [10:36] voidspace, yeah I think so [10:37] cool [10:37] dimitern: thanks [10:37] dimitern: hmmm... that's a one way transform though [10:38] dimitern: (i.e. you won't be able to deterministacally go from the SpaceName on a Subnet to the MAAS space) [10:38] dimitern: you can list spaces to see that though [10:38] which will have both [10:38] probably doesn't matter [10:38] voidspace, when talking to maas we always need to use the provider id - we should have it once the space is imported [10:39] dimitern: sure [10:39] dimitern: but if someone sees a SubnetInfo (i.e. output from subnets list) the space name shown there is *not* the maas space name [10:39] voidspace, but I don't think we'll need to do juju space name -> maas space provider id anywhere [10:39] I'm talking about the user - not us [10:39] voidspace, ah, sure - we need to have ProviderSpaceId on SubnetInfo [10:40] voidspace, in addition to SpaceName [10:40] that means a new facade [10:40] as it's a change to the api output [10:40] are you *sure* we need that as it's a lot of extra work [10:40] (well - a moderate amount) [10:40] voidspace, ProviderSpaceId is optional [10:40] I say we should be aware of it but not do it for now [10:41] voidspace, but good point about the versioning [10:41] dimitern: still need a new facade, if someone writes any code depending on that value being present then they need to know which api version to require [10:41] voidspace, that's the client subnets facade you're talking about? [10:41] dimitern: yep [10:41] as it's in 1.25 [10:41] voidspace, yeah, I think before the release we should bump the version for sure [10:42] dimitern: I say we don't need SpaceProviderId on SubnetInfo yet [10:42] voidspace, but for the time being let's just keep it in mind (probably add a card for it though, once your change lands) [10:42] cool [10:42] nobody may actually care in practise, but if they do we can add it [10:43] voidspace, we should be serious about api backwards compatibility [10:43] voidspace, but given the time frame, it can wait for a while [10:52] dimitern: so I think I do need to add SpaceProviderId to SubnetInfo - as it's needed for Spaces [10:52] dimitern: however I don't need to add it to the apiserver SubnetResults [10:53] voidspace, not yet, yep [10:53] voidspace, it can stay within the provider code [10:53] voidspace, and for importing at bootstrap we don't need to modify the api facades at all, since we have access to state at that point [10:54] dimitern: and access to the provider [10:55] dimitern: but we do need the proper provider id, so it will need to be on SubnetInfo (which is what we build the state model from) [10:56] voidspace, we do have access to the provider when we have the environ config [10:56] voidspace, network.SubnetInfo != params.SubnetInfo btw exactly for that reason [10:57] yep, understood [11:02] Bug #1516975 opened: Incompatible version formats [11:11] Bug #1516975 changed: Incompatible version formats [11:14] Bug #1516977 opened: juju upgrade-juju should guess version number [11:26] Bug #1516977 changed: juju upgrade-juju should guess version number [11:26] Bug #1516975 opened: Incompatible version formats [11:32] Bug #1516975 changed: Incompatible version formats [11:32] Bug #1516977 opened: juju upgrade-juju should guess version number [11:44] Bug #1516989 opened: juju status broken [13:02] straw poll: `type NotifyChan <-chan struct{}`, or `type NotifyChannel <-chan struct{}` [13:03] jam, dimitern, frobware, anyone? ^^ [13:08] frobware, I'd vote for the first one [13:08] sorry :) I meant fwereade ^^ [13:09] dimitern, cheers [13:17] dimitern, fwereade: for me the latter. :) [13:18] fwereade, speaking it out loud (in my head) I'm talking about a "channel" [13:20] frobware, yeah, but the type name is "chan" and it's more common in the go code base to use Chan vs Channel [13:22] dammit, now it's even again, if someone wants to tie-break? [13:23] I do favour Chan myself but can't really muster up very strong arguments either way [13:23] I would vote for the seccond, there is no sense in adding the type to the name of something in a typed language [13:23] fwereade, frankly, I don't care that much :) you can count me towards Channel if it helps [13:24] Channel it is then, thanks all [13:25] and: dimitern, frobware, perrito666: can I trouble any of you for reviews of http://reviews.vapour.ws/r/3143/ and http://reviews.vapour.ws/r/3146/ please? [13:26] if I can get LGTMs there I can apply the various fixes for the pipeline to http://reviews.vapour.ws/r/3147/ and land in one fell swoop [13:27] fwereade, I'll have a look shortly [13:27] 3143 has already been looked at and I *think* just needs an LGTM and/or second opinion on the open issue [13:27] 3146 is just a copy-code-and-fix-imports [13:27] dimitern, tyvm [13:48] frobware, opinion on thumper's comment at http://reviews.vapour.ws/r/3143/diff/3/?file=159341#file159341line96 ? [13:48] frobware, am inclined to drop the issue but if anyone can tell me *why* it's bad I will change it [13:50] fwereade, looking [13:54] fwereade, plan.Init is actively populated with workers at this stage? [13:55] frobware, yeah [13:55] frobware, ha! some asshole could change Init once the func's returned [13:55] frobware, thanks [14:02] dimitern, frobware, voidspace: https://plus.google.com/hangouts/_/canonical.com/maas-juju-net [14:02] dooferlad, omw 2m [14:09] Bug #1516975 opened: Incompatible version formats [14:46] dooferlad: sooo... [14:46] voidspace: yep? [14:46] dooferlad: I'll be changing my implementation to call the /node/{id}/ endpoint which now includes subnet information in the output [14:47] dooferlad: which the test server may or may not already support - probably not [14:47] no, not yet [14:47] dooferlad: so if you could include that in your test server work that would be wonderful :-) [14:47] voidspace: sure [14:47] thanks [14:56] rofl.... the output for juju run is sorted by machineID... alphabetically... i.e. 1, 10, 11, 12, 2, 3, 4 [15:07] katco, not merging lxd in master? I'm excited for it to happen though [15:15] Bug #1517076 opened: juju run output is sorted by machine id alphabetically not numerically [15:16] voidspace: this is more involved than just subnets because they depend on fabrics and VLANs. Fun :-| [15:16] dooferlad: we're not initially going to be using fabrics and vlans though [15:16] dooferlad: so even if they *should* appear in json output you can restrict the test server to just stuff that we're using [15:17] when we need fabrics and vlans we can add them [15:17] voidspace: hmm, OK [15:17] test server support doesn't need to be complete [15:17] voidspace: I will just dump out some sensible defaults [15:17] cool [15:17] it doesn't look like I'll need to query a subnet's ip addresses by the way [15:17] if you haven't done that yet [15:17] if you have there's no harm in keeping it [15:18] voidspace: Oh, I haven't got the basics down yet, but ignoring VLANs removes quite a chunk of work. [15:18] yay \o/ [15:23] fwereade, sorry, I just finished a call now and managed to push first of your reviews [15:23] dimitern, np, I've been tussling with worker/environ again [15:27] mattyw: we're merging lxd into master. waiting for bless from ci [15:27] dimitern, and, thanks for the detailed review: I'm not sure I'll be making any code changes there, but there's no harm in making the comments less stupid [15:28] fwereade, and the other one is done - I hope the reviews are useful :) [15:28] fwereade, yep +1 [15:29] katco, hey, any idea about that blocker bug? didn't we decide to rollback issues like that.. [15:31] dimitern: we're going to try and get a fix in before EOD today, if not we'll revert [15:35] katco, much excite ;) [15:35] mattyw: :) [15:38] katco, ok, thumbs up then :) [15:45] Bug #1517092 opened: [xenial] libgo panic doing a bootstrap on ARM64

[15:48] Bug #1517092 changed: [xenial] libgo panic doing a bootstrap on ARM64

[15:54] Bug #1517092 opened: [xenial] libgo panic doing a bootstrap on ARM64

[16:33] dimitern: frobware: dooferlad: openstack hangout? [16:34] voidspace: we are skipping today I think [16:34] dooferlad: no0pe [16:34] *nope [16:34] dooferlad: we skipped the second one last week [16:35] voidspace: and at this mornings hangout we discussed canceling this one [16:35] voidspace: which I realise you weren't at [16:36] voidspace: so, we should have mentioned that! I guess it should have been canceled in our calendars as well. frobware, dimitern? [16:36] dooferlad: dimitern and the open stack guys are here [16:36] but it's a low key discussion [16:37] dooferlad: probably no harm in you skipping [16:45] if we juju add-machine and then juju deploy, shouldn't the deployed unit get placed on the existing clean machine? [16:45] i.e. instead of having deploy add another machine [16:45] fwereade: ^ [16:50] voidspace: https://code.launchpad.net/~dooferlad/gomaasapi/subnets [16:51] voidspace: I can remove most of the VLAN code (untested) and propose a merge now, or you can just use what is there until I have finished up vlans, spaces and subnets. Shouldn't take long now. [16:52] dooferlad: I can start using what you've got [16:53] dooferlad: I'm "retooling" to use the node api instead of subnets plus addresses for when we're filtering by node id [16:53] dooferlad: that's almost done and I have the start of maasEnviron.Spaces() too [16:53] dooferlad: so plenty to be getting on with, but I'll need this code "real-soon-now" [16:53] dooferlad: if I need it before you're done I'll use your branch until it lands [16:55] voidspace: sounds good. So far I just have basic get/post/put/delete support for subnets. I need to integrate subnets into nodes, add op=?? (if you need that) etc. [16:55] I don't need ip_addresses now [16:56] I still need unreserved_ranges [17:11] sinzui: for the 1.26-alpha2 release notes, should they just be appended to the 1.26.0 release notes doc? [17:11] cherylj: yes please [17:11] sinzui: thanks! [17:12] natefinch, assuming constraints match hardware characteristics, yes [17:13] fwereade: was not seeing that happen last night on 1.25 [17:14] natefinch, I don't think I have the mental bandwidth to do more than say "that sounds like a bug then" [17:15] fwereade: no problem, just wanted to confirm that it really should be putting the unit on the existing machine, assuming the machine would otherwise be ok. [17:15] (clean, matches constraints, etc) === akhavr1 is now known as akhavr [18:09] cherylj: we're back to power8 [18:09] it's no longer a proxy issue [18:09] we're getting a machine-0 log in the next 30 mins === mbarnett` is now known as mbarnett [19:37] natefinch: how's bug fix coming? [19:39] katco: having some trouble reproing... running CI test s manually now to see if I can see it happen [19:39] natefinch: k, lmk what happens. maybe menn0 did fix it :) [19:40] that would be cool :) [20:06] mgz: you around? [20:06] or abentley? [20:07] natefinch: Hi. [20:07] I'm looking at the jes CI failure on master... but when I try to run it I get provider validation failed: invalid EC2 provider config: environment has no access-key or secret-key [20:08] this is when it's trying to do create-environment on top of the controller environment [20:08] ...obviously the controller environment was created fine with an access key and secret key [20:12] natefinch: I could be wrong, but I don't think the controller environment provider config is used to create the hosted environments. [20:13] thumper, menn0: is there a trick to getting the jes CI test to run? It seems to be failing when it tries to create-environment on top of the controller environment, saying it can't find EC2 keys [20:13] natefinch: you need to give it config for the environment, which includes cloud credentials [20:14] natefinch: Which jes test do you mean? [20:15] thumper: I did... it bootstraps the controller environment just fine, then fails to create the child environment (or whatever we're calling them) [20:15] abentley: assess_jes_deploy [20:16] did you pass the config through to the create environment call? [20:17] thumper: I don't know? I gave the script the CLI args it asked for, and it didn't say I was missing any [20:17] hosted environments are what we are calling them [20:18] natefinch: Does environments.yaml contain access-key and secret-key for the "env" you specified? [20:20] abentley: yes. Like I said, it was able to bootstrap the controller environment, so obviously those exist and are correct (also I can bootstrap that environment manually). [20:21] natefinch: assess_jes_deploy worked for me but I had to hack it a little to do an --upload-tools so it would use juju from my machine [20:21] * menn0 checks what he did [20:21] natefinch: I'm not sure how the ci script creates the hosted environment [20:21] natefinch: Not obviously. Perhaps bootstrap respects environment variables and create-environment does not, for example. [20:23] abentley, natefinch: I don't have any EC2 related env vars set [20:23] abentley, natefinch: here's what I did: ./assess_jes_deploy.py menn0-ec2 ~/tmp/workspace/bin/juju ~/tmp/workspace/logs menn0-testing [20:24] menn0-ec2 is an EC2 env defined in my environments.yaml [20:24] menn0-testing is the name of an env that the test will create, cloning the settings from menn0-ec2 [20:25] ~/tmp/workspace/bin/juju is the path to the juju binary (jujud etc are next to it in the same directory) [20:25] menn0: that's what I did, too [20:25] ~/tmp/workspace/logs is where I asked for the artifacts to go [20:27] natefinch: I also modified assess_jes_deploy.py so that the uploadTools arg to boot_context() is True instead of False [20:27] but I don't think that will cause the problem you're seeing [20:27] natefinch: maybe check your env for EC2 variables? Maybe there's some set that are causing a problem. [20:29] natefinch: try "set | grep -e AWS -e EC2" [20:29] hmm, so I do have the AWS keys in my environment [20:29] maybe that's the problem [20:29] natefinch: yep, try removing them [20:32] or rather, lack of them in the environments.yaml [20:41] natefinch: that makes even more sense [20:42] I imagine the AWS keys will need to be in environments.yaml in order for create-environment to work [20:46] gah, now it's giving me "failed to create new environment: admin-secret should never be written to the state" sigh [21:02] natefinch: I bet the config from environments.yaml is just being copied, try not explicitly setting admin-secret in the environments.yaml [21:02] actually, create-environment should read the environment vars [21:04] thumper: maybe the script is stripping out the env vars? [21:04] * natefinch shrugs [21:04] I can try doing it manually when the script is finished (this is a long-ass test) [21:04] * thumper shrugs [21:06] ahh, wondrous exceptions [21:06] OSError: [Errno 17] File exists: '../logs/machine-0' [21:07] tautological errors, next one should be file is file [21:14] lol, I think the CI test failed while trying to dump logs [21:14] thanks, exceptions! [21:14] quiet expectations [21:14] os.mkdir(machine_dir) [21:17] so after 15 minutes of running the test... it fails because the directory it was trying to create... existed. Sigh [21:18] natefinch: I wonder if mgz's changes to grab logs for each environment is trying to write all the logs into the same place [21:18] hmm could be, though it was trying to create machine-0 ... do we have a machine-0 in the host environment? [21:22] yes, that is a very annoying characteristic of CI tests, they assume a pristine env [21:27] menn0: I did not commit my change yet [21:28] CI tends to just be unforgiving about expected start state [21:28] mgz: ah right [21:28] so it's just that natefinch needs to clear the artifacts directory. there's probably stuff from a previous attempt hanging around. [21:30] I need to restart for hangouts, brb [21:35] thanks, retrying with empty logs dir [21:52] um hmm.. weird... cannot run instances: Your quota allows for 0 more running instance(s). You requested at least 1 (InstanceLimitExceeded) [21:53] menn0: do we still need to use st.docID(localid) when creating a doc to insert into a collection, or is the prefixing of the doc id with env-uuid automatic now? [21:53] everything in state still seems to use st.docID() [21:54] wallyworld: it's not necessary any more [21:54] awedome, ty [21:54] wallyworld: we can remove all the calls [21:54] wallyworld: I've got a tech debt card for that [21:54] yay, ok [21:54] katco: I'm not going to be able to figure out this bug before dinner. I have time after dinner, but I've just been strugging with CI too much [21:54] but you know how it is [21:55] natefinch: ok, i'll discuss in the release meeting. can you log where you're at somewhere [21:55] natefinch: that i can look at? [21:56] axw: wallyworld: does the new azure provider use simplestreams? [21:56] katco: no [21:56] it queries the cloud directly [21:56] alexisb: ^^ [21:56] to look up images ids [21:56] wallyworld: ty [21:56] katco: i already told alexisb taht [21:59] katco: not a lot to say. Couldn't repro manually, CI hasn't gotten through the test yet (each attempt is like 20 minutes) [21:59] katco: updated the bug: https://bugs.launchpad.net/juju-core/+bug/1516144 [21:59] Bug #1516144: Cannot deploy charms in jes envs

=== natefinch is now known as natefinch-afk [21:59] natefinch: ok [22:20] wwitzel3: ericsnow_: any updates on your end? [22:21] thumper: review pls: http://reviews.vapour.ws/r/3169/ [22:21] * thumper looks [22:22] katco: patch should be done in the next little while (based on Nate's patch + tests) [22:22] ericsnow_: cool. demonstrably addresses memory issue? [22:22] katco: yes [22:26] super, a bit of rain and power starts doing funny stuff [22:28] thumper: thanks [22:47] katco: cherylj : Do you have time to review http://reviews.vapour.ws/r/3170/ [22:56] katco: not yet, I'll be working on it tonight to try and at least get past this stupid series issue. [22:56] wwitzel3: ok [22:56] katco: my plan was to bug someone who is starting their day to take a look at ti with me [22:56] wwitzel3: not a bad idea. alexis is also going to see if jam or fwereade have any insights [22:57] katco: yeah, that'd be great. I was going to bug people after Jessa and I have dinner. [22:58] wwitzel3: k sounds good. say hi to natefinch-afk; he'll be working on trying to repo the master blocker tonight :p [22:58] repro rather [23:01] :) [23:07] thumper, wallyworld, ^^^ if someone is around when wwitzel3 is back on it would be nice to have a AUS/NZ timezone person to bounce ideas off of [23:07] sure === ericsnow_ is now known as ericsnow