[00:42] Bug #1613487 opened: Fails to build tools when no matching tools found.

[00:57] Bug #1613491 opened: juju service names limited to 66 characters [01:05] wallyworld: i've spent a bit of time looking into this environs/sync issue and nothing jumps out [01:05] wallyworld: I don't see any cases of a gc.C being stored inappropriately or even goroutines being used [01:05] yeah, i had a brief look earlier and am in the same boat [01:06] we really need there to be more info logged [01:07] wallyworld: ha! I had it running with dave's stress script while I was having lunch and managed to have it fail like that once out of ~100 times [01:07] wallyworld: so it is somewhat reproducible [01:08] and yet CI fails more regularly [01:08] hard to know where to target though with no errors [01:09] wallyworld: i'm going to try running it again with -check.vv [01:11] wallyworld: ok it failed quite quickly that time [01:11] just checking the output [01:12] wallyworld: got it [01:12] * menn0 pastebins [01:14] wallyworld: here's the 3 failures I just got: http://paste.ubuntu.com/23060118/ [01:15] menn0: intersting, wtf isn't that deterministic [01:15] i can look into it [01:17] wallyworld: I see it [01:18] wallyworld: GetMockBundleTools is patched in during SetupTest and it keeps a reference to the gc.C that was passed to SetUpTest [01:18] wallyworld: when it fails during the test that gc.C is no longer the current one [01:19] (i.e. the one that gets passed to the test) [01:19] that's what causes these tests that both pass and fail [01:22] wallyworld: moving the GetMockBundleTools patching into the tests themselves makes them fail consistently (due to typed nil) [01:23] wallyworld: if you don't mind, PTAL at http://reviews.vapour.ws/r/5441/ [01:25] menn0: ah, i hate that multiple gc.C thing [01:25] menn0: i have a branch i need to start, i can fix in that [01:26] wallyworld: I'll push what i've done so far [01:26] menn0: ok, np. i haven't had any time so far today to dive in, keep getting pulled left and right [01:27] wallyworld: understood. this one got my attention because a) it's odd and 2) it's bitten me several times in the last day [01:27] yeah, it will be fixed... today! [01:29] axw: lgtm. we'll need to address the @local suffic at some point because I think it was going to be removed [01:29] wallyworld: okey dokey. I'd love it if we just had one canonical form always [01:30] agreed, i think that's where the push is coming from, need to touch base on that again [01:31] wallyworld: pushing now. all the tests pass again except one and I'm not sure if the test is wrong or the implementation. [01:31] wallyworld: the tests were only passing by luck before it seems. [01:31] (due to the typed nil comparison [01:31] menn0: ok, ta, will merge into mine [01:32] wallyworld: https://github.com/mjs/juju/tree/environ-sync-tests [01:32] ty === menn0 is now known as menn0-afk === natefinch-afk is now known as natefinch [02:13] could really use a review: http://reviews.vapour.ws/r/5440/ axw or anyone else [02:18] wallyworld: FYI, possibly not going to have these changes done today. I'm going to have to change a bunch of code in state, API, etc. to use credential tags instead of names [02:18] not surprised [02:18] so probably won't get to LXD till tomorrow [02:20] wallyworld: the auto-upload-tools thing still seems dangerous to me. What if you have 2.0 deployed, have a 2.0.1.1 client/server built locally, but 2.2 is available in streams. What does juju upgrade-juju do? [02:21] upgrade to 2.2 [02:21] well, actually,i need to check the urgrade rules [02:22] if you have a 2.0.1.1, then you have built it and so are expected to know what you are doing [02:22] wallyworld: expected to know what you're doing is a lot different than understand exactly what fine details of the code are going to do [02:22] i can't recall off hand how we select the allowed upgarde version when searching [02:22] wallyworld: it seems like any time both streams and your local are valid upgrade paths, there' [02:22] upgrades have always been like that [02:23] there's no way to know which one the user intends to use [02:23] it searches for packaged tools first [02:23] if it finds them, it will use them in preference [02:23] ok [02:23] so whatever upgrade rules we have for selecting will be used [02:24] i think we look for major.minor [02:24] it only uploads automatically if no packaged tools are there, and only if what you have is different and only if jujud matches the client being used [02:25] I guess my point is - I want to make sure we never automatically upgrade someone from a released version to a development version with some explicit flag or confirmation [02:26] s/with/without/ [02:31] right now, there's not confirmation - it will use a jujud mactching juju [02:31] that seems undesirable [02:32] so if juju is 2.1-beta1 it will upgrade a 2.0 (if the upgrade rules select such a version) [02:32] i'l need to check that [02:39] I just don't want someone to get screwed who happens to build juju from source to test out the latest and greatest... forgets for a week, then upgrades their production server and boom, now they're on devel [02:40] Bug #1611097 changed: cannot add/destroy a model if done too fast

[02:46] menn0-afk: when you are back, here's a PR to fix that test and a CI issue http://reviews.vapour.ws/r/5442/ === menn0-afk is now known as menn0 [02:47] wallyworld: I am back.... forgot to update nick [02:47] looking [02:48] menn0: ah, i see a mistake, a wrong test suite name, fixing [02:49] hmm, scrap that, it can stay as is [02:54] axw: wallyworld: bug 1582021 was addressed, right? [02:54] Bug #1582021: Juju loses track of current controller when destroying model <2.0>

[02:54] think so? [02:55] don't recall exactly [02:55] i'll mark as fixed committed \o/ would have been sooo very lovely to have a functional test that would just tell us :) mayb even have a link to a passing ci test in the bug :) [02:56] veebers: ^^ [03:04] wallyworld: done! [03:04] ta [03:08] wallyworld, thumper: do you think I moved the notification about an available upgrade into the new NOTES status column too? [03:08] axw: why do we store the charm url on the storage instance when you can get it from the owner? [03:09] menn0: +1 [03:09] thumper: that's what I was thinking too [03:09] menn0: showing the most relevant info [03:09] migration > upgrade available > nothing [03:09] menn0: yeah, good idea [03:10] thumper: it's not necessarily the same. it's the charm URL that was in use when the storage instance was created, not the current one [03:10] thumper: yep that's exactly what I was thinking [03:10] axw: but why do we care? [03:10] thumper: I forget why we have it at all. we don't use it AFAIK [03:10] wallyworld: do you recall? [03:10] um [03:11] we could just drop it, yeah? [03:12] thumper: if we haven't needed it thus far, probably safe to drop it [03:12] thumper: axw: i can't even see where it's used [03:12] axw: you ok if I do that as part of the migration branch I'm doing? [03:12] thumper: SGTM [03:12] ok, cool [03:13] thanks [03:13] * thumper is looking how to add storage to the model to test exports [03:36] menn0: thumper: it seems to me that in the multi-model world, stuff is not getting removed when a model is destroyed. it never used to matter in the single model case. but now for example, i can see where we are creating model settings and constraints in modelSetupOps but i can't see where this data is ever removed via calls to removeConstraintsOps etc. I can see where constraints and what not are removed for applications, units, [03:36] machines, but not models. am i on crack? [03:37] wallyworld: I didn't write this stuff but RemoveAllModelDocs catches everything [03:37] wallyworld: very brute force [03:37] * wallyworld looks at that method [03:37] wallyworld: not sure when/how it gets called when models are destroyed normally [03:38] wallyworld: until yesterday it was missing user perms but I fixed that [03:39] wallyworld: looks like the undertaker worker ends up calling it, so everything should be getting removed [03:39] menn0: awesome thanks [03:42] * natefinch tries for like the millionth time to use custom streams with the CI tests [03:49] Bug #1474291 changed: juju called unexpected config-change hooks after read tcp 127.0.0.1:37017: i/o timeout

[03:49] Bug #1613516 opened: AWS instance types should be detected

[03:50] it's really a shame we can't just upload tools to the controller and tell it what series they're for. [03:50] OMG, state.AddApplication is 230 lines... [03:52] at my first job out of college, the main entry function was 1000 lines long [03:52] over 100 lines of that is arg validation [03:52] natefinch: I worked at a bank where the CreateInstrument function was 10000 lines [03:52] OMG [03:52] it was horrible [03:53] and noone wanted to fix it [03:53] that's beyond horrible [03:53] because "what if it broke" [03:53] yes.. what IF it broke? :/ [03:53] :) [03:53] no unit tests, just overnight regression tests [03:54] my last job, we barely had tests. I'm sure it was like 5% coverage.... and many developers openly criticized the idea of writing tests. [03:56] it's a damn miracle anything worked. We did have like 8 full time QA people doing manual tests, I guess. [03:56] wallyworld: so can you show me your clouds.yaml? [03:56] only if you show me yours :-D [03:56] i'll pastrebin [03:57] menn0: http://paste.ubuntu.com/23060299/ [03:57] axw: able to save me some time and point me at the simplest test code that adds a storage instance to the db? [03:58] wallyworld: I see the problem. I had my config at the wrong level... no "config" section. [03:58] axw: factory.MakeApplication? [03:58] wallyworld: I wonder if juju add-cloud should complain if you mess this up? [03:58] (like I did) [03:58] probably [03:58] wallyworld, axw: re: a test for 1582021 as mentioned by anastasiamac, does this sound reasonable for a testplan? test would bootstrap, add a new model, switch to it, (ensure status works) destroy that model then ensure that it could both "list-models" and "status" successfully? [04:00] veebers: if you just destroy a model and don't switch to another one, you will get an error [04:00] it will be interesting to repro the race condiiton [04:05] wallyworld: is that the error that is shown in the bug? Perhaps I'm not sure what the test behaviour should be [04:07] it depends if the bug has been fixed. the bug refers to a not found which is the original symptom. if the current model is cleared out, then there will be a different error refering to no current model selected [04:08] wallyworld: veebers is writting the test to detemrine whether the bug has been fixed. he just needs to confirm if his scenario will veriffy the fix [04:08] right, understood. as i said, there's 2 aspects. the resetting of current model and the race [04:09] the error wrt the resetting of current model will depend on if the bug has been fixed. and you need to switch models if you want to avoid an error of either kind [04:10] if the bug has been fixed, you get the not found, if not you'll get the no current model as stated above [04:10] assuming you don't switch models [04:11] wallyworld: the bug specifically deals with no-explicit-switch situation.. it's covered in bug description [04:11] wallyworld: mark is not happy with "not found"... did u read the bug? :D [04:11] right, but the test also needs to cover the switch case [04:11] yes [04:11] wallyworld: Ok, so I want to confirm the mention bug is fixed, so I destroy and don't switch, doing a list-models or status should give me . . .something? better asked what should the expected result be [04:11] and as i said, if the bug has been fixed (ie current model reset), you will not get the not found [04:12] but you will get a different error [04:12] until you switch to a different model [04:12] i thought i explain it! you will get a different error [04:13] not a not found error but an error saying there is not current model [04:13] and if you switch everything will be ok again [04:14] ok cool, I'll run with that. Cheers [04:14] if you get stuck, let me know; sorry if i was unclear [04:15] thumper: sorry, was having lunch. if you just add a unit whose charm has non-optional storage, it'll add a storage instance automatically [04:16] axw: is there an example of that anywhere? [04:16] do we have a testing charm that has non-optional storage? [04:17] thumper: state/storage_test.go, StorageStateSuiteBase.setupSingleStorage (actually that one's non-optional, but that code sets up the storage constraints) [04:17] thumper: err sorry, that one's *optional* [04:18] that'll do... [04:18] I'll crib from that [04:24] wallyworld: fyi I get the error message "..no model in focus" which seems like the bugfix is in place for that part at least [04:24] veebers: awesome, that means that it is fixed, yeah [04:24] so now the test should switch [04:24] and confirm that everything is ok [04:24] sweet [04:24] i would have thought list-models could still work though [04:24] even with no model in focus [04:25] if it doesn't that's a bug IMO [04:25] how do you know what to switch to [04:25] if you can't list models [04:27] veebers: would be awesome if commented on the bug with your findings. we can change status of the bug based on ur test \o/ [04:29] wallyworld: 'models' and 'list-models' seem to work, although switching to a model doesn't seem to work, although I could be doing something off in my test code. I'll re-pro manually [04:29] list-models works even without a current model? cool [04:29] i think we set current model with a * [04:30] so list-models should not have the * without a current model selected [04:38] wallyworld: I'm trying to use my own streams to run a test. if I bootstrap with --metadata-source ... where is that stored? I don't see that config val in get-model-config or get-controller config [04:38] it's not stored [04:39] what streams? tools or images? [04:39] wallyworld: tools [04:39] tools are stored in blobstore [04:40] metadata-source is only used to tell bootstrap where to find tools to upload and store [04:41] wallyworld: I'm trying to test something on windows, so I need a secondary machine to be able to get the windows binary that I have built. how do I do that? [04:42] it's not trivial [04:42] one way is to use agent-metadata-url pointing to an apache http server [04:42] well, so I have tools & streams generated and sitting in S3 [04:43] if it's a public bucket you could use that url as agent-metadata-url [04:43] wallyworld: ok, I thought metadata-source would have done that, but I can set metadata-url, that's fine [04:43] er agent-metadata-url [04:44] metadataa-source -s just fpr bootstrap [04:44] to tell it where to find a local (as in dir) source of tools/image metadata [04:44] generated using the plugins [04:44] which is then sent to controller suring bootstrap and stored in state [04:45] ahh... then maybe it should complain if I give it a URL :/ [04:45] file:// is a url [04:45] which it uses [04:45] i guess it could complain if not file:// [04:45] or a dir [04:45] but --metadata-source https://s3.amazonaws.com/24b99da448314945835933a0c8edffb3 just fails silently [04:45] not fails... is ignored [04:45] yeah, there's not been much polish on dev only tools [04:46] i didn't think it failed, that must be a new bug [04:47] bootstrap doesn't fail, which is kind of the problem [04:48] thumper or wallyworld or anastasiamac: MM status in tabular output http://reviews.vapour.ws/r/5444/ [04:49] ok [04:55] menn0: i asked for one more test [05:14] wallyworld: I thought of that one but was being lazy. done now :)_ [05:14] :-) [05:15] we've all been there [05:15] you have a shipit anyway [05:19] Bug #1613537 opened: metadata-source doesn't sanity check the value you give it [05:44] axw: if you had a moment at any time, this is a mechanical change to the model defaults api to accommodate cloud/region. not used yet but will be when reed's stuff gets further along http://reviews.vapour.ws/r/5445/ [05:46] wallyworld: ok, in a little while [05:46] sure, no hurry [05:48] wallyworld: one fundamental thing: we have cloud tags now, should be using that instead of just a cloud name [05:48] ah, yeah [05:49] doh [05:49] fixing [05:58] axw: updated for when you get to look later [06:09] thans axw [06:09] np [06:19] Bug #1551959 changed: juju-tools 1.25.0 with juju-agent 1.25.3 has networking issues [06:22] wallyworld: axw: just to confirm - i believe this bug has been fixed with new model naming convention that includes user name, non? :D bug 1576359 [06:22] Bug #1576359: Cannot talk to a new model with a reused name

[06:23] not sure off hand [06:23] i wasn't familar with that bug [06:23] axw: ? :D [06:23] anastasiamac: not sure that that one is fixed [06:23] anastasiamac: I don't think I did anything specifically that would have fixed that [06:24] axw: wallyworld: k. thnx [06:31] wallyworld: https://github.com/juju/names/pull/72 - sorry, third time's the charm perhaps? [06:31] Bug #1100560 changed: Support method of removing a subordinate charm

[06:31] Bug #1432759 changed: Transient error on status while running deployments via quickstart

[06:31] axw: just gotta race out to school pickup, will look when i get back [06:31] np [06:31] am running really late === rogpeppe is now known as rogpeppe1 [06:52] anastasiamac: Hey there. Just to be clear, bug 1613491 is not critical for me - we're working around it easily enough. [06:52] Bug #1613491: juju service names limited to 66 characters

[06:53] miken: awesome \o/ thank you - i've updated the bug [07:01] Bug # changed: 862422, 1228355, 1301255, 1340756, 1468223 === frankban|afk is now known as frankban [07:28] Bug # changed: 1217814, 1283866, 1398589, 1401358, 1403460, 1417149, 1420996 [07:29] wallyworld rogpeppe: this PR should address concerns about the cloud API, and also updates other bits to use the new cloud credential tag: http://reviews.vapour.ws/r/5447/ [07:29] axw: i'll swap ya :) http://reviews.vapour.ws/r/5439/ [07:29] rogpeppe: sure, looking [07:31] rogpeppe: hmm, I think it was intentional to leave them there, so old clients can get a useful error message [07:31] axw: won't they still get the same error message?> [07:32] axw: i think i tried to leave the behaviour unchanged. perhaps i didn't though... [07:32] rogpeppe: maybe? I just saw a test that removed a check for the error message, so assumed it didn't any more [07:32] * axw looks deeper [07:32] axw: ah, yes, maybe i removed that test when i shouldn't have [07:32] the idea is that older 1.25 clients should get a nice message [07:33] not sure of the details though [07:33] wallyworld: yes, they still should, but yeah, i removed some of the infrastructure that supported the test [07:33] if it works with the proposed changes, then that's ok [07:34] +1 to deleting code if it makes sense :-) [07:35] wallyworld: i think we could keep the test but make it manually call Login with version 2; no need for OpenWithVersion, I think. [07:35] sgtm, keep as little as possible [07:40] Bug #1494936 changed: imageSuite.TestDownloadEnvironmentPath

[07:49] Bug #1494936 opened: imageSuite.TestDownloadEnvironmentPath

[07:58] Bug #1494936 changed: imageSuite.TestDownloadEnvironmentPath

[08:09] axw: reviewed [08:13] rogpeppe: ta [08:26] rogpeppe: I'd prefer to leave mention of user/cred syntax in the help docs until we have a means to share [08:27] axw: "a means to share" ? [08:27] rogpeppe: share credentials - ACLs [08:28] rogpeppe: i.e. you may be able to use the syntax user/cred, but it won't work unless user == you [08:28] axw: the problem is that historically that means we'll forget to add it to the help docs [08:28] rogpeppe: I'd rather that than confuse people by having them think they can do something they can't [08:28] axw: i'd add it to the help docs and add a caveat that currently the user must be the same as the model user [08:29] axw: or, tbh, since it'll just fail, that's probably fine [08:29] axw: it'll fail with "permission denied" presumably [08:29] axw: which is exactly how it would fail if the permissions hadn't been shared [08:29] rogpeppe: "not found" at the moment I think [08:30] axw: even if the other user's creds actually exist? [08:30] axw: oh yes, because that's what we always do for perm denied i guess [08:30] rogpeppe: yeah. generally we do that so you can't sniff out info [08:30] axw: so, i think it's fine to document the syntacx [08:31] axw: then when we implement sharing, people will find a use for it [08:32] rogpeppe: ok [08:32] axw: also, describing it in the help docs is often a good indicator as to whether it's a decent feature or not [08:32] axw: i.e. can it be described nicely [08:32] ? [08:54] axw: tell me about CreateLocalLoginMacaroon [08:56] rogpeppe: it's how you get a macaroon that you can log into a controller as a local user [08:56] rogpeppe: what do you want to know about it in particular...? [08:57] axw: i'm just wondering if it should be part of the controller-only API or not [08:57] rogpeppe: yes, it is a controller level thing [08:58] axw: i'm getting a test failure where login fails [08:58] axw: because it's using the model-level connection (that it's using to log in) but then uses that to create a local login macaroon [08:59] axw: and that fails because UserManager is controller-only [08:59] rogpeppe: what's the test? [08:59] axw: featuretests.cmdLoginSuite.TestLoginCommand [08:59] axw: it seems like a legit failure to me [09:00] rogpeppe: ah, it's actually being used in change-user-password [09:00] which should be controller-only.. [09:01] axw: it is [09:01] axw: changePasswordCommand embeds ControllerCommandBase [09:01] axw: hmm, so is login actually [09:02] axw: so now i'm confused :) [09:02] rogpeppe: that should mean they're getting controller API connections [09:02] axw: yup [09:02] axw: so it should work, but i'm getting [09:02] [LOG] 0:00.242 ERROR cmd failed to create a temporary credential: permission denied (unauthorized access) [09:04] rogpeppe: sorry, nothing's jumping out from the code... login is passing "" for the model name to get the API connection [09:05] axw: yeah, there's something odd going on. thanks. [09:06] axw: now i just have to work out how to get the featuretests to show trace-level debug logs of the api server [09:07] rogpeppe: loggo.GetLogger("juju.apiserver").SetLogLevel(loggo.TRACE) [09:07] I think [09:07] axw: i'll try that [09:11] axw: still not seeing any api server traffic (i also tried GetLogger with "" and "") [09:12] axw: ISTR working out a way to do this before, but it's gone from my memory now unfortunately [09:20] rogpeppe: I've never managed to set the root logger level using GetLogger("") - try using ConfigureLoggers instead. [09:20] babbageclunk: ha, thanks [09:21] babbageclunk: the loggo code thinks that it should be "" [09:22] rogpeppe: ah, that might be what I got wrong [09:22] babbageclunk: but i suspect that all the logging is being lost [09:22] babbageclunk: GetLogger should really have a comment [09:22]