[00:00] have just a couple of steps remaining [00:01] alexisb: lol [00:04] menn0: sorry, just got out of meetings, can talk now [00:05] wallyworld: thanks. which hangout? [00:06] https://plus.google.com/hangouts/_/canonical.com/tanzanite-stand [00:18] oh the cannot find encoding bug is fixed in trusty-proposed now [00:24] Bug #1496639 opened: juju get incorrectly reports boolean default values

[01:20] * thumper wanders off to make a coffee [01:24] Bug #1496652 opened: Relation settings watcher exposes txn-revno to uniter [01:33] Bug #1496652 changed: Relation settings watcher exposes txn-revno to uniter [01:39] Bug #1496652 opened: Relation settings watcher exposes txn-revno to uniter [01:52] menn0: here is the master branch for the uniter upgrade step work http://reviews.vapour.ws/r/2682/diff/# [01:53] menn0: just a forward port of the 1.25 with the new step added [01:54] wallyworld: FYI I'm doing the tweak to unblock master, just being held up writing a test [01:54] ty, saw the bug comment [01:54] man I love my coffee machine [01:54] thumper: looking [02:03] davecheney: re bug 1465317, what's the current status of the fixes you have been doing? [02:03] Bug #1465317: Wily osx win: panic: osVersion reported an error: Could not determine series

[02:04] davecheney: should we assign this bug to you? [02:04] for the work you have been doing? [02:08] thumper: ship it [02:08] menn0: cheers [02:12] wallyworld: http://reviews.vapour.ws/r/2683/ [02:13] looking [02:13] thumper, wallyworld: i'm beginning to think this idea of Juju using MAAS' cached images is a non-starter [02:13] because? [02:13] thumper, wallyworld: the images don't appear to be in the same format [02:14] not surprised [02:14] i was sort of afaid of that [02:14] the MAAS images are a tarball of a root filesystem [02:14] menn0: document your findings on the bug, and we'll look for another solution [02:15] the images served by the public server are a tarball with the kernal separate and a .img file which contains the root filesystem [02:15] thumper: ok [02:15] thumper: should I remove it from the 1.25-beta2 milestone? [02:16] axw: ship it, thanks for temp fix [02:16] um... yeah [02:16] wallyworld: ta [02:17] thumper: menn0: this maas image thing is a direct request from dan w so i suspect we'll be asked to make it work somehow [02:17] by we, i mean us and maas [02:18] wallyworld: perhaps mass need to cache the initial format, then unpack for themselves [02:18] * thumper hopes it is more maas [02:18] yes [02:18] * wallyworld does too [02:18] I think there was a mailing thread list about this stuff a while ago... [02:18] rings a bell [02:19] i'll have a look around [02:19] and i'll still send an email to relevant people about this [02:20] menn0: not sure how relevant, "Juju + MAAS + Image Downloads" from may 2014 [02:20] err sorry sept [02:21] can't read US dates :) [02:21] axw: i've seen that one [02:21] axw: that's the discussion around the ticket i'm looking at [02:21] ah ok [02:21] axw: no one seems to have checked if the image formats are actually the same :) [02:21] heh [02:22] maybe it's easy to convert between them [02:22] i'm just pulling down the maas code now to check a few things [02:22] then i'll send this email [02:42] axw: going afk for an hour or so, bbiab [02:43] wallyworld: no worries, ttyl [02:53] wallyworld, thumper: bradm mentioned this to me when I was chatting to him earlier: https://bugs.launchpad.net/juju-core/+bug/1496639 [02:53] Bug #1496639: juju get incorrectly reports boolean default values

[02:53] does that ring any bells? [03:07] menn0: nope [03:07] but interesting [03:13] thumper: attach it to 1.25-beta2? [03:13] menn0: sure, why not [03:15] hmm... [03:15] wrote a simple charm to test a problem [03:15] but the charm's default apt-get is failing [03:16] E: There are problems and -y was used without --force-yes [03:16] seems a bit weird [03:55] was my squid-deb-proxy being out of date somehow [03:58] miken: here I am then :) [03:58] trying to reproduce the failure [03:59] Yep, re-running my mojo spec with the same code revision that caused the config-changed failure, to see if I still see the issue too. [03:59] miken: this one https://bugs.launchpad.net/juju-core/+bug/1494542 [03:59] Bug #1494542: config-changed error does not cause error state

[03:59] miken: ah... are you able to run the environment with --debug? [03:59] miken: then let me have the logs? [04:00] I'm not able to reproduce with my trivial failing charm [04:01] possibly - I'll let the current run finish so I know whether I can reproduce it. If I can, is that an option I can add to the juju-deployer call? /me checks [04:01] no idea [04:01] miken: does the deployer bootstrap for you? [04:02] it is juju I want in debug mode, not really the deplower [04:02] yer [04:02] thumper: which problem is this? deployer/config-changed sounds like the thing eric and natefinch-afk fixed today?: [04:02] Ah - just to bootstrap. Sure. [04:02] miken: if you use --debug when bootstrapping, that flag is passed through to the remote server, and it logs everything at debug too [04:03] you can set it afterwards [04:03] but easiest if bootstrapping to just use --debug [04:03] Nice. I'll try setting it now too in case it does reproduce (takes 10-15mins to run deployment) [04:04] Hmm - how can I set it with the deploy already going? [04:05] juju set-env logging-config=juju=DEBUG [04:05] Great [04:05] rick_h_: https://bugs.launchpad.net/juju-core/+bug/1494542 [04:05] Bug #1494542: config-changed error does not cause error state

[04:05] miken: has rick_h_ hit it too? [04:06] miken: ah ok, that's different then [04:06] thumper: no, the mention of a config-changed issue made me think of https://bugs.launchpad.net/juju-core/+bug/1495681 from today [04:06] Bug #1495681: quickstart delployments broken in 1.24

[04:06] thumper: so just checking, but looks like that's different [04:06] ok [04:06] what causes quickstart to not work? [04:06] * rick_h_ goes back to hiding :) [04:07] thumper: something to do with nil values and the API that got cleaned up [04:07] rick_h_: WTH are you doing answering IRC now anyway? [04:07] rick_h_: you are setting a bad precident [04:07] thumper: meeting in an hour :) [04:07] ew [04:07] well that sucks [04:07] thumper: fortunately not too many of these [04:07] thumper: if only all of you would move to the US :P [04:09] TBH I don't think the US would want us all [04:09] stealing all those jobs from real americans [04:09] thumper: heh we just say that a lot, but we don't really care. [04:12] thumper: ok, running with the same spec and code rev did reproduce it. I can give you logs, but I only switched on --debug half way through (but before the config-change error, afaict). Is it just the /var/log/juju/unit-sca-app-0.log that you're after? [04:12] yes, and machine-0.log [04:12] the unit one so we can see what it thinks [04:13] and machine-0 to check any server side changes [04:13] see where the problem is, client or server [04:13] miken: can you attach the logs to the bug plz? [04:13] Ack [04:13] awesome [04:13] ta [04:19] thumper: are all those safe to attach? e.g. cloud info? /me hasn't looked lately. [04:19] hmm... [04:19] I *think* so [04:20] miken: might just give a quick grep for cloud creds or anything. I know there was work to not outut things but don't recall where it left off tbh [04:22] rick_h_: yeah, I was just checking through after noticing that they're root:root 600 in ~/.juju/local/logs [04:24] I can see lots of "Response": "'body redacted'" 's [04:27] miken: yeah, need to set the logging level to TRACE to get the data in there [04:28] * thumper is on kid duty, will check in later with the bug to look (maybe) [04:28] I have a feeling that all work later will be emails and comments on documents [04:28] thumper: YOu can grab the logs from: [04:28] thumper: chinstrap.canonical.com:/home/michaeln/bug-1494542-logs.tgz If you can see there's nothing sensitive there, feel free to attach. [04:28] * thumper has to go cook dinner before school production tonight [04:29] thumper: Nice - enjoy! [04:29] miken: ok, ta === urulama_ is now known as urulama [07:51] bad status test... embedding actual version numbers in expected output [07:51] * thumper settles down to write a few final emails with a glass of wine [07:51] hopefully the wine will help [07:55] fwereade: I would appreciate a review from you on http://reviews.vapour.ws/r/2685/ when you can. I'd particularly like validation of my assertion in settingsDocNeedsMigration... but really the whole thing is a *little* bit hairy [07:56] axw, cheers [08:40] Bug #1496750 opened: Failed worker can result in large number of goroutines and open socket connections and eventually gets picked on by the OOM killer [08:49] Bug #1496750 changed: Failed worker can result in large number of goroutines and open socket connections and eventually gets picked on by the OOM killer [09:01] jam: fwereade: stdup? [09:01] voidspace: I' believe its strdup :) [09:02] jam: :-p [09:04] Bug #1496750 opened: Failed worker can result in large number of goroutines and open socket connections and eventually gets picked on by the OOM killer [09:37] Bug #1495542 changed: 1.20.x cannot upgrade to 1.26-alpha1

[09:42] unit-err-0[1975]: 2015-09-17 09:39:11 DEBUG juju.worker.dependency engine.go:438 "metric-sender" manifold worker stopped: failed to open spool directory "/var/lib/juju/metricspool": stat /var/lib/juju/metricspool: no such file or directory [09:42] anybody getting this on master? [09:44] bogdanteleaga: windows? [09:44] ashipika: trusty [09:44] it appeared on windows as well fwiw, but it fails on trusty too [09:45] bogdanteleaga: that's a change that landed with the maltese falcon feature branch… [09:45] bogdanteleaga: which test did you run? [09:45] ashipika: I'm trying to deploy a charm [09:46] bogdanteleaga: will look into it! thanks for reporting it [09:46] bogdanteleaga: could you please file a bug on launchpad? [09:47] ashipika: I'll try later [09:48] bogdanteleaga: thank you.. [10:08] fwereade: voidspace: I was looking at https://github.com/juju/juju/wiki/mgo-txn-example the one thing that sticks out as odd is the "e, err = e.st.Environment()" to refresh itself [10:08] it feels really strange to override the 'self' pointer, as it doesn't change the caller, just the local context, right? [10:08] jam, rather than e.Refresh()? [10:09] IMO it would be clearer if we use a real local variable there [10:09] fwereade: well either Refresh or local var [10:09] jam, i.e. `env := e` outside? [10:09] fwereade: right [10:09] jam, there's a convention in state that methods not change unexpected fields, which is why no Refresh() [10:10] jam, I have no objection to local vars though [10:10] jam, would probably be clearer [10:10] +1 for not overriding the receiver... that's certain to cause confusion. [10:10] fwereade: I'm fine with the no Refresh, cause I understand not wanting to have to introspect your object to see if there were any side effects. [10:10] jam, then +1 to fixing it to use local vars [10:12] anastasiamac: when you're working... please see https://github.com/axw/juju/commit/3168dd66426d018a00ff32c61b91c9141351e8b5 [10:24] fwereade: jam: although if you *know* your data is out of date, deliberately *not* refreshing struck me as a little odd [10:24] fwereade: jam: you're choosing to deliberately leave stale data for other "observers" [10:24] but I haven't gone back and read the doc yet [10:25] voidspace, all data is always stale anyway [10:25] I guess the risk is that you leave other observers with inconsistent data, and stale data is better than inconsistent data [10:25] fwereade: possibly stale versus we actually know this is wrong [10:27] voidspace, it's not so much about other observers -- individual entity types are not generally goroutine-safe -- as it is about ensuring every individual client sees a consistent (if not correct) picture of the entity; part of that is putting the responsibility for Refresh in their hands, and not pre-empting it [10:27] fwereade: right [10:27] inconsistent data could be arbitrarily bad [10:28] fair enough, although then it would seem that Refresh is almost never safe [10:28] voidspace, *that said* the value of that approach is certainly diminished -- it came from a time when everything had a direct db connection, and there were many more long-lived *state.Things floating around in the system [10:29] voidspace, almost all our state types are now very short-lived -- one API call and that's it [10:29] fwereade: I've addressed all the issues on my PR except the txn assert and the tests for that [10:29] voidspace, cool, thanks [10:29] fwereade: will ping you when it's done as it's a different beast from when it was last reviewed [10:32] voidspace, (fwiw, it's a slightly subtle notion of inconsistency -- a refresh *will* give you new, consistent data, but it potentially invalidates assumption you checked in the scope you're looking at it from. so we try to keep showing you the same picture; even if it *is* a lie, the cause of any failures will still be reported, and it's up to the client to Refresh and reconsider or just give up) [10:39] fwereade, ping? [10:40] mattyw, pong [10:45] bogdanteleaga, are you blocked on that metric spool error? [10:47] fwereade: sure, understood. A refresh at an arbitrary point on another goroutine could present a very odd view of the world to another observer. [10:48] fwereade: so a Refresh is *only* safe if you know there are no other observers. So Refresh is probably a bit of an anti-pattern. [10:49] voidspace, most of them aren't goroutine-safe, so the two observers are the internal and external views of the object -- but, yeah [10:50] voidspace, (State itself should be, but most of the stuff you get from a state will not be) [10:50] right [11:01] mattyw: nope, it just made logs harder to read [11:01] it does try a lot :) [11:11] bogdanteleaga, yeah - it's the first time I've read the logs from the point of view of someone who doesn't know about the dep engine - how understandable they are leaves a lot to be desired [11:11] bogdanteleaga, we'll fix it, thanks for bringing it up [11:13] mattyw: np, shouldn't it create the directory itself though? it does still error differently if I create it manually [11:16] bogdanteleaga, all of that stuff should get handled automatically when it needs it [11:17] bogdanteleaga, but from the logs that'a all left very unclear :/ [11:18] mattyw: oh, I think I get what you mean [11:21] axw: saw :D [11:48] anyone got an opinion on what HTTP status code is appropriate for the error with code params.CodeOperationBlocked [11:48] ? [11:48] i'm pretty sure that ErrBadRequest is not correct [11:48] s/Err/Status/ [11:48] perhaps StatusForbidden, I guess [11:56] 403 Forbidden seems close (I understood you, but you can't do that) [11:57] 409 Conflict is potentially valid (you made a request which conflicts with the fact it is currently blocked) [11:57] rogpeppe: ^^ [11:58] jam: i've gone with Forbidden [11:58] jam: (thanks) [12:05] [11:47:59] fwereade: sure, understood. A refresh at an arbitrary point on another goroutine could present a very odd view of the world to another observer. [12:06] voidspace: just picking up on that at random [12:06] voidspace: you *can* get a refresh at an arbitrary point in a goroutine [12:06] voidspace: if we had shorter refresh times, it would happen more often [12:08] rogpeppe, expand? we were thinking of Refresh calls on state types [12:08] fwereade: ah, totally out of context then, ignore me :) [12:09] fwereade: i always thought that it was a bit of an anti-pattern that state types had any internal state at all [12:09] rogpeppe, yeah, I think you're right there [12:09] rogpeppe, hey ho, we endure :) [12:14] cmars: still up for meeting in 10 or so? [12:14] jam, yep, i'm in the hangout [12:16] dimitern, ping [12:16] alexisb, omw [12:35] axw: hi, could you please take a look at http://reviews.vapour.ws/r/2633/ ? it's proposed against a feature branch [13:43] abentley: ping? [13:45] perrito666: pong [13:45] abentley: by any chance would you know how I can get a hold of the packaging script of juju-mongodb? [13:47] perrito666: Sorry, I don't know where that's kept. On our team, sinzui is the main packager, but I don't think he did juju-mongodb. I think that was juju-core. [13:48] katco: sorry for the late notice, I'm going to have to jet at 15 past the hour - our 2 year old threw up this morning and so I'm going to have to run an errand that my wife was going to do, while she's home with the kids. I'll only be gone an hour, but it's time sensitive. [13:49] natefinch: no worries... it's just us for the standup, so shouldn't even take 15m [13:49] katco: oh yeah, forgot eric's gone now, too. [13:50] abentley: thank you :) Ill keep hunting [13:51] Sorry I can't help more. [13:52] perrito666: Try jamespage: https://dogfood.paddev.net/ubuntu/+source/juju-mongodb/2.4.10-0ubuntu1 [13:52] abentley: you did, :) one less palce to look for [14:18] abentley: Notes look ok. We are just looking for a time for those non-voting to be enabled [14:19] xwwt: And commitment from core on the ones we can't enable without their help. [14:22] abentley: yeah, commitment on timing. we will continue to work through that on the release meeting too. [14:25] abentley, perrito666: hello [14:25] jamespage: hello :) [14:25] second part of my apiserver cleanup. reviews much appreciated. https://github.com/juju/juju/pull/3307 [14:25] jamespage: do you happen to know how can I get a hold of the scripts required to create a custom package for juju-mongodb ? [14:26] perrito666, bzr branch ubuntu:juju-mongodb [14:26] katco: this carries on from yesterday's branch if you wanna take a look [14:26] jamespage: that was easy, tx :) [14:26] rogpeppe: probably won't have time today... meeting day [14:27] katco: np, thanks for yesterday's review anyway [14:27] rogpeppe: np [14:27] natefinch: wanna take a look? 30 lines deleted :) [14:28] rogpeppe: he had to run an errand... kiddos sick [14:28] katco: ok, thanks [14:28] perrito666 darling, are you around? [14:29] mgz: I am sweetie (I feel a bomb coming my way) [14:30] $ grep -F 1.24 cmd/juju/status_test.go [14:30] - new available version: "1.24.7" "1.24.7 \n"+ [14:30] rev 3b8ee08c9dd6c27c1147f2aab157868987489d8c seems incomplete [14:30] mm, I am pretty sure I replaced that after [14:33] mgz: -n? [14:34] shi*** [14:34] fixing [14:34] perrito666: 3026, 3131 [14:34] you got *most* of them :P [14:36] mgz: I suck [14:39] mm, it is fixed everywhere excepting there [14:40] mgz: running tests now [14:44] perrito666: we'll need to combine your branch with the version bump to get the landing allowed, [14:45] perrito666: see https://github.com/juju/juju/pull/3299 [14:45] mgz: you can pull -r https://github.com/perrito666/juju/tree/hotfix_1.24_hardcoded_version_in_tests into the bump patch [14:45] perrito666: thanks, we'll do that [14:47] mgz: sorry again [14:47] perrito666: worth it just to wind you up :) [14:49] mgz: pleased to please :p [14:50] alexisb: our meeting conflicts with the juju-core meeting. want to move to after our 9am? [15:00] frobware, hey, I'm back again - should we start the call? [15:00] yep [15:00] frobware, standup HO? [15:00] dimitern, yes [15:18] fwereade: ping [15:20] dimitern: ping [15:23] voidspace, pong [15:25] fwereade: I'm struggling to write the assert I need I'm afraid and would appreciate some help [15:25] fwereade: I need to assert that the new address (type state.address) is in either the addresses field or machineaddresses field of the current machine doc [15:26] fwereade: but the $in operator doesn't work like that, and as far as I can tell there are no good references on mgo asserts [15:26] nor any similar examples that *I* can find [15:27] fwereade: this is for state/machine.go [15:27] fwereade: current state of code at http://reviews.vapour.ws/r/2593/diff/# [15:28] fwereade: it would go inside the getPreferredAddressOps method [15:31] voidspace, something like {$or: {addresses: {$in: [ADDR]}, machineaddresses: {$in: [ADDR]}}} ? [15:31] just found the mongodb operator docs [15:32] ah cool :) [15:32] fwereade: will try that and see what happens, I had that as a straight $in and it said it needed an array [15:33] and that addr was not a valid value [15:33] voidspace, yeah, it's ugly but I think it works [15:33] in some of the various permutations [15:33] *I tried [15:33] I have a call, will try in a bit [15:34] does anyone know if the /environment/$uuid/charms GET endpoint is used at all by anything? [15:35] rogpeppe: is that used by the gui to get local charm info? /me isn't sure what endpoint is called but it looks like something we'd want [15:36] rick_h_: i think that's an API call - i'm talking about the REST endpoint here [15:42] fwereade: ping [15:42] rogpeppe, pog [15:42] fwereade: your name seems to be associated with some of this code, so perhaps you can tell me... [15:42] * fwereade gets nervous [15:42] fwereade: i'm looking at apiserver.charmsHandler [15:43] fwereade: it seems that sometimes it sends errors as JSON errors and sometimes just as text (with http.Error) [15:43] * fwereade *thinks* he wants to point at frankban? [15:43] fwereade: can you think of a reason why that might be a good thing? [15:43] rogpeppe, I certainly can't [15:43] rogpeppe, I do remember drivebying some fix there a while back [15:43] rogpeppe, but I don't think that was related? [15:44] fwereade: do you think it would considered API breakage to fix that? [15:44] fwereade: (FWIW i can find any client code in juju-core that does a GET from that endpoint) [15:44] rogpeppe, probably :-/ but if you send the right thing *and* the wrong thing (with a comment explaining why) that might sound more sound plausible? [15:45] fwereade: (and none of those errors have any test coverage) [15:45] rogpeppe, ffs :( [15:45] fwereade: the two things are mutually exclusive [15:45] fwereade: you can't send a JSON-encoded error *and* the error as text [15:45] rogpeppe, oh balls ofc sorry [15:45] rogpeppe, end of day ;p [15:45] fwereade: np :) [15:46] fwereade: for my purposes, I'd very much like it all to standardise on one error format that can return an error code too [15:46] rogpeppe, I think it is api breakage then... can we have a /v2 handler, perhaps? [15:47] fwereade: my take on it is that there's currently no way to write a correct client [15:47] fwereade: so we're actually fixing things not breaking them === tvansteenburgh1 is now known as tvansteenburgh [15:48] rogpeppe, juju error code, not an http error code? [15:48] fwereade: yes [15:48] rogpeppe, I would like that too, but... I don't see why it's impossible to write a client? bloody hindering awkward, yes [15:49] fwereade: BTW if you have any time (unlikely, I know) it would be great to have a review of my ongoing apiserver cleanup http://reviews.vapour.ws/r/2689/ [15:49] rogpeppe, but I can't see how to change it without potentially breaking the clients that have gone to the effort [15:50] fwereade: so, the only decent way that I can see to write a non-breaking client is to check the content type and unmarshal as JSON if json and text otherwise [15:50] fwereade: so if we change the bad endpoints to return JSON, that client would still work [15:51] fwereade: also, all but one of the textual errors will be impossible to get unless there's juju internal breakage (corrupt charm cache files) [15:53] fwereade: the one error that may be possible to get would be "directory listing not allowed", but tbh the worst that can happen is that some client gets an error that doesn't look like the error they expect in that case. [15:54] rogpeppe, honestly it's too hairy for me, I have convinced myself that breakage is impossible, and been wrong, too many times :( [15:54] rogpeppe, would much rather see a new version [15:55] fwereade: the fact that this is only on a very unusual error path and that none of our actual juju code uses the endpoint makes me think that it should be ok [15:56] fwereade: do you know of any external programs that use this endpoint at all, let alone relying on juju internal service error formats? [15:57] rogpeppe, no, and that's the problem -- if I knew who used it I'd be able to find out how [16:00] fwereade: so this as a basic syntax (not yet with $or) at least is correct syntax [16:00] fwereade: bson.D{{"addresses", bson.D{{"$in", []address{addr}}}}} [16:00] fwereade: the value we're checking needs to be in an array, that's what was fooling me [16:00] however... [16:00] that fails with: "cannot set addresses of machine 0: state changing too quickly; try again soon" [16:00] fwereade: i think it's reasonable to change the internal errors as they are not possible to trigger, and thus impossible to rely on anyway. [16:00] (deterministically it seems) [16:01] voidspace, heh, ok, so that assert triggers? dammit [16:02] fwereade: i honestly think there's room for a small amount of pragmatism in the case of the "directory listing not allowed" error [16:02] voidspace, bugger, an address is a struct, isn't it, not sure how that'll play [16:02] fwereade: yup [16:02] fwereade: especially given the marginal cost of a version change [16:02]