[00:03] perrito666: in standup now [00:03] going [00:06] can i make LP:#1585005 a blocker so i can land a fix for it? :) [00:06] Bug #1585005: list-* commands should be aliases for what they're listing

[00:07] alexisb: ping [00:08] alexisb: we know what the problem is for the network address issue [00:08] we know why it fails [00:08] we know where the bad data gets in [00:08] we don't know WHY the bad data gets in [00:08] that is what we are looking at now [00:08] we know how to avoid the problem, but we are trying to work out why the problem is happening [00:09] thumper, ok, please keep the bug updated [00:09] I'll go add that to the bug [00:11] https://bugs.launchpad.net/juju-core/+bug/1588574 [00:11] Bug #1588574: Session already closed in state/presence

[00:11] ^ fix comitted !! [00:12] sweet davecheney! [00:12] thumper: i punted on two issues which I'll work on today [00:12] so we could land this [00:13] alexisb: well, don't open the champaign yet, let's see if I got the fix right :) [00:13] davecheney: which two issues are those? [00:19] cannot resolve URL "cs:trusty/apache-flume-syslog-4": cannot resolve charm URL "cs:trusty/apache-flume-syslog-4": cannot get "/trusty/apache-flume-syslog-4/meta/any?include=id&include=supported-series&include=published": Get https://api.jujucharms.com/charmstore/v5/trusty/apache-flume-syslog-4/meta/any?include=id&include=supported-series&include=published: dial tcp: lookup api.jujucharms.com on 127.0.1.1:53: read udp 127.0.0.1:43666-> [00:19] 127.0.1.1:53: i/o timeout [00:19] WTF? [00:19] this error is shown to the user when deploying charms [00:26] thumper: wow, that is incredebly subtle [00:26] the mgo issue [00:27] yeah [00:27] kinda shit [00:27] thumper: first issue is https://launchpad.net/bugs/1590161 [00:27] Bug #1590161: apiserver/client: panic: Session already closed [00:27] we are going through mgo with logging now [00:27] the second issue is a followup requested by william to make the presence watchers implement worker.Worker [00:27] i'll do that first then tackle https://launchpad.net/bugs/1590161 [00:32] api/client_test.go: [00:32] 31: jujunames "github.com/juju/juju/juju/names" [00:32] you're killing me [00:36] Bug #1590205 opened: list-actions should produce tabular output [00:42] Bug #1590205 changed: list-actions should produce tabular output [00:48] Bug #1590205 opened: list-actions should produce tabular output [00:50] menn0: here is a simple one to warm up on https://github.com/juju/juju/pull/5563 [00:55] * thumper -> dog walk [01:01] wallyworld: free to chat now? [01:02] axw_: give me a few, just doing a review [01:02] wallyworld: sure, ping me [01:10] mongo continues to impress: https://engineering.meteor.com/mongodb-queries-dont-always-return-all-matching-documents-654b6594a827#.k7ljsk6fv [01:15] natefinch: :o [01:15] axw_: right? It's a race condition that should be unlikely, especially in a tiny DB like ours, but still... [01:19] davecheney: sorry, I've been on calls - ship it! [01:23] menn0: ta [01:24] davecheney: one thing, are you aware of the helpers in the worker/workertest package? [01:25] davecheney: there's something in there which negates the need for that separate assertStopped in the presence package [01:25] i was not [01:25] i'm not fussed if you leave it in this case, but it's a good package to know about. there's lots of good stuff in there (all by Will) [01:25] it's fairly new [01:25] worker.Stop(w) does the job [01:26] axw_: sorry, just finished doing review, have 30 minute meeting, will ping when i can [01:26] wallyworld: np [01:28] davecheney: take a look at worker/workertest/check.go. Those helpers handle the case of Stop/Wait not returning. === menn0 is now known as menno-afk [01:29] menno-afk: in my case, they have to return, otherwise the mongo driver will panic [01:29] thumper: i'd love a second look at http://reviews.vapour.ws/r/4973/ when you're back [01:57] axw_: standup ho? [01:57] wallyworld: be there in a mo [02:02] menno-afk: thumper next cab off the rank, https://github.com/juju/juju/pull/5564 [02:05] natefinch: wow, that blog post, what garbage [02:07] davecheney: yuuup. kick you right in the assumptions [02:30] natefinch: got 5 minutes to hear weirdness? [02:31] thumper: I love weirdness [02:32] natefinch: https://hangouts.google.com/hangouts/_/canonical.com/onyx-standup?authuser=0 === menno-afk is now known as menn0 [02:39] davecheney: looking [02:43] menn0: confirmed it is mgo bug [02:43] on the way in to mgo, it is a bson.D [02:44] when it is applying finally, it passes through a bson.M [02:44] so it futzes the ordering [02:44] I've got a bunch of extra logging [02:44] have all the flusher logging too [02:44] but you haven't found the exact place yet? [02:47] looking [02:48] thumper: it's so weird that we can't find the bson.M usage in the code [02:49] menn0: when it needs to run [02:49] it hits a queue [02:49] of four [02:50] thumper: urg [02:50] this pinger api [02:51] it's like pulling out the pin on a hand grenade then loosing it in the couch [02:52] davecheney: haha [02:52] davecheney: so you've dealt with the panics that occurred during the tests... are you sure the recover() wasn't hiding panics in production too? [02:53] davecheney: apart from that concern the change LGTM [02:54] davecheney: i'd be a bit happier if the pinger kept it's own Session rather than trying to copy the one attached to the pings collection every time it wanted one [02:55] menn0: all the bugs are in tests [02:55] their shutdown behaviour is weaksauce [02:55] but we wouldn't know if there were panics in production that the recover was hiding right? [02:56] I agree you've dealt with the test issues. [02:56] i'm not interested in having a discussion about stability by hiding panics [02:56] this code was added ages ago when we didn't understand what was causing the problem [02:56] you cannot take that as a basis for it being the "right" thing, and I am proposing something more dangerous [02:57] panic => test or production, we find hte cause and fix that, not paper over them [02:57] this isn't just a philosophical argument [02:57] i see where you're coming from but a panic in production is still a panic [02:57] panics can break lock invadiants [02:57] menn0: that's why we have upstart to restart processes that die [02:59] i'd like another person to take a look, but apart from this concern the change looks good [03:54] func (s *MachineMockProviderSuite) SetUpTest(c *gc.C) { // Change to environ that supports HasRegion s.commonMachineSuite.SetUpTest(c) [03:54] } [03:54] ^ wtf, [03:54] Bug #1590237 opened: juju2 usability: removing cloud makes list-credentials behave weird

[03:54] Bug #1590239 opened: juju2 usability: I can import credentials of the wrong type, but can't list them/use them.

[03:55] go func() { [03:55] c.Check(a.Run(nil), jc.ErrorIsNil) [03:55] }() [03:55] defer func() { c.Check(a.Stop(), jc.ErrorIsNil) }() [03:55] WTF [03:55] the call to a.Stop() acn happen _after_ the call to a.Run!!! [04:01] goroutines, how do they work? [04:04] Run(c *cmd.Context) error [04:04] ^ sign of a bad API, every call to this method is passes nil! [04:10] func (a *machineAgentCmd) Run(c *cmd.Context) error { machineAgent := a.machineAgentFactory(a.machineId) return machineAgent.Run(c) [04:10] } [04:10] func (a *machineAgentCmd) Run(c *cmd.Context) error { machineAgent := a.machineAgentFactory(a.machineId) return machineAgent.Run(c) [04:10] } [04:10] how does this help ? [04:49] axw_: i have to go get car. here is a wip, based in part on your work yesterday (to pass in and store the cloud name against a model) http://reviews.vapour.ws/r/5012/ needs more tests and i haven't gone through it all yet but i need to head out for an hour or so [04:50] wallyworld: okey dokey. about to have lunch, will look soon [04:51] np, ty, bbiab [05:01] aahhh stupid frigging slow ass tests [05:15] I think this test never fails, it just waits for ever [05:19] hey, naked receive on a channel, thanks, tests. [05:40] wallyworld: test failures are killing me. I'm going to have to pick this up in the morning. === frankban|afk is now known as frankban [07:40] axw_: off to soccer photos, changes pushed, still got some tests to fix, but only minor changes to come [07:41] wallyworld: ok, will look shortly [09:01] frobware, dimitern: hangout time! [09:01] oops omw [09:03] fwereade, jam: standup today? [09:04] dooferlad: thanks, just finishing the other meeting [09:07] jam: would love to catch up about a couple of things, did you want to talk later, maybe an hour's time? [09:08] wallyworld: sure. [09:08] ok, will ping after dinner [09:13] menn0: actually it was block devices that I fixed: https://bugs.launchpad.net/juju-core/+bug/1461871 [09:13] Bug #1461871: worker/diskmanager sometimes goes into a restart loop due to failing to update state

[09:13] sorry, clearly I should have emailed the list :/ [09:14] axw_: all good. the fact that it's happened twice just means we need to prevent it happening again :) [09:14] menn0: +1 [09:45] axw_: can u help me with bug 1590172 [09:45] Bug #1590172: ERROR cmd supercommand.go:448 autorest:WithErrorUnlessStatusCode POST https://login.microsoftonline.com/fb30bf07-xxxx-xxxx-xxxx-02ef08680fb9/oauth2/token?api-version=1.0 fa iled with 400 Bad Request [09:49] ejat: I'll try. just checking, did you redact the UUID in the URL there? [09:49] or is that verbatim [09:50] axw_: yups .. i edit it [09:51] bcause its my tenant-id [09:51] ejat: yep just checking that it wasn't like that in your credentials file [09:52] axw_: its in full in my credentials file [09:52] ejat: hmm, so I *think* that the only reason why would get a 400 is if the application, subscription or tenant ID is invalid [09:52] ejat: you'll get a 401 if they're valid but the password is invalid [09:52] (which is kinda poor security practice) [09:52] ic .. [09:53] ejat: do you have the azure CLI on your laptop? [09:53] axw_: yes i am [09:54] ejat: can you please confirm the tenant and subscription IDs in ~/.local/share/juju/credentials.yaml by comparing with the output of "azure account show" [09:55] yes .. its the same [09:59] ejat: and then please compare application-id to the "AppId" field for the application you created, by running "azure ad app list" [10:00] ejat: I can confirm that if I change the application-id to something invalid, I get the same error message [10:04] i think that might cause me a problem ... [10:05] may i ask why need to Create an Azure Active Directory (AAD) application ? [10:05] i mean .. its in 2.0 not in 1.0 [10:06] and what is the best --name, --home-page, and --identifier-uris based ? project / models ? [10:07] ejat: that's just what MS/Azure says what we (Juju) are supposed to do [10:07] ejat: it doesn't really matter, anything will do [10:07] ejat: BTW there's docs here, they don't seem to be linked very well at the moment: https://jujucharms.com/docs/devel/help-azure [10:08] okie ... noted ... at least i can tell the MS here the same as what u told me :) [10:08] axw_: yeah im refering to that doc [10:09] ejat: also, you may run into this after you get past the 400 issue: https://bugs.launchpad.net/bugs/1589890 [10:09] Bug #1589890: juju2 azure fail with error 409 network conflict

[10:11] error: 'ad' is not an azure command. See 'azure help'. [10:12] ejat: what version of the CLI do you have? you may need to update it. I'm on 0.10.1, and it exists there [10:13] so in short , juju2 not working atm with azure? [10:13] ok let me try to update it [10:13] jam: you free now? https://hangouts.google.com/hangouts/_/canonical.com/tanzanite-stand [10:16] ejat: it is working, see comment #1. you just need to run a few extra azure CLI commands for Juju to work properly [10:19] * ejat update n create ad 1st .. [10:19] thanks for the guiding [10:26] axw_: [10:26] dimitern: What's workload-status? It's always unknown for the applications in the maas-spaces model. [10:26] PS C:\Users\Lenovo> azure --version [10:26] 0.10.1 (node: 4.2.4) [10:26] PS C:\Users\Lenovo> azure ad app create --name "informology" --home-page "http://www.informology.my" --identifier-uris "http://www.informology.my" --password $APP_PASSWORD [10:26] error: 'ad' is not an azure command. See 'azure help'. [10:26] babbageclunk: that's always unknown, unless set by the charm with e.g. 'status-set active "Ready"' [10:27] babbageclunk: i.e. depends on the charm [10:27] dimitern: ahh. So maybe these charms just don't set it? [10:28] babbageclunk: yeah [10:28] dimitern: although I guess that might also mean that they're not actually working. [10:28] babbageclunk: that's why it's unknown - we just don't know :) [10:29] dimitern: How can I check? haproxy's listening on port 80, but in the public space. [10:29] ejat: maybe you need to run "azure config mode arm"? [10:31] babbageclunk: ssh in and check the logs for errors? [10:32] dimitern: yeah, I'll try that [10:33] dimitern: How could I make a web request to the running haproxy? Do I need to create an interface on the same vlan on the host machine? [10:33] babbageclunk: or just open haproxy's public address as a webpage [10:34] babbageclunk: if you can't access the public ip directly [10:34] dimitern: I tried that, nothing. [10:34] babbageclunk: try using sshuttle [10:34] dimitern: Hmm, ok [10:34] Bug #1590362 opened: azure: Azure API errors do not contain information about the cause

[10:34] babbageclunk: e.g. `sshuttle -r maas-hw 10.14.0.0/20 10.30.0.0/20 10.50.0.0/20 10.100.0.0/20 10.150.0.0/20 10.200.0.0/20 10.250.0.0/20 10.99.0.0/20` [10:35] babbageclunk: maas-hw is configured in my ~/.ssh/config to be Host: 192.168.1.104, User: maashw, and I can ssh with my pubkey [10:36] axw_: http://paste.ubuntu.com/17113187/ [10:36] babbageclunk: the ranges after that match any maas subnets you want to access (but otherwise can't except when on the rack machine) [10:36] dimitern: Ok. [10:39] ejat: hrm. did you do "azure login" again after changing the config mode? [10:41] axw_: yups [10:42] axw_: http://paste.ubuntu.com/17113282/ [10:43] ejat: ok, well I guess azure just doesn't let you do that. I'm not sure what we can do about that [10:43] axw_: i need to contact Azure ? [10:44] ejat: worth a try. it may be that it's a limitation of BizSpark [10:51] axw_: Your question was successfully submitted to Microsoft Support using your Azure Subscription Management Support plan. A Microsoft support professional will contact you within 2 hour(s). [10:51] :( [10:52] ejat: that's better than I expected :p [10:52] ejat: I need to head off shortly, could you please update the bug with the outcome? or leave me a PM here, either way [10:52] thats from azure portal .. or else .. i need to ask favor from MS SG @ MY to expedite [10:53] okie will do .. thanks a lot .. will pm you once get feedback from Azure [10:59] axw_: i've updated the bug [11:03] ejat: thank you. I'm heading off now, good night! [11:04] dimitern: I can't juju ssh to any of the machines in my model - I can ssh to them directly using the IPs of the physical interfaces from the MAAS UI (192.168.150.*), but not with the IPs in juju status (192.168.10.*). Is that what sshuttle fixes? [11:04] babbageclunk: yeah, it should fix that [11:05] dimitern: ok, thanks [11:05] babbageclunk: alternatively, you can add static routes for all maas subnets on your local machine, but sshuttle makes it a lot easier [11:07] dimitern: ok, I think that makes sense - I was kind of fumbling towards realising that it was because of missing routes, but good to have it confirmed. [11:14] dimitern: ok, I've got sshuttle running and it says it's connected. I still can't juju ssh to the machines - should I be able to? [11:16] babbageclunk: try passing -v at the end of juju ssh .. [11:16] babbageclunk: also try ping [11:18] dimitern: nope. Maybe the maas controller can't get to the hosts either? [11:18] dimitern: That seems like it would explain it. [12:02] wallyworld: sorry about the delay, are you still around? [12:02] yep [12:03] https://hangouts.google.com/hangouts/_/canonical.com/tanzanite-stand [12:10] jam: ? [12:40] dimitern: are you around? [12:40] babbageclunk: yeah [12:41] dimitern: I can't ping the mysql host on its db space IP from the mediawiki host. But I can ping it on its .150 ip (default space) [12:42] dimitern: The routes look right, I think [12:42] babbageclunk: how is mysql deployed? bindings? [12:43] dimitern: yeah - from the bindings defined in the demo bundle [12:44] dimitern: http://pastebin.ubuntu.com/17115770/ [12:46] babbageclunk: and for mysql? [12:48] dimitern: http://pastebin.ubuntu.com/17115875/ [12:49] babbageclunk: btw it might be worth updating to the tip of master and rebuilding, so you can take advantage of my fix of yesterday; that e/n/i with addresses on both the bridge and the underlying NIC, e.g. ens3.10 might be causing issues [12:49] dimitern: Hmm, ok [12:50] babbageclunk: wait a sec [12:51] babbageclunk: can you paste that last one again, but with the output from 'ip route show' please? I'm not quite sure, but it looks like there are no gateways for the vlan subnets [12:52] dimitern: ok, gathering [12:52] dimitern: babbageclunk: http://fabriclondon.com/club/listing/1294 [12:53] dimitern: http://pastebin.ubuntu.com/17116157/ [12:53] voidspace: ooh nice! thinking of going? [12:54] dimitern: yep [12:54] voidspace: not all of us will be away from our families and going all rumspringa! [12:54] babbageclunk: your choice... :-) [12:54] you're not having a networking meeting in fabric... [12:54] mgz: hah, it would be appropriate... [12:55] voidspace: well, I'd love to go actually.. brush up some degrading social skills :D [12:55] dimitern: :-) [12:57] babbageclunk: ok, it looks fine; and you can't ping 192.168.10.3 from 192.168.10.4 ? [12:58] dimitern: nope - just get destination host unreachable. [12:58] dimitern: Do you think I should rebuild and rebootstrap? [12:58] babbageclunk: can you ping both of those IPs from maas? [12:59] babbageclunk: it will help, but let's try to see if something else is wrong first [13:00] dimitern: nope, can't get to either of them from maas [13:00] dimitern: I haven't got the vlans on the maas controller. [13:01] babbageclunk: well, that's why then :) [13:01] babbageclunk: maas needs to be on those vlans [13:02] dimitern: that's why can't get to them from the maas controller. But does that also affect the connection from mediawiki to mysql? [13:02] dimitern: (I mean, I was going to fix the maas vlan issue, particularly so sshuttle would work, but I still don't understand the other bit.) [13:03] babbageclunk: yeah, since maas is the gateway for all subnets, if maas itself cannot reach them they won't be able to either [13:03] dimitern: Oooooooooooooh [13:03] babbageclunk: :) [13:03] dimitern: ok, that makes a lot more sense. Sorry for extreme dumbness! [13:03] babbageclunk: my e/n/i on maas looks like this: http://paste.ubuntu.com/17116341/ [13:04] babbageclunk: np :) [13:08] * dimitern steps out for ~30m [13:18] voidspace: ping and hello === katco` is now known as katco [13:25] or anyone else available, need an information [13:26] TheMue: hello [13:27] dooferlad: hey, hello [13:28] dooferlad: I need the e-mail address of the HR department (or somebody there) for the request of a special official paper. [13:28] dooferlad: sadly I've got no access to mail directory anymore, hehe [13:29] TheMue: ok, will PM you in a momenty [13:29] dooferlad: great, thanks [13:29] dooferlad: and juju is doing well? [13:30] TheMue: yep [13:31] dooferlad: great, I'm missing my time at Canonical [13:36] TheMue: you are missed! [13:37] dooferlad: am I? [13:38] TheMue: more hands, more brains, more code :-) [13:38] TheMue: and knowledge of Whisky of course [13:38] dooferlad: sadly I don't see an according job offer [13:38] TheMue: you are looking? [13:38] dooferlad: would start immediately [13:38] dooferlad: yep, I do [13:39] TheMue: I will keep my ear to the ground for you [13:39] dooferlad: had no luck with my last employer [13:40] TheMue: :-( [13:41] dooferlad: bad climate, and also lost one important customer when I started, so we only worked halftime. and then they kicked the last employed three. *sigh* [13:42] TheMue: that sucks. I am sorry to hear that. [13:45] dooferlad: already have some new contact, will see [14:53] Bug #1590468 opened: Controller machine log contains worker errors

[14:54] you know a test is probably doing something wrong when you see atomic.StoreUint32 [14:58] s.checkStartInstanceCustom(c, m, "pork", s.defaultConstraints, nil, nil, nil, false, nil, true) [14:59] no wonder fixing tests takes 3x as long as implementing the feature [15:07] redir: standup time [16:15] babbageclunk: hey, did you manage to sort out the vlans stuff? [16:19] dimitern: not yet - was at code club. Trying to change the script to link the subnets at the same time as it adds the vlans to solve the rerun problem I was seeing. [16:21] babbageclunk: ok === frankban is now known as frankban|afk [17:03] fwereade, ping [17:03] you still around? [17:13] voidspace, you still around? [17:13] alexisb: yes [18:57] Bug #1590520 opened: failed to prepare container "0/lxc/2" network config: model "default" networking not supported (not supported)

=== alexisb is now known as alexisb-afk [19:14] katco: ericsnow natefinch would one of you have time to make sure I am understanding the build transaction convention in state in about 30m? [19:14] about to have lunch === redir is now known as redir_lunch [19:14] redir_lunch: sure [19:15] ericsnow: redir_lunch: wouldn't mind sitting in on that just as a refresher [19:18] Ok. I'll ping you both after I hoover some stew [19:36] if at first the test does not succeed... delete the test [19:42] back === redir_lunch is now known as redir [19:44] I have a question about the API, why do I need to specify the model UUID when connecting to a controller? Why can't I just connec to the controller with user/pass? [19:51] marcoceppi: that's a thumper question, I think [19:53] ping katco ericsnow [19:53] ericsnow is gonoe [19:54] s/gonoe/gone [19:54] redir: he's probably just restarting or something [19:54] marcoceppi: because of stuff :P [19:54] watch ericsnow status [19:54] :) [19:55] rick_h_ natefinch it's a really freakin bummer for libjuju library design [19:55] marcoceppi: I htink it's because there's possibility that it's not clear in future uses [19:55] marcoceppi: and the UUID is required to help make sure it finds its way to the api server correctly [19:55] esp since we're not connecting to a controller at that point, we're connecting to a Model, so the idea of a Controller object is really moot [19:56] I would have expected: connect to Controller, get list of models my user has, connect to model [19:57] I agree w/ marcoceppi I would think I'd connect then switch to a model. or connect to model@controller which might do it all at once [19:58] marcoceppi: redir yes, but I think part of the thing is the way we did controllers was a bit "easy route" in that it's just a model that we treat special and it takes the same api calls as models [19:58] so it's just not in all the calls to make it pulled out different and not require it [19:58] but controllers are very special cases [19:58] how do I do ensure-ha? it's not against a model, it's a controller level feature [19:59] list me all your controller endpoints, as another example [20:00] right, I'm +1 on trying to design the lib to hide that, but technically that's how it works is you happen to ask one of the dozen models a special question and he answers [20:05] rick_h_: understood that's how we did it, just saying it isn't how me the user would expect it to be. [20:05] redir: I know, I'm +1 on trying to hide it as an implementation detail as much as we can [20:05] redir: but the api is the api so there will be limits [20:09] rick_h_: i think maybe what redir and marcoceppi are trying to say is that it would be nice if the api were designed differently :) [20:10] katco: I know, and I know alexisb-afk would fly to my house and mount my head on a pike if I suggested changing it atm so a bit stuck [20:10] rofl [20:10] I like my head /me inserts "attached to it" joke here [20:10] rick_h_: yes. yes she would. i don't think anyone is blaming you or anyone who implemented it. them's the constraints we were working under. [20:12] rick_h_, whats up? [20:12] * alexisb-afk reads back scroll [20:12] alexisb-afk: nothing, just noting that I don't want you to kill me :P [20:12] and gets he axe ready [20:12] * marcoceppi ducks [20:14] rick_h_, this discussion should have taken place back in october [20:15]