=== _thumper_ is now known as thumper [00:22] wallyworld, axw: if we are bumping all the API versions and hacking at the wire protocol with a machette, can you please add explicit serialization directives to everything on the wire? [00:22] we can try [00:22] surely this is just a sed script right? [00:22] :-( === JoseeAntonioR is now known as jose [00:22] because sed is all you need [00:23] * wallyworld growls at thumper [00:23] * thumper hums the "all you need is love" tune... [00:23] all you need is sed... sed... [00:23] sed is all you need [00:23] groan [00:24] * thumper is distracting himself from what he really needs to be doing [00:26] Bug #1540076 changed: Storage Hooks before Install [00:32] * thumper is wondering when master will get another CI run... [00:32] last was several days ago... [00:32] like four [00:32] the dep engine merge hasn't been blessed or cursed yet... [00:49] thumper: CI is paused over the weekend, should get another run later tonight our time. i'm keen to see it get a run because after i merged the dep endgine stuff into the api-command-rename feature branch CI reports intermittent leadership failures which i claim have nothing to do with renaming some methods [01:16] wallyworld: was out, Charlotte's first day of grade one. responded to your email [01:16] np, hope she went ok [01:16] yep [01:39] wallyworld: I think one use-case for adding volumes after deploy is, e.g. if you have a MAAS and add a disk to the machine [01:39] wallyworld: MAAS has no agent, so you have no other way of discovering it [01:40] that is true, but that then becomes an add storage thing maybe [01:40] needs thought as to how it all fits the model [01:41] wallyworld: how do you mean? add-storage would always be required, to associate the volume with a unit [01:42] exactly, that's after a unit is deployed, you add a volume, possibly one you added post deploy to the machine [01:42] but at deploy time, i'm not sure it makes sense to have to have storage and placement directives macthing [01:42] maybe i'm wrong [01:43] wallyworld: well if you have the machine already, then you wouldn't be able to deploy a unit that *requires* storage if you only have that OOB-added disk on the machine [01:44] wallyworld: anyway, this is just speculation, needs more requirements [01:44] yep [01:51] thumper: can you let me know if you have 5 minutes for a chat/question [01:51] wallyworld: sure, now is as good as any [01:51] ok, 1:1 [03:48] thumper: it works! [03:48] root@ip-10-251-40-215:~# echo -e "GET /debug/pprof/goroutine?debug=1 HTTP/1.0\r\n" | socat unix-connect:/tmp/.5636.pprof STDIO | pastebinit [03:48] http://paste.ubuntu.com/14847387/ [03:49] * thumper took a minute disecting that command [03:49] very cool [03:50] yeah, it's a mouthful [03:50] davecheney: what are the lines like this: 2 @ 0x437b2a 0x446382 0x445372 0x713277 0x737040 0x464d41 [03:50] most http tools are like "unix socket, wat?!" [03:50] those are the raw program counters [03:50] ah [03:51] is there memory dumps? [03:51] here is what the comp[uter see, http://paste.ubuntu.com/14847399/ [03:51] ah [03:51] the extra lines are for humans, see the ?debug=1 flag [03:51] much nicer to see the full method [03:52] what about memory? [03:52] can we see that? [03:54] http://paste.ubuntu.com/14847418/ [03:54] i don't immediately know how to interpret those numbers [03:55] but pprof can [03:55] i've never found the go heap profiler to tell me useful information [03:55] i wanted to implement this facility so I could find out what the goroutines were doing inside a program [04:34] http://paste.ubuntu.com/14847633/ [04:34] friday the juju deployer worked fine for me [04:34] today, nothing [05:06] see y'all tomorrow [05:07] davecheney, juju-deployer==0.3.6 is an old version of deployer. You probably want 0.6.1-1 from 'deb http://ppa.launchpad.net/juju/stable/ubuntu trusty main' === mup_ is now known as mup [06:54] Bug #1540047 changed: Error in Jujuclient with Python3 [07:00] Bug #1540047 opened: Error in Jujuclient with Python3 [07:09] Bug #1540047 changed: Error in Jujuclient with Python3 [09:03] jam: ho? (though I have no camera today.) [09:03] be there in just a sec [10:02] voidspace, frobware, fwereade__, standup? [10:02] dimitern: omw [10:03] dimitern: got booted out - rejoining [10:20] voidspace: are you rejoining? === dooferlad_ is now known as dooferlad [10:21] ok, 2fa for me. [10:21] yay, in the middle of a hangout! [10:26] dimitern: jam: dooferlad: not sure if any of my last messages got through [10:26] dimitern: jam: dooferlad: internet went down briefly [10:27] dimitern: jam: dooferlad: dependency engine stuff may mean I get to simplify my tests a bit, I think that was all I wanted to say [10:27] voidspace: we didn't get that bit. sounds goodf. [10:27] fwereade__: hey, did you have 5 minutes, i just wanted to run something by you [10:29] voidspace, that sounds like a good next step, yeah [11:07] Bug #1540301 opened: adding storage to a unit should be able to take a specified block device

[11:40] Bug #1540316 opened: provider/maas: bridgescript should support vlans

[11:58] wallyworld: might be a minute or two late [11:58] ok [11:59] jog: i'm running the version of hte juju-deplouyer that ships in trusty [12:00] and it worked right up til friday [12:00] but this week it just decided not to work [12:02] perrito666: meeting? [12:37] fwereade__: ping? === akhavr1 is now known as akhavr [13:53] wwitzel3, you around mate? [13:54] cherylj, don't suppose you know anyone who might be awake and knows about the lxd provider, I'm having problems === akhavr1 is now known as akhavr [14:08] mattyw: ping [14:08] wwitzel3, hey hey, you worked on the lxd provider? [14:08] mattyw: yep, ericsnow and myself [14:09] wwitzel3, awesome, I'm having problems with it (it was working fine) I've almost finished typing it all out in an email I was going to put on juju-dev email list. If it's ok I'll finish that and ping when it's sent. I basically need help debugging and env that seems screwed [14:09] wwitzel3, I'll finish off the email to save time typing it all out again [14:10] mattyw: yeah, ok, I'm not on the list anymore, so if you want to also pastebin it to me, I'll give it a read and see if I have any insight [14:12] wwitzel3, I've emailed you as well [14:12] that works [14:16] menno, ping? [14:23] mattyw: I'd also ping ericsnow and direct him to the list message, he might have some insight on it too .. but yeah, I'll poke it a bit more and see if anything shakes out [14:23] ericsnow, are you around? [14:23] I wish irc displayed timezones for everyone [14:25] mattyw: he'll be around soon (if he isn't out of office) [14:25] mattyw: moonstone standup is in 30 minutes [14:31] mgz, did we ever get a dedicated 1.25 CI run for bug 1532167 - this was the backport of the maas-spaces bridge script [14:31] Bug #1532167: maas bridge script handles VLAN NICs incorrectly

[14:46] Bug #1540394 opened: Juju does not handle 429 "too many requests" from Azure

[14:52] Bug #1540394 changed: Juju does not handle 429 "too many requests" from Azure

[14:55] Bug #1540394 opened: Juju does not handle 429 "too many requests" from Azure

[14:58] mattyw: present, but about to go to standup [14:58] mattyw: I'll ping you when we're done [14:58] ericsnow, ok cool [14:59] mattyw: nevermind, I'm the only one around apparently [14:59] :) [15:01] ericsnow, take a look at my email to juju-dev for the strarting point [15:01] mattyw: will do [15:28] sinzui: ping [15:29] or, mgz: ping [15:29] hi katco` [15:29] sinzui: o/ [15:30] sinzui: at the charmer summit, have a person here trying to try out juju on joyent [15:30] sinzui: and he's getting a strange error on bootstrap [15:30] sinzui: (one sec getting err) [15:31] katco`: joyent has several ill regions over the last few weeks. us-east-3 is good. us-east-2 is locked down. us-east-1 has network issues, us-west-1 is not cleaning up networks [15:31] sinzui: we're using eu regions [15:31] sinzui: 2016-02-01 15:27:04 ERROR juju.cmd supercommand.go:429 there was an issue examining the environment: cannot create credentials: An error occurred while parsing the key: asn1: structure error: tags don't match (16 vs {class:0 tag:23 length:23 isCompound:true}) {optional:false explicit:false application:false defaultValue: tag: stringType:0 timeType:0 set:false omitEmpty:false} pkcs1PrivateKey @2 [15:32] sinzui: the environments.yaml looks like it could be ok, but wouldn't completely rule that out [15:32] katco`: I don't think someone setup their account keys properly. CI has never seen that kind of issue [15:33] sinzui: so we checked the keys in his account, and they match what's in environments.yaml [15:33] sinzui: i've not used joyent. what kind of setup could be wrong in this case? [15:34] katco`: I've never seen an error like that to day [15:34] say [15:34] sinzui: hm ok [15:36] sinzui: manta information should be the same as sdc? [15:36] katco`: my config looks like this https://pastebin.canonical.com/148857/ [15:37] sinzui: ahhh possibly what's wrong. the key he posted looks possibly like a fingerprint [15:37] katco`: yes, sdc-key-id and anta-key-id are the same, as ar sdc-user and manta-user. [15:38] sinzui: scratch that, he's specifying the path to the key [15:39] katco`: private-key-path works, though I don't use it often [15:39] sinzui: he's on os x if that helps [15:40] katco`: I haven't seen a difference between OS X and Ubuntu for joyent. ssh and keys are the same. [15:40] sinzui: hm ok [15:41] katco`: unless someone trued to compile their own juju on that host. OSX requires CGO=1 to link to the native keystore [15:41] sinzui: nah he just brew installed [15:42] sinzui: fyi when he tried to brew install alpha1, he got an error [15:42] katco`: we don't support it [15:42] sinzui: looked like the brew recipe had some kind of utf error [15:42] sinzui: ah ok [15:43] katco we onlt support jujus in *releaed* streams. other wise peoples configs break when they get updates [15:43] sinzui: right, we're just showing alpha1 here to show everyone the new stuff [15:43] sinzui: but seems like a bug in the brew recipe for 2.0? or are you saying we don't do the support for brew? [15:44] katco`: not a bug is the recipe, it is about streams not working at the time the binaries are meade [15:45] sinzui: no, i'm saying the recipe seems broke. as in brew install doesn't work for alpha1 [15:46] katco`: we did suport devel, users were not happy about incompatible configs [15:46] katco`: We solved this by creating cient for testers https://launchpad.net/juju-core/+milestone/2.0-alpha1 got clients *before* ubuntu users [15:47] sinzui: right, that's what he ended up downloading [16:04] Bug #1540437 opened: lxd provider: machines stuck at "pending" [16:04] Bug #1540447 opened: error: r.runner.RunCommands: tomb: dying

[16:07] Bug #1540437 changed: lxd provider: machines stuck at "pending" [16:07] Bug #1540447 changed: error: r.runner.RunCommands: tomb: dying

[16:10] Bug #1540437 opened: lxd provider: machines stuck at "pending" [16:10] Bug #1540447 opened: error: r.runner.RunCommands: tomb: dying

[16:18] dimitern: omw [16:33] morning ericsnow [16:33] natefinch: hey [16:33] natefinch: interested in a late standup? [16:33] ericsnow: yeah, definitely [16:37] cherylj, some success with VLAN nics on 1.25 - https://bugs.launchpad.net/juju-core/+bug/1532167/comments/28 [16:38] Bug #1532167: maas bridge script handles VLAN NICs incorrectly

[16:38] cherylj, this change was also going to get a CI branch as well. I haven't seen any reports. The PR is good to go, just waiting on sanity CI run. [16:39] dimitern: frobware: do we still have the networking thing this afternoon? [16:39] voidspace, yes. [16:39] frobware: thanks [16:40] frobware: okay, I had held off since the initial feedback showed it had problems. I'll get it on the queue [16:41] cherylj, thanks. [16:41] voidspace, we do, but I might skip it [16:46] Bug #1540469 opened: [ARM64][LXD Provider][2.0-alpha1-0ubuntu1~16.04.1~juju1] juju run command ERROR fork/exec /usr/bin/ssh: cannot allocate memory [16:47] dimitern: ok [16:48] dimitern: frobware: dooferlad: unless anyone has any questions/follow up for Jay, maybe we should skip it this week? [16:48] voidspace, +1 from me [16:50] voidspace, dimitern, dooferlad: agreed. need to merge :) [16:50] I've emailed [17:00] cheers voidspace [17:01] dimitern: can I tell the difference, on a buffered channel as I *do not* want to block, between a closed channel and a channel with no messages available [17:01] dimitern: I'm starting to think that the bool is just easier [17:03] dimitern: I think I can [17:03] dimitern: unping [17:03] voidspace: a closed channel never blocks [17:03] voidspace: an empty channel always blocks [17:04] natefinch: and I need to tell the difference between those two cases [17:04] natefinch: but I think I can use the ok parameter for that [17:04] (thanks) [17:04] natefinch: ah, I misread [17:04] natefinch: I want a non-blocking way to tell if a channel is closed or not [17:05] natefinch: just reading from it isn't enough as it will block if not closed [17:05] natefinch: but I can provide a default path in the select for that case [17:08] voidspace: yeah, was goign to say, with a default you get the non-blocking, as long as you're ok with actually taking a value off the channel if there's something there [17:08] voidspace: also, for a := range ch { works really nicely if you're looping anyway [17:09] natefinch: thanks [17:10] natefinch: in my case I'm not reading from the channel, just need to know if it's closed [17:12] voidspace: You can't tell if it's closed without trying to read from it.. the notification that it's closed is a value you read off the channel.... Why do you need to know if a channel is closed without reading off it? [17:14] natefinch: I thought it was a common pattern to use a channel to signal something - the signal being that you close the channel [17:14] natefinch: reading from the closed channel then doesn't block - which is how you know it's closed [17:14] voidspace: yes [17:15] natefinch: the problem was that attempting a read would block, and I need the check to be non-blocking [17:15] natefinch: but that's easy enough with a default path in the select [17:15] natefinch: so no problem :-) [17:15] voidspace: yep [17:25] voidspace, dooferlad, frobware, here it is: http://reviews.vapour.ws/r/3693/ [17:25] dimitern: cool, mine is nearly ready for review too [17:26] in the mean time I'm still finishing a few tests [17:34] dimitern: dooferlad: frobware: http://reviews.vapour.ws/r/3694/ [17:34] dimitern: looking at your review [17:35] voidspace, ta, looking at yours [17:35] dimitern: switching to a channel meant undoing a bunch of code, but it is nicer [17:37] voidspace, I suspected as much ;) [17:42] dimitern, taking a look now; would some manual testing help at all? [17:44] frobware, sure, in fact what I'm seeing now is that it seems to work live (bindings included), but I still have hard time fixing the unit tests [17:45] dimitern, how many unit tests are broken? what's the nature of the breakage? and depending on how many are to be addressed could these be done post the merge as collateral damage? [17:45] frobware, so the biggest issue is one of the apiaddressupdater tests hang [17:46] frobware, the rest are mostly trivial - e.g. endpoint_bindings_test.go, service_test.go - related to incomplete test setup or outdated asserts to fix [17:47] dimitern, for the test that hangs is this something the CI test would run into? [17:47] frobware, well, yeah - the merge bot will hang [17:48] frobware, I'm close to actually skipping that test for now [17:48] dimitern, right. :) [17:48] dimitern, ok, another way. if we skip the test, is there likely to be a genuine functional failure that CI would run into. [17:53] frobware, I don't believe so [17:54] dimitern, so we could throw caution to the wind here. [17:54] dimitern, cherylj: another way is to get a CI run on your branch before we merge to maas-spaces [17:55] frobware, the test in question is related to the legacy address selection by scope followed by filtering of lxcbr0 addresses, both of which are skipped on 1.9 with controller space used for selection instead [17:55] dimitern, as an aside just found an interesting problem with lxcbr0 [17:55] frobware, that's perfect, if it can be arranged [17:55] cherylj, ^^ [17:56] dimitern, is your branch now rebased against maas-spacs? I know on friday you were holding off [17:57] frobware, I rebased before proposing, that's why it took some time - a couple of conflicts - easy to resolve though [17:57] dimitern: frobware: manual test on my branch works fine by the way [17:58] voidspace, I'm still looking, but it will take some time as I have a few suggestions [18:01] * cherylj reads scrollback [18:01] dimitern: ok [18:03] ericsnow: so, for deploy --resources, if we're doing multiple resources as a single string, space separated, how do we handle paths with spaces in them? like --resources "foo=./bar foo2=./my docs/baz"? [18:04] rick_h__: ^^ [18:05] natefinch: --resources foo="./bar" foo2="./my docs/baz" [18:05] natefinch: or --resources "foo=./bar" "foo2=./my docs/baz" [18:06] natefinch: or --resources foo=./bar "foo2=./my docs/baz" [18:06] dimitern: are you passing controller space straight through to maas as a constraint at bootstrap time [18:06] natefinch: any of those should work [18:08] ericsnow: that's different than how constraints work, though [18:09] natefinch: understand, it'll have to be in order to be cross platform/etc. None of the others have actual directories in there like that. [18:09] natefinch: if this is not a problem that constraints has then we'll probably have to do it differently than constraints, no? [18:09] natefinch: constraints are built by us so we got to curate what was in there. I'd be curious if you can create a space or storage pool with a space in the name? [18:09] guessing not [18:10] ericsnow, rick_h__ : it's also sorta hard to parse.... juju deploy mysql --resources foo=bar mydatabase is mydatabase the name of the service or a resource? [18:10] natefinch: you'd have to use quotation marks [18:11] voidspace, not exactly - it's a bootstrap argument which if unset will be inferred and set during bootstrap (see the finalizerWrapper) [18:11] ericsnow: no, that could be legal for deploying mysql with the servicename mydatabase [18:12] ericsnow: or it could be a typo for a resource called mydatabase without a path [18:13] my point is --flag item1 item2 item3 is really hard to parse unless you guarantee it has to be the last thing on the line [18:13] which is probably why constraints have to be a single string [18:15] ericsnow: looks like gnuflag doesn't even support that kind of thing [18:16] natefinch: so we use a comma for a separator? [18:16] ericsnow: that or semicolon.... I think we use semicolon somewhere else.... [18:18] (although now I can't find where... maybe I'm crazy) [18:19] ericsnow: oh, well, loggo uses semicolons *shrug* [18:24] dimitern: right, but is it used to select the space to bootstrap *to*? [18:24] dimitern: (as well) [18:25] cherylj, ping [18:25] voidspace, no, that's what bootstrap constraints are for [18:26] frobware: in mtg, give me a few [18:26] cherylj, ack [18:26] dimitern: so you could set controller-space and end up with an apiserver not in the controller space? [18:26] voidspace, and in fact if you specify --controller-space admin-api, you'll get a node there, as after validating the controller space we also update the bootstrap constraints to match [18:27] dimitern: ok, so the controller-space *is* turned into a bootstrap constraint? [18:27] dimitern: (that's what I was really asking - but I can't see the code that does that) [18:27] dimitern: because if so we have an issue with the space name [18:27] voidspace, but if you don't specify --controller-space, you'll get arbitrary node, from which we'll then infer contr.space and also set env. constraints [18:28] dimitern: I am specifically asking about the case where you *do* specify it [18:28] voidspace, yeah, the issue is commented in a todo [18:28] dimitern: we'll have an issue with space names [18:28] voidspace, that we need to apply the same transformation as in discoverspaces [18:28] dimitern: the bootstrap contraint should be a maas space name, but we require controller space to be a juju space name [18:28] dimitern: right [18:28] dimitern: that's the point I wanted to make [18:28] voidspace, well, both need to be a valid juju space name [18:28] dimitern: but it's still a todo then? [18:29] dimitern: except you can't "untranslate" [18:29] dimitern: our name transformation is one way only [18:29] dimitern: you can't go from juju name back to maas name [18:29] voidspace, so you can't use "space=Default space" in cons or --controller-space "Default space" as well [18:29] dimitern: we'll have to do discovery on the client I'm afraid and then match the name [18:29] voidspace, you can :) that's why we discover them and keep both id and name [18:30] dimitern: and for the bootstrap constraint the user will have to "guess" what the juju name will be [18:30] dimitern: not before discovery has been done you can't [18:30] voidspace, there are still things to fix, I'll give you that :) [18:30] dimitern: and discovery hasn't been done until after machine-0 has been selected and jujud is runnign [18:30] *running [18:30] voidspace, but it's a necessary first step [18:31] anyway, really should be going now, perhaps back later [18:31] dimitern: why don't we just use provider names in state instead of having restricted juju names requiring a transformation? [18:31] because of binding syntax restrictions I suppose :-( [18:31] and because we have provider ids on maas only - i.e. not the common ground [18:32] dimitern: maas is the only place where we need discovery [18:32] dimitern: so technically it is fully common ground... [18:32] voidspace, I'll think about it tomorrow and I'm happy to listen to ideas how to generalize this approach [18:32] but anyway, this is just a distraction [18:33] life would be easier if we dropped the name restrictions and didn't have to do the transformation though [18:33] (well - easier in places, maybe harder in others) [18:33] yeah - mongo special chars in keys, and bad cli ux for constraints and bindings.. [18:34] there's a better way I'm sure [18:35] sinzui: hey, got past the problem. getting a new error "cannot read product data, invalid URL "https://us-east.manta.jpyemt.com/cpcjoyentsupport/public/juju-dist/tools/streams/v1/com.ubuntu.juju:devel:tools.json"" [18:35] sinzui: what environments.yaml thing do i tweak to make that work? [18:36] sinzui: again he's using 2.0-alpha1 [18:37] katco`: per the annoucenment to the juju and juju-devel lists, users of 2.0-alpha1 in aws, joyent, and azure must set "agent-metadata-url: https://streams.canonical.com/juju/tools" which restore 1.x behaviour [18:40] sinzui: ty trying now [18:40] sinzui: sorry for the verbose questions, in a bar trying to get this to work lol [18:41] katco`: :) that situation is crazy. by manually setting the streams to the default location that juju is already checking, it will find the agents in the public cloud it said it could not find. [18:45] sinzui: ty! that seems to be working [18:52] cherylj: will get coffee, might be a couple of mins late depending on the expreso machine mood [18:52] perrito666: np :) [18:57] back, its so hot outside that the machine didnt need to pre-heat === jog_ is now known as jog [19:33] cherylj, just wanted to say that dimiter has committed changes to his dev branch only. the tests that he mentioned in the meeting still need fixing but are simply commented out and skipped for now. Is it possible to get a CI run on his branch so we can look at the net result in the morning? [19:34] cherylj, PR -> https://github.com/juju/juju/pull/4255 [19:59] Bug #1540469 changed: [ARM64][LXD Provider][2.0-alpha1-0ubuntu1~16.04.1~juju1] juju run command ERROR fork/exec /usr/bin/ssh: cannot allocate memory [20:05] Bug #1540469 opened: [ARM64][LXD Provider][2.0-alpha1-0ubuntu1~16.04.1~juju1] juju run command ERROR fork/exec /usr/bin/ssh: cannot allocate memory === akhavr1 is now known as akhavr [20:07] anyone online who knows how deploying bundles works with the juju client? [20:07] natefinch: dont you need deployer for bundles? [20:07] perrito666: it's been added to the client [20:08] Bug #1540469 changed: [ARM64][LXD Provider][2.0-alpha1-0ubuntu1~16.04.1~juju1] juju run command ERROR fork/exec /usr/bin/ssh: cannot allocate memory [20:09] it looks like we just blindly throw away all CLI flags if you target a bundle [20:10] * perrito666 is suddenly attacked in linkedin by solo recruiters offering the dream job, I wonder what is the algo that triggers those [20:10] heh [20:20] natefinch - are you referring to the new "juju deploy" or python juju client? [20:22] cherylj: did you want to talk through those few bugs? [20:23] o/ lazypower|summit [20:23] \o heyo thumper [20:24] thumper: give me a minute, I'm chatting with frobware [20:24] k [20:24]