[00:25] I hate something in my network but I am not sure what [00:25] your computer? [00:25] I had mine [00:26] It might be my isp, or perhaps my wlan card, but I am not sure which of those is the one I hate today [00:38] Hi wwitzel3 ericsnow: Can we get this reviewed and merged by tommorrow so that we can propose 1.25-beta1 for wily https://github.com/juju/charm/pull/157 [00:38] * sinzui not know the policy for reviews and merges in the charm project [01:56] Bug #1500676 opened: use-default-secgroup in environments.yaml not respected [05:11] wallyworld: FYI - https://bugs.launchpad.net/juju-core/+bug/1500703 [05:11] Bug #1500703: provider/gce: DeviceNames populated in volume attachment info are invalid [05:12] ok [05:12] wallyworld: gce volumes don't work properly [05:12] wallyworld: working on a fix now [05:12] sounds good, ty [05:12] hopefully you'll get access to that maas setup soon too [05:14] Bug #1500703 opened: provider/gce: DeviceNames populated in volume attachment info are invalid [05:17] Bug #1500703 changed: provider/gce: DeviceNames populated in volume attachment info are invalid [05:26] Bug #1500703 opened: provider/gce: DeviceNames populated in volume attachment info are invalid [05:29] Bug #1500703 changed: provider/gce: DeviceNames populated in volume attachment info are invalid [05:32] Bug #1500703 opened: provider/gce: DeviceNames populated in volume attachment info are invalid [07:06] Bug #1500721 opened: provider/ec2: volumes should be attached with name "xvdf" instead of "xvdf1" by default [07:43] dimitern: oh great OCR of the day! :-) would you be able to do a review for me, by any chance please? http://reviews.vapour.ws/r/2774/ [07:50] rogpeppe, sure thing :) looking [07:50] dimitern: ta muchly! :) [08:02] dimitern: if you get a chance... http://reviews.vapour.ws/r/2775/ [08:02] frobware, sorry, omw ~2m [08:04] dimitern, ok [08:07] dimitern: actually, extended the test and found a bug... fixing [08:15] dimitern: and done... [08:27] wallyworld: do you have a moment to review http://reviews.vapour.ws/r/2776/? [08:27] axw: sure, about to eat,so will pick it up after if i don't get it done [08:27] wallyworld: thanks [08:30] axw: where does "scsi-0Google_PersistentDisk" come from? [08:30] is that something we made up? [08:50] voidspace, sweet! :) [09:00] Bug #1500760 opened: juju space/subnet subcommands need to respect -e env-name flag [09:02] voidspace, TheMue, standup? [09:02] jam, fwereade ^^ [09:02] dimitern: any chance of that review? :) [09:04] rogpeppe, in standup now, will reply in ~20m [09:04] dimitern: ta [09:04] dimitern: omw [09:15] rogpeppe, LGTM [09:15] dimitern: tvm [09:25] voidspace, reviewed as well [09:29] frobware, how's going so far with the bootstrapping etc? [09:29] dimitern, tis bootstrapped [09:29] dimitern, want to go through the motions over HO? [09:31] frobware, ok - joining the standup one [09:31] dimitern, 10 mins - ok? [09:31] frobware, np [09:33] Bug #1500769 opened: provder/gce: default block source not set [09:33] dooferlad, just a reminder to add cards for the -e flag issue [09:33] :) [09:34] sure [09:44] dimitern, I'm back... [09:50] frobware, omw [10:06] wallyworld: good news, I got Ceph working on GCE with storage. [10:06] farking awesome [10:06] what sort of disks? [10:07] wallyworld: SSDs [10:07] much to change in the charm? [10:07] wallyworld: the non-local kind [10:07] btw, did you see my question above? [10:07] wallyworld: not a lot [10:07] wallyworld: no sorry, reading [10:08] yeah, adding storage should not be a big deal [10:08] wallyworld: it's not made up, that's how what the /dev/disk/by-id path looks like. it's not *documented* though, so it's a bit sketchy. hence the TODO, which involves changing mongo and apiserver, etc. [10:09] np, thanks, i didn't quite fully grok it [10:09] wallyworld: there's two paths in /dev/disk/by-id: that one, and google- [10:09] ah ok [10:09] wallyworld: the diskmanager publishes the undocumented one :) [10:09] :-) [10:10] I better log a bug for that I think [10:10] axw: should we fix that now lest we have mmigration issue later [10:10] before 1.25 ships [10:10] wallyworld: I'll try to do that next [10:11] wallyworld: there's another bug I was about to fix, which is that the gce provider doesn't set a default volume source [10:11] wallyworld: I can leave that for now and fix this [10:11] i think that would be wise, just imo [10:11] wallyworld: I'd prefer to get the quick fix in first though, if you don't mind [10:12] sure, +1 [10:12] in case I run out of time [10:12] done [10:12] wallyworld: thanks [10:12] frobware: so, back from the doc, brain is - at least as much as possible ;) - ok, now continuing on the "online" docs (part of the cmd). want to propose it today [10:29] TheMue, sounds good on all fronts... :) [10:32] frobware: yeah, absolutely. and then I can start with the user docs, already got needed infos by nick [10:39] Bug #1500803 opened: provider/gce: HardwareId should use the documented "google-" format [11:04] dooferlad, do you run spaces tests on CI right now? [11:04] frobware: we don't have any yet, so no [11:05] frobware: that is what I am fixing :-) [11:05] frobware: there are unit tests [11:05] frobware: and those do run [11:05] dooferlad, ok, that's good -- if you had said "yes" then dimitern and I might have been bemused [11:06] dooferlad, because it's not clear at the moment which access-key/secret-key combo allows spaces to work [11:10] frobware, dimitern: http://pastebin.ubuntu.com/12610778/ [11:10] frobware, dimitern: http://pastebin.ubuntu.com/12610786/ [11:10] dooferlad, interesting. that's what I was trying to do. [11:11] dooferlad, as in my test charm for spaces was to use ubuntu [11:11] frobware: well, I have it automated now. Will check in and you can use it. [11:12] dooferlad, if you could bung me some instructions that would be appreciated [11:12] dooferlad, you need to use a different service name if you're deploying the same charm twice [11:12] dooferlad, i.e. juju deploy cs:trusty/ubuntu ubuntu1 --constraints spaces=dmz ; # or maybe even just: juju deploy ubuntu ubuntu2 --constraints... [11:12] dimitern: so http://reviews.vapour.ws/r/2700/ isn't quite the perfect instructions then :p [11:13] dimitern: will try that... [11:13] dooferlad, in the steps there I'm using 1 deploy followed by 1 add-unit [11:13] dimitern: ok, ok, problem between keyboard and chair [11:14] :) [11:19] rogpeppe: urulama: whenever you get time, here's a new charmrepo.v2 PR which adds a repo which sits on top of a given charm dir https://github.com/juju/charmrepo/pull/32 [11:19] wallyworld: kk, sometime during the day, will be ready before your morning [11:20] wallyworld: thanks, will look [11:20] tyvm [11:20] wallyworld: hah, not a small one :P [11:20] nah sorry [11:21] rogpeppe: urulama: it's set up to allow core to juju deploy with a dir path rather than a cs:foo or local:bar url [11:24] wallyworld: i wonder if it's right that the caller should have no control as to which kind of repository is used [11:25] wallyworld: for example if you're a server, you almost certainly don't want to be statting local paths [11:25] wallyworld: when resolving URLs [11:27] rogpeppe: i don't quite follow, if you use the Infer method, the first checks are for cs: or local:. Only if neither of those does it assume a path [11:27] this is used on the client when deploy or upgrade [11:29] wallyworld: hmm, doesn't ParseReference always return a schema of "cs" if none is specified [11:29] ? [11:30] wallyworld: so if you do "juju deploy wordpress" and there's a local wordpress directory that you want to deploy, i think it'll try to deploy cs:wordpress anyway, won't it? [11:30] wallyworld: is that the behaviour you want? [11:30] rogpeppe: ah right so with paths with only single words (not foo/bar) it will assume cs: [11:30] i'll have to change that [11:30] try for path first [11:30] i had it that way and then changed it [11:30] dimitern: can you view the spaces and subnets in juju status? [11:32] dooferlad, no [11:32] dimitern: bother [11:41] dooferlad, use space list for the time being - we'll add it to status at some point [11:47] wallyworld: have you got a few moments for a chat? [11:48] rogpeppe: sure, can i ping you in say 10 minutes? [11:48] wallyworld: sure [11:48] ty [11:49] wallyworld, rogpeppe: do talk about the "supported-series" vs "series" as well, otherwise, we can do it tomorrow [11:49] oh? i changed it to supported-series [11:52] wallyworld: eco guys don't like supported-series unfortunately [11:52] oh damn [11:53] i'll have to add back that parsing stuff [11:54] wallyworld: i've suggested an alternative [11:54] in te doc? [11:56] wallyworld: in the bug on juju/charm that was opened [11:56] wallyworld: see https://github.com/juju/charm/issues/156 [11:56] oh, ok, i don't normally follow gh bugs [11:56] wallyworld: :P [11:58] lp forever [11:59] rogpeppe: free now? https://plus.google.com/hangouts/_/canonical.com/tanzanite-stand [12:09] * fwereade had very little sleep last night and is going to have a nap before his meeting in 2h [12:17] frobware, I found the problem [12:19] Bug #1500843 opened: Windows ftb due to unused import is diskmanager [12:19] frobware, while I *still* don't quite get why it's working fine when using the root keys, the reason we're getting this rate limit error [12:20] frobware, as we saw in the logs is "InvalidGroup.NotFound" for the SG juju created for the machine [12:21] frobware, which is a "classic" SG - not attached to the VPC explicitly but [12:22] Bug #1500843 changed: Windows ftb due to unused import is diskmanager [12:22] frobware, it might be easier if we get on a HO actually :) [12:24] ocr: ^ want to review my branch? [12:25] Bug #1500843 opened: Windows ftb due to unused import is diskmanager [12:31] dimitern: so I added the missing test and extracted the functions as you suggested [12:32] dimitern: switching one of the machines to have JobManageEnviron in the MachineTemplate caused AddMachines to return an empty slice of machines [12:32] dimitern: do you think it's worth me digging into that to work out why? [12:33] dimitern, yep, HO's easier... :) [12:37] dimitern: ah, found it - "state server jobs specified but not allowed" [12:38] hmmm... [12:40] frobware, joining standup HO? [12:40] dimitern, ok [12:49] rogpeppe: here's that charm change https://github.com/juju/charm/pull/158 [12:50] wallyworld: i think he's out on lunch [12:50] urulama: np, i'm about to go to sleep, so if it's +1, could you shipit for me [12:53] wallyworld: will do [12:53] ty [13:43] dimitern, coming back... (wrong tab) [14:40] Bug #1246156 changed: plugins should receive environment to operate in unambigiously [14:43] Bug #1246156 opened: plugins should receive environment to operate in unambigiously [14:46] Bug #1246156 changed: plugins should receive environment to operate in unambigiously [14:51] dimitern: ping [14:55] voidspace, pong [14:56] dimitern: upgrade step added to 1.25 [14:56] dimitern: does it need adding to 1.26 (master) as well (i.e. to steps126) [14:57] dimitern: that would only be needed if someone upgraded from a 1.25 beta, which doesn't have the preferred field, to 1.26 [14:57] without going to 1.25 final first (which would do the upgrade for them) [14:57] dimitern: I don't think that's supported, so I don't think it needs to be in steps126 [14:57] dimitern: but would like confirmation [14:58] to be clear, it is (or will) be in steps124 and steps125 on master - I don't think it needs to be in steps126 as well [14:59] voidspace, hmm.. let me think for a moment [15:00] voidspace, I think since 1.25 is not release yet, we *shouldn't* need to add it to 1.26 steps [15:01] dimitern: that's my understanding [15:01] dimitern: they should upgrade to 1.25 "some released version" first [15:01] which would do the upgrade [15:01] then go to 1.26 [15:04] voidspace, yeah - as far as users are concerned - it might screw us up a bit when testing upgrades etc. with upload-tools [15:07] dimitern: heh, if it does it's a few lines copy & paste to fix [15:08] dimitern: it's merged on 1.24, in the queue for 1.25 and master [15:10] voidspace, awesome! [15:22] modular [15:22] lol wrong window [15:23] how to kill a colleague without being found out [15:23] oops, wrong window... [15:23] rofl [15:23] ;-) [15:28] dimitern: heh, landed on 1.25 - master is blocked [15:28] voidspace, 1.25 is the important one anyway for us now :) [15:29] dimitern: yep, I've marked the bug as fix committed [15:29] dimitern: the blocker on master is trivial, so should be fixed soon anyway [15:30] voidspace, sweet! [15:44] dimitern: hah, so the ec2 provider implementation of NetworkInterfaces already fetches all associated subnets [15:44] dimitern: the only thing needed for SubnetInfo that it doesn't keep is the availability zone information [15:45] dimitern: so easy enough to add (somehow....) [15:45] dimitern: looking at whether extending network.InterfaceInfo to keep this info is sensible [15:48] frobware, *facepalm* .. and I was wondering why it's not working! [15:48] voidspace, I'm not sure adding AZ to InterfaceInfo is worth the effort [15:48] frobware, us-east-1 does not have a default VPC [15:49] dimitern: but I *need* it for the Subnets call [15:49] dimitern: if I add it to InterfaceInfo, then when Subnets(instId) is called I can just call NetworkInterfaces(instid) and build the SubnetInfo from that [15:50] dimitern: otherwise I call NetworkInterface(instId) (which internally calls the ec2 describe subnets for each subnet) - and then I have to call describe subnet *again* for each nic to get the full SubnetInfo [15:50] or I can add []AZ to InterfaceInfo [15:51] frobware, what a way to waste 2h.. [15:51] voidspace, I see [15:51] dimitern: or I can find another way, which might be better than extending InterfaceInfo [15:51] a new method which is used by both NetworkInterfaces and Subnets [15:51] voidspace, ok then - I see no harm in adding AvailabilityZone to InterfaceInfo [15:52] it had better be a slice to support other providers where a subnet can span availability zones [15:55] voidspace, fair point about using a slice - I'll leave it to your judgment then [15:59] dimitern, so what's the redux here? [16:06] frobware, look carefully before spending hours debugging :) [16:06] frobware, eu-central-1 one of the few (if not the only one) regions on the shared aws account with a default VPC [16:07] frobware, I've added a few subnets to it, backed out my changes and I'm trying to recreate the demo there, will paste you some steps shortly [16:07] dimitern, ok. but I guess a better error message is in order for the places/cases where this currently happens. [16:08] frobware, re the redux - default VPC is required for all this MVP functionality to work [16:09] frobware, yes, definitely - e.g. error out if spaces cannot be supported (due to a missing default VPC for example at least for now) [16:09] dimitern: which region do we have default vpc in? [16:09] voidspace, eu-central-1 [16:09] cool, thanks [16:09] dimitern: doing some manual testing [16:09] about to do addressable containers (1.25) [16:11] ooo addressable containers \o/ [16:11] heh [16:12] "ERROR environment destruction failed: destroying storage: destroying volumes: destroying "vol-fa3c3117": The volume 'vol-fa3c3117' does not exist. (InvalidVolume.NotFound)" [16:12] that's a new one on me. I didn't create any volumes. [16:15] voidspace, go in the EC2 web console and check the Volumes page [16:15] dimitern: presumably someone else's volume... [16:15] which is worrying that my call to destroy-environment would touch it... [16:16] and IT WORKS! [16:16] yay [16:16] whatever IT is... [16:16] spaces hopefully... [16:18] voidspace, frobware: it's still deploying some instances, but they all end up where they should and AZ distribution also works: http://paste.ubuntu.com/12613764/ [16:19] dimitern: awesome, nice work [16:19] Bug #1464679 changed: juju status oneline format missing info [16:19] voidspace, frobware, here are the steps: http://paste.ubuntu.com/12613782/ [16:20] dimitern, presumably I should be able to replicate this with my local aws account as it would have been created with a default VPC [16:20] frobware, indeed - that's why it worked with my personal account but not with the shared on us-east-1 [16:22] frobware, and the mystery around root/non-root creds about the shared account - no mystery at all :) I actually tested with my account it turned out (I did set the shared acc keys but *then* overrode them in env. yaml) [16:24] dimitern, if you don't have a default VPC (e..g, us-east-1) is there a way to specify one as part of the setup/demo? [16:24] frobware, not yet [16:25] frobware, because the issues we were facing during the HO [16:26] (with a non-default VPC the behavior is "as prescribed" in the AWS API docs, no "EC2-Classic"-sorta compatible quirks - e.g. groupName vs groupId) [16:26] dimitern, what happens if you don't do this step: for i in subnet-0fb97566 subnet-d27d91a9; do juju subnet add $i default; done [16:29] frobware, nothing - I just wanted to test the 3 cases: using default VPC subnets, non-default subnets with and without auto-public-IP set, [16:29] just for completeness sake [16:29] dimitern, OK. Just wanted to check (and understand) that you could do it with just public/dmz. [16:30] frobware, yeah - surprisingly for me it works even in the last case (no public IPs) .. somewhat [16:31] dimitern, so next steps are to try some charms... :) [16:31] frobware, the machine is up and can talk to the api server, you can even ssh into it (as we're jump-hosting via the bootstrap node), but it can't access the archive (dns works, but not connecting) [16:31] frobware, yep! :) [16:35] frobware, I'll leave the env running for tonight, checking the connectivity periodically with a watch script, just to make sure it works longer than 5m :) [16:35] and then it's time to get a drink \o/ [16:35] dimitern, yep, nice one! === ericsnow is now known as ericsnow_afk [17:26] natefinch: wwitzel3: ericsnow_afk: are all of you available for a chat [17:26] ? === alexisb is now known as alexisb-afk [17:46] Bug #1500981 opened: juju-db segfault while syncing with replicas === ericsnow_afk is now known as ericsnow [17:49] katco: sure [17:51] natefinch: wwitzel3: ericsnow: need the whole team for this one [17:52] ericsnow: i'm in moonstone now if you have a sec though === alexisb-afk is now known as alexisb [18:16] Bug #1500996 opened: Juju says "bad JSON product data" for valid simplestreams === Makyo is now known as Guest96377 [19:01] is there any charm for vivid? [19:40] perrito666: https://jujucharms.com/q/vivid [19:53] natefinch: ping [20:16] rick_h_: thanks a lot [20:17] perrito666: any time [20:17] what is the middle column supposed to be, series? [20:17] perrito666: yes [20:18] more interesting in a search for not a specific series: https://jujucharms.com/q/apache2 [20:19] rick_h_: I tried the dropdown filter but only got the ubuntu charm [20:19] perrito666: yea, we try not to promote the non-LTS too much [20:19] natefinch: ping [20:20] perrito666: so if you ask for it it'll give it to you, but deploying workloads is best done on LTS [20:20] rick_h_: I used the filter to get the url parameter and then changed for vivid [20:20] perrito666: ah, yea it probably had some other params there by default vs a normal search. [20:21] perrito666: since by default in the /store it shows only promulgated (reviewed charms) and that is the only one [20:21] heh, yup, the strange thing is that it returned only one charm, but it was useful for what I was trying [20:21] perrito666: anyway, a handful from there [20:50] natefinch: ping [21:11] wwitzel3: bug 1491688 - where did you get up to? this issue seems familair - did we fix it one other time? [21:11] Bug #1491688: all-machine logging stopped, x509: certificate signed by unknown authority [21:18] wallyworld: yeah, we fixed it a previous time, axw did, but it looks like it might have not made it in to other versions, I got to replication point, but didn't ever start really digging on it [21:19] * thumper looks around carefully [21:19] anything blowing up? [21:19] wwitzel3: ty, i'll follow up with him [21:19] thumper, yes [21:19] we have two criticals: [21:19] :-( [21:19] * thumper sighs [21:20] thumper, you know you love bug squad duties [21:20] except I was poking lxd with a stick [21:20] among other things [21:21] thumper: you can debug my ppc64le if you'd rather [21:21] alexisb: where are these criticals? [21:22] thumper, lxd is a happening in moonstone hangout if you are interested [21:22] yeah [21:22] I am sure wwitzel3 wouldnt mind if you joined the party [21:22] I have some info re:lxd [21:22] criticals: [21:23] bug 1491688 [21:23] Bug #1491688: all-machine logging stopped, x509: certificate signed by unknown authority [21:23] thumper: fwiw ^^^^^ i thought that one was fixed [21:23] which ironically is only high [21:23] https://bugs.launchpad.net/juju-core/+bug/1335885 [21:23] Bug #1335885: destroy-environment reports WARNING cannot delete security group [21:24] thumper, it is happening on cisco site [21:24] wallyworld: I thought it was fixed too [21:24] thumper: andrew fixed it apparently, i'll check with him in an hour or so [21:25] I wonder if it is client version vs. fixed version [21:25] perhaps the fixed version hasn't made its way to the client site [21:25] could be [21:25] thumper, that second one is now causing issues for openstack and sts [21:25] m'kay [21:28] wwitzel3: are you talking lxd now? [21:33] thumper: wwitzel3: bug 1417875 [21:33] Bug #1417875: ERROR juju.worker runner.go:219 exited "rsyslog": x509: certificate signed by unknown authority Released by wwitzel3> [21:34] haven't read description yet, seems related [21:34] fixed in 1.22 [21:36] thumper: wwitzel3: and then this one which seems more likely, fixed in 1.24 bug 1474614 [21:36] Bug #1474614: rsyslog connections fail with certificate verification errors after upgrade to 1.24.2 [21:36] so maybe above fix needs to be checked [21:45] wallyworld: arse [21:46] balls? [21:47] thumper: ? [21:47] wallyworld: the fixed but not fixed [21:48] yeah, but could be different root cause [21:48] not sure, haven't dug [22:08] menn0: tyvm for review!!! [22:10] anastasiamac: no problemo [22:11] Bug #1501084 opened: hitting nova RAM quota results in over-general error [22:32] Bug #1500676 changed: use-default-secgroup in environments.yaml not respected [22:56] Bug #1500996 changed: Juju says "bad JSON product data" for valid simplestreams [22:56] Bug #1501093 opened: unclear simplestreams error [23:30] thumper: you have some lxd info? [23:34] hi wallyworld, thumper - thoughts on https://launchpad.net/bugs/1335885 ? i'm hitting it with high frequency, bootstraps are failing in test automation. [23:34] Bug #1335885: destroy-environment reports WARNING cannot delete security group [23:35] ps, added summary comment [23:37] 1.24.5 --> 1.24.6 seems to have exacerbated it