| davecheney | da fuq ? | 00:01 |
|---|---|---|
| thumper | davecheney: yeah, this seems to be the fundamental problem behind the lxc containers not upgrading | 00:06 |
| * thumper is still digging | 00:06 | |
| * thumper tries to igrore work for a bit and go to lunch | 00:26 | |
| === natefinch-afk is now known as natefinch | ||
| menn0 | thumper: based on circumstantial evidence only it looks like the a stuck lease/leadership worker is behind bug 1466565 | 01:10 |
| mup | Bug #1466565: Upgraded juju to 1.24 dies shortly after starting <cts> <landscape> <sts> <upgrade-juju> <juju-core:Triaged by menno.smits> <juju-core 1.24:Triaged by menno.smits> <https://launchpad.net/bugs/1466565> | 01:10 |
| menn0 | thumper: based on a log message indicating that a watcher fired due to a change in the leases collection long after just about everything else was dead | 01:11 |
| * menn0 goes to try a quick repro | 01:11 | |
| davecheney | thumper: http://paste.ubuntu.com/11776424/ | 01:18 |
| davecheney | 5 races, including the obscure apiserver one | 01:18 |
| davecheney | that we talked about in the standup | 01:18 |
| natefinch | dave, always playing the race card | 01:20 |
| natefinch | ericsnow: you around? | 01:27 |
| === kadams54 is now known as kadams54-away | ||
| davecheney | thumper: who maintains gomass api ? | 01:49 |
| davecheney | https://bugs.launchpad.net/juju-core/+bug/1468972 | 01:49 |
| mup | Bug #1468972: provider/maas: race in launchpad.net/gomaasapi <juju-core:New> <https://launchpad.net/bugs/1468972> | 01:49 |
| mup | Bug #1468972 opened: provider/maas: race in launchpad.net/gomaasapi <juju-core:New> <https://launchpad.net/bugs/1468972> | 01:51 |
| menn0 | thumper: bingo... able to repro | 01:52 |
| menn0 | wallyworld: we're never doing another 1.23 release again are we? | 01:54 |
| wallyworld | no | 01:55 |
| wallyworld | that's the plan | 01:55 |
| menn0 | cool | 01:55 |
| menn0 | wallyworld: the reason I ask is that I'm looking at a problem upgrading out of a 1.23 env which seems fairly easy to hit (almost certainly due to the lease/leadership workers not exiting) | 01:56 |
| wallyworld | hmmm | 01:57 |
| menn0 | wallyworld: adam c has hit it and I can repro it pretty easily | 01:57 |
| menn0 | wallyworld: seems like anyone who ended up on 1.23 could have trouble getting off it | 01:57 |
| wallyworld | i guess we could do another release then | 01:57 |
| menn0 | wallyworld: that wouldn't help | 01:57 |
| wallyworld | or have to | 01:57 |
| === kadams54-away is now known as kadams54 | ||
| menn0 | wallyworld: they wouldn't be able to upgrade to that either | 01:57 |
| wallyworld | ah yeah | 01:58 |
| menn0 | wallyworld: the issue is prevent the agent from exiting to restart into the new version | 01:58 |
| wallyworld | is there a workaround we can document? | 01:58 |
| menn0 | wallyworld: it should be possible to work around it by manually setting the symlink | 01:58 |
| axw | menn0: I *think* killing the jujud process would fix it | 01:58 |
| wallyworld | that will have to be what we do then i guess | 01:58 |
| axw | it just deadlocks when shutting down | 01:59 |
| axw | if it's the bug I fixed | 01:59 |
| menn0 | axw: no that doesn't help because the symlink gets changed as one of the very last things that jujud does b4 it exists | 01:59 |
| axw | ah right | 01:59 |
| menn0 | axw: and b/c some workers aren't finishing it's not getting to that | 01:59 |
| * axw nods | 01:59 | |
| axw | menn0: btw, reviewed your branches. sorry for not doing so yesterday | 02:00 |
| menn0 | adam gets a minute or so of working Juju before it wants to restart and then gets stuck | 02:00 |
| menn0 | axw: thanks. no worries. | 02:00 |
| menn0 | axw: good catches for both of the problems you noticed | 02:02 |
| === anthonyf is now known as Guest78303 | ||
| thumper | davecheney: technically we maintain gomass api | 02:12 |
| thumper | menn0: which repro are you talking about? | 02:13 |
| davecheney | launchpad.net/gomaasappi | 02:13 |
| === kadams54 is now known as kadams54-away | ||
| === kadams54-away is now known as kadams54 | ||
| === kadams54 is now known as kadams54-away | ||
| menn0 | thumper: bug 1466565 | 02:44 |
| mup | Bug #1466565: Upgraded juju to 1.24 dies shortly after starting <cts> <landscape> <sts> <upgrade-juju> <juju-core:Triaged by menno.smits> <juju-core 1.24:Triaged by menno.smits> <https://launchpad.net/bugs/1466565> | 02:44 |
| thumper | menn0: yes? | 02:45 |
| menn0 | thumper: this is pretty serious actually... anyone who upgraded to 1.23 is likely to have a hard time getting off it | 02:45 |
| menn0 | thumper: see the ticket for trivial repro steps | 02:45 |
| menn0 | thumper: manual steps are required to upgrade | 02:45 |
| menn0 | thumper: the culprit appears to be the lease worker not honouring kill requests | 02:46 |
| * thumper nods | 02:46 | |
| axw | wallyworld: I'm playing around with the Azure portal, which looks like it's using the new model... and putting machines in the same AS still forces them to the same domain-name/IP | 02:58 |
| axw | :( | 02:58 |
| wallyworld | oh :-( | 02:58 |
| wallyworld | can you email the ms guys we have been talking to and ask about it? | 02:58 |
| axw | wallyworld: ok | 03:00 |
| wallyworld | ty, may not be the answer we want but at least they may be able to explain why etc | 03:00 |
| mup | Bug #1466565 changed: Upgraded juju to 1.24 dies shortly after starting <cts> <landscape> <sts> <upgrade-juju> <juju-core:Triaged by menno.smits> <juju-core 1.24:Triaged by menno.smits> <https://launchpad.net/bugs/1466565> | 03:09 |
| wallyworld | axw: there's a blue card in the Next lane - binding volumes/filesystems. That one has actually been done as part of the volume deletion work | 03:13 |
| axw | wallyworld: yes, apart from UI to change binding | 03:14 |
| axw | wallyworld: so I'll change it to just the missing bits | 03:14 |
| wallyworld | axw: so i reckon we should add an unplanned card worth 5 or 8 for the work done | 03:14 |
| axw | wallyworld: it was part of the persistent volume deletion | 03:15 |
| axw | which was just woefully underestimated | 03:15 |
| wallyworld | yep, i under estimated the resources card too :-( | 03:15 |
| wallyworld | axw: also, if/when you get a chance ptal at the resources pr again :-) | 03:15 |
| axw | wallyworld: sure, just writing this email to guy | 03:16 |
| wallyworld | np | 03:16 |
| mup | Bug #1466565 opened: Upgraded juju to 1.24 dies shortly after starting <cts> <landscape> <sts> <upgrade-juju> <juju-core 1.23:Won't Fix by menno.smits> <juju-core 1.24:Invalid by menno.smits> <https://launchpad.net/bugs/1466565> | 03:21 |
| axw | wallyworld: sorry, dunno why I thought you were storing the URL now. I think I saw the params struct and thought that's what you were storing in state | 03:27 |
| wallyworld | np | 03:28 |
| axw | wallyworld: LGTM | 03:31 |
| wallyworld | yay, ty | 03:31 |
| mup | Bug #1466565 changed: Upgraded juju to 1.24 dies shortly after starting <cts> <landscape> <sts> <upgrade-juju> <juju-core 1.23:Won't Fix by menno.smits> <juju-core 1.24:Invalid by menno.smits> <https://launchpad.net/bugs/1466565> | 03:33 |
| menn0 | omg so much fail | 04:00 |
| menn0 | you pull a string and broken stuff appears everywhere | 04:01 |
| thumper | wallyworld, axw: can you join a hangout plxz? | 04:12 |
| thumper | https://plus.google.com/hangouts/_/canonical.com/onyx-standup | 04:12 |
| axw | thumper: omw | 04:13 |
| axw | thumper: are you in? just says "trying to join the call" | 04:15 |
| thumper | axw: I had that earlier today too... | 04:15 |
| * thumper tries a direct invite | 04:15 | |
| === kadams54 is now known as kadams54-away | ||
| thumper | axw: when did this commit land BTW? | 04:28 |
| axw | thumper: 1.24 | 04:28 |
| thumper | I'm wondering if we should pull 1.24.1 | 04:29 |
| thumper | because this problem will stop any non-state server upgrading I think | 04:29 |
| axw | thumper: probably not a bad idea. how come this got through CI? is it only affecting things that don't support KVM? | 04:29 |
| thumper | no idea | 04:30 |
| thumper | maybe... | 04:30 |
| thumper | there is an open issue though about CI around upgrades | 04:30 |
| thumper | as we have found so many upgrade problems | 04:30 |
| thumper | which CI didn't catch | 04:30 |
| axw | thumper: got the OK from OIL too I think, though not sure if they do upgrade or clean install | 04:30 |
| thumper | I assigned you to the wrong bug | 04:31 |
| thumper | hang on | 04:31 |
| axw | thumper: ta | 04:32 |
| thumper | bug 1466969 | 04:36 |
| mup | Bug #1466969: Upgrading 1.20.14 -> 1.24.0 fails <canonical-bootstack> <upgrade-juju> <juju-core:Triaged> <juju-core 1.24:Triaged by axwalk> <https://launchpad.net/bugs/1466969> | 04:36 |
| axw | hurngh, can't test because I have vivid | 04:36 |
| axw | should fail from 1.23 I guess | 04:36 |
| thumper | 1.23 is terrible | 04:36 |
| thumper | you can't upgrade from 1.23 due to lease / leadership issues | 04:36 |
| thumper | try 1.22 or 1.20 | 04:37 |
| thumper | I have some 1.20.14 binaries if you want them :) | 04:37 |
| axw | thumper: I can build them, juju 1.20 doesn't work on vivid | 04:37 |
| axw | no systemd | 04:37 |
| thumper | ugh | 04:37 |
| thumper | geez | 04:38 |
| axw | never mind, I'll work something out | 04:38 |
| thumper | axw: you could reproduce in ec2 | 04:38 |
| axw | yep. I think I have a VM anyway | 04:38 |
| thumper | axw: by deploying ubuntu into a container | 04:38 |
| thumper | ok | 04:38 |
| thumper | axw: was this for 1.24.1 or 1.24.0? | 04:39 |
| thumper | axw: because there is another bug about failing to upgrade from 1.24.0 to 1.24.1 | 04:39 |
| axw | thumper: pretty sure 1.24, I'll double check | 04:39 |
| axw | .0 I mean | 04:40 |
| * thumper wouldn't be surprised if it is a different bug | 04:40 | |
| thumper | so many bugs | 04:40 |
| thumper | :-( | 04:40 |
| axw | thumper: yep, 1.24.0 | 04:41 |
| thumper | ok... so this other upgrade problem is something else | 04:41 |
| * thumper takes a deep breath | 04:41 | |
| axw | thumper: how do I work around this syslog upgrade issue? | 04:42 |
| axw | upgrade to 1.24.2.1 failed (will retry): move syslog config from LogDir to DataDir: error(s) while moving old syslog config files: invalid argument | 04:42 |
| thumper | ha | 04:42 |
| thumper | I build from the 1.24.1 tag | 04:42 |
| axw | I see, that was only broken in 1.24.2 ? | 04:43 |
| thumper | or mkdir /etc/juju-<namespace> | 04:43 |
| thumper | yep | 04:43 |
| thumper | it is the commit after updating the version to 1.24.2 | 04:43 |
| axw | okey dokey | 04:43 |
| axw | I'll try that | 04:43 |
| mup | Bug #1468994 opened: Multi-env unsafe leadership documents written to settings collection <juju-core:Triaged by menno.smits> <juju-core 1.24:In Progress by menno.smits> <https://launchpad.net/bugs/1468994> | 05:06 |
| menn0 | thumper: digging into the leadership settings issue... the _id field was being prefix correctly | 05:17 |
| menn0 | thumper: but the env-uuid field wasn't being added | 05:17 |
| menn0 | thumper: so there's no cross-env leakage issues, but the upgrade step definitely gets confused | 05:18 |
| * menn0 updates ticket | 05:18 | |
| menn0 | axw: can I get a quick review of http://reviews.vapour.ws/r/2036/ please | 05:28 |
| menn0 | it's a one-liner :) | 05:28 |
| axw | menn0: sure | 05:28 |
| axw | menn0: is there a minimal test you can add for it? or is that coming later? | 05:29 |
| menn0 | axw: i'll have a look... i didn't have to change any tests when making this change | 05:30 |
| axw | menn0: right, but we had missing test coverage right? | 05:30 |
| axw | menn0: maybe not worthwhile. I'll LGTM and leave it to your discretion | 05:31 |
| menn0 | axw: thinking about it, a test at this layer doensnt make sense since it's actually the responsibility of a lower level to add the env-uuid field | 05:32 |
| axw | menn0: fair enough | 05:32 |
| menn0 | axw: the fact that the lower layer didn't blow up when given a doc like this will be fixed in a later PR | 05:32 |
| menn0 | axw: and tested there | 05:33 |
| axw | menn0: SGTM | 05:33 |
| axw | shipit | 05:33 |
| menn0 | axw: cheers | 05:33 |
| menn0 | thumper: https://github.com/juju/juju/pull/2662 and https://github.com/juju/juju/pull/2661 are merging now. they're the minimum fixes for the leaderships settings doc env-uuid issue for 1.24 and master. More to come to avoid this kind of thing in the future of course. | 05:36 |
| axw | thumper: seems there's another problem too :/ 2015-06-26 06:22:31 ERROR juju.worker runner.go:218 exited "api": login for "machine-1" blocked because upgrade in progress | 06:25 |
| axw | thumper: (machine-1 hasn't upgraded yet) | 06:26 |
| dimitern | voidspace, dooferlad, hey guys, since you're on call reviewers today, along with fwereade, please review any non-reviewed PRs with priority | 08:17 |
| fwereade | dimitern, am so doing :) | 08:17 |
| dimitern | fwereade, cheers :) | 08:24 |
| dooferlad | dimitern: on it. | 08:30 |
| dimitern | dooferlad, ta! | 08:30 |
| voidspace | cool | 08:30 |
| dooferlad | dimitern: the other topic for the day seems to be bootstack related. Should we sync up with Peter now | 08:30 |
| dooferlad | ? | 08:30 |
| dimitern | dooferlad, I'm talking to him in #juju @c | 08:31 |
| dooferlad | dimitern: ah, I was expecting on a different channel. | 08:31 |
| dimitern | dooferlad, standup? | 09:02 |
| mup | Bug #1469077 opened: Leadership claims, document larger than capped size <landscape> <leadership> <juju-core:New> <https://launchpad.net/bugs/1469077> | 09:58 |
| Syed_A | Hello ! | 11:30 |
| Syed_A | submitted two bugs last night. | 11:30 |
| Syed_A | [1] https://bugs.launchpad.net/charms/+source/quantum-gateway/+bug/1468939 | 11:30 |
| mup | Bug #1468939: Instances fail to get metadata: The 'service_metadata_proxy' option must be enabled. <quantum-gateway (Juju Charms Collection):New> <https://launchpad.net/bugs/1468939> | 11:30 |
| Syed_A | https://bugs.launchpad.net/charms/+source/nova-cloud-controller/+bug/1468918/ | 11:31 |
| mup | Bug #1468918: neutron-server fails to start; python-neutron-vpnaas and python-neutron-lbaas packages are missing. <nova-cloud-controller (Juju Charms Collection):New> <https://launchpad.net/bugs/1468918> | 11:31 |
| Syed_A | jamespage: Hello | 11:32 |
| mup | Bug #1469130 opened: tools migration fails when upgrading 1.20.14 to 1.24.1 on ec2 <ec2-provider> <upgrade-juju> <juju-core:Triaged> <https://launchpad.net/bugs/1469130> | 12:01 |
| fwereade | mattyw, would you close http://reviews.vapour.ws/r/1460/ one way or the other? looks like it has ship-its | 12:02 |
| mattyw | fwereade, oooh, had forgotten about this | 12:03 |
| fwereade | mattyw, cheers | 12:03 |
| mattyw | fwereade, the comments seem controversial - care to make a casting vote - land or just close? | 12:03 |
| fwereade | mattyw, I'm inclined to trust dave and andrew's apparent approval; nobody's complained, so land it | 12:04 |
| mattyw | fwereade, landing, thanks very much | 12:05 |
| mattyw | fwereade, thanks for noticing, had totally forgotten about this | 12:06 |
| fwereade | niedbalski, niedbalski_: so, I'm sorry, I don't know what happened with your patches http://reviews.vapour.ws/r/1698/ and http://reviews.vapour.ws/r/1717/ ; it seems they got ship-its but never landed? if you check whether they need updating, and let me know their status, I will make sure they get landed | 12:15 |
| fwereade | dimitern, http://reviews.vapour.ws/r/1403/ ? | 12:16 |
| dimitern | fwereade, looking | 12:24 |
| dimitern | fwereade, that needs to land yes, it's been a while | 12:25 |
| dimitern | fwereade, I'll fix/respond to the current reviews and ask you for a final stamp | 12:26 |
| fwereade | dimitern, cool | 12:26 |
| jamespage | Syed_A, which openstack release? | 12:38 |
| Syed_A | jamespage: Kilo | 12:38 |
| jamespage | Syed_A, for that second bug, neutron-server is not supported on nova-cloud-controller - you have to use the neutron-api charm | 12:38 |
| jamespage | that applies for >= kilo | 12:38 |
| jamespage | Syed_A, can you make sure that your quantum-gateway charm is up-to-date - the kilo template should have the right things set | 12:40 |
| Syed_A | jamespage: Ok, so if i deploy neutron-api charm i wouldn't need to install vpnaas or lbass ? | 12:40 |
| jamespage | Syed_A, the neutron-api charm knows how to deploy those things for >= kilo | 12:40 |
| Syed_A | jamespage: Roger that. | 12:40 |
| jamespage | it will enable them - nova-cloud-controller only supported 'embedded neutron-server' up to juno I think | 12:41 |
| Syed_A | jamespage: This may be a silly question but how can i make sure that quantum-gateway charm is up-to-date ? | 12:41 |
| dimitern | fwereade, updated http://reviews.vapour.ws/r/1403/ PTAL | 12:41 |
| jamespage | Syed_A, are you deployed from branches or from the juju charm store? | 12:41 |
| Syed_A | jamespage: juju charm store. | 12:49 |
| jamespage | Syed_A, which version does 'juju status' tell you have deployed then | 12:49 |
| jamespage | Syed_A, version 16 has the required templates: | 12:50 |
| jamespage | https://api.jujucharms.com/charmstore/v4/trusty/quantum-gateway-16/archive/templates/kilo/nova.conf | 12:50 |
| Syed_A | Ok,,, checking ... | 12:50 |
| Syed_A | jamespage: charm: cs:trusty/quantum-gateway-16 | 12:53 |
| fwereade | dimitern, LGTM | 12:54 |
| jamespage | Syed_A, what's your openstack-origin configuration? | 12:54 |
| Syed_A | jamespage: Unfortunately, in this setup openstack-origin is not present but there is an ansible variable which specify openstack release which is set to kilo. | 12:58 |
| Syed_A | jamespage: The variable is used to set this repository, repo="deb http://ubuntu-cloud.archive.canonical.com/ubuntu {{ ansible_lsb.codename }}-updates/{{ openstack_release }} main" | 12:58 |
| jamespage | Syed_A, I need to understand what the charm thinks it should be doing | 12:59 |
| jamespage | if openstack-origin is not set correctly, it won't use the right templates | 12:59 |
| Syed_A | jamespage: Ok, i am going to set openstack_origin in the config right now. | 12:59 |
| jamespage | irrespective of what you put in sources :) | 12:59 |
| jamespage | Syed_A, this may have working in the past, but for the last release we switched how we determine openstack series to support the deploy from source feature in the charms | 13:00 |
| jamespage | Syed_A, my statement about openstack-origin will apply across all of the openstack charms btw | 13:01 |
| jamespage | the template loader is constructed based on that configuration | 13:01 |
| jamespage | so it will assume a default of icehouse on trusty for example | 13:01 |
| Syed_A | jamespage: Ohhh, i got it, so this might be the reason why this charm which used to work fine, now fails. | 13:03 |
| jamespage | Syed_A, that's quite possible | 13:03 |
| jamespage | Syed_A, before we determined version based on packages installed - however for deploy from source, there are not any openstack packages installed :-) | 13:04 |
| dimitern | fwereade, last look? http://reviews.vapour.ws/r/1403/ | 13:14 |
| fwereade | dimitern, if that's all you changed just land it :) | 13:15 |
| dimitern | fwereade, cheers :) will do | 13:15 |
| Syed_A | jamespage: I am deploying a fresh setup with these configs. [1] http://paste.ubuntu.com/11778630/ && [2] http://paste.ubuntu.com/11778641/ | 13:42 |
| jamespage | Syed_A, openstack-dashboard needs openstack-origin as well | 13:43 |
| jamespage | but looks much better | 13:43 |
| jamespage | Syed_A, I must introduce you to bundles :-) | 13:43 |
| Syed_A | jamespage: bundles ? :) | 13:44 |
| jamespage | Syed_A, hmm - you're doing alot of --to=X to the same machines ? | 13:44 |
| Syed_A | jamespage: Yes, specifying exactly where a service should go ? Isn't a good practice. | 13:45 |
| jamespage | Syed_A, bundles - https://jujucharms.com/openstack-base/ | 13:45 |
| jamespage | Syed_A, pushing multiple services onto the same machines without using containers won;t work | 13:46 |
| jamespage | Syed_A, https://wiki.ubuntu.com/ServerTeam/OpenStackCharms/ProviderColocationSupport | 13:47 |
| Syed_A | jamespage: This is why i was working on a lxc based OpenStack deployment. But for now we are just deploying nova-compute and quantum-gateway on seperate machine which used to work in the past. | 13:50 |
| Syed_A | jamespage: Our lxc based bits are also ready just need to patch the lxc-ubuntu-cloud template for our 3 nics per container requirement. | 13:50 |
| jamespage | Syed_A, I thought you where - good | 13:51 |
| jamespage | you pastebin confused me | 13:51 |
| Syed_A | jamespage: Sorry about that. alice(controller) is 1, bob(compute) is 2 and charlie(quantum-gateway) is 3. :) | 13:52 |
| jamespage | Syed_A, but you are going to use lxc containers right? | 13:52 |
| Syed_A | jamespage: No, not in this setup. | 13:53 |
| jamespage | Syed_A, most of the controller services won't work | 13:53 |
| jamespage | Syed_A, they assume control over the filesystem, so are not safe to deploy without containers | 13:53 |
| Syed_A | jamespage: ohhh that w'd be a problem. :/ | 13:55 |
| jamespage | Syed_A, yeah - I know they will all at-least conflict on haproxy configuration | 13:56 |
| jamespage | Syed_A, we enable that by default now | 13:56 |
| Syed_A | jamespage: for haproxy, we have a customized haproxy.cfg which fixes the issue | 13:57 |
| * fwereade was up until 2 last night, taking an extended break, may or may not be back at a reasonable time | 14:02 | |
| jamespage | Syed_A, you guys are terrifying me - all I can say is ymmv | 14:03 |
| mbruzek | Has anyone seen a problem with the GCE provider today? The juju bootstrap command is giving this error: ERROR failed to bootstrap environment: cannot start bootstrap instance: no "trusty" images in us-central1 with arches [amd64 arm64 armhf i386 ppc64el] | 14:04 |
| sinzui | mbruzek: I am in #cloudware. I haven’t gotten any answers | 14:07 |
| sinzui | mbruzek: there are NO images for gce http://cloud-images.ubuntu.com/releases/streams/v1/com.ubuntu.cloud:released:gce.sjson | 14:08 |
| Syed_A | jamespage: Our goal is to eventually move towards lxc based openstack deployment as suggested by the community. Right now i am only trying to fix this issue for the time being. We have every intention to follow the process as suggested on ubuntu wiki. | 14:08 |
| mbruzek | sinzui: strange that this worked before, I am just seeing this error today | 14:08 |
| sinzui | mbruzek: CI tests gce, we saw the failre about 15 hours ago. | 14:09 |
| mbruzek | sinzui: Did you file a bug that I can contribute to? | 14:10 |
| sinzui | mbruzek: no, because this is an ops issue. I am not aware of a project for gce images | 14:10 |
| sinzui | mbruzek: I am crafting a email asking for someone with power to explain the situation | 14:11 |
| Syed_A | jamespage: You were right about the conflict at haproxy , neutron-api failed to install and logs this:INFO install error: cannot open 9696/tcp (unit "neutron-api/0"): conflicts with existing 9696/tcp (unit "nova-cloud-controller/0") | 14:17 |
| Syed_A | jamespage: Looks like nova-cloud-controller and neutron-api are both installing neutron-server. | 14:20 |
| jamespage | Syed_A, yes | 14:21 |
| jamespage | Syed_A, hmm - yes - that won't work well on a single unit | 14:21 |
| jamespage | Syed_A, there is a huge assumption in the charms that they 'own' the unit | 14:21 |
| Syed_A | jamespage: Ok, so how can i stop nova-cloud-controller from installing neutron-server. | 14:25 |
| Syed_A | jamespage: Will it work if i deploy neutrona-api unit on quantum-gateway node ? | 14:29 |
| jamespage | Syed_A, nope - neutron-api will trample all over the gateway charms config files | 14:30 |
| Syed_A | jamespage: compute node then ? | 14:30 |
| jamespage | Syed_A, nova-cc decides to stop managing neutron-server - but not straight away | 14:30 |
| jamespage | Syed_A, same problem - but this time neutron-openvswitch's config files | 14:30 |
| jamespage | Syed_A, the charms are just not designed for this type of use | 14:30 |
| mup | Bug #1469184 opened: listSuite teardown fails <ci> <intermittent-failure> <unit-tests> <juju-core:Incomplete> <juju-core 1.24:New> <https://launchpad.net/bugs/1469184> | 14:32 |
| mup | Bug #1469186 opened: ContextRelationSuite teardown fails <ci> <intermittent-failure> <test-failure> <juju-core:Triaged> <https://launchpad.net/bugs/1469186> | 14:32 |
| Syed_A | jamespage: Don't you think charms should be able to deploy a standalone controller node say a VM. | 14:32 |
| jamespage | Syed_A, I'm adverse to changing the design principle each charm has in that it 'owns' the unit filesystem | 14:34 |
| jamespage | Syed_A, LXC containers give us a lightweight way to manage this, without having to have alot of complexity in the charms to deal with this problem | 14:35 |
| Syed_A | jamespage: I am inclined to agree with you. LXC works better but the use case that somebody might want to deploy a openstack controller node without using lxc is a valid use case. | 14:37 |
| jamespage | Syed_A, I don't disagree with that - just saying maybe the charms are not the right way to fullfil that | 14:38 |
| natefinch | fwereade: why did we write our own RPC implementation when there's one in the stdlib? | 14:45 |
| mup | Bug #1469189 opened: unitUpgraderSuite teardown panic <ci> <intermittent-failure> <unit-tests> <juju-core:Incomplete> <juju-core 1.24:Triaged> <https://launchpad.net/bugs/1469189> | 15:02 |
| mup | Bug #1469193 opened: juju selects wrong address for API <sts> <juju-core:New> <https://launchpad.net/bugs/1469193> | 15:02 |
| mup | Bug #1469196 opened: runlistener nil pointer / invalid address <ci> <intermittent-failure> <unit-tests> <juju-core:Incomplete> <juju-core 1.24:Triaged> <https://launchpad.net/bugs/1469196> | 15:02 |
| Syed_A | jamespage: Ok let's say i fix the neutron-server manually but what about the instance metadata not working ? | 15:02 |
| jamespage | Syed_A, that should be fixed by correctly specifying openstack-origin | 15:02 |
| Syed_A | jamespage: testing ... | 15:06 |
| === kadams54 is now known as kadams54-away | ||
| === kadams54-away is now known as kadams54 | ||
| Syed_A | jamespage: Ok so instance metadata is working. | 15:33 |
| fwereade | natefinch, I can't remember what it does that the stdlib one didn't, but I know it was something :/ | 15:33 |
| Syed_A | jamespage: As per your suggestion correctly specifying openstack-origin fixed the issue. | 15:33 |
| fwereade | natefinch, rogpeppe would remember | 15:33 |
| rogpeppe | natefinch: there were a few reasons | 15:34 |
| rogpeppe | natefinch: the main one is that with the stdlib version you don't get to have per-connection context | 15:34 |
| natefinch | rogpeppe: ahh, interesting, yeah | 15:35 |
| rogpeppe | natefinch: also, the way you have to phrase the stdlib methods is awkward | 15:35 |
| Syed_A | jamespage: If somebody is deploying openstack on public cloud and they cannot use lxc, so the suggestion here will be to start a new vm and install neutron-api as a standalone unit there ? | 15:38 |
| mup | Bug #1469199 opened: State server seems to have died <cloud-install-failure> <juju-core:New> <https://launchpad.net/bugs/1469199> | 15:41 |
| natefinch | rogpeppe: yeah, the stdlib way is kind of annoying, I'm surprised they didn't do it the way ours does... (traditional val, error return)... but I'm sure there was a reason at the time | 15:41 |
| rogpeppe | natefinch: it's simpler to implement the way they did it | 15:41 |
| rogpeppe | natefinch: but my reasoning was we were going to be writing lots of API entry points, so the additional complexity in the rpc package was worth it | 15:42 |
| voidspace | mgz: ping | 15:46 |
| mgz | voidspace: hey | 15:49 |
| voidspace | mgz: it's alright, I think I've sorted it | 15:49 |
| voidspace | mgz: had a question about gomaasapi which you seem to have touched | 15:50 |
| mgz | voidspace: okay, I hall remain in the dark | 15:50 |
| voidspace | mgz: heh | 15:51 |
| voidspace | mgz: I hate creating JSON maps in Go :-/ | 15:53 |
| mgz | voidspace: it is not the most fun | 15:53 |
| jamespage | Syed_A, yes - but that is very much an edge case | 15:56 |
| jamespage | most clouds are deployed on metal :-) | 15:57 |
| jamespage | Syed_A, infact what you suggest is exactly how we test the openstack charms - we have a small QA cloud (5 compute nodes) which we can standup a full openstack cloud ontop of | 15:57 |
| jamespage | we can run ~15 clouds in parallel | 15:57 |
| jamespage | and do things like test HA etc... | 15:57 |
| Syed_A | jamespage: Correct most clouds are deployed on metal. But with the latest charms neutron-api and nova-cloud-controller cannot be installed on the same physical machine ? | 15:58 |
| jamespage | Syed_A, that is absolutley the case - and you will hit issues with other conflicts as well | 16:00 |
| jamespage | Syed_A, which is why we have https://wiki.ubuntu.com/ServerTeam/OpenStackCharms/ProviderColocationSupport | 16:00 |
| Syed_A | jamespage: We also have a small setup where we test openstack. I set up HA LXC openstack setup last week. Which was fun :) | 16:00 |
| jamespage | :-) | 16:00 |
| jamespage | Syed_A, its neat - the qa cloud i refer to is juju deployed, and is HA control plane under lxc as well | 16:00 |
| Syed_A | jamespage: Cool ! | 16:02 |
| ericsnow | natefinch: regarding RB, did you mean the GH integration isn't working or something else? | 16:18 |
| === kadams54 is now known as kadams54-away | ||
| sinzui | mbruzek: gce streams are back | 16:27 |
| mbruzek | sinzui: thank you | 16:27 |
| natefinch | ericsnow: yes, the GH integration... like, I made a PR vs. juju-process-docker and no review was created on RB | 16:55 |
| natefinch | ericsnow: I probably just missed a steo | 16:56 |
| natefinch | ste | 16:56 |
| natefinch | step | 16:56 |
| natefinch | arg... | 16:56 |
| ericsnow | natefinch: yeah, the repo did not have the web hook set up (I've added it) | 17:06 |
| === kadams54-away is now known as kadams54 | ||
| natefinch | ericsnow: can you document the steps in the wiki? | 17:08 |
| ericsnow | natefinch: sure | 17:09 |
| natefinch | ericsnow: so, process server api in process/api/server.go? | 18:03 |
| ericsnow | natefinch: how about process/api/server/uniter.go | 18:04 |
| ericsnow | natefinch: params would live in process/api/params.go | 18:04 |
| natefinch | ericsnow: is there a reason to split out the params, server, and client stuff? if each one is fairly simple and probably fits in a single file... | 18:05 |
| ericsnow | natefinch: my expectation is that it won't fit well in a single file | 18:06 |
| natefinch | ericsnow: ok | 18:06 |
| === kadams54 is now known as kadams54-away | ||
| natefinch | ericsnow: when are those state functions getting merged into the feature branch? | 19:43 |
| ericsnow | natefinch: likely not before Monday | 19:44 |
| natefinch | ericsnow: ok | 19:44 |
| natefinch | this whole "duplicate every single struct in the API" thing gets really tiresome | 20:21 |
| mup | Bug #1469318 opened: apitserver: TestAgentConnectionsShutDownWhenStateDies takes > 30 seconds to run <juju-core:New> <https://launchpad.net/bugs/1469318> | 21:51 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!