| thumper | anyone on know mocks well? I need to discuss a test | 00:03 |
|---|---|---|
| thumper | babbageclunk, wallyworld[m] ? | 00:04 |
| * thumper goes to make lunch instead | 00:04 | |
| * thumper looks for babbageclunk again | 00:56 | |
| thumper | o/ kirkland | 00:56 |
| * thumper checked calendar and saw babbageclunk out | 01:00 | |
| thumper | wallyworld: I was wanting to chat about a test using mocks | 01:08 |
| thumper | as I'm struggling a little | 01:08 |
| thumper | wallyworld: could probably get it over in 5 minutes | 01:09 |
| wallyworld | sure, i'm free | 01:09 |
| thumper | wallyworld: I'll jump in your standup | 01:09 |
| thumper | wallyworld: https://github.com/juju/juju/pull/11530 | 01:47 |
| wallyworld | ok | 01:47 |
| wallyworld | just finishing this other one | 01:47 |
| thumper | ack | 01:50 |
| wallyworld | thumper: lgtm | 02:05 |
| thumper | ta | 02:16 |
| wallyworld | thumper: a small fix to restore a api method to the latest facade https://github.com/juju/juju/pull/11529 | 03:31 |
| wallyworld | and a +1/0 https://github.com/juju/juju/pull/11531 | 03:43 |
| * thumper loogs | 04:14 | |
| * thumper looks even | 04:14 | |
| thumper | wallyworld: already commented on the first one | 04:14 |
| thumper | approved second | 04:15 |
| wallyworld | ta | 04:17 |
| babbageclunk | wallyworld: hey, you already merged my stuff forward into develop! | 04:18 |
| babbageclunk | thanks! | 04:18 |
| wallyworld | thumper: nah, not useful to me, just a pita to keep up to date. YMMV. i was also going for consistency. i can add them back if you feel strongly | 04:18 |
| babbageclunk | huh, I think I've gotten confused from too much birthday cake | 04:20 |
| wallyworld | babbageclunk: how was the birthday? | 04:21 |
| babbageclunk | about as much fun as you can have at a party! (given that you're not allowed to invite friends) | 04:22 |
| babbageclunk | She decided that you don't turn 4 until you've eaten the cake, it was very cute | 04:22 |
| wallyworld | well, none of *my* parties have friends | 04:23 |
| babbageclunk | Bugger, I landed that change in the wrong branch, backporting | 04:25 |
| babbageclunk | I mean, I intended to get it into develop, but had meant to do it first | 04:25 |
| babbageclunk | in 2.7 | 04:25 |
| wallyworld | ooops | 04:25 |
| babbageclunk | fixing now | 04:29 |
| thumper | wallyworld: I think I do feel strongly enough to keep them, I'd like to see that pattern extended | 04:32 |
| thumper | wallyworld: the key here is that if they change, it should be obvious | 04:32 |
| thumper | and we shouldn't be changing older facades | 04:32 |
| wallyworld | ok, i'll revert. but it's not a great check for facade breakage since you can add methods to a facade and not even know about the interfaces | 04:34 |
| wallyworld | i do question whether they carry their weight | 04:35 |
| wallyworld | thumper: pr much smaller now, interfaces restored | 04:46 |
| babbageclunk | kelvinliu: want to rubber-stamp the PR you've already approved but this time for the 2.7 branch? :) https://github.com/juju/juju/pull/11532 | 04:52 |
| kelvinliu | babbageclunk: yep | 05:01 |
| babbageclunk | thanks! | 05:01 |
| kelvinliu | np : > | 05:02 |
| wallyworld | thumper: small one if you're still here https://github.com/juju/juju/pull/11533 otherwise can wait | 05:09 |
| babbageclunk | wallyworld: I done it | 05:20 |
| wallyworld | ty | 05:21 |
| thumper | ty | 05:43 |
| thumper | well that was a very annoying hour debugging | 06:52 |
| thumper | for the record, if you use a standard http client and have an url that looks like "http://something//action" and you call PostForm, it gets converted into a Get silently | 06:52 |
| thumper | remove the double slash and you're fine | 06:53 |
| thumper | FFS | 06:53 |
| * thumper well past EOD | 07:20 | |
| thumper | later peeps | 07:20 |
| === balloons3 is now known as balloons | ||
| achilleasa_ | jam: do you want me to take a look at 11527 or do you want to land it as it's already approved? | 09:03 |
| jam | achilleasa_, I'm happy to have your feedback, if you're busy I can just land it. | 09:07 |
| achilleasa_ | jam: will take a look now since I am waiting on CI ;-) | 09:08 |
| jam | great! | 09:15 |
| === salmankhan1 is now known as salmankhan | ||
| skatsaounis | Hi, I have a question regarding cloud-init of juju models' machines and juju controllers | 10:17 |
| skatsaounis | Is there an option during juju bootstrap or by providing juju model constraints to set some options to cloud-init of created machines? | 10:17 |
| skatsaounis | for example, I would love to add a user with password to troubleshoot some issues with my OpenStack cloud provider. | 10:18 |
| skatsaounis | Being more specific, I want to use the virsh console to login to a juju machine (nova instance) and login with my user to troubleshoot my connectivity issues | 10:19 |
| Eryn_1983_FL | hey guys | 12:10 |
| Eryn_1983_FL | I got an issue | 12:10 |
| Eryn_1983_FL | i remove my nic config on a server in the cluster and i cant login to it locall | 12:10 |
| Eryn_1983_FL | locally | 12:10 |
| Eryn_1983_FL | this is for openstack | 12:11 |
| achilleasa_ | manadart: I am trying to add an address dedup layer on top of the NetworkInfo API call (uniter). An interface address is a tuple (host, address, cidr). I am trying to recall whether it is legal to have the same CIDR in two different spaces | 12:18 |
| achilleasa_ | or do you think that dedupping on the address alone would be sufficient? | 12:19 |
| manadart | achilleasa_: As of right now, where CIDR:Subnet is 1:1, a CIDR can only be in one space. | 12:19 |
| Eryn_1983_FL | hello | 12:24 |
| Eryn_1983_FL | so how do i get into an admin account for a node on the cluster? | 12:25 |
| Eryn_1983_FL | i cant ssh into it. | 12:25 |
| === balloons4 is now known as balloons | ||
| Eryn_1983_FL | ??? | 12:30 |
| Eryn_1983_FL | can somebody tell me how i can rescue a box w/o any networking working? | 12:37 |
| manadart | Eryn_1983_FL: If you add an interface via OpenStack, Juju should detect the change shortly thereafter, allow you to `juju ssh`. | 12:43 |
| hml | achilleasa_: 11526 is looking good, just running a few of the qa steps again | 12:46 |
| Eryn_1983_FL | ok.. | 12:46 |
| Eryn_1983_FL | i dont think it can even communicate to it manadart | 12:46 |
| Eryn_1983_FL | i cant ping any nics | 12:48 |
| Eryn_1983_FL | heading towards a rescue cd.. | 12:48 |
| manadart | Eryn_1983_FL: This node is part of a Juju model, correct? | 12:49 |
| Eryn_1983_FL | yeah | 12:50 |
| Eryn_1983_FL | i move the netplan file | 12:50 |
| Eryn_1983_FL | and it messed it up | 12:50 |
| Eryn_1983_FL | atleast i found u guys finally | 12:51 |
| Eryn_1983_FL | and the docs the proper docs | 12:51 |
| Eryn_1983_FL | i got serious network issues | 12:51 |
| Eryn_1983_FL | i have broken the fabrics of spacetime/network :( | 12:51 |
| === marcosilva_ is now known as marcosilva | ||
| Eryn_1983_FL | can you explain how openstack is using netplan and vswitch? | 12:55 |
| Eryn_1983_FL | it seems like it was making only one bridge eth0 and thats what i remove thinking it was in error and then everything went down after reboot | 12:56 |
| flxfoo | hi all, | 13:13 |
| flxfoo | sorry to bother, I killed a juju process during bootstrap attempt, now I have `ERROR no API addresses" when trying to do a `juju status`, any idea? | 13:13 |
| flxfoo | thanks in advance | 13:13 |
| flxfoo | (still wanted to create a new controller) | 13:14 |
| Eryn_1983_FL | not sure flxfoo | 13:21 |
| Eryn_1983_FL | manadart: ? | 13:22 |
| Eryn_1983_FL | noobie here | 13:22 |
| flxfoo | Eryn_1983_FL: ok, got something | 13:30 |
| Eryn_1983_FL | ok | 13:32 |
| Eryn_1983_FL | whats the root passes? | 13:32 |
| Eryn_1983_FL | damn it | 13:33 |
| Eryn_1983_FL | i didnt add it to the admin group or whatever | 13:33 |
| Eryn_1983_FL | sigh | 13:35 |
| Eryn_1983_FL | so i got vswitch and netplan running on the same boxes.. | 13:37 |
| Eryn_1983_FL | why is that? | 13:37 |
| Eryn_1983_FL | is that normal? | 13:37 |
| Eryn_1983_FL | ok i almost got it back to normal i hope | 13:56 |
| Eryn_1983_FL | well less broken | 13:56 |
| hml | flxfoo: you need to remove the attempted controller and rebootstrap. remove the controller machine via your cloud, and run juju unregister <controller name> | 14:06 |
| petevg | flxfoo: the --force flags on destroy controller, and the juju unregister <controller> command should help you get into a state where you can cleanup the borked controller. You can then bootstrap a new one. | 14:06 |
| Eryn_1983_FL | ok guys i am going to post some pastebins of whats going on with my stuff | 14:07 |
| petevg | Eryn_1983_FL: the internals of how the OpenStack charms work are documented at https://docs.openstack.org/charm-guide/latest/ | 14:09 |
| petevg | If you have specific questions after reading the docs, we'd be happy to answer them. | 14:09 |
| Eryn_1983_FL | https://paste.debian.net/1145121/ | 14:10 |
| Eryn_1983_FL | ok | 14:10 |
| petevg | Eryn_1983_FL: juju and the charms are meant to abstract a lot of the internals away, so that you don't have to worry about them. You're welcome to experiment with your system, of course, but you will break things if you start removing network interfaces and network configuration files around. :-) | 14:11 |
| Eryn_1983_FL | ok | 14:12 |
| Eryn_1983_FL | well is there a way to go back? | 14:12 |
| Eryn_1983_FL | i think the problem is the co worker used juju and messed soemthing up | 14:13 |
| Eryn_1983_FL | the switching fabric broke | 14:13 |
| Eryn_1983_FL | lets start with how can i verify the status of my openstack, and juju, and then what can i do to reinstall rebuild | 14:14 |
| Eryn_1983_FL | https://paste.debian.net/1145123/ | 14:17 |
| Eryn_1983_FL | that looks good to me petevg | 14:17 |
| Eryn_1983_FL | except for ceph | 14:18 |
| petevg | Eryn_1983_FL: if there are still problems, it looks like juju doesn't know about them. Do you know what actions the co-worker ran? | 14:18 |
| Eryn_1983_FL | no clue | 14:20 |
| Eryn_1983_FL | we ere getting permission denied erros when we try to start instances | 14:20 |
| Eryn_1983_FL | i need to reinstall the switching on the machines is there a way to do that | 14:20 |
| Eryn_1983_FL | i think the rest is fine | 14:21 |
| Eryn_1983_FL | if i were to redeploy like web4 | 14:22 |
| Eryn_1983_FL | does that affect my instances or whatever | 14:22 |
| petevg | Eryn_1983_FL: were you getting permission denied in the OpenStack tooling? What lead you to think that the root cause was networking? | 14:25 |
| Eryn_1983_FL | well i saw the networking when i went in | 14:30 |
| Eryn_1983_FL | i think we saw some errors for br-int was busy | 14:30 |
| Eryn_1983_FL | and then the processes for openvswitch was down | 14:30 |
| Eryn_1983_FL | failed | 14:30 |
| Eryn_1983_FL | the webgui we got the permission denied error | 14:30 |
| Eryn_1983_FL | when we tried to restart the instaces after some external network issues and reboots | 14:32 |
| petevg | Eryn_1983_FL: Open vSwitch is the virtualized networking layer for OpenStack. If you remove the bridges or interfaces or configuration it creates, you'll interfere with the OpenStack services' ability to communicate with each other. | 14:36 |
| petevg | When you fixed the external network issues, did you bring the network back up in the same state that it was in before? i.e., do machines still communicate on the same subnets, and have the same ip addresses? | 14:37 |
| Eryn_1983_FL | not sure what he did on that layer either, i can still ssh from maas | 14:39 |
| Eryn_1983_FL | i can ping out on them | 14:39 |
| Eryn_1983_FL | at this point i just want to build another 4 host cluster and rescuse this data off and put it there.. | 14:39 |
| petevg | Eryn_1983_FL: that sounds like a good idea. There's enough uncertainty here in the history to make it hard to troubleshoot over IRC. If you have the option to rebuild, I'd go with that option. :-) | 14:44 |
| achilleasa_ | manadart: if around can you take a look at https://github.com/juju/juju/pull/11535? | 15:00 |
| achilleasa_ | manadart: you were right, the cause of the bug is indeed https://bugs.launchpad.net/juju/+bug/1855263 but we should probably leave that be and resolve it once your refactoring work lands | 15:01 |
| mup | Bug #1855263: Duplicate link layer device entries when running on AWS <juju:Triaged by manadart> <https://launchpad.net/bugs/1855263> | 15:01 |
| manadart | achilleasa_: Yep; looking now. | 15:02 |
| === achilleasa_ is now known as achilleasa | ||
| Eryn_1983_FL | ok question guyes | 15:45 |
| Eryn_1983_FL | where are the root partitions stored for instances? | 15:46 |
| Eryn_1983_FL | ?? | 15:52 |
| Eryn_1983_FL | found it /var/lib/nova/instances | 16:14 |
| Eryn_1983_FL | petevg: do you know how to open these files? | 16:27 |
| petevg | Eryn_1983_FL: I wouldn't recommend introspecting vm filesystems unless you are very comfortable with what you're doing. If I were troubleshooting this myself, I'd probably be looking for root cause of the issue in log messages. | 16:30 |
| Eryn_1983_FL | yeah its the vswitch | 16:32 |
| Eryn_1983_FL | but nobody know how to help me | 16:32 |
| === salmankhan1 is now known as salmankhan | ||
| petevg | Eryn_1983_FL: what are you trying to accomplish by inspecting the vms? Are you trying to recover data? Or look at logs? | 16:38 |
| petevg | You can use openstack tools to look at the console logs. For example: "openstack console log show <some instance name>" | 16:39 |
| Eryn_1983_FL | i need the conifgs and data from them | 16:40 |
| Eryn_1983_FL | so i rebuild the entire thing | 16:40 |
| Eryn_1983_FL | looks like its qcow | 16:42 |
| Eryn_1983_FL | i think i can handle that | 16:42 |
| petevg | Eryn_1983_FL: yes. They are qcow images. Just as a note: you probably don't need to be looking in the images for information like how things were networked together. That information should exist in your juju bundle.yaml. | 16:44 |
| petevg | If you're looking to recover data unique to applications that you had deployed on top of OpenStack, then it might make sense to look in those images, assuming that you don't have backups elsewhere. | 16:45 |
| petevg | Eryn_1983_FL: also, since you're deploying on MAAS and Juju, do you have a support contract with Canonical? If so, I would reach out through your support channels. Those would be a better place to ask for step by step instructions on disaster recovery. | 16:48 |
| Eryn_1983_FL | haha | 17:01 |
| Eryn_1983_FL | nope no support | 17:01 |
| Eryn_1983_FL | boss was like lets do this and installs it.. | 17:01 |
| Eryn_1983_FL | nah this was not backuped else where, | 17:02 |
| flxfoo | petevg: hml thanks guys ! :) | 17:40 |
| petevg | You're welcome! | 17:40 |
| babbageclunk | hpidcock: wanna do an easy +0/-0 review? https://github.com/juju/juju/pull/11536 | 23:30 |
| babbageclunk | or anyone else - wallyworld? | 23:42 |
| hpidcock | sure | 23:46 |
| hpidcock | babbageclunk: doneski | 23:47 |
| babbageclunk | cheers | 23:47 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!