| === alexpilotti_ is now known as alexpilotti | ||
| === CyberJacob|Away is now known as CyberJacob | ||
| === allenap_ is now known as allenap | ||
| gmb | allenap, rvba: https://code.launchpad.net/~gmb/maas/use-node.start-instead-of-start_nodes-bug-1330765/+merge/238758 needs a review when you’ve time | 08:49 |
|---|---|---|
| bigjools | gmb: sorry I should have reviewed that today | 08:51 |
| gmb | bigjools: no worries. Y’all might be getting spammed in a minute anyway; I just tried to create an MP that had that one as a prerequisite, and forgot to add the prerequisite branch. Twice. | 08:53 |
| gmb | Not sure whether LP will kill the notifications for those two deleted MPs. | 08:53 |
| bigjools | \o/ | 08:53 |
| gmb | allenap, rvba: https://code.launchpad.net/~gmb/maas/use-node.start-instead-of-start_nodes-bug-1330765/+merge/238758 and https://code.launchpad.net/~gmb/maas/use-node.stop-instead-of-stop_nodes-bug-1330765/+merge/238853 are up for review when you’ve time. | 10:17 |
| dimitern | hey guys | 10:19 |
| dimitern | i've just filed bug 1383231 | 10:19 |
| ubot5 | bug 1383231 in MAAS "Can reserve the same static IP twice and cannot release it completely" [Undecided,New] https://launchpad.net/bugs/1383231 | 10:19 |
| gmb | allenap, rvba: Also https://code.launchpad.net/~gmb/maas/remove-stop_nodes-and-start_nodes-bug-1330765/+merge/238857 (these branches are in dependency order, btw) | 10:27 |
| gmb | dimitern: Looking… | 10:27 |
| allenap | gmb: I'll take a look in a bit. | 10:27 |
| gmb | allenap: Ta | 10:27 |
| gmb | allenap: Realise you’re busy as hell with the transaction stuff right now, so no rush; just putting it out there. | 10:27 |
| dimitern | gmb, I've updated the bug a bit - it's more of a case for better error message actually | 10:28 |
| gmb | okay | 10:28 |
| dimitern | nothing is broken, just confusing | 10:28 |
| allenap | dimitern: There's only one 50.100 reservation. | 10:30 |
| allenap | dimitern: Ah, I see you've updated the bug. | 10:31 |
| dimitern | allenap, yes, I initially though there's a duplicate, but that's not the case; just a better error message is needed | 10:31 |
| gmb | dimitern: Confirmed and triaged. | 10:31 |
| dimitern | cheers! | 10:32 |
| gmb | (Also found another bug) | 10:32 |
| allenap | dimitern: The bug description doesn't appear right still: the 50.100 address isn't in the "But the list still shows it:" list. | 10:32 |
| dimitern | allenap, looking | 10:33 |
| dimitern | allenap, dropped the list section, as it's no longer relevant | 10:34 |
| allenap | rvba: https://code.launchpad.net/~allenap/maas/transactions-redux-the-revenge/+merge/238864 passed the full test suite, but I suspect it might fail in practice. | 11:03 |
| rvba | allenap: looking | 11:52 |
| rvba | allenap: why are you using a method to patch things instead of simply configuring Django using the settings? | 11:57 |
| roaksoax | rvba: this is why we changed the isolation stuff right? | 12:31 |
| roaksoax | rvba: because the settings were being used in practice | 12:31 |
| allenap | rvba: To make sure it actually happens. A change was previously made to use SERIALIZABLE but it never made it into production because a change was missed to maas_local_settings.py. This way we can't avoid it. | 12:32 |
| rvba | roaksoax: like allenap said, we never changed the isolation level. We just made it explicit. | 12:33 |
| rvba | allenap: I don't get it, why not put this in the settings the user can't modify? | 12:33 |
| roaksoax | allenap: i do remember the change being made in maas_local_settings.py | 12:34 |
| allenap | rvba: Because they *can* modify maas_local_settings.py. | 12:34 |
| roaksoax | allenap: otherwise my large maas deployment would have been failing altogether | 12:34 |
| roaksoax | rvba: so the settings.py get's completely overwritten by what's in maas_local_settings.py ? | 12:34 |
| rvba | roaksoax: individual settings are overridden by what's in maas_local_settings.py, yes. | 12:35 |
| roaksoax | rvba: ok, so if settings.py set the ISOLATION to SERIALIZABLE, and not setting it in maas_local_settings.py would mean that the individual setting for that was not being replaced with what we have in maas_local_settings.py | 12:35 |
| rvba | allenap: well, okay, I see your point. I think that's completely overkill but okay. | 12:35 |
| roaksoax | rvba: so that means we were using serializable | 12:35 |
| rvba | roaksoax: yes | 12:36 |
| roaksoax | thanks, so 1. we should have never introduced any changes to maas_local_Settings.py 2. we should have not change from serializable to the other read_committed | 12:36 |
| allenap | rvba: It also avoids generating another install-time conflict on maas_local_settings.py | 12:37 |
| rvba | allenap: my point is that it should be in src/maas/settings.py | 12:37 |
| roaksoax | next time, please ask for my review before making changes to config files as the above | 12:37 |
| allenap | roaksoax: No, it was *NOT* using SERIALIZABLE isolation. | 12:37 |
| rvba | allenap: if that's possibly considering the packaging messes with the config. | 12:37 |
| allenap | The setting is per-database. | 12:37 |
| roaksoax | allenap: settings.py was using serializable isolation | 12:37 |
| allenap | maas_local_settings.py replaces the DATABASES setting. | 12:38 |
| roaksoax | allenap: did you test that was the case? | 12:38 |
| rvba | allenap: right, that's unfortunate but that's how it is indeed. | 12:38 |
| allenap | OMFG, read the code! | 12:38 |
| roaksoax | allenap: did you actually ensured that it was the case? if settings.py was using serializable and maas_local_settings.py wasn't specifying the value, did you ensure that was the case? | 12:38 |
| roaksoax | allenap: because IIRC< jtv didn't make the config change because he tested that settings.py wasn't being fully overwritten, only what was modified in maas_local_settings.py | 12:39 |
| rvba | roaksoax: yeah, like I said, individual settings from maas_local_settings.py override what's in settings.py | 12:40 |
| allenap | roaksoax: No, I didn't test. However, the code is almost as simple as: | 12:41 |
| allenap | DATABASES = {... isolation_level: SERIALIZABLE} # in maas.settings | 12:41 |
| allenap | DATABASES = {... isolation_level not specified} # in maas_local_settings.py | 12:41 |
| allenap | IOW, the DATABASES value is being completely replaced. | 12:41 |
| roaksoax | allenap: so then, why do we have a regression where we didn't have it before? | 12:42 |
| rvba | roaksoax: are we sure it's a regression? | 12:43 |
| roaksoax | rvba: yup | 12:43 |
| allenap | roaksoax: I'm fairly confident that any regression has been caused by another change. rvba, gmb: Do we ask OIL to try out SERIALIZABLE as a cowboy patch a while back? | 12:47 |
| allenap | s/Do/Did/ | 12:47 |
| roaksoax | allenap: we did | 12:47 |
| roaksoax | allenap: we did test it | 12:47 |
| allenap | roaksoax: Could the regression come from losing that snippet of configuration in OIL? | 12:47 |
| allenap | Are we sure this is a regression in *trunk*? | 12:48 |
| roaksoax | allenap: so we have been upgrading from trunk | 12:48 |
| roaksoax | allenap: ie. we upgraded to 1.5, then various 1.6, then early 1.7, then late 1.7 | 12:49 |
| roaksoax | allenap: the issue was seen in late 1.7 (beta4/5?( | 12:49 |
| rvba | roaksoax: what's the latest version without that pb? | 12:49 |
| allenap | roaksoax: At some point the person upgrading will have been asked about a conflict in maas_local_settings.py. Could it be that the cowboy patch got overwritten at that point? | 12:49 |
| allenap | roaksoax: Sorry for my outburst earlier. I was/am stressed, tired, not very well, but nevertheless it wasn't cool. | 12:51 |
| === CyberJacob is now known as CyberJacob|Away | ||
| === CyberJacob|Away is now known as CyberJacob | ||
| dimitern | any reviewers available? | 14:22 |
| dimitern | I'd like to have this gomaasapi MP reviewed please - https://code.launchpad.net/~dimitern/gomaasapi/static-ipaddresses/+merge/238893 | 14:22 |
| gmb | dimitern: I’ll take a look presently. | 14:23 |
| dimitern | gmb, thanks! | 14:23 |
| === jfarschman is now known as MilesDenver | ||
| gmb | dimitern: Approved. I’m not an official reviewer of gomaasapi, but it doesn’t look like LP has actually noticed that :) | 14:50 |
| dimitern | gmb, cheers! :) | 14:50 |
| dimitern | gmb, there's no bot for gomaasapi right? I just need to merge my branch into lp:gomaasapi? | 14:51 |
| gmb | dimitern: I think so. | 14:51 |
| gmb | Can’t remember there being a bot. | 14:51 |
| dimitern | gmb, ok then | 14:51 |
| === jfarschman is now known as MilesDenver | ||
| === jfarschman is now known as MilesDenver | ||
| === roadmr is now known as roadmr_afk | ||
| th3rt | I've got maas setup to be my dns server for newly bootstrapped instances. Maas is assigning DNS entries based on IP addresses instead of DNS entries based on hostnames. Can anyone tell me what I misconfigured to cause that to happen? | 17:06 |
| th3rt | For example, instead of a dns entry for 7f9wx.maas, I have one for 10-76-16.1.maas | 17:07 |
| th3rt | where 7f9wx is the hostname of the node | 17:07 |
| th3rt | and 10.76.16.1 is the IP | 17:07 |
| blake_r | th3rt: what version of maas? | 17:08 |
| th3rt | dpkg -l output... 1.5.4+bzr2294-0 | 17:09 |
| blake_r | th3rt: you have ubuntu deployed on the node? | 17:09 |
| th3rt | I do | 17:09 |
| blake_r | th3rt: there is no DNS entry for 7f9wx.maas? | 17:10 |
| blake_r | th3rt: did you enable DNS management on that interface for the cluster? | 17:10 |
| th3rt | No, there is no dns entry for 7f9wx.maas and yes, I did enable DNS (and DHCP) for that interface on the cluster. | 17:11 |
| blake_r | th3rt: any error in region celery.log? | 17:12 |
| th3rt | I have an error from more than 5 hours ago, but nothing current. | 17:14 |
| th3rt | [2014-10-20 06:40:01,767: ERROR/Worker-2] rndc_command: Command `rndc -c /etc/bind/maas/rndc.conf.maas reload` returned non-zero exit status 1: | 17:14 |
| blake_r | check the bind configurations to see if that hostname is in the config | 17:16 |
| th3rt | grep -R '7f9wx' /etc/bind/maas/ returns nothing | 17:17 |
| blake_r | hmm | 17:17 |
| th3rt | When I look at the zone.* files in /etc/bind/maas I have a bunch of DNS entries based on the nodes IP addresses | 17:17 |
| th3rt | So I can dig 10-76-16-1.maas | 17:18 |
| th3rt | but not 7f9wx.maas | 17:18 |
| blake_r | yeah you should have a cname record that maps to that | 17:18 |
| blake_r | check through the other maas log to see if you see any other errors | 17:19 |
| th3rt | Well, I don't have any cname records in my zone files | 17:19 |
| th3rt | celery.log shows | 17:23 |
| th3rt | [2014-10-20 12:20:42,623: INFO/MainProcess] Received task: provisioningserver.tasks.upload_dhcp_leases[83572b37-16af-42ee-b6ba-b6341eb97e24] expires:[2014-10-20 17:21:42.619675+00:00] | 17:23 |
| th3rt | [2014-10-20 12:20:42,628: INFO/Worker-6] The DHCP leases file does not exist. This is only a problem if this cluster controller is managing its DHCP server. If that's the case then you need to install the 'maas-dhcp' package on this cluster controller. | 17:23 |
| th3rt | maas-dhcp is installed and running on the host | 17:23 |
| th3rt | ... and handing out IPs | 17:23 |
| === roadmr_afk is now known as roadmr | ||
| blake_r | th3rt: if it cannot parse the leases file then the cname mapping is not made | 17:30 |
| blake_r | th3rt: sudo maas-dhcp-server status | 17:30 |
| th3rt | Alright. that explains my problem :-) | 17:31 |
| th3rt | service maas-dhcp-server status | 17:31 |
| th3rt | maas-dhcp-server start/running, process 28272 | 17:31 |
| th3rt | /var/lib/maas/dhcp has no leases file present | 17:31 |
| th3rt | and looks like maybe it should | 17:31 |
| blake_r | yeah it should | 17:31 |
| th3rt | /var/lib/dhcp has a leases file... | 17:31 |
| blake_r | maybe permissions issue | 17:31 |
| blake_r | that file is not used | 17:31 |
| th3rt | ok | 17:31 |
| blake_r | check syslog for dhcp issues | 17:32 |
| th3rt | Hah, found it. I moved /etc/dhcp and /var/lib/dhcp out of the way and MaaS created it's own leases file in /var/lib/maas/dhcp/ | 17:36 |
| blake_r | th3rt: awesome, glad its fixed | 17:49 |
| th3rt | blake_r, thanks for the help! | 18:12 |
| blake_r | th3rt: np | 18:12 |
| === jfarschman is now known as MilesDenver | ||
| === CyberJacob is now known as CyberJacob|Away | ||
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!