[00:04] does anyone here know enough about samba in 16.04 to say whether both libpam-winbind and libnss-winbind would both be necessary to auth to AD? [00:05] or is it that libpam-winbind provides AD-based auth to PAM and libnss-bind provides winbindd with AD lookups? [00:06] it seems that at some point, at least, libpam-winbind contained the files that are now in libnss-winbind (pre-trusty, i think) === Kenrinx is now known as kenrin === arooni is now known as arooni__ === arooni__ is now known as arooni === arooni is now known as arooni__ [08:29] coreycb, tempest full test failures against newton-staging http://paste.ubuntu.com/20154691/ [08:29] 14 out of 1300 [08:30] that's good === zubat is now known as Golbat === iberezovskiy|off is now known as iberezovskiy === _degorenko|afk is now known as degorenko === rcj` is now known as rcj [11:11] jamespage, that's great, just 14 failures === degorenko is now known as _degorenko|afk [12:27] rbasak: I'm working on mysql security updates...unfortunately, that means I'll be superseding your xenial-proposed package [12:29] mdeslaur: thank you for the note. It's very unfortunate timing. The SRU fixes a bunch of upgrade issues which users will hit once they upgrade after 16.04.1 is out. [12:29] It might even be better to verify quickly and forget the aging time. [12:29] if you'd get them verified, I'll leave them in the security update [12:30] My original SRU included the MRE, but infinity wanted the two separated. Otherwise the SRU would already contain the version bump. [12:30] I'll verify now, thanks. === _degorenko|afk is now known as degorenko [12:33] cpaelzer: are you available? [12:37] rbasak: hi [12:37] reading ... [12:38] rbasak: available for what - helping to verify soemthing in proposed or wherever it currently is? [12:38] rbasak: actually no matter what - for how much time should I plan ? [12:39] cpaelzer: yes please. https://launchpad.net/ubuntu/+source/mysql-5.7/5.7.12-0ubuntu1.2 is the list. I'll start with 865. Could you start with 712 please, and we'll work our way through? [12:40] cpaelzer: I'd like to do as thorough a job as possible as we'll be cutting the usual aging time short. [12:40] rbasak: yeah I'll try to help - I can continue on that arch dependent curtin bug I'm n atm later [12:40] Thank you. [12:51] rbasak: bug 1574458 done, continuing ... [12:51] bug 1574458 in mysql-5.6 (Ubuntu Xenial) "Logs.var.log.mysql.error.log.txt contains usernames and passwords" [Undecided,New] https://launchpad.net/bugs/1574458 [12:52] you said you are doing 1571865 [12:52] I'm picking bug 1602763 next then [12:52] bug 1602763 in mysql-5.7 (Ubuntu Xenial) "postinst does not print a helpful message when the server will fail to start" [Undecided,Fix committed] https://launchpad.net/bugs/1602763 [12:55] cpaelzer: thanks. Yes, I'm still on 865. I think I need to test 5 combinations, so may be a while. [12:56] cpaelzer: for 763, I noticed that changing the datadir doesn't work - known upstream. The fix should cause the addition of "broken = 1" to /etc/mysql/mysql.cnf.d/... to result in a helpful postinst error though. [12:56] cpaelzer: some people in the bug have reported it doesn't help them, but I think they have different unrelated underlying issues. [13:00] cpaelzer, mdeslaur: SRU verification failure for the 865 bug. When infinity asked me to not bump to 5.7.13, I forgot to drop down the version string it compares against for the upgrade path fix. So it does nothing :-/ [13:00] I'm not sure what to do now. [13:00] well, my version will work :) [13:00] actually... [13:01] dpkg --compare-versions "$2" le-nl "5.7.13-0ubuntu3~" [13:01] It might be wrong anyway. Sorry. [13:01] ah, yeah, it will be wrong [13:01] I'll change it in my package, and will upload it to the the security team public ppa [13:01] if you can verify the others in the meantime [13:01] then you can verify the one in the sec team ppa [13:02] OK, thanks. [13:03] cpaelzer: I'll take 458 next. [13:13] cpaelzer: we just both verified 458? [13:14] ? [13:14] cpaelzer: I thought you were going to do 763? [13:14] yeah, that was the first I did [13:15] see above in chat [13:15] I'm sorry, so you did. [13:16] and I updated the bug itself right away with tags and so === ivoks is now known as mirka_ [13:16] better twice than never verified [13:16] rbasak: sorry I missed when you said you take that one next [13:16] If I'd feel better today I might have realized that before you did id - but that isn't my day :-) === mirka_ is now known as ivoks [13:17] cpaelzer: it's OK, it's my fault. [13:17] cpaelzer: OK, pad time. [13:17] cpaelzer: http://pad.ubuntu.com/8qgZ3lJnHY [13:18] ok, I'm there with you [13:18] Thanks. Does what I have look correct? [13:18] I'll do 712 now then [13:20] rbasak: yes the status in the pad looks like reality atm IMHO [13:20] OK thanks [13:23] rbasak: 763 done, picking 647 next, pad and bug updated [13:23] Thanks [13:41] rbasak: 647 done [13:41] cpaelzer: thank you! [13:41] rbasak: so if I read that correctly all done right? [13:42] Yes. [13:42] great [13:42] cpaelzer: I wonder if we could pause and consider anything we might have missed? [13:42] cpaelzer: can you think of any edge cases which could cause problems given the fixes we're putting in here? [13:43] For 865 I think there are many combinations to test, I'll do that once the package is in security-proposed with the version string fixed. [13:43] * cpaelzer |><| <- shall show a spinning hourglass [13:43] :-) [13:44] rbasak: I think the most critical one could be one based on 763 [13:44] For 712 I checked that the ordering is correct, so it does pick up on the change before attempt to start the server again. That seems to work (say my reading of the logs). [13:44] rbasak: where the prestart to check for errors actually breaks something [13:44] Good point, I agree. [13:44] rbasak: but that is just a risk evaluation - I have no idea of a real case that could do so yet [13:45] It did come from upstream. I found one false negative (changing datadir) but no false positives so far. [13:45] And false negatives should be no worse than if we didn't change it. [13:45] rbasak: yeah false negatives are ok [13:45] rbasak: this is a best effort approach to fix this anyway [13:45] I mean incompatible versions are incompatible [13:46] you can only do so much to auto-transition [13:46] and since you tackled 865 it even is better than just "show them a message" [13:46] as it takes care of the most common things [13:46] OK. So are we agreed that we think this SRU is good, apart from the 865 version check and additional testing I need to do on that? [13:46] the apport fix might in rare cases remove too much lines, but that isn't of reasonable severity [13:47] rbasak: yes, from what I see the SRU is good except thealready identified version check issue [13:47] Yeah the apport fix shouldn't cause a regression in production, only to apport reports, so shouldn't regress production. [13:47] cpaelzer: thank you for your help. I really appreciate it - both in terms of time to get this done and for your second pair of eyes. [13:48] rbasak: you are absolutely welcome [13:48] ok, I'll go back and fight with curtin then [13:48] let me know if anything comes up either in this or in the merges [13:48] Will do. [13:48] rbasak: ah since I missed the IRC yesterday [13:48] rbasak: in terms of prio the NTP>>other merges [13:49] I was reviewing your ntp merge but got interrupted by this. Sorry! I'll go back to it as soon as I'm done with this. [13:49] yeah, and the ntp merge is really messy and bug due to all the req bugfixes [13:49] so I beg a pardon and hope to not trigger too much facepalm mode review moments [13:50] It looks good so far - confusing but that's because it is complex. When I follow it through it has made sense so far. [13:51] mdeslaur: I think we're done. Four bugs verified, the fifth one needs that version string fixing. [13:51] rbasak: ok, I'll let you know once I've uploaded it to our public ppa [13:51] rbasak: thanks! [13:51] mdeslaur: thanks. Do you have a rough ETA please, so I can try and be available? [13:52] couple of hours probably [13:52] OK thanks. [13:52] actually, before that, I'll just change the string and will upload it....30-45 min [13:55] OK [14:19] coreycb lp:~ddellav/ubuntu/+source/keystone ready for push/review [14:21] rbasak: uploaded: https://launchpad.net/~ubuntu-security-proposed/+archive/ubuntu/ppa/+packages [14:29] ddellav, jamespage might have fixed up keystone already - https://git.launchpad.net/~ubuntu-server-dev/ubuntu/+source/keystone/log/ [14:29] coreycb ah ok, disregard [14:29] mdeslaur: thanks! I'll test now. [14:29] ddellav, ok [14:36] mdeslaur: W: Failed to fetch http://ppa.launchpad.net/ubuntu-security-proposed/ppa/ubuntu/dists/xenial/main/binary-amd64/Packages 403 Forbidden [14:36] Oh sorry, that's probably my local proxy. [14:37] rbasak: it's not built yet [14:37] the hamsters are spinning [14:37] Oh, OK. Well, in the meantime I'll fix my proxy :) [14:46] I'm using ubuntu 16.04 and for this server apache dies all the time it seems. Well the process is still running but my test site goes down. [14:47] kpettit: Are you using the packages from the Ubuntu archive? [14:47] It's a default 16.04 running on Rackspace. [14:47] sorry I mean digital ocean. [14:48] I've got 10.04, 12.04, 14.04 and now this one. But it's my only one that just craps out like that. [14:48] kpettit: Ubuntu doesn't come with Apache running by default; how did you install Apache on the system? [14:49] I did tasksel and selected lamp [14:49] OK, cool. [14:49] From there the MPM settings are always too high so I adjusted those. This is what I'm using http://pastebin.com/VXqEeMQz [14:50] kpettit: Do you see the issue you're having without any customised configuration? [14:50] I think it's very minimal, but who knows. The site I'm doing isn't public, it's a locked down wordpress site with at most 2 users [14:50] kpettit: (Just trying to narrow down what's causing it :) [14:50] yes. [14:50] usually I don't have to mess with apache until a site goes public and starts getting real load. So it threw me off out of the box with just me it would die frequently [14:51] kpettit: Are there any untoward messages in /var/log/apache? [14:51] I don't see anything obvious when I do TOP. And restarting apache always fixes it [14:53] I've never really used Apache in anger, so I'm not going to have a huge amount of useful input from this point onward. :p [14:54] Yeah, usually apache is solid. I've been doing lamp stuff for a decade. Out of the box things normally work. This is the first time I've had it just suck out of the box. [14:54] But I haven't used 16.04 in production yet either so was curious if anybody else has issues or maybe it's just this one VM or something weird on it [14:55] kpettit: is there anything in the logs why it dies? [14:55] I've been looking. It just stops. [14:55] kpettit: either the apache logs or in the journal for apache? [14:55] kpettit: so process gone, and no message anywhere? [14:56] The process is still there. [14:56] it's more like apache gets hung up [14:56] so doing top everything still looks normal. [14:56] Kehet: define "goes down" then. Does it refuse new connections? Or accept and then hang? Or hang before accept? [14:56] kpettit: so it doesn't accept new connections in that situation then? [14:57] yes. I don't have the exact error up. But it doesn't accept anything new for sure. [14:57] I'm trying to re-create error.... [15:17] Hi does anyone know much about php.ini ? I'm using FastCGI and as far as I've read each website such as /home/domainname/ should have their own php.ini file you can set in /home/domainname/etc/php5/ - but when I amend a site's php.ini it still loads the one from /etc/php5/cgi/php.ini [15:17] How do I get Virtual Servers to use their own php.ini files correctly? [15:23] heh? [15:23] ddellav, python-pecan [15:23] each fastcgi php instance loads a php.ini file (the same php.ini) [15:23] you sound like your only using a single fastcgi php instance [15:23] not sure who told you each website has it's own, that is not true [15:24] unless you WORK VERY HARD, to make that the case [15:34] jonah there are a few ways to do it. Usually you'd specify it with a php admin flag in the virtual host definition. I'm not sure if this works with nginx though (you can google to find out). [15:35] also it's not strictly necessary to have a different php.ini for each site, that's only necessary if you need completely different settings for each domain, or if you want to have certain domains more secure than others. This is a typical use-case for hosting companies, to allow customers to have a completely different setting file than other customers on the same box [15:36] but inside php scripts you can set/get ini settings by using the ini_set and ini_get methods [16:05] mdeslaur: it still doesn't work :-( [16:05] ddellav, that won't work [16:05] mdeslaur: this is another oversight. mysql-server-5.7 is a new package when upgrading from Trusty, since it was mysql-server-5.5 (or 5.6) back there. So the maintainer script doesn't treat it as an upgrade. [16:05] php flag in the virtual host does not work for fastcgi [16:05] This is frustrating. [16:06] patdk-wk ok, i thought it would still pick it up but the last time i did it was with the apache module [16:06] the .user.ini file [16:06] but really, that isn't the issue, the issue is any other customer can read the other customers configs [16:06] rbasak: oh, right, duh [16:06] that means he'll probably have to setup different pools for each domain [16:07] unless yo uhave each php running as a different user [16:07] I guess maybe the logic should be "if installing fresh and /etc/mysql/my.cnf.migrated exists, or upgrading and the previous package is prior to 5.7.13-0ubuntu0~, then run fix_old_config_options" [16:07] Not ideal, but it'll work. [16:08] rbasak: can you give me a diff? [16:08] ack [16:10] mdeslaur: http://paste.ubuntu.com/20192824/ maybe? Untested. I'll test now, but would appreciate your opinion. [16:14] rbasak: how about removing the version and just using le instead of le-nl [16:14] le should treat empty as less [16:15] Good point, thanks. [16:15] sorry, s/the version/the empty string check/ [16:20] That seems to have worked. I need to test further though. [16:33] I have some virtual machines running ubuntu 16.04. Of course ye olde eth0 is now ens3. A) What's the proper way to get it back to eth0? There seems to be a udev rule that people are 50/50 on the fact that it may work. or there is a modification to GRUB that can be done. B) Why the heck did this change? [16:34] blizzow: A) net.ifnames=0 on kernel command line; B) because in the case of multiple NICs, it was always racy and broken before. [16:34] ninja'd >.< [16:34] https://www.freedesktop.org/wiki/Software/systemd/PredictableNetworkInterfaceNames/ if you care to read about the new predictable network interface names system [16:34] blizzow: ^ [16:36] the new system for naming nics only seems predictable if I know what bus and where on it the nic is located [16:37] This is indeed true. The old system for naming nics was only predictable if you only had one. [16:37] or didn't change them around.. [16:37] There are other modes available I believe. [16:38] rattking: technically, no, even if you didn't physically alter the order, you weren't guaranteed the enumeration was the same [16:38] rattking: so if you never saw that bug, jsut consider yourself lucky :) [16:38] It would be nice if in the case of one NIC it defaulted to not doing this, but I didn't ask for this because AFAIK it is impossible to determine if you have only one NIC because hotplug. [16:39] yeah, it's not just having one NIC, but forever only having one NIC :) [16:39] heh I definitely had the drive enumeration problem before uuid's but for NICs on servers I guess I have been lucky [16:40] rattking: yeah, it's a very similar problem to the drive enumeration issue; it also is a "good thing" in that it makes the user interface (even if an ugly naming) not dependent on kernel naming (which eth* was) [16:40] rattking: but yeah, i've seen it a few times with NICs on big machines ... debugging that was no fun :) [16:47] mdeslaur: looks good with that fix. I'm just writing up. [16:58] mdeslaur: I've written up my testing in https://bugs.launchpad.net/ubuntu/+source/mysql-5.7/+bug/1571865/comments/16 [16:58] Launchpad bug 1571865 in mysql-5.7 (Ubuntu Xenial) "mysql fails to start after upgrade if previous defaults were customised" [High,Fix committed] [16:58] mdeslaur: so on top of your version string match fix, we also need to s/le-nl/le/ on that line. [16:59] rbasak: ok, I'll upload a new package to the ppa, thanks! [17:00] mdeslaur: thank you for working through this with me. Do you need anything else from me? [17:00] rbasak: nope, I should be good. Thanks! [17:00] mdeslaur: OK. FYI, I'm EOD now. I'll keep an eye on IRC but will be slow to respond. === cmagina is now known as cmagina_lunch === iberezovskiy is now known as iberezovskiy|off === arooni__ is now known as arooni === degorenko is now known as _degorenko|afk === seventy_ is now known as seventy === Golbat is now known as GitGud [23:09] can anyone recommend a mysql server backup tool? i'm using `automysqlbackup` now but having problems using NFS mount as the target. we don't have a huge setup, but still the database server locks up for too long and the various CMSs and apps using it lose their connection [23:11] House: can you save the output locally and then move it over nfs later? [23:13] sarnold : yes, automysqlbackup has a nice option for pre- and post-run commands, so i'm using an rsync there to shift it over, but while i'm looking at the various issues we have with the current setup, i thought i'd get recommendations for alternatives too [23:13] House: aha :) good thinking. [23:15] i love rsync. previous job had a network admin who thought rsync initiated by the remote to central server was the only way to move scheduled traffic. so, backups were pushed into the DC, and data bundles were periodically (like every 5mins) polled for and retrieved when they existed... love/hate relationship with rsync, but it's now grafted to my soul and cannot be ignored! [23:16] haha. I know the feeling... [23:19] house, that is the issue with using it [23:19] use it on a slave mysql [23:19] use lvm snapshot and backup from that [23:19] or use percona xtrabackup [23:22] lvm snapshot with a database? o_O [23:36] sarnold, why not? [23:36] isn't that the whole point of lvm snapshots? [23:36] used it years ago, when that was the only option [23:37] yeah but you've normally got to bring the database to a quienscent state so you don't snapshot garbage [23:37] hmmmm http://dba.stackexchange.com/questions/18017/how-to-create-snapshot-backups-in-mysql [23:37] flush tables with lock on mysql [23:37] snapshot lvm [23:37] unlock tables [23:37] then copy/backup/... [23:37] sarnold : link shows flush+lock, snapshot, unlock [23:37] but these days, just xtrabackup :) [23:38] House: it's certainly encouraging that what you found matches exactly with patdk-lap's advice :) [23:38] poor percona, they had such grand dreams, and now they're known primarily for the backup tool? heh [23:38] so patdk-lap, you're cloning a filesystem snapshot of a quiesced database server's data volume, out to a backup location? [23:38] no [23:38] then i presume releasing the snapshots immediately or some time in the future [23:39] I did that when I wanted to make a new slave, like 10years ago [23:39] If I make a change to /etc/network/interfaces, it seems like a 50/50 chance that the change takes place immediately with no 'services networking restart'. Is there some newfangled, proper way to give a server a static IP? [23:39] ah, k [23:39] blizzow: "service networking restart" isn't safe, and should report an error ... [23:39] blizzow: ifdown and ifup after making the changes [23:41] patdk-lap, sarnold: this approach is quite similar to vmware+vdp/veeam with windows VSS. I'm not sure how well it works in linux guests. (i had poor experience with dell/equallogic integration with rhel6) [23:41] vmware supports it for linux [23:41] you just have to setup the freeze and unfreeze scripts to sync/flush the filesystems properly that you are using [23:42] personally, anything that doesn't flush properly to begin with, is improperly designed [23:44] sarnold: that's the problem, I copy a new file into /etc/networking/interfaces, and half the time my connection dies instantly and is changed over to the new config, the other half it stays and I should expect to reboot or do the ifdown ifup dance. [23:46] so I'm wondering if there is some new accepted stable way to change IP addresses. [23:57] blizzow: i think it's rather unexpected that an "immediate" change to eni takes effect.