[01:03] karstensrage, that is not a very safe thing to do with security [01:20] patdk-lap, what do you mean? [01:21] you are not matching the whole string, only the start of the process name [01:38] so why is that bad? patdk-lap [01:39] whatever that check is doing, can be bypassed by using a process starting with the same name [01:52] well its an nss library like nss_ldap [01:52] and those processes are the ones that open the library but dont do anything with it [01:53] or close it [01:53] so that code is necessary to short circuit out if those processes open the library [01:53] so if there is process that has the same starting name, i guess i would want to short circuit out as well [01:54] this same problem is apparently with nss_ldap [01:54] but they handled it differently [01:55] debian sure makes things painful [01:55] karstensrage: was just about to say, nss-ldap doesn't seem to have any of those process names hard-coded in it; what did they do differently? [01:56] tarpman, the work around afaict was to set a flag to either do a strong connection or a weak connection to the ldap server, in the former case keep trying after a failure, in the latter, abort if it fails the first time [01:56] i dont have that luxury [01:57] i hate this way of doing it btw with the hardcoded names [01:57] but im not seeing a good way around it [02:00] tarpman, basically if you google "dbus nss_ldap" you can find all the discussions about the troubles nss_ldap had [02:01] it was really hard to narrow it down to dbus, but once i did that, i was able to put in the right debugging to see these processes and filter them out [02:01] this is ringing some bells now... some of these bugs look very familiar [03:06] tarpman, any other suggestions? [03:11] teward: I think you can set DEB_BUILD_MAINT_OPTIONS=hardening=-pie or something like that. [03:11] teward: https://wiki.debian.org/Hardening#dpkg-buildflags [03:34] karstensrage: in your position I'd be trying very hard to detect the "network unreachable" state from my module... that wasn't possible for libnss-ldap since libldap hides the network state behind the LDAP result code === nchambers is now known as hammwch === JanC_ is now known as JanC [07:41] is this the right channel to ask for assistance with data recovery with Linux Raid 5 / LVM ? [08:10] jakst: it is one channel to ask - there is no one specific to ubuntu + lvm/raid [08:11] jakst: you can still go on to the wider community in #ubuntu if you find no help in the more server specific group around here [08:12] cpaelzer: Of course, just wanted to make sure I wouldn't be chased away with torches and pitchforks for asking here :) Already tried #ubuntu but didn't get much of a response [08:15] jakst: it maybe is still a slow satrt of the year [08:15] Well, I'll give it a shot [08:15] The thing is, my physical and logical volumes have disappeared from LVM, and my raid array has status 'active, degraded, not started' and reports the wrong size [08:16] jakst: did any of the links that were linked there help you already ? [08:16] No, not really [08:17] jakst: if all is gone (pv and lv and likely also vg) you have to start looking bottom up [08:17] jakst: so #1 are the raw devices like /dev/sd... still there? [08:17] jakst: from there go on with pvdisplay, maybe pvscan ... to find your pv's - and from there to vg and lv and so on [08:17] I can see them with fdisk -l [08:18] jakst: the question is where it breaks [08:18] jakst: ok so disks are there - and for the moment we assume they are intact [08:18] jakst: you said LVM / Raid before - is it only LVM or is also an md involved? [08:18] cpaelzer: pvdisplay and pvscan display nothing [08:18] yes [08:19] stacked which way - are the pv's on the md array - or have you made a md array out of lv's ? [08:19] cpaelzer: /dev/md0 consists of raid5 of sdc sdd and sde [08:20] jakst: ok and the pv(s) is on /dev/md0 to shape off lvms from there right? [08:20] jakst: is cat /proc/mdstat still happy about /dev/md0? [08:20] Good morning, happy new year! [08:20] good morning and year lordievader [08:21] Happy new year! :) [08:21] https://www.irccloud.com/pastebin/JcWed20J/ [08:21] This is /proc/mdstat [08:21] jakst: ok, so not lvm is broken (maybe it is later) but your md is down [08:23] Yeah that seems to be the case [08:23] jakst: http://superuser.com/questions/603481/how-do-i-reactivate-my-mdadm-raid5-array [08:23] jakst: that should get you to activate it again [08:23] jakst: there are also commands to gather status on each member disk and such [08:23] jakst: I'd do so and store that away before starting/assembling it [08:24] Do you mean mdadm --examine /dev/sdc etc? [08:24] jakst: and all other raid devs [08:25] jakst: I like to store debug info before changing something [08:25] Thanks for the tip! [08:25] jakst: and then likely go with [08:25] jakst: mdadm --stop /dev/md0 [08:25] jakst: mdadm --assemble --scan -v [08:25] jakst: and let us know if it worked or why not if not [08:26] jakst: the linked example has a case with out of date disks and uses force to reenable, but most of what follows depends so much on your case that you have to decide (e.g. if force is ok) [08:28] https://www.irccloud.com/pastebin/2FlDN3O1/ [08:28] So this is the output of assemble [08:29] jakst: that is a good start [08:29] jakst: as I read it it means it could reassemble the state and currently syncs up one of your devices [08:30] your /proc/mdstat should show it syncing with an ETA [08:30] jakst: after that you should be able to start it [08:30] jakst: what does proc/mdstat show now? [08:30] https://www.irccloud.com/pastebin/nMMfc1Ir/ [08:31] jakst: also the state of the examine output should have changed now - the disks are now part of an array [08:31] jakst: there is something like "Device Role" at the end of examine [08:31] hrm - does that mean they are all as spares (S) [08:31] need to check [08:33] cpaelzer: Device Role is the same as before, Active device 0, 1 and 2 [08:34] jakst: it very likely just needs the --force, but it is your data so I'm refusing to just say you should do so [08:34] jakst: do you have enough spare storage to dd away the raw disk content before you do so? [08:34] No, I don't [08:34] what does force do? [08:35] jakst: essentially it starts it anyway referring to the last line in https://www.irccloud.com/pastebin/2FlDN3O1/ [08:35] jakst: from the bit I see in your case it is 98% fixing your issue, but 2% killing your data - that is why I need you to make the call [08:37] jakst: "if you search for "assembled from 2 drives and 1 rebuilding - not enough to start the array while not clean - consider --force" the net is full of recommendations to just do it [08:37] Well I don't have enough space to backup, and it's not ultra critical to recover. Just very very nice if it works [08:38] jakst: so do the assemble with force, then start it [08:38] It says my devices are busy -.- [08:38] jakst: it should be in recovery mode then [08:38] jakst: stop before reassemble [08:39] Ok, but should I assemble manually? Don't know what it get sr0 from and suh [08:40] Nvm that [08:40] Now I forced it. Should I just mount it now? [08:41] jakst: now that you forced the assemble you should madam start it and check /proc/mdstat [08:42] is that mdadm -A /dev/md0? [08:43] jakst: assemble might start it automatically - it is too long ago since mine just works for years now [08:43] jakst: what does /proc/mdstat show now (before searching for a start command that might not exists) [08:44] https://www.irccloud.com/pastebin/8LzgkVch/ [08:44] Recovering [08:44] jakst: good [08:44] jakst: when that happened to me it was the day to read about upgrading to raid6 for the day two disks will break :-) [08:45] jakst: you can use it now, after the recovery is done it will provide the extra level of failsafe again [08:45] Haha yeah, a lot of thoughts about upgrading have been passing through my head [08:45] jakst: I waited to be recovered before using it thou [08:46] cpaelzer: Yeah I'll just check if it mounts properly, then I'll leave it to recovering [08:46] jakst: in your case pvscan might be the next [08:46] jakst: as you have pvs on the md [08:46] jakst: and then vgscan, lvscan, mount [08:49] cpaelzer: Well it appears in pvscan, but without a volume group [08:50] jakst: it apears without vg in pvscan because the vg isn't active I think [08:51] It's supposed to belong to vg group0 [08:51] I think. Was a while since I set it up [08:52] jakst: so pvdisplay shows your pv's [08:52] jakst: but vgdisplay shows nothing - not even inactive? [08:53] vgdisplay shows my volume group, but only cointaing a caching disk that I never bothered to activate [08:54] jakst: and vgscan is not re-finding your pvs now? [08:54] Nope =/ [08:55] jakst: sorry I'm out of remote-usable-skills now I guess [08:55] jakst: has the pvdisplay all your pv's at least? [08:57] cpaelzer: pvdisplay shows md0, but not the individual drives [08:58] jakst: That makes sense? Right? [08:58] Ja [08:58] Whoops [09:00] lordievader: Well I think I recall that each drive was listed under pvs [09:01] jakst: if you did pvcreate on /dev/md0 you will only see /dev/md0 in pvdisplay [09:01] For mdraid perhaps... but if you layer lvm on top of mdraid you won't see all drives in pvs/pvdisplay. [09:01] jakst: the member disks are no more to be accessed directly or you will kill your raid [09:01] ^ that. [09:01] lordievader: ack [09:02] If you'd let LVM do the raid5 then yes, you'd see all disks. [09:03] Ok, but trying to mount the array I get 'mount: wrong fs type, bad option, bad superblock on /dev/md0' [09:04] jakst: You put lvm on the mdraid right? [09:04] LVM ain't a filesystem ;) [09:05] lordievader: No I guess I haven't. How would I do that without destroying the data? [09:06] jakst: What is the output of 'sudo pvscan && sudo vgscan && sudo lvscan'? [09:06] that ^ [09:07] https://www.irccloud.com/pastebin/v8VXxrij/ [09:07] sdb is the device I was meaning to use as cache, but never did [09:09] Hmm, md0 contains a PV signature but is not assigned to any volume group? [09:09] Before the crash I had a logical volume called data [09:09] Heh, yeah [09:10] jakst: Could you pastebin the output of 'sudo lsblk -o NAME,KNAME,FSTYPE'? [09:11] https://www.irccloud.com/pastebin/0oD8Q8cw/ [09:14] jakst: it will likely just complain not knowing about "data" but what does this give you?: "vgchange -ay data" [09:14] Yeah not found [09:15] Sda contains rootfs I presume? [09:15] jakst: sudo vgcfgrestore --list data [09:15] yes [09:16] sudo vgcfgrestore --list data [09:16] No archives found in /etc/lvm/archive. [09:16] :-/ [09:16] But if I ls that directory I can see them [09:16] ? [09:16] https://www.irccloud.com/pastebin/4xEbEngI/ [09:17] jakst: well you have backup of the group0 cache, but not of a data vg [09:17] jakst: I slowly lean to assuming you once had a data lvm, but stopped using it a while ago [09:17] My system was up and running before new years [09:18] jakst: What happened that you lost it? [09:18] Power outage? [09:18] Might have been, not sure. I was away [09:19] But I also might have messed it up in my early rescue attempts [09:19] I just checked your former pastes - since /dev/md0 is a proper PV it was used as PV - I wonder why it would auto-backup the cache but not the data config [09:20] But group0 countained the lv data, so should be correct right? [09:20] data wasn't it's own group [09:21] jakst: are the files in /etc/lvm/archive human readable - and if yes is data in there? [09:23] jakst: Is the data lv defined in /etc/lvm/backup/* [09:23] ? [09:24] not in backup, but in archive [09:24] # Generated by LVM2 version 2.02.98(2) (2012-10-15): Wed Jul 15 12:27:07 2015 [09:24] contents = "Text Format Volume Group" [09:24] version = 1 [09:24] description = "Created *before* executing 'lvcreate group0 -L20M -n dataCacheMe$ [09:24] creation_host = "NAS" # Linux NAS 3.16.0-43-generic #58~14.04.1-Ubuntu SMP Mo$ [09:25] but that seems only to be the cache device [09:25] or came more before flood control kicked you [09:25] jakst: Could you pastebin that file? [09:26] https://www.irccloud.com/pastebin/xrkslaQV/ [09:26] Yeah, I accidentally pasted raw :P [09:27] Hard to copy long texts from console... [09:27] nice, it really has a backup [09:27] not sure but you might be able to reload that with vgcfgrestore [09:30] I could! [09:33] And it mounted!!! My data is back!!!! [09:33] yeah [09:33] Whoop whoop [09:33] gz jakst [09:33] jakst: Nice [09:33] Love you guys cpaelzer lordievader [09:33] jakst: What was the actual command you used to restore the backup? [09:34] sudo vgcfgrestore -f /etc/lvm/archive/group0_00008-621465970.vg group0 [09:34] Ah, cool. [09:34] Thanks [09:35] I really couldn't have figured that out on my own, and I already spent a whole day trying [09:35] Now I learned a lot as well! Thanks :) [09:36] your welcome [09:36] So, futureproofing.... Raid6. Anything else? [09:37] I'd do the raid in LVM, but that is me ;) [09:38] What's the upside? [09:38] More flexibility. LVM uses dmraid, like mdraid, but does so per LV instead of per disk. [09:39] So you can determine per LV if you want linear, raid0, raid1, raid-whatever. [09:39] jakst: also maybe share your won insight in something like http://askubuntu.com/questions/13981/recover-lvm-after-hdd-crash or a new post [09:40] cpaelzer: Absolutely, I'll do that! [09:40] lordievader: Okay, sounds nice. I'll have to look at that when I get more disks [09:48] i need to set up sendmail on my ubuntu xenial server. the server just needs to send emails to users, it doesn't need to receive anything. i found this guide on DO, whose guides i've found to be excellent in the past, but this one seems a little more confusing to me https://www.digitalocean.com/community/tutorials/how-to-install-and-configure-postfix-as-a-send-only-smtp-server-on-ubuntu-16-04 [09:49] particularly, i'm confused about hostname settings. should it even matter if all i'm doing is sending email? === chmurifree is now known as chmuri [10:08] ghostal: unless you're using a service provider's email relay, if your hostname doesn't resolve to your source IP (and in reverse) then many hosts will block your emails for spam. [10:10] rbasak: well, i'm not using a relay, i know that much :) [10:10] my hostname is just "mir" [10:14] but there is a DNS a record for the machine === petevg_holidays is now known as petevg === jgrimm-holiday is now known as jgrimm [14:10] dasjoe: hmm, I think it's supposed to default to the last LTS === PaulW2U_ is now known as pcw === med_` is now known as medberry === medberry is now known as med_ === med_ is now known as Guest46232 === Guest46232 is now known as medberry [15:30] coreycb: ping can you update your upstream report please? [15:30] zul, that's in progress, did you see I moved that btw? [15:31] coreycb: yeah im using the new location [15:31] zul, ok [15:32] zul, i'm working on barbican update-excuses failure. waiting on a s390 instance to debug the neutron autopkgtest failure. [15:33] coreycb: ack [15:40] zul, do you have an MIR open for monasca-statsd? === wwalker_ is now known as wwalker [15:40] coreycb: no there needs to be one i think === bladernr is now known as bladernr-doc [15:41] zul, ok i'll open one [15:43] coreycb: k [16:31] anyone ever used chrony in ubuntu before? I'm trying to query a chrony client on my network as "chronyc -h 192.168.1.22 tracking" but I'm wondering if it needs to be allowed first inside chrony.conf [16:31] allowed ? [16:31] if you're specifying it on the command line it won't take that parameter from the config [16:35] ikonia: chrony operates as an ntp client by default, if I allow a host inside chrony.conf then it becomes a server for that client (if it needs to) but I just want to query it for skews [16:35] I'll test it [16:36] jge: right, but you're specifying -h on the command line so it won't care about that option in the host [16:36] (host config) [16:36] ikonia: I'm querying the remote server from another host in the network [16:37] jge: yes, I understand that, [16:37] jge: however the fact that you're setting -h on the command line replaces that parameter from the config [16:37] same thing as an "ntpq -p [16:38] but the other end needs to allow the connection [16:38] no? [16:38] jge: so you're talking about the config on the remote servcer it's querying [16:38] rather than the client [16:38] yep [16:38] jge: ok, so yes you'll need to tell it to allow queries [16:39] yeah I did, let me test it [16:39] never worked with chrony so I wasnt sure [16:39] jge: works %80 the same as ntp [16:41] yeah the guy who has it running here swears by it [16:41] "it's so much better than ntpd" [16:41] but no explanation as to why he thought that.. had to look it up. [16:42] I'm not sure why it's "better" [16:42] I've found it "fine" but nothing to write home about as a big song and dance [16:42] I don't see any real world benifit over ntp [16:50] Hello. I'm attempting to mount a filesystem from an iscsi server. My fstab has the _netdev option for the filesystem and it is using UUID. The problem is that during the boot sequence iscsi-open start script hasn't run at the time the system attempts to mount the disk. How do I get iscsi-open to run after network start and before the mounting of filesystems? [16:54] ctjctj: i wonder if you need to include iscsi into your initramfs [16:55] nacc, no. I'm not booting off an iscsi disk. [16:55] UUID="xyzzy" /var/lib/mysql defaults,_netdev 1 1 [16:56] So we boot of a local disk and then we should mount the iscsi disk before mysqld (mariadb) starts [16:56] ctjctj: ah sorry [16:56] nacc, it was a great answer, just not the one I needed. [16:56] ctjctj: 16.04? [16:56] 14.04 LTS [16:57] ctjctj: hrm, so maybe an upstart ordering is needed? [16:59] I thought that. But we have S45open-iscsi in rcS.d which I *think* means to do this before we change out of single user mode and into a multi-user runstate. [17:00] My understanding was that by putting the _netdev it would cause the mount of network devices to wait until after open-iscsi completed. [17:03] zul, monasca-statsd is optional so I added it to suggests [17:05] zul, upstream report is updated now too for ocata === medberry is now known as med_ [17:26] ikonia: that worked but now I'm getting "517 Protocol versin mismatch" not much about it online.. the client querying is running Ubuntu and the other CentOS.. wondering if this is the problem [17:27] shouldn't be [17:27] ubuntu has chrony version 1.29 and Centos 1.29.1 [17:28] I have multiple distros using it with each other [17:29] ikonia: looking at source here https://github.com/SuperQ/chrony/blob/master/client.c [17:30] something to do with a bad header? [17:30] jge: not sure, I'll need to look into it, but it works on mine [17:31] ikonia: are you able to query other clients as "crhonyc -h ip tracking" [17:31] ? [17:32] jge: I can't check it at the moment as I don't have access to those hosts from where I am [17:32] hmm ok. [17:32] I don't know then :( [17:33] it would be nice to have a switch for verbose [17:35] I can try it for you later on [17:56] coreycb: cool beans [18:13] beisner, hi can you promote python-cryptography 1.0.1-1ubuntu1~cloud1.2 to liberty-proposed please? [18:27] coreycb, done, re: https://bugs.launchpad.net/horizon/+bug/1601986 [18:28] Launchpad bug 1601986 in OpenStack Dashboard (Horizon) "RuntimeError: osrandom engine already registered" [Undecided,New] [18:28] beisner, ty sir [18:29] coreycb: mind if i sync python-muranoclient over from debian? [18:29] zul, fine by me [19:02] How would I go about doing a smart scan of a disk if my ubuntu server is hosted in an ESXi hypervisor? In Ubuntu or in ESXi? [19:28] How do I force open-iscsi to start before network mounts? At this point I have a _netdev in fstab for the disk in question. open-iscsi attaches the device correctly when it runs but upstart/systemd(?) are attempting to mount the disk before open-iscsi starts === Spydar007 is now known as Guest70417 === bladernr-doc is now known as bladernr [21:38] beisner, hey these are ready to promote from liberty-proposed->liberty-updates: cinder, heat, manila, nova, openstack-trove, sahara [21:48] I'm looking at an issue that /etc/init/mountall-net.conf will attempt to mount devices that are attached via open-iscsi. But mountall-net.conf runs before open-iscsi runs. Is there a fix for this? [22:26] hey all, could I install a version of a package that's meant for say Xenial to Trusty? [22:27] jge: that's not recommended or supported [22:29] jge: no [22:29] so I'm better of intsalling from source if I need a version that's not available in repos? [22:31] jge: what do you actually want [22:31] ikonia: chrony version 2.3 [22:31] ttp://chrony.tuxfamily.org/doc/2.3/manual.html#Installation [22:32] jge: why do you want that version ? [22:32] jge: 2.3 is not available in xenial either, afaict [22:33] win 12 [22:33] oops [22:33] ikonia: I'm getting an error trying to query another chrony client in Cent0S, "Read command packet with protocol version 5 (expected) 6" and from the mailing list here it looks like it might be related to the version: https://listengine.tuxfamily.org/chrony.tuxfamily.org/chrony-users/2010/06/msg00005.html [22:34] so I wanted to test if upgrading to the latest release will help [22:34] I've got 16.04 and Centos 7 hosts in sync from each other [22:34] what version of chrony on both? [22:35] saly, I can't check as I ended up not going home tonight [22:36] I was able to test earlier from different clients (one 1.29 and the other 2.2) both CentOS and it worked, so I'm thinking is the version of Ubuntu.. [22:37] jge: what version does ubuntu use [22:37] it appearently sends protocol version 5 when the other ends expects 6 [22:37] what actual chrony version does ubuntu use [22:37] (not got a box here to check) [22:37] ikonia: it's on 1.29.1 which is the latest stable [22:37] jge: so applying logic, you have a 1.29 box working and 2.2 box working [22:38] i don't think a 1.29.1 "won't" work, when a 1.29 box does [22:38] same OS though [22:38] jge: so ? [22:39] well, I'm thinking it might be implemented differently.. it's clearly sending a different version of the protocol [22:39] so if you think it's a different implmentation, upgrading it won't do anything [22:39] if it would be the same code base then it shouldn't complain [22:39] jge: have you actually looked at the config or arguments to see if things can be set [22:40] jge: it is the same code base [22:40] you've just said that [22:40] you have a 1.29 client that works [22:40] 1.29.1 is the same codebase [22:41] my idea with upgrading is that the latest release could have better (compatability) with earlier versions as opposed to the opposite [22:42] jge: sorry, thats just blind randomess [22:42] maybe downgrade connection protocol, I don't know ..just spitting ideas [22:42] jge: have you even done basic research to see if the clients support both versions of the protocol [22:42] and if you can force the protocol, and what the default is [22:43] i looked up focing the protocol but manual doesn't have anything for that.. [22:43] client obviously does not support one of the protocols [22:45] why though [22:45] as it's in the code base [22:45] logically it's more likley to be a configuration option [22:47] ikonia: https://github.com/mlichvar/chrony/blob/master/NEWS [22:48] check out the security fix under version 1.29.1 [22:48] incompatible with previous protocol version.. [22:48] there you go then [22:48] so you need to use the other protocol [22:48] but would that be referring to 1.29 or 1.28? [22:49] would what ? [22:49] previous protocol version [22:49] so 1.29 seems to support both [22:49] 1.29.1 seems to patch one to fix a problem [22:50] so the logical approach is to use the one that is supported by both [22:50] how to foce it the question [22:50] if you look there is a similar change in 1.27 [22:51] hm yeah I see it === hammwch is now known as nchambers [22:53] ikonia: I dont have chronyd open on the internet, maybe I could just go back to 1.29 [22:54] wait a minute, I was looking at another box... the ubuntu box is already on version 1.29 [23:18] For anybody that cares about the open-iscsi mount on boot issue I was describing. When we went to upstart we created a helper tool called "mountall" which processes fstab and mounts drives as they become available. Once upstart starts the network /etc/init/mountall-net.conf runs and kills the mountall process. BUT /etc/init.d/open-iscsi start has not yet run so any iscsi targets have not yet been mounted. Thus the mount [23:18] fails and boot hangs. The original intention was for the _netdev in /etc/fstab to keep any mount of the iscsi device from happening. All the other remote devices would then be mounted by commands like "mount -a -t nfs -O _netdev" Thus /etc/init.d/open-iscsi also does a "mount -a -O _netdev" because it runs after all of NFS/CIFS and such. Catch 22. [23:21] Any built-in way with systemd to have an escalating set of shutdown commands for a service? Specifically, I have a VirtualBox VM set up as a service, and I'd like it to first try VBoxManage controlvm $vmname acpishutdown, and (perhaps after a timeout) try poweroff instead of acpishutdown if the process hasn't halted. [23:22] Hi! is it possible to compile squid into a package such that no shared dependency exist? [23:23] * keithzg has tried to read systemd documentation, but for instance https://www.freedesktop.org/software/systemd/man/systemd.unit.html doesn't even *mention* ExecStop, much less document it. [23:30] keithzg, i believe there is a method. The easiest that I can think of is to just have two shutdown VM commands. One does the acpishutdown and waits upto 30 seconds Then the second VM shutdown runs and does the poweroff. Since all VMs that could be shutdown with acpishutdown will already be shutdown this only catches the once still on. [23:32] ctjctj: Fair enough, I was thinking perhaps there was some native systemd way of doing this but that certainly sounds like it'd work. I'll try just using `/usr/bin/VBoxManage controlvm Sibrel acpipowerbutton && /bin/sleep 30 && /usr/bin/VBoxManage controlvm Sibrel poweroff` [23:48] how would i soft link a path like /Volumes to my main path /media/dumpebut/ i need it to see the soft link path in a script.