[01:33] Could anyone help me? I'm experiencing a problem similar to what is described here: [01:34] http://lists.freebsd.org/pipermail/freebsd-questions/2004-September/058852.html [01:34] But my ubuntu 12.04 LTS server doesn't seem to have an /etc/rc.conf file [01:34] So I don't know where to implement the answer given in that article or even if that answer is relevant on my version of Ubuntu Server [01:36] shawn1: you're getting "permission denied" when you're trying to ssh to a specific server? [01:36] yes. Let me copy and paste my own outputs [01:37] Shawns-MacBook-Pro:~ shawnshipley$ ssh -vvv 192.168.1.255 [01:37] OpenSSH_5.6p1, OpenSSL 0.9.8r 8 Feb 2011 [01:37] debug1: Reading configuration data /etc/ssh_config [01:37] debug1: Applying options for * [01:37] debug2: ssh_connect: needpriv 0 [01:37] debug1: Connecting to 192.168.1.255 [192.168.1.255] port 22. [01:37] debug1: connect to address 192.168.1.255 port 22: Permission denied [01:37] ssh: connect to host 192.168.1.255 port 22: Permission denied [01:38] shawn1: I have a wild guess that your laptop and your server do not agree on netmask or their network address [01:39] how can I fix that? [01:40] shawn1: you'll have to find the correct settings to use, probably from your router, and then ensure that your server and your macbook both have the same settings for netmask, broadcast, and so forth [01:40] shawn1: (the 'permission denied' here is probably because you're trying to ssh to a broadcast address, for a network 192.168.1.0 and netmask 255.255.255.0 (a /24)) [01:41] shawn1: if you manually assigned an IP address for your server, don't use .255. :) [01:42] I didn't manually do that. Why would that present a problem, though? (just so that I understand for future reference). [01:43] shawn1: normally the 'last' address in a network range is a broadcast address that can be used to contact all hosts in a subnet. but it only works with UDP and ICMP, not TCP, and it requires superuser privileges, and most hosts don't respond to broadcast ping requests any more anyway :( [01:44] shawn1: so, if your network is 192.168.0.0 and the netmask is /16, the broadcast address would be 192.168.255.255; if the network is 192.168.0.0 and the netmask is /24, then the broadcast is 192.168.0.255. [01:45] .255 is the global address, .5 is the local address, but it won't ping to .5 [01:45] shawn1: it's not always 255, that's just convenient with /8 or /16 or /24 netmasks -- but it does stand out clearly here :) hehe [01:45] This is an odd question, but is your name Sarah Arnold? [01:46] shawn1: indeed no, seth arnold. hehe. :) [01:46] I know a Sarah Arnold, so I was just wondering [01:47] My question: Do you know a way that I can completely reset the router? [01:47] I configured it in windows and then later had to reinstall windows and it would only let me connect as a guest. [01:47] shawn1: depends on the router. often holden the power button for twentyseconds or something will zero the memory. [01:47] It'll completely reset everything and let me reconfigure? [01:50] Any ideas on why wlan0 wouldn't start on boot when configured in interfaces file but does with 'ifconfig wlan0 up' [01:50] shawn1: quite often; it varies from vendor to vendor, but that's a good first shot. hehe. :) [01:51] ag763: pastebin your interfaces? [01:55] sarnold, http://pastebin.com/1CGSyeQK [01:57] ag7631: do you need a module loaded first? maybe add the module name to /etc/modules? [01:58] ag7631: hope that helps, it's time for me to bail :) have a good night, have fun [01:58] shawn1: any luck with 20-seconds? :) [01:58] sarnold, I would expect to see a dmesg or syslog error for that right? [01:59] ag7631: hrm. dunno. seems plausible though :) [02:01] sarnold, looking through both I don't see anything. I can start it right after booting without any issues, seeing nothing in dmesg or syslog I'm at a loss. [02:04] Netgear can be frustrating.... [02:04] Okay. I think I found what I needed === Lcawte is now known as Lcawte|Away [02:16] the netmask address matches [02:18] I was making a very dumb mistake! [02:18] I was accidentally trying to ssh the broadcast address [02:19] I'm connected remotely [02:20] VICTORY!! [02:20] (you can tell that I'm new at working with servers) [02:42] Thanks, Sarnold [02:55] how do we ceck the health of a filesystem? [03:00] anepanaliptos: fsck /filesystem [03:00] a good practice is umount that filesystem first [03:00] and you fsck the partition [03:00] what can do you do over ssh [03:00] ? [03:00] is there anything just to show the health 'status' so to say? [03:02] well, fsck is the most secure way to see if a filesystem is health or not [03:02] do you have any kind of read/write errors or something? [03:03] well i just ran the server too many times on/off the harsh way [03:03] and i just want to make sure the disks are ok [03:03] anepanaliptos: uhmm, fsck all th way [03:04] but , yeah, its really importat unmount those filesystem first [03:04] yeah i once did it mounted [03:04] and also there is a way to force fsck before boot in the next reboot / power on [03:04] made everything brand new [03:04] o yeah? how's that? [03:04] uhmm .. i didn't recall, but let me search for you [03:04] oh no that's ok [03:04] ill rtfm [03:05] im not lazy, just unknowlegeable. [03:05] http://linux.aldeby.org/post/linux-ubuntu-force-fsck-filesystem-check-at-reboot.html [03:05] that web page was written in 2011, info could be old .. but i know there is a way to force that [04:05] I am trying to reconnect a blu-ray player to my router wirelessly and keep get dhcp cannot be acquired. Is there a way to fix this? [04:08] where is dhcp provided from? [04:08] I do not knwo [05:04] can anyone help me? [05:05] I set up port forwarding and made my server IP static in order to connect through SSH to my server over the internet. [05:05] but I don't know what command to use from the client machine in order to access the server. [05:05] ssh [05:05] well, yes [05:06] but to I just ssh my server login name and external ip [05:06] yes [05:06] or do I need to specify port 80 [05:06] ssh youruser@1.2.3.4 [05:06] absolutely not, port 80 is for http [05:06] oh [05:06] well that's the point [05:06] I'm trying to connect over http [05:07] maybe I stated that incorrectly, sorry [05:07] you can't ssh over http as far as I am aware [05:07] I don't know why you would want to [05:07] in fact, quite the opposite is a typical case, http tunneled over ssh [05:08] well [05:08] here's what I did [05:08] in my router settings, I clicked on the 'port forwarding' option [05:08] if you want to SSH from the internet, you need to forward port 22 [05:08] okay [05:09] let me change things here [05:09] and I suggest that you either install fail2ban or configure a limiting firewall to defend against brute force attacks [05:10] well sweet [05:10] that was easy [05:10] okay. [05:10] Folks, Having problem starting Jboss server, it says Jboss home pointing to different installation , Output can be seen in the URL( http://paste.ubuntu.com/5953884/ ) please suggest some solutions [05:10] Thank you very much :) [05:18] shawn1 Can you resolve my problem, any idea on Jboss server ? [05:18] shawn1: Can you resolve my problem, any idea on Jboss server ? [05:19] qman__: Can you suggest something about Jboss? any idea? [05:23] sorry, I don't know anything about it, I try to stay as far away from java as possible [05:27] Sorry, indistylo [05:28] I just got your messages [05:28] and I'm one of those people who are on here for help [05:28] Shawn1: No issues [05:28] in questions related to server issues, qman_ will know much more than me :) [05:29] shawn1: Its alright I am juggling with problem still, Ya i had already asked qman__ : waiting for his reply ! [05:39] Hm, I want to change my home directory but usermod complains about me being logged in (which is true), but I am the single logon-user available on the machine... [05:48] foo357: root. [05:50] anepanaliptos: I connect to the machien through ssh and login with my account [05:50] anepanaliptos: I don't think it's possible to directly login as root that way [05:53] Use sudo -i [05:54] Folks, Eclipse not starting in ubuntu12.04, I installed in /usr/share/eclipse directory and created eclipsed.desktop but its not starting, Kindly suggest solutions [06:05] Ask for help on #ubuntu since that's not a server issue. [06:50] indistylo [06:50] oops [07:02] got it! [07:02] thanks! [07:07] =]=] === thumper is now known as thumper-afk === three18t- is now known as three18ti [08:43] <_ruben> ugh .. why is Azure so limited .. they make it really hard to make linux clusters (no floating ips for HA services, etc) [08:44] <_ruben> need to find something decent to replicate/sync my data between nodes somehow === smb` is now known as smb === Lcawte|Away is now known as Lcawte === Jikan is now known as Jikai === Jikai is now known as Jikan === matanya_ is now known as matanya [11:37] stgraber: Hi, can I ask you something about the isc-dhcp-server6 upstart job? I'm trying to figure out how to fix bug #1186662 [11:37] Launchpad bug 1186662 in isc-dhcp "isc-dhcp-server fails to renew lease file" [Undecided,Confirmed] https://launchpad.net/bugs/1186662 [11:38] geser: sure [11:38] sounds like a regression that would have happened when we switched from our own privilege dropping code to upstream's --paranoia option [11:39] it could be that in the past, dhcpd only dropped privileges after all the files were opened but now it happens a bit earlier, causing the -ENOPERM [11:40] the lease file itself get updated, but it can't get rotated as dhcpd is running as dhcpd but everything is owned by root [11:40] so no updates to leases~ [11:41] ah right, so we probably should just give the dhcpd user ownership of the dir and be done with it? [11:41] almost, with chown dhcpd it gets a little bit farther till: dhcpd: Can't backup lease database /var/lib/dhcp/dhcpd6.leases to /var/lib/dhcp/dhcpd6.leases~: Operation not permitted [11:42] the apparmor profile needs an update too: kernel: [2773234.120934] type=1702 audit(1375788247.840:13): op=linkat action=denied pid=31888 comm="dhcpd" path="/var/lib/dhcp/dhcpd6.leases" dev="dm-0" ino=2757 [11:44] do I need to specify that dhcpd can write (create new files) to /var/lib/dhcp into the apparmor profile? [11:46] I'm surprised the existing apparmor rule doesn't cover that [11:48] there is "/var/lib/dhcp/dhcpd{,6}.leases* lrw" but it seems it doesn't cover creating new files [11:49] looking at the code, dhcpd removes the old leases~ and renames leases to leases~ before it creates a new leases to write into [11:51] Hello [11:52] i need some info, i have 200 servers under my management. Is there any way i can have a key management system, say i want one key to all servers and i dont want to ssh them manualy or in a script, since a server can be now offline and later online [11:52] i need to be able to generate keys and distribute them [12:10] jdstrand: can you help me to understand why the dhcpd apparmor profile issue? I've in syslog: "dhcpd: Can't backup lease database /var/lib/dhcp/dhcpd6.le [12:10] ases to /var/lib/dhcp/dhcpd6.leases~: Operation not permitted" [12:11] and later "kernel: [2773234.120934] type=1702 audit(1375788247.840:13): op=linkat action=denied pid=31888 comm="dhcpd" path="/var/lib/dhcp/dhcpd6.leases" dev="dm-0" ino=2757 [12:11] isn't "/var/lib/dhcp/dhcpd{,6}.leases* lrw" enough to allow it? [12:14] zul, bah - ordering of package build in havana-proposed is creating installability issues [12:16] jamespage: like what? [12:16] zul, "python-keystone : Depends: python-sqlalchemy (< 0.8) but 0.8.2-1~cloud0 is to be installed" [12:17] jamespage: grr.. [12:18] jamespage: hmmmm [12:18] zul, if I rebuilt it now against staging it gets the correct versioned depends [12:18] geser: what version of ubuntu are you seeing this? [12:18] jamespage: right [12:19] jamespage: so we need to rebuild things? [12:19] zul, yeah - I'm just pushing stuff with a ~cloud1 with 'No change rebuild for new version of SQLAlchemy." [12:20] jamespage: ack [12:20] i just uploaded a new version of pbr for trunk packages (as of 5 minutes ago) [12:20] jdstrand: ubuntu server 13.04 [12:22] zul, OK [12:22] geser: /var/lib/dhcp/dhcpd{,6}.leases* lrw is not in the 13.04 profile. did you add it yourself? [12:22] oh, dhcp*d* [12:22] hold on [12:23] hello, i'm trying to get my head around Tar syntax...i got two folders.../mnt/source & /mnt/destination...how can i tar the contents of /mnt/source to a tar file in /mnt/destination without running the command from neither of them? [12:26] jdstrand: for the background: I'm trying to fix #1186662, and got this far after changing the owner of /var/lib/dhcp to dhcpd so that dhcpd can write to it again [12:29] geser: see 'man apparmor.d'. do the source and target files meet the criteria for 'Link mode'? [12:32] geser: it isn't clear to me if the apparmor.d man page is talking about the apparmor permissions or the apparmor+DAC permissions [12:32] geser: I need to call in reinforcements [12:33] jjohansen: I'm not sure what is going on with geser ^ and his quest to fix bug #1186662 [12:33] Launchpad bug 1186662 in isc-dhcp "isc-dhcp-server fails to renew lease file" [Undecided,Confirmed] https://launchpad.net/bugs/1186662 [12:33] jjohansen: backscroll 24 minutes from this timestamp [12:34] jdstrand, jjohansen: does owner of the file matter when linking? dhcpd is run as dhcpd and dhcpd6.leases is owned by root currently [12:34] that's the bit I'm not sure of [12:35] I would think you would see a dac_override entry if that were the case [12:37] geser, jjohansen: I wonder if it has anything to do with 4.2.4-1ubuntu4 and bug #1028526 [12:37] Launchpad bug 1028526 in isc-dhcp "dhcpd failed to start with apparmor denied: capname="dac_override"" [High,Fix released] https://launchpad.net/bugs/1028526 [12:47] roaksoax: y0. [12:51] lifeless: are you still seeing the libvirt memory leak with virt-manager? I was running under valgrind to inspect... but now i can't reproduce it, even without valgrind [12:57] roaksoax: i have afresh pcmk 1.1.10+git here that fixes some nasty bugs. === p0wp0w_ is now known as p0wp0w [13:20] jamespage: https://code.launchpad.net/~zulcss/cinder/babel/+merge/178747 [13:24] jamespage: ping i was thinking....shouldnt we building against the -proposed pocket in the openstack-ci lab since we would be catching stuff much sooner [13:25] like the sqlalchemy stuff [13:51] Madkiss: perfect ill get that synced [14:05] roaksoax: where is your stuff? [14:06] stgraber: http://people.canonical.com/~serge/lxc-resolve.debdiff [14:06] biab [14:08] hallyn: I think it's fine [14:14] I am looking for suggestions for snmp trap software. Currently I am using snmptrapd but I would like to add interface descriptions from switches [14:15] Madkiss: ill put it in github in a bit [14:19] ok [14:20] stgraber: cool [14:27] zul, sorry - was OTP [14:28] zul, thats a really good point [14:28] jamespage: i could do that if you want [14:28] * jamespage thinks [14:32] jamespage/roaksoax: https://code.launchpad.net/~zulcss/cinder/babel/+merge/178747 [14:38] jamespage: typo fixed [14:40] zul, someone synced over your python3 changes in stevedore btw (might have been Daviey) [14:40] jamespage: grrr... [14:41] * zul shakes his fist at Daviey [14:43] jamespage: mind if i push this cinder branch i fixed the changelog entry [14:43] zul, +1 [14:44] jamespage: thnak [14:46] jamespage: the sqlalchemy fix we had in nova got merged fyi [14:53] great [14:59] jamespage/roaksoax: https://code.launchpad.net/~zulcss/keystone/oslo.sphinx/+merge/178778 [15:13] SpamapS: pinger [15:15] jamespage: what did you do to fix the autopkgtests with nova? [15:19] mr hallyn [15:19] are you around sir ? [15:37] smoser: yup [15:37] [ERROR] ./stack.sh:698 nova-api did not start [15:37] guess i'm really not meant to use this on saucy [15:38] i have 2 [recurring] lxc questions. a.) lxc-start-ephemeral minus the ephemeral . b.) lxc clone hooks. [15:38] hallyn, i dont know. i think i used it last on raring fairly painlessly. [15:39] i think my ud-devstack worked start to end last time i used it. [15:39] jamespage/roaksoax: this is really needed to get python-pbr through: https://code.launchpad.net/~zulcss/heat/sqile-fix/+merge/178786 [15:40] smoser: what is ud-devstack? [15:40] stgraber: this is great, and i can also resolve c1.lxc from c2.lxc from there. cool. pushing. [15:44] zul: pongitola [15:45] https://gist.github.com/smoser/4795358 [15:45] SpamapS: any idea when heat is going to switch over to neutronclient? [15:45] hallyn, ^ [15:45] hallyn, see my questions above ? [15:45] i was poinged by juju team. [15:45] they're interetsed in clone being faser [15:45] and those are 2 things that they'd need. [15:45] * hallyn reading up [15:46] smoser: what time, which chan? [15:47] zul: got a bug? I'll grab it and submit a patch now. [15:47] (or file the bug) [15:48] oh i see. weird [15:48] my eyees just totally glazed over that [15:49] hallyn, what? [15:49] oh. [15:49] questions above. [15:49] i have 2 [recurring] lxc questions. a.) lxc-start-ephemeral minus the ephemeral . b.) lxc clone hooks. [15:49] and ud-devstack is : launch instance with --user-data of that 'ud-devstack.yaml' and then wait. [15:49] SpamapS: https://bugs.launchpad.net/heat/+bug/1197208 [15:49] smoser: so those things all work great using the ubuntu-lxc/daily ppa. [15:49] Launchpad bug 1197208 in heat "Migrate Quantum references to Neutron" [Undecided,In progress] [15:49] a. lxc-start-ephemeral minus the ephemral, becomes: [15:49] lxc-create -t ubuntu -n orig [15:49] lxc-clone -B overlayfs -o orig -s -n ephem1 [15:49] b. lxc clone hooks - thye should be there (in ppa), lemme check [15:50] yup, lxc.hook.clone is there [15:50] note the saucy lxc should be merged from upstream git in (iirc) august. stgraber is gonna strangle me soon because i can never remember the dates he has in mind [15:51] hallyn, k. thank you. [15:53] smoser: np, shout if you need more [16:04] jdstrand: I did some more tries: after chowning both the directory and dhcpd6.leases back to dhcpd (after starting it), the leases rotated now without error in syslog. So it looks like it's related to that bug you fixed in the past. (testing is time-consuming as dhcpd rewrites the leases file once per hour (hardcoded)) [16:04] zul: is LP having issues right now? [16:05] zul: can't seem to get to that bug report :p [16:05] SpamapS: not that i know of [16:06] SpamapS: meh...that patchset seemed to got abandoned [16:06] zul: yeah will try to revive [16:07] if launchpad will talk to me. :-P [16:07] hallyn, because i'm that lazy... [16:07] lxc ppa link ? [16:07] smoser: pad.lv/~ubuntu-lxc/daily [16:07] nope [16:07] not that [16:07] https://launchpad.net/~ubuntu-lxc/+archive/daily [16:10] SpamapS: you have to talk to it nicely :) [16:10] hi SpamapS [16:10] zul: and s l o w l y ... [16:11] [1235993.357149] systemd-hostnamed[17620]: Warning: nss-myhostname is not installed. Changing the local hostname might make it unresolveable. Please install nss-myhostname! [16:11] wtf [16:12] stop using fedora ;) [16:26] jamespage: before you go https://code.launchpad.net/~zulcss/heat/sqile-fix/+merge/178786 [16:26] zul, +1 [16:26] jamespage: thanks === alamar is now known as julian === julian is now known as alamar [16:52] jdstrand, geser: so yes file ownership matters. However the link rule unless you stick the owner conditional won't enforce that restriction. Where you will see the restriction is apparmor's stacking with capabilities. Capabilities may require dac_override to access a file with different ownership [16:52] you will see capability messages with an apparmor message, and this will require a capability dac_override, permission in the profile [16:53] jjohansen: the weird thing is dac_override wasn't logged. maybe kernel logging is getting in the way... [16:53] geser: can you do sudo sysctl -w kernel.printk_ratelimit=0 [16:53] smoser: I'm debugging a problem that has cropped up since we started building raring images.... [16:53] geser: and report back your denials? [16:54] smoser: could you see a problem with having this: https://github.com/stackforge/diskimage-builder/blob/master/elements/cloud-init-nocloud/install.d/05-set-cloud-init-sources [16:54] jjohansen: have I mentioned how annoying kernel logging is? :P (/me knows you've mentioned it to me) [16:55] smoser: having trouble debugging because my console is local kvm vga and I can't see what was on it easily. :-/ [16:55] jdstrand, geser: possible, also note the linkat message is not an apparmor message. geser is that file on a different device? you can't hard link across devices [16:56] can anybody tell me why my domain name keeps redirecting to my mail server for? [16:56] jdstrand: kernel logging is an absolute mess [16:57] jjohansen: re linkat> oh, duh-- it isn't :) I'm so used to seeing apparmor denials seeing another kernel denial was not even in my headspace :) [16:57] note, that is not me blaming apparmor-- that is me doing far too much profiling ;) [16:58] something else in the kernel prints denials? [16:58] jdstrand: heh, no its an easy mistake to make, I had to double check it [16:58] sarnold: a few things [16:58] sarnold: 07:11 < geser> and later "kernel: [2773234.120934] type=1702 audit(1375788247.840:13): op=linkat action=denied pid=31888 comm="dhcpd" path="/var/lib/dhcp/dhcpd6.leases" dev="dm-0" ino=2757 [16:59] jjohansen: no, all files are in /var/lib/dhcp, dhcpd runs as dhcpd while the dhcpd6.leases is owned as root (I've chowned /var/lib/dhcp back to dhcpd so it can create the temporary file there again) [16:59] jdstrand: wow, cool, crazy :) never seen a 1702 before. [16:59] sarnold: yeah, me either :) maybe I can be forgiven for thinking it was apparmor then :) [17:00] jdstrand: definitely :) I hope I'd wonder why it feels so short... [17:00] I definitely thought it was a weird looking line [17:01] I even looked around for it in the apparmor docs, then got distracted by my rememberance of dac_override and the previous bug [17:01] anybody know the solution to my problem [17:02] TimR: sorry, you haven't described it in enough detail to even hazard guesses. can you pastebin commands that work, commands that don't work, and what you think those ocmmands ought to do differently? maybe then someone could help.. [17:03] jdstrand: I can only see a dac-override message if I reproduce the state for the old bug (#1028526) [17:03] the next leases should rotate in around 30 min [17:03] Ok I dont see how much more clear I can get with that issue I am having when domain name is redirecting to my mail server [17:04] TimR: maybe pastebin your host or dig output and your zone files? [17:05] domain name and mail server is pointing at the same address [17:06] geser, jdstrand: its the kernel link protections [17:12] jdstrand, jjohansen: when both the directory and the leases file is owned by dhcpd:dhcpd dhcpd doesn't start (old bug), with root:root dhcpd starts but can't rotate the leases file (current bug), when I change the owner back to dhcpd:dhcpd (both dir and file) *after* dhcpd started leases file rotating works (till restart) [17:13] would chowning to dhcpd:dhcpd in the upstart job and adding the dac_override cap to the apparmor profile fix both (old and current) bugs? [17:14] geser: why would you chown the file in an upstart job? That is just papering over the problem [17:15] geser: what is the bug # of the old problem? I want to make sure I am correct in my understanding of the old issue before I answer? [17:16] jjohansen: I guess it was done to ensure the right permissions, see http://launchpadlibrarian.net/111078972/isc-dhcp_4.2.4-1ubuntu3_4.2.4-1ubuntu4.diff.gz for the fix for the old bug [17:16] jjohansen: old bug: bug #1028526 ; current bug: bug #1186662 [17:16] Launchpad bug 1028526 in isc-dhcp "dhcpd failed to start with apparmor denied: capname="dac_override"" [High,Fix released] https://launchpad.net/bugs/1028526 [17:16] Launchpad bug 1186662 in isc-dhcp "isc-dhcp-server fails to renew lease file" [Undecided,Confirmed] https://launchpad.net/bugs/1186662 [17:17] jdstrand: ^ this is a problem, it runs foul of the kernels link restrictions [17:19] geser: so adding capability dac_override to the profile will fix any apparmor induced problems. However that is not the problem here [17:19] we have a conflict betwen dhcpd privilege sep, and kernel link restrictions === andreas__ is now known as ahasenack [17:23] jjohansen: what about the owner of the leases files: dhcpd recreates it after it drops priv as user dhcpd, but dhcpd can't open if for append at startup as root (it happens before dhcpd drops priv). Who should own that file? root or dhcpd? [17:25] eww. sounds like dhcpd folks didn't design their privsep correctly? [17:26] I get slowly that impression too [17:29] sarnold: I agree, they have a broken priv sep design, and the only solutions are [17:29] 1. Fix dhcpd [17:29] 2. turn off kernel link restrictions (which I would consider only a temporary solution) [17:39] geser: you can temporarily fix this by setting /proc/sys/fs/protected_hardlinks to 0, please not this will globally disable kernel link restrictions [17:39] unfortunately this is not something we can control on a per profile or task basis [17:42] jjohansen: my temporary fix for now is to remember to chown the leases file and dir back to dhcpd after I restart dhcpd or the whole server (shouldn't happen too often) till it gets fixed properly [17:42] geser: yeah, that is a more localized fix === ffio is now known as hack === hack is now known as nerd [18:02] jjohansen: you pointed me at backscroll, but I'm not sure what you were pointing to. are you saying what we did for quantal was wrong? === nerd is now known as baba [18:08] jdstrand: I am saying that the dhcpd priv sep patch is in conflict with the kernel link restrictions [18:09] jjohansen: yes, that seems clear now. at the time (quantal), Ubuntu dropped our privsep patch that had worked [18:09] and the new one behaved differently-- and I didn't want to grant dac_override [18:10] funny how this is only coming up now [18:10] yeah [18:11] jdstrand: its likely because of changes made to the kernel link restrictions during upstreaming vs what was in yama [18:11] anyways that is a guess [18:11] huh [18:11] jdstrand: just a guess as to why its surfacing now instead of before [18:12] jdstrand: there where some changes but I'd have to go back to the ml to figure out what they where [18:12] s/where/were/ [18:12] do you both have an idea how to fix it? [18:12] am I interpreting that the current workaround is to chown root:root /var/lib/dhcp/dhcpd6.leases~ if it exists, start dhcpd, then chown dhcpd:dhcpd /var/lib/dhcp/dhcpd6.leases~ after it starts? [18:12] (that would be insane) [18:13] jdstrand: yes or disable kernel link restrictions [18:13] jdstrand: yes, that works for me [18:13] yikes [18:13] please file an upstream bug :P [18:13] jdstrand: I did 'sudo chown dhcpd /var/lib/dhcp{,/dhcpd6.leases}' [18:13] jdstrand: that is because if the process has an open file handle to the file, the restrictions are applied differently [18:15] interesting. I have not looked at the code at all, but it seems fairly obvious that if the lease files are going to be handled as the dhcpd user, oh, I don't know, open them as the dhcpd user [18:15] hrmmm, actually no this one is just doing an ownership test on startup, and then ignoring that in the future === james_ is now known as Guest7355 [18:15] jdstrand: yeah [18:16] jdstrand: dhcpd opens the leases file, drops priv to dhcpd and does later the leases file rotation as part of normal operation [18:17] * jdstrand nods [18:18] geser: it makes the very broken assumption that it can hard link a file it doesn't own as part of the rotation [18:19] I guess upstream assumes that dhcpd can write that file and create files in that dir [18:20] but this conflicts with trying to open the leases file for append as root during the startup phase [18:20] well, it can-- but this is linking files and hardlink restrictions are now part of the linux kernel, so it needs to handle it correctly [18:21] they either need to open as root and rotate as root, or open as dhcpd and rotate as dhcpd, aiui [18:21] again, I've not looked at the code [18:22] http://bazaar.launchpad.net/~ubuntu-branches/ubuntu/saucy/isc-dhcp/saucy/view/head:/server/db.c#L1083 new_lease_file() does the rotation [18:24] geser: I think reporting the bug upstream and employing a short term workaround until they fix it is reasonable [18:24] (we could then cherrypick the fix) [18:24] no O_EXCL in that open(2) call? hrm. === Ursinha is now known as Ursinha-afk [18:27] http://bazaar.launchpad.net/~ubuntu-branches/ubuntu/saucy/isc-dhcp/saucy/view/head:/server/dhcpd.c#L703 is where the leases file is read during startup while the privs get dropped later in line 775 === chuck__ is now known as zul [18:49] Hi folks; I'm looking for information on some of the modifications required for running ubuntu on azure; we are trying to isolate the source of some intermittent io issues and our next step is looking at the distribution provided by microsoft. [18:49] It looks like Ben Howard is the goto person for this; but I have no idea what channels or methods I would use to ask technical questions on this matter. [19:00] jdstrand: do you know if doing the chown to dhcpd call for those files in a post-startup stance in the upstart job would work? (as a ugly workaround till it can get properly fixed) === Ursinha-afk is now known as Ursinha [19:07] jkew: Ben Howard is utlemming on here, or try the ubuntu-cloud mailing list [19:08] rbasak: thanks [19:19] geser: I don't-- I only mentioned that based on your comments [19:19] geser: it seems plausible and worth testing [19:20] jdstrand: I tested it, but it doesn't work (unless I made a mistake with the upstart job) [19:21] will file the problem upstream to get it fixed properly [19:26] stgraber: do you know if it's possible to use a post-start stance in an upstart job where the daemon get started in the foreground? [19:29] Hmm, since when does ubuntu-server name it's network devices as nm ? [19:30] this is a vanila install and i have not added network-manager [19:30] geser: I think so, though if it really starts in the foreground and doesn't send a signal back to upstart or fork at some point, I'd then expect post-start to happens immediately after the command is started, so quite possibly before the daemon is actually operational [19:30] mac_nibblet: /etc/udev/rules.d/70-persistent-net.rules is usually responsible for NIC names.. [19:31] sarnold: but is this really expected behavior ? [19:31] mac_nibblet: I've never seen that (and it has nothing to do with Network manager as someone said) [19:32] mac_nibblet: maybe? :) what specifically are you seeing? [19:32] file does not exist [19:32] stgraber: I tried to chown some files (the isc-dhcp-server issue) as a workround after dhcpd get started but they stayed root:root [19:34] sarnold, guntbert: im wondering if i should try and just reinstall the server before i get more weird things [19:34] takes like 8 minutes to reinstall so .. [19:35] geser: so I really think the right way is to switch the ownership of the directory and figure out exactly what's going on with apparmor that prevents isc-dhcp from doing the whole create/rename/destroy thing [19:35] mac_nibblet: I would not expect to see different results [19:35] geser: sarnold or jjohansen should be able to help you there [19:35] stgraber: we know what the problem is (it was discussed in backscroll) [19:36] stgraber: dhcpd is doing their priv separation wrong [19:36] stgraber: it's a combination of AppArmor, dhcpd's priv sep and Kernel Link Protection [19:37] during startup the file needs to belong root and during operation dhcpd :( [19:37] jdstrand: just read the backlog now, that'd indeed explain it... [19:38] it really isn't apparmor, it is the kernel link protection [19:38] note that ISC isn't terribly good at fixing bugs or giving any feedback outside of security issues [19:38] getting an upstream priv dropping code took over 2-3 years so having it changed/fixed may take just as long [19:39] so we should either look into fixing that and carrying a patch until they eventually merge it or find a robust workaround (I don't want that bit to be racy in 14.04) [19:39] omg, perhaps we should add some chown calls before dhcpd calls setuid() [19:41] jdstrand: without AppArmor we could let dhcpd own those dir and files (AppArmor was the reason they got changed to root:root) [19:42] I hope you aren't suggesting dropping apparmor :) [19:43] it needs to either open and rotate the files as dhcpd, or open and rotate the files as root [19:43] opening as root and rotating as dhcpd is the problem [19:43] jdstrand: certainly not suggesting it [19:44] jdstrand: what about changing the owner of those files before calling the setuid()/setgid()? [19:44] they are probably opening as root and handing off the fd as a security protection [19:45]