/srv/irclogs.ubuntu.com/2017/01/24/#ubuntu-server.txt

=== manjo` is now known as manjo
fishcookershould i restart the box after configure network interfaces? http://vpaste.net/HQirT AFAIK im used to do /etc/init.d/networking restart or ifconfig eth0 down && ifconfig eth0 up, but it doesn't work04:52
=== disposable3 is now known as disposable2
lordievaderGood morning.08:55
rbasakcpaelzer: did you intend to drop the openldap merge from the blueprint?09:32
cpaelzerrbasak: yeah it was a dup, check the second to last line09:38
cpaelzerrbasak: I was updating mine and picked a few of the more simpler merges to fill into my qemu testing idle time slots09:38
cpaelzerrbasak: and so I found a few things that were duplicates or not updated yet09:38
cpaelzerrbasak: I hope there is no openldap != openldap magive that I'd have missed - did I ?09:39
rbasakcpaelzer: ah OK. Just checking!09:56
saju_mIn what order10:25
saju_mIn what order I should install these packages http://dpaste.com/1Q388JC ?10:25
=== synapse is now known as Guest89004
zulcoreycb: i uploaded a newer tempest last night btw13:15
coreycbzul, ok thanks13:16
zioprotocoreycb: can you confirm that Openstack Kolla in not yet package for Openstack Newton in Ubuntu ?13:42
coreycbzioproto, confirmed.  although we were talking about packaging it.13:44
zioprotook, I am testing it at the moment, right now working with pip then13:52
jamespagezioproto, hey13:54
zioprotohey13:54
jamespagezioproto, are you looking at kolla itself or one of the kolla-ansible/k8s deployment projects?13:54
zioprotowe have this problem here, that we dont feel like rewriting our puppet stuff from puppet3 to puppet413:55
zioprotoat the moment looking at kolla in stable/newton13:55
zioprotothat is still just kolla without the additional kolla-ansible repo13:55
zioprotothat starts in ocata13:55
zioprotokolla-ansible repo only has a master branch13:55
jamespagezioproto, ack13:56
zioprotostill did not look at Kubernetes13:56
jamespagezioproto, ok so can I suggest a slight different approach to packaging kolla then ?13:56
zioprotosure13:57
zioprotoI am all ears !13:57
jamespagezioproto, have you heard of snaps?13:57
zioprotois k8s different from https://github.com/openstack/kolla-kubernetes ?13:57
jamespagekubernetes == k8s13:57
jamespagefor folk like me that can't spell or be bothered to type a long name :-)13:57
zioprotook, so this is the same stuff ? https://launchpad.net/kolla-k8s13:57
jamespageyah13:58
zioprotoso what different approach you would suggest ?13:59
zioprotoah snaps13:59
zioprotoyes13:59
jamespagezioproto, yeah13:59
zioprotoyes13:59
jamespagezioproto, https://github.com/openstack-snaps/snap-rally13:59
zioprotopackaging distribution agnostic, right ?13:59
jamespageyep13:59
zioprotoI hear of it13:59
zioprotobut I dont have experience13:59
jamespagezioproto, we've been looking at it alot in the last few months13:59
jamespagezioproto, have a mostly functional set of openstack snaps13:59
zioprotoactually is how we deploy RC packages on Cumulus Linux14:00
jamespageI just raised the review to get rally and tempest snaps up under /openstack14:00
joeliojamespage: interesting, I'm working on rally today. Code available?14:00
zioprotodo you have a stable/newton openstack snap for Kolla that I can test ?14:00
jamespagezioproto, not yet - I was going to suggest that you have a run at it14:00
zioprotojamespage: are you coming to Milano to the ops meetup ?14:01
jamespagethey are pretty quick to write14:01
jamespagezioproto, might be14:01
jamespagezioproto, plans tbc14:01
zioprotookay, so I will first give it a try with pip14:01
jamespagejoelio, https://github.com/openstack-snaps/snap-rally14:01
joeliothanks14:01
jamespageis installable from the snap store - take a read of the README14:01
zioprotoKolla is already new, and I cant introduce in the team 3 new technologies at the same time :)14:01
jamespagezioproto, just think of it as a static wrap of pip14:02
jamespageinstallable without infecting the rest of your install with pip-ness14:02
jamespagejoelio, fwiw I think you need the snapd from xenial-proposed for the rally snap - its a classic mode version14:03
jamespagewhich is pretty new feature14:03
jamespagejoelio, 2.21 is the version14:04
joeliojamespage: yea, I'll take a look - I'd be keen to look at this for the cli tooling too, currently we provide a Vagrantfile for customers and a web interface, a snap would be cool too14:04
jamespagejoelio, oh I already have one in flight14:04
joelioace :)14:04
jamespagejoelio, openstackclients14:04
jamespagejoelio, idea is that it provides all openstack project cli tools14:04
joeliobrill, sounds ideal14:04
jamespagethe snap store is due to support series soon as well, so we can have a openstackclients aligned with each stable release of openstack14:05
joelioawesome14:05
jamespagejoelio, we're bootstrapping snap bits atm - if you wanted to join #openstack-snaps that's where most irc discussion is happening14:06
joeliojamespage: ack14:07
jamespagezioproto, ^^ if you're interested as well :-)14:07
zioprotook ! I will try to join also there14:13
zioprotoso when I run commands like14:13
zioprotosudo snapper create -d "Installing DHCPd pre-release fix" --command "dpkg -i ./isc-dhcp-*deb"14:13
zioprotoI am already use snap14:13
zioprotois the same tool right ?14:13
zioprotoor snapper is yet another thing ?14:13
joeliozul: different, thats for snapshotting your local stuff.. this is more akin to an image registry/catalogue14:31
joeliozioproto: ^^ (sorry zul)14:31
joelioI just tested rally out, took less than a minute including setting up xenial-proposed, lgtm14:31
smoserdannf`, please feel free to verify bug 164051914:35
ubottubug 1640519 in curtin (Ubuntu Xenial) "arm64 xenial maas images don't include u-boot-tools package" [Medium,Confirmed] https://launchpad.net/bugs/164051914:35
zioprotojoelio: so we are talking about this ? http://snapcraft.io/14:52
joelioyep14:52
dannf`smoser: yep - i'll ask sfeole to try it in his setup15:05
=== dannf` is now known as dannf
cpaelzerrbasak: could you run the importer on tgt ?15:23
cpaelzerrbasak: currently old/new debian is still outdated so it needs an import anyway15:23
cpaelzerrbasak: but I'm eager to see what it does since this already has a zesty merge15:23
cpaelzeryet I'm considering a re-merge to pick up latest (and drop more delta IIRC)15:23
jamespagezioproto, have you considered switching to charms from your current puppet approach for deployment of openstack?15:26
zioprotommm… no, not at all :)15:27
rbasakcpaelzer: sure15:31
cpaelzerthanks rbasak15:32
rbasakcpaelzer: done15:36
coreycbEmilienM, is ocata-proposed working ok for you know that webob is back to 1.6.2?16:14
EmilienMcoreycb: I haven't tested16:15
EmilienMmwhahaha: ^16:15
mwhahahano16:15
mwhahahawe've got other issues16:15
* mwhahaha hasn't had time to dig16:15
mwhahahabut nova's broken16:15
coreycbmwhahaha, in what way?16:16
mwhahahacoreycb: http://logs.openstack.org/66/422766/4/check/gate-puppet-openstack-integration-4-scenario001-tempest-ubuntu-xenial-nv/790428e/console.html#_2017-01-24_15_07_22_54698016:17
mwhahahahttps://review.openstack.org/#/c/422766/ if you want to follow along16:17
coreycbmwhahaha, is there any chance that run has the old webob?16:18
mwhahahanope16:18
coreycbmwhahaha, hmm16:18
mwhahahahttp://logs.openstack.org/66/422766/4/check/gate-puppet-openstack-integration-4-scenario001-tempest-ubuntu-xenial-nv/790428e/logs/dpkg-l.txt.gz16:18
mwhahaha1:1.6.2-2~cloud016:18
coreycbmwhahaha, any details in the scheduler log as to why no valid host was found?16:19
mwhahahahttp://logs.openstack.org/66/422766/4/check/gate-puppet-openstack-integration-4-scenario001-tempest-ubuntu-xenial-nv/790428e/logs/16:19
mwhahahalike i said, i haven't looked deeply yet. there was a vif error that is saw when i took a few seconds16:20
mwhahahahttp://logs.openstack.org/66/422766/4/check/gate-puppet-openstack-integration-4-scenario001-tempest-ubuntu-xenial-nv/790428e/logs/nova/nova-compute.txt.gz#_2017-01-24_15_01_25_38016:21
jamespagemwhahaha, coreycb: hold a mo16:27
jamespageI think that might be due to a os-vif incompat we had16:27
jamespageRebuilding amd64 build of nova 2:15.0.0~b2-0ubuntu2~cloud0 in ubuntu xenial RELEASE16:27
jamespagejust triggered that16:27
coreycbjamespage, hmm16:27
jamespagesorry that fell off my plate yesterday16:27
* mwhahaha buys jamespage a bigger plate :D16:28
coreycb:)16:28
jamespagecoreycb, mwhahaha: hmm16:30
jamespagecoreycb, mwhahaha: no I did do it yesterday16:30
jamespage2:15.0.0~b2-0ubuntu3~cloud016:30
jamespageis the right version16:30
jamespagewe might have a general os-vif compat issue in that case16:31
mwhahahahttp://logs.openstack.org/66/422766/4/check/gate-puppet-openstack-integration-4-scenario001-tempest-ubuntu-xenial-nv/790428e/logs/nova/nova-conductor.txt.gz#_2017-01-24_15_00_57_58216:34
jamespagemwhahaha: [Errno 2] No such file or directory16:46
jamespagelooks like something fails on the compute node with regards to port creation16:46
coreycbjamespage, yeah would be nice to know what that file or director is16:47
jamespage2017-01-24 15:01:52.220 4178 DEBUG oslo.privsep.daemon [-] u'brctl addbr qbr9926d47b-cb' failed. Not Retrying. out_of_band /usr/lib/python2.7/dist-packages/oslo_privsep/daemon.py:19416:47
jamespagebrctl by the looks of things16:47
mwhahahahttp://logs.openstack.org/66/422766/4/check/gate-puppet-openstack-integration-4-scenario001-tempest-ubuntu-xenial-nv/790428e/logs/syslog.txt.gz16:48
mwhahahathere's the syslog if you want to go looking16:48
mwhahahalooks like there's a bunch of aodh errors too we'll need to cleanup at some point16:49
jamespagemwhahaha, coreycb: indeed bridge-utils is not installed16:50
jamespagemwhahaha, coreycb: ok looks like a dep change in libvirt is causing this16:53
jamespagexenial libvirt has a hard depends libvirt-bin -> bridge-utils16:53
jamespagein ocata uca16:53
jamespagelibvirt-daemon-system -> recommends bridge-utils only16:54
mwhahahapesky libvirt16:54
mwhahahaif you're fixing deps, python-gabbi is also required by tempest. we fixed it in our testing since we don't use the packages but it wasn't getting pulled in16:55
coreycbmwhahaha, ah ok we'll take a look at that16:56
jamespagecoreycb, we don't see this as juju install bridge-utils automatically on all units for container bridging and addresssing16:56
jamespagebut it does need a tweak16:56
jamespageI'd suggest actually having the dep on neutron-openvswitch-agent or nova-compute - not quite sure which is best tbh16:57
jamespagenova uses brctl directly I think16:57
jamespagecoreycb, prob nova-compute methinks16:58
mwhahahafor what it's worth, rdo has nova-compute -> bridge-utils16:59
mwhahahahttps://github.com/rdo-packages/nova-distgit/blob/rpm-master/openstack-nova.spec#L14416:59
coreycbjamespage, mwhahaha: yeah seems to make sense to add as a dep for nova-compute16:59
coreycbwe only have it for nova-network atm16:59
coreycbalright let me make those 2 updates17:00
coreycbmwhahaha, it doesn't look like tempest uses gabbi. could it be a project-specific plugin?17:10
mwhahahait's possible17:11
mwhahahacoreycb: ceilometer test17:12
mwhahahahttp://logs.openstack.org/48/422248/8/check/gate-puppet-openstack-integration-4-scenario001-tempest-ubuntu-xenial-nv/1c85705/console.html#_2017-01-20_22_23_32_19916617:13
coreycbmwhahaha, ok i'll add that dependency for ceilometer.  these will build and i'll let you know when they're backported and promoted to ocata-proposed.17:30
mwhahahacoreycb: thanks17:30
greyollaIs there a way I can do my own live kernel patching without using canonical's livepatch service? ie looking to self host/maintain.17:40
jgrimmcaribou, do you still have merges in progress for kdump-tools, clamav, and nut packages?  thought i'd make sure they are still on your radar17:56
jgrimmoh, kdump-tools was for secureboot, not a merge17:57
cariboujgrimm: kdump-tools/makedumpfile is synced with Debian; clamav is awaiting on the MIR of tomsfastmath to become a sync17:57
cariboujgrimm: nut still needs to be completed afaik17:58
jgrimmcool enough, i was just going through the blueprint and checking in on INPROGRESS items17:58
cariboujgrimm: I'd like to get the kexec-tools done too depending on my b/w17:59
jgrimmack17:59
JemalMohaHello All, nice to be part of the community!18:13
ctjctjHello.  I just upgraded to 16.10 and iscsitarget no longer exists.  What is the replacement package?18:19
Noname01x2i need assistance18:33
ctjctjNoname01x2, ask your question. don't ask for attention.18:34
Noname01x2ok18:35
Noname01x2so on my ubuntu server i have apparmor, which i need. and its not working. getting an error in the log.18:36
Noname01x2im open to any solution to get rid of the error, although i would prefer not to remove mysql-server in the process.18:37
sarnoldplease pastebin your DENIED entries18:37
Noname01x2how do i do that? im no pro18:37
Noname01x2i took a screenshot18:38
Noname01x2saved it in paint18:39
sarnoldNoname01x2: dmesg | grep DENIED should do it18:39
sarnoldyou could use the pastebinit program to automatically copy-and-paste that to a pastebin site18:40
sarnoldctjctj: I don't know about _the_ replacement for iscsitarget, but there's a few choices.. both libsiscsi and tgt appear to be in main in 16.10: http://packages.ubuntu.com/source/yakkety/tgt  http://packages.ubuntu.com/source/yakkety/libiscsi18:41
Noname01x2hmm let me try that18:41
ctjctjtgt and libiscsi and I'm checking istgt now.  All my google foo is pulling up iscsitarget references.  THanks sarnold.18:41
Noname01x2im in emergincy mode18:44
ctjctjNoname01x2, *grins* I understand.  I've got 10+ devs that are grumping because we lost all of our iscsi targets.18:45
sarnoldowww18:46
ctjctjI'll blame you for the solution.18:47
sarnoldwhen devs are grumbling is probably not the ideal time to be doing research, but last time I looked into iscsiland, this table seemed useful http://scst.sourceforge.net/comparison.html18:48
noname01x3sending you a file, please accept sarnold18:49
sarnoldeh?18:49
noname01x3its a screenshot18:49
sarnoldwhy not just pastebin the dmesg | grep DENIED?18:49
noname01x3cant18:49
noname01x3let me try again18:49
noname01x3im in the emergency mode terminal thing18:50
noname01x3i coppied the msg18:51
noname01x3now what18:51
ctjctjnoname01x3, if you don't have it installed, install pastebinit   and use it to create pastebin documents.  You'll get back a URL which you post here.18:52
ctjctj!pastebin18:52
ubottuFor posting multi-line texts into the channel, please use http://paste.ubuntu.com | To post !screenshots use http://imgur.com/ !pastebinit to paste directly from command line | Make sure you give us the URL for your paste - see also the channel topic.18:52
noname01x3im rebooting18:53
Noname01x2ohh i get it18:53
DammitJimsarnold, are you around today? good afternoon18:55
noname01x3pastebin came up. now all i have to do is get the text....(still rebooting)18:55
DammitJimis there a problem with setting systemd-journald to persistent?18:56
sarnoldhey DammitJim :) I wouldn't have thought so, but in that bug report yesterday I got the impression that using it might lead to .. wel,l the exact issue you were facing yesterday18:56
DammitJimLOL18:57
DammitJimwell, making it persistent has actually somehow helped and the server reboots just fine18:57
sarnolddude18:57
DammitJimshould I be concerned with leaving it the way it is?18:57
sarnoldjust when I think I understand how things work..18:57
noname01x3Right now I'm at "Welcome to emergency mode! After logging in, type "journalctl -xb" to view system logs, "systemctl" to reboot, "systemctl default" or ^d to try again to boot into default mode. Press Enter for maintenance.18:57
DammitJimLOL... same thing here... I mean, the log still says that it couldn't unmount /var18:57
DammitJimdammit, that means that there are more problems, huh?18:58
sarnoldDammitJim: it should be fine to keep it on persistent. your logging fielsystem may see twice the write rate that it had before, since now rsyslog -and- journald are logging, but for most people that really shouldn't be a burden.18:58
DammitJimok, thanks18:58
DammitJimit's sad that one only has a limited amount of time and to get to the root of a problem sometimes one has to just skip it until later18:59
DammitJimwhatever later is...18:59
noname01x3interesting18:59
JemalMoha How to Stop any daemons that is listening on the default SMTP port. (Newbie Here!)18:59
sarnoldDammitJim: yeah. from that bug report it looks sort of like there's no real fix in sight, either18:59
sarnoldNoname01x2: so you're stuck in a rescue mode? o_O oy. that sounds like an annoying thing to sort out.19:00
noname01x3yeah19:00
noname01x3it boots into this mode and thats what I was trying to tell you guys. I cant even open any programs19:00
noname01x3how can I copy and paste the error if I cant even copy n paste19:01
sarnoldaha19:01
noname01x3I see a red Failed19:01
noname01x3Failed to start LSB: AppArmor initialization. at the end of the line.19:01
noname01x3everything else has a green OK19:02
noname01x3Fix it please. Thanks.19:02
noname01x3What about the packages? Can we do something with that?19:03
sarnoldnoname01x3: what does dmesg | grep -i apparmor report?19:06
JemalMohaHelp ?   How to Stop any daemons that is listening on the default SMTP port. (Newbie Here!)19:07
sarnoldJemalMoha: use netstat -anp | grep :25 to find whatever it listening on port 25, then either use systemctl stop <name> to stop the thing (16.04 lts and newer), or 'stop name' to stop the thing (14.04 lts and older)19:07
JemalMoha@sarnold thanks!!19:09
noname01x3sarnold it reports: {       0.001320] AppArmor: AppArmor disabled by boot time parameter19:10
sarnoldnoname01x3: hrm, I wouldn't have expected that to cause the boot to fail.19:10
noname01x3[       29.453556] system[1]: systemd 229 running in system mode.19:11
noname01x3its all ur fault. u broke it. :-)19:11
noname01x3fyi i made up that last line.19:11
noname01x3..about u braking it...19:11
noname01x3also it says: (+PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK...19:12
sarnoldthat bit just says that systemd was configured with support for all those different tools19:13
noname01x3Well the app armor is important.19:13
noname01x3oh19:13
noname01x3app armor is in red19:13
sarnoldhehe, yes, I feel the same way, I've been working on apparmor since 2000 :)19:13
noname01x3:-)19:14
noname01x3I'm confident you can help me.19:14
sarnoldnoname01x3: are there better errors in the logs? /var/log/syslog for example?19:17
noname01x3how do i get that19:17
sarnoldless /var/log/syslog probably -- there's a chance that the things causing you trouble may not say 'apparmor' directly in the lines..19:18
noname01x3I would live to give you all possible info.19:18
noname01x3ok a lot of info cam ein19:18
sarnoldjump to the end and then start scrolling backwrads..19:19
tyler3332I have an ssh key with a passphrase, but Im not getting asked to put my passphrase in, why is this? Its for a local VM.19:19
sarnoldtyler3332: ssh-agent or a keyring daemon perhaps?19:19
ctjctjnoname01x3, in case you did not know, you can use a slash (/) to start a search in less.  So find your apparmor error, then use page up down to move up and see what was happing near there.  You can use 'G' to go to the end of the file and '?' to search backwards.19:19
tyler3332how do I check?19:19
ctjctjtyler3332, ssh-add -L19:20
tyler3332yup there is a key there19:20
tyler3332is that ssh-agent?19:20
ctjctjThat means you are connected to an ssh-agent.  When the key is added to the agent you were asked to provide the passphrase.19:21
tyler3332ah ok19:21
ctjctjIt is assumed that ssh-agent will take good care of your private keys.  I.e. it wont leak them to just any old person.19:21
tyler3332ok thanks.19:22
ctjctjssh-add is how you add (and remove?) keys from the agent.  -l will give you finger prints and -L will give you the public key half.19:22
tyler3332do keys get added automatically? cause I didnt add it myself19:22
ctjctjWhen I'm dealing with clients lacking clues we'll get them to log in with a password verify that they have a working agent with ssh-add -l then have them give us the ssh-add -L output so that we can update "authorized_keys" and turn off password access.19:23
sarnolddefinitely good idea19:24
ctjctj(reality is that we have a special VM with a guest account that the log into to runn ssh-add -L.  That way we never allow password access to our servers to our clients.19:24
sarnoldDUDE19:24
sarnoldthat's great :)19:24
noname01x3sarnold i cant tell what msg is good and whats bad19:24
ctjctjThere is *exactly* one password on our servers.  For root which I know for emergency access from the KVM console.19:24
sarnoldnoname01x3: can you copy-and-paste it to paste.ubuntu.com?19:25
noname01x3I have no idea how.19:25
noname01x3maybe i should have booted into advanced mode19:26
ctjctjsarnold, take a look at pam_ssh_agent_auth for how to allow sudo access based on ssh keys.19:26
ctjctjnoname01x3, use pastebinit like I pointed you at.19:26
sarnoldctjctj: he's stuck in a rescue mode, no networking, no nothing :/19:27
ctjctjsarnold, service networking start ?19:27
sarnoldworth a try19:28
coreycbbeisner, can you promote nova - 2:15.0.0~b2-0ubuntu4~cloud0 to ocata-proposed please?19:30
noname01x3i believe i have network access19:31
noname01x3it says starting raise network interfaces....19:31
sarnoldctjctj: pam_ssh_agent_auth.. that's odd. it's better than passwords for sudo but I'd be afraid that the ssh key lifetimes useful for logging in to hosts may not match sudo lifetimes very well :)19:31
noname01x3[OK]Started ifup for eth0.19:31
ctjctjsarnold, I don't follow.  what do you mean by ssh key lifetimes?19:32
sarnoldctjctj: well, back in the day I used to use ssh-add -t 30019:32
coreycbbeisner, that should help get puppet CI back in order for mwhahaha19:32
ctjctjsarnold, ah, we don't  we leave the key in for the duration of the session.19:33
sarnoldctjctj: but then I started doing tasks on VMs that would take an hour, etc, so I wound up moving to ssh-add -t 3600 ... and then eventually the tasks got long enough that I gave up on -t entirely and just add keys to the agent that last -forever-19:33
ctjctjyep.19:33
ctjctjSo the life time would be fine for sudo work.19:33
sarnoldbut I certainly don't want more than ~five minutes of allowed sudo window19:33
sarnoldnot that that's perfect19:33
sarnoldan open terminal is a disaster anyway19:34
sarnoldbut still19:34
ctjctjsarnold, ah.  I understand.  I'm dealing with a bunch of Devs that are Drupal/Wordpress people.  Teaching them to do anything safely is hard.19:34
sarnoldctjctj: it certainly doesn't come naturally. :)19:35
noname01x3Maintenance mode19:35
beisnercoreycb, ok, promoted. ye shall haz bridge utils.19:35
coreycbbeisner, awesome19:35
ctjctjMy current favorite is that our "director of creative direction" has been giving out the company user name/password (.htpasswd) to clients instead of making them use their individual passwords)19:35
noname01x3ctjctj so if I  have network access. then what19:35
sarnoldnoname01x3: sweet. apt-get install pastebinit, and then pastebinit /var/log/syslog -- if it looks like something you'd want to share with the world19:36
ctjctjnoname01x3, now you can do things like apt-get install pastebinit and use that to get us bits and bobs of what's going on in your system.19:36
sarnoldctjctj: ouch. just .. ouch.19:36
noname01x3reading package19:37
noname01x3done19:37
ctjctjsarnold, yeah.  Ouch is the word for this guy.  He manged to make his way onto my s___ list about 6 months ago.  Normally it takes about 2 weeks to work your way off.  He is deeper on the list than when he was placed on it.19:37
sarnoldctjctj: gotta hand it to him, he does sound -creative-19:38
noname01x3installed19:38
noname01x3so now what19:38
noname01x3im at root19:38
sarnoldpastebinit /var/log/syslog19:38
ctjctjFriday I get a client to agree to X,Y and Z.  This makes things so much easier for what I'm developing.  Monday he talks to the client and undid all of that client work.  So now the project is in limbo for another two weeks until client reaches a decision, again.19:39
noname01x3sarnold it says: http://paste.ubuntu.com/23859388/19:39
ctjctjnoname01x3, *yes*!19:39
sarnoldnoname01x3: great :) moment..19:39
noname01x3Congrats to me.19:40
noname01x3This is fun19:40
noname01x3My boss will be unhappy if this isn't fixed, however.19:40
ctjctjnoname01x3, sorry about that.  Part of the problem is that we forget to specify everything.19:40
noname01x3ctjctj ok so thats ok. I get it. You didn't understand how low my skill level was.19:41
noname01x3VERY LOW. But I'm a smart guy.19:41
ctjctjsarnold, that looks like a desktop boot.  I'm seeing boatloads of gnome-session stuff.19:41
noname01x3I sound like I'm almost not stupid.19:42
sarnoldI can't spot anything that looks like trouble ;(19:43
noname01x3sarnold A desktop boot? How dare you?19:43
ctjctjsarnold, would dmesg hold it?19:43
ctjctjI'm thinking that systemd might not be running yet.19:43
ctjctjsyslogd19:43
ctjctjnoname01x3, on that server: dmesg| pastebinit19:44
noname01x3uhh ok. let me see19:44
noname01x3ctjctj Http://paste.ubuntu.com/23859412/19:45
sarnold[    0.000000] Command line: BOOT_IMAGE=/vmlinuz-4.4.0-59-generic.efi.signed root=/dev/mapper/BNY--VS--CA--vg-root ro security=selinux selinux=119:46
sarnoldno wonder apparmor init script isn't happy :)19:46
ctjctjnoname01x3, translation.  fix your boot command line.19:46
ctjctjselinux and aptarmor don't play well together (IIRC)19:47
noname01x3okay. how do we do that19:47
sarnoldindeed :)19:47
sarnoldmaybe someday19:47
sarnoldbut not this year19:47
sarnoldand probably not next year19:47
noname01x3Thats ok. We have time.19:47
ctjctjsarnold, apt-get purge selinux ?19:48
noname01x3But we need this up and running asap19:48
noname01x3lol19:48
ctjctjmaybe with an * in there some place?19:48
noname01x3ctjctj someone doesn't like selinux.19:49
noname01x3ctjctj yes thank u.19:49
ctjctjnoname01x3, you don't at this point.  Or you can explain to "boss" why the system is down? *GRIN*19:49
noname01x3:-)19:49
ctjctjI'm waiting for sarnold to approve that line19:49
sarnoldgood question, I don't know selinux packging well enough to know if they automatically add selinux=1 stuff to the kernel command line..19:49
sarnoldapt-get purge 'selinux*' would be a decent starting point anyway19:49
ctjctjsarnold, I'd start with the purge, check /boot/grub.menu (or whatever it is this week) and if it is still bad we'll update it via grub stuff.19:50
tarpmancareful, that might match libselinux as well19:50
noname01x3sarnold will it give me a warning if it will do something i dont like?19:50
sarnoldnoname01x3: not really19:50
sarnoldlinux is like that19:50
ctjctjnoname01x3, nope.  It's unix.  If you want to shoot yourself in the foot then it will ask you how big of a hole you want to make after the first shot.19:50
sarnoldit'll let you cut off your arms if you want19:51
noname01x3lol19:51
ctjctjsarnold, *laughs* great minds and low ethics think alike.19:51
noname01x3ok thanks for the tip.19:51
sarnoldlol19:51
coreycbzul, yay python-statsd approved19:52
noname01x3I like the purge selunix idea.19:53
ctjctjI'm going to send a cuss word filled bug report to whomever wrote the option parsing for tgtadm  '-y' is not allowed/supported.  I ended up in the source code to find out that -y means --blocksize...19:53
coreycbzul, that should unblock some things19:53
zulcoreycb: i saw19:53
zulcoreycb: oslo.middleware19:53
ikonia-y means blocksize ???19:53
ikoniawho thought that out19:53
noname01x3I have a backup checkpoint from this morning.19:53
noname01x3I can put it right back if it goes bad19:53
ctjctjikonia, yeah.19:53
noname01x3or maybe I dont know what im talking about.19:53
sarnoldctjctj: eww!19:54
ctjctjnoname01x3, I'm glad you have that checkpoint.  Means we don't have to worry about xxxbadcommamndXXX -rf / mean people.19:54
noname01x3ctjctj, thanks. Yeah I believe I can try it with no worries. And my boss is constantly asking me whats new and I would love to have an answer, other than repeating what you guys are saying, which causes him to ask "And".19:56
ctjctjnoname01x3, what version of ubuntu are you running? 14.04LTS?19:57
sarnoldprobably 16.04 lts, with kernel version 4.4.x19:58
ctjctjsarnold, ok.19:58
noname01x316.04 LTS19:58
ctjctjnoname01x3, once you have the purge done: pastebinit /boot/grub/grub.cfg19:59
noname01x3ctjctj, ok i will do the purge now.19:59
noname01x3apt-get purge selinux20:00
noname01x3Done20:00
noname01x3waait it says click y to continue20:00
ctjctjnoname01x3, wrong command.20:01
ctjctjapt-get purge 'selinux*'20:01
noname01x3pressed y20:01
ctjctjThat's ok.  When you are done run the second purge too.20:01
ctjctjsarnold, updated my original command.20:01
ctjctjif it says anything about "grub" as it is doing that purge let us know please.20:02
sarnoldgood idea, apt-get purge 'selinux*' looks better20:02
ctjctjsarnold, well I did cut and paste your version... *GRIN*20:02
sarnoldhehe20:03
noname01x3ctjctj, http://paste.ubuntu.com/2385948420:03
noname01x3whats the second purge?20:04
sarnoldthe second purge would remove any selinux policies that might have been installed via tha packages20:04
ctjctjit adds a wildcard to search for other packages named selinux[anything]20:04
noname01x3ohh ok lets not lol20:04
ctjctjSo if there is a selinux-break-nonames-system the * would match -break-nonames-system20:04
ctjctjYou want to delete those other things noname01x3.20:05
noname01x3ctjctj, unless you insist20:05
ctjctjsarnold, looks like the security=selinux and selinux=1 are still in the grub.cfg20:05
sarnold"security=selinux selinux=120:05
sarnoldyes that :) heh20:05
noname01x3ahhhhhhhhh20:05
noname01x3lets get it out!!!!!!20:05
noname01x3ok whats the purge. lets do it20:06
ctjctjnoname01x3, what's happening is that sarnold and I could very easily tell you how to remove that part in grub.cfg and update grub to have your system boot.  But my fear is that the next time you got a kernel update the selinux issue would come back.20:06
ctjctjapt-get purge 'selinux*'20:06
ctjctjnoname01x3, pastebinit /etc/default/grub.cfg20:07
ctjctjnoname01x3, do you speak vim, emacs or ed/ex?20:07
Noname01x2what the???20:10
Noname01x2i was disconnected20:10
ctjctjNoname01x2, pastebin /etc/default/grub.cfg after the purge20:11
ctjctjNoname01x2, do you speak vim, emacs, ed/ex, or nano?20:11
noname01x3im back20:11
noname01x3ctjctj, none of the above20:11
ctjctjnoname01x3, pastebinit /etc/default/grub.cfg20:11
ctjctjif that purge completed.20:12
noname01x3unable to read from /etc/default/grub.cfg20:12
ctjctjsorry /etc/default/grub20:13
ctjctjto many windows with some being 14.04 and some 16.10 with slightly different naming conventions.20:13
noname01x3ctjctj, http://paste.ubuntu.com/23859650/20:14
ctjctjnoname01x3, nano /etc/default/grub  go to the line that says "GRUB_CMDLINE_LINUX_DEFAULT" and remove everything between the quotes.20:15
ctjctjnano is designed for my 6yo.  So I'm hoping you can drive it... *GRIN*20:15
sarnoldthat's the first package I purge every install :)20:16
ctjctjsarnold, my IDE is emacs.... *GRIN*20:16
noname01x3ctjctj, I'm doing what you told me. Stand by...20:17
sarnoldctjctj: do you also use it for irc, email, web, etc? :)20:17
ctjctjsarnold, is there any other way? *grin* Nope.  just my programming editor.  I'm equally at home with vim, vi, ed, ex, jove and a few others.20:18
sarnoldctjctj: no kidding? I started with vi too long enough, and never figured out how to do more than quit emacs..20:19
ctjctjthough I have used it for email, irc and uunet news20:19
ctjctjsarnold, I started with a home grown editor that I wrote to interface with a distribute console editing system. (scredit at MSU).  But I was playing "hack" or "nethack" and my fingers learned the vi key movements.20:20
sarnoldand once vi was in my fingers, nothing else quite fits right20:20
ctjctjabout 1989 I was working with Mike Muuse (author of ping and a bunch of other great things) and watched him drive "jove" (jonathans own version of emacs).  Jove was light weight enough to load in a reasonable amount of time vs emacs.  And seeing some of the things  he was doing, multiple windows into the same file, two files open at the same time, cut and paste from one window to another.  and all of it in a 24x80 green20:21
ctjctjscreen terminal.20:21
noname01x3ctjctj, do i remove the ""?20:22
ctjctjI switched to jove the next day.  Used jove until emacs for programming and vim for "quick edits"20:22
ctjctjnoname01x3, no.  Leave the ""20:22
ctjctjsarnold, I have one of my logical units back.  Thanks for the pointer to tgt.20:22
sarnoldctjctj: ahhh that makes sense. yeah. emacs was way too heavy to load on the shared system I started with, I didn't want the other users upset :)20:23
sarnoldctjctj: sweet! how's the transition?20:23
noname01x3ctjctj, im done.20:23
ctjctjthe transition is a pita.  It is all command line controlled.  I haven't located a configuration file yet so about 6 long command lines to define the target, define the LUN, define the users, attach the users, open the target to the initiator.20:24
noname01x3I think I want to become an ubuntu expert now.20:24
ctjctjnoname01x3, update-grub20:24
noname01x3ctjctj, im still in nano.20:25
noname01x3ctjctj, do i do exit?20:25
ctjctjisn't it in the menu at the bottom? (ctrl-x)20:25
ctjctjit will ask you to save the file and such.  Do so.20:25
noname01x3ctjctj, yeah I just wanted to confirm.20:25
* ctjctj ups the age requirement from 6 to 820:25
sarnoldctjctj: damn :/ what a pain in the butt :(20:25
noname01x3ctjctj, lol. I get it. Its very easy to use. I just like to follow instructions to a tee.20:26
noname01x3ctjctj, ok now we do that other thing yo usaid....20:26
noname01x3updating...20:27
ctjctjsarnold, as far as I can tell this is a cleaner interface.  iscsitarget has a command line interface but it does not save any dynamic modifications.  So you have to modify the target/lun dynamicly, update the configuration file.  Hope you got it right.  If you restart iscsitarget your initatiors hate you.20:27
ctjctjsarnold, this is all dynamic which is good in that way.20:27
noname01x3ctjctj, done.20:27
ctjctjnoname01x3, pastebinit /boot/grub/grub.cfg20:28
noname01x3ctjctj, http://paste.ubuntu.com/23859711/20:28
ctjctjnoname01x3, what is your favorite God?20:29
ctjctjnoname01x3, cross fingers, start praying and type "shutdown -r +1"20:29
sarnoldctjctj: owww, I never really thought about trying to change configs of these things on the fly20:32
sarnoldctjctj: (i've really only got the one good computer, so even though I've read all the docs, once upon atime, I never needed to use the things)20:32
noname01x3sorry. disconnected20:34
noname01x3ctjctj, what now20:34
ctjctjsarnold, I'm sitting in my home office with primary box 16.10, two vm's 14.04lts, two other desk tops for wives and kids computer.  And then a dozen primary servers and two dozen dev VM sandboxes running 14.04lts.  I've broken everything atleast once.20:34
ctjctjnoname01x3, cross fingers, start praying, and type "shutdown -r +1"20:34
ctjctjand stop disconnecting. *grin*20:34
sarnoldctjctj: hehe, breaking things, a good way to learn20:35
ctjctjsarnold, changing /etc/default files is new to you?20:35
sarnoldctjctj: yeah. I felt more at home with editing the one big /etc/rc with the ifconfig and route commands and what not just stuffed in there.20:36
sarnoldsysvrc was a stretch, but I got there.20:36
sarnold/etc/default/ .. newfangled things. I never think to look there first, only when reading the sources eventually shows the ". /etc/default/whatever" in the script I'm reading20:36
ctjctjsarnold, I was once at the National Cancer Research center in Fredricks MD (I think I remember correctly).  I had 48 hours to install unicos on their cray X/MP.  Documentation says it takes two weeks.  It took me 32 hours.  Having the director pop in ever 2 hours to check on things was stressful.20:36
ctjctjOn the other hand the fact that pizza and Coke-a-cola were always fresh was nice.20:37
noname01x3ctjctj, sarnold, if this works...20:37
sarnoldctjctj: daaaang. things were different in those days. linux was a breath of fresh air compared to those cranky old unix machines.20:38
sarnoldctjctj: my first job had an sco unix thing. no compiler. no header files. but we did have perl..20:38
noname01x3a start job is running...20:38
ctjctjsarnold, not really.  Early linux was the pits compared to BSD 4.4 though I hated solaris.  SunOS 4.3+ was a joy to work with.20:38
noname01x3hmmm.20:38
noname01x3"OK" AppArmor initialization.20:39
ctjctjsarnold, I once had to get a CC compiler on to a "binary only" box in order to compile GCC20:39
sarnoldctjctj: I dunno. I recall my first slackware install much more fondly than the terrible old sco box... heh.20:40
sarnoldctjctj: the bsdi system I only ever used as a user. it just worked. someone else dealt with everything :)20:40
sarnoldctjctj: boostrapping gcc though. ugh.20:40
ctjctjSCO is not a fun version of Linux.20:40
noname01x3ctjctj, sarnold, its all showing OK so why is it still in emergency mode?20:40
sarnoldctjctj: rofl20:40
ctjctjnoname01x3, no idea.  Why did you tell it to boot to emergency mode?20:40
noname01x3uhhh20:40
sarnoldnoname01x3: why is it in emergency mode this time?20:40
noname01x3dont know20:41
ctjctj(or you can show us syslog and let us try and figure it out from data instead of guessing?)20:41
ctjctjsarnold, BSDI is not the same as BSD.  BSDI is based on BSD 4.4 open release.  Which is what set Unix free.  Thank you U of California.20:42
* ctjctj needs a control dial for snark level.20:42
sarnoldctjctj: yeah, I didn't actually use a bsd 4.4. system. just the bsdi derivative. (Thanks microsoft!) and then a freebsd system somewhere along the way, for a month, just for kicks.20:43
ctjctjI have a few contributions to FreeBSD.20:43
ctjctjBut that was a long long time ago.20:44
noname01x4wow i came back automatically this time20:44
ctjctjnoname01x4, I told you to stop disconnecting.  How rude.  Makes it hard to help you.... *GRIN*.  We invite you to show us data so we can figure out how you asked your system to boot into emergency mode.20:45
ctjctj(that's a quiz)20:45
noname01x4http://paste.ubuntu.com/23859756/20:46
noname01x4what the20:46
noname01x4how can I quit20:46
ctjctjyour name changed.20:46
sarnoldping timeouts are funny things20:46
noname01x4Im on wifi so maybe thats why20:46
noname01x4or pint timeouts20:47
sarnoldyour irc client didn't respond to the server's PING? request in time, so the server disconnects you20:47
noname01x4okj20:47
ctjctjsarnold, that's exactly the same syslog as he posted last time.20:47
sarnoldnormally your client will notice that it hasn't had a PING? request in a while and disconnect/reconnect on its own. and your name would still be in use.20:47
sarnoldwell now that's odd. I thought those timestamps didn't look right..20:48
noname01x4so what now20:48
ctjctjnoname01x4, I'm thinking.20:48
ctjctjsarnold, he can write to his disk.  So the disk is not read-only.20:48
ctjctjnoname01x4, do: ls -l /var/log/syslog and take note of the size and date20:49
ctjctjnoname01x4, then service rsyslog start20:49
ctjctjcheck /var/log/syslog and see if anything changed.20:49
noname01x4ctjctj, ok20:55
noname01x4ctjctj, its doing something....20:55
ctjctjsyslog is growing or the date changed?20:56
noname01x4nothing is happening20:56
noname01x4just sitting there.20:56
ctjctjnoname01x4, did you run ls -l /var/log/syslog again?  Did the values change?20:58
noname01x4ctjctj, no they did not change.20:58
sarnoldo_O20:58
noname01x413:1920:58
noname01x4wow u made the guy quit20:59
ctjctjdid it tell you why it was entering emergency mode?20:59
noname01x4nope20:59
ctjctjDid you see any "fails" on the boot?20:59
noname01x4i just typed reboot20:59
ctjctjnoname01x4, it is almost always better to use "shutdown -r +1" than a simple "reboot"20:59
noname01x4shutdown -r +1?21:00
noname01x4what is that21:00
ctjctjyes.  Like I said up above.21:00
ctjctjtry "man shutdown"21:00
noname01x4ok so far it says "mounted /boot/efi.21:01
noname01x4ok21:01
noname01x4I will use that in the future.21:01
noname01x4if possible21:02
noname01x4i tried it and it didnt take, but maybe i typed it wrong21:02
ctjctjyou should be able to type "man command" for any command we give you.  If you don't know what the command does or the options we are giving you, you should run man to see what it is you are being told to do.21:02
noname01x4its dpoing something21:03
noname01x4qq21:03
ctjctjWhich doesn't help when people are intentionally attempting to mess with you, the old: Your documents don't look right.  You need to make them look better.  Open up a DOS window and type "format c:" that will format all your documents to C things better21:03
noname01x4interesting21:04
noname01x4I just cant get this boot going21:04
noname01x4always emergency mode21:04
ctjctjOf course *I* never did that.  I just sent out CD's with Linux or FreeBSD labeled "Windows '95 update'21:05
noname01x4cant we just change somethjing in the boot to say startttt normally21:05
ctjctjwhat does journalctl -xb tell you?21:05
noname01x4so much info21:06
ctjctjOk.21:06
ctjctjstart the network then21:06
noname01x4nothing looked bad except maybe one or two things21:06
ctjctjjournalctl -l -xb | pastebinit21:06
sarnoldoff to lunch, back eventually, good luck guys :)21:06
ctjctjsarnold, thanks for the discussions.21:06
sarnoldthanks to you, too, very helpful :D21:07
noname01x4sarnold, you are fantastic. enjoy your meal.21:08
noname01x4ctjctj, http://paste.ubuntu.com/2385986521:08
ctjctjnoname01x4, did you see anything in RED as that scrolled by?21:10
noname01x4Yeah a few thinks., ctjctj.21:11
noname01x4pci had a problem21:11
noname01x4but who needs a pci21:12
ctjctjOk.  how many disk drives do you have attached to this machine?21:12
ctjctjpastebinit /etc/fstab21:12
ctjctjIt is looking bad for a disk drive.  I would also like to see "blkid | pastebinit"21:14
noname01x4pci: fatal?21:14
ctjctjAccording to your journal you failed to mount /data because /dev/sdc1 does not seem to exist.  I'm exploring that right now.  So if you can get me those two pastebin's it would be helpful.21:15
noname01x4ctjctj, http://paste.ubuntu.com/2385990421:15
noname01x4there was a third hard drive thats not there21:15
ctjctjnoname01x4, *blinks* what do you mean there is suppose to be a third drive that is "not there"?21:16
ctjctjDid you take the drive out of the box?21:16
noname01x4ca boto and ca data21:17
ctjctjnoname01x4, I don't follow.21:17
noname01x4cs data21:17
noname01x4wait21:17
noname01x4im typoing21:17
noname01x4wait my boss is telling me something about hard drives.21:17
noname01x4he is not sure what he's talking about21:18
noname01x4he says there should be a drive called data21:18
noname01x4ok nvm21:18
ctjctjYes, there should be a drive that is mounted on /data21:18
ctjctjAnd I asked for "blkid | pastebinit" so I can see what drives you have21:18
noname01x4ctjctj, http://paste.ubuntu.com/23859927/21:19
ctjctjOk.  Ask yourself and your boss "DID YOU REMOVE A DRIVE FROM THIS BOX?"21:20
noname01x4yeah I would hope not. uhggg. now hes on the phone.21:20
noname01x4timed out waiting for device dev-sdc1.device21:23
noname01x4that may be normal i guess21:23
ctjctjnoname01x4, There is a sda and a sdb drive that shows in your dmesg boot sequence.  There is no reference to sdc.21:24
ctjctjYour fstab says to mount /dev/sdc1 as /data.  If that fails then bad things happen.  Such as ending up in emergency mode.21:24
noname01x4ohh21:24
ctjctjSo when I hear the answer to the question "Did you remove a drive from the box" I can move forward on debugging your issue.21:24
noname01x4ok stand by21:25
ctjctjThis is one of those times where people get slapped with a dead fish "My system doesn't boot"  four hours of debugging later "Do you think that removing this extra card from the computer might have caused and issue?"21:25
ctjctjnoname01x4, is this a bare metal box or is it virtualized?21:26
ctjctjAnd yes, my snark level is high.21:26
noname01x4virtualized21:28
=== alexisb is now known as alexisb-afk
ctjctjAre all three drives provided as virtualized drives or is one of them provided as an iscsi target?21:28
noname01x4that is an interesting question. give me a few more minutes to get the answers.21:29
=== keithzg_ is now known as keithzg
ctjctjnoname01x4, any answers yet?21:43
ctjctjnoname01x4, I'm out of here in 16.25 minutes for an hour.21:44
ctjctjAFK22:01
noname01x4hey hey22:09
noname01x4ctjctj, my boss is saying there was a third physical drive that was removed22:09
noname01x4ctjctj, but there error was there before the drive was removed.22:09
noname01x3ok22:14
noname01x3ctjctj, all the drives are virtual22:15
noname01x3ctjctj, and the error happed before the third physical drive was removed.22:15
Noname01x2crazy22:28
Noname01x2my boss told me to continue working on this until its fixed22:29
=== alexisb-afk is now known as alexisb
jelly"flogging shall continue until morale is improved"23:27
ctjctjnoname01x3, are you here?23:50
=== CodeMouse92 is now known as JasonMc92

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!