[07:08] <lordievader> Good morning.
[07:09] <cpaelzer> hi lordievader
[07:09] <lordievader> Hey cpaelzer, doing good?
[07:09] <cpaelzer> yeah so far so good :-)
[07:12] <cpaelzer> lordievader: is your day already crazy?
[07:13] <lordievader> Nah, still quiet.
[07:14] <lordievader> Though Google has decided to only show the header bar -.-
[07:16] <cpaelzer>  /msg chanserv info #ubuntu-server
[07:16] <cpaelzer> bad space :-)
[07:45] <cpaelzer> Today is Ubuntu Server Bug Squashing Day #1: Announcement goo.gl/B98ER and a pad to track current activities at http://pad.ubuntu.com/U5GLIMwc2u
[07:45] <cpaelzer> Current chairs: cpaelzer
[07:46] <cpaelzer> rbasak: ping me when you are around as well, so that I can add you
[07:47] <cpaelzer> I thought to post that e.g. hourly, giving up on title edit as according to my checks many measures are in Place to keep that as-is
[08:34] <rt_ctx> Hi all, I have a problem at my work. I have more than 200 ubuntu virtual machine on production. Some of them are using a lot of memory (98% usage). I want to decrease this value. I know that I can decrease the usage of cache RAM but I don't know how ? Somebody can help me ? Thx
[08:35] <cpaelzer> rt_ctx: before I start, how hard do you want to go on these machines?
[08:35] <cpaelzer> rt_ctx: just try a bit, or really force them - potentially even trying critical things
[08:36] <cpaelzer> rt_ctx: and finally do you want something to tweak them now or a regular self regulating system?
[08:36] <rt_ctx> I have some test VMs. So I can test the harder without any problem. But for the production I must not broke any services (We are a Cloud company.)
[08:37] <rt_ctx> And regular regulating system if it's possible
[08:37] <cpaelzer> rt_ctx: ok, lets start so simple manual "throw away some cache" is: echo 3 > tee /proc/sys/vm/drop_caches
[08:38] <cpaelzer> rt_ctx: but eventually that will grow again and there are certain structures which can't be dropped, but they could be shrinked if the system would know about memory pressure
[08:39] <cpaelzer> rt_ctx: experiment wise you can just "eat" memory in there e.g. with stress-ng, the system will then try to shrink to survive and once you stop the memory eater you will ahve more free than before
[08:39] <cpaelzer> rt_ctx: I know people that add temp swap, eat mem so until it slightly swaps, then stop the eater and disable the swap
[08:39] <cpaelzer> rt_ctx: but then this is for experiments as I said
[08:40] <cpaelzer> rt_ctx: If you are looking for a sustainable setup you should look into ballooning IMHO
[08:41] <rt_ctx> Ok I tried "echo 3 > tee /proc/sys/vm/drop_caches". And I'll see about ballooning IMHO.
[08:41] <rt_ctx> Nothing special about RAM usage when I execute echo 3 > tee /proc/sys/vm/drop_caches
[08:42] <cpaelzer> that "just" drops caches, dentries and a few others
[08:42] <cpaelzer> let me pull the link
[08:42] <cpaelzer> https://www.kernel.org/doc/Documentation/sysctl/vm.txt
[08:43] <rt_ctx> Ok thank you fot the link.
[08:43] <cpaelzer> Ballooning should work these days, you just have to configure it, but I haven#t touched it in a while
[08:44] <rt_ctx> But can I decrease the purcentage (%) of cache used by the system on the RAM ?
[08:44] <cpaelzer> This will give the guest the "impression" of pressure to stay small
[08:44] <rt_ctx> Ok look great
[08:44] <cpaelzer> rt_ctx: on your last question "But can I decrease the purcentage (%) of cache used by the system on the RAM ?" I can only recommend not to go down that route as the primary target
[08:45] <cpaelzer> rt_ctx: there are plenty of discussions and all I know end with people understanding that this is not the problem
[08:45] <cpaelzer> Linux uses all ram it has, and that is smart
[08:45] <cpaelzer> if it is virtual ram you might want to make more complex thoughts, but that is where ballooning and such kick in
[08:45] <cpaelzer> not a straight "do not cache"
[08:46] <rt_ctx> Ok. So i'll try ballooning
[08:46] <cpaelzer> vice versa of you have an operation where you know that caching is stupid then you can work on that
[08:46] <cpaelzer> but the kernel has learned some tricks to avoid wrong cacheing (like read once being the first evicted and such)
[08:47] <rt_ctx> I'll check the kernel side.
[08:47] <rt_ctx> Thank you for your help cpaelzer !
[08:50]  * cpaelzer can't hide his performance past
[09:07] <TafThorne1> Good morning.  When I follow the link http://pad.ubuntu.com/U5GLIMwc2u I see "Either you have not been granted access to this resource or your entitlement has timed out. Please try again."
[09:09] <TafThorne1> In the mean time I will skim over http://packaging.ubuntu.com/html/ again.
[09:10] <cpaelzer> TafThorne1: hi
[09:10] <cpaelzer> TafThorne1: nacc has created that pad, I don't know how he had set it up and it worked for me after login
[09:10] <cpaelzer> TafThorne1: let me check
[09:13] <TafThorne1> bbs need to restart this machine after an update.
[09:22] <cpaelzer> TafThorne: welcome back
[09:22] <cpaelzer> TafThorne: I can't find why the pad isn't "open enough" - I'll let nacc sort that out later
[09:22] <cpaelzer> nacc: ^^
[09:22] <cpaelzer> TafThorne: for now lets just work here together
[09:23] <cpaelzer> TafThorne: if needed I can sync some of what we do there so others can see it at real time
[09:23] <cpaelzer> TafThorne: IIRC you were working on bug 1630516 right?
[09:23] <ubottu> bug 1630516 in logrotate (Ubuntu Zesty) "Logrotate doesn't clean old system logs, allowing them to fill the disk" [Critical,Triaged] https://launchpad.net/bugs/1630516
[09:26] <cpaelzer> TafThorne: do you want me to guide you learning on this bug for the Ubuntu Server Bug Squashing Day?
[09:26] <cpaelzer> TafThorne: or do you have other preferences
[09:29] <cpaelzer> Since I have a report on the pad not being very accessible I don't know how useful it is atm, but lets repeat the banner still
[09:29] <cpaelzer> Today is Ubuntu Server Bug Squashing Day #1: Announcement goo.gl/B98ER and a pad to track current activities at http://pad.ubuntu.com/U5GLIMwc2u
[09:29] <cpaelzer> Current chairs: cpaelzer
[09:29] <TafThorne> I have not really been working on any bug but that was the bug that brought me here.  If that is a suitable one to start out on I am happy to look at it.  If there is something easier you would recommend for a new guy I am happy to look at that.
[09:29] <cpaelzer> rbasak: if you are around as well let me know
[09:29] <cpaelzer> TafThorne: there might be easier ones, let me check one for you
[09:31] <cpaelzer> TafThorne: it always depends on the rating of the one doing the tagging - but this is the list that is considered "easy"
[09:31] <cpaelzer> TafThorne: https://bugs.launchpad.net/ubuntu/?field.searchtext=&orderby=-importance&search=Search&field.status%3Alist=NEW&field.status%3Alist=CONFIRMED&field.status%3Alist=TRIAGED&field.status%3Alist=INPROGRESS&field.status%3Alist=FIXCOMMITTED&field.status%3Alist=INCOMPLETE_WITH_RESPONSE&field.status%3Alist=INCOMPLETE_WITHOUT_RESPONSE&field.subscriber=ubuntu-server&field.tag=bitesize+
[09:31] <cpaelzer> TafThorne: is there an issue on that list that would be interesting to you?
[09:32] <rbasak> cpaelzer: o/
[09:32] <cpaelzer> hi rbasak
[09:33] <cpaelzer> That makes it
[09:33] <cpaelzer> Today is Ubuntu Server Bug Squashing Day #1: Announcement goo.gl/B98ER and a pad to track current activities at http://pad.ubuntu.com/U5GLIMwc2u
[09:33] <cpaelzer> Current chairs: cpaelzer, rbasak
[09:33] <cpaelzer> rbasak: if you happen to know how/if we could "open up" the pad more - I've got reports people can't access it
[09:33] <rbasak> The pad had problems with abuse in the past
[09:34] <rbasak> People need to join https://launchpad.net/~ubuntu-etherpad
[09:34] <cpaelzer> TafThorne: if there is none on the list which sounds like "yeah" we can as well work slowly on the logrotate issue
[09:34] <cpaelzer> TafThorne: are you fine with C coding - which seems it needed in that case
[09:35] <cpaelzer> well not coding but making sure patches apply well and such
[09:37] <cpaelzer> TafThorne: there is also much that can be done outside of packaging/coding - e.g. verifying fixes and such
[09:37] <TafThorne> I do C/C++ in my day job and have done Ruby and a few other languages in the past.
[09:37] <cpaelzer> Ok, that should be good then
[09:39] <TafThorne> cpaelzer: I am at least vaguely familiar with the problem I reported.  I was just looking over the rsyslogd, postfix and systemd: spamassassin bugs on that list you sent.  I think I have messed with the configration of most of those systems in the past.
[09:41] <TafThorne> In 2.5 hours I will be on lunch and can focus more on this.  Until then I will be a bit in and out as I do other tasks.  Sorry if I go quite for a while now and then.
[09:42] <cpaelzer> TafThorne: this is for you and others - I'm fine that you spend time here and not to force you do more :-)
[09:43] <cpaelzer> TafThorne: good cases for help that works at coming in-and-out would IMHO be the three cases titled "unmatched entries ..."
[09:43] <cpaelzer> TafThorne: those are essentially waiting to be reported upstream
[09:44] <cpaelzer> TafThorne: hopefully not hard to do, but helpful for Ubuntu overall as it will make it better eventually
[09:44] <cpaelzer> TafThorne: read the bugs and if you want to tackle those upstream reports let me know
[09:44] <cpaelzer> TafThorne: I'd add you to the pad that you can't access (sorry) to consider those, and I can help with any question arising
[09:45] <cpaelzer> TafThorne: and if you later today want to also tackle the logrotate one let me know
[09:47] <cpaelzer> TafThorne: I actually might go for the logrotate one now if you don't mind - as I read into that in the past that looks tempting to me
[09:47] <cpaelzer> TafThorne: I I'm successful you can help testing later on after you are over the three upstream reports I mentioned above
[09:48] <cpaelzer> TafThorne: FYI - those would be bug 1578004 bug 1583705 bug 1583706 - I'd be really happy if you'd take the effort or reporting them upstream and linking the upstream bugs in the Ubuntu bugs
[09:48] <ubottu> bug 1578004 in logwatch (Ubuntu) "unmatched entries for courier" [Medium,New] https://launchpad.net/bugs/1578004
[09:48] <ubottu> bug 1583705 in logwatch (Ubuntu) "unmatched entries for postfix" [Undecided,New] https://launchpad.net/bugs/1583705
[09:48] <ubottu> bug 1583706 in logwatch (Ubuntu) "unmatched entries for rsyslogd" [Undecided,New] https://launchpad.net/bugs/1583706
[09:48] <cpaelzer> TafThorne: give me an ack if you take them
[09:49] <TafThorne> cpaelzer: please feel free to work on the logrotate one if it interests you.  I will look over the other areas that you have suggested.
[10:50] <cpaelzer> rbasak: I ran into tricky versioning if you look at $ rmadison logrotate
[10:50] <cpaelzer> rbasak: X/Y/Z have the same
[10:50] <cpaelzer> rbasak: zesty (still prior to release) will get 3.8.7-2ubuntu2->3.8.7-2ubuntu3
[10:51] <cpaelzer> rbasak: but on X/Y I'm unsure - they would get 3.8.7-2ubuntu2.1 IMO
[10:51] <cpaelzer> rbasak: but then they would be fully the same version
[10:51] <cpaelzer> rbasak: if I make one with yakkety and one with xenial in the changelog the will e.g. overwrite the changes file
[10:51] <cpaelzer> rbasak: could I mark one upload for "both" or what would be the right approach here?
[10:52] <Trefex> hi all. we have a openstack with 3 ctrl, 6 compute, and we have a nice issue with dhcp_agent. the problem is that when a VM is deleted, the /leases file entry does not get removed. This means when a new VM gets the same IP it doesn't receive an OFFER from DHCP
[10:52] <cpaelzer> rbasak: never mind
[10:52] <Trefex> Does anybody know what I can do to fix this? restarting dhcp_agent refreshes the leases file and then it works fine, until VM delete / spawn again
[10:52] <cpaelzer> rbasak: In think that was the "insert release version" trick
[10:54] <cpaelzer> rbasak: so it would be 3.8.7-2ubuntu2.16.04.1 and 3.8.7-2ubuntu2.16.10.1 then - could you confirm?
[10:55] <rbasak> cpaelzer: yeah that sounds right
[10:56] <Trefex> I for the life can't figure out what is going on :(
[10:56] <rbasak> Trefex: I'd guess that your problem isn't the leases file.
[10:57] <rbasak> Trefex: it's that renewals don't work but discovers do.
[10:57] <Trefex> rbasak: how so? both the /host /addn_hosts get updated on removal
[10:57] <rbasak> Trefex: try reducing your lease time massively. I'd guess that you'll find that renewals don't work either.
[10:58] <Trefex> rbasak: and on new creation
[10:58] <rbasak> Trefex: I could be wrong of course.
[10:58] <rbasak> My hunch is that it's something to do with addressing and routing. Renewals look different at protocol level wrt. arps etc.
[11:02] <Trefex> rbasak: now you lost me
[11:06] <Trefex> rbasak: is there somebody from Canonical that can help me in this? We have support for 10 server nodes and basically I'm sure we could pinpoint the issue fast as we're trying out things since 7 days now
=== sikun is now known as tikun
[11:21] <Trefex> rbasak: i changed lease time and restart agent, when i reboot a VM you see this in the log http://paste.openstack.org/show/603740/
[11:21] <Trefex> rbasak: so i guess renewal works ?
[11:23] <ivyyy> Trefex, i'm with Canonical, i'm PMing you
=== JanC_ is now known as JanC
[12:01] <cpaelzer> TafThorne: I'm going for lunch now, I have builds for all affected releases and updated bug 1630516
[12:01] <ubottu> bug 1630516 in logrotate (Ubuntu Zesty) "Logrotate doesn't clean old system logs, allowing them to fill the disk" [Critical,Triaged] https://launchpad.net/bugs/1630516
[12:01] <cpaelzer> TafThorne: it would be great if later today you could verify those in your environment
[12:02] <cpaelzer> TafThorne: I posted all you'll need in the last bug update
[12:02] <cpaelzer> TafThorne: please let me know if you need any help doing so
[12:06] <cpaelzer> rbasak: since I go to lunch I update with only you for now
[12:06] <cpaelzer> Today is Ubuntu Server Bug Squashing Day #1: Announcement goo.gl/B98ER and a pad to track current activities at http://pad.ubuntu.com/U5GLIMwc2u
[12:06] <cpaelzer> Current chairs: rbasak
[12:39] <cpaelzer> Today is Ubuntu Server Bug Squashing Day #1: Announcement goo.gl/B98ER and a pad to track current activities at http://pad.ubuntu.com/U5GLIMwc2u
[12:39] <cpaelzer> Current chairs: cpaelzer, rbasak
[13:18] <rbasak> cpaelzer: I was going to look at bug 1503611 as you flagged it Critical, but given that you already understand the issue well would it be easier for you to continue driving it?
[13:18] <ubottu> bug 1503611 in spamassassin (Ubuntu) "systemd: spamassassin not starting after upgrading to 15.04" [Critical,Triaged] https://launchpad.net/bugs/1503611
[13:21] <cpaelzer> rbasak: please feel free to look at it, I'm open to a quick HO to sync on the actual final todo (which is way less than the whole bug text)
[13:21] <cpaelzer> rbasak: I don't think I'd have a huge advantage over you handling that bug
[13:21] <cpaelzer> rbasak: quite the opposite, while I debugged the reasons the solution might be more in your experience than mine
[13:22] <rbasak> OK I'll take a look thanks
[13:25] <rbasak> cpaelzer: I think I follow. Let me check. In the past, the user was supposed to set ENABLED=1 in /etc/default/spamassassin. Now, the user is supposed to run "systemctl enable spamassassin.service" instead. So, on upgrade, the state goes to enabled to disabled as there is no upgrade path. Correct?
[13:25] <cpaelzer> rbasak: exactly
[13:25] <rbasak> cpaelzer: no need for a Hangout I think?
[13:25] <rbasak> OK. Thanks!
[13:25] <cpaelzer> rbasak: no you got it
[13:26] <oneil> hi, one persone know openwrt ?
[14:09] <rbasak> cpaelzer: when you tested that spamassassin bug, did you also see the conffile prompt on upgrade?
[14:14] <cpaelzer> Not that I remember
[14:14] <cpaelzer> rbasak: ^^
[14:14] <cpaelzer> should I test anything to confirm?
[14:15] <rbasak> No need if you don't recall.
[14:31] <bgardner> Hey all, I have a t1.micro AWS instance running 16.04.2 LTS that had unattended-upgrades turned on.  It emailed me asking for a reboot which I did and now it just kernel panics.  Final log message is:  "VFS: Cannot open root device "LABEL=cloudimg-rootfs" or unknown-block(0,0): error -6"
[14:33] <bgardner> This is not a mission-critical server and I can just blow it away and start over, but I'd like to learn what happened and fix it if I can...
[14:34] <cpaelzer> bgardner: do you think this is bug 1661292?
[14:34] <ubottu> bug 1661292 in OpenStack Compute (nova) "VFS: Cannot open root device "LABEL=cloudimg-rootfs" or unknown-block(0,0): error -6" [Undecided,New] https://launchpad.net/bugs/1661292
[14:34] <bgardner> cpaelzer: Reading...
[14:34] <cpaelzer> From the bug it seemed to be a racy case - here not an aws, but local KVM it seems
[14:35] <cpaelzer> It seems like populating the LABELs vs using it could race to me
[14:36] <cpaelzer> The message itself doesn't help that much as it just mean "can't find this", why it can't is the bigger question
[14:37] <cpaelzer> yet as races are hard to debug one would need at least a dump of the case, or even better some test that gets it to show up reliably
[14:41] <cpaelzer> bgardner: thanks for getting me to think on that bug, should be filed against the kernel if-any IMHO
[14:42] <bgardner> cpaelzer: Interesting, let me bounce a few times and see if it is intermittent for me.  I'll comment on the bug if it seems relevant to this case.
[14:42] <bgardner> cpaelzer: And thank you\
[14:42] <cpaelzer> bgardner: thank you, no matter how weird that sounds - I hope it fails all the time for you
[14:43] <cpaelzer> (just this instacne reboot)
=== din0 is now known as dino82
[14:44] <bgardner> cpaelzer: Agreed, easier to debug
[15:06] <nacc> cpaelzer: TafThorne: looking into the pad issues
[15:07] <nacc> cpaelzer: yes, that's the right versioning, documented ont he security wiki
[15:07] <TafThorne> nacc: thank you
[15:08] <cpaelzer> nacc: welcome
[15:08] <bgardner> cpaelzer: Nine reboots later, still exactly the same error.  I'll head to the ticket.
[15:08] <TafThorne> cplaelzer: I will try and test out the updates at the end of the day.
[15:08] <cpaelzer> Thanks
[15:08] <TafThorne> cplaelzer: which is ~2 hours away for me.
[15:08] <cpaelzer> TafThorne: if until then you also find the time to do the upstream reports on the three minor issues you are my #1 today
[15:08] <cpaelzer> not that being my #1 is worth anything but a thankful cpaelzer
[15:09] <cpaelzer> nacc: your appearance makes me update my regular post
[15:09] <cpaelzer> Today is Ubuntu Server Bug Squashing Day #1: Announcement goo.gl/B98ER and a pad to track current activities at http://pad.ubuntu.com/U5GLIMwc2u
[15:09] <cpaelzer> Current chairs: cpaelzer, rbasak, nacc
[15:09] <nacc> cpaelzer: thanks :)
[15:09] <nacc> TafThorne: it sounds like (based upon rbasak's comment), you need to join a specific team on launchpad (~ubuntu-etherpad)?
[15:12] <TafThorne> > "if until then you also find the time to do the upstream reports on the three minor issues you are my #1 today" eh?
[15:12] <jge> hey all, can someone shed some light.. my fresh ubuntu server install is showing it allocated 63.9G to SWAP, that seems excessive to me leaving my root partition to be 25G.. why did it pick so much?
[15:12] <cpaelzer> TafThorne: the three bugs you picked this morning with the "unmatched ..." on logwatch IIRC
[15:13] <TafThorne> cpaelzer: Oh I see.  Well I will see what I can do.
[15:13] <cpaelzer> jge: I think there were some changes recently, but in the past I think it had 50% of memory or even 100% for suspend
[15:13] <cpaelzer> jge: so you might have a lot of memory, which means you should tweak the default
[15:14] <cpaelzer> jge: might I ask what ubuntu release you installed?
[15:14] <jge> cpaelzer: I do, 64G
[15:14] <cpaelzer> IIRC xnox was working on those things in the installer, but he is away this week
[15:14] <nacc> iirc, default is 300% of system memory, maybe?
[15:14] <jge> cpaelzer: im running 16.04 LTS
[15:14] <TafThorne> nacc: So I got to https://launchpad.net/~ubuntu-etherpad and click "Join the team" and wait?
[15:14] <nacc> i know i've seen some preseed docs that say to alter that by default, for example
[15:14] <nacc> TafThorne: yes (and I might be able to approve it, not sure)
[15:15] <nacc> TafThorne: or ask someone to :)
[15:15] <jge> cpaelzer: is there a way to resize without reinstalling again?
[15:15] <nacc> jge: reformatting swap shouldn't be a big deal, but resizing the other partitions to grow them (presuming you want to) might be hareder
[15:15] <nacc> *harder
[15:16] <jge> yep, resizing my root partition to grow them .. I'll just reinstall no big deal.
[15:18] <jge> since I have to tweak my swap partition on install, any pointers as to what my swap should be with this much memory?
[15:18] <cpaelzer> jge: I knew I remembered something https://lists.ubuntu.com/archives/ubuntu-devel/2016-November/039538.html
[15:18] <cpaelzer> jge: but the thread didn't come to a conclusion, not sure if anything was changed on the defaults
[15:18] <cpaelzer> jge: in the thread are also suggestions on a proper 2017 server swap size
[15:18] <jge> aha
[15:19] <jge> perfect, will read through it.. thanks cpaelzer
[15:20] <nacc> yeah, swap files are probably the better choice for that much memory (tbh)
[15:21] <nacc> jge: do you actually overcommit your systems?
[15:21] <nacc> jge: it really depends on workload what we'd recommend, i'd think
[15:21] <jge> not really, 64G is way more than we will ever need
[15:22] <nacc> jge: yeah, then just don't have swap :)
[15:22] <nacc> jge: you don't technically need it
[15:22] <nacc> jge: and you can always use swapfiles (presuming you have lots of disk) later if you find that you could use it
[15:22] <cpaelzer> nacc: a bit to just not die can't hurt, it gives you the chance to realize slow before dead :-)
[15:23] <cpaelzer> just monitor swapping which should never occur
[15:23] <jge> good points
[15:24] <cpaelzer> jge: and I see xnox completed it in 17.04 - see http://blog.surgut.co.uk/2016/12/swapfiles-by-default-in-ubuntu.html
[15:24] <nacc> cpaelzer: yeah, it really depends on if they never get close to 64G
[15:25] <jge> I just did 8 servers last night, so used to just hitting enter on those prompts failed to see it allocated that much to swap
[15:25] <jge> have to redo them again , pita
[15:25] <cpaelzer> preseed and/or maas to the rescue
[15:29] <nacc> yeah if they are duplicate servers, i would automate it
[15:45] <nacc> cpaelzer: fyi, LP: #1654011, they appear to be on trusty
[15:45] <ubottu> Launchpad bug 1654011 in linux (Ubuntu) "iscsi target not DKMS compiling for kernel 4.4.0-57" [Undecided,Incomplete] https://launchpad.net/bugs/1654011
[15:48] <nacc> cpaelzer: using the HWE stack. I just duped it to the bug i just fixed in the package
[15:49] <cpaelzer> thanks nacc
[15:50] <nacc> cpaelzer: np, also, if we do see future bugs against it, let's see if we can't get peopel to stop using it (on 16.04 and on) and use in-kernel iscsi_target_mod and tgt
[15:51] <nacc> cpaelzer: i will try and write something up for that
[15:51] <cpaelzer> yeah, fixing dkms in old packages for newer kernels is always a pain
[15:52] <cpaelzer> lets get rid of at least that
[15:54] <cpaelzer> Today is Ubuntu Server Bug Squashing Day #1: Announcement goo.gl/B98ER and a pad to track current activities at http://pad.ubuntu.com/U5GLIMwc2u
[15:54] <cpaelzer> Current chairs: rbasak, nacc
[16:27] <jge> hey guys, forgot to ask.. I'm thinking of doing software raid (raid1) with two disks as my root directory and do my /boot parition on a separate SSD drive.. is this a wise design? how easy would it be to recover if my boot (ssd drive) fails?
[16:28] <jge> just get a new drive, format, put my /boot partition there again and then mount the others on the raid array?
[17:12] <powersj> nacc: rbasak: could either of you read my comment to LP: #1625043? I'm hoping I fully grasp what is going on there.
[17:12] <ubottu> Launchpad bug 1625043 in tomcat7 (Ubuntu) "tomcat7 package not compliant with tomcat specification" [High,Confirmed] https://launchpad.net/bugs/1625043
[17:12] <nacc> powersj: looking
[17:13] <nacc> Today is Ubuntu Server Bug Squashing Day #1: Announcement goo.gl/B98ER and a pad to track current activities at http://pad.ubuntu.com/U5GLIMwc2u
[17:13] <nacc> Current chairs: rbasak, nacc
[17:14] <nacc> powersj: there are a number of bugs about this, actually, we should dupe them all together once we have a fix (still reading)
[17:15] <powersj> (I think I got both of the dups I saw)
[17:15] <nacc> powersj: cool
[17:16] <nacc> powersj: we may also want to loop jamespage in
[17:16]  * jamespage hides
[17:17] <powersj> nacc: so 2 concerns, 1) dropping support for other older JRE seems wrong, but it is broken so I feel slightly justified, especially since Debian did and 2) haven't thought through an SRU and possible impact. Yes it is broken, but what else might I break?
[17:19] <nacc> powersj: right, it's probably not suitable for SRU in and of itself
[17:19] <nacc> powersj: was tomcat7 also changed liek that in debian?
[17:20] <powersj> nacc: I don't believe so because it was dropped from unstable a while back I believe
[17:20] <nacc> powersj: ah right
[17:22] <powersj> I'm kind of on the page of, get tomcat8 synced with Debian for 17.10 to get the fix there, note the problem and how using Java 8 or later is better, and move on.
[17:30] <TafThorne> I was approved for the team list by jorge.  That has allowed me to follow the link through to the Public Pad.
[17:31] <nacc> TafThorne: ah great!
[17:31] <nacc> TafThorne: sorry, meant to ping you back after jcastro approved that
[17:32] <TafThorne> nacc: not a problem and saw an email notification and knew what it related to.
[18:32] <nacc> powersj: i'm starting to think (for your own sanity) to have two bugs, one for tomcat7 (with tasks as appropriate, given that zesty no longer ships a tomcat7 binary package)
[18:33] <nacc> powersj: and one for tomcat8
[18:33] <nacc> powersj: normally, i'd say those could be in the same bug, but i think the fixes will be different and i think most users are ocmplaining about tomcat7 not being b-c to java7, not tomcat8 to java7 (unless i misundersatnd)?
[18:34] <powersj> nacc: you are correct the focus is on tomcat7, but since I was looking at 7 figured looked at 8 while I was at it.
[18:34] <powersj> however, if I am not going to SRU, is there even a need for the tomcat 8 one?
[18:34] <powersj> or would that be useful for documentation purposes?
[18:34] <nacc> powersj: understood, i just thinking it's less of an issue for tomcat8 (i think)
[18:35] <nacc> *was just
[18:35] <jge> trying to do a manual partition of my drive and set up LVM, I would like to use 80% and leave the rest as available space for LVM use.. I created 2 partitions so far. 1 primary with 80% of my disk mounted as /. Second 2G for swap and the remaining 22G of free space, how would I set LVM with all of this?
[18:35] <jge> did I do it right?
[18:36] <nacc> powersj: so does tomcat7 work with java8? if you were to fix it to be java7 compatbile, it would then not work with java8, right?
[18:37] <powersj> tomcat7 will work with java 8 as it stands today. The fix proposed is to just drop support for java 7. However, if we were to compile with java 7 it should still be compatible with java 8 afaict.
[18:38] <nacc> powersj: also consider LTS -> LTS upgraders -- and mabye look at what tomcat6 -> tomcat7 did (precise -> trusty)
[18:39] <nacc> powersj: in that i think if you were on 14.04 and using tomcat7, it should continue to work on 16.04 (although tomcat7 went from main -> universe)
[18:39] <nacc> powersj: it was not intentional that it would be broken, at least
[18:40] <powersj> nacc: by continue to work, does that also include working with the same version of java?
[18:41] <nacc> powersj: that's what i'd like to know for 12.04 -> 14.04 :)
[18:42] <nacc> powersj: that transition was both tomcat6 (main) -> tomcat7 (main) and openjdk-6-jre (main) -> openjdk-7-jre (main)
[18:44] <powersj> I can only go off of what the Tomcat compatibility guide says, which claims tomcat 7 should support java 6 and later. However, if Tomcat 7 is the default JRE in 14.04, then there will of course be this exact same situation if someone tries to use Tomcat 7 with Java 6
[18:44] <powersj> ugh if java 7 is the default jre in 14.04 is what I meant
[18:45] <nacc> powersj: right, but i wonder if some twiddling was done to the package :)
[18:45] <nacc> powersj: agreed on the upstream, i'd like to know what happens in practice
[18:46] <nacc> powersj: it might be that this just needed to be better documented in the release notes
[18:46] <powersj> Right, however based on the debian bug I don't believe this is just a doc issue
[18:47] <powersj> I will see if anything changed between precise and trusty though
[18:47] <nacc> powersj: right, it might not be -- and if this broke between precise and trusty in a similar way, then I'm happy to not fix anything :)
[18:50] <nacc> powersj: are you ok with confirming that 12.04 -> 14.04? you can also throw it back to me if not :)
[18:51] <powersj> nacc: confirming the issue can occur there as well? sure. I can check the init file for previous support and give an example of something found in Java 7 but not Java 6
[18:52] <nacc> powersj: yeah
[18:54] <jge> anyone? trying to manually partition a disk and use LVM with it, I would like to end up with 80% of the disk being used for OS and the rest as free space so I can resize VG later on.. anyone can point me in the right direction
[18:54] <jge> I've always done it using the guided partitioning but have no idea how that works in a manual install
[18:56] <nacc> jge: you already created/mounted a primary partition for /
[18:56] <nacc> jge: ?
[18:57] <jge> nacc: yep, so far I have allocated 80% of disk to / , then i have another as swap and the remainder free space as LVM
[18:58] <nacc> jge: but you don't want / on lvm?
[18:58] <jge> looks like this: primary (96G) ext 4 / , logical (2G) swap, logical (22G) lvm
[18:59] <jge> nacc: Yes i do, that would allow me to take snapshots, expand and resize it later if i need
[18:59] <nacc> jge: then your partitioning is incorrect
[19:00] <nacc> jge: lvm is below filesystems not above them
[19:00] <nacc> jge: https://wiki.ubuntu.com/Lvm
[19:01] <jge> hmm ok, so i need to get rid of that primary partition and set it to be used for LVM
[19:02] <jge> and then mount my paritions on them
[19:02] <nacc> jge: a) LVM partition -> b) LVM PV -> c) LVM VG -> d) multiple (potentially) LVM LVs
[19:02] <nacc> jge: you don't mount partitions on an LVM
[19:02] <nacc> jge: i suggest you read the above wiki page
[19:14] <nacc> Today is Ubuntu Server Bug Squashing Day #1: Announcement goo.gl/B98ER and a pad to track current activities at http://pad.ubuntu.com/U5GLIMwc2u
[19:14] <nacc> Current chairs: nacc
[19:15] <jge> nacc: hmm ok, thanks for the link .. think I got it now. Selected Logical Volume Manager from the manual menu, created a volume group with my disk, then two logical volumes (root with about 80% of my disk space, and swap of about 2G)
[19:18] <jge> nacc: probably not needed to have swap be a logical volume right?
[19:19] <nacc> jge: probably not; like i said you might not need swap at alla nd you can always use swapfiles :)
[19:19] <jge> that's true, thank you nacc
[19:20] <nacc> jge: but yes, i'm not sure you need swap in a lvm, beyond consistency
[19:21] <jge> nacc: I put it in there, because I remember when using the guided partitioning swap showed up in lvm.
[19:21] <nacc> jge: right, because guided partitioning is trying to cver all use case (if i had to guess)
[19:21] <nacc> jge: i mean, yes you can have it, it may never get used
[19:21] <jge> i guess so
[19:21] <nacc> jge: i don't think it really matters if it's on lvm or not
[19:22] <jge> feel like I learned something new today even if it took this long to figure out ;)
[19:23] <jge> now I gotta figure out how to do software raid and lvm
[19:23] <powersj> nacc: if I am preparing a second SRU for a package, and the first has not already been accepted, what should I put as the version in the change log?
[19:24] <nacc> powersj: not accepted or not verified?
[19:24] <powersj> nacc: rbasak still needs to accept the changes, so not even in verification yet
[19:24] <nacc> powersj: if not accepted, you can ask the sru team to reject your existing upload
[19:24] <nacc> powersj: and then you can upload the same version up with more changes
[19:30] <nacc> i think technically two people can even uplaod the same version into the queue, but i'm not 100% on that
[19:30] <nacc> but only one would get accepted
[19:43] <powersj> nacc: when I run dch my email shows up as username@hostname, however in my .bashrc I set DEBEMAIL, is there something else I'm suppose to set?
[20:26] <drab> https://bugs.launchpad.net/ubuntu/+source/biosdevname/+bug/1284043
[20:26] <ubottu> Launchpad bug 1284043 in biosdevname (Ubuntu) "udev renaming the same hardware network i/f to different name, breaks networking and firewall" [High,Confirmed]
[20:27] <drab> any idea what to do about that?
[20:27] <drab> it's pretty darn annoying and problematic :(
[20:27] <drab> screwing up all my bridges and bonds and forcing me to do installation manually/do quite a few hacks around it
=== dzragon^ is now known as dzragon
[21:20] <sarnold> drab: that bug is three years old and has enough comments that it's probably accumulated a lot of unrelated cruft. Probably it'd be best to file a new bug.
[21:43] <powersj> nacc: when I did a usd clone of mongodb and switched to the yakkety/devel branch it shows up as version 2.6.12-2ubuntu2, however the latest version in yakkety is 2.6.11-1ubuntu1. Am I missing something?
[21:55] <drab> sarnold: I found this which actually is very recent and exactly the same as mine: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1578141
[21:55] <ubottu> Launchpad bug 1578141 in linux (Ubuntu) "Predictable interface names partially broken with igb driver" [Medium,Confirmed]
[21:55] <drab> same igb driver
[21:57] <drab> he's even got basically the same mobo...
[21:57] <drab> for a second I thought I filed it and forgot about it :P
[21:57] <sarnold> heh
[22:00] <sarnold> drab: nice that bug hasn't been polluted yet. I suggest reporting it to systemd upstream, add a link to it in a comment here, and try to get some traction again
[22:01] <drab> sarnold: last few comments on that bug seems to suggest it's a hw problem tho :/
[22:01] <drab> bios to be precise
[22:01] <drab> and systemd not handling that misinformation well
[22:01] <drab> altho there's also the mention of an ubuntu patch
[22:01] <drab> those guys have done quite in depth debugging
[22:01] <sarnold> drab: yeah, it's a good bug.
[22:02] <drab> minus the almost one year with no resolution :P
[22:02] <sarnold> drab: but bioses have bugs and it's unrealistic for systemd to say "go get a fix from your vendor" to every one of them. most bioses are thrown over a wall and you never have a chance to even complain to the vendor..
[22:02] <drab> fair enough
[22:32] <jge> anyone ever configured software raid-1, I have three disks in total: 1 SSD, 2 HD (part of the array) and 3 HD (part of the array). When I get to the last step on Ubuntu's installation where it asks to install GRUB boot loader, it gives me an option to select which drive to install it to.. I select my second disk but it fails to boot up. Any clues?
[22:32] <jge> the SSD drive is not being used for anything
[22:34] <drab> jge: I run quite a few raid1s and install the boot loader on all disks part of the mirror (so taht I can boot from any)
[22:35] <drab> it's failing to boot up probably because it's not trying the second disk for some reason
[22:35] <sarnold> is that in itself cause for concern? e.g. maybe the raid for booting isn't as useful as it could be or should be?
[22:35] <jge> drab: yeah the plan is to install it on the second disk once I boot up into my system, but wouln't this be redundant since it's probably mirrored there anyway?
[22:35] <sarnold> or is it expected that manual effort to boot from the right media in the case of drive1 failing is necessary?
[22:37] <jge> sarnold: I take it you're not a fan of software raid? :)
[22:39] <drab> sarnold: booting from a failed mirror isn't the most common thing if that's what you mean
[22:39] <sarnold> jge: my new computer uses both mdraid and zfs. I use zfs on the data, mdraid on the OS drives, and I have -no idea- how the mdraid works. none. I pray I never need to figure it out..
[22:39] <drab> unless the disk leads the system to freeze
[22:39] <drab> normally you'd put in a new drive without the need to shutdown, assuming you have hot swap, which pretty much any modern system does
[22:39] <drab> that'd be the other case and maybe the more frequent one
[22:40] <drab> needing to shut down the machine to add the drive or the box locking up when you try to hot swap
[22:40] <drab> it used to happen back in the days, not sure about now, been out of the loop for a while
[22:41] <drab> sarnold: I do the same thing for my NAS, 2 m2 SSDs + 6 HDs in zfs and a SLOG nvme
[22:41] <jge> drab: hmm yeah so what I'm doing is right then? selecting one of the disks of the array when it asks where to install the GRUB boot loader and then making sure my BIOS settings are set to boot from that drive?
[22:42] <drab> jge: you should be able to select them all
[22:42] <sarnold> drab: hehe yeah very similar setup: 2 ssds + 9 hdds + nvme slog/l2arc (slog appears unused, I keep thinking I should take it back out and re-partition that nvme to just be l2arc)
[22:42] <jge> drab: tested on another identical machine I was working on and it booted fine! something must be up with that other
[22:43] <drab> sarnold: are you finding l2arc useful? I've been mostly just reading and it didn't seem that it made a lot of sense if you had enough RAM and didn't read a lot of new things all the time, but maybe you do
[22:43] <drab> jge: maybe, maybe disk order
[22:43] <drab> some bios settings
[22:43] <jge> yep, that must be it.. thanks drab
[22:44] <drab> mmmh bridging over bonding... this is getting a tad complicated to automate...
[22:44] <jge> drab: should I just do a grub-install /dev/sdc (other disk on the array) once I boot up?
[22:45] <drab> jge: the mirror should take care of it, but honestly I have never directly tested that part, like I said I install it on all drives part of the mirror
[22:45] <drab> but it should work I guess
[22:46] <drab> does it even let you choose the md device at all? or is it just offering raw devices?
[22:48] <jge> drab: just raw devices
[22:48] <sarnold> drab: l2arc is insanely useful for me; I have to admit the nvme storage isn't anywhere near as fast as I had hoped it would be when I bought it, but it's a drastic assistance for my workload
[22:49] <drab> jge: k, well, up to you, I'd just select them all and sleep better, but if you're just testing I guess give it a try
[22:49] <drab> jge: also fyi in case you haven't seen it, there's a bug affecting mdadm/initrd and degraded boot
[22:49] <jge> drab: it wont allow you to pick multiple, only one
[22:49] <drab> as it stands xenial won't boot from degraded raid
[22:50] <drab> it'll just sit there and eventually time out adn drop you in a initrd shell
[22:50] <drab> (you can resume from there tho)
[22:50] <jge> ohh wow, no I was not aware of it
[22:51] <jge> so if the array is degraded and reboot the machine it wont come back uup?
[22:51] <drab> correct
[22:53] <jge> I've never done software raid with ubuntu, in case a drive goes bad does it allow you to swap it out for a new one and rebuild it without rebooting? cause that would be bad.
[22:53] <jge> drab^
[22:55] <drab> jge: yeah that's no prob
[22:56] <jge> ok got it, will not it thanks from bringing it up
[22:56] <jge> note*
[22:57] <drab> http://askubuntu.com/questions/789953/how-to-enable-degraded-raid1-boot-in-16-04lts
[23:01] <jge> thank you drab
[23:01] <nacc> powersj: sorry, was afk for a bit, let me look