[04:06] <drab> anybody got any suggestion why testing with fio reads would come back slower than writes?
[04:06] <drab> using O_DIRECT, no BBU caches, "simple" nvme device
[04:15] <patdk-lap> hmm, flash tends to buffer writes
[04:26] <average_guy> where are the commands you enter stored? so when you hit up they reappear
[04:27] <drab> patdk-lap: mmmh, that could be, however I had a spare one of these nvme and put it into a machine that's slower/worse than the server I was testing on and the gap doesn't show
[04:28] <drab> so it's something on this paricular machine that's causing the reads to be 2/3 slower
[04:28] <drab> I'm testing with fio using a test script so that's consistent on the machines with the same nvme device and same pcie adapter in a x16 slot
[04:28] <drab> even more interesting, I found out that's really concurrency killing it
[04:29] <drab> if I run fio with 1 job, the newer server obliterates the others and reads are 2x writes almost
[04:30] <drab> but if I just up jobs from 1 to 4 booom, reads crawl from 2.7GMps down to 200MBps
[04:31] <drab> writes also slow down, but of only a "few" hundred MBs, staying about 700 or so, which is btw what the manufacturer spec'ed this for, so that's cool
[04:37] <drab> if I didn't have the other system to compare to I'd agree, since flash is indeed caching and this paricularly one has a nice SLC cache before writing to TLC Nands
[04:38] <drab> so maybe the real issue here is that concurrency simply kills performances on this box period but doesn't on another fo reasons I dont get (same ubuntu xenial with same kernel)
[07:40] <vamadir> I buy vps server with ubuntu. And now i have 2 process GETTY, i think its server provider connet(some reboot etc...) from web panel. But i want delete this, its will be ok?
[07:40] <vamadir> i need only ssh
[08:01] <lordievader> Good morning
[09:48] <jamespage> rbasak, around? I have a query re the current inflight ceph sru's
[10:37] <m1dnight_> Anyone here have experience with msmtp?
[10:41] <m1dnight_> msmtp keeps setting my from address to "default", while I have clearly configured the from to be "foo@example.com"
 Sender address rejected: need fully-qualified address
[11:18] <jamespage> rbasak, nm I figured it out - the tag was not quite 'verification-done' - doh!
[14:01] <frickler> cpaelzer: https://bugs.launchpad.net/ubuntu/+source/strongswan/+bug/1587886 is still affecting me on Xenial it seems (when running neutron-vpnaas), is there a chance to backport the fix? or get the version from zesty into UCA?
[14:10] <cpaelzer> hi frickler, let me take a look
[14:10]  * cpaelzer has no ECC-memory in his brain :-/
[14:12] <cpaelzer> frickler: are you the Dr. that did the last update a few minutes ago on  the bug?
[15:25] <frickler> cpaelzer: yes. and neutron does use the plain xenial package indeed
[15:28] <cpaelzer> frickler: ok, since I have no good local repro I have provided a ppa with the fix backported to Xenial
[15:28] <cpaelzer> frickler: it is currently in some more testing, but if you could verify it solves it for you that would be great
[15:29] <cpaelzer> frickler: if it does comment on the bug please (I did so announcing the ppa)
[15:30] <cpaelzer> frickler: also if you could provide a short "Test Case" section for https://wiki.ubuntu.com/StableReleaseUpdates#SRU_Bug_Template that would be great as well
[15:30] <cpaelzer> frickler: I'm good adding the rest
[15:31] <frickler> cpaelzer: sure, thx
[15:54] <frickler> cpaelzer: so the fix works, minor issue: updating only strongswan itself doesn't pull in the updated other packages automatically, needed to update strongswan-charon and -starter explicitly
[15:55] <frickler> cpaelzer: I'll try to find a simpler test case than install OpenStack with certain settings and configure a VPN link now
[15:57] <cpaelzer> thanks
[17:26] <genii> drab: Did you make any headway on the preseed file?
[17:42] <matsaman> just installed 12.04.5 as a virtualbox guest, and it seems like trying to apt-get install anything throws 'no installation candidate'
[17:45] <nacc> matsaman: did you run `apt-get update` first?
[17:46] <matsaman> I've run that, yes
[17:47] <nacc> matsaman: can you pastebin that output, and then an example `apt-get install ...` output?
[17:50] <matsaman> looks like https://bugs.launchpad.net/ubuntu/+source/debian-installer/+bug/1430648
[17:51] <matsaman> I'm doing an update, upgrade, & dist-upgrade now
[17:51] <matsaman> I'm assuming this is because I opted out of automatic updates at install time
[17:52] <matsaman> which is a habit I've developed because that has, IME, _increased_ the likelihood of the installer failing
[17:52] <matsaman> the irony, there is no escape
[17:52] <nacc> matsaman: :)
[17:53] <matsaman> as if I'd be installing 12 if I wanted updates anyway! heh
[17:53] <zul> cpaelzer/coreycb: im not able to reproduce http://paste.ubuntu.com/23949041/
[17:58]  * matsaman sighs
[17:58] <matsaman> no improvement
[18:02] <zul> coreycb: rc1 incoming
[18:02] <matsaman> oh thank heaven
[18:02] <matsaman> fixed with http://askubuntu.com/questions/297757/why-after-fresh-ubuntu-12-04-installation-update-arent-being-installed#answer-300930
[18:27] <drab> genii: no, I got stcuk with another nasty issue, trying to debug some nvme device that's giving out seemingly random performances...
[18:28] <drab> also trying to reconcile dd with fio... I thought if I ran fio with 1 job same filesize same block value I would get the similar readings, but nope
[18:28] <drab> fio's value are way way lower for whatever reason I don't get (in a reading O_DIRECT should play no role and a sync and flush buffers before running dd)
[19:32] <sarnold> drab: what exact dd commandline are you using?
[19:45] <blueking> I have samba share  on mine ubuntu server.. but workgroup/ubuntu server aren't visible on my lan, tho I could connect to mine samba share as it was configured before... changes I've made  added two more nic, made those two nic into one  interface name bond0  interface that samba share was on before are disabled..  I've changed smb.conf    I need to change somewhere else too ?
[19:50] <sarnold> blueking: check samba logs, maybe it's telling you the error..
[19:50] <blueking> ok
[19:57] <blueking> how long time should it take to have devices visible at explorer  networked devices visible 'WORKGROUP' as an example..
[19:59] <sarnold> try using smbclient directly without trying to browse
[20:00] <sarnold> browsing is handled by a different protocol, and it may not be indicative of the success or failure of the sharing protocol
[20:04] <blueking> but samba share works from old setup.. I could access it from mine windows pc
[20:06] <sarnold> when troubleshooting it's worth trying to break big problems into small problems and then solve them one at a time, from the most basic upwards
[20:06] <sarnold> you know how it goes, starting with "is it plugged in and turned on?", "is the network cable plugged in?", "are there link lights on both the NIC and the switch?" and onwards and upwards :)
[20:07] <sarnold> bonding makes things a bit more complicated since I don't know how to e.g. check if the machine's responding to arps correctly; but you could test ping and ssh inbound and outbound and make sure that the machine still has networking at all
[20:07] <sarnold> and if it appears to have networking still, then you can focus on the two protocols in windows sharing -- the sharing bit (smb) and the browsing bit (nmb)
[20:09] <sarnold> bonding makes things a bit more complicated since I don't know how to e.g. check if the machine's responding to arps correctly; but you could test ping and ssh inbound and outbound and make sure that the machine still has networking at all
[20:09] <sarnold> and if it appears to have networking still, then you can focus on the two protocols in windows sharing -- the sharing bit (smb) and the browsing bit (nmb)
[21:08] <ikonia> win 10
[21:08] <ikonia> oops
[21:09] <compdoc> wot?!   ban!
[21:39] <powersj> nacc: can you import dlm for jgrimm and I?
[21:40] <nacc> powersj: running now
[21:41] <powersj> nacc: great thx!
[21:44] <nacc> powersj: of course, should be done soon
[21:44] <nacc> powersj: done
[22:25] <powersj> nacc: thanks again
[22:35] <nacc> powersj: np!
[23:13] <drab> sarnold: to write I'm testing with dd of=/dev/nvme0n1 if=/dev/zero count=10000 bs=128K conv=fdatasync,notrunc oflag=direct
[23:13] <drab> sarnold: for reading sudo dd if=/dev/nvme0n1 of=/dev/null count=10000 bs=128K
[23:13] <genii> drab: What kind of drive is it?
[23:14] <genii> ( the EVO apparently have some issue)
[23:14] <sarnold> drab: good good
[23:15] <drab> genii: http://mydigitalssd.com/pcie-m2-ngff-ssd.php
[23:15] <sarnold> drab: the downside of course is that now I'm out of suggestions why the two would have different values :)
[23:15] <drab> sarnold: :)
[23:15] <sarnold> drab: if fio were -faster- that'd make loads of sense, I could say "oh it's probably doing multiple requests"
[23:16] <sarnold> but fio is slower and I'm 99% sure dd isn't going to be submitting multiple IO requests :/
[23:16] <drab> yeah, that's also what I don't get...
[23:16] <sarnold> drab: maybe capture .1s of strace output? I'm curious what the write patterns are for both
[23:17] <drab> the other thing is that results are inconsistent... sometimes I get lots of context switches and low tput, sometimes high tput and pretty high ctx, other times high tput and low ctx
[23:17] <drab> these are test servers doing nothing other than being hammered with fio, so I don't get why the huge flactuation in ctx
[23:17] <drab> but it's the only thing I can see diff besides the speeds
[23:17] <drab> (in fio output)
[23:19] <drab> it might be the hw, but blaming it on the hw isn't a great solution... we're a non profit and money is tight, trying to get some older drives to behave by putting a decent cache in front of them
[23:19] <drab> and this seemed like a very good option, low price, good perfs, really good reviews on tomshardware & a few other places
[23:19] <drab> it's not top of the line, but plenty for us
[23:20] <drab> except that I just can't get consistent numbers no matter what
[23:20] <sarnold> drab: ohhhhhhh
[23:21] <drab> what, you solved it? :)
[23:21] <sarnold> drab: my new theory -- the drive's firmware is compressing the hell out of a string of zeros. it's writing almost nothing most of the time.
[23:21] <sarnold> drab: probably fio knows about this and is sending uncompressable blocks to the disk to write.
[23:22] <genii> According to the webpage for the drive, it does have some fancy firmware
[23:22] <drab> sarnold: that's an interesting thought, I might try to find a way to test around it
[23:23] <drab> thank you
[23:23] <drab> I'm still left with the problem of inconsistent tests tho
[23:23] <sarnold> yeah :/
[23:23] <drab> I've had maybe 3 or 4 times in a row fio giving me back 1.5GBs reads
[23:24] <drab> and then seen those plummet to 700MBps
[23:24] <drab> and no way to get them back up, not even with a reboot, nothing
[23:24] <sarnold> I started by looking at the page to see if they mentioned how much overprovisioned space they had, in the hopes that that might help dictate just how much garbage compaction it might do, and -that- is what kicked off the neuron that remembered that most flash firmwares try hard to reduce writes..
[23:24] <sarnold> OUCH
[23:24] <sarnold> that's one heck of a penalty once you hit it
[23:25] <drab> then at some point I tried to write a 10GB file as a test, and got 1.5GPBs again...
[23:26] <drab> and that is stable on one test machine, meaning I keep getting 1.5GBPs, but I can never get that speed on a 1GB file sample, needs to be at least 3GB to get close to 1GBps
[23:26] <drab> on the other machine however going back and forth with 10GB makes no diff, it's "stuck" at 700MBps
[23:26] <sarnold> where is your 1gig file coming -from-?
[23:26] <drab> I have no idea how fio gens that
[23:27] <sarnold> unless it was already stuffed in RAM you might be testing the read speed of the drive holding it, or the throughput of the pcie bus that's got the devices..
[23:27] <drab> I guess I might look into that since I'm out of options
[23:27] <sarnold> probably fio does the right thing and generates a megabyte or two of uncompressable data at startup and just re-uses it
[23:28] <sarnold> though that might encourage folks to do de-dup. so maybe they'd have light-weight modifications in place too..
[23:38] <powersj> nacc: finished dlm merge
[23:38] <powersj> set you as the reviewer so you can judge how well jgrimm taught me ;)
[23:39] <jgrimm> ha! nice
[23:40] <nacc> powersj: thanks, will probably get to it tmrw
[23:41] <sarnold> drab: what kind of drives will you be putting behind this nvme? even 700MBps is way faster than spinning metal drives, maybe the odd performance bumps might not matter in the end
[23:41] <sarnold> drab: I'm curious what the cause is, but at some point it might not really matter
[23:46] <JoseLuis_> Good afternoon
[23:49] <jgrimm> nacc, i am curious why dlm wasn't auto-updated by the importer?
[23:49] <jgrimm> powersj, ^^
[23:49] <nacc> jgrimm: how do you mean?
[23:50] <jgrimm> since it had been imported before.. i thought it would be kept fresh when newer debian available
[23:50] <JoseLuis_> Does anyone know those files ( /usr/bin/xjrcsubbir and /etc/init.d/xjrcsubbir) ? is running in my server ubuntu Ubuntu 16.04.1 LTS
[23:50] <nacc> jgrimm: it will be updated when a new publish is seen in launchpad
[23:50] <nacc> jgrimm: relative to when the importer itself last ran
[23:52] <sarnold> JoseLuis_: I've never seen it. my archive mirror doesn't have anything like it.
[23:53] <sarnold> and just as troubling, google reports "Your search - xjrcsubbir - did not match any documents"
[23:53] <JoseLuis_> thanks sarnold, my server is running in in linode.com
[23:53] <jgrimm> nacc, how is that useful?
[23:54] <nacc> jgrimm: sorry?
[23:54] <nacc> jgrimm: maybe we can discuss this in our 1x1
[23:54] <sarnold> JoseLuis_: -maybe- it's something they built but even so I'd expect to find some reference to it somewhere on the internet.
[23:54] <sarnold> JoseLuis_: try asking in #linode on irc.oftc.net
[23:54] <jgrimm> nacc, yep! all good. not urgent in the slightest. :)
[23:54] <JoseLuis_> thanks sarnold
[23:55] <JoseLuis_> Does anyone know those files ( /usr/bin/xjrcsubbir and /etc/init.d/xjrcsubbir) ? is running in my server Ubuntu 16.04.1 LTS on linode.com
[23:56] <JoseLuis_> mistake!!