/srv/irclogs.ubuntu.com/2017/09/26/#ubuntu-server.txt

=== JanC is now known as Guest73457
=== JanC_ is now known as JanC
AegonTargWhat's a good use for my ubuntu server? (16.04). I've already setup an email server, web server, VPN, game server. I want a cool project for my apartment, like sensors or something. Not sure what though.05:21
AegonTarglike for example, a temperature sensor that I post to my server and I can read the values from anywhere across the internet.05:22
andolAegonTarg: Sounds like you already have an idea, like in that senor setup you described? :)05:28
AegonTargandol, I already did that haha.05:29
AegonTargnot sure what else to do..05:29
andolAegonTarg: Jukebox?05:29
AegonTargandol, oh it's a cloud server.05:30
AegonTarglol05:30
andolAhh, just assumed a local physical server when you started talking about sensor.05:30
andolAegonTarg: In that can I think you should use it to setup your own DNS server, letting it be the DNS master, and using a hosted service for DNS secondaries.05:31
andolAegonTarg: Running DNS yourself might not always be the most effective solution, but it's a good learning experience, and properly understanding DNS is a good thing.05:32
AegonTargandol, honestly I'm not entirely sure how DNS works. Like I always assumed the ISP's regulated who gets what domain name based on the DNS registrats05:33
AegonTargregistars*05:33
andolAegonTarg: Time to start figuring it out then :-P05:34
andolWay too common that people kind of know what DNS does, without any understanding about how the different pieces fit together.05:35
AegonTargandol, I've tried looking it up but never can get a clear answer.05:35
AegonTargSo basically I could get a domain name "helloworld.org" for free by hosting my own DNS server05:37
AegonTargOr basically do I still need to register it (which costs money) and all I'm doing is serving the IP address that points to the domain name?05:39
andolAegonTarg: There are two different aspect here, the DNS name delegation and the acutal DNS hosting06:16
andolAegonTarg: No matter what, the .org top domain (registry) will need to delegate the example.org domain to you. For that there is a free. The top domain registries don't deal directly with "customers", but rather you have to deal with a reseller, in this case a registrar.06:18
andolAegonTarg: Then there is the DNS hosting, when the .org top domain delegates the example.org domain to you it does that by telling the world which authorative DNS-servers are resposible for the example.org domain.06:19
andolAegonTarg: It's not uncommon for registrars to also provide the service of DNS hosting, but technically that is a different service.06:19
andolAegonTarg: So yes, you will need to pay a registrar for the example.org domain, and then you can use the registrar to communicate with the .org top domain that your DNS server(s) are resposibly for the example.org domain.06:20
AegonTargandol, ah alright, that explains it a bit.06:54
AegonTargStill not sure if I want to do a DNS server though, I don't really need one honestly.06:55
lordievaderGood morning06:55
andolAegonTarg: Nope, unless you plan to do any special integrations, or have other special requirements, there is very little reasons to run your own DNS server, except for the learning experience.06:56
AegonTargandol, ok thank you!07:58
albechbeen trying to route traffic back through the interface where the connection is established on, but with no luck. I have a setup with three interfaces. eth0: internal network in the DC, eth1: external interface with static ip, tun0: vpn interface over eth1, which is used as default gw. I wish to route connections coming to eth1 back through eth1 and not via default gw. I have looked at https://unix.stackexchange.com/questions/4420/reply-on-same-inter11:51
albechbut it isnt working as expected.11:51
albechnot sure if rp_filter should be off for this, but i have already tried without success.11:53
dpb1https://trello.com/b/E0g6etCl/1709-retrospective13:21
dpb1whoops, wrong channel, that's not publicly visible13:22
coreycbjamespage: beisner: hello, can you promote python-oslo.middleware 3.30.0-0ubuntu1.1~cloud0 to pike-proposed please?13:32
coreycbjamespage: beisner: and also the point release in newton-staging is ready to promote to newton-proposed.13:33
DannySHi everyone, I am having an issue on my Ubuntu server 14.04 where I can't seem to ping anything, nothing works, I can't download packages, or ping google.com or even ping 8.8.8.813:50
DannySAny help? I have looked online and did everything online says, but nothing is working13:50
DannySI think there is some issue with DNS resolution13:50
maswanif ping to an ip doesn't work, it's networking, not DNS13:51
maswanhttps://help.ubuntu.com/lts/serverguide/network-configuration.html13:52
maswanmight have some help13:52
DannySmaswan: I get this: eth0      Link encap:Ethernet  HWaddr 4c:72:b9:d2:c0:8a13:53
DannySifconfig -a | grep eth13:53
DannySBut I can connect to the network, and my websites and that work...13:53
DannySBut nothing can resolve internally? maswan13:54
maswanbut you just said that you couldn't ping IPs13:56
maswananyway, both IP addressing and resolving is on there13:56
DannySmaswan: This is already configured correctly, everything seems to show right...?13:58
DannySLike eth0 shows it using multicast DNS?13:59
DannySmaswan: So when I try and ping google.com it doesn't work, my website is still showing, my connection is still working and all of it works still... but nothing can resolve locally, therefore applications just error, like my game server won't allow connections, my IRC bouncer keeps telling me disconnected, could not resolve host.14:00
DannySCan't download packages, anything that requires a lookup fails14:01
DannySIf I give someone like access can they look? Like i am honestly done with this... I can't understand what is wrong14:04
albechDannyS: will 'dig google.com' work?14:06
DannySI don;t know enough about networking, and all the posts I have followed, or config I have looked at and made sure mine is the same, nothing works.14:06
DannySalbech: Will look14:06
DannySalbech: Doesn't do anything, just hangs14:06
albechDannyS: then try 'dig @8.8.8.8 google.com14:06
DannyS; <<>> DiG 9.9.5-3ubuntu0.16-Ubuntu <<>> @8.8.8.814:07
DannySgoogle.com ; (1 server found) ;; global options: +cmd ;; connection timed out; no servers could be reached14:07
DannySalbech: Nope, I get this: ^14:07
sarnoldhow about ip route get 8.8.8.814:07
DannySSorry what do you mean?14:07
DannySsarnold: ^14:08
albechlooks like a routing issue14:08
sarnoldwhat is the output from running the command "ip route get 8.8.8.8"14:08
DannySsarnold: I get this: ip route get 8.8.8.8 8.8.8.8 via 192.168.1.1 dev eth0  src 192.168.1.178  cache14:08
sarnoldDannyS: okay, ping 192.168.1.114:09
DannySSame issue as pinging IPs, just hangs?14:09
DannySsarnold: ^14:09
sarnoldd'oh14:09
albechhmm14:10
sarnoldhow about ping -n 192.168.1.114:10
* sarnold smacks self14:10
DannySsarnold: Same issue, just hangs14:10
DannySAll I did was reboot the server >.<14:10
albechDannyS: and you are chatting with us through the same router?14:11
DannySalbech: No, I am on my computer, I am ssh'd into my server14:11
DannySI can't use my IRC bouncer anymore, as it just disconnects me because can't resolve host14:12
patdk-lprogue dhcp server :) those are the best :)14:12
DannySI will happily give access if it will make it easier?14:12
patdk-lphow will give access work when you don't have a working network?14:12
DannySpatdk-lp: I am SSH'd into the server...14:13
AureliusODannyS, what does iptables -L look like?14:13
DannySAureliusO: It's big, let me put it on a paste14:13
albechDannyS: ahhh..14:13
AureliusOI smell ufw muckery.14:14
DannySOh?14:14
albechIndeed14:14
DannyShttp://git.dannysmc.com/snippets/4514:14
sarnoldi suspect icmp hsa been blocked somewhere14:14
sarnoldand maybe udp too14:14
* patdk-lp smells begals!14:14
sarnoldoh damn I want bagels14:14
patdk-lphaven't been able to use irc for months :(14:15
DannySAureliusO, sarnold: I linked the output above.14:15
patdk-lptechnically stil lcan't, too busy, but ignoring work14:15
albechpatdk-lp: get to work! ;)14:16
DannySAnyone? Anything? Please :(14:18
AureliusOReading the mess, sec.14:18
DannySAureliusO: Ahh, is it that bad? :/14:18
DannySI just use ufw, because not sure about the whole iptables thing14:18
AureliusONah, reading ufw is just a lot of back and forth jumping.14:19
patdk-lpheh?14:20
patdk-lpit looks perfectly fine14:20
patdk-lpdidn't take much to verify it at all14:20
sarnoldhow the heck did you read 16k of rules so quickly? :D14:21
patdk-lpyou don't need to14:21
patdk-lpyou just follow it till you hit, ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED14:21
patdk-lpthen the rest doesn't matter14:21
patdk-lpatleast for our purposes14:21
patdk-lpand everything before that is empty so14:22
DannySSo no one has said anything? Is my iptables correct? or?14:22
DannySSorry I don't mean to come across rude, just networking is something I don't know, so not really sure what is happening.14:23
patdk-lpok, read it all, not interesting :)14:24
jamespagecoreycb: do you still need that stuff promoting?14:24
patdk-lphow that doesn't mean something with nat isn't screwed up, or in pre-routing14:24
coreycbjamespage: yes please14:24
DannySSo if I give someone SSH access maybe they can look? or someone can tell me commands I need to run? Honestly very grateful for the help already14:24
jamespagecoreycb: ok doing that now14:25
coreycbjamespage: thanks14:25
coreycbjamespage: horizon can also be promoted to newton-proposed14:25
jamespagecoreycb: looking14:26
DannySAureliusO: ?14:26
AureliusOUnless an interface is specified incorrectly somewhere, the iptables set is fine, as patdk-lp said.  Still odd that 192.168.1.1 isn't reachable.14:26
patdk-lpI really don't get those user-input rules though, so much crap opened that shouldn't be14:26
jamespagecoreycb: and for pike as well14:26
DannySpatdk-lp: I have a lot of game servers, that run on different ports14:26
patdk-lpyou have something on udp port 22?14:26
patdk-lpudp port 80 and 443?14:26
coreycbjamespage: yeah probably. i uploaded horizon for mitaka->pike. still need to upload kilo but was having troubles building it with .egg issues.14:27
DannySErrr? I have a web server? git server? game servers? nodejs applications?14:27
sarnoldDannyS: try nmap -sT or nmap -sU from this server to your router to see if you can reach it via nay other mechansisms14:27
patdk-lpDannyS, yes, but what about any of that uses udp?14:27
DannySpatdk-lp: I wouldn't think so, not sure, might need to clear all my rules and start again, if that;s the case14:28
DannySsarnold: nmap -sT comes back with: Starting Nmap 6.40 ( http://nmap.org ) at 2017-09-26 15:27 BST WARNING: No targets were specified, so 0 hosts scanned. Nmap done: 0 IP addresses (0 hosts up) scanned in 0.05 seconds14:28
sarnoldDannyS: nmap -sT 192.168.1.1  ?14:28
DannySsarnold: Starting Nmap 6.40 ( http://nmap.org ) at 2017-09-26 15:28 BST mass_dns: warning: Unable to determine any DNS servers. Reverse DNS is disabled. Try using --system-dns or specify valid servers with --dns-servers Note: Host seems down. If it is really up, but blocking our ping probes, try -Pn Nmap done: 1 IP address (0 hosts up) scanned in 0.51 seconds14:28
sarnoldjeezus nmap just do what I mean!14:29
DannySsarnold: ?14:29
AureliusOEven though we can't see anything wrong, part of me still wants to tell him to put in input & output rules to just blindly accept from his gateway and test again14:29
sdezielDannyS: "tcpdump -ni eth0 not port 22" should tell you if ARP is working14:29
jamespagecoreycb: do you want me to promote all of the new newton point releases to newton proposed as well?14:29
coreycbjamespage: yes14:29
AureliusODannyS, iptables -t nat -L -v14:29
AureliusODannyS, also, just to be sure, iptables -L -v14:30
sarnoldDannyS: alright try nmap -sT -n -Pn 192.168.1.114:30
DannySsdeziel: I get a lot of content coming through?14:30
DannySAureliusO:14:30
sdezielDannyS: well, I just realized that your were connected by SSH so that was a moot point, sorry14:31
DannySsdeziel: No worries!14:32
DannySAureliusO: http://git.dannysmc.com/snippets/4614:32
DannySAureliusO: http://git.dannysmc.com/snippets/4714:32
DannySThat's both the commands you sent me14:32
DannySsarnold: Seems to be hanging?14:33
DannySsarnold, AureliusO tell me what you think14:34
sdezielDannyS: "arp -na | grep -F 192.168.1.1"14:35
DannySsdeziel: ? (192.168.1.1) at 00:07:b4:00:01:01 [ether] on eth014:36
sdezielDannyS: OK, so that's not the problem14:37
DannySErrrr14:37
DannySAnyone want access? haha14:37
DannySMight be easier to get information?14:37
patdk-lpDannyS, ping 96.83.110.22814:38
DannySpatdk-lp: Just hangs, doesn't do anything14:39
DannySpatdk-lp: When I close it: 12 packets transmitted, 0 received, 100% packet loss, time 11011ms14:39
sarnoldDannyS: it might take forever..14:39
DannySsarnold: What will take forever?14:39
patdk-lpdo it again14:39
sarnoldDannyS: nmap14:40
DannySDoing it again14:40
patdk-lpi see nothing at all, odd14:40
patdk-lpreboot your router/firewall/gateway14:40
DannySufw?14:40
patdk-lpis it one of those home device things?14:40
DannySNo this is at a hosting company14:41
patdk-lpusing a private ip block?14:41
DannySpatdk-lp: They verified it's nothing on their end14:41
DannySand yes I assume so?14:41
DannySI have 2 dedicated IPs14:41
DannySsarnold: Nothing is happening, shall I just leave it running?14:42
DannySsarnold: IT FINISHED!14:43
DannySsarnold: Starting Nmap 6.40 ( http://nmap.org ) at 2017-09-26 15:39 BST Nmap scan report for 192.168.1.1 Host is up. All 1000 scanned ports on 192.168.1.1 are filtered Nmap done: 1 IP address (1 host up) scanned in 201.28 seconds14:43
sarnoldDannyS: what's the firewall on 192.168.1.1 doing?14:44
DannySsarnold: How do I find out?14:44
sarnoldDannyS: log in to 192.168.1.1's administrative interfaces and look around14:44
AureliusOsarnold, sounds like is the hosting company's gateway -- he might not get to know14:44
DannySsarnold: It's my hosting company, I don't have access sorry14:45
sarnoldahh14:46
sarnoldare there "security groups" there that might be tweakable?14:46
AureliusOSomething doesn't add up here.  The iptables rules are fine, inbound traffic is apparently working OK, related outbound is working or SSH wouldn't do anything...14:47
DannySsarnold: Well I wouldn't think so, but I can't see how this would be an issue? As it's worked before? It's not like a new server, it's just stopped working since I deleted /var/cache and restarted14:47
DannySthe server before had been running for over 3 months non stop14:47
DannySWith many packages being installed etc14:47
DannySSo idk14:47
DannySAureliusO: Shall I give you a sudo user? so you can look around?14:49
sarnoldit sure feels like icmp and udp is being blocked _somewhere_14:49
DannySsarnold: Again, shall I give access?14:49
sdezieldig +tcp @8.8.8.8 google.com14:53
patdk-lpaccesss isn't going to solve anything14:53
DannySpatdk-lp: So I don't understand? What am I supposed to do?14:53
patdk-lpif we knew, we would have solved this long ago14:54
AureliusODannyS, realistically, you have told us everything we would want to derive from having access -- it's more of a "what's next" thought right now.14:54
sarnoldDannyS: the thing is I just don't have the time to dedicate to it :( just poke in ideas from time to time..14:54
patdk-lpwe didn't setup your server, setup your network, setup your hosting provider14:54
patdk-lpthere is only so much we can do14:54
DannySsarnold, AureliusO I see, so not really any way of fixing it?14:54
sarnoldno, just that it might take more time and effort than I can dedicate14:55
patdk-lpgiven your server is working fine on the local network14:55
patdk-lpbut it's firewall and routing are ok, and arp works14:55
patdk-lpthe next step is to check the firewall/router14:55
patdk-lpbut we cannot do that14:55
DannySOh :( who can? xx14:55
AureliusOYou should get your hosting company on the phone and get someone who can capture traffic at or after 192.168.1.1 and see if they can find where your pings are being dropped14:55
DannySignore xx14:55
patdk-lpyour hosting provider, but they said it is working fine you said14:56
AureliusOThe fact that patdk-lp was unable to see the INCOMING ICMP traffic is pretty telling.14:56
sdezielDannyS: could you tell us one of the 2 dedicated IPs that server has? And also which service should be publicly accessible?14:58
albechseems rather strange that a hosting company would be using 192.168.1/24 for their clients14:58
sarnoldhehe that's part of why I assumed the router was DannyS's :)14:59
AureliusOalbech, depending on the company it could be as little as 1 or 2 people operating as a reseller -- that's not odd for some game hosting companies.14:59
AureliusOThey don't tend to be *too* savvy themselves.14:59
AureliusOIn that case, of course.  There are plenty who are very good & technical.15:00
hateballthere's also cg-nat these days15:01
DannySSoYouStart is the company reseller of OVH15:01
patdk-lpcgnat has it's own ip block, that isn't it15:03
danpawlikcoreycb: Hello15:06
danpawlikcoreycb: Do you recognize https://bugs.launchpad.net/puppet-gnocchi/+bug/1687235   zhongshengping ?15:06
ubottuLaunchpad bug 1687235 in puppet-gnocchi "Package gnocchi-indexer-sqlalchemy does not exist on Ubuntu" [High,In progress]15:06
coreycbdanpawlik: i've not seen that but it seems we should add that binary package15:13
danpawlikcoreycb: yes... Should I create a task for it? Or maybe I can push some proposal?15:15
coreycbdanpawlik: i've added a task to that bug15:15
danpawlikcoreycb: thanks!15:16
DannySsdeziel: 94.23.41.10115:16
sdezielDannyS: what service/port should I be poking?15:16
DannySYou can poke port 8015:16
DannySThat seems to work still15:16
sdezielDannyS: FYI, I can ping and reach your HTTP server15:17
sdezielDannyS: really looks like a misconfigured firewall upstream to your machine. As if the firewall would not let you initiate outbound connections15:17
beisnercoreycb: promoted newton to uca proposed for point release.  the oslo middleware pike one looks like it's already done.15:20
coreycbbeisner: \o/15:20
danpawlik:D15:20
DannySsdeziel: No idea what to do, but thanks15:34
sdezielDannyS: I'd bring that info to the attention of the hosting provider/firewall manager15:35
DannySsdeziel: Will let them know15:35
mike-zalmaybe some will explein it to me. ufw is installed but not enabled. however IP tables do have some rules and virtualmin shows them as active. does it mean that firewall runs? I kinda am confused without ufw.16:23
sarnoldI think iptables -L is probably your best source of truth16:24
naccufw = frontend for iptables, iirc16:28
nacc"The Uncomplicated FireWall is a front-end for iptable"16:28
sarnoldit is16:28
sarnoldbut if you use a different tool to manage the firewall then I wouldn't rely upon ufw's output to tell you much about iptables or the other tool16:28
hateballmike-zal: maybe you are using fail2ban ?16:29
tomreynor virtualbox, vmware workstation, lxc/lxd/docker or some other kind of virtualization / containerization16:30
tomreyn"sudo iptables-save" may provide a better idea of what the existing policies are for (than "sudo iptables -L").16:32
naccsarnold: good point16:32
mason/etc/network/interfaces "up /etc/network/rules" FTW.16:34
mike-zalthanks sarnold, iptables -L does show rules so it seems to be active. I am asking, because now when I use virtualmin, I don't need ufw, since virtualmin shows and manage rules.16:39
mike-zalhateball: yes, I am using fail2ban16:39
hateballmike-zal: maybe it has created some rules then16:39
mike-zalprobably. I was learning how it all works, but a friend recommended me virtualmin and it really is awesome, but in a way I must learn things anew.16:40
mike-zalhowever, it's less likely I screw something ;)16:40
mike-zalalso, I'm quicker and more effective with gui then with terminal, so such solution is working for me16:41
mike-zalso, are iptable rules active by default on clean ubuntu server install?16:42
hateballThere's not a single rule, no16:42
mike-zalI always thought I need to enable and set ufw and that was what I did before16:43
mike-zalI mean on a clean ubuntu server install, 16.0416:43
hateballiptables itself is active as it is part of the kernel, but there are no rules16:43
mike-zalah, ok16:43
mike-zalthanks, that clarifies things for me16:44
mike-zala new question. can sql file be password protected somehow? doing some backups and having sql files seems to be a voulnerability16:46
hateballtar it, compress it, encrypt it16:47
hateballup to you16:47
mike-zalhas tar a password option? will have to investiage it. encrypting also sounds a good idea, although I just don't know anything about it but that can be changed ;)16:58
naccrbasak: excellent, dpkg-parsechangelog between x and a behave differently16:58
naccrbasak: which means i need to build dpkg from source too16:59
sarnoldmike-zal: yeah you can use whatever frontend you like for iptables :) ufw is simple and works for a lot of people but whatever works17:30
sarnoldmike-zal: gpg -c is a good way to password protect a file17:31
mike-zalsarnold: thanks, noted17:38
=== JanC_ is now known as JanC
trippehstrange, systemd-timesyncd isnt synchronizing, the ntp requests and responses show up in tcpdump and timesyncd logs nothing.18:14
sarnold:(18:15
trippeh# /lib/systemd/systemd-timesyncd --help18:16
trippehThis program does not take arguments.18:16
trippehhelpful ;)18:16
rbasaknacc: :-(18:16
naccrbasak: and ... might have found another bug in snapcraft :/18:17
rbasaknacc: perhaps add a test to check that we are getting the right side of the difference?18:18
naccrbasak: yeah, I'll do that18:18
rbasaknacc: git-ubuntu dev discussion with Launchpad team in half an hour (1500 UTC). Would you like to join via HO?18:20
naccrbasak: i am going to be at lunch18:20
rbasaknp18:20
powersjrbasak: nacc: how well do you know the launchpad api?18:49
powersjCurrently running into the following: https://paste.ubuntu.com/25618073/18:50
powersjwhich had been working18:50
powersjIn the past I got a single result for the artful release18:50
rbasakpowersj: not sure. Try #launchpad?18:54
powersjok filed LP: #171971518:55
ubottuLaunchpad bug 1719715 in Launchpad itself "getDevelopmentSeries fails to find a series" [Undecided,New] https://launchpad.net/bugs/171971518:55
rbasaknacc: bug 1719715 (probably Invalid) is relevant to git-ubuntu also I think.18:59
ubottubug 1719715 in Launchpad itself "getDevelopmentSeries fails to find a series" [Undecided,New] https://launchpad.net/bugs/171971518:59
coreycbjamespage: opened a bug for pike point release: https://bugs.launchpad.net/ubuntu/+source/nova/+bug/171972820:22
ubottuLaunchpad bug 1719728 in nova (Ubuntu Artful) "[SRU] pike stable releases" [Undecided,New]20:22
naccrbasak: maybe, we don't use it20:28
naccrbasak: we use current_series directly20:29
naccrbasak: dpb1: do you have time for a HO?20:36
naccrbasak: fyi, we have a c-m with tomcat8 now20:51
naccdpb1: --^20:51
naccit's on my todo to fix20:51
nacc(as a new mir team member)20:51
nacccomponent mismatch20:51
naccto whoever just asked :)20:51
JenshaeHello everyone.21:01
coreycbbeisner: can you promote horizon 1:2015.1.4-0ubuntu3 to kilo-proposed please?21:02
beisnercoreycb: thanks.  promoted that^21:04
JenshaeI tried many flavours of the debian branch, debian itself, ubuntu desktop, xubuntu, lubuntu and so forth. To get a SATA fakeraid RAID 5 to work, I have resort to using Ubuntu desktop for two PCs and I am also using it instead of Kubuntu server (same bug) for an archive server I am building.21:09
Jenshaehttps://www.mail-archive.com/ubuntu-bugs@lists.ubuntu.com/msg5172827.html21:10
JenshaeOn the desktops, having GUIs installed, I am finding them slow to respond and unstable.21:10
JenshaeOn the archive server, I have left it as a command prompt for now and I don't know what to do with it. :D21:11
JenshaeI saw something about elevator=noop for something to do with data packets to and from the drives, which might be causing the PC's response lag.21:11
JenshaeDoes anyone here have some good guides on either working around that bug or on how to convert a ubuntu-server into a gaming installation?21:12
sarnoldelevator=noop is recommended for disks that ZFS controls because ZFS already does internal io scheduling tasks21:12
JenshaeWhat is ZFS?21:13
sarnoldother filesystems would probably benefit from one of the other schedulers that knows how to handle rotational disks or ssds or whatever21:13
JenshaeThe last time I fiddled with server versions was Kubuntu server 7 about a decade ago.21:13
JenshaeI have three SSHDs in this machine21:13
JenshaeLubuntu-desktop seems the most responsive and stable.21:13
sarnoldZFS is a combined storage system that is sort of like LVM / RAID controllers married directly to the filesystem, with end-to-end checksums, configurable redundancy, etc.21:14
JenshaeAs I say that and hit Enter, there is a half second freeze before it sends my message to you.21:14
JenshaeThank you, that makes some more sense.21:14
sarnoldthat sounds intolerably bad21:14
sarnoldactually figuring out what needs to be changed might be difficult21:14
JenshaeIt isn't freezing while I type at least :D21:15
sarnoldi'm on one side of the contry typing into a shell o the other side of the conutry and the latency is ~140 ms tops.21:15
sarnold80ms. even better than I expected. :)21:15
sarnoldespecially since there's wifi involved...21:15
sarnoldso 500ms is outright terrible.21:16
JenshaeYeah, wifi. Hate that stuff, number of people that I try and convince to get an ethernet cable ...21:16
JenshaeSo, what would I need to run? I think there was a command, hdparm or something that shows the transfer rates of the HDs?21:17
sarnoldI like iostat -dmx 121:18
sarnoldwell, I _love_ zpool iostat -v 121:19
sarnoldbut if you're not running zfs it's not going to be useful :)21:19
JenshaeI don't know what I am running.21:19
JenshaeThis machine has the raw Ubuntu Server install without any additions like Samba server and then I have slapped multiple desktops on it.21:20
JenshaeStandard Intel Bios RAID config.21:20
Jenshae3x Seagate FireCuda Laptop 500GB 2.5" Hybrid Hard Drive - SSHD 7mm (I somehow screwed up and got 2.5" instead of 3.5" (I have insomnia and sleep deprivation problems so I do some bizarre things))21:23
JenshaeWhat do I run to see if I am using ZFS?21:23
sarnoldyou'd know if you were using it :)21:26
Jenshaehttps://wiki.ubuntu.com/ZFS21:26
sarnoldit still takes some effort ot use zfs on linux these days21:26
JenshaeHazarding your best guess, can I just slap that in now, post installation?21:27
sarnoldhere's a series of blog posts about ZFS that got me started https://pthree.org/2012/12/04/zfs-administration-part-i-vdevs/21:27
JenshaeIt has taken me three weeks of testing flavours to end up on Ubuntu Server so I am persistant.21:27
JenshaeThank you21:27
sarnoldyou certainly could move to ZFS but that would take some effort -- new drives would be the easiest way to get there.21:27
JenshaeThese have nothing valuable on them. Can wipe the RAID and start again at any point.21:28
sarnoldRAID controllers are usually rubbish. soft-raid doubly-so :(21:28
JenshaeYou recommend Soft Raid over fake raid?21:29
JenshaeWould a hardware RAID controller get around the desktop bug?21:29
JenshaeI am seeing a performance boost ... when I don't get a lag spike.21:29
trippehheh, so re earlier, the home ntp server had gone off the rails, claiming accuracy of +/- 18 seconds21:30
JenshaeIt feels like the machine sometimes waits for enough data in RAM before writing it to the drives or reading from them or something.21:30
sarnoldtrippeh: 18 seconds??? ouch21:30
trippehsarnold: yeah. everything looked fine in chronyc sources21:31
sarnoldJenshae: I recommend ZFS over fake raid things. dm or md things are okay, but I never learned how to use them. ZFS provides reliability guarantees that are very difficult to get otherwise.21:31
trippehnot sure how that would happen21:31
JenshaeThat's nothing. We have 9 servers at work and they are minutes out from each other. I tried synching to Google's NTP server (should be the best, right?) well that seems to be 4 minutes out according to all the Apple dorks in the office. :P21:31
tomreynJenshae: i think a software raid configuration is a better option than any fake raid.21:32
tomreynJenshae: for i/o performance testing you could try this https://www.thomas-krenn.com/en/wiki/Linux_I/O_Performance_Tests_using_dd21:33
trippehsarnold: ETOOMUCHINFRASTRUCTUREATHOME I guess ;)21:33
sarnoldbe careful with google's NTP servers. they smear their seconds when leap seconds are added. You should only use google's ntp servres if you understand what this means. :)21:33
sarnoldtrippeh: lol21:33
JenshaeThanks guys. My first RAID attempt this year was with Kubuntu server and software raid using mdadm. I partitioned mixed drive sizes of 160, 200, 320 and 500 into 160 chunks and tried to string them together unsuccessfully (guessing part of that is that it was 3x the load to the 500 drives)21:33
tomreynJenshae: the I/O latency you are experiencing will probably not depend much on which linux distribution you use, but on your hardware, how it works with linux generally (i.e. which quality the drivers are), and on how you configure the system.21:33
sarnoldJenshae: heh, that'll probably utterly DESTROY your latency and throughput.21:34
JenshaeIt was more about just creating one whole partition space for my ... very varied understanding of IT users.21:34
JenshaeSo it was just a RAID 0 with backup tape. The stuff they were to put on there is over 5 years old, things they should delete but can't let go of.21:35
JenshaeLose a partition, good, got rid of a terabyte or more of rubbish.21:35
JenshaeI have since gone around grabbing all the 500GB drives I can out of the workstations.21:36
JenshaeHence why my apprentice has a 160 GB RAID 5, I waved that carrot in his face while I yoinked his drive.21:36
JenshaeAlso, the machine is a proof of concept before I can get funding for real hardware.21:37
sarnoldwow raid0 with a bunch of crazy partitioned drives .. brave indeed :)21:37
JenshaeReally, I want to just select by modified date and purge ... but they would lynch me.21:38
JenshaeIf I can blame a machine and say it is just the gambles we take in life, they will cry a bit and let it go.21:38
JenshaeThey distrust computers and expect them to fail. They also think I am a wizard, Hagrid. They want solutions to all problems within 5 minutes and don't get why one fix is fast and another "small" one takes weeks.21:40
JenshaeQuite a few of the staff ... are past retirement age. So, the started their careers on paper ledgers and type writers.21:40
JenshaeSo, they*21:41
JenshaeGoing to go jump in my bath before it is frozen. Thank you for the reading material.21:41
JenshaeBizarre thing, the lags seem to be gone.21:42
sarnoldI can see why it might be hard to get a budget for real hardware then :)21:42
sarnoldthey'll be back :/21:42
JenshaeThe machine seems to be speeding up while I use it.21:42
sarnoldit will21:42
tomreynsounds like https://www.youtube.com/watch?v=BKorP55Aqvg21:42
sarnoldas data is read off the disks into memory, it won't need to be re-read from disk the next time it's needed21:42
sarnoldtomreyn: :)21:42
JenshaeAlso being SSHD it is moving frequently used data from magnetic to solid.21:43
Jenshaetomreyn: Exactly! :D21:45
sarnoldI'm pretty skeptical of the benefits of sshd; I could believe they might be an improvement for 'standard desktop users', but it's hard to imagine how they could pull off a general improvement for all use cases21:46
JenshaeI have U.2 ports ... but have you ever tried sourcing a drive for them?21:47
trippehnot much but intel SSDs for those21:48
trippehpricey ones21:48
JenshaeYup21:48
JenshaeNo M.2, so I guess I should get a card.21:49
trippehI use a m.2 to PCIe slot adapter for my retro 2008-era computer.21:51
trippehit wont boot of it however.21:51
JenshaeI would give you my specs ... if I could find something to list them P21:52
Jenshae:P21:52
JenshaeIs there an angry bot lurking in this channel that would kick me for pasting multiple lines?21:55
tomreynthere should be, and there probably is21:56
JenshaeWill try find my pastebin in that case21:57
tomreyn!paste21:57
ubottuFor posting multi-line texts into the channel, please use http://paste.ubuntu.com | To post !screenshots use http://imgur.com/ !pastebinit to paste directly from command line | Make sure you give us the URL for your paste - see also the channel topic.21:57
sarnoldthe pastebinit tool is nice21:57
Jenshaehttp://paste.ubuntu.com/25623182/21:58
JenshaeThat work?21:58
sarnoldyeah. those are very busy disks.21:59
sarnoldin those four seconds they're not doing anything but the cumulative stats sure look like they're pegged to capacity nearly all the time.22:00
JenshaeI have three tabs of firefox, IRC and a Terminal emulator running.22:00
JenshaeNot exactly a heavy load.22:00
sarnoldwatch the 'si' and 'so' columns of vmstat 1   output22:01
sarnoldthat _might_ be heavy swapping activity if you don't have enough memory22:01
trippehwere they resyncing? or was it raid0, which does no such things22:03
Jenshaehttp://paste.ubuntu.com/25623204/22:03
JenshaeI did have it OC'ed to 4.1GHz on a single regular SATA but dropped it to defaults to try get more stability for now.22:03
sarnold64 gigs ram not bad.22:06
sarnoldnice thing about zfs is you can turn on lz4 compression, it's a lot like getting magically faster disks22:06
sarnoldtime to run, have fun22:06
Jenshaesi and so are all 0s - http://paste.ubuntu.com/25623220/22:06
JenshaeThanks for the help sarnold22:07
JenshaeI made the machine to manipulate national data arrays at home on some consultancy work for the government.22:08
JenshaeTurns out that I would have been better set filling all 8 channels with 4GB rather than the initial 8GB in 4 slots. The RAM usage has never been high but speed increased with more units of hardware.22:09
JenshaeCurrently torn about getting a Vega graphics card. The Vega 64 would be a big enough jump to justify the upgrade from my R9 390 but the cost is high. The Vega 56 isn't a big enough jump but is the right price I would pay for the Vega 64.22:11
JenshaeAlso, I would probably be in the same boat I was with this card two years ago.22:11
Jenshae... waiting for AMD to release proprietary patches to fix bugs because Mesa is much slower and coming out with latest card support.22:12
Jenshaeat coming22:12
JenshaeAnyhoo, AFK. Thanks again. Most helpful Ubuntu IRC I have been in.22:13

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!