[03:23] <genewitch>  /wc
[07:09] <lordievader> Good morning
[08:56] <sinhue> Hi, I'm trying to install Ubuntu Server 18.04 on Virtual Machine But all I'm getting is= "Error setting up gfx boot." I'm getting desperate. Even tried older releases. What could be the problem?
[09:06] <blackflow> sinhue: did you try with google, there are some suggestions in first few results, for "Error setting up gfx boot"
[09:06] <sinhue> I did but nothing helped me.
[09:07] <sinhue> What helped me was noticing that VM does in fact have 8 mb ram, instead of 8gb :D :D
[09:07] <sinhue> Sorry for bothering
[09:08] <blackflow> sinhue: which hypervisor is this? Did you see this, and try as suggested to reach the menu? https://askubuntu.com/questions/895148/error-setting-up-gfxboot/920907
[09:09] <sinhue> blackflow, I solved it already bro. But this was ESXI 6.5 hypervisor.
[09:09] <blackflow> this is ubuntu server support, you're not bothering, but it is expected you try to help yourself first as much as possible :)
[09:09] <blackflow> solved how?
[09:09] <blackflow> I'm suspecting missing or invalid graphics device?
[09:10] <blackflow> (and please don't call me bro)
[10:19] <kstenerud> cpaelzer: I'm beginning to think that a keepalived backport isn't in the cards. The commits are all merges containing other merges containing yet more merges, which completely obscures what's actually happening. I've found vrrp and IP deletion code strewn through the logs, but it spans years, fixing various problems when using ipv4, ipv6, static ro
[10:19] <kstenerud> utes, physical vs virtual devices, etc. I'm not confident that I'd be able to find it all :/
[10:21] <kstenerud> All references to fixing keepalived + systemd-networkd only refers back to the "beta branch", with no commit refs
[10:23] <kstenerud> The workaround I posted does work. Perhaps we're better off waiting to see if this solves the issue for enough people? The issue should be fixed in keepalived 2.0 in disco, which I'm testing now
[10:24] <kstenerud> It's actually quite a complicated problem, as keepalived must rely on heuristics to determine if a device or ip being removed was intentional or not...
[10:29] <cpaelzer> ok, sounds right
[10:30] <cpaelzer> do we have a time limit when we check if "this solves the issue for enough people"  is true or not?
[10:30] <cpaelzer> and if so  planned action (e.g. as discussed yesterday docs/blogs/...)
[10:47] <kstenerud> The bug report has the workaround, but we should definitely update any docs we maintain that deal with keepalived. I don't see much other than https://www.ubuntu.com/kubernetes/docs/keepalived which uses juju
[10:48] <kstenerud> I suppose it comes down to which approach we would bless? apt? snaps? charms?
[10:56]  * blackflow bets 5€ on the snap.
[10:57] <lotuspsychje> lol
[10:58] <blackflow> makes sense, if there's a lot of issues to shoehorn it into specific base env.
[13:05] <blackflow> Is the ZFS in Bionic patched to support latest 0.7.x featureset?
[14:00] <lordcirth> blackflow, bionic has 0.7.5
[14:01] <blackflow> as baseline version yes, but a megaton of patches to make it work with 4.15+ kernels
[14:01] <blackflow> so I'm asking what else is there featureset wise, or most importantly, can it fully use pools created with 0.7.12 (which I'm preparing to test in a few)
[14:02] <ahasenack> wrt pools, most likely, We are settling on a very baseline set of features to remain compatible with other pools out there
[14:02] <ahasenack> but you could ask in the kernel mailing list perhaps: https://lists.ubuntu.com/mailman/listinfo/kernel-team
[14:02] <blackflow> if so that's awesome. I don't enable any non-default features, I only need it to rw to pools created with 0.7.12 defaults
[14:02] <blackflow> but... about to test all that now
[14:03] <blackflow> and make that 0.7.13, being latest actually :)
[14:03] <ahasenack> I have 0.8 elsewhere, and the story there is different
[14:04] <ahasenack> as one might expect given the version bump
[14:04] <blackflow> oh yeah, that's a whole new bag of goodies in that :)Č
[14:04] <blackflow> s/Č//
[14:05] <lordcirth> We are very glad for the faster scrubs in 0.8. Our current production storage takes 6 days to do the weekly scrubs, and it's less than half full. We need the speed to keep up.
[14:05] <ahasenack> lordcirth: and with 0.8, how long does it take?
[14:05] <lordcirth> Haven't tested yet
[14:06] <lordcirth> We are still putting together the new system
[14:06] <blackflow> lordcirth: why weekly tho? esp. if that sized. scrub is just "let's find problems now before regular block access does anyway"
[14:06] <lordcirth> blackflow, best to be safe.
[14:07] <ahasenack> that much activity could also reduce the lifetime of the drives, no?
[14:07] <lordcirth> Then we RMA them.
[14:07] <blackflow> lordcirth: but if you have properly set up pools (redundancy, hot swappable drives) then you gain very little from frequent scrubs
[14:07] <lordcirth> 3-year evergreen due to warranty anyway
[14:07] <blackflow> from *too frequenty I mean
[14:07] <ahasenack> lordcirth: have these scrubs ever found problems?
[14:08] <blackflow> statistically, there should be one bit corrupt for every 10TB accessed, according to (now old) google research
[14:08] <ahasenack> well, "ever" is a long time :)
[14:08] <ahasenack> let's change that to "frequently" :)
[14:08] <lordcirth> We've had two drive failures. IIRC at least one of them was found during a scrub?
[14:08] <ahasenack> or provoked by one ;)
[14:09] <ahasenack> by several, that is
[14:09] <lordcirth> Better to have 2 known failures than 1 unknown one, I guess
[14:09] <blackflow> I second what ahasenack said. at that scale, too frequent scrubbing is just adding to the wear
[14:09] <lordcirth> Weekly scrubs are the default in Ubuntu's packages, presumably for a reason.
[14:09] <blackflow> no, monthly are
[14:09] <blackflow> first sunday in the month
[14:10] <ahasenack> # Scrub the second Sunday of every month.
[14:10] <ahasenack> 24 0 8-14 * * root [ $(date +\%w) -eq 0 ] && [ -x /usr/lib/zfs-linux/scrub ] && /usr/lib/zfs-linux/scrub
[14:10] <ahasenack> in my disco system
[14:10] <lordcirth> Really? hmm I thought it was every sunday
[14:10] <blackflow> *Second, okay.
[14:10] <ahasenack> I didn't do the date +\%w math, just going by the comment in there
[14:10] <blackflow> (yeah I change that to first in the month)
[14:12] <blackflow> ahasenack: 8-14 'th day in the month, then date +%w checks if it's sunday
[14:13] <sdeziel> scrub should be read only unless there is anything to correct so presumably this is lighter on the disks, no?
[14:14] <lordcirth> Scrubs are still a lot of seeks, at least until 0.8 when it's sequential
[14:14] <sdeziel> I agree that scrubbing 6 days per week is very excessive though
[14:14] <sdeziel> ah
[14:14] <lordcirth> Not 6 days per week, once per week
[14:14] <lordcirth> Ah, you mean total time
[14:15] <sdeziel> yeah :)
[14:15] <lordcirth> Well, hopefully 0.8 will be much less load
[14:15] <sdeziel> must absolutely kill the normal performance, except on Sundays where nobody's there to enjoy the real speed ;)
[14:17] <lordcirth> Actually, part of why the scrubs take so long is that they are quite low priority compared to normal usage
[14:20] <zetheroo> blackflow: you here?
[14:22] <blackflow> yup
[14:28] <zetheroo> why ewww ... cloudflare? I am being told that it's "the best" etc ...
[14:30] <blackflow> zetheroo: personal observation, nothing more.   btw, that "search mt.local" if you're using .local for domains, resolved might have problems with that. .local should be reserved for mDNS
[14:39] <zetheroo> blackflow: Ok, was just wondering if there was anything about cloudflare specifically ... since I never heard about it until the other IT guy said "We are switching to it because its the best" :D
[14:40] <zetheroo> ... and "google sucks" ... because we had been using 8.8.8.8 etc
[14:41] <blackflow> zetheroo: "best hyped" more likely. nothing special about it tho'.  technically, some networks erroneously treat 1.0.0.0/8 as a test range so it might not work everywhere, but that's probably very edge case
[14:42] <lordcirth> cloudflare got 1.1.1.1 in exchange for being able to handle and study the huge amounts of random traffic it gets
[14:43] <lordcirth> Not many companies can handle that much traffic.
[14:46] <blackflow> which only puts them in the position to abuse that power
[14:47] <blackflow> personally, been running own Bind authoritative and resolvers for many years.
[14:50] <blackflow> ahasenack: lordcirth: fwiw, tests have shown Bionic having normal rw access to 0.7.12 generated zpool with default features. thanks for your feedback.
[14:50] <lordcirth> blackflow, cool
[14:50] <ahasenack> nice
[14:53] <sdeziel> cloudflare's anycast network is pretty cool on paper though
[14:54] <lordcirth> Cloudflare has a lot of cool articles on HA
[15:26] <zetheroo> blackflow: about systemd-resolved ... you said that it tries the first two nameservers but no more than that ... or did I misunderstand you?
[15:28] <blackflow> zetheroo: you misunderstood.
[15:28] <blackflow> zetheroo: glibc is the one that doesn't take more than 3 "nameserver" entries in resolv.conf
[15:28] <blackflow> zetheroo: systemd does something else. tries one nameserver, and if that fails, tries another and then keeps on using that other, ignoring the order defined by resolv.conf
[15:29] <blackflow> there's been huge issue/debate upstream about that unexpected behavior
[15:30] <zetheroo> sorry, that's what I meant - so if you have three nameservers ( A, B and C) systemd tries A, then B and then keeps trying B?
[15:30] <zetheroo> or does it just randomly try A, B or C ?
[15:32] <blackflow> keeps trying B
[15:32] <blackflow> I mean, keeps _using_ B until it fails, then tries C
[15:33] <blackflow> or in other words, it sticks to using the nameserver last known to work
[15:34] <zetheroo> how long does it take to fail on B?
[15:35] <blackflow> zetheroo: note that NXDOMAIN isn't a failure, and your B is cloudflare, and if you're asking for mt.local, you'll get NXDOMAIN
[15:35] <zetheroo> Just was odd to hit a brick wall like that
[15:35] <zetheroo> blackflow: oh, darn ... that config was what was changed to after the issue ...
[15:36] <zetheroo> originally the cloudflare IP was the third or forth
[15:37] <zetheroo> It was: Win server primary DNS, Win server secondary DNS, Gateway IP, Cloudflare Primary, CF secondary
[15:37] <blackflow> too much. more than 3 isn't effective for glibc anyway
[15:41] <zetheroo> is that a recent limit in glibc?
[15:42] <blackflow> no, it's from ay one. bug in LP shows 2007 at least
[15:43] <blackflow> bug #118930
[15:43] <blackflow> it's like that in pretty much all the distros, a glibc default nobody thinks is important to raise
[15:43] <zetheroo> whoa
[15:43] <blackflow> (and they be right. if 3 NS fails, you've got more serious issues :) )
[15:55] <rbasak> Seems to me that the bug is requesting much more complex DNS handling - more than what glibc does.
[15:55] <rbasak> In the VPN use cases for example, and where different non-Internet domains are being added by some subset of configured servers.
[15:56] <rbasak> In these cases, surely something like systemd-resolved would be more appropriate, rather than asking for implemnetation in the glibc resolver?
[15:57] <blackflow> would, if resolved behaved properly.
[16:00] <lordcirth> blackflow, what does it do wrong?
[16:00] <blackflow> lordcirth: for starters, breaks the decades old expectation of nameserver priorities. then it has issues with DNSSEC. then it has issues with VPNs.
[16:01] <blackflow> the NS order is very much important, it's not round robin. you have primary and then failover resolvers. resolved breaks that entirely with round-robin only resolving.
[16:02] <blackflow> you get round robin if you explicitely set options rotate in resolv.conf
[16:08] <rbasak> blackflow: if resolved has issues then they need to be resolved in resolved, sure. I don't think we can expect glibc to implement more complex behaviour than it already does though. resolv.conf is overloaded enough as it is.
[16:08] <rbasak> (and it's got to do everything in process, can't realistically keep state, etc)
[16:09] <rbasak> So I think the answer to that bug really needs to be "Won't Fix; use resolved, or implement and use some other proxy".
[16:09] <rbasak> However I don't consider myself expert enough in the area to decide that for the project without the opinion of some other Ubuntu devs who are more expert.
[16:11] <blackflow> rbasak: I agree. I just mentioned it because zetheroo had more than 3 nameserver entries so I pointed at that being useles
[16:12] <rbasak> Ah, sorry.
[16:12] <blackflow> rbasak: btw the issues all exist upstream. unfortunately some of them are being WONTFIX'd, like theh round robin thingy
[16:27] <supaman> what is the unit name for the nfs daemon?
[16:28] <supaman> nfs-server?
[16:29] <Ool> nfs-kernel-server is the service name
[16:29] <supaman> ok, thanks
[16:34] <blackflow> some like to call it nfs<tab> :)