* enyc meows04:24
apwenyc, not heard anything against 5.4 this cycle so far08:37
enycapw: indeed may be complete misnomer, turned out that user 5.4.0-47 still09:02
enycapw: ooi any idea where there is a 'gap' in kernel cycle announced on kernel.ubuntu.com ?09:02
apwenyc, i think it is mentioned at least on the front page09:02
enycapw: sorry i mean   WHY ther eis  ...  as annoinced on ...09:03
* enyc thinks... maybe working on ubuntu 20.10 release kernel09:03
apwenyc, the cycle would have been badly aligned against the 20.10 release which isn't a great plan, and there is some infrastructure work going on which makes it hard to realign that cycle; so it was decided to pause over that period09:04
enycapw: i see =)09:04
enychrrm groovy 5.8 kernel09:05
apwenyc, nothing earth shattering or concerning; we will be right back at it the cycle after09:05
iraOn 20.04 I have a kernel hanging crypto tasks during a ceph install using ceph-ansible, to set up encrypted volumes.17:10
iraWhere's the best place to send info, and whatnot.17:10
iraI've also run the proposed kernel, shows the same issues.  I have not tried mainline yet.17:11
=== Eighth_Doctor is now known as Conan_Kudo
=== Conan_Kudo is now known as Eighth_Doctor
sarnoldira: which tasks? does strace show them hung? if so, where does the /proc/pid/stack for the processes say they are hung?19:35
iraI'm getting kernel wait messages.  I wiped the machines and I'm installing the OEM kernel + 20.04 to see if things change.19:36
iraThe whole setup is MAAS + ansible, so reconfiguring it isn't the end of the world.  19:38
iraAnd yeah they are hung, the install hangs.19:40
iraIt is using ceph-volume to create dm-crypt encrypted disks.19:41
sarnoldif it's moments after boot I wonder if you're hitting the /dev/random entropy stuff. can you shove in a random seed from the maas server to the cloud-init on those things?19:44
iraIt's way after boot.19:45
sarnolddays? or minutes?19:46
sarnoldstill plausibly randomness, a machine you never touch has limited ability to gather its own19:47
iraExcept it installs fine on 18.04.19:47
sarnoldoh interesting :)19:47
iraSame containers, scripting etc...19:48
iraRebooting the boxes right now to pick up 5.6.  I19:48
ira'll do the install there, and see how it goes... if it flies through it takes about 20-30m.19:49
iraIf it all fails, I'll reset it to stock 20.04 and be glad to do a debug session.19:50
ira(Even if not, I'd like to get this fixed, so we can use 20.04 and not 18.04 here.)19:50
ira@sarnold the 5.6 kernel does it.20:34
iraSo I got machines hanging for interrogation :)20:34
sarnoldsweet :) well, I mean, ugh :) but you know.. a reproducer is handy :)20:35
sarnoldcan you spot which processes look hung? what does strace say they are doing? /proc/pid/stack?20:35
iraWhat's the same thing as fpaste on fedora here?20:36
ira(old red hat engineer here... sorry man :) )20:36
ira(ex-red hat)20:36
tomreynpastebinit as a CLI20:37
tomreynor just    echo foobar | nc termbin.com 999920:38
iraAnd 5.6 locked faster than 5.4 it looks like.20:39
ira(the stack of that process.)20:40
iraThank you for helping :).20:41
iraAnything else off the machines you'd like?20:56
iraLooks like it doesn't happen every time, but it does happen, I see it made a few OSDs on that host successfully.21:12
iraI've also tried the non-containerized built for ubuntu ceph octopus packages, and they don't work either, in the exact same way.  In case there's a question there :)21:16
iraReinstalling the machines, with blacklisting intel_qat, and also blocking install.21:49
sarnoldira: oh wow looks like you've been very productive :D22:03
sarnoldira: I think yo'ure right, blacklisting qat looks promising22:03
irahere's to hoping.22:04

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!