[01:20] Ubuntu kernel builds appear to put out a bunch of stuff, .deb files, .udeb files, and a .tar.gz file, that I don't see specified in a document accurately. Is there a document I SHOULD be reading, or can I help updating one that is old? [01:20] I have been using https://wiki.ubuntu.com/Kernel/BuildYourOwnKernel, and for instance it says I should see 3 .deb files instead of dozens of files. [01:20] Please advise how I can be most helpful in either asking a question, reading other documents, or updating inaccuracies. [01:31] I apologize. I lost my connection. [07:13] xcyclist, running 'fakeroot debian/rules binary' (from the wiki page you reference) simply does a build of the binary packages for your currently running architecture (which results in 3 debs). A full official build will result in a whole lot more packages: binaries for all the supported architecture, indep packages, source package, ... [08:26] udebs for the installer and for x86 a signing tarball [09:21] sforshee: ping bug 1752507 [09:21] bug 1752507 in linux-firmware (Ubuntu Artful) "add i915/glk_dmc_ver1_04.bin" [Undecided,New] https://launchpad.net/bugs/1752507 [10:01] hello. [10:05] I'm on 4.13.0-36-generic, backported to 16.04, and see this crash/lockup ocasionnally: http://thre.sh/stuff/v05.crash.txt [10:05] does that sound similar to something already known? [13:04] jjohansen, tyhicks: either of you have any ideas about this? https://paste.ubuntu.com/p/JBHVsV9SsZ/ [13:04] it's happening during adt, but only on ppc64el [13:16] sforshee: hey - what's being tested when that's happening? [13:20] tyhicks: I don't think we know, cascardo? [13:23] I am not sure, but it might be apparmor [13:23] that sounds reasonable [13:23] or something way before that, cause we don't know exactly when it starts [13:24] it looks like something is testing/fuzzing the /proc//attr/current interface [13:25] the audit backlog messages start about 1.5 minutes later, possibly these two things aren't related [13:27] the "AppArmor: change_hat: ..." messages don't go through the audit subsystem [13:27] so they're unrelated in the sense that they're not causing the audit backlog issue [13:28] but it is still possible that AppArmor is generating the audit messages that are causing the audit backlog issue [13:28] I would disagree [13:28] [121737.681674] audit: type=1400 audit(1520969595.539:16276): apparmor="DENIED" operation="setprocattr" info="current" error=-22 profile="unconfined" pid=7572 comm="bash" [13:28] but, yeah, I also agree that might be a red herring [13:29] cascardo: to be clear, I suspect that something is testing apparmor and apparmor is generating a bunch of audit messages [13:30] what problems is it causing? is the testbed failing because of this? [13:30] tyhicks: the test does fail, we're not sure that it's related to all of those messages but it's one of the few leads we've got right now [13:31] rather the vm seems to get hung somehow [13:32] but I'm also now suspecting that it may be that a test is trying to suspend/resume the vm and that it's dying there [13:33] neither of those sets of error messages (the ones from AppArmor nor the ones from audit) indicate a dire situation that would take the vm down [13:33] they're error conditions for sure but they're handled error conditions [13:34] ok, thanks tyhicks [13:34] from my little experience with autopkgtest, sometimes the backend will look for the login prompt on the console [13:34] and I thought that was the problem, that so many errors on the console would prevent it from finding it [13:35] but maybe the nova backend is not that broken [13:35] that's possible [13:35] I'd argue that the apparmor error message is mostly useless and should go away [13:35] the audit message is important and can't go away [13:35] so, getting a clear console might not be possible [13:36] that's okay [13:36] I think the problem here is how it would generate that so many messages in that small amount of time [13:36] when trying to sigstop auditd, I couldn't get that to happen even when writing to attr/current in a loop [13:37] huh, why is auditd running in the VM? [13:37] that's surprising to me [13:37] I got a slightly different behavior as well, a "lost" message as well [13:37] ah, I am not saying it was [13:37] oh, I see what you're saying [13:37] that's just the only thing I thought would have caused those backlog messages [13:38] like: there need to be an open auditd socket, right? [13:38] the audit subsystem forwards to the syslog when an auditd hasn't registered itself with the kernel [13:39] that's the default configuration on ubuntu [13:39] I would expect that the test runners would be in that configuration (auditd isn't installed by default) [13:40] but how does it forward that? by the kernel log/printk only? [13:40] yeah, forwarding is in-kernel [13:42] okay, so I got that same TAP13 message as from the ADT log [13:46] right after running the step_after_suspend_test [13:46] and my VM can't respond anymore [13:46] what is supposed to be waking it up, by the way? [13:46] and nothing seemed to have changed on that test, either from our last kernel or on the testing repos [13:47] more likely, we broke suspend on ppc [13:47] or worse we made it do something, and the wake up does not work [13:48] i have this odd feeling we don't run those in adt in some sense [13:49] I could just test it with the older kernel and see how it behaves [13:49] yeah, the problem is we modify the kernel self tests etc to turn them off, and that can stop working [13:49] has done before [13:49] just have to set it up all over again, cause this openstack is broken [13:50] I looked it up, we didn't just turn it on again recently [13:50] so, it should have been running before [13:50] hrm, i'm confused, I was sure it was disabled for x86 [13:50] cking thinks we haven;t turned it back on [13:50] well this is ppc which is breaking on suspend [13:50] maybe I'm mistaken, a lot changes around here [13:50] so i am suspicious we should not be running it there either [13:50] as an unrelaible to wake up test [13:51] in a VM environment [13:51] it maybe that the RTC wakeup is broken in a VM, or it the RTC wakeup is too short, or a bunch of other reasons [13:52] hum... it was commented out on 4.15.0-10.11 [13:52] and clone didn't work on 4.15.0-11.12 [13:53] from a previous log which did run to completion, I don't see that test being run [13:54] https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-bionic/bionic/ppc64el/l/linux/20180221_002951_88802@/log.gz [13:55] okay [13:55] that file has changed [13:55] and the sed line is not working anymore [13:56] it's not a += anymore, it's a := [13:56] \o/ [13:56] I am fixing that on the testsuite, then will retrigger it [13:57] awesome, thanks cascardo [13:57] gotta make sure I write something that will work on new and old kernels [13:58] thank you for making me look past the audit backlog thing [14:02] nice! [14:03] now, let's hope the only problem we see is the kallsyms one [14:04] apw, can you please accept the nomination for #1755817? [14:05] juergh: approved [14:10] cascardo, thanks === himcesjf_ is now known as him-cesjf [15:06] Thank you juergh. Is there a document with the taxonomy of the different build combinations, both setups, and product files? [15:12] * apw doubts we have anything that detailed ... the intent of packages is one thing and one thing only to build in the defined debian/ubuntu build environment [17:11] cascardo: your ppc64el retry finished, the only failure is the expected kernel_symbols_missing [17:12] I'll add a hint for that, then we should be in good shape to promote [17:45] apw: the testing for our bionic kernel is all in order but it looks like we need to clear NBS [17:49] sforshee, i think i did that already, about 15:30 my time [17:49] but as it is 17:50 i am suspicious [17:50] apw: the adt matrix still shows it is NBS [17:50] sforshee, hmmm, should be resolved there soon, cirtainly britney is now showing it only held by the bug [17:51] ack, ta [18:40] sforshee: thanks!