[01:08] apw: finally got around to file #1559609 regarding arcmsr timeouts with ARC1882 RAID controllers... hope the bug report is useful enough to get things going [06:09] I'm continuing trying to troubleshoot https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1259861 [06:09] Launchpad bug 1259861 in linux (Ubuntu) "5-10 second delay in kernel boot" [Medium,Confirmed] [06:09] My plan is to do a small bisection by using the kernels from the mainline ppa, and see when it broke [06:10] It was after 3.2 and before 3.8, I'll pinpoint it better in a while [06:10] Then I'm planning to compare the configs of those two kernels, because I'm thinking it might be a config issue and not a source code (ipconfig.c) issue that caused it [06:10] If you guys have some better suggestion, please ping me :) === zequence_ is now known as zequence [06:13] Erm, do the mainline ppa kernels use a different config from the ones shipped in the ubuntu archives + live CDs? [06:23] Trying with Ubuntu 12.04 and the stock kernels, [06:23] 3.5 does not have the issue, while 3.8 does [06:23] So the problem was somewhere after Ubuntu 12.10 and before 13.04 [06:23] I'll compare the configs now [06:26] 3.5: # CONFIG_IP_PNP is not set [06:26] 3.8: CONFIG_IP_PNP=y [06:26] CONFIG_IP_PNP_DHCP=y [06:27] Is it reasonable to ask that to be reverted? [06:32] Also, can I find the Ubuntu config commit for that, so that I can read the reasoning behind it? [06:32] E.g. maybe it's needed for some ARM devices, if so it could be conditionally enabled only on those arches... [07:10] I think this is the relevant upstream commit: https://github.com/torvalds/linux/commit/c1b362e3b4d331a63915b268a33207311a439d60#diff-364c3610ebc6899c22148ba10636c71c [08:13] So, Ubuntu and openSUSE have CONFIG_IP_PNP=y and experience the issue [08:13] Debian and Fedora don't have it [10:14] alkisg: put that info in the bug, needs thinking about indeed [10:19] apw, I did put it [10:19] Thank you :) [10:20] ok so the real issue is that the initramfs and kernel option overlap [10:20] Well, it's the same option, it can be handled in 2 places [10:20] So if the kernel was smart enough to avoid the delay if it can't initialize the NIC, all would be fine [10:21] Or if the Ubuntu kernel didn't include CONFIG_IP_PNP/ipconfig.c when it doesn't need it [10:21] (does anyone need it? or was it put there by just copying the new upstream kernel defaults?) [10:23] I think it would be best to unset CONFIG_IP_PNP in Ubuntu, like Debian and Fedora do, and also report the ipconfig.c issue upstream to the kernel, to be fixed by whoever maintains it, without any hurry... [10:23] it would likely sound useful and get enabled [10:26] I imagine that it's only useful when the modules are built-in, so if someone is building their own kernel, they may as well set CONFIG_IP_PNP themselves... [10:26] Now it would only be useful in virtio environments, right? (and I'm not sure if all the NFS support is configured...) [10:27] ogra_: have you ever needed to netboot any boards without using an initramfs? I.e. has CONFIG_IP_PNP=y ever been useful to you? [10:28] (it's causing 10 sec delay to ltsp clients) [10:50] no, never [10:51] Ty :) [10:54] which doesnt mean there are no people using it like that indeed [10:56] Sure, but if we can't find anyone, maybe it'd be better to disable it like other distros do [10:57] Our initramfs now is close to 40 MB, it doesn't help to include all options... [10:57] iirc (apw might remember better) there was once a task "make ubuntu bootable without initrd" ... not sure if that has ever been mandatory or was just a nice-to-have thing [10:57] And I'm not sure it even works; I put ip=dhcp under kvm, with virtio drivers built-in, and it does nothing [10:58] well, i'm pretty sure our cloud setup uses it that way [10:58] Without an initramfs? [10:58] no [10:58] kvm based netboot [10:59] The clould setup doesn't use an initramfs?! [10:59] (with initrd, but it surely does more than nothing ;) ) [11:00] When an initrd is present, the ip=xxx and other netbooting options are handled there... [11:00] right [11:00] So the cloud people don't need CONFIG_IP_PNP either [11:00] oh, you meant you used kvm without initrd and virtio builtin [11:01] I'm just trying to see if CONFIG_IP_PNP actually works or if anyone's using it, because if it doesn't, the easiest solution would be to disable it [11:03] (it is disabled in other distros and it was disabled up to 12.04.1) [11:04] if you are lucky there is some papertrail about why it was enabled [11:04] ogra_: https://github.com/torvalds/linux/commit/c1b362e3b4d331a63915b268a33207311a439d60#diff-364c3610ebc6899c22148ba10636c71c [11:04] I think we just followed hpa's upstream commit [11:05] thats more like 8.10 though [11:05] quite old [11:06] xenial server image doesn't support usb keyboards? that's a surprise :) [11:06] if we had it disabled til 12.04 it sounds like some decision on your side [11:06] tjaalton, yeah, convergence ... its all touchscreen or nothing now :P [11:07] bummer [11:07] ogra_, we did a full review and enable a lot of stuff, so it may not have been a specific decision on that option as much as part of a general cleanup [11:07] but if the initramfs does this and does it better, we may well be wrong in having it enabled [11:07] apw, well, thats the decision i mean5t then ;) [11:08] if we know it just came in with housekeeping it is easy to decide to disable it again [11:08] Note that we still have CONFIG_IP_PNP_BOOTP and CONFIG_IP_PNP_RARP unset, which is different from upstream, so yup some thought went into that [11:11] Is there a bazaar branch for the kernel config, where we could pinpoint that commit and read the commit message? [11:11] has always been kernel.ubuntu.com i think [11:12] apw, comparing upstream with our config, I see e.g. CONFIG_E1000=y in upstream while CONFIG_E1000=m in ours. Same for realteks and other common NICs. [11:12] trying to install a gen8 hp microserver, 1404/1510 images just have syslinux output "boot error", and xenial has no usb.. oh well [11:13] apw, so I don't think CONFIG_IP_PNP helps us if we don't also include those... [11:13] alkisg, yep, and if one is not building those in, it is likely IP_PNP is an error, especially as initramfs will sort them out [11:14] apw, cool, who do you ping to remove it? could you do it? I can do whatever tests are needed... :) [11:14] *do we [11:19] alkisg, oh i have the power, i need to think about it and check feature parity [11:19] remind me of the bug # [11:21] apw, https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1259861 [11:21] Launchpad bug 1259861 in linux (Ubuntu) "5-10 second delay in kernel boot" [Medium,Confirmed] [11:21] that is confirmed as "with kernel command line ip= option [11:21] right ? [11:22] Right [11:22] and i think it is also confirmed as a 12s delay [11:22] * apw fixes the title [11:22] It's from 2 to 12 [11:22] So maybe it's 10 [11:23] I don't know if it's started at 2 or at 0 though (in which case it would be 12) [11:23] Maybe it's this one: #define CONF_CARRIER_TIMEOUT 120000 /* Wait for carrier timeout */ [11:24] anyhow, its all about feature parity, if initramfs-tools does more than the kernel we are good we can just wack the option [11:24] if not we may have to get more creative [11:25] initramfs-tools certainly does more than the kernel [11:25] It also supports other protocols that the kernel doesn't, e.g. NBD [11:27] only if you have the module [11:28] alkisg, yeah its if it does less in any way that someone could be using [11:28] (i dont think it is there by default) [11:28] that is really the only thing that matters [11:31] AFAIK the initramfs ip= handling is in all cases superior to the in-kernel ip= handling [11:32] The only case that I imagine someone would prefer the in-kernel one, is if he didn't use an initramfs at all [11:33] apw, thank you for working on this [11:33] * alkisg waves, later... === alkisg is now known as alkisg_away [11:34] for reference, that change to the common config was introduced by 3166c8772 in March 2010 [11:35] (for Quantal) [11:36] before that it was only used for armhf highbank arch [11:41] correction, 301b4bb in 13.10 (Saucy) [12:24] 14.04 desktop works on ths microserver, so there must be some usb diff with server/desktop kernels that cause non-working usb with server image.. and I'm not the only one it seems [12:26] is usbhid included in base kernel? [12:26] doesn't look like it [12:27] https://blog.al4.co.nz/2014/12/ubuntu-14-04-no-usb-keyboard-after-upgrading-kernel/ [12:27] so that's why 14.04.1 works and newer ones don't [12:33] filed a bug [13:15] tjaalton, whats the number ... [13:17] tjaalton, that blog post says that a server won't work without linux-image-extra which is correct [13:17] tjaalton, -extra is for hardware support essentially, beyond what is in a virt image [13:41] alkisg_away, ok ... i think i can see no reason to not turn this off for sure in xenial, so i have done that for the first upload after beta === alkisg_away is now known as alkisg [13:41] alkisg_away, if that does not throw regressions then we can consider it for sru, do remind me in a week or so [13:41] apw, thanks a lot, I'll do so [13:42] that means it will be off in linux-lts-x in trusty as well when that propogates back, so not the next one but the one after [13:42] Will it reach even precise-lts-trusty in the long run? [13:43] apw: so server image has -extra too? then my bug description is all wrong :) [13:43] tjaalton, yes, we only have linux-generic and linux-virtual, servers use linux-generic [13:44] ok [13:44] the only way to install akernle without linux-image-extra and get updates is to use linux-virtual [13:44] * alkisg managed to get down the 16.04 ltsp client boot process from 45 seconds to 12 seconds, same as in 12.04... there were 4 unrelated timeouts involved :) [13:44] linux-server points to linux-generic [13:44] alkisg, nice [13:44] bug 1559692 [13:44] bug 1559692 in linux (Ubuntu) "usbhid missing from main kernel package" [Undecided,Incomplete] https://launchpad.net/bugs/1559692 [13:44] hehe [13:45] can't update it right now [13:47] if there's something to try on cmdline i'll give that a go later [13:58] tjaalton, are you saying the live desktop images work ok for xenial images but d-i ones do not ? [13:59] apw: i didn't try xenial live, had only trusty available [14:00] it is more likely that there is something missing from a udeb if live works and server images not [14:00] but we'd want to compare liek with like [14:00] xenial live with xenial server [14:00] sure, that's what i'll test next [14:01] in about six hours from now :) [16:00] bug 1559609 [16:00] bug 1559609 in linux (Ubuntu) "arcmsr times out with ARC1882 RAID card" [Undecided,Confirmed] https://launchpad.net/bugs/1559609 [16:03] apw, this is the bug I filed upon your suggestion [16:28] martink3, is that the one where there is a couple of small 4.5 ocmmits which might make it work ? [16:32] yes, exactly. it should be sufficient to take the 4.5 commits regarding arcmsr.h and arcmsr_hba.c, and then it should work. [16:33] I was about to try to see if the current xenial kernel compiles with just these two changes, but then I had some dependencies issues for the build chain. I am trying it again using a recent xenial to build a -test1 with those changes === alkisg is now known as alkisg_away === shuduo is now known as shuduo-afk === shuduo-afk is now known as shuduo [16:48] martink3, i should be able to make you a test kernel if you can test it [16:54] apw, happy to do so! [17:40] martink3, that (btw) is not two changes, that is the diff from v4.4 to v4.5 for those two files, which is 6-8 changes [17:41] martink3, kernels will be here in a few mins, they are pushing now -- http://people.canonical.com/~apw/lp1559609-xenial/ [17:42] apw, you are right. I didn't see it this way, but you are right, it's a couple of driver versions higher in areca's version numbering scheme, and they (areca) had some trouble getting anything accepted in mainline for some time [17:42] upstream always resists, it is their job [17:44] I agree, and the driver became more readable, too [17:48] apw, I think all files arrived now, I will get them now... [17:48] martink3, still pushing -extra, don't be fooled [17:49] martink3, and you only need the arch you are, i386 or amd64 [17:49] likely just linux-image and linux-image-extra for that arch [17:55] ugg my uplink sucks today [17:55] but for downlink, traffic is no consideration, right? If so, then I will just wget your whole lp1559609-xenial/ folder ... [17:57] martink3, you can hammer my fileserver all you want [17:57] the amd64 bits look complete, i386 -extra is still pusin [17:58] amd64 is what I need. will try it in a virtual machine first, and then I hope I can get my hands on one of the real fileservers soon for this test [18:00] martink3, push complete [18:02] martink3, please report your testing in the bug, and let me know here you did so [18:07] apw, got the files, will do, thanks so far! [18:32] jsalisbury: updated 1543863 for our sadness [18:33] lp: #1543863 [18:33] Launchpad bug 1543863 in phpmyadmin (Ubuntu) "package phpmyadmin 4:4.0.10-1 failed to install/upgrade: subprocess installed post-installation script returned error exit status 2" [Undecided,New] https://launchpad.net/bugs/1543863 [18:42] apw: also, I fail at typing bug numbers [18:42] lp: #1543683 [18:42] Launchpad bug 1543683 in linux (Ubuntu Xenial) "Fails to detect (second) display" [Medium,Fix released] https://launchpad.net/bugs/1543683 [18:42] yeah that one [18:43] jsalisbury: 1543683 that is === alkisg_away is now known as alkisg === alkisg is now known as alkisg_away [21:02] apw: confirmed, xenial desktop image has working input devices and server doesn't [21:30] tjaalton: ack could you update the bug to that effwct pls [21:31] done [21:31] i checked the obvious udebs and they look fine