[01:08] <martink3> apw: finally got around to file #1559609 regarding arcmsr timeouts with ARC1882 RAID controllers... hope the bug report is useful enough to get things going
[06:09] <alkisg> I'm continuing trying to troubleshoot https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1259861
[06:09] <alkisg> My plan is to do a small bisection by using the kernels from the mainline ppa, and see when it broke
[06:10] <alkisg> It was after 3.2 and before 3.8, I'll pinpoint it better in a while
[06:10] <alkisg> Then I'm planning to compare the configs of those two kernels, because I'm thinking it might be a config issue and not a source code (ipconfig.c) issue that caused it
[06:10] <alkisg> If you guys have some better suggestion, please ping me :)
[06:13] <alkisg> Erm, do the mainline ppa kernels use a different config from the ones shipped in the ubuntu archives + live CDs?
[06:23] <alkisg> Trying with Ubuntu 12.04 and the stock kernels,
[06:23] <alkisg> 3.5 does not have the issue, while 3.8 does
[06:23] <alkisg> So the problem was somewhere after Ubuntu 12.10 and before 13.04
[06:23] <alkisg> I'll compare the configs now
[06:26] <alkisg> 3.5: # CONFIG_IP_PNP is not set
[06:26] <alkisg> 3.8: CONFIG_IP_PNP=y
[06:26] <alkisg> CONFIG_IP_PNP_DHCP=y
[06:27] <alkisg> Is it reasonable to ask that to be reverted?
[06:32] <alkisg> Also, can I find the Ubuntu config commit for that, so that I can read the reasoning behind it?
[06:32] <alkisg> E.g. maybe it's needed for some ARM devices, if so it could be conditionally enabled only on those arches...
[07:10] <alkisg> I think this is the relevant upstream commit: https://github.com/torvalds/linux/commit/c1b362e3b4d331a63915b268a33207311a439d60#diff-364c3610ebc6899c22148ba10636c71c
[08:13] <alkisg> So, Ubuntu and openSUSE have CONFIG_IP_PNP=y and experience the issue
[08:13] <alkisg> Debian and Fedora don't have it
[10:14] <apw> alkisg: put that info in the bug, needs thinking about indeed
[10:19] <alkisg> apw, I did put it
[10:19] <alkisg> Thank you :)
[10:20] <apw> ok so the real issue is that the initramfs and kernel option overlap
[10:20] <alkisg> Well, it's the same option, it can be handled in 2 places
[10:20] <alkisg> So if the kernel was smart enough to avoid the delay if it can't initialize the NIC, all would be fine
[10:21] <alkisg> Or if the Ubuntu kernel didn't include CONFIG_IP_PNP/ipconfig.c when it doesn't need it
[10:21] <alkisg> (does anyone need it? or was it put there by just copying the new upstream kernel defaults?)
[10:23] <alkisg> I think it would be best to unset CONFIG_IP_PNP in Ubuntu, like Debian and Fedora do, and also report the ipconfig.c issue upstream to the kernel, to be fixed by whoever maintains it, without any hurry...
[10:23] <apw> it would likely sound useful and get enabled
[10:26] <alkisg> I imagine that it's only useful when the modules are built-in, so if someone is building their own kernel, they may as well set CONFIG_IP_PNP themselves...
[10:26] <alkisg> Now it would only be useful in virtio environments, right? (and I'm not sure if all the NFS support is configured...)
[10:27] <alkisg> ogra_: have you ever needed to netboot any boards without using an initramfs? I.e. has CONFIG_IP_PNP=y ever been useful to you?
[10:28] <alkisg> (it's causing 10 sec delay to ltsp clients)
[10:50] <ogra_> no, never
[10:51] <alkisg> Ty :)
[10:54] <ogra_> which doesnt mean there are no people using it like that indeed
[10:56] <alkisg> Sure, but if we can't find anyone, maybe it'd be better to disable it like other distros do
[10:57] <alkisg> Our initramfs now is close to 40 MB, it doesn't help to include all options...
[10:57] <ogra_> iirc (apw might remember better) there was once a task "make ubuntu bootable without initrd" ... not sure if that has ever been mandatory or was just a nice-to-have thing 
[10:57] <alkisg> And I'm not sure it even works; I put ip=dhcp under kvm, with virtio drivers built-in, and it does nothing
[10:58] <ogra_> well, i'm pretty sure our cloud setup uses it that way
[10:58] <alkisg> Without an initramfs?
[10:58] <ogra_> no
[10:58] <ogra_> kvm based netboot
[10:59] <alkisg> The clould setup doesn't use an initramfs?!
[10:59] <ogra_> (with initrd, but it surely does more than nothing ;) )
[11:00] <alkisg> When an initrd is present, the ip=xxx and other netbooting options are handled there...
[11:00] <ogra_> right
[11:00] <alkisg> So the cloud people don't need CONFIG_IP_PNP either
[11:00] <ogra_> oh, you meant you used kvm without initrd and virtio builtin 
[11:01] <alkisg> I'm just trying to see if CONFIG_IP_PNP actually works or if anyone's using it, because if it doesn't, the easiest solution would be to disable it
[11:03] <alkisg> (it is disabled in other distros and it was disabled up to 12.04.1)
[11:04] <ogra_> if you are lucky there is some papertrail about why it was enabled
[11:04] <alkisg> ogra_: https://github.com/torvalds/linux/commit/c1b362e3b4d331a63915b268a33207311a439d60#diff-364c3610ebc6899c22148ba10636c71c
[11:04] <alkisg> I think we just followed hpa's upstream commit
[11:05] <ogra_> thats more like 8.10 though
[11:05] <ogra_> quite old
[11:06] <tjaalton> xenial server image doesn't support usb keyboards? that's a surprise :)
[11:06] <ogra_> if we had it disabled til 12.04 it sounds like some decision on your side
[11:06] <ogra_> tjaalton, yeah, convergence ... its all touchscreen or nothing now :P
[11:07] <tjaalton> bummer
[11:07] <apw> ogra_, we did a full review and enable a lot of stuff, so it may not have been a specific decision on that option as much as part of a general cleanup
[11:07] <apw> but if the initramfs does this and does it better, we may well be wrong in having it enabled
[11:07] <ogra_> apw, well, thats the decision i mean5t then ;)
[11:08] <ogra_> if we know it just came in with housekeeping it is easy to decide to disable it again 
[11:08] <alkisg> Note that we still have CONFIG_IP_PNP_BOOTP and CONFIG_IP_PNP_RARP unset, which is different from upstream, so yup some thought went into that
[11:11] <alkisg> Is there a bazaar branch for the kernel config, where we could pinpoint that commit and read the commit message?
[11:11] <ogra_> has always been kernel.ubuntu.com i think 
[11:12] <alkisg> apw, comparing upstream with our config, I see e.g. CONFIG_E1000=y in upstream while CONFIG_E1000=m in ours. Same for realteks and other common NICs.
[11:12] <tjaalton> trying to install a gen8 hp microserver, 1404/1510 images just have syslinux output "boot error", and xenial has no usb.. oh well
[11:13] <alkisg> apw, so I don't think CONFIG_IP_PNP helps us if we don't also include those...
[11:13] <apw> alkisg, yep, and if one is not building those in, it is likely IP_PNP is an error, especially as initramfs will sort them out
[11:14] <alkisg> apw, cool, who do you ping to remove it? could you do it? I can do whatever tests are needed... :)
[11:14] <alkisg> *do we
[11:19] <apw> alkisg, oh i have the power, i need to think about it and check feature parity
[11:19] <apw> remind me of the bug #
[11:21] <alkisg> apw, https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1259861
[11:21] <apw> that is confirmed as "with kernel command line ip= option
[11:21] <apw> right ?
[11:22] <alkisg> Right
[11:22] <apw> and i think it is also confirmed as a 12s delay 
[11:22]  * apw fixes the title
[11:22] <alkisg> It's from 2 to 12
[11:22] <alkisg> So maybe it's 10
[11:23] <alkisg> I don't know if it's started at 2 or at 0 though (in which case it would be 12)
[11:23] <alkisg> Maybe it's this one:   #define CONF_CARRIER_TIMEOUT	120000	/* Wait for carrier timeout */
[11:24] <apw> anyhow, its all about feature parity, if initramfs-tools does more than the kernel we are good we can just wack the option
[11:24] <apw> if not we may have to get more creative
[11:25] <alkisg> initramfs-tools certainly does more than the kernel
[11:25] <alkisg> It also supports other protocols that the kernel doesn't, e.g. NBD
[11:27] <ogra_> only if you have the module 
[11:28] <apw> alkisg, yeah its if it does less in any way that someone could be using
[11:28] <ogra_> (i dont think it is there by default)
[11:28] <apw> that is really the only thing that matters
[11:31] <alkisg> AFAIK the initramfs ip= handling is in all cases superior to the in-kernel ip= handling
[11:32] <alkisg> The only case that I imagine someone would prefer the in-kernel one, is if he didn't use an initramfs at all
[11:33] <alkisg> apw, thank you for working on this
[11:33]  * alkisg waves, later...
[11:34] <TJ-> for reference, that change to the common config was introduced by 3166c8772 in March 2010
[11:35] <TJ-> (for Quantal)
[11:36] <TJ-> before that it was only used for armhf highbank arch
[11:41] <TJ-> correction, 301b4bb in 13.10 (Saucy)
[12:24] <tjaalton> 14.04 desktop works on ths microserver, so there must be some usb diff with server/desktop kernels that cause non-working usb with server image.. and I'm not the only one it seems
[12:26] <tjaalton> is usbhid included in base kernel?
[12:26] <tjaalton> doesn't look like it
[12:27] <tjaalton> https://blog.al4.co.nz/2014/12/ubuntu-14-04-no-usb-keyboard-after-upgrading-kernel/
[12:27] <tjaalton> so that's why 14.04.1 works and newer ones don't
[12:33] <tjaalton> filed a bug
[13:15] <apw> tjaalton, whats the number ...
[13:17] <apw> tjaalton, that blog post says that a server won't work without linux-image-extra which is correct
[13:17] <apw> tjaalton, -extra is for hardware support essentially, beyond what is in a virt image
[13:41] <apw> alkisg_away, ok ... i think i can see no reason to not turn this off for sure in xenial, so i have done that for the first upload after beta
[13:41] <apw> alkisg_away, if that does not throw regressions then we can consider it for sru, do remind me in a week or so
[13:41] <alkisg> apw, thanks a lot, I'll do so
[13:42] <apw> that means it will be off in linux-lts-x in trusty as well when that propogates back, so not the next one but the one after
[13:42] <alkisg> Will it reach even precise-lts-trusty in the long run?
[13:43] <tjaalton> apw: so server image has -extra too? then my bug description is all wrong :)
[13:43] <apw> tjaalton, yes, we only have linux-generic and linux-virtual, servers use linux-generic
[13:44] <tjaalton> ok
[13:44] <apw> the only way to install akernle without linux-image-extra and get updates is to use linux-virtual
[13:44]  * alkisg managed to get down the 16.04 ltsp client boot process from 45 seconds to 12 seconds, same as in 12.04... there were 4 unrelated timeouts involved :)
[13:44] <apw> linux-server points to linux-generic
[13:44] <apw> alkisg, nice
[13:44] <tjaalton> bug 1559692
[13:44] <tjaalton> hehe
[13:45] <tjaalton> can't update it right now
[13:47] <tjaalton> if there's something to try on cmdline i'll give that a go later
[13:58] <apw> tjaalton, are you saying the live desktop images work ok for xenial images but d-i ones do not ?
[13:59] <tjaalton> apw: i didn't try xenial live, had only trusty available
[14:00] <apw> it is more likely that there is something missing from a udeb if live works and server images not
[14:00] <apw> but we'd want to compare liek with like
[14:00] <apw> xenial live with xenial server
[14:00] <tjaalton> sure, that's what i'll test next
[14:01] <tjaalton> in about six hours from now :)
[16:00] <martink3> bug 1559609
[16:03] <martink3> apw, this is the bug I filed upon your suggestion
[16:28] <apw> martink3, is that the one where there is a couple of small 4.5 ocmmits which might make it work ?
[16:32] <martink3> yes, exactly. it should be sufficient to take the 4.5 commits regarding arcmsr.h and arcmsr_hba.c, and then it should work.
[16:33] <martink3> I was about to try to see if the current xenial kernel compiles with just these two changes, but then I had some dependencies issues for the build chain. I am trying it again using a recent xenial to build a -test1 with those changes
[16:48] <apw> martink3, i should be able to make you a test kernel if you can test it
[16:54] <martink3> apw, happy to do so!
[17:40] <apw> martink3, that (btw) is not two changes, that is the diff from v4.4 to v4.5 for those two files, which is 6-8 changes
[17:41] <apw> martink3, kernels will be here in a few mins, they are pushing now -- http://people.canonical.com/~apw/lp1559609-xenial/
[17:42] <martink3> apw, you are right. I didn't see it this way, but you are right, it's a couple of driver versions higher in areca's version numbering scheme, and they (areca) had some trouble getting anything accepted in mainline for some time
[17:42] <apw> upstream always resists, it is their job
[17:44] <martink3> I agree, and the driver became more readable, too
[17:48] <martink3> apw, I think all files arrived now, I will get them now...
[17:48] <apw> martink3, still pushing -extra, don't be fooled
[17:49] <apw> martink3, and you only need the arch you are, i386 or amd64
[17:49] <apw> likely just linux-image and linux-image-extra for that arch
[17:55] <apw> ugg my uplink sucks today
[17:55] <martink3> but for downlink, traffic is no consideration, right? If so, then I will just wget your whole lp1559609-xenial/ folder ...
[17:57] <apw> martink3, you can hammer my fileserver all you want
[17:57] <apw> the amd64 bits look complete, i386 -extra is still pusin
[17:58] <martink3> amd64 is what I need. will try it in a virtual machine first, and then I hope I can get my hands on one of the real fileservers soon for this test
[18:00] <apw> martink3, push complete
[18:02] <apw> martink3, please report your testing in the bug, and let me know here you did so
[18:07] <martink3> apw, got the files, will do, thanks so far!
[18:32] <lamont> jsalisbury: updated 1543863 for our sadness
[18:33] <apw> lp: #1543863
[18:42] <lamont> apw: also, I fail at typing bug numbers
[18:42] <lamont> lp: #1543683
[18:42] <lamont> yeah that one
[18:43] <lamont> jsalisbury: 1543683 that is
[21:02] <tjaalton> apw: confirmed, xenial desktop image has working input devices and server doesn't
[21:30] <apw> tjaalton: ack could you update the bug to that effwct pls
[21:31] <tjaalton> done
[21:31] <tjaalton> i checked the obvious udebs and they look fine