[12:08] <kirkland> is there a such place where one can find a nightly (or regular) tarball of bcollins/ubuntu-2.6.git tree?
[12:09] <kirkland> i see /topic, but those look a little out of date...
[01:00] <crimsun> kirkland: I don't know offhand, but you can always just crontab a ``git pull''
[04:56] <mjg59> BenC: When you updated the bcm code, you seemed to stomp over my patches...
[04:56] <BenC> crap, I probably did
[04:56] <mjg59> BenC: Should just reapply, with luck
[04:57] <BenC> wish bcm had a 2.6.15 git tree :/
[04:57] <BenC> yeah, I can probably re-base them in git
[04:57] <mjg59> You're pulling from the 2.6.16 one and then merging back?
[05:00] <BenC> your changes or bcm43xx in general?
[05:00] <mjg59> bcm43xx in general
[05:01] <BenC> nah, grabbing daily-snapshot tarball and just copying
[05:01] <BenC> 2.6.16 would be hard to pull from
[05:02] <crimsun> BenC: (just a minor, but s/David/Daniel/ in the changelog)
[05:02] <BenC> ah, sorry :)
[05:02] <crimsun> no biggie :)
[05:03] <BenC> done
[05:03] <crimsun> gracias
[05:06] <mjg59> BenC: Ah, ok
[05:06] <mjg59> BenC: If you could reapply my diff, that would be great
[07:42] <lamont> BenC: isn't ide-generic still the module that acutally triggers ide subsystem initialization?
[07:44] <Keybuk> BenC: ping
[07:46] <BenC> lamont: no
[07:46] <Keybuk> BenC: it appears I missed a discussion this afternoon
[07:51] <makx> BenC: are you sure? (i mean considering isa)
[07:57] <BenC_> ide works with ide-generic
[07:57] <Keybuk> BenC_: so it appears I missed a discussion this afternoon?
[07:57] <BenC_> Keybuk: appears so :)
[07:58] <Keybuk> BenC_: which is a shame, because your conclusions were all wrong
[07:58] <Keybuk> anyway, am replying now via e-mail
[07:58] <BenC_> I don't recall having any conclusions
[07:58] <Keybuk> "wait 10s before loading ide-generic"
[07:59] <BenC_> that's not a conclusion, that's a suggestion :)
[07:59] <Keybuk> it's a decision, according to your e-mail

[07:59] <BenC_> based on Mith's argument
[07:59] <Keybuk> it won't help a monkey's
[07:59] <Keybuk> the kernel already has that "sleep 10" in the IDE init code
[07:59] <Keybuk> which is in the module loading bit
[07:59] <Keybuk> so blocks the return of modprobe
[08:00] <BenC_> but if the module load is blocking, so are all other modules
[08:00] <Keybuk> "all other modules" ?
[08:00] <BenC_> I didn't know ide-generic had a sleep(10) i it
[08:00] <Keybuk> ide-generic doesn't
[08:00] <BenC_> or is this ide-core?
[08:00] <Keybuk> every IDE driver does
[08:00] <Keybuk> ide-core
[08:00] <Keybuk> modprobe real-pci-driver
[08:01] <Keybuk> consistently doesn't return until udev has the events
[08:01] <BenC_> is this the "Probing later" thing?
[08:01] <Keybuk> it can take a few seconds to return on most things
[08:01] <Keybuk> I don't know what "probing later" is
[08:01] <BenC_> are you sure you read Mith's arguments that promoted me suggesting that?
[08:01] <BenC_> prompted
[08:02] <BenC_> he statement was that the main reason we don't use UUID more widely is that it has some dependency on the bus type
[08:02] <Keybuk> right
[08:03] <Keybuk> and the sleep 10 won't change that
[08:03] <Keybuk> it doesn't have a dependency on the bus type
[08:03] <Keybuk> this is the bit where me being there would have helped
[08:03] <BenC_> he stated differently
[08:03] <Keybuk> ok, so here's how the initramfs bits work
[08:03] <Keybuk> first we look at the ROOT= to detect the bus type
[08:03] <Keybuk> either IDE or "everything else" (which are always scsi)
[08:04] <BenC_> then it does have code that depends on ide or !ide
[08:04] <mjg59> Keybuk: Eh? That's entirely untrue, as far as I can tell
[08:04] <Keybuk> All the drivers under the scsi subsystem (which includes sata, usb, firewire) behave one way
[08:04] <mjg59> Keybuk: I can't find any sign of a sleep(10) in the ide-generic load path
[08:04] <Keybuk> the modprobe returns immediately, then at some point in the future the driver finds its devices
[08:04] <Keybuk> mjg59: not a literal sleep(10); not ide-generic; the ide-core probes and binds for the REAL PCI DRIVERS before modprobe can return
[08:05] <mjg59> Keybuk: How?
[08:05] <Keybuk> mjg59: please let me finish first
[08:05] <Keybuk> ok, so for scsi and friends; we have to iterate the bus, modprobe all the drivers, then sit back and wait for the devices to show up
[08:05] <Keybuk> sit back and wait up to three minutes
[08:06] <Keybuk> for especially slow scsi controllers (fabbione delights in owning these)
[08:06] <Keybuk> so we do it with a simple while loop; while the root device hasn't shown up on disc, sleep another tenth of a second
[08:06] <Keybuk> if it at the end of that loop, the root device still hasn't shown up, we give up
[08:06] <BenC> ok, the ide portion is more important :)
[08:07] <Keybuk> for dapper+1 I'd like to extend that to "wait forever", and have some pretty on screen message saying something along the lines of "if your root disk isn't plugged in, could you do that?"
[08:07] <Keybuk> so that's scsi
[08:08] <Keybuk> ide we do it differently, because the ide subsystem has different quirks
[08:08] <Keybuk> the drivers behave the other way, the modprobe doesn't return until the driver has done the probe-and-bind
[08:08] <Keybuk> there's no asynchronous component to them
[08:09] <Keybuk> we also may have to load ide-generic on legacy ISA machines
[08:09] <Keybuk> and in some cases with a root device on an ancient PCMCIA controller (which counts as "legacy ISA" in my mind)
[08:09] <mjg59> Keybuk: I'm still not finding anywhere where the PCI IDE drivers differ from normal PCI drivers (where there is an asynchronous component)
[08:09] <mjg59> Hm. PCMCIA ought to come under pcmcia_cs
[08:09] <mjg59> Uh, ide_cs
[08:10] <BenC> regular PCI IDE drivers use pci probing, and normal device layer callbacks
[08:10] <makx> pcmcia bridges are not incl. in latest initramfs by default
[08:10] <Keybuk> mjg59: critical path.  the ide core already knows what devices are available.  so there's no issuing of a probe command, and waiting for it to return.  it just iterates an already-in-memory structure and does the magic
[08:10] <mjg59> There's a corner case where it's not actually PCMCIA, but an on-board IDE interface is connected via the PMCIA slot
[08:10] <BenC> the difference is that IDE detects the drives at the same time it detects the controllers
[08:11] <BenC> so Keybuk is correct in how it works
[08:11] <Keybuk> right
[08:11] <Keybuk> if you detect the controller, you also just detected the drives
[08:11] <BenC> how do you decide to load ide-generic?
[08:11] <Keybuk> scsi after detecting the controller you have to ask the controller nicely to go detect the drives and get back to you on that
[08:11] <Keybuk> right
[08:11] <Keybuk> so because of the way ide works in this regard, what we do is
[08:11] <Keybuk> iterate each PCI device in order, and load the module for it, waiting for everything to settle in between
[08:12] <mjg59> Keybuk: You load piix. That registers itself as an IDE driver, which in turn calls pci_register_driver. After pci_register_driver, the probe routine in piix (piix_init_one) is called. If that binds to something, it does ide_setup_pci_device (which actually allocates the interface)
[08:12] <mjg59> Isn't that exactly the same as loading a network driver, except that pci_register_device is done in the IDE layer rather than in the module itself?
[08:12] <Keybuk> mjg59: right, nothing there returns to userspace
[08:12] <Keybuk> thanks for verifying that for us :)
[08:12] <Keybuk> so we know after we've loaded the PCI drivers, if the root device still hasn't shown up, it isn't going to show up later
[08:13] <Keybuk> so we load ide-generic as a last resort
[08:13] <Keybuk> scsi looks like "load drivers; wait for root device to show up; abort after 3 minutes"
[08:13] <Keybuk> ide looks like "load drivers sequentially; if root device hasn't shown up, load ide-generic"
[08:14] <mjg59> So the original discussion appears to have been based on the false premise that we need to know whether the root device is on ide or not in order to only load ide-generic at the appropriate time?
[08:14] <Keybuk> well, no
[08:14] <BenC> no, it's correct
[08:14] <Keybuk> we still need to know whether we may or may not need to load ide-generic
[08:14] <BenC> the only incorrect info I got was the loading ide-generic was racy and could cause problems
[08:14] <mjg59> Why not load scsi drivers first, then ide drivers (at this point based on PCI IDs), then unconditionally load ide-generic if there's still no rootfs?
[08:15] <mjg59> Loading ide-generic isn't going to hurt in the case where the PCI driver has already bound the io-ports
[08:15] <Keybuk> mjg59: because of FUCKING SATA
[08:15] <mjg59> Keybuk: But you've already loaded the driver
[08:15] <Keybuk> SATA as far as I'm concerned is a scsi driver
[08:15] <BenC> mjg59: according to Mith, Fabio's crappy scsi controller breaks if you load ide-generic
[08:15] <Keybuk> it behaves like scsi drivers
[08:15] <Keybuk> most importantly, it returns from modprobe immediately, before performing the controller-specific initialisation
[08:15] <Keybuk> so modprobe piix-sata will return straight away, and udev will get the devices some few seconds later
[08:16] <Keybuk> on some older SATA controllers, on a busy system, this can be 10-15s
[08:16] <BenC> Keybuk: ok, here's the problems I see....
[08:16] <Keybuk> loading ide-generic at any point before the SATA SCSI driver has finished can sometimes "steal" the devices from it
[08:16] <Keybuk> the solution wwould be to only load ide-generic after the three minutes are up
[08:17] <Keybuk> which means everyone on ISA would have to wait 3 minutes before their machine started booting
[08:17] <mjg59> Keybuk: ata_piix calls piix_init, which calls pci_module_init, which will then provide a probe method which registers the pors
[08:17] <BenC> Keybuk: IMO, we should detect that ide-generic is needed at install, and make a note of it for initramfs to use :)
[08:17] <mjg59> Keybuk: I'm still not seeing how this is different to the IDE case (though there may be something very simple I'm missing)
[08:17] <BenC> or when initramfs is built, it could detect that
[08:17] <Keybuk> mjg59: right, and then it returns to userspace, and libata later calls the probe method
[08:18] <Keybuk> BenC: I agree.  We can look in /sys in the installer and see whether the driver is ide-generic or not;  if not, use UUID
[08:18] <mjg59> Keybuk: Sorry, /which/ probe method?
[08:19] <Keybuk> mjg59: I can't remember offhand; when I looked through the code (a few months ago) the SATA stuff was very much callback orientated
[08:19] <Keybuk> and hooked into the SCSI bits of the kernel which are totally async
[08:19] <BenC> mjg59: from what I can tell, scsi device probing happens in the scsiX kernel process and not in the module probe code path
[08:19] <Keybuk> whereas IDE was all one obvious line of function call codes
[08:19] <mjg59> BenC: What are you defining as "scsi device probing"?
[08:19] <Keybuk> mjg59: "devices handled by the kernel scsi subsystem", which includes SATA ide drives
[08:19] <BenC> where it probes for devices attached to the controller, as opposed to the controller itself
[08:20] <mjg59> BenC: But by that time it's already registered the i/o ports, surely?
[08:20] <Keybuk> mjg59: not always,no
[08:20] <Keybuk> loading the controller driver doesn't guarantee anything other than you've opened the controller
[08:20] <BenC> mjg59: i/o for the controller yes, but for the ports no
[08:20] <Keybuk> at that point, you may be lucky enough for the ports to be registered, but not generally
[08:21] <Keybuk> device probing on the controller happens later
[08:21] <mjg59> pci_request_regions is done in ata_pci_init_one
[08:21] <mjg59> Which is called directly from the ata_piix PCI probe routine
[08:21] <Keybuk> mjg59: there's an easy way to test it btw
[08:21] <Keybuk> boot with break=top
[08:21] <mjg59> Keybuk: Well, if I had any SATA machines...
[08:22] <Keybuk> modprobe -a piix-sata ide-generic
[08:22] <mjg59> (That is, SATA machines with legacy i/o)
[08:22] <Keybuk> about 7/10 times, your drives will be /dev/hda rather than /dev/sda
[08:22] <Keybuk> or just trawl malone for the ~100 bugs about that
[08:22] <mjg59> Keybuk: Right, I'm willing to believe that, but I still don't see any mechanism by which this can be true for sata but not for IDE
[08:22] <Keybuk> one of them from mdz :)  his laptop loves to trigger that case
[08:22] <mjg59> Uh. When did mdz get a new laptop?
[08:23] <Keybuk> sorry, strike that; mdz had a different bug, I think
[08:23] <mjg59> Yeah, he had ide-generic binding before piix
[08:23] <Keybuk> yeah
[08:23] <Keybuk> he doesn't have a sata laptop
[08:23] <mjg59> (Which you've just been telling me is impossible, but :) )
[08:23] <Keybuk> no we loaded ide-generic first in his case
[08:23] <Keybuk> that was a clear-out bug :p
[08:24] <Keybuk> mjg59: if you start with IDE from the top, you'll see that nothing returns to userspace until the drives are already probed for and bound
[08:24] <Keybuk> mjg59: if you start with SCSI/SATA from the top, there are plenty of return to userspace b
[08:24] <Keybuk> points before the drives are probed for
[08:24] <mjg59> Keybuk: As far as I can tell, in both the IDE and sata cases, the io regions are requested in the call into the core layer that occurs at the end of the pci probe
[08:24] <BenC> ok, from what I'm hearing, ide-generic should really be the corner case instead of ide in general
[08:24] <Keybuk> I'm not sure about registering of i/o ports, I wasn't looking for that
[08:24] <Keybuk> and I don't really understand that
[08:25] <mjg59> Keybuk: It's the registration of i/o ports that's then /entire point/
[08:25] <Keybuk> I was looking at pure "when does the kernel create sdXY and associate it with that driver"
[08:25] <mjg59> If the i/o ports are registered, ide-generic won't bind. Otherwise, it will.
[08:25] <Keybuk> but ide-generic does bind
[08:25] <mjg59> Keybuk: Yes, but that's not the problem
[08:25] <BenC> yeah, which driver grabs the ports from the kernel is the winner
[08:25] <mjg59> So. How can that situation arise in the sata case, but not the ide one?
[08:25] <BenC> but from what I can tell, there not only i/o per controller, but i/o per device
[08:26] <mjg59> BenC: No
[08:26] <mjg59> One i/o region per channel
[08:26] <BenC> ok, one per primary and secondary
[08:27] <Keybuk> I'm quite happy to change the initramfs code to look like:
[08:27] <Keybuk> - load pci drivers
[08:27] <Keybuk> - load ide-generic
[08:27] <Keybuk> - wait for up to three minutes for the root device to show up
[08:27] <Keybuk> - abort
[08:27] <mjg59> Keybuk: Could you take a look at drivers/scsi/ata_piix.c and tell me where it enters userspace between loading and hitting line 4843 of libata-core?
[08:28] <mjg59> I'm genuinely failing to get this
[08:29] <Keybuk> I can't see where it gets to libata-core
[08:29] <Keybuk> ah, sorry
[08:29] <Keybuk> at the bottom
[08:30] <Keybuk> right so piix_init calls pci_module_init calls piix_init_one calls ata_pci_init_one
[08:30] <mjg59> Yes
[08:30] <mjg59> Which is identical to the IDE case, as far as I can tell
[08:31] <Keybuk> right
[08:31] <Keybuk> and is this true for every single sata driver?
[08:31] <Keybuk> if it's true that all the sata drivers reserve their regions in init, then I'm happy
[08:32] <Keybuk> that may not have been true earlier, or we may have had a different bug
[08:32] <mjg59> ahci seems to do it internally, but doesn't do legacy IDE anyway as far as I can tell
[08:32] <Keybuk> and as long as reserving the regions guarantees ide-generic can't steal the devices
[08:32] <BenC> that still doesn't help out issue where scsi in general is delayed device probing
[08:32] <Keybuk> how does it effect that issue?
[08:33] <mjg59> BenC: That's fine. Load all SCSI controllers, loda all PCI IDE, load ide-generic, wait up to three minutes (or whatever)
[08:33] <BenC> sata behaving like ide doesn't help anything :)
[08:33] <Keybuk> this wouldn't distinguish between SCSI and PCI IDE drivers
[08:34] <Keybuk> can anyone recall off-hand how we avoid loading both the SATA and PATA driver for a controller?
[08:34] <Keybuk> do we do that in the kernel?
[08:34] <mjg59> They have different PCI IDs
[08:34] <BenC> so basically, from my point of view is, load scsi, ide, ide-generic, loop for 3 minutes checking for device path and/or UUID
[08:34] <mjg59> BenC: Yeah
[08:35] <mjg59> Keybuk: We still need to deal with the case where ata_piix binds to the controller but can't drive it
[08:35] <Keybuk> mjg59: we do?
[08:35] <Keybuk> do we deal with that nw?
[08:35] <mjg59> No
[08:35] <mjg59> Which breaks several machines
[08:35] <Keybuk> oh, well, if it's not a regression... :)
[08:35] <Keybuk> was this what we talked briefly about in the DeathSprint?
[08:35] <mjg59> It's not a regression, but it's been an open bug since before breezy
[08:35] <mjg59> Think so
[08:36] <mjg59> ahci and ata_piix share PCI IDs. We should always try to bind ahci first.
[08:36] <Keybuk> can't remember what the decision was there though
[08:36] <mjg59> Currently, the reverse happens
[08:36] <Keybuk> right, and some cards work for ahci, some for ata_piix
[08:36] <Keybuk> and you can't tell which?
[08:36] <mjg59> ahci should always fail when it's not drivable by ahci
[08:36] <mjg59> But ata_piix will sometimes bind even when it can't drive it
[08:37] <Keybuk> why doesn't ata_piix do the same?
[08:37] <Keybuk> that sounds like the bug to me
[08:37] <mjg59> Broken BIOS
[08:37] <Keybuk> they might have a second ide controller that needs ata_piix
[08:37] <mjg59> Or, rather, the detection code in ata_piix assumes the BIOS is sensible
[08:37] <mjg59> Keybuk: I can't see how
[08:37] <mjg59> Keybuk: You can't get them on PCI cards
[08:38] <mjg59> Though it would be more of a problem once ata_piix is needed for driving PATA devices
[08:38] <mjg59> (So not a dapper issue)
[08:38] <Keybuk> yeah, my inclination with anything involving two fighting modules is to fix the modules
[08:38] <Keybuk> as there's always a corner case where you may HAVE to load both modules on a given machine
[08:38] <mjg59> I'm not convinced it's possible to fix the module in this case
[08:39] <Keybuk> we have similar examples today where people have OSS and ALSA sound cards in the same machine
[08:39] <Keybuk> and have filed bugs saying that ALSA OSS emulation doesn't work
[08:39] <mjg59> Right, but that's clearly a bug in alsa
[08:40] <Keybuk> or there's one where they need both snd_bt87x and bt878 loaded; but snd_bt87x won't work if bttv (dep of bt878) has been loaded first, etc.
[08:40] <BenC> yeah, that's a bttv bug, IMO
[08:41] <Keybuk> I
[08:42] <Keybuk> I do want to think about "blacklist-if" kind of thing for dapper+1
[08:42] <mjg59> Keybuk: Ok. Almost all sata drivers call pci_request_regions themselves. The only one that doesn't is ata_piix, and that's the case we've already discussed
[08:43] <mjg59> So if there's no race in ide, I can't see any race here
[08:45] <Keybuk> hmm
[08:45] <Keybuk> the only thing about modifying modprobe to selectively blacklist, or force a particular order, is it only works if one alias requests both modules
[08:46] <Keybuk> it fails for when you have two devices which cause both modules to be loaded
[08:46] <Keybuk> which again, leads me back to pointing at the driver itself and saying "iz module bug"
[08:48] <mjg59> Well, I'm fairly convinced it's a BIOS bug
[09:02] <Keybuk> we can't ship updates to those via APT though :p
[09:08] <mjg59> Yeah, so the issue is whether we can work around it
[09:09] <mjg59> Based on what you were saying earlier, just loading ahci first should be sufficient, right?
[09:09] <mjg59> Because it should bind to the PCI device before you've had a chance to load ata_piix
[09:09] <Keybuk> the problem is when ata_piix and ahci are loaded due to different devices
[09:09] <Keybuk> no, it's a race condition :(
[09:09] <mjg59> That's not going to be a problem in this specific case
[09:10] <mjg59> Hang on. How is that a race condition and our earlier conversation wasn't?
[09:10] <Keybuk> (on phone, will be more eloquent in a sec)
[09:11] <mjg59> If there's a race between ahci and ata_piix being loaded, surely there's a race between piix and ide-generic?
[09:11] <mjg59> (I'm ignoring the case of multiple PCI devices, because that doesn't apply in the problem case as far as I can tell)
[09:17] <lamont> BenC: I could swear that was the case in 2.6.12 for ia64 (at least...)  ISTR uploading 3 times to get it right...
[09:58] <Keybuk> mjg59: sorry, off phone now
[09:58] <Keybuk> damned relationships are so hard work <g>
[09:59] <Keybuk> race condition was with multiple devices, as we call modprobe at basically the same time
[09:59] <Keybuk> we could do a modprobe patch to handle the case of both drivers being expanded from an alias for a single device
[09:59] <Keybuk> would be trivial
[09:59] <Keybuk> it builds a list of them, and just have a "force-order" config option or something
[09:59] <Keybuk> sorting depmod output would be another option
[10:02] <mjg59> Keybuk: Right. In the cases I've seen, there's only one device
[10:02] <mjg59> So just having it load ahci first would be fine
[10:31] <cjb> Has anyone seen problems with bluetooth on latest dapper?  I can boot 2.6.15-15, but 2.6.15-19 spawns gdm and then presumably tries to do something with hcid that never returns; I never get any gettys launched.
[10:32] <cjb> If I hit ctrl+alt+del on the empty tty1, the shutdown gets to stopping hcid and then hangs there.
[10:36] <lamont>  4) load ide-generic for legacy ISA systems
[10:36] <lamont> hrm...
[10:36] <mdz> cjb: try to confirm that hcid is in fact getting hung up, and then see if you can find out what's causing it
[10:36] <lamont> what do we do on ia64 boxes?
[10:37] <mdz> lamont: the same thing?
[10:37] <lamont> if we loaded any ide modules, i think we have to load it.
[10:37] <lamont> unless they fixed 2.6.15 to not do ide init (on ia64) in ide-generic...
[10:38] <mdz> lamont: read that "for" as "for the benefit of" not "only for"
[10:38] <lamont> ah, pok
[10:38] <infinity> Indeed.
[10:38] <lamont> s/p//
[10:38] <infinity> ide-generic is still required for anyone with root on /dev/hd* ... More or less.
[10:38] <infinity> So we load it last, after the rest of the world settles (or doesn't)
[10:39] <lamont> ciik
[10:39] <lamont> cool
[10:39] <mdz> infinity: this seems to have been called into question during the discussions today, but we're still loading it at the end
[10:42] <infinity> mdz: Yeah, I've read the (heated) backscroll a bit while attempting to wake up. :)
[10:43] <mjg59> lamont: That hasn't been the case for a while
[10:43] <mjg59> lamont: legacy IDE init is done when ide-generic is loaded, but anything else will be driven by the appropriate PCI driver
[10:44] <mjg59> Anyway, after discussing it with Scott, we've come to the conclusion that there are no races now
[10:44] <lamont> mjg59: the issue I ran into was with an ide hard drive...
[10:44] <mjg59> lamont: In the old days, you loaded PCI drivers and then had to load ide-generic in order to get the PCI drivers to bind
[10:44] <mjg59> That was a bug, and it's been fixed
[10:44] <lamont> 0000:a0:02.0 IDE interface: Silicon Image, Inc. (formerly CMD Technology Inc) PCI0649 (rev 02)
[10:44] <lamont> ah, ok.
[10:45] <lamont> for old days == something including breezy's 2.6.12?
[10:45] <mjg59> Probably, yeah
[10:45] <lamont> ok
[10:46] <mjg59> lamont: I don't seem to get ide-generic loaded at all now