=== chihchun_afk is now known as chihchun === chihchun is now known as chihchun_afk === chihchun_afk is now known as chihchun === chihchun is now known as chihchun_afk [14:23] Good morning everyone! === yofel_ is now known as yofel [16:43] I am -><- this close to screaming at the UEFI code [16:43] I hate doing first-article bring-up of new chips sometimes [16:44] Booting can be a real pain in the ass [16:46] Anyone here have extensive experience with UEFI and arm64? [16:50] Martyn: I have some [16:50] HEY! You're online! [16:51] Martyn: what trouble are you having? [16:52] okay, so I am trying to bring up on a Juno board [16:52] Those have upstream support in EDK2, so what exactly are you trying to do? [16:53] I'm trying to boot over PCIe [16:53] so I need to initialize the PCIe bus early, in UEFI [16:54] ( this is on the A57 ) [16:55] However, once I do that initialization and copy/unpack the kernel into a bootable memory address -- The kernel fails to boot as -it- tries to re-initialize it [16:56] Martyn: which Juno revision? [16:57] r2 [16:57] r2p0 [16:58] oh wait -- that's wrong [16:58] I have Cortex A57 - r1p1 [16:58] and the a53 is r0p3 [16:58] so it's juno r1 [16:59] Hmm, that _should_ work. Which FW, which kernel, and are you using ACPI or DT? [17:00] I'm using devicetree, kernel 4.6-rc5 ( current mainline ), and unsure which firmware [17:01] Also, I've tested booting from SATA just to make sure that I'm not having some weird XAUI issue, and I can boot from SATA just fine [17:01] also I can boot from 10GigE ( also XAUI ) [17:02] Martyn: So which PCIe device(s) do you have plugged in in the failing case? [17:02] but when I try to cluster boot over PCIe .. somehow initializing the PCIe bus early results in a deadlock condition once the mainline kernel starts to boot. I don't have a 1wire debug available, or KEIL debug over JTAG, so I'm kind of stuck [17:03] Martyn: I was under the impression that the SATA was over PCIe, so what do you mean by "boot over PCIe"? Which device? [17:03] just a PCIe debug board I built. I'm clocking the data in, and all the data is well ordered [17:04] I'm trying to do something San Mehat did back at VA Linux Systems back in the early 2000's ... I am putting together a TCP/IP stack over PCIe, so that I can use one a64 card to boot another directly. [17:05] Martyn: given that SATA works (over PCIe), the issue sounds specific to your debug board; I'm not sdure I can be of much help. :/ [17:06] so the board that's plugged into the PCIe bus is fairly simple -- a SPEAr 1340 w/ single channel PCIe 2.0 RC/EP port [17:06] and all it does it clock in the data -- the kernel does load correctly into memory, and I have uncompressed it into RAM and can even start booting it [17:07] the -problem- is that the entire PCIe bus locks up right afterwards. I don't think the PCIe bus likes being re-initialized by when the kernel reboots [17:07] reboots/boots [17:07] Is there a way to prevent the kernel from attempting to re-initialize the PCIe bus, given that I've already done it during UEFI? [17:12] oh .. interesting. I just tried to checkout a fresh copy of EDK2 -- issues with git [17:12] warning: remote HEAD refers to nonexistent ref, unable to checkout. [17:12] Martyn: I don't follow why that would only be a problem when your board is plugged in? Surely it re-initialises it anyway? [17:13] That's what I was thinking... PCIe has to be initialized to get SATA working, so I figured this would be safe as houses [17:13] however, I'm not doing anything significantly more complex than what happens during SATA boot [17:14] I am copying data from the card I built, the data copies in, and as soon as the kernel does a PCIe probe -- BANG [17:14] deadlock [17:15] Bah .. I need to get JTAG on this [17:19] It may be worth asking on the linux-arm-kernel mailing list; this sounds like a PCIe issue, and people there are more familiar with the ARM/ARM64 details for that than I am [17:19] Yeah. With the A57/v8 chips starting to get a lot more popular out there ... I figured that y'all might be doing some unusual cluster booting techniques like I am here [17:19] Will do [17:20] LKML can be ... contentious :) [17:20] LAKML is also .. contentious :) [17:22] I boot my Juno with GRUB and TFTP, which works well enough for me [17:23] There are some nice people on LAKML (and LKML), if you can ignore the noise [17:25] Heh .. this work doesn't end with the Juno, of course. just like with Smooth-Stone/Calxeda -- early steps [17:25] Well, for now I'll stick to booting on 10gigE [17:25] But I need to crack this PCIe issue [19:03] mrutland : GAH! solved it. [19:04] mrutland : It was an out-of-order transaction happening on the PCIe bus... for some weird reason, the PCIe IP on the Juno isn't allowing it. [19:05] IS there a limitation on the a57's architecture that breaks when there are out-of-order transactions on PCIe? ( like there was on the A9's? ) [19:06] Is relaxed ordering allowed? [19:06] ( If I turn off the relaxed ordering bit in the TLP, things work ) [19:07] that would be bad news for things like 8 and 16 lane graphics cards...