=== TheMuso_ is now known as TheMuso
[10:25] <lool> cjwatson: Around?
[10:25] <lool> cjwatson: I need some help to document the next steps on #345534 a bit more verbosely
[10:25] <lool> cjwatson: It's not entirely clear to me in what form the last actions should be implemented
[10:25] <lool> I'll add an ubiquity task as well
[10:27] <cjwatson> "moving stuff out of postinst to a separate script as to allow ubiquity to run it" delete that
[10:27] <cjwatson> it's entirely feasible for ubiquity to just call the postinst
[10:27] <cjwatson> fconfig I don't know about, I'll have to leave that to you guys
[10:28] <cjwatson> I can do a first pass at the ubiquity work
[10:29] <lool> cjwatson: Would be happy to have that
[10:29] <lool> cjwatson: fconfig is to update the bootscript which has the kernel cmdline
[10:29] <lool> The one we use on the install SD card will be replaced by the one to boot the final system
[10:30] <lool> e.g. the cmdline with boot=casper in live SD will be replaced with one which has an UUID= and all
[10:30] <cjwatson> ok
[10:34] <ogra> cjwatson, have you seen my updated syslog for the pre-partitioned issue ?
[10:35] <ogra> i did a try with ext3 as well for which i didnt upload logs (to make sure its no ext4 specific prob) which had the same result
[10:35] <cjwatson> ogra: yes, haven't dug into it yet though
[10:35] <ogra> just wanted to point out its not caused by selecting the wrong fs
[10:36] <ogra> behavior is identical with the right one selected
[10:58] <cjwatson> ogra: that just means there are two bugs :-)
[10:59] <cjwatson> ogra: there are several places where parted_server might incorrectly decide that the partition table needs to be rewritten; this is why working on a device with busy partitions is so delicate, because a very minor logic flaw in any one of those will cause the whole thing to break
[11:16] <ogra> cjwatson, yeah, i understand that
[11:49] <CIA-28> ubiquity: cjwatson * r3150 ubiquity/ (debian/changelog scripts/install.py): Refactor architecture/subarchitecture detection slightly.
[11:59] <CIA-28> ubiquity: cjwatson * r3151 ubiquity/ (7 files in 5 dirs): First pass at flash-kernel integration for armel (LP: #345534).
[12:00] <cjwatson> lool,ogra: ^- that's the general approach but of course may well need adjustment (it's untested); feel free to shout if anything else is needed in ubiquity
[12:00] <ogra> will do, i'll test with the next iso
[12:01] <ogra> note that there is still someting missing that sets the UUID in flash-kernel-installers postinst
[12:01] <cjwatson> right, happy for you guys to fix that though ;-)
[12:01] <ogra> i'm working on that today and might need your help later (i have no idea how to find the target device to get the uuid, i guess there is a d-i function or variable))
[12:02] <cjwatson> ha, not really
[12:02] <ogra> oh
[12:02] <cjwatson> it's just reimplemented with mount wherever it's needed
[12:02] <cjwatson> see the findfs function in grub-installer, for example
[12:02] <ogra> how does the grub installation udeb know it ?
[12:02] <cjwatson> findfs () {
[12:02] <cjwatson>         mount | grep "on $ROOT${1%/} " | cut -d' ' -f1
[12:02] <cjwatson> }
[12:02] <ogra> yeah, thats what i planned to look at
[12:03] <cjwatson> hardly worth abstracting elsewhere
[12:03] <cjwatson> grub-installer takes $ROOT as a parameter but you can just hardcode it to /target
[12:03] <ogra> oki
[12:03] <ogra> that should suffice
[12:03] <ogra> just need to assemble a proper cmdline and add some fconfig glue
[12:04] <cjwatson> and then obviously vol_id -u. Careful of paths though
[12:04] <ogra> yep
[12:04] <cjwatson> partman-target does: PATH="/lib/udev:$PATH" vol_id -u blah
[12:05] <ogra> ok
[12:05] <cjwatson> since it's been variously in /lib/udev/vol_id and /sbin/vol_id IIRC
[12:05] <ogra> why vol_id (which needs root) and not blkid ?
[12:06]  * ogra would have used blkid by default
[12:06] <cjwatson> the installer is root
[12:06] <ogra> indeed
[12:07] <cjwatson> also, blkid has a cache and it doesn't check whether it's up to date
[12:07] <cjwatson> so it can give wrong answers
[12:07] <ogra> ah, ok, that makes sense
[12:07] <cjwatson> especially when you've just done something like putting a new filesystem on the device ;-)
[12:07] <ogra> oh, and apparently blkid doesnt have a short output, i would need to parse it
[12:07] <cjwatson> the installer uses vol_id consistently everywhere
[12:08] <cjwatson> I don't think blkid is available in a udeb in any case
[12:08] <ogra> ah
[12:08] <cjwatson> hmm, the reason why I've been half-dozy all morning is that I FORGOT TO HAVE COFFEE
[12:09] <ogra> ouch and its alreayd past morning ... you have to travle back in time to correct that
[12:09] <ogra> *travel
[12:09] <lool> ogra: flash-kernel already has findfs
[12:09] <ogra> lool, oh, i didnt know ... in the udeb ?
[12:10] <lool> debian/flash-kernel-installer.postinst
[12:10] <ogra> yeah, i see it
[12:10] <ogra> i should have looked first :)
[12:10] <ogra> so my patch turns even more trivial
[12:12] <ogra> the ugly part is that fconfig turns \n into just \ ... so the quoting is nontrivial
[12:12] <lool> Uh?
[12:13] <ogra> the redboot script needs \ as newline char
[12:13] <ogra> silly convention
[12:13] <ogra> FCONFIG_STRING=" $LOAD_INITRAMFS\ $LOAD_KERNEL\ e -r 0x1000000 -s $INITRAMFS_SIZE -c $CMDLINE"
[12:14] <ogra> luckily it doesnt break if you add leading spaces to lines
[12:14] <ogra> so =" $LOAD_INITRAMFS\ $LOAD_KERNEL\ e ..." works
[12:14] <ogra> ugly nontheless
[12:20] <lool> ogra: are the backslashes needed?!
[12:20] <lool> ogra: I see space in the raw config bytes
[12:20] <lool> 005f030: 0062 6f6f 745f 7363 7269 7074 0066 6973  .boot_script.fis
[12:20] <lool> 005f040: 206c 6f61 6420 6b65 726e 656c 0a65 202d   load kernel.e -
[12:20] <lool> 005f050: 6320 2263 6f6e 736f 6c65 3d74 7479 6d78  c "console=ttymx
[12:20] <lool> sorry, it's 0a
[12:21] <lool> if (*dest == '\\') { *dest = '\n';
[12:21] <lool> So there's a conversion from \\ to \n (0a above)
[12:22] <lool> But you could as well use real newlines  :)
[12:22] <lool> And on print there's no conversion
[12:22] <lool> It ends with 0a 00
[12:24] <lool> ogra: (I'd personally use \\ instead of \ though)
=== mcasadevall is now known as NCommander
[12:49] <ogra> lool, using \n makes me end up with something like http://paste.ubuntu.com/141370/
[12:52] <lool> ogra: \n isn't interpreted as 0x0a in bash; you want something like `printf "%s\n%s\n%s\n" "fis load initramfs" "fis load initrd" "exec -c ..."`
[12:53] <ogra> ah, well, escaped backslashes do it as well
[12:56] <lool> Yeah
[12:59] <lool> ogra: So I guess you're picking this up?
[12:59] <ogra> working actively on it already
[12:59] <ogra> so yes, feel free to assign to me
[13:05] <mvo> woah, thanks cjwatson for your super-quick fix for #351547
[13:09] <cjwatson> that's ok
[13:09] <cjwatson> did it work? :)
[13:12] <CIA-28> oem-config: cjwatson * r642 trunk/debian/ (changelog control):
[13:12] <CIA-28> oem-config: Suggest checkbox/checkbox-gtk rather than recommending
[13:12] <CIA-28> oem-config: hwtest/hwtest-gtk.
[13:20] <lool> cjwatson: Looked at the ubiquity changes for flash-kernel, they look really slick and nice, and I saw you patched the place which was patched for beta to allow us to continue without bootloader previouslyt
[13:32] <mvo> cjwatson: not sure, I have not rsyned a daily yet .) but I will try soon(ish) - my test machines mainboard died today, so I need to wait for the replacement before I can continue
[13:44] <CIA-28> partman-partitioning: cjwatson * r705 ubuntu/ (debian/changelog lib/resize.sh):
[13:44] <CIA-28> partman-partitioning: When resizing a non-virtual partition, update it after committing
[13:44] <CIA-28> partman-partitioning: changes, otherwise we might miss a change of filesystem that was just
[13:44] <CIA-28> partman-partitioning: committed (LP: #333356).
[13:47] <CIA-28> partman-partitioning: cjwatson * r706 ubuntu/debian/changelog: releasing version 65ubuntu2
[14:00] <CIA-28> ubiquity: cjwatson * r3152 ubiquity/ (debian/changelog ubiquity/components/usersetup.py): Fix yes/no question text in weak-password warning (LP: #352348).
[15:26] <CIA-28> oem-config: cjwatson * r643 trunk/debian/ (64 files in 2 dirs): Update translations from Launchpad.
[15:34] <CIA-28> casper: cjwatson * r607 trunk/ (2 files in 2 dirs):
[15:34] <CIA-28> casper: Fix writing of "$@" to diverted update-initramfs script (here-documents
[15:34] <CIA-28> casper: perform parameter expansion unless the delimiter is quoted).
[15:37] <CIA-28> casper: cjwatson * r608 trunk/ (debian/changelog scripts/casper-bottom/24preseed):
[15:37] <CIA-28> casper: Don't bring up a temporary network interface while fetching the preseed
[15:37] <CIA-28> casper: file when netbooting, as that will disconnect our root filesystem
[15:37] <CIA-28> casper: (LP: #351982).
[15:39] <ogra> cjwatson, for the initramfs size i have: initrd_size=$(wc -c /target/boot/initrd.img-$(uname -r)|cut -d ' ' -f1) in flash-kernel-installer, lool thinks thats not safe enough, but i am not sure the /initrd.img symlink exists at the point where flash-kernel-installer runs, do you know more ?
[15:39] <ogra> (i simply assumed $(uname -r) matches our installer kernel anyway)
[15:41] <cjwatson> ogra: the symlink should exist; although uname -r should indeed match
[15:41] <cjwatson> well, in ubiquity it should match otherwise you'll have trouble booting; not necessarily in d-i
[15:41] <cjwatson> so it's probably safer to use the initrd.img symlink
[15:41] <ogra> ok
[15:42] <ogra> i wasnt sure it exists yet since we're just configuring kernel-img.conf in this place
[15:42] <CIA-28> casper: cjwatson * r609 trunk/debian/changelog: releasing version 1.168
[15:42] <cjwatson> in d-i, base-installer sets it up, which runs well before flash-kernel-installer
[15:43] <cjwatson> in ubiquity, it's created by configure_hardware, which runs immediately before configure_bootloader
[15:43] <ogra> ah, good
[15:43] <NCommander> cjwatson, did persia's lpia fixes make it in to the last upload? I'm working on the resolving the kernel problems ATM, but its not that useful if we can't install a kernel
[15:43] <cjwatson> besides, other bootloader installers need the initrd.img symlink
[15:43] <cjwatson> NCommander: which ones? I know I've merged several of his fixes
[15:44] <NCommander> the one that added lpia to base-installer
[15:44]  * NCommander goes hunting for the bug
[15:44] <cjwatson> apparently not
[15:44] <cjwatson> I'll merge that now
[15:45] <NCommander> cjwatson, https://bugs.edge.launchpad.net/ubuntu/+source/base-installer/+bug/291670
[15:45] <ubottu> Ubuntu bug 291670 in base-installer "LPIA installer missing kernel" [Medium,Triaged]
[15:45] <NCommander> I'm working on resolving #347458 which will make cdrom-detector work, and make installation possible.
[15:48] <lool> ogra: Forget initramfs size calculation, we don't need them anymore
[15:49]  * ogra has http://paste.ubuntu.com/141467/
[15:49] <ogra> lool, got it booting ?
[15:49] <ogra> wow
[15:49] <lool> Yup
[15:49] <lool> ogra: I can actually get it to boot for you without changing your kernel :)
[15:49] <lool> ogra: Pass ramdisk_size=65536 on your kernel command-line and replace flash-kernel with a script I'll hand yuo
[15:49] <ogra> but i assume we need a special cmdline
[15:50] <lool> ogra: I'd like to push a new flash-kernel just updating the main flash-kernel script (for that change), ok with you?
[15:50] <ogra> sure
[15:50] <lool> ogra: i'm using: exec -r 0x01000000 -s 0x00940000 -c "console=ttymxc0,115200 root=/dev/mmcblk0p2 ro rootdelay=2"
[15:51] <lool> It makes booting slower because we're reading from flash (padding)
[15:51] <ogra> yeah
[15:51]  * NCommander notes something is seriously wrong with his laptop
[15:51] <lool> ~ 10 M instead of 4 M
[15:51] <CIA-28> base-installer: cjwatson * r358 ubuntu/ (11 files in 4 dirs): merge from lp:~persia/base-installer/add-lpia
[15:51] <ogra> 10M should be fine for the start
[15:54] <CIA-28> base-installer: cjwatson * r359 ubuntu/kernel/ (4 files in 2 dirs): try linux-lpia first, as is standard behaviour for Ubuntu
[15:56] <NCommander> thanks cjwatson
[15:58] <CIA-28> base-installer: cjwatson * r360 ubuntu/debian/changelog: releasing version 1.98ubuntu5
[16:06] <CIA-28> oem-config: cjwatson * r644 trunk/ (d-i/manifest debian/changelog):
[16:06] <CIA-28> oem-config: Automatic update of included source packages: console-setup 1.28ubuntu7,
[16:06] <CIA-28> oem-config: user-setup 1.23ubuntu15.
[16:07] <lool> ogra: I pushed the updated flash-kernel
[16:07]  * ogra waits patiently
[16:07] <ogra> i cant get it booting
[16:08] <ogra>  e -r 0x1000000 -s 0x00940000 -c "console=ttymxc0,115200 console=tty1 file=/cdrom/preseed/ubuntu.seed ramdisk_size=65536 boot=casper LIVEMEDIA=/dev/mmcblk0p1 --"
[16:08] <ogra> gives me a kernel panic ... so i assume you added something special ?
[16:08] <lool> ogra: did you pad your initrd with zeroes?
[16:08] <ogra> indeed i didnt
[16:08] <lool> ogra: also, check fis list -d
[16:09] <lool> ogra: It should show a datalength of 0x00940000; that was what prevented me using the actually correct padding I had earlier
[16:09] <ogra> ah
[16:09] <lool> http://people.ubuntu.com/~lool/flash-kernel
[16:09] <lool> Run that, and it should just work (tm)
[16:10] <lool> ogra: I pushed that one to the archive, it's much cleaner, but sadly even more complex
[16:10] <ogra> ogra@osiris:/var/build/babbage/images$ sudo fis -d /dev/mmcblk0 -o 0x40000 list|grep initramfs
[16:10] <ogra>        initramfs: addr = 0x00560000, size = 0x002e0000, entry = 0x01000000, length = 0x00452133, cksum = 0x1d051f73
[16:10] <lool> It's the only arch where we have to create an intermediate file and it also requires fis partitions instead of mtd
[16:10] <lool> ogra: Oh you need to use -s $size
[16:10] <ogra> indeed thats not 0x00940000
[16:12] <lool> ogra: Do you have changes in progress for flash-kernel?  Otherwise I could as well do that flash-kernel.conf thingy
[16:12] <lool> ogra: BTW I'm using a fixed kernel
[16:13] <ogra> feel free, i have http://paste.ubuntu.com/141467/
[16:13] <ogra> but that will need modification after your changes
[16:13] <ogra> i can push my stuff later if all your changes are in
[16:15] <lool> ogra: load_initrd versus ld_initrd
[16:16] <ogra> that makes the var name so long
[16:16] <ogra> (it was load_initrd load_kernel before :)
[16:17] <ogra> but i can indeed change it back
[16:18] <ogra> ah, intresting, with -s $size i get "RAMDISK: ran out of compressed data"
[16:20] <ogra> http://paste.ubuntu.com/141491/
[16:27] <lool> ogra: That's without ramdisk_size?
[16:27] <ogra> no, still with
[16:27] <lool> With padded ramdisk?
[16:27] <lool> and correct datalength?
[16:27] <ogra> no
[16:27] <ogra> its based on the current image
[16:28] <ogra> oh, wait, thats with ramdisk_size
[16:28] <lool> ogra: You need to have zeroes in memory, if you just fis load initramfs, it will be random RAM data after the initrd
[16:28] <ogra> yeah
[16:29] <ogra> i dont have the intramfs handy since i build the image on antimony ...
[16:33] <ogra> lool, pad=$(expr $ifissize - $ifilesize) .... pad=$(($ifissize - $ifilesize))
[16:33] <ogra> saves you the call to expr
[16:35] <lool> ogra: I copied this from Thecus; I didn't want to diverge from the rest of the file, but now that i look at it, i see that $(()) is used in the file as well
[16:35] <ogra> yeah, dirty
[16:35] <ogra> just noticed the same
[16:36] <CIA-28> oem-config: cjwatson * r645 trunk/debian/changelog: releasing version 1.54.10
[16:43] <ogra> lool, hrm, that flash-kernel change will produce probs for users that already have a babbage install
[16:44] <ogra> we would need to repartition the fis setup on the fly
[16:45] <ogra> initramfs: addr = 0x00560000, size = 0x002e0000, entry = 0x01000000, length = 0x00452133, cksum = 0x1d051f73 ...
[16:45] <ogra> 0x002e0000 = 3014656 bytes
[16:45] <lool> ogra: ?
[16:45] <ogra> ogra@osiris:/var/build/babbage/images$ wc -c armel.initrd-imx51
[16:45] <ogra> 4530483 armel.initrd-imx51
[16:46] <lool> ogra: I will fail on upgrades with the version I just pushed
[16:46] <ogra> check_size "$initrd" $ifilesize $ifissize
[16:46] <ogra> that will always fail
[16:47] <lool> Why?
[16:47] <lool> Oh you're saying your fis entry was too small
[16:47] <lool> Well I don't know how you picked your values; mine work   :-P
[16:47] <ogra> yes, since it doesnt hold anything
[16:47] <lool> But note that it's not a flash-kernel change, it's just an issue with newer kernels and your partition
[16:47] <lool> (more modules, not enough space)
[16:48] <ogra> yours uses addr and size
[16:48] <lool> ogra, NCommander, persia: you should create a /etc/flash-kernel.conf with:
[16:48] <lool> fis_dev="/dev/mmcblk0"
[16:48] <lool> fis_offset_hex="0x40000"
[16:48] <ogra> while the old way was to use addr and length
[16:48] <lool> fis_size_hex="0x1F000"
[16:48] <lool> ogra: Uh?
[16:49] <NCommander> huh?
[16:49] <ogra> size only tells how big the fis entry is ...
[16:49] <lool> ogra: Yes, and I'm padding it in full
[16:49] <ogra> not how big the initramfs is
[16:49] <lool> Yes, I don't care how large it is
[16:49] <lool> I'm just checking it fits
[16:49] <ogra> into what ? the fis entry only has meta info we dont need it that big
[16:49] <lool> ogra: I think you're confused
[16:50] <lool> ogra: check_size "$initrd" $ifilesize $ifissize is a check to make sure that the initramfs data fits in the initramfs partition
[16:50] <ogra> i think check_size should check on lenght
[16:50] <lool> $ifissize is the size of the initramfs partition, as listed in fis -l output
[16:50] <ogra> right
[16:50] <lool> ogra: What "length"?
[16:51] <lool> $ifilesize is the size of the file we want to write
[16:51] <ogra> initramfs: addr = 0x00560000, size = 0x002e0000, entry = 0x01000000, length = 0x00452133, cksum = 0x1d051f73
[16:51] <lool> length is the data length of the *old* data
[16:51] <lool> It's irrelevant
[16:52] <lool> ogra: Your fis partition for initramfs is too small; it's 3014656 bytes; my initrd is currently 3913573
[16:52] <ogra> so why did the 4530483bytes big initramfs we use atm ever get loaded at all if it was stored in a 3014656 bytes big partition ?
[16:52] <lool> ogra: Yours?  I don't know
[16:53] <ogra> the fis partition only holds metadata
[16:53] <lool> ogra: My partition is large enough; are you sure your ramdisk was loaded?  perhaps it was cut?
[16:53] <ogra> lool, we released beta with that
[16:53] <ogra> and yes, i'm sure it wasnt cut
[16:54] <lool> ogra: Perhaps the initrd was smaller?  I don't know
[16:54] <lool> ogra: Did flash-kernel run with the beta image?
[16:54] <lool> ogra: Perhaps you truncated it when creating the SD card image
[16:54] <ogra> length = 0x00452133 == 4530483 bytes
[16:54] <lool> Ouch
[16:54] <lool> I know what you did
[16:55] <lool> You wrote past the end of the fis partition for the initramfs
[16:55] <lool> fis is really stupid
[16:55] <lool> ogra: You had a fis entry for <-----> and you wrote <-----------> to it
[16:55] <ogra> fis only contains metadata that tells redboot from where to load the actual payload
[16:55] <lool> ogra: This is exactly like a partition table
[16:56] <ogra> so the fis partition doesnt need to have the actual size of the initramfs
[16:56] <lool> erf
[16:56] <ogra> entry = 0x01000000, length = 0x00452133 are the relevant infos redboot uses ...
[16:57] <lool> ogra: No
[16:57] <lool> Not if you fis create
[16:57] <ogra> i do fis create
[16:57] <lool> It will flash past the end of your partition
[16:57] <lool> ogra: You realize that if anything is after the initramfs partition it will be overwritten?
[16:57] <ogra> i actually copied your code from flash-kernel to do that
[16:58] <lool> You're lucky to have 20M as the first partition
[16:58] <ogra> and used the settings the FSL binary blob had as defaults
[16:58] <lool> ogra: No, my code in flash-kernel enforces the size to be large enough
[16:58] <lool> You might have copied the code doing the write, but not copied the test...
[16:59] <lool> ogra: Anyway, done is done; it's a really serious bug for that flavour but fortunately it's used by a handful of people
[17:00] <lool> ogra: let's move it to -arm
[17:03] <ogra> grr
[17:04] <ogra> lool, i always used whats created by default by sd_init_babbage.bin
[17:04] <lool> ogra: I never used that
[17:05] <ogra> you never used updater.sh ?
[17:06] <lool> No
[17:06] <ogra> ah
[17:06] <ogra> well, everyone else did :)
[17:06] <lool> ogra: I think I mentionned the size of the partitions I created on my 16M SD card
[17:06] <lool> And mentionned that they were a bit tight, but enough
[17:07] <lool> ogra: The way I created my partition table was by running fis init -f and fis create on a running system; then I did it again by using the fis command
[17:07]  * ogra adjusts the builder script to use 0x00940000 by default from now on
[17:08] <lool> ogra: I think you would have caught that error with a tighter size for the first partition
[17:09] <lool> ogra: Hey let's have a quick phone call
[17:09] <ogra> one sec
[17:10] <ogra> ready
[17:10] <ogra> (we should probably also take that conversation to #ubuntu-arm :) )
[18:33] <NCommander> This may be a stupid question, but what is a driver update disk, and when do we produce them?
[18:34] <cjwatson> https://wiki.ubuntu.com/Ubiquity/DriverUpdates
[21:21] <CIA-28> ubiquity: cjwatson * r3153 ubiquity/ (16 files in 9 dirs): bump to 1.12
[22:23] <CIA-28> ubiquity: cjwatson * r3154 ubiquity/debian/ (80 files in 2 dirs): Update translations from Launchpad.
[23:17] <CIA-28> ubiquity: cjwatson * r3155 ubiquity/ (d-i/manifest debian/changelog):
[23:17] <CIA-28> ubiquity: Automatic update of included source packages: base-installer
[23:17] <CIA-28> ubiquity: 1.98ubuntu5, flash-kernel 2.13ubuntu5, partman-partitioning 65ubuntu2.
[23:25] <CIA-28> ubiquity: cjwatson * r3156 ubiquity/debian/changelog: releasing version 1.12.1