cjwatsonxnox: Don't know if you saw, but we had a tester show up overnight who can reproduce 1080701 reliably - can we make use of him later today?08:21
stgraberbug 108070108:22
ubot2Launchpad bug 1080701 in ubiquity (Ubuntu Raring) "After 'Preparing to install Ubuntu' screen, raring installation hangs" [High,Confirmed] https://launchpad.net/bugs/108070108:22
xnoxcjwatson: noticed. will think how to best use "remote hands" =) I also want to try kentb's reproducer case. I think it might be another case of phantom lvm metadata getting left-over. Something similar to previous one bug 15408608:27
ubot2Launchpad bug 154086 in partman-auto-lvm (Debian) "Installing to HDD with previous ubuntu fails to create fresh LVM claiming group already in use" [Unknown,New] https://launchpad.net/bugs/15408608:27
cjwatsonBeware of possible multiple causes08:29
jackweirdyHello :) I'm trying to preseed installation for 12.04 on machines which will dual boot with MS Windows for a classroom setting. I can't find any information about using/resizing existing partitions in d-i, but I've noticed ubiquity can do the "install alongside" stuff. Is that preseedable?10:16
xnoxjackweirdy: look into using partman-auto/init_automatically_partition select biggest_free. That will install into largest free disk-space. That assumes that e.g. you pre-shrink windows installs.10:22
jackweirdyawesome; I'll look into that :) Presumably I could use early_command to do the shrinking itself?10:22
xnoxyou can try resize_use_free, but it can choke up on resizing window installation, and that would be harder to troubleshoot.10:22
xnox(resize_use_free should resize windows & then install into biggest free)10:23
xnoxjackweirdy: yes, you can do early-command as well, I'd recommend "partman/early_command" as that should have all the partitioning/fs utilities available to you.10:24
jackweirdyawesome, thanks for that :D10:26
dankTracing through bug 1171185 by inserting 'set -x; exec 2>/tmp/foo' in /bin/partman et al13:45
ubot2Launchpad bug 1171185 in ubiquity (Ubuntu) "Ubuntu installer appears to hang on "Installation von Ubuntu wird vorbereitet" screen" [Undecided,Incomplete] https://launchpad.net/bugs/117118513:45
xnoxdank: yeap, that would be handy. Also when you boot the image, edit the boot paramater to have "debug-ubiquity" in it, that way all logs will be more verbose.13:46
xnoxdank: i'm trying to reproduce it here as well based on kentb-out  comments, but i have not been successful yet.13:47
dankhanging in /lib/partman/display.d/10initial_auto, adding set -x there13:48
danknow hanging in /lib/partman/automatically_partition/15reuse/choices13:49
xnoxdank: also interesting if exectuing "os-prober" hangs. And what sort of set of operating systems are installed.13:49
xnoxdank: what about output of "mount" to see what's mounted where?13:50
xnox(i guess the fact that grub-mount uses mount namespaces will not help much)13:50
dankos-prober does not hang13:50
dankmount sez:13:52
dank/cow on / type overlayfs (rw)13:52
dankproc on /proc type proc (rw,noexec,nosuid,nodev)13:52
danksysfs on /sys type sysfs (rw,noexec,nosuid,nodev)13:52
dankudev on /dev type devtmpfs (rw,mode=0755)13:52
dankdevpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=0620)13:52
danktmpfs on /run type tmpfs (rw,noexec,nosuid,size=10%,mode=0755)13:52
dank/dev/sr0 on /cdrom type iso9660 (ro,noatime)13:52
dank/dev/loop0 on /rofs type squashfs (ro,noatime)13:52
danknone on /sys/fs/cgroup type tmpfs (rw)13:52
danknone on /sys/fs/fuse/connections type fusectl (rw)13:52
danknone on /sys/kernel/debug type debugfs (rw)13:52
danknone on /sys/kernel/security type securityfs (rw)13:52
danktmpfs on /tmp type tmpfs (rw,nosuid,nodev)13:52
danknone on /run/lock type tmpfs (rw,noexec,nosuid,nodev,size=5242880)13:52
danknone on /run/shm type tmpfs (rw,nosuid,nodev)13:52
danknone on /run/user type tmpfs (rw,noexec,nosuid,nodev,size=104857600,mode=0755)13:52
danktmpfs on /var/lib/polkit-1/localauthority/90-mandatory.d type tmpfs (rw)13:52
dankhmm, why does chatzilla think /cow should be italics?  :-)13:52
xnoxright, nothing unsual. / is this italics for you? /13:53
xnoxdank: can you give me output of os-prober?13:53
xnoxdank: and "parted -l"13:53
dankwait for it13:53
dankhmm, how does one edit boot parameters?  f6 gives only a small set of choices.13:55
xnoxdank: also I'm now thinking a tarball of /var/lib/partman would be most useful to debug why reuse is hanging.13:56
dankbefore triggering the problem?13:56
xnoxdank: well it will not be populated yet. so get to at least the point were partman has started and /var/lib/partman/devices got populated with folders.13:58
dankos-prober outputs13:58
xnoxand then create a tarball of /var/lib/partman/devices and email it to me or attach to a bug report.13:59
dank"/devv/sd1a:Ubuntu 13.04 (13.04):Ubuntu:linux"13:59
dankwhat's your email?13:59
=== kentb-out is now known as kentb
xnoxdank: my-irc-nick@ubuntu.com13:59
xnoxdank: or see PM14:00
dankparted -l says:14:03
dankModel: ATA WDC WD5000AACS-0 (scsi)14:03
dankDisk /dev/sda: 500GB14:03
dankSector size (logical/physical): 512B/512B14:03
dankPartition Table: msdos14:03
dankNumber  Start   End     Size    Type      File system     Flags14:03
dank 1      1049kB  14.0GB  14.0GB  primary   ext4            boot14:03
dank 2      14.0GB  22.0GB  8000MB  primary   linux-swap(v1)14:03
dank 3      22.0GB  500GB   478GB   extended14:03
dank 5      22.0GB  500GB   478GB   logical   ext414:03
dankok, I added set -x to /lib/partman/automatically_partition/*/choices14:05
xnoxdank: ok. Let me send you a couple of files and instructions on what to try. One moment please.14:05
dankand.... this time it didn't hang.14:06
xnoxdank: ok. still I'd like to send you something.14:07
dankmaybe debug-ubiquity makes it not hang14:09
xnoxdank: that's possible =) in that case reboot "normal", replace the reuse & replace choices with my patched scripts from people.canonical.com/~xnox/reuse and people.canonical.com/~xnox/replace14:11
xnoxwget -O /lib/partman/automatically_partition/15reuse/choices http://people.canonical.com/~xnox/reuse/choices14:12
xnoxwget -O /lib/partman/automatically_partition/25replace/choices http://people.canonical.com/~xnox/replace/choices14:12
xnoxchmod +x /lib/partman/automatically_partition/*/choices14:12
xnoxthis is my current guess, that grub-mount rightfully gracefully exits with non-zero, but ro-mount subsequently chokes up completely =)14:17
xnoxI guess it would be useful to redirect grub-mount and mount output somewhere useful from above.....14:17
xnoxnot sure where to though.14:17
dankI did the usual set -x and redirect inside partman.  Now to run...14:21
dankno hang14:22
dankwant the log?14:22
xnoxdank: yeah.14:22
xnox/var/log/partman /var/log/syslog14:22
xnoxand the output from set -x, if it's not in /var/log/syslog (should be)14:23
xnoxdank: if you install pastebinit package you can simply do " cat foo | pastebinit"14:27
dankoh, you kids, get off my lawn with this pastebin stuff :-)14:28
dankwhile you look at that I may do one more run with plain old files and judicious set -x to see if I can catch it hanging again14:28
xnoxawesome =)14:30
dankit hung14:37
dankI'll send you another batch of logs14:37
dankfun fact: ps shows 15reuse/choices still running, in S state14:42
xnoxthanks. Now, that it is hanging, can you try with replaced choices as above ^^^^ ? but please reboot. Once it's hanging, there is no clean way to go back to original state, and most likely it will work if you kill partman/choices and restart the installer instead of rebooting.14:45
cjwatsonIt's *usually* sufficient to kill all ubiquity/partman/parted* processes and rm -rf /var/lib/partman14:46
cjwatsonBut it requires some care and it's probably best to avoid introducing new variables while debugging14:46
xnoxcjwatson: but that won't cleanup anything which is already mounted by e.g. reuse/replace recipes which at this point may or may not have mounted something.14:49
xnoxi guess one can also go and do $ ls /dev/sd* | xargs -L 1 umount14:49
xnoxArgh "Apr 22 14:35:47 ubuntu rsyslogd-2177: imuxsock begins to drop messages from pid 4575 due to rate-limiting"14:51
xnoxcjwatson: have you considered making the limits basically unlimited for partman/ubiquity, just in case one is trying to debug it =)14:52
dankhmm.  How would I disable rate limiting for this run?14:55
dankmaybe I'll just redirect stderr.14:56
xnox$SystemLogRateLimitInterval 014:56
xnox$SystemLogRateLimitBurst 014:56
xnoxand then restart it.14:57
dankI've swapped out the two files, marked them executable, turned off rate limiting.  Here we go...14:59
danksent.  Included output of ps augxw and lsof in files in /tmp15:08
dankand now I'm turning into a pumpkin.  I can run more tests for you tomorrow.15:08
xnoxdank: did it still hang with my swapped in files?15:08
xnoxdank: thanks a lot for your help!15:08
xnoxwell i guess i should see that in the logs...15:09
xnoxit did, sigh.15:09
cjwatsonxnox: It might be a plan to unlimit logging in casper, perhaps15:39
cjwatsonxnox: Are you going to be in the London office tomorrow?15:40
xnoxcjwatson: yeah =)15:40
cjwatsonExcellent, if all else fails we can pair-debug it there15:40
plarsdank: still around?20:39
slangasekxnox: hi, so I see from logs that dank was able to send you some interesting debug output20:49
slangasekxnox: does this get us closer to having a reproducer?  plars is trying to reproduce the bug, so maybe if the debug output is interesting you could share it with him20:50
plarsyes, would love to find a way to reliably reproduce this20:51
xnoxplars: there wasn't muh useful in it. my current hypothesis is that when ext4 needs to recover journal or otherwise have rw access to the hard drive. replace/reuse will hang. My other idea is to axe reuse/replace and simply use os-prober instead of that code.21:12
xnoxsimilarly dirty / hybernated / not-cleanly mounted ntfs partitions could cause the same. But maybe not as much after the top crasher got fixed for it (bug 1019806)21:14
ubot2Launchpad bug 1019806 in ntfs-3g (Ubuntu) "ntfs-3g crashed with SIGABRT in get_node()" [Medium,Confirmed] https://launchpad.net/bugs/101980621:14
xnoxI will run these past cjwatson tomorrow, and see what he thinks.21:15
xnoxI think my other hdd had that once where I had to wipe it clean it was: intel raid metadata -> full-disk lvm -> lvm volumes used for VMs and thus having -> (full disk ubuntu install, nested lvm installs, etc) But (a) I wouldn't want to reproduce such a setup (b) not sure we want to support anything like that.21:16
xnoxdank reported that there are no errors from os-prober, it just works fine.21:17
slangasekxnox: well, the hope is that plars would be able to get you a reproducer for this bug before tomorrow21:17
xnoxslangasek: my next step is to remaster a ubiquity cd which defaults to full debugging and set -x partman and disable rate-limitting, such that hopefully lower the "debugging" skills of those who hit this bug.21:19
GrueMasterxnox: Wouldn't it be quicker/easier to modify the files in an already created cd image than remaster it?21:42
xnoxGrueMaster: sure. are you affected by 1080701 ?21:43
xnoxbug 108070121:43
ubot2Launchpad bug 1080701 in ubiquity (Ubuntu Raring) "After 'Preparing to install Ubuntu' screen, raring installation hangs" [Critical,Confirmed] https://launchpad.net/bugs/108070121:43
xnoxso far I can't think of anything more than https://wiki.ubuntu.com/DebuggingUbiquity#Deeper_debugging_of_partman to help us find the problem21:44
GrueMasterNo, I stopped testing the live images after 12.04.21:44
xnoxI understand.21:44
GrueMasterBut I used to do this almost weekly when I was doing Arm QA.21:45
GrueMasterAnd before you say that is different, I used to verify issues I found on x86/amd64.21:45
GrueMasterUbiquity is one of the ugliest programs I have seen to try to debug.  the kernel parameters for adding debug output are largely ignored from what I have seen.21:53

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!