[10:11] <xivulon> cjwatson, evand, please see https://bugs.edge.launchpad.net/wubi/+bug/243105
[10:12] <xivulon> I made a tiny progress
[10:12] <xivulon> basically the problem is in apt_pkg -> Acquire -> Run
[10:13] <xivulon> which explain the Type Int error in Pulse (it is the pulse inside acquire.cc not in install.py...)
[10:14] <xivulon> I still do not know why bindmounts create issues here though
[10:14] <xivulon> any hint welcome
[10:27] <xivulon> hmm could the issue be in cdrom.py mountpoint handling? I cannot test anymore
[11:02] <holst> hmm
[11:03] <holst> is there some way to format if no file system exist otherwise just mount, in parted?
[11:09] <holst> http://rafb.net/p/6KigAK96.html
[11:10] <holst> do you see any problem with my /NOBACKUP partition?
[11:15] <holst> anyone? :)
[11:15] <holst> it reformats my disk even if i have method{ keep }
[11:20] <cjwatson> holst: did you see the comments I addressed to you yesterday evening?
[11:22] <holst> well, I need to format /
[11:23] <holst> hmm, reading up on what you said
[11:23] <holst> gimme a sec
[11:24] <holst> hmm
[11:24] <holst> can you run a script that, before the installation starts, runs some formatting?
[11:25] <cjwatson> yes, unfortunately it's a bit complicated due to bug 239348
[11:26] <holst> ohh, how bitter, I really wanted to solve this
[11:26] <cjwatson> but you can do it by using preseed/early_command to write out a script in /lib/partman/display.d/00local and make it executable
[11:26] <cjwatson> then that will be run at the start of the partitioner
[11:26] <cjwatson> holst: I'm still not clear on whether your existing /NOBACKUP is a separate partition or just an ordinary directory on /
[11:27] <holst> I solved it!
[11:27] <holst> I have two CDs
[11:27] <holst> one is labeled: FORMAT
[11:27] <holst> one is labeled: KEEP
[11:27] <holst> with the obvious meanings for /NOBACKUP :)
[11:27] <holst> what do you think?
[11:27] <cjwatson> I'm not sure I understand
[11:27] <holst> well, one cd formats /NOBACKUP
[11:28] <holst> and if you know you have a /NOBACKUP,
[11:28] <holst> then you install with the other CD
[11:28] <holst> the one labeled KEEP
[11:28] <holst> which just mounts /NOBACKUP
[11:28] <cjwatson> it's still tricky to deal with existing partitions in the current automatic partitioner
[11:29] <cjwatson> 21:22 <cjwatson> holst: however, unfortunately, it's really hard to do this automatically at the moment, because the automatic partitioner doesn't have any support for selecting an existing partition
[11:29] <cjwatson> 21:22 <cjwatson> this is something I'm hoping to fix - but at the moment all it can do is create new partitions
[11:29] <cjwatson> 21:23 <cjwatson> if /NOBACKUP is a separate partition, then I've seen ways to do this by deleting all the other partitions you *don't* want to keep before the partitioner runs, then telling the automatic
[11:29] <cjwatson>                  partitioner to operate on the free space, then mounting it at the end - but they're all a bit hackish
[11:29] <cjwatson> as I said yesterday
[11:31] <xivulon> cjwatson how is the cdrom repository handled by apt during installation? is /proc/mounts involved in the process? could bindmounts in /cdrom create problems?
[11:32] <cjwatson> it just calls mount as far as I know
[11:33] <cjwatson> though I thought we told it not to do that, generally
[11:33] <cjwatson> please see the configure_apt method in ubiquity/scripts/install.py
[11:34] <cjwatson> what I mean is, we configure apt so that it shouldn't muck about with mounting/unmounting /cdrom itself, although ubiquity bind-mounts /cdrom into /target/cdrom
[11:35] <xivulon> it would still need a mountpoint though, could it be that the mountpoint is retrieved from the device?
[11:35] <xivulon> so that you end up with 2 mountpoints
[11:35] <cjwatson> I'm not sure that question makes sense ...
[11:36] <cjwatson> apt's cdrom mountpoint is part of apt's configuration, /cdrom by default
[11:36] <xivulon> hm assuming the cd is mounted, apt would need to know the mountpoint I assume
[11:36] <cjwatson> it isn't retrieved from the device (that doesn't make sense)
[11:36] <cjwatson> apt-pkg/cdrom.cc:199:   string cdromPath = _config->FindDir("Acquire::cdrom::mount","/cdrom/");
[11:36] <cjwatson> apt-pkg/cdrom.cc:523:   string CDROM = _config->FindDir("Acquire::cdrom::mount","/cdrom/");
[11:36] <cjwatson> apt-pkg/cdrom.cc:583:   string CDROM = _config->FindDir("Acquire::cdrom::mount","/cdrom/");
[11:36] <cjwatson> grep is your friend ;-)
[11:38] <xivulon> I know, but cannot access the source now, I assume that "/cdrom" is a fallback for "Acquire::cdrom::mount"
[11:40] <xivulon> and FindDir("Acquire::cdrom::mount") does not parse /proc/mounts by any chance (in case the cdrom is stored as "device")
[11:40] <cjwatson> no it does not
[11:41] <cjwatson> if you don't mind me saying so I think you may be a little bit obsessed with /proc/mounts format :-) in most cases it is not an issue
[11:42] <cjwatson> practically everything either just accesses it as an ordinary directory or uses mount/umount
[11:42] <xivulon> re 243105 so far we know that the issue is due to cdrom being bindmounted and the trace ends at apt_pkg.PkgAcquireRun
[11:43] <cjwatson> no, you know that it happens when cdrom is bind-mounted
[11:43] <cjwatson> that's not quite the same thing as "due to"
[11:43] <xivulon> true
[11:44] <cjwatson> I think strace might be more useful than a python stack trace
[11:45] <xivulon> yes but that is where I ended yesterday night, will resume tonight with strace
[11:46] <cjwatson> (strace -f, mind, there will be a lot of processes involved)
[11:48] <cjwatson> in particular I'd be very interested if something in apt were calling umount
[11:48] <cjwatson> but you should also be able to see it statting files under /cdrom
[11:49] <cjwatson> oh, hmm!
[11:49] <cjwatson>    and the '..' file in the mount point and see if they are on the same device.
[11:49] <cjwatson>    By definition if they are the same then it is not mounted. This should
[11:49] <cjwatson>    account for symlinked mount points as well. */
[11:49] <cjwatson> drat, sorry
[11:49] <cjwatson>  // IsMounted - Returns true if the mount point is mounted              /*{{{*/
[11:49] <cjwatson>  // ---------------------------------------------------------------------
[11:49] <cjwatson>  /* This is a simple algorithm that should always work, we stat the mount point
[11:49] <cjwatson>     and the '..' file in the mount point and see if they are on the same device.
[11:49] <cjwatson>     By definition if they are the same then it is not mounted. This should
[11:49] <cjwatson>     account for symlinked mount points as well. */
[11:49] <cjwatson> if /cdrom is bind-mounted from the same device then that algorithm would fail
[11:50] <xivulon> ahh
[11:50] <xivulon> great catch
[11:50] <cjwatson> though I don't see why a symlink to something on the same device would be any better
[11:51] <cjwatson> so, while this looks like a plausible guess, it could do with some verification
[11:51] <cjwatson> furthermore, shouldn't the CD content still be on a different device (albeit loop-mounted) from /target?
[11:53] <xivulon> cjwatson, would you mind pasting the relevant chunk of code, otherwise I am a bit blind
[11:53] <cjwatson> I don't have a relevant chunk of code
[11:53] <cjwatson> the IsMounted code doesn't matter, it's adequately described by the comment above
[11:54] <cjwatson> it stats /cdrom/ and /cdrom/../ and compares the st_dev fields
[11:54] <xivulon> I see
[11:54] <cjwatson> oh, hmm now, apt is running in the live CD root filesystem not in /target
[11:55] <cjwatson> maybe it just needs to be configured to use /target/cdrom
[12:03] <cjwatson> you might try something like http://paste.ubuntu.com/23063/
[12:04] <cjwatson> I'm not entirely sure that will work but it makes a twisted kind of sense
[12:04] <xivulon> ah thanks, will try tonight, I am sure evenad will be able to do some testing earlier on :)
[12:04] <xivulon> a shame davmor2 is not around
[12:11] <xivulon> I have updated the bug report with the ^
[12:55] <CIA-48> hw-detect: cjwatson * r82 ubuntu/ (6 files in 3 dirs): merge from Debian 1.63
[12:57] <CIA-48> hw-detect: cjwatson * r83 ubuntu/debian/changelog: releasing version 1.63ubuntu1
[13:15] <xivulon> cjwatson re 226622, what is the correct behaviour of panic messages? it seems like they are not displayed unless you are in recovery mode
[13:15] <xivulon> is this by design?
[13:15] <cjwatson> TheMuso: ^--
[13:16] <xivulon> I would expect that whenever you are thrown in busybox you should see the error message, whether you booted with usplash or quiet
[13:16] <cjwatson> (I don't know the answer, but I think TheMuso dug into that last)
[13:16]  * TheMuso reads scrollback.
[13:17] <xivulon> TheMuso, basically 226622 patch simply adds a "Meaningful message" to panic
[13:17] <xivulon> but that is not always displayed
[13:17] <TheMuso> xivulon: hang on, let me have a look at the bug also.
[13:20] <TheMuso> You shouldn't need to boot to recovery mode just to get the panic message. Let me look at the hardy initramfs-tools code where the dirty flag message is displayed. I saw it during the intrepid merge, but I'll look at the hardy package to be sure.
[13:22] <TheMuso> The code looks ok, the only thing I can think of is that mount is not returning the correct status all the time. I currently don't have a windows/wubi setup to test with right away, but I'll set one up tomorrow to see if I can reproduce the issue.
[13:23] <xivulon> TheMuso, if you run panic "XXX" when booting with quiet splash, do you see "XXX" once you hit the busybox?
[13:23] <xivulon> I mean in a normal setup
[13:23] <TheMuso> xivulon: Yes. The panic code kills usplash to display text on the console.
[13:24] <xivulon> could splash have some "screen artifacts" that hide the text somehow?
[13:24] <xivulon> like the screen taking some time to reset
[13:24] <TheMuso> I don't know. All the code does is if it finds usplash_write, it tells it to shut down usplash.
[13:25] <xivulon> Yep I noticed that, could you test with a panic message in a normal setup booting with quite splash?
[13:26] <TheMuso> The only way I know to trigger a panic message is to make sure the root device cannot be found, and I don't have full installs on other boxes to test with currently without rebooting this one.
[13:27] <xivulon> ah you can just add "panic mymessage" somewhere in the initrd
[13:27] <TheMuso> Yes, but I have a better idea of triggering one. Forcing initramfs to think the root device is something it is not.
[13:27] <TheMuso> anyway brb will test
[13:27] <xivulon> much appreciated
[13:36] <TheMuso> Ok. Its nothing to do with the code at all, well not really. It has to do with the TTY usplash is running on, and the TTY that the error message gets displayed on. What happens is that usplash is on TTY8, and when it gets shut down, the active console is still TTY8.
[13:36] <TheMuso> However, when text is echoed, its echoed to TTY1, however busybox still gets launched on tty8 because thats where usplash was.
[13:36] <TheMuso> SO the reason why its seen without usplash is because the tty doesn't get changed.
[13:37] <TheMuso> So we either echo the text to tty8 if usplash is running, (harder to do), or we switch to tty1 when usplash quits.
[13:39] <TheMuso> ...which may be harder, as the initramfs doesn't have chvt...
[13:45] <xivulon> TheMuso not sure if we can push that in, but it possible it would a lot :)
[13:46] <TheMuso> xivulon: There may be another way I haven't yet thought of.
[13:46] <xivulon> shall I open a bug against initramfs-tools?
[13:47] <TheMuso> Yes please do.
[13:51] <xivulon> #243226
[13:52] <TheMuso> Thanks./
[13:52] <xivulon> ah thank you
[13:53] <xivulon> do you think we can have that in hardy point release?
[13:53] <TheMuso> 8.04.1 is highly unlikely now.
[13:53] <xivulon> guess not :(
[13:53] <xivulon> well it would be almost as good if users could get that via normal updates
[13:53] <TheMuso> yep.
[13:54] <xivulon> in most cases that manifest after in post-installation
[14:21] <xivulon> evand ping
[14:22] <evand> xivulon: pong
[14:24] <xivulon> did you see ^
[14:24] <xivulon> evand do you think we can test the cjwatson patch? looks promising
[14:25] <xivulon> we also need ferification for 136682
[14:27] <evand> I'll give it a shot in a second.
[14:27] <evand> re 136682> I'll give it a shot after we resolve the python-apt mess.
[14:33] <xivulon> evand does the new iso contain wubi selfextract?
[14:33] <xivulon> rev 504
[14:34] <evand> I don't see a new ISO
[14:34] <evand> link?
[14:41] <xivulon> ah didn't even notice...
[14:41] <xivulon> I also see 20.1 :(
[14:42] <xivulon> anyway, new ISOs will contain rev 504 self-extact correct?
[14:49] <evand> correct
[15:31] <giosue_c> I'm having trouble with cdimage/debian-cd and I was hoping someone had a few minutes to help me troubleshoot...
[15:32] <cjwatson> I'm on the phone, but say what your problem is and stick around
[15:32] <giosue_c> ok. thanks.
[15:32] <cjwatson> IIRC you've asked a question several times but left before I could answer ...
[15:32] <giosue_c> sorry about that.  I think our time zones are quite different :)
[15:33] <giosue_c> So I've been working on getting a smaller local repository that will work with cdimage, and i think i'm getting close.  but now debootstrap is complaining about the following missing packages:
[15:34] <giosue_c> Missing debootstrap-required libtext-charwidth-perl
[15:34] <giosue_c> Missing debootstrap-required libtext-iconv-perl
[15:34] <giosue_c> Missing debootstrap-required libtext-wrapi18n-perl
[15:34] <giosue_c> Missing debootstrap-required bsdmainutils
[15:34] <giosue_c> Missing debootstrap-required debconf-i18n
[15:34] <giosue_c> Missing debootstrap-required liblocale-gettext-perl
[15:34] <giosue_c> these packages are all in my repository and show up in the Packages lists... but obviously i've got something wrong.
[15:49] <cjwatson> IIRC (I'll check in more detail once I'm off the phone, but maybe this will help) that usually indicates that the Priority fields of those packages are wrong
[15:49] <cjwatson> packages installed by debootstrap are always 'required' or 'important' and it does matter
[15:54] <giosue_c> ok.  will look into that.
[15:59] <giosue_c> Looked at each entry in the Packages file and it looks like they are all 'required' or 'important'.
[16:00] <giosue_c> I was just looking at the main/binary-i386 Packages file.  does debootstrap care about any of the other ones?
[17:24] <evand> http://evalicious.com/tmp/strace-20080626-1.txt
[17:24] <evand> read(11, "403 Media Failure\nMedia: Ubuntu "..., 64000) = 97
[17:25] <evand> write(2, "result could not be parsed", 26) = 26
[17:30] <xivulon> O_o ETA 55m
[17:31] <evand> it's nearly 200MB
[17:31] <xivulon> noticed it
[17:57] <cjwatson> evand: might want -s 1024 there
[17:57] <cjwatson> otherwise it tends to get truncated
[18:02] <evand> cjwatson: noted.  I'm assuming the python process running scripts/install.py was the right thing to attach to, correct?
[18:04] <cjwatson> yeah
[18:04] <cjwatson> giosue_c: those should be enough
[18:06] <giosue_c> ok.  my repository must be missing something...
[18:12] <giosue_c> i wish my repo were publicly visible so I could let you have a quick look at it... :(
[18:24] <giosue_c> cjwatson: if the tasks aren't specified properly (meaning not all overrides are in indices) will that cause problems for debootstrap?
[18:24] <cjwatson> shouldn't
[18:24] <cjwatson> only priority overrides matter
[18:37] <giosue_c> cjwatson: in creating a fairly stripped down local repo I noticed that something in cdimage was complaining if I didn't have $DIST-proposed $DIST-security $DIST-updates in my repo.  However I don't think I need anything more than $DIST to create a CD.  Am I right?
[18:40] <cjwatson> should be fine ...
[18:45] <CIA-48> apt-setup: cjwatson * r137 apt-setup/ (apt-setup-verify debian/changelog):
[18:45] <CIA-48> apt-setup: Initialise local variable 'file' so that apt-setup-verify doesn't get
[18:45] <CIA-48> apt-setup: excruciatingly confused when you pass file=/path on the kernel command
[18:45] <CIA-48> apt-setup: line.
[18:46] <CIA-48> apt-setup: cjwatson * r138 apt-setup/debian/changelog: releasing version 1:0.37ubuntu2
[21:03] <giosue_c> I've a general question about creating your own repository.  how does apt-ftparchive know which debs to list in the packages file under main, restricted, etc...  the control file of a package doesn't say anything about where packages belong in the repo.
[21:04] <giosue_c> Perhaps you use the file list option instead of letting it grind through your entire pool?
[21:05] <giosue_c> that would be a bummer because you'd have to run it once for every dist and component in your repo.
[22:07] <cjwatson> giosue_c: we use the file list option for the main Ubuntu archive, and record the information in the override file (see indices/ on any mirror); it's possible for the same version of the same package to be in different components in e.g. hardy and intrepid
[22:08] <cjwatson> for example it's not uncommon for something to be in universe in hardy and then in main in intrepid, because something in main starts depending on it
[22:08] <giosue_c> makes sense.
[22:10] <giosue_c> for the local repo i am trying to create i am generating the Packages and Release files myself and I realized apt-ftparchive wasn't doing the right thing just by scanning the pool.  seems like i need to pull the override files to try to recreate the file lists.
[22:31] <giosue_c> cjwatson: is there a place i can see the scripts and config files used to build and maintain the main Ubuntu archive?
[22:31] <cjwatson> I'm afraid that's part of Launchpad, whose source code is not published
[22:32] <cjwatson> you can look at dak, which does the same kind of job for Debian
[22:32] <giosue_c> dak.  ok thanks.
[22:33] <giosue_c> are you a volunteer contributor to ubuntu or do you work for canonical?
[22:33] <cjwatson> I work for Canonical
[22:33] <giosue_c> yea.  i thought that you were pulling a pretty heavy load for a volunteer.
[22:43] <giosue_c> cjwatson: the nonprofit i'm working for is build a distro off xubuntu.  my boss was just down in capetown with some folks from the shuttleworth foundation working on a project to deliver phone service to some townships in S. Africa.
[22:44] <giosue_c> cjwatson: cdimage has been awesome for getting our own distro off the ground.
[22:45] <cjwatson> glad to hear it
[22:45] <cjwatson> have to go to bed now, though :)
[22:46] <giosue_c> g-nite.  see u round.
[23:00] <xivulon> what package is on the CD which is not already installed in the live desktop?
[23:01] <xivulon> this is for testing whether installing something from a bindmounted /cdrom can reproduce the issues without going through the full installation
[23:02] <xivulon> well never mind I can find out easily
[23:13] <xivulon> I tried to apt-get install "patch" from a bindmounted /cdrom dir containing iso content and it fails
[23:13] <xivulon> will try python-apt next