[05:42] <twb> So I'm trying to install via netboot.  It doesn't ask me to set an http proxy (unlike Debian), and despite being told to use the internal mirror (an apt-cacher daemon), it's clear from the debugging log that it's trying to fetch .../Release directly from archive.ubuntu.com -- which doesn't work in my environment because http traffic that doesn't go through the proxy is dropped.
[05:50] <twb> Where .../Release is http://archive.ubuntu.com/ubuntu/dists/gutsy/Release
[10:46] <twb> What generates these files? http://archive.ubuntu.com/ubuntu/dists/gutsy/main/installer-i386/current/images/netboot/ubuntu-installer/i386/{linux,initrd.gz}
[10:53] <cjwatson> superm1: I don't think your r2181 is correct. apt-setup should take care of that already.
[10:54] <cjwatson> twb: part of the debian-installer source package build process
[10:54] <cjwatson> twb: we have a session scheduled for the next developer summit to fix a number of issues with networkless installation, which also covers this case
[10:54] <cjwatson> tepsipakki: erm, I forget whether this was one of the things that Anton already nacked ...
[10:57] <twb> cjwatson: well, I need to fix this bug NOW; before I start ripping apart the cpio, it occurred to me to investigate if there were already existing abstractions that could make the process less tedious.
[10:58] <cjwatson> twb: choose-mirror is hardcoded to go to archive.ubuntu.com to find out whether it needs to use a proxy, but that has a very short timeout
[10:59] <cjwatson> twb: can you post your preseed file, minus any sensitive information like passwords?
[10:59] <twb> I'm not using any preseed file yet
[10:59] <cjwatson> err
[10:59] <cjwatson> so how are you telling it to use the internal mirror?
[10:59] <twb> The problem is that here, any non-proxy web traffic is DROPPED
[10:59] <twb> Not rejected
[10:59] <cjwatson> I heard you
[10:59] <twb> So it's taking at least five minutes to time out before asking for a proxy
[10:59] <twb> By which time I give up
[11:00] <twb> I'm telling it to use the internal mirror by hand, when it asks which mirror to use
[11:01] <twb> I apologize if I'm being silly and thus sounding brash.
[11:02] <cjwatson> you can work around it using apt-setup/use_mirror=false and installing entirely from the CD
[11:02] <twb> I don't have a CD.
[11:02] <cjwatson> unfortunately that does mean you'll either have to set up sources.list to ... oh
[11:02] <tepsipakki> cjwatson: ah, so in that case debian won't be using it? I'll revert the commit then :)
[11:03] <cjwatson> tepsipakki: I didn't say that, I said I couldn't remember
[11:03] <cjwatson> tepsipakki: please don't revert it, I think Debian *should* be using it
[11:03] <tepsipakki> cjwatson: ok, I'll discuss it with gravity on FOSSCamp
[11:04] <cjwatson> tepsipakki: please grab me too :)
[11:04] <tepsipakki> maybe xsf could help getting this in d-i/console-setup
[11:04] <tepsipakki> cjwatson: sure :)
[11:04] <cjwatson> tepsipakki: oh, I have commit access, that's not a problem, I just need to refamiliarise myself with where Debian is in terms of console
[11:05] <cjwatson> twb: hmm, the case where you have a CD is the only one I know of where archive.ubuntu.com is hardcoded
[11:05] <cjwatson> twb: could I see your logs, please?
[11:05] <twb> Er, which log?
[11:05] <cjwatson> twb: syslog
[11:05] <tepsipakki> cjwatson: yep, but getting it acked would be easier if, say, xsf wanted to use it too :)
[11:05] <twb> Hmm... how can I get that off the system?  The initrd doesn't contain ssh or scp
[11:07] <cjwatson> twb: nc
[11:07] <cjwatson> (and you can use 'anna-install openssh-client-udeb' to get scp, but only if it can talk to a mirror to retrieve it ...)
[11:08] <cjwatson> twb: or use the "save debug logs" item on the main menu
[11:08] <twb> menu?
[11:08] <cjwatson> back up ...
[11:08] <twb> OK, that menu
[11:08] <twb> The one you get when you click "Go Back"
[11:09] <cjwatson> correct
[11:14] <twb> Two more data: if I have the server tell (via DNS) the netboot client that "archive.ubuntu.com" --> 127.0.2.1, then it fails straightaway and immediately asks about the proxy; secondly, this issue occurs with both feisty and gutsy.
[11:18] <cjwatson> yes, it would
[11:19] <cjwatson> any luck on the logs?
[11:20] <twb> ftp://twb.ath.cx/tmp/syslog
[11:22] <cjwatson> hmm, of course, I misread that code path
[11:26] <cjwatson> bah, that code is so broken. There's really not much of a way around this in the existing code, short of something gross like replacing wget with a wrapper that just exits immediately if it's asked for something starting with http://archive.ubuntu.com/
[11:27] <cjwatson> this code was one of the more desperate approaches to reducing the number of questions asked by d-i in the warty era. Now that we have ubiquity I'm very tempted to just rip it out.
[11:27] <twb> Please do.
[11:27] <twb> AFAICT it's simply wrong, and only ever worked by accident.
[11:27] <cjwatson> well, it makes a valiant attempt to avoid timeouts
[11:27] <cjwatson> but obviously not enough
[11:28] <cjwatson> it's not wrong *as such*, it's just very slow in some circumstances; in your case, I would be inclined to just preseed everything and not worry about the timeout because it could happen on all my machines in parallel and I could spend it having a cup of coffee
[11:29] <twb> It makes testing the preseed a horrible pain
[11:29] <cjwatson> that is not an attempt to argue that it isn't a bug, just that it shouldn't be a showstopper :)
[11:29] <twb> Shall I tell you the other problem?
[11:29] <cjwatson> sure
[11:29] <twb> 192.168.67.0/24 is a network just between the netboot server and the clients
[11:31] <twb> If I tell the client to use the proxy, which runs outside that NATted environment, then the proxy can't find the apt-cacher on 192.168.67.1 (for obvious reasons)
[11:31] <twb> But if I tell it NOT to use the proxy, er, something bad happens, I can't remember what.  Lemme check
[11:32] <cjwatson> it doesn't sound like you should be telling it to use a proxy
[11:33] <cjwatson> while apt-cacher is a proxy in some sense it's acting more like a mirror here
[11:33] <twb> It's purpose is basically to reduce the download time because it's only going over the 100Mbit crossover, not the public internet
[11:34] <cjwatson> I understand, but that doesn't make it a proxy as far as d-i is concerned :)
[11:35] <twb> Oh no no no
[11:35] <twb> I'm telling it to use the squid proxy
[11:35] <cjwatson> it's a proxy if the requests are named for some host outside your network, but actually being sent to an internal host
[11:35] <cjwatson> doesn't sound like you can do both that and use the apt-cacher
[11:38] <twb> Right
[11:38] <twb> Essentially I want to set no_proxy=192.168.67.1
[11:38] <cjwatson> but the proxy is *only* used to talk to the mirror
[11:39] <cjwatson> it's not used for any other purpose
[11:39] <twb> It's not used for security repo?
[11:39] <cjwatson> oh, yes, it is. you don't mirror that then?
[11:39] <twb> I could, but I can't TELL it to use my security mirror, AFAICT
[11:39] <cjwatson> you can, but only by preseeding
[11:39] <twb> Ah, OK.\
[11:39] <cjwatson> apt-setup/security_host
[11:39] <twb> Where do you learn what these preseed keys are?
[11:40] <twb> For example, all the Debian documentation still talks about console-data/keymap instead of console-data-at/keymap
[11:40] <cjwatson> they're in the installation guide
[11:40] <cjwatson> https://help.ubuntu.com/7.04/installation-guide/i386/appendix-preseed.html
[11:40] <twb> Thanks.
[11:41] <cjwatson> I don't think there's a published version for 7.10 yet, but the differences are small
[11:41] <twb> It looks like a fork of the Debian Install Guide
[11:41] <cjwatson> it is
[11:42] <cjwatson> but with details adjusted, for example the one you pointed out above (except that it's not console-data-at/keymap either)
[11:42] <twb> Sorry, I may mis-remember
[11:42] <cjwatson> you'll want the console-setup/* questions for that
[11:44] <twb> Is that guide available as a PDF?
[11:48] <twb> html2ps doesn't appear able to cope with the https protocol.
[11:48] <cjwatson> yes, in the installation-guide-$ARCH package (e.g. installation-guide-i386)
[11:49] <twb> Thanks.
[22:42] <superm1> cjwatson, that's why i have it in our branch only for now.  It didn't seem sensible to me either that it was acting that way.  I wasn't sure if apt-setup was missing something.  Apparently it is, and mirrors aren't properly setup either.
[22:42] <superm1> cjwatson, but in an effort to get our disk out i put that in