| === fginther|afk is now known as fginther | ||
| Riddell | how come this doesn't work? http://paste.kde.org/760544/ | 09:27 |
|---|---|---|
| Riddell | grantlee slipped to universe | 09:28 |
| === doko_ is now known as doko | ||
| wgrant | Riddell: Without options it will override the named binary. Use -S to override a source and its binaries. | 09:57 |
| Riddell | thanks wgrant | 10:51 |
| === cking_ is now known as cking | ||
| === hggdh_ is now known as hggdh | ||
| bdmurray | Could somebody have a look at https://code.launchpad.net/~brian-murray/ubuntu-archive-tools/phased-update-percentage/+merge/167871 and then use it to set the phased-update-percentage for checkbox in raring-updates to 0? | 14:44 |
| === mmrazik is now known as mmrazik|afk | ||
| bdmurray | cjwatson: ^ | 15:03 |
| cjwatson | bdmurray: Merged - just checkbox or the other binaries too? | 15:05 |
| bdmurray | cjwatson: the other binaries too please | 15:06 |
| cjwatson | also fixed your code to handle -z 0 :) | 15:06 |
| plars | something happened between yesterday and today on the server images. I'm not sure what yet, but today's images are no good | 15:11 |
| bdmurray | cjwatson: hrm, thanks | 15:11 |
| plars | still debugging, but I cannot seem to get todays image to install | 15:11 |
| plars | cjwatson: I think publishing to current is still the default correct? I think it would be good to push yesterdays image back as current if that's the case | 15:12 |
| cjwatson | plars: sec, you're in the queue | 15:12 |
| infinity | plars: Is it the usual kernel version mismatch confusion? They may have built just as d-i migrated. | 15:13 |
| plars | infinity: could be, kernel is the same as yesterday's image | 15:14 |
| cjwatson | bdmurray: ok, fixed up further so that 'change-override -s raring-updates -S -z 0 checkbox' works for this | 15:15 |
| plars | infinity: is there an easy place to check the version? | 15:15 |
| cjwatson | bdmurray: and overridden | 15:15 |
| infinity | plars: I was hoping for some sort of error message from syslog. | 15:15 |
| cjwatson | bdmurray: https://launchpad.net/ubuntu/raring/i386/checkbox | 15:15 |
| plars | infinity: waiting on jenkins to come back up, so I'm looking at it manually right now | 15:16 |
| infinity | plars: Anyhow, if it's booting with 3.9.0-4, it'll be mismatched, cause the ISO has 3.9.0-3 on it. | 15:16 |
| cjwatson | plars: punted current back to 20130606 for now | 15:16 |
| plars | infinity: first thing is that network seems to not be detected | 15:16 |
| bdmurray | cjwatson: how do you get to that url from https://launchpad.net/ubuntu/+source/checkbox? | 15:16 |
| cjwatson | plars: that's a classic result of kernel mismatch | 15:16 |
| plars | though I think it got past that point on the preseed install | 15:16 |
| cjwatson | bdmurray: god, I can never remember, I memorised the URL pattern :) | 15:16 |
| bdmurray | cjwatson: okay, I'll work on that then | 15:17 |
| cjwatson | the binary package publishing history pages aren't well-linked | 15:17 |
| cjwatson | I think you can get at them using a search from /ubuntu/raring | 15:17 |
| plars | then right after partman it fails | 15:17 |
| plars | mount: mounting /dev/loop0 on /mnt failed: No such device | 15:18 |
| plars | we are also fighting jenkins issues today, so I'm doing this by hand at the moment | 15:18 |
| cjwatson | so - https://launchpad.net/ubuntu/raring -> "Architectures and builds for Raring" i386 -> search for checkbox and if it ever doesn't time out you should get your answer | 15:18 |
| cjwatson | plars: no point continuing past the first failure anyway | 15:18 |
| cjwatson | We have a current d-i now so I'll just rebuild | 15:19 |
| plars | infinity: kernel version is 3.9.0-4 | 15:19 |
| infinity | plars: Right. mismatch it is. | 15:19 |
| plars | infinity: sorry | 15:19 |
| plars | infinity: it's -3, I typod | 15:19 |
| cjwatson | Rebuilding now | 15:20 |
| infinity | ... | 15:20 |
| plars | infinity: I was asking if there was an easy way to check the d-i version to see if it had gone up before the kernel for some raeson | 15:20 |
| infinity | I'm all for not caring until it's rebuilt. :) | 15:20 |
| plars | ok | 15:20 |
| cjwatson | the way to check is to run uname -a | 15:20 |
| plars | cjwatson: for kernel, yes, and it's 3.9.0-3, was asking about d-i | 15:21 |
| cjwatson | who cares | 15:21 |
| cjwatson | what matters if the kernel embedded in d-i | 15:21 |
| plars | ok, will wait for respin | 15:21 |
| cjwatson | and uname -a tells you that | 15:21 |
| cjwatson | *is the kernel | 15:21 |
| * cjwatson grabs an image in parallel to check | 15:22 | |
| cjwatson | plars,infinity: The problem was that due to unfortunate timing of seed changes vs. the image build only a few module udebs were present. | 15:47 |
| cjwatson | A rebuild should clear it up. | 15:47 |
| cjwatson | (Basically, those module udebs that were pulled in by dependencies of various random things.) | 15:47 |
| bdmurray | cjwatson: what will change on the archive now that the p-u-p is set to 0 for checkbox? | 16:02 |
| cjwatson | A Phased-Update-Percentage: 0 control field in the Packages files for those packages | 16:02 |
| bdmurray | cjwatson: so the Packages file here http://archive.ubuntu.com/ubuntu/dists/raring-updates/main/binary-amd64/ and look for checkbox? | 16:07 |
| cjwatson | Yep | 16:09 |
| bdmurray | okay, I'm not seeing it... should I be more patient? | 16:10 |
| cjwatson | Your mirror's probably out of date | 16:12 |
| cjwatson | It's there on the master | 16:12 |
| bdmurray | okay, great | 16:13 |
| cjwatson | (To be fair, I think the master only pulsed about eight minutes ago) | 16:13 |
| bdmurray | cjwatson: okay, I've tested update-manager if you want to set to 100 now | 16:19 |
| cjwatson | bdmurray: OK, done. You have until the next publisher run to keep testing :) | 16:21 |
| bdmurray | cjwatson: thanks for the help | 16:22 |
| === plars is now known as plars-afk | ||
| === plars-afk is now known as plars | ||
| plars | cjwatson, infinity: respin still fails it seems, looking | 18:24 |
| plars | Jun 7 16:49:28 main-menu[1606]: (process:13288): FATAL: Module squashfs not found. | 18:25 |
| plars | Jun 7 16:49:28 main-menu[1606]: (process:13288): libkmod: kmod_lookup_alias_from_builtin_file: | 18:25 |
| plars | Jun 7 16:49:28 main-menu[1606]: (process:13288): could not open builtin file '/lib/modules/3.9.0-3-generic/modules.builtin.bin' | 18:25 |
| plars | Jun 7 16:49:28 main-menu[1606]: (process:13288): mount: mounting /dev/loop0 on /mnt failed: No such device | 18:25 |
| plars | Jun 7 16:49:28 main-menu[1606]: WARNING **: Configuring 'live-installer' failed with error code 1 | 18:25 |
| plars | still d-i/kernel mismatch it seems? | 18:26 |
| infinity | Why would it have 3.9.0-3? | 18:32 |
| infinity | Methinks something is very confused. | 18:34 |
| infinity | plars: Hrm, it seems the mirror on cdimage is failing to update. I'll poke in a sec. | 18:44 |
| plars | thanks infinity | 18:44 |
| plars | infinity: ping me if you start another respin and I'll make sure it gets priority for testing | 18:44 |
| ScottK | libc++, there's a name that won't cause any confusion ... | 19:01 |
| infinity | ScottK: It's llvm's stdc++ ... And yeah, not confusing AT ALL. | 19:03 |
| infinity | plars: 20130607.2 should be more pleasant. | 19:38 |
| plars | thanks infinity | 19:38 |
| infinity | cjwatson: So, looks like the kernel skew issues were because the mirror wasn't being updated pre-build. I ran anonftpsync by hand, and the images were sane afterward, but up until then, the mirror hadn't been updated for over a day. | 19:39 |
| infinity | cjwatson: Not sure why that would be, and too tired right now to try to make sense of it. | 19:39 |
| plars | infinity: ok, now something new is broken at least :) | 20:31 |
| plars | 'can't get a pty: No such file or directory' | 20:31 |
| plars | xnox: is that related to the stuff you were worried about earlier? ^ | 20:31 |
| xnox | plars: can you point me to the logs? | 20:33 |
| plars | xnox: nothing external unfortunately due to the jenkins madness right now | 20:34 |
| xnox | plars: well i can get to internal jenkins instance. or is it not even there? | 20:34 |
| plars | http://10.98.0.1:8080/view/Saucy/view/Smoke%20Testing/job/saucy-server-amd64-smoke-default/45/ is the best I can give you, it will publish to the visible jenkins instance eventually I hope | 20:34 |
| xnox | plars: well I grepped everything there, and can't filed "can't get a pty" | 20:35 |
| plars | xnox: no, I had to hand boot it to see this message | 20:36 |
| xnox | oh. | 20:36 |
| plars | it wasn't clear to me from the logs there why it failed, so I tried it myself | 20:37 |
| plars | and the message I pasted above seems to be happening continuously, just after selecting install from the boot menu | 20:37 |
| plars | xnox: upstart1.8-0ubuntu4 did show up in this release it looks like, was that the one you were talking about earlier? | 21:00 |
| plars | also several things got dropped it seems, like busybox, cryptsetup, ecryptfs... | 21:02 |
| slangasek | cryptsetup+ecryptfs> expected | 21:02 |
| slangasek | busybox> doesn't sound problematic, probably related to cryptsetup+ecryptfs | 21:02 |
| plars | slangasek: any thoughts as to the upstart bump? was this expected to be something that could cause problems? | 21:03 |
| plars | slangasek: xnox mentioned wanting to watch the results when it showed up | 21:03 |
| slangasek | plars: we never expect the upstart versions we upload to cause problems ;-) | 21:03 |
| plars | slangasek: yes yes, of course :) | 21:03 |
| slangasek | sounds like xnox was just being responsibly cautious and wanting to follow up in case there /were/ any problems | 21:03 |
| slangasek | I don't know why the new version would have caused pty errors | 21:04 |
| slangasek | given that upstart and the kernel both changed at the same time (right?), I think the kernel is the more likely culprit... the changes to upstart should only affect the behavior of 'telinit u' | 21:04 |
| plars | slangasek: any suggestions for debugging this? I don't get very far in the boot unfortunately | 21:04 |
| slangasek | plars: splice together something using the previous kernel to see if the problem is reproducible, so we can figure out first of all if we should be going after upstart or kernel | 21:06 |
| slangasek | in fact, you probably want to try separately downgrading (or upgrading) both the kernel and upstart, to make sure which one is the culprit... if either | 21:06 |
| cjwatson | infinity: mm, I thought I'd noticed something like that a while back when there was a stale lock ... | 22:51 |
| cjwatson | Yep, /srv/cdimage.ubuntu.com/etc/.sem-build-image-set existed, but dated very recently so hard to say | 22:54 |
| cjwatson | I've cleared it, hopefully that will fix life a bit | 22:55 |
| cjwatson | plars: can't possibly be anything to do with upstart, which doesn't run during server installation | 22:56 |
| cjwatson | I'm syncing an image to investigate | 22:57 |
| plars | cjwatson: the booting kernel on both images seem to be the same too though | 23:00 |
| cjwatson | With what you have so far I would guess udev, but we'll see | 23:02 |
| cjwatson | (Since the relevant startup script supplied by udev-udeb was screwed up not that long ago, and might have been again) | 23:03 |
| plars | there are some differences pertaining to rtc in the udev rules when I look in the initrd | 23:06 |
| cjwatson | I'll probably use BOOT_DEBUG=3 and step through by hand | 23:07 |
| cjwatson | (tends to require 'modprobe vesafb' blind ...) | 23:07 |
| * cjwatson SIGSTOPs his nightly mirror run for a bit to try to recover some bandwidth | 23:08 | |
| plars | start-udev seems to run, but the next one after that is when I start getting the error | 23:11 |
| plars | if I use your boot_debug=3 trick | 23:11 |
| cjwatson | capitalisation relevant :) | 23:12 |
| plars | yes, sorry | 23:12 |
| cjwatson | but I assume you got it right or you wouldn't have seen start-udev | 23:12 |
| cjwatson | "next one after that"? | 23:13 |
| plars | I saw it run some steps that seemed to be after it was doing start-udev | 23:13 |
| plars | I was given a prompt | 23:13 |
| plars | when I exited the shell at that point, was when I started getting the error | 23:13 |
| plars | rebooting now | 23:13 |
| cjwatson | right, I'm going to end up stepping through the individual scripts by hand | 23:14 |
| cjwatson | here it is, let's see | 23:14 |
| cjwatson | wonder if the lack of /dev/pts is relevant | 23:17 |
| cjwatson | comparing with raring | 23:17 |
| cjwatson | mm, raring at this point has a devpts mount | 23:17 |
| cjwatson | I blame systemd | 23:18 |
| cjwatson | Again | 23:18 |
| plars | ah, yes | 23:18 |
| plars | comparing with the image from june 6 (which installed ok) /dev is very populated at this point | 23:19 |
| cjwatson | /dev is populated, just not quite enough | 23:20 |
| cjwatson | Yep, the old script ended with | 23:22 |
| cjwatson | mkdir -p /dev/pts | 23:22 |
| cjwatson | mount -t devpts devpts /dev/pts | 23:22 |
| cjwatson | Completely missing from the new one | 23:22 |
| cjwatson | Might POSSIBLY matter | 23:22 |
| cjwatson | * debian/extra/udev.startup: Drop devpts mounting again, already done by | 23:23 |
| cjwatson | /usr/share/initramfs-tools/init. | 23:23 |
| cjwatson | That doesn't even run | 23:23 |
| cjwatson | LIES | 23:23 |
| plars | heh | 23:24 |
| cjwatson | So this was entirely unnecessary and wrong code cleanup ... | 23:25 |
| plars | thanks a lot for helping track it down cjwatson | 23:26 |
| cjwatson | plars: Is there already a bug for this that I can close? (If not, I don't need one) | 23:28 |
| plars | cjwatson: I was going to open one real quick if that's ok, for purposes of having something to designate why it is red on the results (we get asked sometimes) | 23:29 |
| plars | one sec... systemd I guess since udev is there now? | 23:29 |
| cjwatson | Well, OK, if *you* need one | 23:29 |
| cjwatson | systemd, yes | 23:29 |
| cjwatson | It'll need a debian-installer rebuild after systemd is built, but I don't usually open bugs for those (it's a hassle and not very useful) | 23:29 |
| cjwatson | You can just make the bug pro-forma describing the bare symptoms if you like, I don't need anything detailed | 23:30 |
| cjwatson | (And am waiting for the bug number so I can upload this ...) | 23:32 |
| plars | cjohnston: https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1188864 | 23:32 |
| ubot2` | Ubuntu bug 1188864 in systemd (Ubuntu) "/dev/pts not getting mounted before install" [Undecided,New] | 23:32 |
| plars | grr | 23:32 |
| plars | cjwatson: https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1188864 | 23:32 |
| cjwatson | thanks | 23:32 |
| plars | cjohnston: you know you like it when I do that :) | 23:33 |
| cjwatson | Uploaded | 23:34 |
| cjwatson | https://launchpad.net/ubuntu/+source/systemd/204-0ubuntu3 | 23:35 |
| cjwatson | slangasek: ^- Do you think that once the above has built and published to saucy-proposed on all architectures, you could do a no-change upload of d-i (lp:~ubuntu-core-dev/debian-installer/ubuntu as usual)? | 23:36 |
| cjwatson | It may or may not be worth respinning server images after that, versus just waiting for the cronned build in ~7 hours | 23:37 |
| cjwatson | But either way, I could do with some sleep rather than waiting up to insert a d-i upload in between those times :) | 23:37 |
| slangasek | cjwatson: that's after systemd publishes? sure I can take care of that | 23:58 |
| cjwatson | Yep. Thanks a lot | 23:59 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!