[01:38] hm.. [01:38] anyone still awake? [01:39] https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1061977 [01:39] Launchpad bug 1061977 in cloud-init "Machine fails to commission when console=ttyS0 is present on kernel opts" [Undecided,New] [02:02] smb`, https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1061977 . i'd appreciate your input when you awake. [02:02] Launchpad bug 1061977 in cloud-init "Machine fails to commission when console=ttyS0 is present on kernel opts" [Undecided,New] === cyphermox_ is now known as cyphermox === lifeless_ is now known as lifeless === smb` is now known as smb [07:55] ikepanhc: around? May I have some help compiling a test highbank kernel please? With "fakeroot debian/rules binary-highbank" (cross compile), I get this: http://paste.ubuntu.com/1261463/ . A zImage does exist but it doesn't seem to work. How should I be building test kernels instead? [07:56] * ikepanhc reads [07:57] rbasak: do you `fakeroot debian/rules clean` before fdr binary-highbank? [07:57] `fdr clean` will generate changelog, control in debian folder [07:57] ikepanhc: when I tried that it wouldn't even start a build. I'll do that now and paste the error. [07:58] rbasak: you using cross-compiler? [07:58] ikepanhc: yes [07:59] rbasak: try `export $(dpkg-architecture -aarmhf)` then `fdr clean` then `CROSS_COMPILE= fdr binary-highbank` [08:00] alias fdr="fakeroot debian/rules" first [08:00] Hmm. [08:00] It seems to be working now [08:00] I swear I got an error doing the clean step first yesterday [08:01] So I tried using git clean only instead [08:02] fdr clean is not only to clean those objects, but also generate control and changelog which we need for make [08:02] I see [08:03] * rbasak waits for the build to finish === henrix_ is now known as henrix === henrix is now known as henrix_ === henrix_ is now known as henrix [08:58] * ppisati needs to go out for ~30mins, brb === henrix_` is now known as henrix [11:17] henrix is having ISP issues === doko_ is now known as doko [11:43] when i run ubuntu-bug linux today, it's reports 'Problem in linux-image-3.5.0-16-generic [11:43] The problem cannot be reported [11:44] This is not an official Ubuntu package. Please remove any third party package and try again' [11:44] message box is popping up [11:47] this is a fresh install btw [12:18] * rtg thinks that was quite a large update for barely being post beta [13:06] smb, around ? [13:07] smoser, yup [13:07] https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1061977 [13:07] Launchpad bug 1061977 in cloud-init "Machine fails to commission when console=ttyS0 is present on kernel opts" [Undecided,New] [13:07] you have any thoughts there? [13:08] i would really rather not take 'console=ttyS0' off the cmdline (as I really want output there by default rather than to any graphical console which has no chance of being logged), but that is about the only option i see. [13:08] * rtg uploaded linux-lts-quantal 25 hours ago and it _just_ started building. [13:09] smoser, well my first thought was that the first pastebin does not look like console output [13:09] smb, on random other note, i know in your cloud-images testing you've had to add ways to get into an image, and that is more involved than you'd like. [13:09] i put together http://bazaar.launchpad.net/~smoser/+junk/backdoor-image/view/head:/backdoor-image [13:10] that basically can mount an image, inject a user into it and then unmount it in one command. [13:10] smb which pastebin ? [13:10] smoser, By now I have made my scripts add a nocloud file for newer releaes [13:11] its not console output, its cloud-init output. he couldn't get console output as it is going to a video console. [13:11] smoser, http://paste.ubuntu.com/1261109/ shows the cloud-init log. [13:11] oh, for "nocloud" there is a util in cloud-utils called 'cloud-localds' that will basically do it also. [13:12] this one actually modifies the image to add the user, so cloud-init isnt involved/required at all [13:13] let me see if i can get it, smb, but its not really important. (the output of running it) [13:13] it fails [13:13] and 'echo "hi" | sudo tee /dev/console' fails also [13:13] the issue is /dev/console writes fail [13:13] which causes all sorts of stuff to be unhappy [13:14] smoser, I think I saw something for this nocloud thing and at least at that point in time it was still too complicated to get working. Still having run some of the cloud init code is usefull to find problems, so this is what gets done right now. [13:15] smoser, Yeah I think it probably makes sense to move some of the info from later comments into the description. I think I just stopped somewhere at the top as I could not really guess what you/the report is up to [13:17] smb, well, run 'cloud-localds' it gives a full example of booting an image even. iss simple for 12.04+. [13:17] smoser, So for the question: as far as I remember everytime you do a console= the kernel assigns a console device, just the last one I believe is a bit different as there is one sort of preferred console [13:17] smb, well, comment 4 has just about everything you need. [13:17] smoser, You assume I care to read that far [13:18] no, I SAY YOU CARE! [13:18] :) [13:18] You *hope* [13:18] if you' dlike i can move the explanation up to the description. [13:18] :) [13:18] but anwya. [13:18] yeah, each 'console=' ends up receiving kernel printk [13:18] which is good. [13:19] and it is good that 'console=ttyS0' when there is no serial device present does not stop the kernel from booting. [13:19] however. [13:19] the final console= parameter gets assigned to /dev/console [13:19] and init writes output to /dev/console, and cloud-init inherits that as its stdout [13:19] the preferred console thing, yeah [13:19] and any writes to stdout then fail [13:20] which can cause problems :) [13:20] what i had *thought* (apparently wrong) the kernel did was assign /dev/console to the last *good* console= on the cmdline. [13:20] but i guess it only makes intelligent decisions if you give no console= [13:21] otherwise it does exactly what you tell it to do. [13:21] the basic problem is, then: [13:21] Not sure it makes any great decisions on its own. [13:21] * if there is a serial device, i want output to go there (including /dev/console) [13:22] * if there is no serial device, i would rather writes to /dev/console not give IO error. [13:22] smb, well http://www.kernel.org/doc/Documentation/serial-console.txt says "If no console device is specified, the first device found capable of acting as a system console will be used." [13:23] "At this time, the system first looks for a VGA card and then for a serial port. So if you don't have a VGA card in your system the first serial port will automatically become the console." [13:23] i'd really just like that order switched. [13:24] and i kind of thought that is what i was getting by adding multiple console=. but i guess not. [13:24] so i think i understand the issue, and i'm just bothering you in a hope that you know something i do not. [13:24] and you can come up with a solution [13:24] and i have not read any kernel code, only doc. [13:24] smoser, Really you want a decision about something the kernel does not know until it finds out done before it can find out. And then you cannot say if there is a serial port whether it is a serial console. So normal boot might siddenly go wrong [13:24] hi good people, I'm sorry to say that my suggested patch from RedHat does not seem to work on all Nvidia chipsets. https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1043518 As the RC is up coming. PPC needs a bit of help. Arguing with 'X' does not solve the issue, it is a kernel change that was undertaken and has serious regression issues. My ask? please make it work, the PPC people are runnning around getting patches in. the kernel is where [13:24] Launchpad bug 1043518 in linux "live cd is unusable due to video degradation with the splash boot option enabled" [High,Fix released] [13:25] smb, right. i know its tricky. [13:25] but the doc says it already does such trickyness. [13:25] if you dont pass any console= [13:26] smb, the other option i considered was that I could have the initramfs notice that /dev/console was busted and "fix it". [13:26] Yeah, using a VGA (compatible) card which really is there most of the time and then try on serial (just in case) [13:26] http://www.mail-archive.com/linux-kernel@vger.kernel.org/msg246433.html discusses changing it [13:26] but i dont think it really seems to work. [13:26] at least it wasn't extremely simple to change the target device of /dev/console [13:29] smoser, I think that part of the kernel "suffers" from not needing much special cleverness as long as you stick to a certain architecture. And it gets even more complicated when there are things like grub in between passing some information or setting up stuff [13:29] well, in this case the thing thats pasin the cmdline is pxe boot [13:30] and i really, *really* want a "one set reasonably fits all" solution. [13:30] as on the first pxe boot of a node, i know zero information about that node. [13:30] s/pasin/passed in/ [13:31] Yeah, I understand the issue. (btw grub seems to do the passing not by cmdline but directly transferring some structure). I just don't think it will be an easy solution. [13:32] smb, well, could you at least read a bit and see if there is a way i could change /dev/console during initramfs ? [13:32] that'd be sufficient for me, if i could recognize its busted and poke around to fix it. [13:33] before /sbin/init came up. [13:34] then, my solution would be to just do what i'm doing (multiple console= parms) and have initramfs walk them backwards until it finds one that does not give io error. [13:34] and then fix to that. [13:35] smoser, I put it on my list but I cannot really say when I get to it. There are enough other things broken that take up time. [13:36] smb, well, at least read that thread on lkml, and tell me if it looks like it is possible or not. [13:36] because i came away with "um... maybe" [14:15] I'm trying to track down a regression between two ubuntu kernels, what's the easiest way to see what patches there are between two kernel versions? [14:16] diwic: git log? [14:16] diwic, it also depends on whether there has been a rebase between them. [14:17] the person claims "3.2.0.31" is not working and "3.2.0.30" is. [14:18] diwic, vanilla stable is linear. git log -p v3.2.0.30..v3.2.0.31 [14:19] you can also restrict your search to just sound, e.g., 'git log -p v3.2.0.30..v3.2.0.31 -- sound' [14:19] rtg, should that be working on a precise tree? I get "unkown revision or path not in working tree" [14:19] diwic, lemme check [14:20] smoser, Hm, *if* it is correct, I would think that it only will work when the current /dev/console driver is a pty. Which would need the console=tty0 last. And there can only be one re-assignment active and not sure whether it can be done when upstart already has changed its stdin and out there. [14:20] yeah. oh well. [14:20] diwic, oh, likely not since stables patches are rebased onto ubnutu. clone the real stable repo at git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git [14:20] thanks for reading, smb. thanks for your time. [14:20] s/are/are not/ [14:22] diwic, hmm, you're talking about Ubuntu versions, not stable versions, right ? i.e., Ubuntu-3.2.0-30.48 [14:23] rtg, okay, so replacing v3.2.0 with Ubuntu tags should do it [14:23] Sometimes you just get unsure of what a command really does [14:24] so then you can get a list of patches by doing 'git log --reverse -p Ubuntu-3.2.0-30.48..Ubuntu-3.2.0-31.49' [14:24] diwic, you can also look in the changelog [14:25] hmm, it seems there are no relevant patches in the sound directory that could have caused this error AFAICS [14:26] I'm reassinging it to the linux package and let someone of you bisect it, chances are it's something completely unrelated [14:27] diwic, make sure jsalisbury is aware of the bug number. note that he's out today. [14:28] rtg, ack [14:29] * henrix -> brb === henrix_ is now known as henrix [15:24] * henrix -> out for ~20mins === henrix is now known as henrix_ [15:38] apw: what triggers the CVE tracker to move a CVE from "triage needed"? I've looked at 1012-4467 and updated the break-fix. Do I need to nudge it in any other way? === henrix_ is now known as henrix [15:50] hi - bug 1057024, last comment - those messages about /dev/sda, are they serious? serious enough to disregard other bugs from that system (when they can't be reproduced elsewhere)? [15:50] Launchpad bug 1057024 in libvirt "internal error Process exited while reading console log output: char device redirected to /dev/pts/1 error when creating a vm" [High,Confirmed] https://launchpad.net/bugs/1057024 [15:51] sconklin, if its pushed to kernel it should be found in an hour sort of thing [15:51] smb: ^ do you mind taking a quick look? [15:51] hallyn, just opening [15:51] thx! [15:52] sconklin, the processing kicks off at :20 and if there is a change can take some time, when did you do the update [15:53] (note the msgs go on, you can see more in the dmesg file in the tar.gz she appended) [15:53] hallyn, serious in the sense that a write failed. not much more info info that piece of output. [15:53] hallyn, Let me check the full log [15:54] smb: thanks. (i also see kvm: VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL does not work properly. Using workaround) [15:55] hallyn, I think I see that in some of my machines too. Without any apparent effect. I would say it *does* us a work-around. :) [15:57] smb: ok, thanks. just making sure [16:02] hallyn, But yeah, there seems to be quite a bit of trouble with that disk. Might be worth trying to check all cabling and maybe trying to limit the speed to 1.5G... [16:04] smb: i think it's a laptop [16:05] hallyn, Ok harder to check cables. But something is wrong even before [16:05] [ 975.481263] ata1.00: exception Emask 0x0 SAct 0x3f SErr 0x0 action 0x6 frozen [16:05] [ 975.481274] ata1.00: failed command: WRITE FPDMA QUEUED [16:05] apw: I did the update around 30-40 mins ago I think [16:06] so perhaps just missed it [16:06] worst case delay [16:06] probably. no worries, I'll wait [16:06] bzr: ERROR: Invalid http response for https://xmlrpc.launchpad.net/bazaar/: Unable to handle http code 502: Bad Gateway [16:06] or perhaps something bad happened [16:07] i'll watch it for the next run [16:07] is bzr down for maintenance perhaps [16:08] apw: I was hearing grumbling recently within the hour about LP issues [16:08] could be that too [16:10] apw: in other news, I'm discovering that in order to use the --include-removal flag for packaging, we have to change our debian/source/format to 3.0 (quilt) [16:11] ogasawara, thats not going to work well for us, as we have debian.* outside our debian directory [16:11] apw: yep, I'd then have to shove an --auto-commit flag in there as well, or it vomits because of that fact [16:12] smb: dunno what that means [16:12] ogasawara, i wish we had chose debian/branch not debian.branch ... with hindsight [16:12] hallyn, Don't know in detail but looks like some dma write timed out [16:13] apw@dm:~/git2/ubuntu-quantal$ cat debian/debian.env [16:13] DEBIAN=debian.master [16:13] though as its encoded that way we ought to be able to move it, yeah right [16:13] something to talk about at uds in teh bar [16:13] apw: indeed. I think its harmless for now for us to leave things as they are for now. that highbank clock.c file is never used, and the redundant firmware is not a huge issu [16:14] its not ending up in the binaries which is mostly what we care about [16:14] * ppisati -> gym [16:15] hallyn, Just one other note that apw recently had to reboot to get virt-manager working. We have not really found why that was [16:16] indeed, logging out and in was not enough for sure [16:16] as i tried that before giving up and doing a windoze on it [16:17] smb: after installing libvirtd, you do have to log out and back in to be in the libvirtd group (or run newgrp). had he tried that first? [16:17] a reboot would of course effect the same thing :) [16:17] yes logging out and in changed nothing [16:17] hallyn, Yep. [16:17] apw: hm [16:17] i know i was pissed [16:17] apw, Did you not also have the complaint about not being able to load the kvm module (though it was loaded) [16:18] apw: can you reproduce that? i've been doing these tests for awhile the last two days and haven't seen that [16:18] that rings a bell but you test my memory [16:18] so if you can reproduce, can you give as much detail as possible? [16:18] apw: oh, hey, did you install kvm *after* installing libvirt by chance? [16:18] hallyn, very possible yes [16:20] hallyn, While I usually go for "apt-get install virt-host^" which takes care of those details [16:20] i think i assumed kvm would be a dependancy [16:20] though thinking about it clearly it will not be [16:20] so i think i did do them in that order [16:20] apw: ok, then that makes sense. and iiuc 'restart libvirt-bin' wouldn't even work, you'd have to 'stop libvirt-bin; start libvirt-bin' to get it to work [16:21] that i wouldn't have tried indeed [16:21] * smb cannot remember whether we tried the one or the other [16:22] sconklin, this is not the right format: [16:22] break-fix: 644595f89620ba8446cc555be336d24a34464950 - ed6fe9d614fc1bca95eb8c0ccd0e92db00ef9d5d [16:22] its a two field thing [16:22] huh, ok. I copied what I found in another file. [16:23] Oh wait. I'll bet the '-' was a placeholder for a nonexistent break [16:23] I'll fic it [16:25] perhaps upstart should warn you when it knows 'restart' won't restart [16:26] Maybe restart should restart [16:27] apw: pushed. Now we wait another cycle. I have to run a quick errand anyway === tyhicks` is now known as tyhicks [17:19] * apw is about done ... === yofel_ is now known as yofel === henrix is now known as henrix_ [19:43] * rtg -> EOW