=== vanhoof_ is now known as vanhoof === MTecknology is now known as billmeye === billmeye is now known as MTecknolgoy === MTecknolgoy is now known as MTecknology === sconklin-gone is now known as sconklin === sconklin is now known as sconklin-gone === MTecknology is now known as pfSensory === pfSensory is now known as MTecknology === lag` is now known as lag [07:39] how to use dma functions to flush caches? [08:23] simple question, we use > to write the output to a file [08:23] how to append the output to the existing content of the file [08:24] fairuz, >> [08:28] apw: ok ty === smb` is now known as smb [08:42] <_ruben> weird .. the link local address "oddity" with sit tunnels i see also on an old SLES9 box (2.6.5 based kernel) .. yet my sixxs interface (created by aiccu) doesn't seem affected somehow .. bah [08:51] i use flush_cache_all() in kernel space to flush L1 cache, do similar function exist in user space? [08:57] apw, In a quiet minute, maybe you have a pointer for that booty thing you mentioned [09:04] commit 299c56966a72b9109d47c71a6db52097098703dd [09:04] Author: Don Zickus [09:04] Date: Mon Feb 7 23:25:00 2011 -0500 [09:04] x86: Use u32 instead of long to set reset vector back to 0 [09:04] smb ^^ [09:04] apw, ta [09:14] apw, Looks like one of those things you hope other people know what they are doing. When setting its done high, then low. And when resetting its just writing 32bit, which would be low, then high... [09:16] i think that in thory we can spin on it not being 0 or something, so there is magic going on [09:16] It is probably so old magic, that it might be documented in my ancient tome of pc internals... :) [09:17] heh yeah. i think on reset they jump to a routine which is doing: [09:17] while (!reset_vector); [09:17] *reset_vector() [09:45] I asked yesterday about searching for regresions in kernel, I'm working with bug https://bugs.launchpad.net/ubuntu/+source/alsa-driver/+bug/662009 [09:45] I tested it and last working kernel-ppa version is 2.6.34.7-maverick and in 2.6.35-rc2-maverick which is next sound it's already broken. What next, should I compile kernel from git? [09:45] Launchpad bug 662009 in alsa-driver "[Lenovo Y530 - Realtek ALC888] Regression from 10.04: No sound with snd_hda_intel model=lenovo-sky" [Undecided,Triaged] [09:45] I asked yesterday about searching for regresions in kernel, I'm working with bug https://bugs.launchpad.net/ubuntu/+source/alsa-driver/+bug/662009 [09:45] I tested it and last working kernel-ppa version is 2.6.34.7-maverick and in 2.6.35-rc2-maverick which is next sound it's already broken. What next, should I compile kernel from git? [09:45] I asked yesterday about searching for regresions in kernel, I'm working with bug https://bugs.launchpad.net/ubuntu/+source/alsa-driver/+bug/662009 [09:45] I tested it and last working kernel-ppa version is 2.6.34.7-maverick and in 2.6.35-rc2-maverick which is next sound it's already broken. What next, should I compile kernel from git? [09:46] ? [09:46] argle [09:50] I asked yesterday about searching for regresions in kernel, I'm working with bug https://bugs.launchpad.net/ubuntu/+source/alsa-driver/+bug/662009 [09:50] I tested it and last working kernel-ppa version is 2.6.34.7-maverick and in 2.6.35-rc2-maverick which is next sound it's already broken. What next, should I compile kernel from git? [09:50] Launchpad bug 662009 in alsa-driver "[Lenovo Y530 - Realtek ALC888] Regression from 10.04: No sound with snd_hda_intel model=lenovo-sky" [Undecided,Triaged] [09:51] nusch, yes we saw that the previous 3 times you said it too! [09:51] as you have now a small range v2.6.34..v3.6.35-rc2 on mainline it seems appropriate to do a bisect between them [09:52] apw: sopry my irc client is bugged id didn't show send message after presing enter [09:57] and is there a way to not compiling it such long? I left compilation of Ubuntu-2.6.35-1.1 for night, and after few hours it stopped due to no space left on hdd. [09:57] I'm compiling with this line DEB_BUILD_OPTIONS=parallel=2 AUTOBUILD=1 NOEXTRAS=1 skipabi=true fakeroot debian/rules binary-generic am I compiling more than my amd64 architecture or this is normal? [09:57] Also is there a method to shorten compile time while switching betwwen git tags - to not compile again modules which didn't changed ? Are there such modules ? [09:57] nusch, what are you trying to compile on [09:57] i have some faster kit i could do a build on for you [09:58] and what release are you running [09:58] maverick [09:59] nusch, ok so v2.6.34 is good and v2.6.35-rc2 s bad ... correcet ? [09:59] yes [10:00] or maybe I should also test compiled vanilia kernels to test if the bug was introduced by ubuntu o r kernel maintainers? [10:02] you tested the ones in the mainline archive i pointed you to right ? [10:02] yes [10:02] and this http://kernel.ubuntu.com/~kernel-ppa/mainline/v2.6.35-rc2-maverick/CHANGES says only 2 people were involwed in snd_hda_intel [10:02] nusch, ok so you did test virgin upstream builds, those have no ubuntu ness applied to them [10:03] nusch, are you 32 or 64 bit ? [10:03] so if the bug was introduced by ubuntu devs it should be quite easy to locate [10:03] no [10:03] no? [10:03] I tested ones from http://kernel.ubuntu.com/~kernel-ppa/mainline [10:03] right ... those _are_ mainline virgin builds without any ubuntu sauce [10:04] that is the point of them, they don't have anything we do applied so we can tell if its our fault [10:04] 32 or 64 bit builds ? [10:04] so why they have maverick in the name ? [10:04] 64 [10:04] because that tells me which configuration was used, which release they are most likely to work on [10:04] * apw notes that this is described in the faq [10:05] ok, so I used thos virgin build from there for 64bit and only tagged as maverick [10:05] thats fine, we can do this using bisect from here [10:07] should I continue Ubuntu-2.6.35-1.1 build or wait for your script ? [10:08] nusch, normally it takes about half an hour a build here, so i shouuld have something to test in that kind of time [10:09] nusch, this is a sound issue right? [10:09] yes [10:09] ahould I also do something with alsa? [10:10] i think a bisect is our best hope, it says 207 revisions to test, 8 steps so it shouldn't take forever to get through [10:12] so simpy I should compile every time tag good/bad with bisect and compile again until I find revisions ? And shouldn't I compile alsa also every time ? Doesn't alsa need to fit currently running kernel ? [10:12] nusch, alsa is part of the kernel [10:12] and nusch i was volunterring to use my fast build box to make the kernels for you [10:13] I'd be grateful [10:14] and what's about userspace as alsamixer, I had issue that after installing new kernel all channels in sound card were muted so I need restore it once [10:15] nusch, yep it often does that, you'll need to check the basics like that every time [10:16] So i need to compile alsa-tools also ? [10:16] no you should need to do nothing other than check the mixers are not muted when testing the kernels i provide [10:17] ok [10:46] nusch, ok first test kernel at: http://people.canonical.com/~apw/bisect-maverick/ [10:47] ok, I don't have any irc bnc so I'll be back after testing [11:01] \query apw the one you provided is ok, waiting for next [11:01] nusch, ack [11:23] nusch, ok second set there ( the later numbered ones in the same place) [11:44] apw, this one was bad [11:45] nusch, ack [11:47] cking, smb how does this look: http://people.canonical.com/~apw/misc/key.html [11:49] great. Should "critical" be red and "high" be orange? [11:55] cking, oh the colours are someone elses fault :) [11:55] but possibly, i got distracted before i pointed out the change [11:55] wh [11:56] which is to include all tasks of a bug, but sqashed on status where they match, se 600453 as an example [11:56] * cking looks === zul__ is now known as zul [12:15] nusch, next one is up [12:16] apw, thx I'll check [12:30] apw, this one is broeken in early boot, last message it says is usb 2-5: new high speed USB.. [12:31] no errors or, warning, i unplugged all usb but know my laptop camera also uses USB bus [12:31] after presing ctrl alt del i get md: stopping all md devices and blinking curor - at this stage even SysRq doesn't work [12:35] apw, I must left for a while, if it won't be a problem please prepare more than one most probable build, I'll test all and provide you with results [12:36] nusch, thats annoying as we don't know whether it is good or bad to tell bisect [13:17] nusch, two new builds in the bucket ... please test both and report back === shadeslayer_ is now known as shadeslayer === herton_ is now known as herton [14:00] the function flush_cache_all() flushes all cache or only L1 cache? [14:01] it's from asm/cacheflush.h === sconklin-gone is now known as sconklin [14:54] apw, first of these also not booting to the end, testing second [14:54] nusch, bah thats most unhelpful as it provides no information [14:55] but did you changed anything, i tested some times ago one of the most recent kernels in mainline build and it booted to the end (without sound) [14:56] I'll try second one, be back in a moment [14:57] nusch, those two are as if the former one was bad and good respectivly [14:58] as the original did not boot far enough for us to know [15:01] apw, second one did boot [15:01] but no sound [15:02] crap that is worst outcome as we have no idea where the good has gone [15:02] i mean bisect201103011259 booting, no sound, bisect201103011238 not booting can't check [15:03] yep, indeed the worst possible combination of results [15:34] ## [15:34] ## Ubuntu Kernel Team Meeting - Today @ 17:00 UTC - #ubuntu-meeting [15:34] ## agenda: https://wiki.ubuntu.com/KernelTeam/Meeting [15:34] ## === zul__ is now known as zul [15:59] nusch, ok we are likely wasting our time here, with such a big hole in the bisect, but here is another one to test [16:00] nusch, let me know if it works or not [16:03] apw, we aren't done with alpha 3 yet are we, just in the freeze bit? [16:03] * JFo brain isn't cooperating today [16:03] * JFo digs for the schedule [16:03] yeah we are frozded for it [16:03] as of today, releasing thursday [16:04] k, thought so [16:04] thank you sir :) === herton is now known as herton_lunch [16:15] bjf, i have some further updates for lpltk for your consideration: lp:~apw/arsenal/python-launchpadlib-toolkit-milestones-etal [16:18] bjf, been playing with the task oriented view of our tasks, ending up with a hybrid which collapses common status: http://people.canonical.com/~apw/misc/key.html [16:18] bjf, note 600453 has two nominations, what do you think [16:32] manjo, about? did you get a chance to test the natty pre-proposed kernel; to confirm it fixes your s/r issues [16:36] apw, looking it over, looks pretty good, personally, the "Summary of bugs" at the top just wastes space and I don't care about "Closed bugs" but they are at the bottom and I can ignore them [16:37] bjf, not really sure myself what we use that for, i've not touched that bit yet [16:37] bjf, does the multiple nominations bit make sense [16:37] apw, i like the "Pending" pulled out [16:37] apw, yes, i think it makes sense [16:38] apw, if you needed the room, for nominations you could use a single capital letter [16:38] i was thinking the milestones and noms might merge into one [16:39] as they are logically pairs, Lucid:lucid-updates, stylee [16:41] can you have mileston: lucid-updates without being nominated: Lucid ? [16:41] ## [16:41] ## Kernel team meeting in 20 minutes [16:41] ## [16:41] bjf, sort of, if the linux (Ubuntu) task is given a milestone === herton_lunch is now known as herton [16:41] though i actually think that that is stupid and any linux (ubuntu) task which is not actually nominated should be converted to Natty [16:41] as really thats what i means [16:42] we already have to elide those when they are nom'd to natty, else you get duplicates [16:44] apw, now what i want is for us to come up with a "kernel heat" index that we add to each bug based on criteria that matter to us [16:44] bjf, any idea what the criteria would be [16:45] apw, there are a few "Incomplete" up in the "Active" section [16:46] yes, those are correct, they are Incomplete (with response) ie ... active [16:46] apw, so not really incomplete [16:46] we haven't got the auto flipper working yet, thats on my list to sort [16:47] right, they need flipping into another state, but lp doesn't do that by default [16:47] i have a prototype that knows how to do it, just not had time to verify its not going to spack up all our bugs [16:47] before letting it loose [16:47] so anything "incomplete" but "active" needs some manual love [16:47] for right now [16:48] yeah, its on jfo's list of 'daily' tasks to flip them back [16:48] the key is its hard to flip them to the right place [16:48] you have to look at the activity log for hte bug to know where they have come from, then use some magic to convert that to an appropriate place [16:49] that and I haven't even touched the list yet this week [16:49] :-/ [16:49] the prototype also handle the New bugs, for when people flip an Incomplete back to New (as thats the only place they can) it can then flip them back to trigaed or whatever they should be [17:14] headed for some lunch [17:25] apw, kernel version for A3 is 2.6.38-rc__ ? [17:25] * skaet_ started into the TechOverview docs now... [17:33] skaet_, -rc6 [17:35] thanks apw, adding now. Can you give me a nice summary paragraph of the changes for this release to add to the Tech Overview? [17:35] skaet_, will do === bjf changed the topic of #ubuntu-kernel to: Home: https://wiki.ubuntu.com/Kernel/ || Natty Kernel Version: 2.6.38 || Ubuntu Kernel Team Meeting - March-08 - 17:00 UTC || If you have a question just ask, and do wait around for an answer! === sforshee is now known as sforshee-lunch [18:05] sconklin, http://people.canonical.com/~apw/lp658307-natty/ [18:05] arrggle [18:05] http://people.canonical.com/~apw/misc/key.html [18:25] brb, gonna step over and hit the big red update button on my Natty netbook [18:48] bjf: tgardner is gone at the moment, can you maybe check a CVE in his absence? nelhage noticed an issue [18:49] bjf: I noticed that http://people.canonical.com/~ubuntu-security/cve/2010/CVE-2010-2653.html claims the CVE only affects ppc, because of CONFIG_HVC_CONSOLE [18:49] nelhage: Race condition in the hvc_close function in drivers/char/hvc_console.c in the Linux kernel before 2.6.34 allows local users to cause a denial of service or possibly have unspecified other impact by closing a Hypervisor Virtual Console device, related to the hvc_open and hvc_remove functions. (http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2010-2653) [18:50] But the affected file is hvc_console.c, and looking at the Makefile: obj-$(CONFIG_HVC_DRIVER)+= hvc_console.o [18:50] And CONFIG_HVC_DRIVER is enabled x86 [18:50] er, on x86 [18:51] I'm guessing someone just checked the wrong config option (the naming is certainly confusing), but maybe there's some reason HVC_CONSOLE matters anyways? [18:56] kees, nelhage will look into it [18:56] bjf: cool; thanks [18:56] thanks [18:59] bjf, Sounds sensible to me. I remember that ppc had a console driver named hvc and xen also has one of the same name. So I guess for that cve both drivers were confused and it affects x86 [19:00] kees, It looks like it should probably be re-activated (the cve tracker file), so we will look at it again === sforshee-lunch is now known as sforshee [19:12] apw, for last kernel you provided I've got panic at early boot stage, swapper tainted - not syncing attempted to kill init and ~ following backtrace http://pastebin.com/JJi289qf (written from screen) [19:15] bjf: btw, for uct, it's just "released", not "fix-released" [19:15] kees, sorry, got mixed up hopping between lp and the cve tracker files [19:15] bjf: but, I'm curious, since you have it as "released" but without a version. [19:16] bjf: where was it fixed? [19:16] kees, let me look ... [19:16] bjf: do you mean "not-affected"? [19:17] kees, if it was already fixed as part of a upstream stable release, is that "not-affected" or "released" [19:17] bjf: if it's part of upstream stable release, it must still be included in -security and announced in a USN. [19:18] bjf: i.e. it must show up in the changelog [19:18] bjf: it should be marked "pending" so that we flip it to "released" on USN [19:18] bjf: if it's already in an earlier release, then the first version that fixed it should be listed. released (2.6....-abi.m) [19:20] and if a package started its life with it fixed, it would be not-affected [19:30] kees, fixed up the active status file and pushed the changes [19:30] bjf: cool; thanks. pulling now [19:30] bjf: I'm about to publish USNs for lucid and maverick. have all the CVEs fixed in those two been called out in the changelog? [19:31] kees, as far as i know [19:31] kees, i didn't go through every commit and verify [19:31] bjf: well, I mean, last time there were CVEs fixed in the stuff that came from stable, and people yelled at me for leaving CVEs out of the USN. [19:32] bjf: did all the open CVEs get compared to the stable patches? sounds like no? [19:32] kees, we just apply the stable commits as is, if they don't have the proper "CVE-xxxx-yyyy" line in the commit then they don't make it into the changelog [19:33] kees, we have no control over upstream [19:33] bjf: right, but we need to have the CVEs actually reported as fixed. that's part of the triage. (this was in my email after the last maverick USN) [19:34] kees, that means that i'd have to go through all the outstanding CVEs and see if they were fixed in a stable upstream release that we are applying and we don't do that [19:34] kees, we only look at the CVE as we work them [19:35] bjf: right, this was the process change I requested last time. [19:36] bjf: you don't have control over upstream's changes, but you do over your changelog. if you have already verified that upstream has fixed it, then you need to update the changelog, otherwise no one (not us, not other kernel team members, not ksplice, not users, no anyone) knows what was fixed [19:40] I thought this was already worked out? [19:40] jdstrand, not to my knowledge [19:40] well, it is pretty clear from an accounting perspective that we need to correlate fixes coming in with CVEs even if upstream isn't doing it [19:40] jdstrand: you just look in the git tree ;P [19:40] jdstrand, if a CVE was fixed in some previous release and we just now find out about it as we work the cve from our end, you want us to update the changelog ? [19:40] jdstrand, we update the cve tracker status, that is the accounting we are correlating [19:40] bjf: if it is the same release, yes [19:40] bjf: that works for part of the accounting, but not the part that users see [19:40] jdstrand, that's not the issue [19:40] if something was vuln in lucid, but not maverick, then maverick's kernel doesn't need it [19:40] jdstrand, however it could be, we don't look at all the outstanding CVEs when we apply a stable release [19:40] there are two cases. I outlined them both in my email: stuff that is so late to get a CVE it was in a prior release, and CVE fixed in stable patch series for current release [19:40] where "release" means publication of an update [19:40] the former case requires manually editing the USN database. the latter requires manually editing the kernel changelog. [19:40] both require code inspection during tirage [19:41] kees, if i get a set of stable patches today, i apply them. I do not go through the list of outstanding cves and see which of them got fixed in that batch of upstream stable commits === zul_ is now known as zul [19:41] bjf: sure, but when you are looking at CVEs, you are presumably verifying if maverick is affected [19:41] bjf: and you do that by looking at the commits [19:42] bjf: right; that's the process I asked to be fixed. the merging of security and stable requires it, else we're not accurately reporting what CVEs have been fixed. [19:42] bjf: when you see maverick has the commit, you update the changelog [19:42] jdstrand, correct and i update the cve tracking bits, i do not rework the commit of a stable upstream commit to add the CVE verbage [19:42] jdstrand, nope, we don't do that [19:43] jdstrand, our changelog is updated via script that culls the commits between versions [19:43] bjf: you need to be adjusting the debian/changlog in some manner to indicate that it has CVE-20XX-NNNN [19:43] jdstrand, we are not doing that and i am not committing to doing that [19:43] bjf: whether that is part of the automated commit message you guys do, or somewhere up above or below the automated ones, it doesn't matter [19:44] bjf: how are user's supposed to know if their kernel is fixed? [19:44] bjf: how are USNs supposed to be written? [19:44] I suppose the USNs could be written based on the status of the CVE in the tracker, but users don't know [19:45] bjf: I don't understand why a stanza can't be added to the debian/changelog when you have all the info [19:45] * CVE fixes as part of stable tree merge: [19:46] - CVE-2011-NNN1 [19:46] jdstrand, if a cve was fixed as part of a stable upstream commit pulled into one of our releases, two updates ago users didn't know that the cve got fixed [19:46] - CVE-2011-NNN2 [19:46] jdstrand, because we didn't go through the entire list of outstanding cves looking for them [19:46] bjf: when you pull the commits, there needs to be review of what is fixed [19:46] jdstrand, no [19:46] two updates ago is a separate issue that we can fix separately by pulling from UCT and stuffing it into the USN db. [19:46] bjf: if it went into upstream stable, there is already a CVE in the CVE database [19:47] that CVE in the database already has a commit [19:47] (typically) [19:47] bjf: no as in 'no, you are not doing it'? [19:47] kees, the issue is no different in my mind whether it was a commit two releases ago or this one now [19:47] jdstrand: no, UCT will have upstream not stable commit id [19:48] jdstrand, no we are not reviewing all the patches from stable upstream releases [19:48] bjf: the difference is that if the CVE is known to the world now, and it has been fixed now, it must be announced as fixed now. [19:48] bjf: if it was fixed in the past, and the CVE identified now, then we can only update the historical USN [19:48] kees, it was fixed in the past, known in the past but we (the kernel team) had not gotten to it yet [19:48] bjf: I am not asking for review of every commit. I am saying look at the open CVEs which have commit ids. correlate those to what was just pulled in [19:49] kees, therefor we don't know that it was fixed in a stable release [19:49] jdstrand, you are asking us to review every cve that we have not already applied against every stable upstream release [19:50] bjf: yes? that is how one determines if that release is affected by a CVE [19:50] and that is something you need to know anyway, for what to fix [19:50] bjf: correct. "had not gotten to it yet" is the problem. in the past, CVE triage always happened before stable updates, so all CVE fixes were already identified prior to do the stable patch series. [19:50] * jdstrand feels like he is missing something [19:54] bjf: I realize that during the 'catch-up phase' there needs to be some understanding that there will be accounting issues. I get that. However, the only way I see to make sure that things don't go awry is if triage happens before stable release commits. if that happens, then it is easy to add the non-automated changelog stanza I mentioned [19:54] kees, i don't know how CVE triage was done in the past, i know that i do it by trying to apply the patch and if it fails, looking at why [19:55] kees, bjf I would say it never was guaranteed. Just because application of patches was slow with all the review and limited resources, it may have ended up that way [19:56] stable release updates come in asynchronous to any work we do with CVEs and will arrive after some and before others [19:56] bjf: well, that works for isolated commits to prior releases. but it doesn't work for wholesale stable updates from upstream, because they are bug fixes *and* security fixes [19:56] I'm just saying we have a responsibility to report what was actually fixed. [19:57] I mean, at some point, someone has to say-- "this doesn't affect maverick because it is already part of the stable release patches" [19:58] kees, Would it not be possible to scan a cve file when moving it to inactive and check for the released status? [19:58] we do that. At the moment we triage a CVE and try to apply it to a kernel, we learn whetehr it has been previously applied [19:58] jdstrand, and i'm saying i find out it doesn't affect maverick because it is already part of a stable release patches because i just tried to apply it and i see that it already has been patched [19:58] since users need to know if their kernels are vulnerable, not to mention comparisons of our kernels ot other distros and 3rd parties like ksplice, why can't the applying of the stable patches be done in accordance to CVE triage, if CVE triage is out of date [19:59] There is no mapping of stable upstream commits to CVEs [19:59] bjf: perhaps the answer is do do CVE traige immediately after applying the stable patches? [19:59] There is a mapping of CVEs to Linus's commits [20:00] there has to be a correlation between Linus' commits and the upstream stable commits [20:00] jdstrand, if that was what we were doing, there would be no backlog of CVEs needing to be worked and all CVEs would be processed before we see stable updates [20:01] bjf: I'm sorry, I don't follow you [20:02] jdstrand, i think the disconnect is what you call "cve triage" and how we do "cve triage" [20:03] bjf: there is initial triage and 'for reals' triage [20:03] bjf: for a non-kernel update, our team gets the initial CVE, see it probably affects our releases, and gets a priority [20:04] bjf: someone is assigned to it and they dig down and see what releases it really affects [20:04] bjf: and then we fix, etc, etc [20:04] bjf: we do the intial, you guys do the 'for reals' [20:04] look [20:04] jdstrand, i only do "for reals" triage and I do it by applying a patch if the patch applies, i update a tracking bug, post the patch and the patch goes in [20:05] bjf: yes [20:05] bjf: why can't the changelog be a part of that? [20:06] jdstrand, when I "apply" a patch, i first take the upstream commit, and add the necessary text so that when it gets applied, and we run "insertchanges" it gets picked up [20:06] jdstrand, that will add it to the changelog [20:07] our changelog is generated just before building our package, by a script that processes the git log. It isn't a manual incremental process [20:07] bjf: but you aren't adding the CVE reference. you said that [20:07] jdstrand, that's not what i said [20:07] jdstrand, i said i don't go back and add it for stable release commits [20:07] alright, when you do 'for reals' and send it up, you ignore whatever it doesn't apply to [20:08] I see that [20:08] jdstrand, oh and by the way, since we now get stable updates at different frequencies for different series, lucid may get a cve fixed before maverick [20:08] but, this is a real problem. it isn't us just whining and asking for busy work [20:09] eg, ever since Ubuntu issued security updates, the changelogs include CVEs references. this dates back to before Ubuntu far back into Debian [20:09] people look at changelogs to decide if the update is worth upgrading for [20:09] many people won't apply updates without CVE references in them [20:09] so people remain vulnerable because they don't know they need to upgrade [20:10] (or won't reboot into the new kernel without them) [20:10] jdstrand, and it still includes CVE references, we just don't guarantee that it's for all the CVEs that were fixed in a particular release (and we never did guarantee it) [20:10] the USNs try to help with that, but we don't know what was fixed because the CVEs aren't in the changelog [20:10] bjf: I understand it will have some. I understand you are saying you never guaranteed that. I am saying that is a problem [20:10] But it was the case before that this happened [20:10] all other packages do this [20:11] before is neither here nor there [20:11] the problem seems exacerbated lately, but if it was broke before and now, it still needs to be fixed now [20:12] jdstrand, it is "worse" now because of the two week cadence and that most releases now have at least one cve in them and you are looking closer [20:12] I'm not sure what the fix is, and I want to reiterate that the work being done is not lost on me. I very much appreciate it and have been quite public about prasing all the hard work you guys have been doing [20:12] (to management and others) [20:13] (and that includes your manager_ [20:13] ) [20:14] upstream stable commits are not going into -proposed without other SRU or CVE bugs are they? [20:14] jdstrand, yes, we take them wholesale [20:14] yes [20:15] with or without, whenever they land we take them [20:15] so it is possible that a kernel will hit -proposed with no open CVEs or SRU bug fixes? [20:15] jdstrand, that is correct [20:15] ok [20:15] theoretically yes [20:16] so, after everything is caught up, there won't be such a backlog of CVEs [20:16] and it's almost happened - we've had kernels with dozens of patches and only one open bug that needed verification [20:17] therefore, it should not be hard to then apply the stable commits, then go through the open CVEs and try to apply the patch. this happens before the upload to -proposed. then you can add an non-automated 'CVE stanza' before uploading to -proposed [20:18] this is not 'extra work' because this is work that you will be doing anyway [20:18] jdstrand, then there would be no backlog of cves right ? [20:18] except that we'd have to go through ALL the unapplied CVEs [20:18] sconklin: I said 'after everything is caught up' [20:18] when pigs fly [20:18] bjf: there isn't a 'backlog', but there might be open CVEs [20:19] jdstrand, i don't see a day when there won't be a backlog [20:19] Despite the rapid rate at which we're working, I don't see the backlog getting a whole lot smaller [20:20] bjf: define backlog. I am talking about all the CVEs that didn't get fixes. it is my understanding that these are getting fixed and people are helping out [20:20] jdstrand, http://people.canonical.com/~apw/cve/pkg/linux.html [20:20] sconklin, bjf: you need to talk to your manager if you can't catch up. if you can't catch up, any hiccup means you can't keep up [20:20] jdstrand, everything on that page that says "needs-triage" is backlog [20:20] bjf: yes, I am familiar with that page [20:21] bjf: you have already gone through almost all the 2010 CVEs [20:21] jdstrand: we're working on that as a separate issue ;) [20:21] jdstrand, you asked me to define backlog and i just did [20:21] there is one outstanding 2011 CVE on that page [20:21] bjf: k [20:21] I have been very encouraged by how much you guys have been catching up [20:22] that page looked much worse 10 days ago [20:22] we are getting caught up on the wrong things [20:22] jdstrand, so, we are making progress, that has nothing to do with what started this discussion [20:22] jdstrand: is it the case that for every CVE we care about that's in yoru CVE tracker, we know the upstream commit ID? [20:23] if part of the work is triaging a CVE by trying to apply it to a branch, then when you do that isn't 'extra work', it is 'rescheduled work' [20:23] bjf: btw, given that the sru-report is empty for dapper hardy karmic, why are there "pending" on http://people.canonical.com/~apw/cve/pkg/linux.html ? [20:23] sconklin: not always, that assumes there is a fix [20:24] kees, no idea [20:24] sconklin: or assumes that the location of this fix was known during triage [20:24] therefore, if you reshedule CVE triage to 'right after' you pull stable upstream commits, you can both adjust the changelog, accurately reflect the state of that kernel, etc, etc [20:25] kees, i'm assuming we are not merging with your updated active files [20:26] kees: isn't it now part of our process to push our updates to the kernel team? [20:26] kees: (our new CVEs) [20:26] bjf, jdstrand: we pull the kernel team tree daily. [20:26] right [20:26] s/daily/at every CVE triage cycle, which is supposed to be daily/ [20:26] jdstrand: yes, but CVE triage has now been redefined to mean 'review every open CVE, determine the upstream commit identified with it, and see if we applied the same patch' [20:27] kees: right, but how often does the kernel team pull from our tree? [20:27] sbeattie: not sure [20:29] sconklin: ok. whatever you call it, can't you just do what I suggested for trees that pull upstream stable updates? it would completely solve the problem for those kernels, and wouldn't hurt the full triage done for the others. plus, you could conceivably add that commit id to the tracker as a pointer on where to look later, so it could make it easier down the road [20:30] I really don't see how that adjusted workflow would adversely affect you guys. that work has to be done regardles [20:31] I'm questionin the premise that this must be done [20:31] jdstrand, that is maybe true, but you are adding an ordering constraint which does not exist currently [20:31] sconklin: see backscroll for users trying to decide on updating. USN publication, 3rd parties like ksplice. accurate accounting. comparison with other distros [20:32] it is an important change [20:32] people are confused [20:32] And I'm sorry, I'm not intending to avoid discussion, but I have four more kernel packages I have to get built today so I'm going to have to get to those [20:33] how do you do it for packages where you do not control uploads [20:33] apw: not to mention, it is the same people applying upstream stable commits as fixing CVEs: the kernel stable team [20:33] when grub2 gets a 'sync'd with debian' how do you ensure it has USN numbers in it before it is uploaded [20:33] jdstrand, how about someone from security team doing a rotation onto the kernel team ? [20:34] apw: we don't sync from Debian for stable releases. we backport patches. we don't write USNs for the devel release. the devel release is ok to have only 'released' or 'not-affected' in the CVE [20:34] what i don't see is how it differs from when a CVE is discovered after the fix was accidentally applide to natty [20:34] or any release [20:34] bjf: I'm not sure where you are going with that [20:35] there the fix preceeds the USN, why can we not postumously indicate a previous release has a fix [20:35] apw: because people don't run the devel release in production? [20:35] as we triag and find the fix, and mark it released (xxx) [20:35] ignore the word natty, it just came out [20:35] jdstrand, i'm suggesting that that to properly understand each others point of view and the work involved, perhaps someone should do a rotation onto the kernel team [20:35] so if we apply .11 and the next week you tell us about a cve that was already released [20:36] how do we handle that, that must be a situation we have a process for [20:36] apw: it's two situations, a CVE being identified months after a fix is published already and a fix being applied when the CVE is already known. [20:36] bjf: no trying to be daft, but we have worked very closely with the kernel team for years on this [20:36] the latter is what needs to be fixed, as it is causing a lot of confusion now since merging stable and security updates [20:36] kees, well i'd say if its not been triaged, its not know yet [20:36] apw: that might be true for you, but for people who watch public CVEs, this is not true. [20:36] till thats done we don't know if the random sha1's in the CVE report even are the right fix [20:37] apw: it looks like Ubuntu is "silently fixed" CVEs [20:37] *fixing [20:37] bjf: sometimes the patching, etc was more on us, and sometimes on the kernel team. there are too many kernels for us to maintain [20:37] jdstrand, ah, nevermind, thought we were in disagreement about something, guess i was wrong [20:37] kees, right but that does happen, we sometimes do pull in a fix for a bug and push it [20:37] without knowing it also fixes a cve [20:37] it is bound to happen isn't it ? [20:38]