| htorque | ogasawara: hi! https://bugs.freedesktop.org/show_bug.cgi?id=41059 - there are two new patches to test in comment 31 and comment 34. can you maybe build two new kernels for soren to test (unless he wants to build them himself)? | 01:20 |
|---|---|---|
| htorque | soren: ^^ | 01:20 |
| ubot2 | Freedesktop bug 41059 in Driver/intel "XRANDR operations very slow unless (phantom) HDMI1 disabled" [Major,Assigned] | 01:20 |
| maco | bjf: you're quick | 03:47 |
| === jibel_ is now known as jibel | ||
| smb | Dry morning .+ | 07:01 |
| abogani | morning | 07:10 |
| brendand | where do i find tar files with the vmlinuz and initrd for different ubuntu kernels? trying to do a live CD customization | 08:45 |
| jjohansen | brendand: tar files? You can find the packages and pull them appart, the source packages and pull them appart, the kernel git tree | 08:47 |
| brendand | what i'm really looking for then is the package with the proposed kernel | 08:53 |
| brendand | for natty, that is | 08:55 |
| jjohansen | brendand: specifically the proposed kernel package or just the kernel source? git is the easiest way to get that | 09:19 |
| brendand | jjohansen - i'm just trying to test a fix, so i only need the package | 09:21 |
| jjohansen | brendand: and you only want the kernel package not the rest of packages in -proposed? | 09:22 |
| jjohansen | brendand: also i386 or x86-64 | 09:24 |
| brendand | jjohansen - definitely just the kernel. it's a fix for an install issue with one system, so i need to do a live cd customization to include the proposed kernel. the steps recommend just copying over the original vmlinuz and initrd with the new ones | 09:24 |
| brendand | jjohansen - it's i386 | 09:24 |
| jjohansen | brendand: oh also which release, natty? | 09:25 |
| brendand | jjohansen - it is, but i'm doing the customization on an oneiric box | 09:27 |
| jjohansen | brendand: well there is no oneiric-proposed kernel yet, so you are you looking for the natty-proposed kernel | 09:28 |
| brendand | jjohansen - absolutely. i'm just inferring that i don't (think i) have the option to just enable -proposed and install it | 09:29 |
| jjohansen | brendand: http://us.archive.ubuntu.com/ubuntu/pool/main/l/linux/linux-image-2.6.38-9-generic_2.6.38-9.43_i386.deb | 09:32 |
| jjohansen | brendand: you can find them by going to http://us.archive.ubuntu.com/ubuntu/dists/ | 09:32 |
| jjohansen | clicking through to the package set natty-proposed | 09:33 |
| brendand | jjohansen - duly bookmarked. thanks | 09:33 |
| jjohansen | natty-proposed/main/binary-i386/Packages.gz | 09:33 |
| jjohansen | unzip and look for linux-image in the Packages text file. It will give you the pool address | 09:34 |
| jjohansen | eg. pool/main/l/linux/linux-image-2.6.38-9-generic_2.6.38-9.43_i386.deb | 09:34 |
| jjohansen | and then you need to prepend which archive mirror, ... | 09:34 |
| jjohansen | its not as easy as it should be, you can use packages.ubuntu.com for updates, backports, and for release packages but not proposed | 09:35 |
| === arun_ is now known as arun | ||
| ogasawara | apw: just so someone else knows what I've done, per the request of the release team I'm uploading just the fix for the linux-image-extra's install dependency issue | 13:48 |
| ogasawara | apw: only they've requested I upload to -proposed, that way if it takes too long to build, they'll just let it sit in -proposed and promote to -updates as a day 0, otherwise they'll promote to the actual release pocket. | 13:49 |
| apw | ogasawara, ack, and i assume that the linux-meta changes went up too | 13:49 |
| ogasawara | apw: yep, linux-meta went up yesterday | 13:49 |
| apw | ogasawara, sounds good, sorry for the mess | 13:49 |
| apw | ogasawara, also once its in the queue get them to score it up | 13:50 |
| ogasawara | apw: no worries, like you said it's an elective install, I'm actually surprised they just don't prefer to wait for the first SRU | 13:50 |
| apw | it may be making errors appear in their builds, though why we didn't find out about it sooner from those same errors i am unsure | 13:51 |
| ogasawara | apw: yah, I'm indeed surprised it went undetected until yesterday | 13:51 |
| apw | also they seem to have a lot of other problems meaning they need a rebuild mondayish anyhow | 13:51 |
| ogasawara | apw: yep, I think that's the plan. | 13:51 |
| apw | we release unity updates on mondays too :) | 13:52 |
| brendand | bjf, sconklin - you free to talk testing? | 14:26 |
| bjf | brendand: yup, i'm here | 14:26 |
| * ogasawara back in 20 | 14:30 | |
| * roadmr also here for the testing thing | 14:30 | |
| brendand | bjf, sconklin - so, as i mentioned yesterday we plan to do as much as we can next cycle to improve the SRU test suite that we run in our certification labs | 14:30 |
| brendand | bjf, sconklin - a major area we've neglected so far has been wireless testing. which is a big oversight, but we're on it now and we want to make sure our efforts are as worthwhile as possible | 14:31 |
| brendand | also audio testing has been minimal and graphics testing is okay, but not as good as it should be (admittedly graphics testing is tricky to do automatically) | 14:32 |
| brendand | obvious points to address in wireless testing are scanning, and some sort of connectivity test | 14:34 |
| bjf | brendand: as i mentioned yesterday, i look at QA testing as being fewer systems involved but deeper testing and cert. as being more shallow testing but accross more systems | 14:34 |
| bjf | brendand: and i think you agreed | 14:34 |
| brendand | bjf - exactly, so we're not trying to cover every possible use case, just the most important ones | 14:34 |
| bjf | brendand: i think it would be a *huge* win if tests can be shared between cert. and QA | 14:34 |
| sconklin | brendand: I'm here, sorry I'm late | 14:35 |
| brendand | sconklin - no problem | 14:35 |
| bjf | brendand: for wireless, what are you asking / thinking ? | 14:35 |
| bjf | brendand: we agree it needs testing | 14:36 |
| brendand | bjf - you said yesterday it was one of the areas where you most commonly saw regressions | 14:36 |
| brendand | bjf - the nature of the regressions, was it mainly total breakage of the wireless? | 14:37 |
| bjf | brendand: it's the area where regressions can have a major impact on users, people are just so dependent on wireless these days | 14:37 |
| sconklin | one way that wireless problems show up is in low throughput or in repeated disconnect/connect. | 14:38 |
| bjf | brendand: one area is rfkill, there occasionally seem to be changes that cause rfkill to stop working or work in unexpected ways | 14:38 |
| sconklin | Throughput can be tested automatically, not sure about the other, unless by log inspection | 14:38 |
| sconklin | ack on rfkill | 14:38 |
| bjf | for wireless, you need to test scanning, association, throughput | 14:39 |
| brendand | sconklin, bjf - excellent. do you know are QA already testing throughput? | 14:39 |
| bjf | it would be nice to test suplicants | 14:39 |
| sconklin | don't know | 14:39 |
| hggdh | brendand: no, we are not testing wireless at all | 14:39 |
| bjf | don't think so, but there should be some QA folks here | 14:39 |
| hggdh | (except for ad hoc testing) | 14:40 |
| brendand | okay, so wireless regressions are wholly the responsibility of the community? | 14:40 |
| gema | hggdh: do we have any ethernet testing that we could leverage on to automate some wireless testing? | 14:40 |
| brendand | (not causing them obviously. catching them) | 14:40 |
| hggdh | not yet, no | 14:40 |
| bjf | brendand: yes | 14:41 |
| brendand | bjf - i was thinking including different security tests, e.g. WPA, WEP and no security | 14:41 |
| bjf | brendand: another area is we get reports that "with latest update/upgrade my wireless is 50% slower" | 14:41 |
| brendand | bjf - we have a bandwidth test, but obviously it's only wired at the moment and probably could be improved | 14:42 |
| roadmr | hm, trending is something we don't do at the moment, but it looks like it would be useful in this case | 14:42 |
| bjf | brendand: the wireless can be tricky because it depends on how noisy the wireless environment and how good the AP is the bug submitter is talking to | 14:42 |
| brendand | bjf - our testing environment would be *extremely* noisy | 14:43 |
| roadmr | yes I see a problem if all systems under test start slamming the AP at the same time | 14:43 |
| gema | bjf: but in a testing lab, all those things are sort of controlled | 14:43 |
| brendand | gema - i would say the opposite in ours | 14:43 |
| gema | or at least, consistent | 14:43 |
| brendand | gema - theoretically it could be, but since we are testing so many systems at once | 14:43 |
| bjf | gema, depends on your setup | 14:43 |
| brendand | gema - and they all have different speeds, an unknown % of them might be exercising the wireless at a given point | 14:44 |
| bjf | gema, also some of the wireless nics don't work as well in very noisy environs like conferences with hundreds of attendees | 14:44 |
| sconklin | In think that in general, trending is something we should strive for in any metric that makes sense. Boot speed, benchmarks, network, file systems. But I think that falls more under QA 'deep' testing than cert 'broad' testing | 14:44 |
| brendand | gema - we could control it but that would involve some clever server side synchronisation | 14:44 |
| bjf | gema: that's actually good testing (of a sort) | 14:45 |
| * gema is thinking faster than she can write | 14:45 | |
| brendand | sconklin - being able to detect trends would be great, again we'd need to invest a lot of time in infrastructure and results analysis to acheive that across all our systems though | 14:46 |
| bjf | brendand: gema: we talked to the intel wireless folks this week, they are very interested in working with us on better testing for the community as well as upstream developers | 14:46 |
| gema | bjf: brendand: I am thinking we probably should put some sort of test environment together in our new lab for this testing | 14:46 |
| roadmr | bjf: do the intel folks have testing tools we could use? | 14:46 |
| brendand | gema - the new lab is in Lexington, right? | 14:46 |
| gema | brendand: yep | 14:46 |
| brendand | gema - half our hardware is there, approximately | 14:47 |
| bjf | roadmr: some, yes, and we will be talking to them about those, and we'll be sure to include cert. and qa | 14:47 |
| sconklin | brendand: simply collecting the data and presenting it graphically in a report might be cheaper, and then humans can spot anything worrisome. That might be low hanging fruit | 14:47 |
| bjf | sconklin: that's an interesting idea, like we are doing with boot-speed testing right now | 14:48 |
| brendand | sconklin - the question is if we have the bandwidth in this cycle. i'll make a note of it and perhaps we can brainstorm something between ourselves and QA | 14:48 |
| roadmr | bjf: awesome, thanks, proven tools will be quite helpful | 14:48 |
| brendand | i'd like to interject though that our first goal needs to be fill in the obvious gaps | 14:48 |
| hggdh | more than proven tools, an idea of how they setup the environment will give us clues | 14:48 |
| sconklin | +1 | 14:48 |
| gema | brendand: are you in london next week? | 14:49 |
| sconklin | hggdh: I don't understand what you mean | 14:49 |
| brendand | gema - no, UDS is my next destination | 14:49 |
| bjf | roadmr: http://linuxwireless.org/en/developers/Testing/wifi-test I have *no* idea if this is used at all or not | 14:49 |
| gema | brendand: ok, so let's talk about it at UDS | 14:49 |
| roadmr | hggdh: that'd be good, although given the bulk nature of what we do in cert, that might not be feasible for us (i.e. can't put a faraday cage around the lab or isolation-test 100+ systems) | 14:49 |
| hggdh | sconklin: I am worried with noise -- we would need a few APs, and tens of machines will be chatting at the same time. As a result tests will potentially be contamined unless we set it right | 14:50 |
| brendand | roadmr - +1 | 14:50 |
| bjf | roadmr: hggdh this could be a good place where the testing that QA does is different that the cert. testing, cert can be in a noisy environ. QA in a "quiet" environ. ? | 14:50 |
| brendand | can we agree to discuss this topic at UDS and focus on purely functional tests for now? | 14:51 |
| gema | brendand: +1 | 14:51 |
| brendand | we have two more topics to discuss at well | 14:51 |
| hggdh | bjf: +1 | 14:51 |
| gema | bjf: +1 | 14:51 |
| hggdh | brendand: +1 for going ahead | 14:52 |
| hggdh | (and flesh it out at UDS) | 14:52 |
| brendand | i think we have a really good base for improving wireless testing now, so i'll be adding these notes to the blueprint and everyone is welcome to subscribe: https://blueprints.launchpad.net/certify-planning/+spec/hardware-p-cert-sru-coverage | 14:52 |
| hggdh | just a Q -- sconklin, bjf: is there any plans for meeting with Intel soon? | 14:52 |
| brendand | let's talk video/graphics testing | 14:52 |
| sconklin | brendand: what's your definition of 'functional test'? | 14:53 |
| brendand | sconklin - one where we don't depend on previous results or heuristics to determine a results. it either works or doesn't | 14:53 |
| sconklin | would that include a test in which we want to insure that network bandwidth meets a certain lower bound? | 14:54 |
| gema | sconklin: if there is a value defined for that lower bound, yes | 14:54 |
| brendand | sconklin - if we can say for sure what that lower bound is and it doesn't depend on environmental factors then yes | 14:54 |
| bjf | hggdh: we just had our twice yearly meeting with them | 14:55 |
| roadmr | sconklin: we have a test like that for ethernet bandwidth, it gives false positives often because throughput ends up depending on network load | 14:55 |
| roadmr | sconklin: I think that kind of info would be more useful if collected and analyzed for trends than for pass/fail tests | 14:55 |
| brendand | roadmr, sconklin - also for disk read speed. we put an arbitrary lower limit of 30MB p/s and one system achieves only 29 | 14:56 |
| sconklin | ok, then I think I'm clearer on the limitations of what's under discussion today. Thanks | 14:56 |
| brendand | as i said, video/graphics | 14:57 |
| brendand | what we do at the moment is quite simplistic. check if x is running. check if compiz *can* be run (not if it is) | 14:58 |
| brendand | we also take a screengrab and anlayse those in a batch. but the tool used is not 100% reliable | 14:58 |
| brendand | s/anlayse/analyse/ | 14:58 |
| sconklin | brendand: do you test external monitors? | 14:59 |
| sconklin | Do you test hotplugging of those? | 14:59 |
| bjf | multi-monitors ? | 14:59 |
| brendand | sconklin - no, and we don't, unfortunately | 14:59 |
| brendand | not in the scope of this. we'll have to achieve testing of that in a different way | 14:59 |
| sconklin | but that's a major area where we see problems, and don't we certify that behavior? | 15:00 |
| brendand | sconklin - we do. but certification is done over 2 weeks | 15:00 |
| sconklin | oh, right, that's the full cert, and what we're talking about here is the quick test that's done for SRU kernels | 15:00 |
| roadmr | sconklin: we just don't test it during the automated SRU and/or weekly tests | 15:00 |
| brendand | sconklin - so we would have a 2 week regression-testing phase for each SRU and the 7 person hwcert team would have no other role but testing SRUs | 15:00 |
| sconklin | problem is, these are problems that show up as regressions in SRUS. | 15:01 |
| brendand | sconklin - yep. really, no manual tests | 15:01 |
| bjf | brendand: that works for us, sold! | 15:01 |
| roadmr | heheh :) | 15:01 |
| brendand | bjf - oh forgot </sarcasm> | 15:01 |
| brendand | :) | 15:02 |
| bjf | brendand: that bit is always flipped on me | 15:02 |
| bjf | brendand: i really don't know how to give guidance on video testing, i think you need to talk to the xorg guys | 15:02 |
| brendand | bjf, sconklin - actually the lab engineers (like roadmr) already have enough unavoidable manual work | 15:03 |
| bjf | brendand: our issue is that we *do* see regressions there and they can be really painful | 15:03 |
| brendand | e.g. powering on laptops, pressing keys to go through bios screens, resuming systems from suspend | 15:03 |
| sconklin | I'm trying not to focus on your processes, but on which actual failures we have seen which aren't currently covered by cert testing. | 15:04 |
| brendand | sconklin - that's good, but i need to point out when something won't be feasible | 15:04 |
| sconklin | I shouldn't have to worry a lot about how you do your testing, only whether the results are valuable to us :-) | 15:04 |
| sconklin | so take it or leave it, but I want for you to know where we see problems | 15:05 |
| bjf | brendand: i'd like to change that "not feasible" to "cert. is unable to perform that testing on sru kernels today" | 15:05 |
| roadmr | yes, well if we're in brainstorm mode anything goes, determining whether it's feasible or not can come later | 15:05 |
| roadmr | sconklin: it's really useful to know that's a regression-prone area | 15:05 |
| sconklin | ack +1 | 15:05 |
| bjf | brendand: that way, QA can pick up testing that cert. can not do or we identify testing that could be done sometime in the future | 15:06 |
| brendand | bjf - true. 'not feasible' is always qualified with 'at this time, with the resources we have' | 15:07 |
| sconklin | ok, we sidetracked while talking about grahics testing . . . | 15:07 |
| hggdh | I would rather go, then, in brainstorm mode, and later on decide what can, or cannot be done right now, in 6 months, in 1 year, etc | 15:07 |
| hggdh | but we need to know what the kernel team see as problem areas | 15:07 |
| sconklin | hggdh: agreed, talk about what we want to do then figure out how much it would cost, time frames, etc later | 15:07 |
| bjf | so ... during the dev cycle we can see a lot of churn in the upstream drivers themselves which cause problems | 15:08 |
| brendand | agreed. i'll put notes of things that might be worth brainstorming like the external monitor and wireless speed trends | 15:08 |
| bjf | once a kernel goes "stable" we see less churn, we see more 'quirking' to enable graphics for specific hw | 15:08 |
| brendand | i shall be inviting bryce to the session, to get some input on graphics testing | 15:09 |
| bjf | however, some of this 'quirking' can impact systems other than those it was intended to target | 15:09 |
| brendand | outside of external monitors, what else is there? | 15:09 |
| bjf | and there are some, more general, bug fixes that also impact multiple systems | 15:10 |
| brendand | bjf - how do these bugs manifest themselves, is it usually corruption? | 15:10 |
| bjf | often it's they boot to a black screen | 15:10 |
| === kamalmostafa is now known as kamal | ||
| sconklin | brendand: failures generally occur either at boot or at login, based on my memory | 15:11 |
| hggdh | or on resume | 15:11 |
| brendand | and the symptom is mostly a black screen | 15:11 |
| sconklin | yes, no graphics at boot is the most common, along with "sparkles", "lines" or other symptoms of incorrect timing. | 15:11 |
| bjf | brendand: often yes | 15:11 |
| bjf | brendand: can you explain "batch" testing of screen shots ? | 15:12 |
| brendand | bjf - the imagemagick 'import' command is used to do a capture (which as i mentioned may not be reliable). this is stored with the test results | 15:13 |
| brendand | bjf - and we have a script which downloads all of these into a single directory and i just browse over them and check for blank ones or ones with graphical glitches. takes about 5 minutes | 15:14 |
| bjf | brendand: thanks | 15:14 |
| brendand | bjf - this week we found one blank one when testing the maverick SRU. turns out the system failed to resume from suspend (not a regression, it was like that since release) | 15:15 |
| bjf | brendand: sidetracking a bit here ... do you do suspend/resume testing ? | 15:15 |
| brendand | bjf - basic suspend/resume testing. i.e. make sure the system still can | 15:16 |
| bjf | brendand: that causes major issues with everything we've been talking about, wireless, graphics audio | 15:16 |
| brendand | bjf - i would like to expand that to include testing subsystems after suspend | 15:16 |
| bjf | the system will rfkill a device on suspend but not restore it on resume | 15:16 |
| brendand | bjf - we'll probably include some element of post-S3 testing for each area | 15:17 |
| bjf | the graphics worked perfectly fine before suspend but it resumes to a black screen | 15:17 |
| brendand | audio, bluetooth, graphics, wireless | 15:17 |
| bjf | and so forth | 15:17 |
| brendand | etc. | 15:17 |
| sconklin | Yeah, I think just by tatking what we have and also running it all post-resume, we gain a lot | 15:17 |
| sconklin | taking | 15:17 |
| bjf | sconklin: +1 | 15:17 |
| bjf | sorry to throw yet another thing it ... do we do any hybernate / resume testing ? | 15:18 |
| brendand | bjf - no. isn't that a dodgy area? i.e. not all systems support it and that's just one of those things? | 15:19 |
| bjf | and we should probably move on to audio | 15:19 |
| brendand | yep | 15:19 |
| bjf | brendand: yes, thought i'd ask though | 15:19 |
| brendand | before that though, just one question | 15:19 |
| sconklin | I've heard a lot of factoids thrown around like "only server people care about it", but never any good data to back up who actually cares, or what the use cases are. I know we get bug reports for it, so some people use it. | 15:20 |
| sconklin | hibernate, that is | 15:20 |
| brendand | bjf, sconklin - to get an idea of the suitability of the screen capture tools we're using, it would be good to know when one of these regressions is seen (hopefully on a system we can get access to) | 15:20 |
| brendand | i want to know if we've maybe been missing stuff | 15:21 |
| bjf | brendand: noted | 15:21 |
| brendand | onto audio anyway | 15:21 |
| herton | also about suspend/resume, do we have any stress testing? like suspend/resume 100 times, some problems are intermittent, and only appear after some iterations | 15:21 |
| sconklin | noted, we can get some historical data I think | 15:21 |
| brendand | herton - no, we even are struggling with the problem of how to achieve that in our certification | 15:22 |
| brendand | herton - i'll note it down though | 15:23 |
| brendand | so, audio | 15:23 |
| bjf | brendand: cking is worth talking to about suspend/resume testing (if you haven't already) | 15:23 |
| bjf | if diwic is around maybe he can speak to the kinds of regressions he sees | 15:24 |
| brendand | bjf - not directly in relation to SRU testing | 15:24 |
| bjf | and the kinds of testing that would be helpful | 15:24 |
| cking | yep, I'm happy to discuss this with you brendand | 15:24 |
| roadmr | we've had a look at cking's fwts testing, but it may be worth revisiting | 15:24 |
| brendand | we have a couple of fwts tests in there | 15:24 |
| * diwic is around | 15:24 | |
| cking | especially with the improved goodness of the Oneiric features | 15:24 |
| brendand | cpu_scaling and wake_alarm | 15:24 |
| brendand | cking - will you subscribe to the blueprint? https://blueprints.launchpad.net/certify-planning/+spec/hardware-p-cert-sru-coverage | 15:25 |
| bjf | brendand: for audio you want to test internal and external equally | 15:25 |
| cking | sure | 15:25 |
| brendand | diwic - you're discussing regressions with audio in -proposed | 15:25 |
| bjf | usb speakers would be nice (yes i'm completely ignoring if these can be automated or not) | 15:26 |
| bladernr | brendand: we could also add the other fwts test as it's full auto and runs most of Colin's automated tests | 15:26 |
| diwic | For for testing audio regressions in general I'd just say test playback and recording of various inputs and outputs, the more the better. | 15:26 |
| brendand | diwic, bjf - so if we can cover record/playback on internal, external mic and usb then it's all good? | 15:27 |
| brendand | usb will be most difficult i think | 15:27 |
| brendand | external we can use a patch cable | 15:27 |
| bjf | brendand: that would be pretty good, yes | 15:27 |
| bjf | brendand: lots of bluetooth headsets in use these days | 15:27 |
| diwic | brendand, yeah to a reasonable degree...I mean, you could go on by testing volume controls, low-latency / high-latency scenarios | 15:28 |
| brendand | diwic - if the audio breaks does it usually break completely? testing audio quality might be tricky | 15:28 |
| cking | so are we ultimately hoping to be able to write per-SRU tests to catch regressions? | 15:28 |
| bjf | brendand: diwic is the domain expert for audio | 15:28 |
| brendand | diwic - if you could attend the session about this at UDS that would be great (link just a little but up in the scrollback) | 15:29 |
| diwic | brendand, that's a good question. In general I think we haven't that many regressions in SRUs for audio in the past - do you agree bjf? | 15:29 |
| diwic | brendand, ok, added myself to blueprint | 15:30 |
| brendand | i think we've got a load of good ideas together now | 15:30 |
| brendand | i'll sit down on monday and try and sift through this | 15:30 |
| diwic | brendand, maybe testing sound after suspend/resume could make sense as well | 15:30 |
| bjf | brendand: yah, more than an hr is just asking for trouble (no sarcasm) | 15:31 |
| brendand | diwic - yeah. a question there though. if the mixer settings get changed after suspend, is that a proper problem? | 15:31 |
| hggdh | actually we should test sound, video, and network on resume | 15:31 |
| sconklin | yeah, this is a good start | 15:32 |
| brendand | hggdh - that's the plan. perhaps bluetooth too | 15:32 |
| hggdh | brendand: yes, bluetooth, I forgot it | 15:32 |
| diwic | brendand, hmm, I think it is, but minor in the sense that if it just happens to one of the machine, it should be fixed, but maybe it is not enough to block an SRU with fixes to thousand of users | 15:32 |
| brendand | actually, i don't know will we be able to address it much, but at least one functional test for bluetooth will be nice (before and after suspend) | 15:33 |
| bjf | brendand: for bluetooth, you want to actually pair to a device | 15:33 |
| brendand | diwic - at the moment i think we already have certified a lot of systems which won't keep the mixer settings after suspend (roadr, bladernr?) | 15:33 |
| gema | brendand: do you do targetted testing for each SRU or do you run all the test cases on each or all the test cases on a mix of SRUs? | 15:34 |
| bjf | brendand: and be able to do that after resume | 15:34 |
| bladernr | brendand: yeah, that's a pain point, personally, but we have | 15:34 |
| brendand | bjf - we would plan to include a file transfer. we're doing this automatically in cert so it shouldn't be too hard | 15:34 |
| bjf | brendand: nice | 15:34 |
| brendand | gema - same test suite on each one | 15:34 |
| gema | separately? | 15:34 |
| gema | and then together? | 15:34 |
| brendand | gema - seperately what? | 15:35 |
| diwic | brendand, interesting. I was not aware of this problem (and haven't seen loads of bugs about it either) | 15:35 |
| gema | I am trying to figure out whether the SRUs would interfere with each other and in which order you test them | 15:35 |
| roadmr | brendand, diwic: yep, as long as audio does work, we don't care that much about the mixer going up/down after resume | 15:35 |
| diwic | roadmr, it feels like one of all minor annoyances we should fix for the P cycle | 15:36 |
| brendand | gema - i'm not quite sure what you're trying to say | 15:36 |
| gema | brendand: no worries, I will ask offline | 15:36 |
| brendand | gema - sorry | 15:36 |
| roadmr | diwic: yep, it's annoying, a papercut if you will | 15:36 |
| hggdh | cking: asnwering you question about per-SRU tests: ideally, we should have a collection of tests that grow as time goes by, regarding regressions | 15:37 |
| brendand | hggdh - we can't afford to let our test suite grow unchecked | 15:37 |
| hggdh | brendand: indeed, for certification, but not so for QA | 15:38 |
| diwic | brendand, as long as test don't require manual intervention I guess testing is cheap, but for manual tests we should carefully consider every one | 15:38 |
| brendand | hggdh - yeah, you guys feel free ;) | 15:38 |
| hggdh | heh | 15:38 |
| gema | diwic: tests are never for free, they need to be maintained | 15:38 |
| gema | diwic: like any other code | 15:38 |
| bjf | brendand: will you be sending out an edited version of this discussion ? | 15:39 |
| sconklin | gema +1 | 15:39 |
| brendand | bjf - yeah, it's going to be what i do on monday probably. i'll probably attach it to the blueprint | 15:40 |
| diwic | gema, fair point | 15:40 |
| bjf | brendand: ok, look forward to it | 15:40 |
| gema | brendand: looking forward to it too, thanks for all the information :) | 15:41 |
| brendand | regarding the mixer settings, if we got a baseline of systems which can restore them properly than at least we could look for sudden regressions in those | 15:41 |
| brendand | for the ones that could never restore them, we can't really hold SRUs back for that | 15:41 |
| diwic | brendand, makes sense | 15:41 |
| brendand | that's my last though | 15:41 |
| brendand | last thought, that is | 15:41 |
| brendand | thanks for everyone's input. i hereby return ubuntu-kernel to its normally scheduled programing | 15:42 |
| sconklin | move to adjourn | 15:42 |
| sconklin | :-) | 15:42 |
| sconklin | thanks! | 15:42 |
| hggdh | brendand: one of the things we should look for is deviation from "standard" -- i.e. those tests that consistently failed/succeeded and are now succeding/failing | 15:42 |
| hggdh | sconklin: +1 | 15:43 |
| roadmr | thanks everyone, it was really useful, hope to continue this at some point / UDS | 15:43 |
| kamal | . | 16:01 |
| roadmr | .. | 16:02 |
| kamal | no. just dot. ;-) | 16:02 |
| diwic | .... | 16:02 |
| diwic | ........ | 16:03 |
| diwic | I'm getting tired. | 16:03 |
| roadmr | it started as a dot and now we have a progress bar going | 16:03 |
| diwic | Maybe it's time to call it a day. | 16:03 |
| === yofel_ is now known as yofel | ||
| bjf | jsalisbury: your making me do real work! :-) | 18:19 |
| jsalisbury | bjf, heh, sorry bout that | 18:20 |
| Claudio9641 | Hi, I have a problem with the new firewire stack in 11.04 (but also still exists in 11.10 beta2). Where would be the best place to ask for help. Can provide error messages and detailed information. | 19:11 |
| bjf | Claudio9641: you should file a bug, add your error messages and detailed information and then come back here and tell us the bug # | 19:14 |
| Claudio9641 | Problem is: I have a firewire external harddisk which is not recognized automatically by Kubuntu 11.04 when connected. Strange thing: when I boot up a life-CD of 10.04 it works perfectly (old firewire stack). When I then reboot into 11.04, the drive also works in 11.04. | 19:14 |
| Claudio9641 | Hi bjf: file a bug - where? | 19:15 |
| bjf | Claudio9641: from the command line you can type: ubuntu-bug | 19:15 |
| Claudio9641 | Great, now I have a problem with ubuntu-bug. I tell it: "Problem with external storage device" and next window that pops up is asking for "Which audio devicee are you having a problem with?" - Hey, dude, I said 'storage', not 'audio'. Grmpf | 19:19 |
| Claudio9641 | ... and when I press "abort", ubuntu-bug hangs .. great! | 19:21 |
| jjohansen | Claudio9641: ouch, try ubuntu-bug linux | 19:23 |
| jsalisbury | ogasawara, bjf, Regression from 11.04 to 11.10, but does not happen with latest mainline build: | 19:26 |
| jsalisbury | https://bugs.launchpad.net/ubuntu/+source/linux/+bug/870123 | 19:26 |
| ubot2 | Launchpad bug 870123 in linux "Twinhan USB DVB device failed to operate after upgrade to 11.10" [Undecided,Confirmed] | 19:26 |
| ogasawara | jsalisbury: ack, I'll take a look and see if there's a patch from upstream we should pull in for SRU | 19:27 |
| jsalisbury | ogasawara, thanks. | 19:28 |
| ogasawara | jsalisbury: cool, I think the patch to resolve that bug actually made it into upstream stable v3.0.5 (ie we have the patch queued for Oneiric's first SRU). | 19:46 |
| ogasawara | jsalisbury: am going to build them a test kernel just to confirm | 19:46 |
| jsalisbury | ogasawara, that's good news | 19:46 |
| ogasawara | jsalisbury: indeed, thanks for the heads up. | 19:46 |
| Claudio964 | Ok, I now filed a bug. Bug #870250. Hope that helps. | 19:46 |
| ubot2 | Launchpad bug 870250 in linux "Problem with external firewire disk and new firewire stack in Kubuntu 11.04 and 11.10 beta2" [Undecided,Confirmed] https://launchpad.net/bugs/870250 | 19:46 |
| jsalisbury | ogasawara, np, thanks for looking at it | 19:46 |
| bjf | jsalisbury: ping | 20:11 |
| jsalisbury | bjf, pong | 20:12 |
| bjf | jsalisbury: pm | 20:12 |
| Claudio964 | bjf, jsalisbury: thanks for the quick comments on my bug report. Will test the upstream kernel tomorrow. Must leave now ... thanks so far for the directions and instructions!!! | 20:14 |
| jsalisbury | Claudio964, np, let us know how the testing goes | 20:14 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!