| cr3 | hi folks, question about submitting test results for the ubuntu distribution: does it make sense to target the results to the distribution itself (ubuntu) in some cases, the distro series (lucid) in other cases, and the distro arch series (lucid i386) in yet other cases? | 14:50 |
|---|---|---|
| fader_ | cr3: That's an awfully broad question :) Do you mean "should I expose this in the UI?" Because otherwise I think the answer is trivially yes. | 14:52 |
| cr3 | fader_: it's intended to be broad, think of testing irrespective of any particular implementation other than ubuntu itself that's in dire need of more testing :) | 14:53 |
| fader_ | I am assuming this is a question relevant to Checkbox or similar tool; if I misunderstood, I apologize. | 14:54 |
| fader_ | Hmm | 14:54 |
| cr3 | fader_: the question is broader than just checkbox, which I didn't even have in mind when I asked actually :) | 14:54 |
| fader_ | cr3: Pondering it I can come up with pretty reasonable justifications for the distro series and the distro arch series, but not for the distro as a whole. Do you have an example of where you think that might be interesting? | 14:55 |
| cr3 | fader_: what I did have in mind though are bugtasks in launchpad which can be targetting to projects, project series, distro, distro series but not distro arch series | 14:55 |
| charlie-tca | cr3: still yes, since it does depends on the actual tests | 14:55 |
| cr3 | fader_: the same thought crossed my mind, but then I saw bugs reported directly against the distro here: https://bugs.launchpad.net/ubuntu | 14:56 |
| charlie-tca | sometimes 386 fails and 64bit passes, sometimes the test is very specific to the release itself, sometimes it is broad, and includes any release | 14:56 |
| fader_ | I'm also tempted to argue that you could provide all the information for each report and then let the people using the information decide what is relevant and what is not | 14:56 |
| fader_ | cr3: IMO it's a bug to allow bug reports against the distro :) | 14:57 |
| cr3 | charlie-tca: do you see these uses cases occuring withing the same test run? | 14:57 |
| fader_ | The only bug I can think of that is relevant to the distro is Bug 1, but even that should probably be put into some sort of metaproject | 14:57 |
| charlie-tca | depends on what actually fails, doesn't it? | 14:58 |
| cr3 | fader_: maybe the reason for reporting bugs against the distro is to lower the barrier to entry: when in doubt, here's a catchall | 14:58 |
| charlie-tca | today you have an update-manager bug, it is in both arches, specific to oneiric only | 14:58 |
| mvo | charlie-tca: what is the issue with update-manager? | 14:59 |
| charlie-tca | tomorrow you might have the same bug, in 386 only | 14:59 |
| cr3 | charlie-tca: you can't report a bug against an arch though | 14:59 |
| charlie-tca | but you can tag it arch specific | 14:59 |
| fader_ | cr3: Sure, I suspect it's about 20% to lower the barrier for reporting and 80% because "that's just how Launchpad works". But I'd not use historical reasons as a justification for doing it that way in the future | 14:59 |
| charlie-tca | breaking things down individually can cause things to get lost in the breakdowns | 14:59 |
| cr3 | charlie-tca: that's different from a distro series though, one is unstructured and the other is structured. however, I appreciate your point that there might sometimes be a need to report against an arch, thanks! | 15:00 |
| charlie-tca | seaching on update-manager, I get all bugs for it. I can limit the search using i386 or amd64 in tags | 15:00 |
| fader_ | ^^ that's an example of what I was meaning earlier -- gather the data and let it be filterable if it is relevant | 15:01 |
| cr3 | charlie-tca: that's a weird thing in launchpad that if I report a bug against a project series, I can't see it under the project itself even though it's a container of serieses | 15:01 |
| cr3 | charlie-tca: what if bugs could only be reported against a distro (ubuntu) or a project (checkbox), and the rest of the data would be in tags like "lucid", "i386", "trunk", etc.? | 15:03 |
| * cr3 finds that somewhat appealing but is scared :( | 15:04 | |
| * cr3 scares easily | 15:04 | |
| charlie-tca | I don't much like that idea | 15:05 |
| cr3 | charlie-tca: hm, I would've expected you to like the idea the most actually. why do you like the idea of tags for the architecture but not for the series? | 15:06 |
| charlie-tca | It could make the tags multi-lines long, and most users do not know how to use them | 15:06 |
| charlie-tca | Have you seen some of the user tags? | 15:06 |
| cr3 | charlie-tca: so why not structure the architecture then instead of leaving it to the whim of users putting it in tags? | 15:06 |
| cr3 | charlie-tca: some of the tags I've seen are insane, I can't make heads or tails of 'em | 15:06 |
| charlie-tca | They think that is a field just for them to put anything they want. | 15:06 |
| charlie-tca | I like being able to look at a specific package, and see all the bugs for it there | 15:07 |
| charlie-tca | I also like being able to see just 386, just 64bit, and just xubuntu bugs, because I can filter on the tag | 15:07 |
| cr3 | charlie-tca: what would you think of these three use cases: | 15:08 |
| charlie-tca | If all bugs were against Ubuntu, how would you ever find a bug against modem-manager? | 15:08 |
| cr3 | 1. when looking at https://bugs.launchpad.net/ubuntu, you would see all bugs for all series and all architectures | 15:08 |
| cr3 | 2. when looking at https://bugs.launchpad.net/ubuntu/lucid, you would see all bugs for that series for all architectures | 15:08 |
| charlie-tca | Why you you ever want to see all 100,000 + bugs at one time? | 15:09 |
| cr3 | 3. when looking at https://bugs.launchpad.net/ubuntu/lucid/i386, you would see all bugs for that series and that architecture | 15:09 |
| charlie-tca | and how would you see the bugs for a specific package? | 15:09 |
| cr3 | charlie-tca: project and project series would behave the same as described above: | 15:09 |
| cr3 | 1. when looking at https://bugs.launchpad.net/checkbox, you would see all bugs for all series | 15:10 |
| cr3 | 2. when looking at https://bugs.launchpad.net/checkbox/trunk, you would see all bugs for that series | 15:10 |
| charlie-tca | That sounds like a lot more work for triagers and developers, really. | 15:10 |
| cr3 | to be more explicit, the difference with the current behavior is that /ubuntu does not show bugs targetted specifically to /ubuntu/lucid... and similar behavior for projects | 15:11 |
| charlie-tca | We already have hundreds of bugs reported against Ubuntu, that have to be re-assigned to the proper packages | 15:11 |
| charlie-tca | We don't have enough people to re-assign every bug reported, there's 1000+ everyday | 15:11 |
| charlie-tca | At least now we have better than 50% chance the reporter willg et the package right | 15:12 |
| cr3 | so what I'm hearing is that being able to target bugs is specifically as possible from the start would be useful, and the same would apply to test results if such a thing existed :) | 15:12 |
| charlie-tca | The more complicated it becomes, the less chance of getting them right, and the more workload for those who need to find their bugs | 15:12 |
| cr3 | s/is specifically/as specifically/ | 15:13 |
| charlie-tca | In my opinion, yes | 15:13 |
| cr3 | charlie-tca: so if there was a distro arch series target (/ubuntu/lucid/i386 for example), that would be preferable than having a tag, right? | 15:13 |
| charlie-tca | It would limit those who know to use it, and make it harder to keep straight, I think | 15:14 |
| charlie-tca | I think that would result in a lot more work | 15:14 |
| cr3 | ok, so two levels of indirection (project/distro and release/series) is a good compromise to keep things simple, as opposed to having a third level for architecture in the case of distro series, right? | 15:15 |
| charlie-tca | I really don't want to see a ton of bugs against that instead of the package itself | 15:15 |
| charlie-tca | right | 15:15 |
| cr3 | charlie-tca: and whatever we can do to target a project rather than a distro series would also be useful, right? | 15:16 |
| charlie-tca | right | 15:16 |
| charlie-tca | I think so | 15:16 |
| charlie-tca | project being package specific, right? | 15:17 |
| cr3 | charlie-tca: your experience is most valuable, much appreciated :) | 15:17 |
| cr3 | charlie-tca: I'm not sure I fully understand the multiple relationships between a package and a project in launchpad, it's been explained to me with sven diagrams but it's still complicated | 15:17 |
| cr3 | charlie-tca: but I think we're more or less talking about the same thing :) | 15:18 |
| charlie-tca | I don't understand all of it either. I just know if I want to see the bugs in something like abiword, I don't want to see all the bugs in 386 or 64 | 15:18 |
| charlie-tca | I want "abiword" only, I don't care if they showed up in any release | 15:18 |
| cr3 | charlie-tca: but you won't see bugs in /ubuntu/+source/abiword though, ie downstream bugs which might very well affect your upstream abiword project | 15:20 |
| cr3 | in other words, you need to look in two places to get very similar information | 15:20 |
| charlie-tca | right. | 15:20 |
| charlie-tca | But upstream does not really want to see everything reported downstream | 15:20 |
| charlie-tca | most of the time, anyway | 15:21 |
| charlie-tca | UPstream could include bugs from any distribution, ubuntu will only include bugs downstream cares about | 15:21 |
| charlie-tca | I think, ideally, as upstream, I would like a way to say "show me all the bugs in every distribution reported in launchpad" but at the same time, how do I get it to include all the bugs in fedora bugzilla, or OpenSuse bugzilla, or ... | 15:24 |
| charlie-tca | I don't think downstream bugs should ever show up automatically in the upstream bug tracker. | 15:24 |
| cr3 | charlie-tca: if we started reporting test results, to see what works in addition to what doesn't work, do you think the same principles as for bugs would apply? | 15:30 |
| cr3 | (here's a sneak preview of what I have in mind: http://ec2-50-18-75-140.us-west-1.compute.amazonaws.com/) | 15:30 |
| charlie-tca | I don't know about breaking results down that far. | 15:34 |
| charlie-tca | I think it can be beneficial, but I don't like more than about once a week. | 15:34 |
| charlie-tca | I am using http://2tu.us/3ebn for Xubuntu daily testing, which is working for us. | 15:34 |
| charlie-tca | I think getting both sides of the test is helpful, most of the time, because if you don't what worked, how do you know what is really tested today? | 15:37 |
| cr3 | fader_: how would you feel if you *had* to report results against a fined grained project series (checkbox trunk) or a distro arch series (lucid i386), but you *could* generate reports for a coarser grained project (just checkbox), distro series (lucid) or distro even (all of ubuntu) | 15:37 |
| cr3 | charlie-tca: that's exactly the intend of that results tracker site. the way I see it: bugs are a measure of sickness, tests a measure of health :) | 15:38 |
| fader_ | cr3: I don't follow... if I have the option of generating coarse-grained reports, how do I have to report against fine-grained project series, etc.? | 15:38 |
| cr3 | charlie-tca: another intent is that it breaks my heart to see lots of teams coming up with all kinds of ways to track their testing efforts, like your spreadsheet for example. it would be nice to consolidate that in a single location for everyone to share the results and get a general overview of what actually works out there | 15:39 |
| charlie-tca | Yes, but as a unsupported flavour, most of the time, we are left on our own to do such things | 15:39 |
| cr3 | fader_: in other words, strict on what goes in, flexible on what comes out. makes more sense? | 15:39 |
| fader_ | cr3: Ah, got it -- I was confused by "report" the verb and "report" the noun :) | 15:40 |
| cr3 | charlie-tca: why isn't there https://launchpad.net/xubuntu? | 15:40 |
| fader_ | cr3: That's what I [think I] am advocating for... data comes in fine-grained but then can be viewed more coarsely if needed | 15:41 |
| fader_ | Or useful | 15:41 |
| charlie-tca | I don't know why | 15:41 |
| fader_ | With the caveat that we should provide tools that automate the data collection as far as possible -- nobody should have to fill out a three page form to submit a bug report | 15:41 |
| cr3 | ironically, that seems like the opposite of my general approach for interfaces: be flexible on what you accept and be strict on what you return :) | 15:41 |
| fader_ | (This is the royal "we", obviously ;) ) | 15:41 |
| cr3 | charlie-tca: there is no kubuntu either, there's something I don't quite understand then | 15:42 |
| charlie-tca | but we have ~xubuntu-team, ~xubuntu-meta, etc | 15:43 |
| charlie-tca | I think we all fall under Ubuntu, but I am not sure | 15:43 |
| cr3 | charlie-tca: that makes sense, thanks for the explanation. I also found xubuntu-desktop, but I still think it would be nice to have just /xubuntu under launchpad.net | 15:44 |
| charlie-tca | I don't understand all that, myself. | 15:44 |
| cr3 | fader_: I'm allergic to forms, so your concerns are probably safe with me | 15:44 |
| charlie-tca | cr3: it might be because when launchpad started, it was ubuntu only? | 15:45 |
| * cr3 needs one of those bracelets: severe allergies to forms and stupidity | 15:45 | |
| fader_ | cr3: It just seems to me that you're frequently going to have cases where you just don't know if the arch is relevant until you start looking at the data that has come in. Stuff that is trivial to get like that is better to have and not need than need and not have. | 15:45 |
| charlie-tca | +1 fader_ | 15:46 |
| cr3 | fader_: dude, you're making sense! | 15:46 |
| fader_ | Whoa, everybody agrees with me... time to change position! | 15:46 |
| fader_ | BUGS SHOULD ONLY BE EXPRESSED IN TERMS OF COLORS AND SMELLS | 15:47 |
| * charlie-tca going hide pretty soon. After all this, fader_ sums it up in two sentences :) | 15:47 | |
| fader_ | Bug against checkbox: blue and cabbage | 15:47 |
| fader_ | charlie-tca: Hehe | 15:48 |
| brendand | i've got one in unity that definitely smells of cheese | 15:48 |
| brendand | not sure what color it is though | 15:48 |
| fader_ | brendand: Shh, that's just cr3 | 15:48 |
| brendand | probably blue | 15:48 |
| cr3 | fader_: I always found it curious that we have such a limited vocabulary specifically for smells, it mostly seems to consist of comparisons to other things | 15:48 |
| davmor2 | fader_: so to get developer attention the critical bug would be brown, pie smell right? | 15:48 |
| brendand | really? didn't know xchat had smell-o-vision | 15:48 |
| fader_ | Mmm... pie | 15:48 |
| fader_ | brendand: Be grateful you don't have that extension installed | 15:50 |
| === yofel_ is now known as yofel | ||
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!