#ubuntu-testing 2009-06-15
<ara> good morning!
<davmor2> Morning All
<cr3> is there a known problem with wired networking in the daily-live/current install? I installed on two systems, one with broadcom and another with intel ethernet controllers, and both have wired ethernet greyed out
<cr3> (that kind of problem is going to be difficult to detect automatically :(
<cr3> all testing and no networking makes cr3 a dull boy...
<fader> cr3: There's a bug right now with the broadcom driver at least
<fader> cr3: bug 384861
<ubot4> Launchpad bug 384861 in linux "Broadcom NetXtreme II (BCM5708) not detected by installer [karmic]" [High,In progress] https://launchpad.net/bugs/384861
<cr3> eth0 seems to come up though, in both cases, but it's not being assigned an address :(
<fader> Hmm, not what I was seeing, but then I was also looking in the alternate rather than the live install
#ubuntu-testing 2009-06-16
<ara> good morning!
<xivulon> Hi all, would it be possible to test http://people.ubuntu.com/~evand/wubi/karmic/wubi-r136.exe as a stand alone application (letting it download and install the Jaunty ISO)
<xivulon> Please report any bug in launchpad, tagging as r136
<eeejay> hi cr3, fader
<cr3> eeejay: yo homie
<fader> eeejay: Wassap, yo?
<eeejay> cr3, fader: how do i get around having to provide a 15 alphanumy?
<eeejay> cr3, fader: certify_prompt.py
<cr3> eeejay: --config=checkbox-compatibility/plugins/blacklist=certify_prompt? :)
<eeejay> cr3, brilliant
<cr3> eeejay: slightly verbose but possible
<cr3> "make common things easy, rare things possible" -- someone who knew what he was talking about
<eeejay> "You've been leading a dog's life.  Stay off the furniture." -- /usr/games/fortune
<cr3> this is a question for all you qa guys who've probably reported a bug or two: I was reporting a new bug in launchpad and then I accidentally pasted outside the text area in firefox. that brought me to a new page and pressing back lost all my carefully crafted but report. wtf?!
<fader> cr3: Don't do that. ;)
<sbeattie> cr3: sometimes firefox remembers what you type, and sometimes not; I'm not sure if any of the plugins help with that.
<eeejay> Phá» break
<cr3> sbeattie: yeah, I noticed the "sometimes" part too
<sbeattie> cr3: of course, you could install itsalltext and configure it to pop up vi in a terminal when you want to edit large blocks of text.
<eeejay> later
<sbeattie> eeejay: mmm, Phá».
<sbeattie> eeejay: are you in seattle now?
<cr3> eeejay: I thought you were mostly vegatarian...
<eeejay> sbeattie: yup, seattle
<eeejay> cr3, got a tofu version
<fader> eeejay: Is that particular to that place or is it fairly common?
<eeejay> fader: seems to be pretty common
<fader> Indeed... I was just trying to figure out where to get vegetarian food for dinner tonight.  You may have talked me into pho.
<fader> :)
<eeejay> cr3: http://paste2.org/p/268511
<eeejay> cr3, i think the "attachments" field cannot be a dict, whic it is
<eeejay> cr3, i think it needs to be a string.
<eeejay> err, i mean it is a list
<eeejay> should be a string, or something else is broken
<cr3> eeejay: sorry for the lag, I'm not sure I follow what's wrong: attachements are lists of dicts containing "name" and "content" keys
<eeejay> cr3, from the stacktrace, it doesn't look like they work well
<cr3> eeejay: ok, so please pastebin the content of checkbox-compatibility/plugins/certify_schemas.py
<cr3> eeejay: the one that's actually being used by your script, in case you happen to have the checkbox-compatibility package installed as well as the source from trunk
<cr3> eeejay: anyways, I'll be willing to bet all my quatloos that you're using revision 200 of certify_schemas.py which changed the definition of attachments on the subsequent revision
<cr3> eeejay: ie, specifically from: attachments": List(String(), required=False)
<cr3> eeejay: to: attachments": List(attachment_schema, required=False)
<cr3> so schema checking is working just fine and just prevented you from submitting invalid data to the server :)
<cr3> that's not entirely true though, the server should've still checked the version of the client
#ubuntu-testing 2009-06-17
<cr3> I be out
<ara> morning!
<plars> ara: so... no meeting then?
<ara> it is at 16:30UTC
<ara> so in 1 hour and a half
<ara> plars: ^
<ara> plars: what did you understand from my email?
<plars> ara: oh, I'm not awake yet!
<ara> plars: ;-)
 * plars suddenly realizes that 1500 is in fact, NOT > 16:30
<plars> heh
<ara> plars: yes, these meetings around the globe are confusing
<eeejay> hello lovely clowns
<ara> morning eeejay!
<eeejay> hi ara, nice to catch you awake :)
<ara> eeejay: I am missing you in my time zone
<eeejay> :)
<ara> hello heno
<heno> hey ara
<ara> cgregan, jcollado, eeejay, plars: meeting ping
<jcollado> ara: pong
 * plars is (mostly) awake
 * eeejay is here
 * heno waits for the wiki page to load
<jcollado> eeejay: Let's talk here, review messages in launchpad are getting messy
<eeejay> jcollado: sure :)
<jcollado> eeejay: So would you prefer to have two different options? Filtering by file name and filtering by name in the XML file?
<eeejay> jcollado: so before your changes, I was able to run mago from the build directory. This made me very happy.
<eeejay> jcollado: yes!
<heno> For the general audience in the chan: we're starting a weekly desktop testing meeting in here
<heno> https://wiki.ubuntu.com/Testing/Automation/Desktop/Meetings
<jcollado> eeejay: OK. I can do that.
<ara> jcollado, eeejay: can you talk about that a bit later, please? or add an item to the agenda ;-)
<jcollado> ara: OK
<heno> indeed let's start
<heno> eeejay: I just added an item for you to catch everyone up on your desktop test deployment work
<cgregan> do we have a mootbot here?
<heno> #startmeeting
<heno> ?
<heno> no bot :(
<eeejay> heno: you mean the first agenda item? yeah, i saw that
<heno> eeejay: right, you've made some checkbox changes
<eeejay> heno, yup
<eeejay> i backed out of most of them
<eeejay> just for the sake of simplicity, it would still be nice to merge them in
<eeejay> but we don't want that to block
<eeejay> right?
<heno> and we agreed the tests would go in the checkbox-qa package
<eeejay> yes
<eeejay> i plan to talk to schwuk about it tomorrow
<heno> what would be blocked by that?
<ara> eeejay: is there any documentation about the work your doing? meeting minutes or something like that
<eeejay> heno: pending checkbox core changes would block the mago plugin
<eeejay> heno: if the plugin depended on the changes
<heno> ok, I see
<eeejay> ara: https://wiki.ubuntu.com/QATeam/Specs/DailyDesktopTesting
<eeejay> so the plugin should be complete this week
<ara> eeejay: thanks
<eeejay> next week is checkbox-qa, i guess
<eeejay> and after that deployment
<eeejay> where cr3 will need to hold my hand
<ara> eeejay: and is the changes your making on a public branch?
<eeejay> ara: https://code.launchpad.net/~eeejay/+junk/checkbox-mago
<heno> eeejay: who is setting up checkbox-qa? cr3 or schwuk?
<eeejay> heno, i understood from our chat that schwuk is, no?
<ara> eeejay, heno: and yet another question? is checkbox-qa a package meant to be in karmic, or just an internal thing?
<schwuk> heno: I'd planned to, with input from cr3
<heno> ok, great
<schwuk> ara: in universe for karmic
<eeejay> schwuk, i hope we could talk about it at tomorrow's meeting?
<heno> ara: in karmic
<ara> schwuk, heno: ta
<schwuk> eeejay: sure
<plars> and what is the purpose of checkbox-qa? this is the first I've heard about it, is it to satisfy the requirements for daily desktop testing?
<eeejay> speaking of.. is mago going into karmic universe?
<heno> https://wiki.ubuntu.com/QATeam/Specs/CheckboxExpandTestCoverage
 * heno afk
<plars> ok, so is it just a temporary branch to do this, that will later be merged back into checkbox trunk? or is it assumed it will always be a separate project?
<ara> eeejay: mmm, no idea. We haven't planned that yet.
 * eeejay don't know
<ara> eeejay: is there a point of having checkbox-qa in karmic and not having mago in karmic?
<eeejay> ara: well, it seems like mago will be a checkbox-qa dependency, so i am just hoping it's packaged somewhere.
<schwuk> plars: https://wiki.ubuntu.com/QATeam/Specs/CheckboxEnhancements, about halfway down.
<eeejay> although I would hate to see us constrained to the distro's schedule
<eeejay> don't want jcollado to be held back :)
<jcollado> eeejay: Thanks
<ara> eeejay: then a package is needed, indeed. I can take care of that task
<schwuk> eeejay: if checkbox-qa ends up in a PPA because of Mago, we can deal with that
<schwuk> Or (temporarily) have a checkbox-qa-mago until Mago is in the archives
<schwuk> But I'd like to avoid that if possible
<eeejay> all seem like viable alternatives
<schwuk> I'll add about the split package/PPA into the spec, just so it's captured.
<eeejay> as we discussed at UDS, the additional non-default packages that are needed for testing should not be installed as packages, but be run from CWD
 * heno back
 * eeejay is talking about daily testing
<eeejay> not community and user checkbox usage
<schwuk> eeejay: got you
<heno> thanks eeejay
<eeejay> so that's my report.
<heno> any other topics?
<eeejay> jcollado's branch
<ara> Suite discovery changes -- jcollado
<heno> (is 30 min too short for this meeting?)
<jcollado> Just wanted to clarify how to remove the roadblocks for merging
<eeejay> jcollado's branch is important for mago checkbox plugin
<cr3> eeejay: me love hand holding long time
<eeejay> :)
<jcollado> Let's talk about the XSL file
<cr3> jcollado: roadblocks for merging? have you submitted some branch for review?
<jcollado> I think it would be a good idea to have a unique XSL file
<ara> cr3: he's talking about mago
<jcollado> that is easy to locate
<jcollado> to avoid the problems that you, eeejay, are having
<eeejay> jcollado: this is one alternative
<jcollado> Maybe in a data subdirectory in the python module
<eeejay> actually two:
<jcollado> could be a good place
<eeejay> 1. have MAGO_XSL in addition to MAGO_PATH
<eeejay> 2. instead of MAGO_PATH, have MAGO_SHARE that is not a ":" separated list, just one path
<eeejay> 2.1. that points to /usr/local/mago/share
<eeejay> ^ $(prefix)/share/mago
<jcollado> MAGO_PATH is for test suite discovery, but XSL file I think isn't part of the test cases it's something that should be part of the library
<eeejay> jcollado: than maybe MAGO_SHARE in addition to MAGO_PATH
<cr3> jcollado: couldn't MAGO_PATH be inferred by MAGO_SHARE? ie, something like $(MAGO_SHARE)/tests
<jcollado> cr3: The idea is that MAGO_PATH contains multiple directories
<jcollado> cr3: with test cases located anywhere
<eeejay> jcollado: i kind of like that too
<cr3> jcollado: gotcha, hence what you said about: MAGO_SHARE that is not a ":" separated list, just one path
<jcollado> cr3: However, the XSL file is unique
<cr3> s/you/eeejay
<eeejay> how about this
<jcollado> Maybe it's better to have MAGO_PATH and MAGO_XSL
<eeejay> MAGO_PATH and MAGO_SHARE, where by default:
<cr3> MAGO_SHARE seems reciprocal to CHECKBOX_SHARE, which seems to have worked well so far
<eeejay> MAGO_PATH = $(MAGO_SHARE)/tests
<cr3> eeejay: I like that
<heno> ok, we need to wrap up, the main QA meeting is starting now in #ubuntu-meeting
<heno> (but feel free continue here)
<jcollado> I don't really like to have the XSL file depend on an environment variable
<jcollado> It's supposed to be something that isn't going to change frequently
<jcollado> Is not as the test cases that someone may try out different things
<cr3> jcollado: I've found that having an environment variable like that makes it easier to be able to run the application from the source directory and then, when building the package, simply change that variable so that it can run from an installed package
<eeejay> jcollado: the env variable should be an override to the defaults
<eeejay> jcollado: they shouldn't be mandatory, nobody should know/use them
<eeejay> jcollado: unless they are developing, like us
<jcollado> Ok, maybe we can use MAGO_SHARE to point to the shared test cases (and the XSL file)
<jcollado> and MAGO_PATH for the extra test cases
<jcollado> MAGO_SHARE: single directory
<jcollado> MAGO_PATH: multiple directories
<jcollado> Would that be ok for you?
 * jcollado assumes everybody agrees
<cgregan> :-)
<sbeattie> cr3: that's just cause the IS team is out to get you. :-)
#ubuntu-testing 2009-06-18
<ara> morning!
<davmor2> is someone running karmic with todays updates?
<stgraber> I do
<davmor2> stgraber: can you add telepathy-idle log onto this channel and let me know if the contacts show up please (empathy should of been installed today)
<stgraber-test> davmor2: Nobody in the right panel
<davmor2> That'll be a bug then :)
<fader> stgraber: How's karmic treating you?  I'm thinking of upgrading my main machine this weekend...
<davmor2> fader: Nooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo
<fader> davmor2: o_O
<davmor2> fader: Only teasing :)  I just wanted to see your reaction :)
<fader> davmor2: I stopped believing anything you said a long time ago ;)
<davmor2> not when I'm testing though then I'm serious :)
<fader> See, I don't believe that either.
<stgraber> fader: works fine for now, just some X crashes from time to time
<fader> stgraber: Who needs X anyway, right? :)
<stgraber> right ;)
 * fader can just interpret the clicks from the hard disk spinning and doesn't need a monitor at all.
<jpds> fader: What about the people using SSDs?
<fader> jpds: They have to watch the disk activity light ;)
<davmor2> fader: don't be daft they just use the force right
<schwuk> fader, cr3, eeejay: I suspect this is going to be made of fail.
<fader> Your suspicions seem to be accurate.
<eeejay> schwuk, you are loud and clear
<eeejay> schwuk: so you could drive :)
<fader> I just got dropped
#ubuntu-testing 2009-06-19
 * mvo hugs sbeattie
<davmor2> Morning All
<fader> eeejay: Good morning
<sbeattie> stgraber: is it possible to have someone review a qa blog post before publishing it to the front page?
#ubuntu-testing 2009-06-20
<ara> sbeattie: are you around?
<ara> sbeattie: no worries, I just wanted to make sure that the testing day was announced, but I saw it in the planet :-)
<ara> I'll follow it on Monday on european time zone
<ara> sbeattie: thanks!
#ubuntu-testing 2009-06-21
<phoenixcomm> hody
<phoenixcomm> can I use multiple monitors(my windows desktop has 4)  w/ unbuntu?
<Shane_Fagan> phoenixcomm: This isnt a support channel. Please ask your question in #ubuntu
#ubuntu-testing 2010-06-21
<davmor2> morning all
<davmor2> ara: did you turn twitter into just a method to publicise what your liking on last.fm ;)
<ara> davmor2, did I? I didn't know it was somehow linked, I have to unlink them
<davmor2> ara: ignore me it's my new android phone.  I forgot it had my last.fm details too.
<ara> davmor2, good to hear :)
#ubuntu-testing 2010-06-22
<ara> good morning all
<davmor2> morning all
<davmor2> fader_: you never said bye yesterday you git
<fader_> davmor2: You know I can't quit you
 * davmor2 Yay!
<ara> hey davmor2
#ubuntu-testing 2010-06-23
<davmor2> morning all
<fader_> davmor2: Hey dude
<davmor2> fader_: hey dude hows taipei
<fader_> Hot and muggy, same as always :)
<davmor2> fader_: Oh so like over here only hot
<fader_> Indeed :)
<fader_> <-- eod, time to find some food... ciao!
#ubuntu-testing 2010-06-24
<ara> good morning all
<kermiac> hey ara (or anyone else) I just had someone ask if the should assign a bug to themselves when they are triaging as they read https://wiki.ubuntu.com/QATeam/TriageAtOpenWeek   The section under the "Assignment & Subscription" heading tells triagers to assign bugs to themselves to avoid having more than one triager work on a bug. AFAIK that is what happened in the past but is not how things are done anymore.
<kermiac> should that section (Assignment & Subscription) be removed or just changed?
<ara> kermiac, I think it should be removed, but I'd better ask pedro_ when he's online
<kermiac> thanks ara :)
#ubuntu-testing 2010-06-25
<ara> good morning all!
#ubuntu-testing 2011-06-20
<hakimsheriff> Hey Guys
<njin> Hi hakimsheriff
#ubuntu-testing 2011-06-22
<patrickmw> QA meeting  ~ 5mins in #ubuntu-meeting
<nagappan> jibel, patrickmw, whenever you are free, can you please help me provide info, how to setup GTK3 in 11.04 ? I would like to reproduce this bug https://bugzilla.gnome.org/show_bug.cgi?id=636062
<ubot4> Gnome bug 636062 in general "comboselect does not work with GTK3" [Normal,Unconfirmed]
#ubuntu-testing 2011-06-25
<preecher> join/ #ubuntu+1
#ubuntu-testing 2012-06-18
<xnox> I would like to contribute to UTAH, I want to work on installation/reboot testing with unusual filesystem's setup. How do I get started? Where are the current installation tests?
<gema> xnox: we are in the process of migrating them, but we haven't really started yet.
<xnox> gema: ok. so where are the current ones? are they for jenkins?
<xnox> gema: or should I just fiddle/start with UTAH?
<xnox> are there any good existing UTAH examples?
<gema> xnox: the current ones are here http://bazaar.launchpad.net/~ubuntu-server-iso-testing-dev/ubuntu-server-iso-testing/trunk/files
<gema> xnox: but if I were you I'd start with utah directly, nuclearbob is finishing off the part where you can preseed installation and we are adding automated testing for utah for that to jenkins this week
<gema> xnox: so early next week we should be in a good position to start adding that kind of tests
<xnox> sounds good to me =)
<gema> xnox: you are welcome to work with us on the moving current tests to utah
<xnox> we'll see =) if the work overlaps with my goals then sure ;)
<gema> xnox: cool, let us know your goals so that we can figure that out
<gema> our goals are to have existing smoke testing working with utah
<xnox> my goals is to add more smoke tests for raid, lvm, btrfs
<xnox> across all affected images
 * xnox grammar fail "my goals are"
<gema> xnox: ok, so whenever we get to migrate those cases, I will ping you
<xnox> =)))))
<gema> we should have a clearer view on how to do it properly by then
<gema> expect a ping either during this week or early next ;)
<gema> xnox: we don't have the capacity to train everyone individually, so we are going to be working with you guys to get all the testing off the ground as we go
<xnox> I was hoping I will be able to learn by myself from the migrated scripts and make changes/additions & merge proposals for additional tests
<gema> xnox: then you'll have to wait two/three weeks
<balloons> phillw, you about?
<phillw> balloons: yup :)
<balloons> did you notice something about http://iso.qa.ubuntu.com/?
<phillw> I'm on http://iso.qa.ubuntu.com/qatracker/milestones/219/builds and have not noticed anything?
<balloons> http://iso.qa.ubuntu.com/qatracker/milestones/219/builds/17439/testcases/4/results
<balloons> look a bit different? :-) Anyways, the new build has landed.
<balloons> I was hoping it would land today, and boom.. it's live
<stgraber> balloons: told you they wouldn't notice ;)
<balloons> stgraber, it's a good thing in many ways!
<balloons> so what that means now is we've got to get the help pages revamped for user facing help, and convert the testcases out of the wiki and into the tracker
<balloons> which is awesome.. I'm super excitied
<balloons>  and while "converting" we get to verify the testcases, and enhance their presentation by putting them all into the templated format, and have them be easy to use
<phillw> looks good,  few typos to correct :)
<phillw> +a
<balloons> Have a look at this: http://packages.qa.dev.stgraber.org/qatracker/milestones/223/builds/16266/testcases/1300/results
<balloons> notice we can do things in the "Link to the installation instructions", and "Link to bug reporting instructions"
<balloons> also, the testcase is in our templated form, and has a bolierplate on the bottom for quicklinks to record what happened
#ubuntu-testing 2012-06-19
<gema> anyone that wants to be up to date or start using utah soon, or collaborate, please join https://lists.ubuntu.com/mailman/listinfo/ubuntu-utah-devel
<gema> we'll be helping everyone, including ourselves through the list
<gaspa> o/
<gema> gaspa: good, welcome :D
<gaspa> thanks :)
<czajkowski> balloons: care to help a user out who's very lost and a bit more than frustrated https://bugs.launchpad.net/ubuntu/+bug/1014054    we've had him on LP on multiple other questions regarding the same issue.
<czajkowski> balloons: context https://answers.launchpad.net/ubuntu/+question/200348
<balloons> czajkowski, I sent a response to have him do some basic troubleshooting
<balloons> that's odd how lp will mark dupe of a private bug
<czajkowski> balloons: thanks
<czajkowski> balloons: it's not odd, it's how it has to be done till bug linking lands
<balloons> well. true.. I'd call it an unintended side effect perhaps
<czajkowski> the private bug can contain info, we suggested he contact someone in the community to get a public version made but he is a bit lost
<czajkowski> balloons: aye well sinuzi and the purple squad are making changes if you keep an eye on blog.launchpad.net you'll be able to see the new stuff coming
<balloons> czajkowski, glad it's being worked. However, troubleshooting his bug is pretty straightforward. I hope my info helps.. What's in the other bug report doesn't have to matter
<czajkowski> exactly
<czajkowski> think he got so wound up not being able to get it he got frustated on questions and we did point out the orignal bug to fix the issue plus blog but that didnt help  him understand
<bladernr_> hey, do you guys know if there's a way to "follow" a particular tag, beyond doing an advanced search for the tag and bookmarking the URL?
<bladernr_> nm, figured it out
<bladernr_> learn something new and nifty every day.
<balloons> phillw, you about?
<phillw> balloons: just finished dinner :)
<balloons> phillw, :-) so your fresh
<balloons> I'm just messing about with the testcases, so I thought if you were around you could have a go at it as well
<phillw> by all means
<phillw> any one you'd like me to take a look at?
<balloons> well, first of all, let's see if you can get in and have access
<balloons> if your logged in, log out. and then when logging in again, make sure you pass the team membership to the tracker
<phillw> http://testcases.qa.ubuntu.com/Install/AlternateWhole is where I am at. I can edit that one.
#ubuntu-testing 2012-06-20
<jibel> started manual testing of ubiquity with yesterdays builds
<jibel> 4 bugs found in 20 minutes
<astraljava> Hi gang, I tried to get data out of the qatracker via the API, using the example file on the page. Probably a dumb question, but what's the actual URL for the xml-rpc service? Surely it's not on example.net. :D
<stgraber> astraljava: <domain>/xmlrpc.php
<astraljava> stgraber: Ahh... thanks. I was sure the qatracker was there somewhere.
<stgraber> astraljava: so if you want the iso tracker, that's http://iso.qa.ubuntu.com/xmlrpc.php, if you want the laptop tracker, that's http://laptop.qa.ubuntu.com/xmlrpc.php, ...
<astraljava> stgraber: Excellent! Thanks! :)
#ubuntu-testing 2012-06-21
<astraljava> balloons: Hi, sadly I cannot participate to the roundtable today. I asked knome if could instead, but he wasn't sure either.
<balloons> astraljava,  I missed your message earlier :-( Well it's happening now
<balloons> phillw, jriddell, hggdh, jibel, Effenberg0x0, superrm1 you guys about?
<Effenberg0x0> o/ here
<hggdh> ~Ã´~
<balloons> I don't know some of the IRC names of folks
 * balloons goes off to look
<superm1> howdy balloons
<balloons> howdy!
<balloons> ok, so we have Effenberg0x0 hggdh and superm1
<balloons> let's see who else we can get trickling in here
<superm1> tgm4883 should be here too from mythbuntu
<tgm4883> I am now
<tgm4883> I forgot and walked away to the server room :/
<balloons> hello Thomas ;-)
<tgm4883> o/
<balloons> ok, looks like astraljava and knome can't make it so, I don't think we'll get anyone from xubuntu
<balloons> any lubuntu folks here?
<balloons> or kubuntu folks?
<balloons> ok, well I will record this so I'll launch a meeting
<balloons> #startmeeting QA Community Roundtable
<meetingology> Meeting started Thu Jun 21 16:10:27 2012 UTC.  The chair is balloons. Information about MeetBot at http://wiki.ubuntu.com/meetingology.
<meetingology> Available commands: #accept #accepted #action #agree #agreed #chair #commands #endmeeting #endvote #halp #help #idea #info #link #lurk #meetingname #meetingtopic #nick #progress #rejected #replay #restrictlogs #save #startmeeting #subtopic #topic #unchair #undo #unlurk #vote #voters #votesrequired
<patdk-wk> hm?
<balloons> Julien and Phil from ubuntu both don't seem to be around ;-(
<balloons> checking on kubuntu folks then we'll get started
<Effenberg0x0> balloons: you had scheduled 2 hours. Maybe they will still make it
<balloons> ok, pm's sent to everyone ;-)
<balloons> so, let's start
<balloons> [TOPIC] UTAH Demo
<balloons> huh, bot doesn't like topics
<balloons> well, anyways, first item on the agenda was to demo UTAH
<tgm4883> balloons, perhaps #topic
<balloons> don't think we can get a demo together, but hggdh can explain a little bit about what it is
<balloons> #topic UTAH Demo
<superm1> i thought there was going to be a google hangout or somethign to show it off?
<tgm4883> hmm, meetingology apparently is a liar when it comes to available commands
<hggdh> no, no demo right now, unfortunately
<balloons> superm1, hggdh should be able to demonstrate how vm's are used in automated testing. but sadly, not a UTAH demo atm
<superm1> so for now everyone stare at http://www.enchantedlearning.com/usa/states/utah/map.GIF and imagine as hggdh describes :)
<balloons> :-)
<Effenberg0x0> I like the green. Very vivid.
<hggdh> UTAH (Ubuntu Testing Automation Harness) is the new baseset we are moving to for automated testing on Ubuntu
<hggdh> it will be (when fully implemented) flavour-agnostic, so it can be used for all *ubuntu
<hggdh> right now, beginning of development, it only supports VMs (via libvirt)
<superm1> is the input it takes an ISO image?
<hggdh> yes, it uses ISO images to build the target test environment
<superm1> specifically, "desktop" ISO images, not alternate
<hggdh> the installation is preseeded, so there is no input required.
<hggdh> not only desktop, but also alternate and server
<superm1> okay i see
<superm1> sorry, i can hold off questions until the end if you would like, i just realized this might be a bit rude to interject
<hggdh> (which is to say, ubiquity and debian-installer based installs)
<hggdh> superm1: no, please shoot the questions as we go
<hggdh> I do not mind :-)
<superm1> Ok. will you have a place to put preseeds in this tool for the different flavours?
<hggdh> this, on the other hand, means that you might have to adjust the preseeds to your specific needs
<hggdh> yes, we will
<superm1> i know at least mythbuntu and ubuntu studio do have custom preseeds
<superm1> Ok
<hggdh> being able to adjust preseeds is pretty much a requirement if you want to automate tests
<superm1> so the unfortunate flaw in doing it this way that comes to mind is that sometimes you will have bugs that are exposed only when the installation is ran in an interactive mode or only in an automatic mode etc
<superm1> so it can't be a complete replacement to lots of user testing, but instead a valuable supplement
<hggdh> UTAH was originally called 'uath'. But we found that (1) nobody could pronouce it, and (2) it means 'fear, horrorÂ´ in ancient Gaelic
<hggdh> generically speaking, automated testing *cannot* replace actual hands-on
<hggdh> it is just a way of getting the bits that do not directly depend on user input tested, and out of the way
<hggdh> but we still need to have manual installs, and testing
<Effenberg0x0> hggdh: Just to be clear, can you mention some cases in which UTAH will raise a flag?
<hggdh> yes
<hggdh> let's say you are testing upgrades from desktop -- it is easily automated: having an existing (say) Precise install, you run 'sudo do-release-upgrade -d', and use pexpect to drive the answers
<hggdh> in this case, we are looking for failures to upgrade -- missing pre-reqs, new package version fails to install, etc
<hggdh> of course, we cannot check for positioning of windows, and text visibility in windows, etc. But we will get the "backoffice" errors
<hggdh> or you are testing a specific package with the testset provided by it (say, mysql, or even coreutils). So you install the image at the version of Ubuntu you want, and run these tests
<hggdh> (for coreutils, you need to _build_ the package again, coreutils tests are intermixed with the build process)
<hggdh> but you will get errors because of a change on libraries
<superm1> what kind of failures get raised in the upgrade testing?  will bugs get filed?
<hggdh> or you want to check on ecryptfs -- we have a set of tests for it, and they are fully automated
<hggdh> no bugs are opened automagically. We still need to look at the failures and identify the cause
<Effenberg0x0> hggdh: got it. Some debugging skills are needed by a human tester.
<hggdh> the point here is to weed out false positives -- errors caused by test code, not by what is actually being tested
<hggdh> Effenberg0x0: always
<hggdh> when you are testing you have to look for real and false positives and negatives
<hggdh> usually, we assume the negatives (which is to say, the expected results) are correct
<hggdh> but, every so often, you should look at your "correct" results, and verify they are indeed correct
<superm1> there will be some sort of notification mechanism for those flavors interested when things fail and need some further intervention?
<hggdh> UTAH has a provision to alert users. YOu can set it and use it. In our case, most of the UTAH tests will be run via Jenkins (http://jenkins-ci.org/), so we use Jenkins to do the alerting
<hggdh> another point we all should be careful on is on destructive/non-destructive tests
<hggdh> I personally define a destructive test as a test that unilaterally changes your system configuration
<hggdh> look at 'change your system configuration' as 'can destroy your system'
<hggdh> it does not matter if it actually completely borks, but it might -- for example -- change the DNS resolution
<superm1> will flavours be able to run utah in the canonical jenkins instance, or need to set up their own?
<hggdh> on the other hand, this Monday we had a ecryptfs test that actually forces a reboot of the system: a piece of the kernel goes haywire, and I/O to ecryptfs cannot be guaranteed to work until the reboot
<hggdh> superm1: I cannot really answer that, sorry. But... we -- right now -- do not have the resources to guarantee test space for the flavours
<superm1> OK
<superm1> Daviey warned that jenkins is a PIA to get setup
<hggdh> so, at least right now, please do not expect we will be able to run tests for other than Ubuntu
<Effenberg0x0> hggdh: The typical ISO-Test (Install/Partitioning, writing MBR/Grub, installing/removing drivers/kernel modules like VGA) is handled by UTAH?
<hggdh> Effenberg0x0: yes indeed
<hggdh> superm1: not really a pita, but it does have a learning curve
<hggdh> the easiest way of using UTAH would be to deploy VMs
<superm1> can UTAH be ran without jenkins then on its own via VM deployments?
<hggdh> deploying bare-metal will, of course, require bare-metal and additional packages/networks (so that MAAS, for example, can be deployed)
<hggdh> superm1: yes, it can, and this is how I was starting to test it
<superm1> ah great
<balloons> if I can interject, it would also be good to throw some links at you for this : https://launchpad.net/utah
<hggdh> all you need is libvirt and friends
<hggdh> balloons: thank you, did not really have time to prepare
<superm1> so really need to just check it out and start playing to see where questions crop up
<balloons> there's a wiki off that page with more info and a picture
<hggdh> (and I am, right now, testing a new set of kernel SRU tests, and the KVM I am using is driving me nuts, popping up on my monitor every time the machine thinks about doing something
<balloons> additionally, if you have further specific questions once your playing with it, there's a mailing list setup for it now ubuntu-utah-dev@lists.ubuntu.com
<hggdh> superm1: yes. Not only to learn, but to tell us where we, ah, did something wrong
<superm1> great, thanks!  just need to find some time to actually use it and experiment now :)
<hggdh> the code resides at...
<balloons> it's linked above hggdh https://launchpad.net/utah, lp:utah
<hggdh> heh
<Effenberg0x0> hggdh: How do we prioritize/filter UTAH-Testing bug reports, amidst the constant flow of new bug reports on LP (correct/incorrect ones) everyday?
<hggdh> so please brz branch, and play -- or install the daily utah package from the PPA
<hggdh> Effenberg0x0: what we are doing internally, is to tag all bugs we find (via whatever test process, including manual testing) as a qa finding
<balloons> fyi, there's a daily and stable ppa.. you can find them here: https://launchpad.net/~utah
<balloons> link to stable: https://launchpad.net/~utah/+archive/stable
<Effenberg0x0> hggdh, OK
<balloons> ok anymore questions on utah for hggdh ?
<balloons> alright if not, next up is testcase management
<balloons> which you get to listen to me for ;-)
<balloons> I'll include visuals..
<balloons> I'm wondering if it makes sense for me to type it or speak it
<balloons> I'm concerned if I only have the video it won't make the log
<balloons> I can screenshare and type to you all, or you can view a live feed of me speaking with my desktop :-)
<balloons> let's try the type and view
<balloons> http://www.screenleap.com/ubuntuqa
<balloons> hopefully everyone can see my screen now?
<Effenberg0x0> OK here balloons
<superm1> yup
<cariboo907> OK here too
<balloons> As you know the qatracker just went through an update to bring testcase management
<balloons> I'm going to show you how it works from a behind the scenes admin perspective
<balloons> we'll use the staging site on dev.stgraber.org
<balloons> so I have a couple products laid out here, mimicking the iso.qa.ubuntu.com tracker
<balloons> testcases and pages look pretty similar, except for the addition of the extra links
<balloons> clicking on a test (http://packages.qa.dev.stgraber.org/qatracker/milestones/224/builds/16281/testcases/1305/results) you can see there are some new links, and the testcase is now included inline
<balloons> hi highvoltage!
<highvoltage> hi balloons :)
<balloons> http://www.screenleap.com/ubuntuqa
<balloons> you can watch and follow along
<balloons> ok, so that's nice having the testcase in there
<balloons> you'll also notice the boilerplate text on the bottom for submitting your result and filing a bug
<balloons> the bug reporting instructions are currently set at a product level
<balloons> the testcases are defined and then grouped into testsuites which can then be used by any product
<superm1> product meaning "ubuntu" "mythbuntu" etc?
<balloons> superm1, yes
<balloons> or, a package
<balloons> like the calls for testing of the kernel
<balloons> I'll show that quickly
<balloons> you can see we've had a call for testing for the kernel, and it's had 2 versions
<balloons> silly me called the 3.4 version precise1, because I was still learning the tool
<balloons> so, there's one testcase in here, smoke test
<balloons> similar boilerplate text
<balloons> we're looking at (http://packages.qa.dev.stgraber.org/qatracker/milestones/223/builds/16283/testcases/1301/results)
<balloons> filing a bug on this package has some further instructions than normal, and includes a link
<balloons> that link is tagging the bug as well
<balloons> if we look at installation instructions for the package, we get a howto install and howto uninstall
<balloons> finally the detailed infromation on the testcase has the history of what the testcase looked like
<balloons> this is useful for when we update a case, but have old results
<balloons> the results and case will match up
<balloons> So you can see some of my earlier attempts and playing with formatting, etc
<balloons> ok, so let's take a look at the admin side of things
<balloons> As you can see, we allow you to define a template for new testcases
<balloons> heh, I changed it a bit
<superm1> does that mean we need to work through an admin like you to make our own test cases?
<balloons> err, actually.. there
<balloons> the template has gone thru a couple revisions before going live
<balloons> superm1, no it doesn't mean you'll need me to help you make testcases
<balloons> we'll get to that in a min ;-)
<balloons> ok, so anyways, that's the example template for a testcase as decided upon last dec by the qa community
<balloons> it could change of course if we decided to change it.. and if so, we could update the template at that time
<balloons> :-0
<balloons> ok, so let's look at the testcases quickly
<balloons> you can see my smoke test mostly follows the template -- barring it has no expected results.
<balloons> but the admin side, you simply title it, and add the testcase
<balloons> notice we do use some html / drupal markup
<balloons> some is allowed and we use it to help display as well as for machine parsing
<balloons> here's an example of an isotest testcase
<balloons> this wiki page http://testcases.qa.ubuntu.com/Install/DesktopFree has been converted into the testcase you see
<balloons> http://packages.qa.dev.stgraber.org/qatracker/milestones/224/builds/16281/testcases/1306/results
<balloons> ok, so that's testcases more or less
<balloons> now, we can organize testcases into testsuites
<balloons> you see the 'free software only' testcase is part of the ubuntu desktop suite
<balloons> well.. 'ubuntu desktop extras' actually :-)
<balloons> there are 3 tests defined in here you can see
<balloons> I can set the order, weight and status as expected
<balloons> now, let's go assign our testcases/testsuites to a product
<balloons> you can see I've linked both the 'ubuntu desktop' and 'ubuntu desktop extras' testsuites to the ubuntu desktop amd64 product
<balloons> for precise
<balloons> now, some of this may be rather foreign to all of you, but there is a guide to this interface which I'll link to at the end
<balloons> the new stuff I am showing you is not yet in that guide. since, it's new (as in this week new :-) )
<balloons> now to answer superm1's question a little bit, there is a new role to the admin interface
<balloons> we will have a testcase admin role which will allow you to define and manage the testcases
<balloons> I've setup a team on lp to do this
<balloons> anyone who is on the team will have access to manage testcases
<balloons> https://launchpad.net/~ubuntu-testcase is the team
<balloons> Phil graciously agreed to help trial out this new interface
<balloons> you'll notice the team is restricted
<balloons> my goal is not to prevent anyone from contributing, but some control is needed
<balloons> for anyone running the tests, I would like them to be able to suggest new tests or improvements by filing a bug against ubuntu-qa
<balloons> like any other community, make a few contributions and it will make sense to make you an admin should you wish to be
<balloons> this is all VERY new, so I'd love input from all of you on how to shape this
<balloons> suffice to say, the flavors should all have at least one person who has this access
<superm1> that's what i was just going to say
<superm1> i'm glad tgm4883 volunteered for mythbuntu
<balloons> :-)
<mrand> hahaha
<balloons> I'd like everyone on the team to also make sure the tests stay maintained and not fall into dis-use or out of date
<balloons> so it's a bit of responsibility, but not too much I don't think
<balloons> more or less if your active in testing, you would be doing / have done this anyway
<balloons> so questions?
<Effenberg0x0> Balloons: Doesn't it sort of overlap with UTAH?
<balloons> besides the testcase management piece, the new qatracker should be able to support all of our testing needs (insomuch as we want to have a test and record results)
<balloons> it's my hope we can consolidate what we're doing by using it
<balloons> Effenberg0x0, how so?
<balloons> This is intended for manual testing, UTAH is intended for automated testing
<balloons> of course, over time we will continue to close the gap on that.. how/where the results get recorded, etc
<Effenberg0x0> I know, I mean some automated tests might kill the need for some manual test-cases
<Effenberg0x0> How to keep things paired up
<balloons> Effenberg0x0, my longer term view is to automate away everything that makes sense
<balloons> the pairing up if you will of what is automated vs manual happens via our communication
<Effenberg0x0> Ok, so the community looks at test-cases and define what's still valid or not, got it
<balloons> this cycle once we have migrated all of our pre-existing tests over I would like to see us take a review of them
<balloons> I took a work item personally to review the testcases for iso testing to ensure they make sense, are needed, etc
<balloons> but yes, as a community, on an on-going basis, we should help frame what is needed for manual testing and where it makes sense for us to help
<balloons> example of this is the kernel tests. you notice the "smoke tests" are rather simple and light
<balloons> the intense tests are being automated by hggdh and the canonical and kernel teams
<balloons> the manual testing piece of that is getting it out to many more different workloads and bits of hardware
<Effenberg0x0> Ok
<balloons> anything else?
<balloons> If not we'll migrate onto the next piece and i'll shut down the screenshare for the time being
<balloons> ok, so we talked about the new tracker and testcases
<balloons> briefly I wanted to mention milestones.. Kate asked me to remind the flavors that you can skip milestones
<balloons> but if you do commit to doing a milestone she asks you do it with full force.. aka, you see it through to the end ;-)
<balloons> feel free to interrupt me at any time btw
<balloons> next up we wanted to discuss collaboration
<superm1> i haven't kept up with the thread about abolishing milestones, but what if that happens?
<superm1> have you thought about how the tracker would scale for that?
<balloons> superm1, yes, that thread has grown and been quite a discussion
<balloons> from a tools point of view, our "milestones" can remain the same.. We can nothing but dailies for iso testing if needed, and calls for testing are already of variable length
<balloons> as far as what's going to happen, I am not completely sure as it's still be discussed
<balloons> however, from a ubuntu perspective we're being asked (again, we were asked at UDS / before UDS) to test more regularly; meaning not just at milestones
<balloons> from that perspective nothing in theory has changed.. but the cadence of exactly when we do this "regular" testing is being suggested to be 2 weeks
<superm1> i see, okay
<balloons> the schedule we adopted after UDS was pretty much the same.. about every 2 weeks
<superm1> and i understand that flavours can follow the cadence they would like in this regime too
<balloons> yes, of course flavors can choose to follow the cadence or not
<balloons> I recommend adopting a cadence that works for the flavor
<balloons> considering the devs, testers, etc
<balloons> hence, adopting every ubuntu milestone doesn't always make sense..
<GridCube> (i could speak very very unoficially for xubuntu)
<balloons> I liked the LTS only approach some flavors have thought about as well
<balloons> GridCube, hello :-)
<GridCube> :)
<balloons> on colloborating, part of us all getting together is to talk about needs we might have and how we can work together to benefit each other
<balloons> maintaining testcases in mutual fashion and sharing them across flavors as it makes sense, is one such example
<balloons> but I also think we can do things like collaborated calls for testing (like we have done with the kernel testing), or on specific packages that mutliple flavors use like firefox
<balloons> whatever it is.. floor is open to whomever has a need or idea for colloberation
<GridCube> o/
<balloons> yes GridCube
<balloons> brb, type away :-)
<GridCube> hello, i present myself, im a bug tester and support collaborator for xubuntu, i've been so for over a year now, never participated here tho.
<GridCube> i have proposed a small change on the qa tracker a while ago, it was that the tests cases should have some sort of area, where all the reported bugs for that particular case on previous days could be seen
<tgm4883> wait, what
 * tgm4883 scowls at superm1 
<superm1> ;)
<balloons> that delay!
<balloons> epic
<balloons> GridCube, this exists: http://iso.qa.ubuntu.com/qatracker/reports/defects
<balloons> additionally when you look at a testcase, it has a list of previously reported bugs
<balloons> but only for that build I believe
<GridCube> only for that build
<balloons> regardless, it's worth filing a bug to discuss how it might look
<balloons> I like the idea
<GridCube> the wishlist bug is already there
<GridCube> im trying to find it
<balloons> GridCube, ahh, ok, point it out to me.. I'll subscribe
<GridCube> ok
<GridCube> give me a sec
<balloons> ok, so anything else.. This last piece is general Q & A time. hggdh is still around (though dealing with his flailing computer) feel free to ask any more questions
<GridCube> https://bugs.launchpad.net/ubuntu-qa-website/+bug/994816
<GridCube> :)
<balloons> it appears like that was implemented?
<balloons> ahh I think you mean https://bugs.launchpad.net/ubuntu-qa-website/+bug/994812
<GridCube> ahm, it was like "granted" but no one actually implemented it
<balloons> I subbed.. I'll look into it.
<GridCube> balloons, yes those two go thogether :)
<balloons> thanks for the suggestion
<GridCube> no problem, i just though it would make reporting and testing easier, because people would know what to look at
<hggdh> I am here
<balloons> ok, so if there's no more questions we can discuss the testcase management stuff quickly.
<balloons> The old wiki needs converted, of which I have done a couple.. In addition to converting I placed them into the new templated format
<balloons> you can see everything in powerpc and amd64+mac has the new testcase format
<balloons> http://iso.qa.ubuntu.com/qatracker/milestones/219/builds/17586/testcases
<balloons> I take that back, heh, 'Live Session' doesn't
<balloons> So I will be going through and doing the same with the other testcases used by the ubuntu iso's
<balloons> however, many of those testcases are used by the flavors as well
<balloons> for instance, the xubuntu desktop i386 testcases
<balloons> they use those same 4 tests
<balloons> the only one not converted is the wubi (windows installer) testcase
<balloons> convert that and you can convert that iso over
<balloons> of course, the flavors can then pick and choose which tests to keep in common and which to write for themselves
<balloons> you can pull any testcase you wish and include it with any other combination of testcases to go into a testsuite
<balloons> the examples I gave that are live actually are 2 testsuites, so that even the testsuites can be shared across multiple isos
<balloons> so for example, the xubuntu desktop i386 iso can use the 'ubuntu desktop' testsuite which contains all 4 testcases it's using, barring the wubi testcase
<balloons> write the wubi testcase and then add it to a testsuite and include it on the iso
<balloons> I trust that all makes sense :-)
<balloons> So, please ping me to get access to help out in this area. Your iso's and testcases will remain usable as-is, (you can see the legacy mode on the tracker), but can be converted now at any time
<balloons> let me know who is interested in joining this new team and I'll help get them going on how to use the tool, etc
<balloons> and with that I think we're done. ;-) Thanks to you all for coming out. I appreciate your time. And thanks for suggesting we meet during the cycle. I think it was helpful
<balloons> #endmeeting
<meetingology> Meeting ended Thu Jun 21 18:00:10 2012 UTC.
<meetingology> Minutes (wiki):        http://ubottu.com/meetingology/logs/ubuntu-testing/2012/ubuntu-testing.2012-06-21-16.10.moin.txt
<meetingology> Minutes (html):        http://ubottu.com/meetingology/logs/ubuntu-testing/2012/ubuntu-testing.2012-06-21-16.10.html
<Effenberg0x0> Thanks balloons
<balloons> I'll post the log on the qa mailing list for reference.. Thanks everyone
<hggdh> thank you balloons
#ubuntu-testing 2012-06-22
<davmor2> jibel: can you try something in a 64bit virtualbox quantal instance, fire it up with 3d enable and wait for jockey to startup and tell you there are additional drivers then try to install the additional drivers for me it worked on i386 but fails on 64 saying I don't have permission
<jibel> davmor2, sure, with todays build ?
<davmor2> jibel: it was yesterday mornings
<davmor2> jibel: I did the install last night but only fired it up for the first time this morning
<jibel> davmor2, it fails but doesn't tell me I don't have permission. anyway jockey will be replaced by ubuntu-drivers at some point during the release
<davmor2> jibel: agreed I was just wondering if it was more of a permissions issue than anything else so thought it worth a second opinion :)
<phillw> balloons: ping
<balloons> phillw, pong
<phillw> balloons: you got time for a PM?
<balloons> of course ;-)
#ubuntu-testing 2012-06-23
<phillw> balloons: poke
#ubuntu-testing 2012-06-24
<astraljava> balloons: I have an option to include ubuntu-testcase into my SSO on iso.qa.ubuntu.com now, but I don't seem to be authorized. Should I be, or is that something I haven't signed up for, yet? If latter, where can I do that?
