[18:26] <dkessel> shrini: regarding your mail about the broken autopilot test - yes, this bug is currently blocking us from executing the tests
[18:28] <dkessel> well, not from executing - but from executing them with succcess ;)
[18:29] <shrini> Ok
[18:29] <balloons> true, they are running :-)
[18:31] <DanChapman> o/
[18:31] <shrini> Failing for me
[18:55] <DanChapman> dkessel: so I got the test to run up to this point http://i.imgur.com/uFmPYTm.png this dialog is new so there is no autopilot code to handle this
[18:57] <dkessel> DanChapman: ok, so we need a bug for fixing that
[18:57] <balloons_> life in the cloud sometimes isn't fun
[18:57] <dkessel> care to share your changes to get it so far? ;)
[18:57] <balloons_> sorry, my IRC has decided to be sketchy
[18:58] <dkessel> also, I couldn't get it to run against my local branch... it would always check out lp:ubiquity...
[18:58] <balloons> and we're back  . .
[18:58] <dkessel> oopsie
[18:59] <DanChapman> dkessel, I'm basically removing the test code at each step to see what's broken.
[18:59] <DanChapman> not fixed anything yet :-)
[18:59] <dkessel> heh
[18:59] <balloons> nice. Well, it's more or less time.
[18:59] <balloons> So who's about to talk about automated ubiquity testing?
[18:59] <DanChapman> IIRC there's an emulator (what do we call them now balloons?? ) for GtkDialog so it should be  easy enough to pick it up
[19:00] <balloons> DanChapman, we prefer 'helpers' now :-)
[19:00] <svij> o/
[19:00] <dkessel> o/
[19:00] <DanChapman> o/
[19:00] <flocculant> and I'm about to read :p
[19:00]  * DanChapman makes note of 'helpers' for future discussion :-)
[19:02] <balloons> Excellent. So I thought we could start by addressing the open questions we still have in the document, along with any other questions people might have
[19:02] <balloons> You can follow along here: https://docs.google.com/document/d/1TILHcVLJzJ6Cz9gaUA0D-Aa5FbdYeP-Jc19nH7Xf1lc/
[19:04] <balloons> I'm essentially looking at the Issues section. The first item up for discussion is about hosting and maintaining the jenkins servers
[19:04] <svij> I could help with that
[19:05] <balloons> We need someone who is willing to try out the current tests and see if real hardware is required or not?
[19:05] <balloons> Anyone have the capability of setting up a test jenkins instance to help give guidance in this area?
[19:07] <DanChapman> I can supply a temporary azure vm for testing if that's of any use. I don't have time to setup the jobs though :-)
[19:07] <svij> setting up an jenkins server isn't a big deal for me, but I don't have much experience in the actual tests. I'm happy to help where I can.
[19:07] <DanChapman> dkessel: FYI bug 1480430
[19:08] <balloons> hmm. I think in this case svij, as long as you can follow the docs and setup the jobs, we should get enough information about what we need for hosting
[19:08] <balloons> you can always ask for help as well, heh :-)
[19:08] <svij> sure
[19:08] <shrini> I Can also help in setting Jenkins
[19:08] <balloons> either way, the setup only needs to exist long enough for us to decide on how we setup jenkins and who we have host it
[19:09] <balloons> once we have a decision on that, we can make a request for funding so we have a proper server to do the rest of the work on.
[19:10] <balloons> shrini, svij thanks for helping out. Can we leave it to you two to make some recommendations on what we should puruse for jenkins and hosting?
[19:10] <shrini> Ok
[19:10] <shrini> Will setup a vps
[19:10] <shrini> And install it
[19:11] <shrini> By tomorrow
[19:11] <shrini> So that we can play around it
[19:11] <dkessel> great
[19:11] <DanChapman> \o/
[19:11] <balloons> sounds perfect, thanks.
[19:11] <svij> I could too ;)
[19:11] <balloons> svij, one or both works for me ;-) Either way both your inputs on how to setup things would be useful
[19:12] <shrini> svij: thanks
[19:12] <svij> shrini: where do you want to host it?
[19:12] <shrini> Ec2 or
[19:13] <shrini> Digital ocean
[19:14] <svij> ok
[19:14] <shrini> Will buy one tonight
[19:14] <shrini> Pm your email
[19:14] <shrini> Will share credentials
[19:15] <svij> shrini: great :) svij@ubuntu.com
[19:15] <shrini> Ok
[19:15] <balloons> thanks! ok, so moving on to the topic
[19:16] <shrini> Guys, for me the ubiquity tests are failing
[19:16] <balloons> as you've noticed, the autopilot tests for ubiquity don't complete successfully at the moment
[19:16] <balloons> shrini :-)
[19:16] <shrini> Need to fix
[19:16] <balloons> yes, indeed. So DanChapman and dkessel have been working on this. What are your thoughts? Do you need any help?
[19:16] <shrini> :-)
[19:16] <shrini> Ok
[19:17] <shrini> Let us wait for them to fix
[19:17] <shrini> Thanks
[19:18] <balloons> Sorry, I was intending to ask DanChapman and dkessel about the tests and if they need help getting them running again, etc
[19:18] <dkessel> I need help getting the tests to run against my local branch :p
[19:19] <DanChapman> balloons: not sure how much time I can commit to fixing them myself (not sure how busy dkessel is either) so help would be great.
[19:19] <balloons> is anyone interested in helping out in the area?
[19:19] <dkessel> I also have limited time during the timeframe we're talking about
[19:19] <shrini> I will try
[19:19] <DanChapman> dkessel you just pass in --shareddir=/path/to/ubiquity/
[19:19] <shrini> But need docs to start
[19:19] <balloons> DanChapman, ack. I imagine your expertise will be most useful but understand you have commitments to dekko
[19:20] <dkessel> DanChapman: oh okay =)
[19:20] <DanChapman> I'm happy to help anyone willing to get involved with them. It's a beast to get your head into so I wouldn't leave anyone dangling :-)
[19:20] <balloons_> DanChapman:  is there anything specific to debugging ubiquity and AP tests?
[19:20] <balloons_> I mean, like a guide
[19:21] <balloons_> or perhaps you can write something up?
[19:22] <DanChapman> The only guide is the README atm. I could write something up tomorrow morning
[19:22] <DanChapman> do yo uwant it on the wiki or just in the src dir?
[19:23] <shrini> Wiki
[19:23] <balloons_> excellent. Once you finish we can recruit some folks again to give help specifcally for the tests
[19:23] <shrini> Would be great
[19:23] <balloons_> DanChapman: I would say source actually
[19:23] <balloons_> We can link from the wiki or even clone I guess. But the source is a great place to keep it long term
[19:24] <DanChapman> yeah I was thinking the same. I'll do that then
[19:24] <balloons_> ok excellent. And thanks for diving in shrini . It'll be a little bit of a learning curve at first, but don't be intimated by the size of the GTK tree
[19:26] <shrini> :-)
[19:26] <shrini> Will learn
[19:26] <balloons> alright, those are the primary tasks to tackle, but there's more things still undecided in the doc. Do you want to talk about reporting at least for now?
[19:27]  * svij has a few questions for jenkins-hosting
[19:27] <balloons> svij, sure. ask away
[19:27] <svij> oh, they're in the doc somehow, wanted to ask about the jenkins slaves
[19:28] <DanChapman> balloons: so ideally you want reports on the tracker?
[19:28] <svij> so, do we really need physical hardware for the tests?
[19:29] <balloons> DanChapman, we'll hold that thought for on sec :-)
[19:29] <balloons> svij, that's what we are hoping you and shrini will answer for us
[19:29] <balloons> We don't think we do, but CI has hinted that we do
[19:29] <svij> well jenkins master is fine in a VM
[19:29] <balloons> svij, right, that's a vm either way
[19:30] <svij> I didn't look into the tests yet, if they run fine in a vm, thats fine too
[19:30] <svij> we could also run it on the master
[19:30] <flocculant> balloons: I don't read that hint that way
[19:30] <balloons> flocculant, oh?
[19:30] <flocculant> assuming you mean "(it uses a local kvm instead of depending on the cloud"
[19:30] <balloons> flocculant, I'm referring to the text left by CI in the RT wxl filed
[19:31] <flocculant> from Evan?
[19:31] <balloons> Let me paste it
[19:31] <svij> shrini: are you an ubuntu member? Asking because it might be better if I set up the jenkins master, so I can request the money from the donations program.
[19:31] <flocculant> svij: or balloons does that
[19:32] <flocculant> being as he's on the team that says yes or no ...
[19:32] <flocculant> if he can :)
[19:32] <balloons> mmm, probably better for svij to do it, heh. But yes, I'm part of the team who decides, hah.
[19:32] <flocculant> :)
[19:32] <balloons> http://paste.ubuntu.com/11975277/ is the text from CI/Evan
[19:33] <balloons> anyway, I think that indeed we'll be able to run everything on a cloud server
[19:33] <balloons> we simply need to verify it
[19:33] <flocculant> balloons: right - so I read the kvm/cloud bit to mean they used local kvm so they didn't need cloud - not that they couldn't
[19:33] <svij> oh ok, is that fine for you shrini, if I set it up and give you access to the jenkins instance?
[19:34] <flocculant> balloons: I guess it's not particularly clear :)
[19:34] <balloons> flocculant, right, it's not :-)
[19:34] <svij> so we do need 2 vms? One jenkins master and a slave where the tests runs?
[19:34] <flocculant> rt's ...
[19:35] <balloons> ok, any other questions on jenkins before we talk about reporting?
[19:35] <balloons> ohh, svij. We can go with whatever setup you guys recommend
[19:35] <balloons> I was thinking having a master and a few slaves would be best. The images arrive more or less at the same time
[19:36] <balloons> it would be nice to have a little parallization
[19:36] <svij> right
[19:36] <shrini> svij: ok
[19:36] <shrini> That's fine
[19:36] <shrini> You set up
[19:36] <svij> we could spin up a few vms as slaves and drop them after running the tests, that saves costs ;)
[19:36] <balloons> how many is a question for you guys to answer as well. So how many instances to obtain, how big, how many slaves?
[19:37] <balloons> svij, ahh right.. None of them have to be permanent
[19:37] <flocculant> balloons: re when images arrive - there are quite a few hours between
[19:37] <balloons> besides master
[19:37] <shrini> Mail me once done
[19:37] <svij> I didn't have a look into the tests yet, so I cant say much about how much VMs we need.
[19:37] <svij> shrini: I do
[19:37] <flocculant> xubuntu turns up ~10am - Lubuntu ~5pm
[19:38] <balloons> flocculant, wow, really? That's more than I thought. I was thinking everything in about a 4 hour window
[19:38] <shrini> Tshrinivasan@gmail.com
[19:38] <flocculant> balloons: kubuntu is ~5am
[19:38] <flocculant> balloons: from the cd-build-logs
[19:38] <svij> sounds like we will have a few jobs anyway who doesn't run on the same time?
[19:38] <svij> don't*
[19:39] <balloons> svij, yes, several tests against the same image, so we could still have more than 1 slave
[19:39] <balloons> anyways, do you have the full list of questions now to answer, heh?
[19:39] <flocculant> flocculant: given that I'm not going to be doing anything else - if when they run is important I can grab those times
[19:39] <flocculant> silly /me
[19:39] <balloons> flocculant, sure, why not. Can you get an average time / timeframe for the images?
[19:39] <flocculant> yep
[19:40] <flocculant> who do we need that for?
[19:40] <balloons> all flavors, expect kubuntu
[19:40] <flocculant> ok
[19:40] <balloons> they will all be running, minus kubuntu as it uses a different installer
[19:40] <flocculant> not Ubuntu?
[19:41] <balloons> we'e not concerned with ubuntu proper in this case
[19:41] <flocculant> ok - just double checking :)
[19:41] <flocculant> so all on iso.qa - except Kubuntu and Ubuntu
[19:41] <balloons> flocculant, feel free to include it for giggles though
[19:41] <balloons> ok, anything else on jenkins?
[19:42] <svij> the tests will not run for the normal ubuntu?
[19:43] <svij> no new jenkins questions for now, from me.
[19:43] <balloons> svij, they do run for normal ubuntu, but we were focusing on testing flavors specifically at first
[19:43] <svij> balloons: ahh ok
[19:43] <balloons> we can certainly run them all honestly
[19:43] <svij> I missed that part.
[19:43] <balloons> kubuntu is excluded only because they use the qt installer, and we have no tests for it
[19:44] <balloons> thought perhaps reaching out to them and seeing what they think about a testsuite would be useful at some point
[19:44] <balloons> anyways, moving on to reporting then and DanChapman's question
[19:44] <svij> oh, quick question: I'll use digitalocean for 10$/month with backups (+20% … 2$) is that fine?
[19:44] <balloons> btw, for your sanity, I'll send along notes of what we discussed to the list
[19:45] <balloons> svij, yes, that's fine. For the costs, we simply have to outline what we are using it for and why. That is not a large amount of money
[19:45] <svij> yep, ok :)
[19:46] <balloons> alright, so on reporting, let's just get some ideas from everyone. It's not something we have to decide now, and we've been chatting for a bit already
[19:46] <balloons> the basic idea is we want to display results to everyone about what is happening with the tests
[19:47] <balloons> this information is especially useful to folks running manual tests on the images.
[19:47] <flocculant> the put it on iso.qa at Notices for a basic pass/fail line seems ideal
[19:47] <DanChapman> +1 ^^
[19:47] <flocculant> but more detail elsewhere for those who want to look at that when necessary
[19:48] <dkessel> +1. that way we have a dashboard in a well-known place
[19:48] <balloons> wow, lots of love for that idea.
[19:48] <flocculant> assuming we can send Xubuntu to Xubuntu, etc etc
[19:48] <balloons> I was going to say, for the initial work, it would be nice to keep reporting requirements to a miniumum
[19:48] <flocculant> balloons: simple often is best
[19:48] <svij> you mean this one? http://iso.qa.ubuntu.com/qatracker/milestones/340/builds/98875/testcases
[19:49] <flocculant> svij: yea
[19:49] <balloons> yea, basically that text at the top of the page
[19:49] <balloons> we could include whatever text in there we wish
[19:49] <flocculant> or alternatively - list them at http://iso.qa.ubuntu.com/qatracker
[19:49] <flocculant> but just list fails
[19:49] <flocculant> who needs to know it passed
[19:50] <balloons> there's a python API for that site. Essentially then we would need to get the build information and push an update to the tracker via the API
[19:51] <balloons> now that includes more work as someone will need to write a script / jenkins job to do this
[19:51] <balloons> svij, shrini is there any easier options for displaying the information on jenkins itself?
[19:51] <balloons> and are folks in favor of looking at jenkins jobs and parsing the information at first?
[19:52] <flocculant> balloons: that's not a problem for me
[19:53] <svij> displaying a status on jenkins is the default (or didn't I understand the question?)
[19:53] <balloons> svij, yea, indeed. I guess I was asking if there was a nitfy plugin you knew of, or a cheap way to display results from builds without us having to write something
[19:53] <DanChapman> or me, the rss feed and email on fails is also a handy thing to have
[19:54] <balloons> ^^ that's an idea
[19:54] <svij> yea, that sounds good DanChapman
[19:54] <dkessel> yeah, e-mails on fails are good
[19:54] <balloons> we could have a mailing list that posted results
[19:54] <balloons> you could signup
[19:54] <svij> emails are the easiest
[19:54] <svij> but feeding an API with a few data doesn't sound too hard anyway (as a second step)
[19:55] <balloons> ok, we won't decide right away, but I think both approaches sound lightweight enough for the initial work
[19:55] <balloons> Let's defer making the decision until we know a little more about the jobs and what it would talk to send updates
[19:56] <balloons> sound good?
[19:56] <svij> +1
[19:56] <DanChapman> yep
[19:56] <flocculant> yep
[19:56] <balloons> With that, I'm through chatting about everything I wanted to cover today. The last bit I wanted to ask is when can everyone meet again?
[19:57] <balloons> I've got a few action items noted that folks have agreed to do, I think giving everyone some time before we meet would be good. Next time we should be able to decide on jenkins/hosting questions, and have a good plan for getting the AP tests working (or even fixed)
[19:58] <balloons> is this time / day good for everyone?
[19:58] <svij> yep
[19:59] <DanChapman> yeah good for me.
[19:59] <flocculant> this time is good for me - days usually not an issue
[19:59]  * balloons realizes he'll be out next Friday travelling
[20:00] <balloons> hmm..
[20:00] <DanChapman> thursday?
[20:00] <flocculant> quick point - mythbuntu
[20:00] <balloons> that works fine for me
[20:01] <DanChapman> seems test runner for ubuntu-gnome is broken :-(
[20:01] <flocculant> is that  going to be looked at - because they're LTS to LTS and also builds have been failing
[20:01] <dkessel> thursday's okay
[20:02] <svij> thursday is also fine for me
[20:02] <balloons> flocculant, excellent point. Do you want to include that in your findings for when the images build?
[20:02] <balloons> perhaps you can suggest which images we start with first, and which to bring on later
[20:02] <balloons> I wouldn't want us to try and test images that will give us trouble at first.. and since they are LTS only anyway, it adds nothing for this cycle
[20:02] <flocculant> balloons: that's how I know it's been failing :D
[20:03] <DanChapman> does mythbuntu have any additional steps in ubiquity like edubuntu does?
[20:03] <balloons> excellent, thursday at 1900 it shall be.
[20:03]  * DanChapman makes note in calendar
[20:03] <flocculant> balloons: looks like Myth started fails on 21st July
[20:04] <balloons> ack
[20:04] <flocculant> http://pad.ubuntu.com/JenkinsTimes
[20:05] <flocculant> there you go - -that's the start/finish times for all except Ubuntu/Kubuntu
[20:05] <balloons> Ok, thanks again everyone for meeting. I'll see you next Thursday @ 1900 UTC. I'll send a summary of what we discussed to the list
[20:05] <balloons> flocculant, thanks. I'll add to the doc
[20:05] <DanChapman> thanks guys!
[20:05] <dkessel> thanks for organizing and notekeeping balloons
[20:05] <balloons> or you can add them
[20:05] <balloons> you are all most welcome. Thanks so much for helping make this a reality!
[20:05] <svij> thanks, I'll setup the jenkins tomorrow
[20:05] <flocculant> balloons:  ta not got a gmail
[20:06] <balloons> sounds good. Enjoy your evenings!
[20:06] <flocculant> night all
[20:07] <svij> evening? nearly bedtime … ;)
[20:07] <flocculant> if there's anything people think I might be able to help with - just shout
[20:07] <balloons> svij, sun is still up here, hah. But yes
[20:08] <balloons> goodnight to you!
[20:08] <svij> thanks ;)
[20:28] <shrini> Thanks all