=== vrruiz_ is now known as rvr === alex_abreu is now known as alex-abreu === chihchun is now known as chihchun_afk [18:26] shrini: regarding your mail about the broken autopilot test - yes, this bug is currently blocking us from executing the tests [18:28] well, not from executing - but from executing them with succcess ;) [18:29] Ok [18:29] true, they are running :-) [18:31] o/ [18:31] Failing for me [18:55] dkessel: so I got the test to run up to this point http://i.imgur.com/uFmPYTm.png this dialog is new so there is no autopilot code to handle this === rainfallshard is now known as balloons_ [18:57] DanChapman: ok, so we need a bug for fixing that [18:57] life in the cloud sometimes isn't fun [18:57] care to share your changes to get it so far? ;) [18:57] sorry, my IRC has decided to be sketchy [18:58] also, I couldn't get it to run against my local branch... it would always check out lp:ubiquity... [18:58] and we're back . . [18:58] oopsie [18:59] dkessel, I'm basically removing the test code at each step to see what's broken. [18:59] not fixed anything yet :-) [18:59] heh [18:59] nice. Well, it's more or less time. [18:59] So who's about to talk about automated ubiquity testing? [18:59] IIRC there's an emulator (what do we call them now balloons?? ) for GtkDialog so it should be easy enough to pick it up [19:00] DanChapman, we prefer 'helpers' now :-) [19:00] o/ [19:00] o/ [19:00] o/ [19:00] and I'm about to read :p [19:00] * DanChapman makes note of 'helpers' for future discussion :-) [19:02] Excellent. So I thought we could start by addressing the open questions we still have in the document, along with any other questions people might have [19:02] You can follow along here: https://docs.google.com/document/d/1TILHcVLJzJ6Cz9gaUA0D-Aa5FbdYeP-Jc19nH7Xf1lc/ [19:04] I'm essentially looking at the Issues section. The first item up for discussion is about hosting and maintaining the jenkins servers [19:04] I could help with that [19:05] We need someone who is willing to try out the current tests and see if real hardware is required or not? [19:05] Anyone have the capability of setting up a test jenkins instance to help give guidance in this area? [19:07] I can supply a temporary azure vm for testing if that's of any use. I don't have time to setup the jobs though :-) [19:07] setting up an jenkins server isn't a big deal for me, but I don't have much experience in the actual tests. I'm happy to help where I can. [19:07] dkessel: FYI bug 1480430 [19:07] bug 1480430 in ubiquity (Ubuntu) "Ubiquity autopilot test get's stuck at "Overwrite disk" dialog" [Undecided,New] https://launchpad.net/bugs/1480430 [19:08] hmm. I think in this case svij, as long as you can follow the docs and setup the jobs, we should get enough information about what we need for hosting [19:08] you can always ask for help as well, heh :-) [19:08] sure [19:08] I Can also help in setting Jenkins [19:08] either way, the setup only needs to exist long enough for us to decide on how we setup jenkins and who we have host it [19:09] once we have a decision on that, we can make a request for funding so we have a proper server to do the rest of the work on. [19:10] shrini, svij thanks for helping out. Can we leave it to you two to make some recommendations on what we should puruse for jenkins and hosting? [19:10] Ok [19:10] Will setup a vps [19:10] And install it [19:11] By tomorrow [19:11] So that we can play around it [19:11] great [19:11] \o/ [19:11] sounds perfect, thanks. [19:11] I could too ;) [19:11] svij, one or both works for me ;-) Either way both your inputs on how to setup things would be useful [19:12] svij: thanks [19:12] shrini: where do you want to host it? [19:12] Ec2 or [19:13] Digital ocean [19:14] ok [19:14] Will buy one tonight [19:14] Pm your email [19:14] Will share credentials [19:15] shrini: great :) svij@ubuntu.com [19:15] Ok [19:15] thanks! ok, so moving on to the topic [19:16] Guys, for me the ubiquity tests are failing [19:16] as you've noticed, the autopilot tests for ubiquity don't complete successfully at the moment [19:16] shrini :-) [19:16] Need to fix [19:16] yes, indeed. So DanChapman and dkessel have been working on this. What are your thoughts? Do you need any help? [19:16] :-) [19:16] Ok [19:17] Let us wait for them to fix [19:17] Thanks [19:18] Sorry, I was intending to ask DanChapman and dkessel about the tests and if they need help getting them running again, etc [19:18] I need help getting the tests to run against my local branch :p [19:19] balloons: not sure how much time I can commit to fixing them myself (not sure how busy dkessel is either) so help would be great. [19:19] is anyone interested in helping out in the area? [19:19] I also have limited time during the timeframe we're talking about [19:19] I will try [19:19] dkessel you just pass in --shareddir=/path/to/ubiquity/ [19:19] But need docs to start [19:19] DanChapman, ack. I imagine your expertise will be most useful but understand you have commitments to dekko [19:20] DanChapman: oh okay =) [19:20] I'm happy to help anyone willing to get involved with them. It's a beast to get your head into so I wouldn't leave anyone dangling :-) [19:20] DanChapman: is there anything specific to debugging ubiquity and AP tests? [19:20] I mean, like a guide [19:21] or perhaps you can write something up? [19:22] The only guide is the README atm. I could write something up tomorrow morning [19:22] do yo uwant it on the wiki or just in the src dir? [19:23] Wiki [19:23] excellent. Once you finish we can recruit some folks again to give help specifcally for the tests [19:23] Would be great [19:23] DanChapman: I would say source actually [19:23] We can link from the wiki or even clone I guess. But the source is a great place to keep it long term [19:24] yeah I was thinking the same. I'll do that then [19:24] ok excellent. And thanks for diving in shrini . It'll be a little bit of a learning curve at first, but don't be intimated by the size of the GTK tree [19:26] :-) [19:26] Will learn [19:26] alright, those are the primary tasks to tackle, but there's more things still undecided in the doc. Do you want to talk about reporting at least for now? [19:27] * svij has a few questions for jenkins-hosting [19:27] svij, sure. ask away [19:27] oh, they're in the doc somehow, wanted to ask about the jenkins slaves [19:28] balloons: so ideally you want reports on the tracker? [19:28] so, do we really need physical hardware for the tests? [19:29] DanChapman, we'll hold that thought for on sec :-) [19:29] svij, that's what we are hoping you and shrini will answer for us [19:29] We don't think we do, but CI has hinted that we do [19:29] well jenkins master is fine in a VM [19:29] svij, right, that's a vm either way [19:30] I didn't look into the tests yet, if they run fine in a vm, thats fine too [19:30] we could also run it on the master [19:30] balloons: I don't read that hint that way [19:30] flocculant, oh? [19:30] assuming you mean "(it uses a local kvm instead of depending on the cloud" [19:30] flocculant, I'm referring to the text left by CI in the RT wxl filed [19:31] from Evan? [19:31] Let me paste it [19:31] shrini: are you an ubuntu member? Asking because it might be better if I set up the jenkins master, so I can request the money from the donations program. [19:31] svij: or balloons does that [19:32] being as he's on the team that says yes or no ... [19:32] if he can :) [19:32] mmm, probably better for svij to do it, heh. But yes, I'm part of the team who decides, hah. [19:32] :) [19:32] http://paste.ubuntu.com/11975277/ is the text from CI/Evan [19:33] anyway, I think that indeed we'll be able to run everything on a cloud server [19:33] we simply need to verify it [19:33] balloons: right - so I read the kvm/cloud bit to mean they used local kvm so they didn't need cloud - not that they couldn't [19:33] oh ok, is that fine for you shrini, if I set it up and give you access to the jenkins instance? [19:34] balloons: I guess it's not particularly clear :) [19:34] flocculant, right, it's not :-) [19:34] so we do need 2 vms? One jenkins master and a slave where the tests runs? [19:34] rt's ... [19:35] ok, any other questions on jenkins before we talk about reporting? [19:35] ohh, svij. We can go with whatever setup you guys recommend [19:35] I was thinking having a master and a few slaves would be best. The images arrive more or less at the same time [19:36] it would be nice to have a little parallization [19:36] right [19:36] svij: ok [19:36] That's fine [19:36] You set up [19:36] we could spin up a few vms as slaves and drop them after running the tests, that saves costs ;) [19:36] how many is a question for you guys to answer as well. So how many instances to obtain, how big, how many slaves? [19:37] svij, ahh right.. None of them have to be permanent [19:37] balloons: re when images arrive - there are quite a few hours between [19:37] besides master [19:37] Mail me once done [19:37] I didn't have a look into the tests yet, so I cant say much about how much VMs we need. [19:37] shrini: I do [19:37] xubuntu turns up ~10am - Lubuntu ~5pm [19:38] flocculant, wow, really? That's more than I thought. I was thinking everything in about a 4 hour window [19:38] Tshrinivasan@gmail.com [19:38] balloons: kubuntu is ~5am [19:38] balloons: from the cd-build-logs [19:38] sounds like we will have a few jobs anyway who doesn't run on the same time? [19:38] don't* [19:39] svij, yes, several tests against the same image, so we could still have more than 1 slave [19:39] anyways, do you have the full list of questions now to answer, heh? [19:39] flocculant: given that I'm not going to be doing anything else - if when they run is important I can grab those times [19:39] silly /me [19:39] flocculant, sure, why not. Can you get an average time / timeframe for the images? [19:39] yep [19:40] who do we need that for? [19:40] all flavors, expect kubuntu [19:40] ok [19:40] they will all be running, minus kubuntu as it uses a different installer [19:40] not Ubuntu? [19:41] we'e not concerned with ubuntu proper in this case [19:41] ok - just double checking :) [19:41] so all on iso.qa - except Kubuntu and Ubuntu [19:41] flocculant, feel free to include it for giggles though [19:41] ok, anything else on jenkins? [19:42] the tests will not run for the normal ubuntu? [19:43] no new jenkins questions for now, from me. [19:43] svij, they do run for normal ubuntu, but we were focusing on testing flavors specifically at first [19:43] balloons: ahh ok [19:43] we can certainly run them all honestly [19:43] I missed that part. [19:43] kubuntu is excluded only because they use the qt installer, and we have no tests for it [19:44] thought perhaps reaching out to them and seeing what they think about a testsuite would be useful at some point [19:44] anyways, moving on to reporting then and DanChapman's question [19:44] oh, quick question: I'll use digitalocean for 10$/month with backups (+20% … 2$) is that fine? [19:44] btw, for your sanity, I'll send along notes of what we discussed to the list [19:45] svij, yes, that's fine. For the costs, we simply have to outline what we are using it for and why. That is not a large amount of money [19:45] yep, ok :) [19:46] alright, so on reporting, let's just get some ideas from everyone. It's not something we have to decide now, and we've been chatting for a bit already [19:46] the basic idea is we want to display results to everyone about what is happening with the tests [19:47] this information is especially useful to folks running manual tests on the images. [19:47] the put it on iso.qa at Notices for a basic pass/fail line seems ideal [19:47] +1 ^^ [19:47] but more detail elsewhere for those who want to look at that when necessary [19:48] +1. that way we have a dashboard in a well-known place [19:48] wow, lots of love for that idea. [19:48] assuming we can send Xubuntu to Xubuntu, etc etc [19:48] I was going to say, for the initial work, it would be nice to keep reporting requirements to a miniumum [19:48] balloons: simple often is best [19:48] you mean this one? http://iso.qa.ubuntu.com/qatracker/milestones/340/builds/98875/testcases [19:49] svij: yea [19:49] yea, basically that text at the top of the page [19:49] we could include whatever text in there we wish [19:49] or alternatively - list them at http://iso.qa.ubuntu.com/qatracker [19:49] but just list fails [19:49] who needs to know it passed [19:50] there's a python API for that site. Essentially then we would need to get the build information and push an update to the tracker via the API [19:51] now that includes more work as someone will need to write a script / jenkins job to do this [19:51] svij, shrini is there any easier options for displaying the information on jenkins itself? [19:51] and are folks in favor of looking at jenkins jobs and parsing the information at first? [19:52] balloons: that's not a problem for me [19:53] displaying a status on jenkins is the default (or didn't I understand the question?) [19:53] svij, yea, indeed. I guess I was asking if there was a nitfy plugin you knew of, or a cheap way to display results from builds without us having to write something [19:53] or me, the rss feed and email on fails is also a handy thing to have [19:54] ^^ that's an idea [19:54] yea, that sounds good DanChapman [19:54] yeah, e-mails on fails are good [19:54] we could have a mailing list that posted results [19:54] you could signup [19:54] emails are the easiest [19:54] but feeding an API with a few data doesn't sound too hard anyway (as a second step) [19:55] ok, we won't decide right away, but I think both approaches sound lightweight enough for the initial work [19:55] Let's defer making the decision until we know a little more about the jobs and what it would talk to send updates [19:56] sound good? [19:56] +1 [19:56] yep [19:56] yep [19:56] With that, I'm through chatting about everything I wanted to cover today. The last bit I wanted to ask is when can everyone meet again? [19:57] I've got a few action items noted that folks have agreed to do, I think giving everyone some time before we meet would be good. Next time we should be able to decide on jenkins/hosting questions, and have a good plan for getting the AP tests working (or even fixed) [19:58] is this time / day good for everyone? [19:58] yep [19:59] yeah good for me. [19:59] this time is good for me - days usually not an issue [19:59] * balloons realizes he'll be out next Friday travelling [20:00] hmm.. [20:00] thursday? [20:00] quick point - mythbuntu [20:00] that works fine for me [20:01] seems test runner for ubuntu-gnome is broken :-( [20:01] is that going to be looked at - because they're LTS to LTS and also builds have been failing [20:01] thursday's okay [20:02] thursday is also fine for me [20:02] flocculant, excellent point. Do you want to include that in your findings for when the images build? [20:02] perhaps you can suggest which images we start with first, and which to bring on later [20:02] I wouldn't want us to try and test images that will give us trouble at first.. and since they are LTS only anyway, it adds nothing for this cycle [20:02] balloons: that's how I know it's been failing :D [20:03] does mythbuntu have any additional steps in ubiquity like edubuntu does? [20:03] excellent, thursday at 1900 it shall be. [20:03] * DanChapman makes note in calendar [20:03] balloons: looks like Myth started fails on 21st July [20:04] ack [20:04] http://pad.ubuntu.com/JenkinsTimes [20:05] there you go - -that's the start/finish times for all except Ubuntu/Kubuntu [20:05] Ok, thanks again everyone for meeting. I'll see you next Thursday @ 1900 UTC. I'll send a summary of what we discussed to the list [20:05] flocculant, thanks. I'll add to the doc [20:05] thanks guys! [20:05] thanks for organizing and notekeeping balloons [20:05] or you can add them [20:05] you are all most welcome. Thanks so much for helping make this a reality! [20:05] thanks, I'll setup the jenkins tomorrow [20:05] balloons: ta not got a gmail [20:06] sounds good. Enjoy your evenings! [20:06] night all [20:07] evening? nearly bedtime … ;) [20:07] if there's anything people think I might be able to help with - just shout [20:07] svij, sun is still up here, hah. But yes [20:08] goodnight to you! [20:08] thanks ;) [20:28] Thanks all