[00:12] cyphermox: still around? [00:54] back now yeah [00:55] rsalveti: pong [00:56] cyphermox: seems fine now, I just copied over the new phablet-tools that was pending in the ppa since nov-08 [00:56] ok [00:56] that had a few fixes that are useful for the emulator bootstrap [00:57] cyphermox: everything is in manual still, right? [00:57] cyphermox: which then probably means that someone had to file a landing request at that time [00:57] rsalveti: yeah [00:57] which didn't happen, so it was stuck in the ppa [00:58] but since jenkins if down, and this only affects the emulator use case, I decided to push push it directly [00:58] hey, if it's pressing just upload directly to the archive ad we'll sync it back [00:59] right, I know there might be a missing step in there [00:59] like publishing in another ppa (for different series), and also updating trunk [00:59] with the released version [00:59] but is that part of a jenkins job as well? [00:59] yes [01:00] but like I said, you could jsut as well upload phablet-tools to distro directly and I'll merge the changelog [01:00] cyphermox: right, that's kind of what I did, but I decided to copy the binary packages instead [01:00] if you can merge it manually it'd indeed be helpful [01:05] ok I'll take a look at that first thing tomorrow [01:05] great, thanks :-) === kalikiana_ is now known as kalikiana === Ursinha-afk is now known as Ursinha [05:26] love the amount of jenkinses up! I wonder what's the status of mergers? [05:29] ah, up but jobs don't launch at jatayu at least [08:40] hmm, something tried an image test it seems but with a weird version [08:44] yep [08:44] and I see stuck jobs on ex-magners [08:44] I guess vila would know ^ [08:44] ogra_: btw, can we publish 101 on saucy? request from stephan :) [08:44] ogra_: IIRC, this one is correct [08:45] didrocks, i was waiting for asac's "go" on that ... [08:46] didrocks: on it but it's mostly guess work at that point [08:46] ogra_: hum? what asac knows more than us on publishing a new image in the stable release (between minor ones, 100 and 101) [08:46] we are not talking about saucy->trusty, but rather saucy->saucy [09:08] didrocks: is mangers up already but on some other IP or something? ;) [09:09] sil2100: you do have http://q-jenkins.ubuntu-ci:8080 [09:09] but it's not fully up yet [09:10] missing executors, firewall rules for syncing on the archive admin machine and the otto machines [09:10] sil2100: I guess vila is on it, but let's sync up during a short meeting [09:10] I would have to know the IP of q-jenkins.ubuntu-ci anyways! [09:11] sil2100: hum, no, you need to change the vpn config as per wiki [09:11] sil2100: it's IP changed overnight, stop *thinkinh* about IPs ;) [09:11] :< [09:12] * sil2100 has a lot of syncing up as he does not know about any wiki [09:12] didrocks: we're on it with jibel but blocked by access rights :-/ no sudo anymore on jatayu which is the new home for q-jenkins and inherits the restricted sudo policy from m-o apparently [09:12] didrocks: those stuck jobs were started by me earlier today, just to see what's up. daily-release-executor seemed down. [09:13] Mirv: starting that slave wasn't documented for m-o and wasn't migrated, jibel just documented the needed steps but we lack the rights to process them [09:13] vila: ok [09:17] sil2100: look at the engineering ML (mails from yesterday) [09:17] vila: and do you know about the firewall? this is an orthogonal question I guess :) [09:18] didrocks: what e-mail is it? Since I don't see anything, could you give me the title? [09:18] didrocks: nope [09:18] I might not be on that mailing list? [09:19] Ah, the earlier update [09:19] Got it [09:19] sil2100: great ;) [09:19] vila: do you know who is tracking it then? [09:20] ev: ^ [09:21] (better to figure it out now than too late) === ev changed the topic of #ubuntu-ci-eng to: Ubuntu CI Engineering Team | Vanguard: ev | Landing instructions: http://paste.ubuntu.com/6292280/ | Known issues: Some services are down (1SS move) [09:23] didrocks: when you say firewall, do you mean https://rt.admin.canonical.com//Ticket/Display.html?id=65887 ? [09:23] * didrocks looks [09:25] ev: is tachash q-jenkins? [09:25] didrocks: no [09:26] didrocks: q-jenkins was magners but it's now hosted on jatayu [09:26] I don't see then the request between snakefruit and jatayu [09:26] which is used for daily release [09:26] * didrocks tries [09:27] $ rsync rsync://jatayu.ubuntu-ci [09:27] rsync: failed to connect to jatayu.ubuntu-ci (10.98.3.12): Connection refused (111) [09:27] rsync error: error in socket IO (code 10) at clientserver.c(122) [Receiver=3.0.9] [09:27] from snakefruit [09:27] * didrocks add this to the ticket [09:29] * didrocks updates the cu2d config to point the final mirror copy from jatayu.ubuntu-ci instead of magners [09:30] plars: ogra_: vila: sil2100: Mirv: ev: short meeting [09:30] didrocks: we're trying to identify service names instead of machine names, can you come up with a name for that one ? (doesn't have to be definitive we'll fix them later) [09:30] vila: the cu2d machine? [09:30] then, call it as you wish [09:31] didrocks: no, the need to use rsync, what's the service there, isos ? [09:31] didrocks: archive ? [09:32] joining in a sec [09:35] === Image #22 building === [09:37] didrocks: ogra_: lets talk about the saucy update thingy [09:37] so we have now 101? [09:37] didrocks: ok, I lost my account on that jenkins, and now when I re-created it and log in I don't see cu2d actually - what do I need to get access to it? [09:37] Or is it not visible? [09:38] asac, since a month, idling in saucy-proposed ... i remember you asked me to hold it back, but i dont remember why anymore [09:38] asac, stgraber would like us to promote it so we test the stable channel promotion [09:38] ogra_: was 101 produced _before_ we started producing trusty? [09:38] yes [09:38] it only has fixes for the uevent spam on maguro [09:39] built from SRU fixes only [09:40] can we in theory produce a 102? [09:40] didrocks: ok, I was looking in the wrong jenkins [09:41] asac, any time, yes, but there were a lot more SRUs now ... so that would need careful testing and looking at [09:41] of course [09:41] http://reports.qa.ubuntu.com/smokeng/saucy/touch_mir/ [09:41] 101 looks even better than 100 [09:41] hmm or was that under touch_ro [09:42] (where it looks even better better :) ) [09:42] sure. i just wonder if we should also test other parts of the update infra [09:42] e.g. that we can produce saucy still [09:42] that we can test it [09:42] we surely can roll another saucy image ... but i would wait until we can get test results [09:43] asac: ogra_: we can promote 101 to stable right now and kick an 102 build [09:43] yeah. how about we target saucy update for next week thu [09:43] didrocks, no [09:43] why? [09:43] didrocks, lets wait with kicking a new build until the infra is fully back [09:43] i dont want to experiment with that [09:43] so they idea was always that we release an update after one month [09:43] right [09:43] ogra_: ok, but we can promote 101 right now still :) [09:44] so lets promote 101 and roll an 102 build on monday [09:44] +1 [09:44] lets discuss final things monday [09:44] right, so that we see image testing works as expected [09:45] but we still promote 101 now? [09:45] not please [09:45] asac: can you please tell that to stephan then? [09:45] sure [09:45] asac: because he's blocking on it [09:45] thanks [09:45] he is blocking? not: he feels blocked :)? [09:45] not sure… [09:46] ok let me figure [09:46] right he wants to finish stuff on the weekend but needs results from our side [09:46] I don't know why we are not promoting 101 without any reason, but well :) [09:46] I think we're not the stackholder there [09:46] the plan was to release an image to stable channel after one month [09:46] thats what we sold to exec mgmt etc. [09:47] * ogra_ doesnt care about stakeholders ... but it is simply a better image than 100 [09:47] i.e. brings benefit to our users [09:47] that was true for the last 3 weeks :) [09:47] asac: so a month, we do release on sunday? :) [09:47] asac, well the plan was surely too that we do more regular image builds then ;) [09:47] so i am not sure why the sense of urgency today [09:47] asac, and not have something rotting in proposed for 4 weeks [09:48] didrocks: just saying: for me it was last thursday [09:48] hence i am not ready mentally [09:48] asac, because stephane needs info if the setup works [09:48] because for me it was in the longer future and i want to talk to a few folks first [09:48] well, then lets keep it til monday ... [09:48] but it seems weird to leave images rot for 4 weeks "just because" [09:49] i will check with him [09:49] once he is up [09:49] there were a lot of SRUs, if 102 breaks it will become really hard to find out why [09:49] "The goal behind this image, besides fixing the issue on maguro was to [09:49] test our upgrade process for 14.04. [09:49] " [09:49] "This end to end test was a requirement before we can officially [09:49] discontinue support of the saucy image in favor of the trusty one, so [09:49] (unrealted to when we promote 101 ... we should do more frequent proposed builds) [09:49] it'd be good if this could be done ASAP." [09:50] so I guess we can wait for Monday [09:50] but would be good not say on Monday "let's wait even more" [09:50] thats his perspective. there is more parts of the process we should pipeclean [09:50] asac: however, if we promote image 101 next week [09:50] it will mean we flip the stable channel to trusty in a month? [09:50] (to not have more than one update a month?) [09:50] didrocks, no [09:51] didrocks, stable is stable [09:51] it wont switch before we release trusty [09:51] (as stable) [09:51] ogra_: that's not what I understood… [09:51] hehe [09:51] dont argue about that part [09:51] everything else would be nonsense [09:51] i disagree with that statement, but lets not go down here [09:52] trusty is in constant flux ... it isnt stable we cant call it stable [09:52] well, if we take a snapshot, are happy about it quality-wise [09:52] thast still not stable [09:52] I don't see the difference with saucy unmaintained :) [09:52] api versions might still change etc [09:52] you have no guarantee your click packages still work and the like [09:53] (we have a devel channel for a reason ... why would you make them the same) [09:53] anyway [09:54] (just sounds pretty messed up to me ) [10:30] ogra_: is a build in progress? [10:30] hmm, might actually be done [10:30] no, not yet [10:30] and yes, see above [10:30] 22 is in the making [10:31] (cdimage part is done) [10:39] ah [10:39] * popey does "/hilight ===" [10:46] 22 installing [10:52] right [10:52] === image #22 DONE === [10:58] ogra_, popey: if you send yourself a bunch of text messages in the indicator does it change, send about 5. I'll try and get a photo [11:00] does the keyboard work again (it didnt in 21 ... after you rotated the device it didnt come up anymore) [11:00] davmor2: i sent a bunch of texts and they were all received [11:01] ogra_: unlikely, that bug was only filed last night [11:01] * ogra_ doesnt have his maguro around atm [11:01] davmor2: can you be more specific about the issue [11:01] yeah, thats what i expected [11:01] davmor2: http://popey.com/~alan/phablet/device-2013-11-15-110127.png [11:01] popey: no I get them but in the indicator the icon changes from text to globe and doesn't give the same effect if you click on it to send back [11:05] ogra_: yes, still happens in #22 [11:05] as expected then [11:34] bug 1251597 [11:34] bug 1251597 in Unity 8 "Scope header sometimes appears in the middle of the scope" [Undecided,New] https://launchpad.net/bugs/1251597 [11:34] anyone else ever see that? [11:34] (not a regression, seen it quite a bit) [11:38] popey: yeah, I think there is already a bug from asac related to it [11:38] popey, all the time [11:40] confirmed [11:42] ah okay [11:43] popey: yes [11:54] popey, ogra: http://ubuntuone.com/20tSyBkGQWHDRmOVznvWcw this is what you normally see right, every now and again I get this http://ubuntuone.com/4OPQyVQdfhliEcdIyq3jlK [11:54] ah, i never noticed that [11:55] (but i must admit i also never paid much attention to this icon) [11:55] ogra_: it's normally the 5-7 text but it's not just the icon I can't reply to this one [11:57] ogra_: I noticed it on 21 but couldn't seem to reproduce it but now I can woohoo \o/ [12:01] not seen that === MacSlow is now known as MacSlow|lunch [12:19] cool. i something on dashboard about todays image :) [12:19] even if 0% [12:19] oh, a proper versioned entry ? [12:19] * ogra_ checks [12:19] not proper [12:19] but something :) [12:20] 20131115 ? [12:20] well, thats just the dashboard going mad i think [12:20] (there was no such thing like 20131114 ) [12:20] its more than all the days before :) [12:20] e.g. there is a heart beating somewhere again [12:20] right, but most likely still not proper [12:21] of course not... i would hope we are better than 0% and still have a proper version [12:21] * ogra_ guesses another firewall issue, though i dont know the actual setup [12:23] ogra_: no, it's what we discussed this morning, the phone don't start [12:23] ah, that [12:23] yeah [12:23] so the job is stuck in install [12:23] well, the odd versioning usually means the phones are unreachable [12:23] i just didnt know why [12:23] didrocks: the jenkins timeout of 30 mins is not working :/ [12:24] psivaa: I hope it's not an uninstalled plugin :p [12:29] didrocks: Build timeout plugin is shown as installed, but not sure if the correct version was restored after the migration.. === ev changed the topic of #ubuntu-ci-eng to: Ubuntu CI Engineering Team | Vanguard: - Type: "cihelp" for help | Landing instructions: http://paste.ubuntu.com/6292280/ | Known issues: Some services are down (1SS move) === alan_g is now known as alan_g|lunch === josepht changed the topic of #ubuntu-ci-eng to: Ubuntu CI Engineering Team | Vanguard: - josepht | Landing instructions: http://paste.ubuntu.com/6292280/ | Known issues: Some services are down (1SS move) === josepht changed the topic of #ubuntu-ci-eng to: Ubuntu CI Engineering Team | Vanguard: josepht | Landing instructions: http://paste.ubuntu.com/6292280/ | Known issues: Some services are down (1SS move) [13:37] psivaa: which plugin are you concerned about? [13:38] plars: the 'Build timeout plugin' [13:38] psivaa: that's a plugin? I thought that was just part of jenkins [13:38] the isntalled version is 1.11 but 1.12. is available and the jobs are not timing out with 30 mins [13:39] plars: http://q-jenkins:8080/pluginManager/ has more details [13:39] psivaa: interesting, but I'm more concerned with why the phones are dying when we try to install at the moment :( [13:40] plars: yea, that is concerning. have seen in 4 devices [13:40] but i ran provision.sh in the same manner locally and it works fine [13:40] psivaa: I've been through multiple runs locally and can't get it to happen [13:41] plars: not sure if any kinnara side pkg versions are any different to what we had in phoenix [13:41] psivaa: I think I saw somewhere that they got some sort of new usb hub, so I wonder if that has something to do with it === MacSlow|lunch is now known as MacSlow [13:50] plars: possibly, not sure why so determinant soon after flashing. I tried doing some reboots with one of the devices and it comes up fine in adb [14:02] morning === alan_g|lunch is now known as alan_g === alan_g is now known as alan_g|tea === alan_g|tea is now known as alan_g [16:05] vila: ev: retoaded: so the otto machines... i thought those were in the DC and wired up yet. [16:05] is that not the case? [16:07] asac, they are all in the DC and wired as far as KVM, network and power are concerned. There are still a few systems that were migrated out of their desktop cases into rack mount cases that require some internal wiring (extended power leads, system fan adapters, etc ...). rfowler is picking those pieces up today so the can be brought online. [16:08] retoaded: are those the otto machiens? [16:08] or are those wired up etc.? === gatox is now known as gatox_lunch [16:08] * asac thinks those are desktop machines and its hard to migrate those into server cases [16:08] but guess rackmount is something else [16:08] asac: see https://wiki.canonical.com/UbuntuEngineering/CI/1ss-move-current-issues missing harware pieces [16:09] asac: they were desktop machines - they were reconfigured into rackable units [16:09] asac, yes, those are the otto machines [16:09] asac, the otto machines were desktop machines now in rack mount cases. The problem was that some of the smaller form factor desktops had very short outputs from the power supplies. [16:10] ok so those are our x86 test boxes. [16:10] what about the phones? [16:10] do we know yet what the issues are with running our jobs? [16:10] http://reports.qa.ubuntu.com/smokeng/trusty/touch/ [16:10] looks like the dashboard has at least one proper version there now [16:10] :) [16:10] * asac is purely color driven :) [16:11] We have 20 phones with two down [16:11] * vila sends green all over asac [16:11] lol [16:11] ev: that sounds good. but why dont the job run? [16:12] ok i heard that a new attempt is currently running [16:12] lets see [16:12] and wait a bit [16:12] * ogra_ was playing with the emulator all day ... [16:13] i doubt we'll ever get it to make the systemsettle tests :P [16:19] ok, the image smoke tests seem to be rolling along on both mako and maguro now [16:20] install_and_boot is done on both, and default has passed on maguro, should start to see results on reports.qa.ubuntu.com soon [16:25] ogra_: I'm less worried about systemsettle tests, and more concerned with whether sensitive timings in the autopilot tests will be a problem [16:28] thy most likely will [16:32] http://reports.qa.ubuntu.com/smokeng/trusty/touch/ [16:33] didrocks, are you going to be able to pick up the new source for the core apps? [16:34] sorry, I mean to say, are you going to land the stuff in the pipeline for core apps? [16:36] fginther, I'm confused by the output from the merge bot on this: https://code.launchpad.net/~nskaggs/ubuntu-rssreader-app/add-activity-indicator-check/+merge/195322. http://91.189.93.70:8080/job/generic-mediumtests-trusty/147/console [16:37] balloons: use the vanguard to start with please :-) [16:37] balloons, interesting. looks like something fell over. Will investigate [16:38] * balloons smacks balloons around a little bit [16:38] sorry cjohnston.. bad habits [16:39] plars: smoke is showing up on the dashboard now: http://reports.qa.ubuntu.com/smokeng/trusty/touch/mako/22:20131115:20131111.1/4977 [16:39] well done [16:39] doanac: yep, I posted just a bit ago ^ :) [16:40] doanac: looks promising at least, will be interesting to see the overall results [16:41] cjohnston, can I bug the vanguard about landing asks as well? :-) [16:41] balloons: probably not... I guess it would depend on what it is, but we don't yet have alot to do with that [16:44] balloons: I think only sergiusens can (once the CI infra back) [16:44] doanac, and its all GREEN !!! [16:44] asac will love that [16:44] didrocks, ty.. I figured the infrastructure might be the holdup [16:44] heh [16:45] there should be merges for everything.. so I'm curious to see it all get pulled and go green [16:45] I wouldn't too hastily hope for 'go green' in the nearest days ;) [16:46] hope springs eternal sil2100 [16:46] sil2100, well, if we stop the image tests right now they will be green :) [16:47] hahah ;) === gatox_lunch is now known as gatox [16:53] ogra_: you need to perfect the following line, " asac these aren't the results you're looking for, move along, move along" [16:53] haha [16:53] that only works in hangouts, else he cant see the specific hand wave [16:59] o| o/ o_ [17:00] sil2100: ogra_: robru: kenvandine: cyphermox: ev: vila: plars: landing meeting time! [17:00] pfft [17:00] your late [17:00] ! [17:00] * sil2100 joins [17:00] *you're [17:02] kenvandine: ping! [17:04] sil2100, pong [17:11] cyphermox, vila: oh hai [17:11] so the problem we're trying to solve, to recap: [17:11] we want the openvpn server to push the DNS server's IP address to the client, but we want this to be a fallback for whatever the person currently has set up [17:12] so if I have 8.8.8.8, I don't want to start suddenly routing all DNS requests through batun [17:12] right [17:12] so, using openvpn directly, there's just one way to do it; via the up script as per the VPN wiki page [17:13] we can tweak that to some degree to be even more useful if we were to push config to dnsmasq rather than to resolvconf directly [17:13] otherwise, NM should be able to do the right thing without change... if not, there's a bug [17:14] would `push "dhcp-option DNS 10.99.244.1"` not work? [17:14] my understanding is that even for the openvpn trick documented on the wiki you end up with the pushed dns on top of yours in /etc/resolv.conf [17:14] i.e. the one pushed from the vpn becomes the primary one [17:15] currently mine is: nameserver 10.99.244.1 [17:15] nameserver 192.168.0.254 [17:15] search ubuntu-ci [17:15] ev: correct, push dhcp-option DNS (and DOMAIN) should be all you really no [17:16] argh [17:16] .. really all you need [17:16] :) [17:16] vila: that's why you need resolvconf or dnsmasq to properly handle the multiple DNS servers and domains [17:16] cyphermox: so to be clear, that wouldn't overwrite 8.8.8.8? [17:16] shouldn't, no [17:17] yay [17:17] cyphermox: additional magic then ? [17:17] resolvconf and dnsmasq will do things differently, but both should work provided they are set up properly [17:17] ev: hold on your enthusiasm ;) [17:17] cyphermox: I'm fine if we have proper instructions to give but as of today that won't work out of the box [17:17] e.g. resolvconf depends on putting , iirc, the VPN dns first, and making sure it only responds for the domain and anything else NXDOMAIN (I think) [17:18] and dnsmasq can do everything nicely [17:18] vila: are the two push commands currently in the VPN config? [17:18] how ? [17:18] dnsmasq is able to say hey, I have this dns server but only for domain XYZ, and ship the requests only to it [17:19] cyphermox: no, but the trick I used in the wiki fakes it so that should give the same end result [17:19] right [17:20] didrocks, hey wait [17:20] robru: yep? [17:21] didrocks, i tried to kick a friends stack build but it failed immediately without even trying. [17:21] robru: can you point vila and I to it? [17:21] cyphermox: so currently the VPN dns is first but last I checked with wireshark, it receives all dns requests [17:21] vila: I'm just saying, what you need is that the push lines are there in the config, or adding the settings to NM; and NM out of the box will do the right thing, otherwise, if you use openvpn directly, you need an up / down script [17:21] robru: and please, don't land friends before we know the image is green :p [17:21] robru: but nice for trying ;) [17:21] vila: with resolvconf, that's expected [17:21] didrocks, no, i won't land it, i just wanted to see some recent friends commits built in PPA ;-) [17:21] didrocks, http://q-jenkins.ubuntu-ci:8080/view/cu2d/view/Head/view/Friends/job/cu2d-friends-head/347/ [17:21] yeah ;) [17:22] vila: with resolvconf, you'd have the first DNS server get all the requests, and possibly skip to the next if it answers with SRVFAIL [17:22] cyphermox: ha, nm is a different story, indeed I have only namesserver 127.0.1.1\nnseach ubuntu-ci there and I didn't check with wireshark [17:23] vila: I'll get you an updated up script to try [17:23] ahah, or actually, just a file that could be left in place always... [17:23] robru: "only one instance of a stack can be queued for building [17:23] " [17:24] robru: did you see this? [17:24] robru: but yeah, nothing is running [17:24] vila: ev: it seems jenkins was backed up in a very very weird state [17:24] didrocks, yes... so where does it show that the stack is already building? jenkins lists zero builders active, it says totally idle. [17:24] robru: yeah, the backup was done while stuff was running. I know where to clean that [17:25] didrocks, ok, otherwise i won't be able to build anything today, right? ;-) [17:25] robru: well, at least not friends, maybe other stacks are in a more lucky state ;) [17:26] rm: cannot remove `head/platform/stack.started': Permission denied [17:26] grumph the ACL are not restored :/ [17:26] retoaded: ^ [17:26] I can't restore the state until then [17:26] normally desktop-team can touch /var/lib/jenkins/cu2d [17:28] robru: when someone from the CI team will be available, can you ask on jenkins-q.ubuntu-ci to: [17:28] cd /var/lib/jenkins/cu2d/work [17:28] rm */*/*started */*/*building [17:28] didrocks, who? can fginther do this? [17:28] probably, not sure [17:28] we used to have access to that directory as desktop-team [17:28] so restoring the ACL can be helpful [17:30] josepht, need some help with q-jenkins: http://paste.ubuntu.com/6422247/ [17:31] geeee [17:31] didrocks, robru: josepht is your guy. He's the vanguard right now. [17:31] didrocks: do you expect *started to match both getting-started and .started ? [17:31] vila: confirm that you have dnsmasq running? [17:31] vila: getting-started? [17:31] vila: where is it? [17:32] ./head/misc.bak/ubuntu/cordova-docs/cordova-docs-3.0.0+13.10.20130930/docs/en/2.4.0/guide/getting-started for example [17:32] ev: you use NM or openvpn to connect to the VPN? [17:32] cyphermox: both [17:32] I use openvpn, as do a few other people. Others use NM [17:32] vila: that doesn't match rm */*/*started [17:32] ideally we'd like a solution for both [17:32] ev: yes [17:32] find . -name '*started' -ls | wc -l [17:32] 125 [17:32] vila@jatayu:/var/lib/jenkins/cu2d/work$ find . -name 'getting-started' -ls | wc -l [17:32] 106 [17:32] vila: yeah, this is in a subdirectory of subdirectory of subdirectory of… [17:33] not in */*/*started which is just 2 level down [17:33] ev: stop-gap while the openvpn config doesn't push the info, is a file in /etc/NetworkManager/dnsmasq.d and /etc/dnsmasq.d that contains just "server=/ubuntu-ci/10.99.244.1"... doesn't need to be removed or modified ever [17:34] ev: and I'm writing a proper up/down script to handle openvpn when the settings do get pushed [17:34] ^^ that stop-gap requires removing the up/down script for openvpn [17:35] vila: I think you are handling it and will give a sign to robru, right? (also, can you add to your list of "things to do" to restore the desktop-team ACL?) [17:35] didrocks: rm done, that's the kind of commands we don't want to have to run anymore [17:36] vila: thanks! [17:36] robru: should be good now [17:36] didrocks: ask ev, I think that's not how we want to proceed in the future, instead we would probably want to reduce that kind of access [17:36] vila: ok, as long as the vanguard can answer quickly [17:37]