| rogpeppe | mornin' all, and Happy New Year! | 09:01 |
|---|---|---|
| axw | hey rogpeppe, happy new year :) | 09:04 |
| rogpeppe | axw: hiya | 09:04 |
| rogpeppe | axw: hope you've had a good break | 09:04 |
| axw | rogpeppe: yeah it was pretty relaxing, and you? | 09:04 |
| rogpeppe | axw: yeah, good thanks. | 09:05 |
| natefinch | fwereade, rogpeppe: I guess no meeting this morning? | 10:03 |
| rogpeppe | natefinch: i dunno, i'd assumed there would be one, just to get us off to a start | 10:04 |
| natefinch | rogpeppe: maybe most people are taking these two days off too? | 10:05 |
| rogpeppe | natefinch: that's entirely possible | 10:05 |
| fwereade | rogpeppe, would you kjoin us quickly please? | 10:06 |
| rogpeppe | fwereade: where? | 10:07 |
| fwereade | rogpeppe, https://plus.google.com/hangouts/_/calendar/bWFyay5yYW1tLWNocmlzdGVuc2VuQGNhbm9uaWNhbC5jb20.8sj9smn017584lljvp63djdnn8 | 10:07 |
| === gary_poster|away is now known as gary_poster | ||
| rogpeppe | feeling stupid trying to upgrade my system - i click the "Upgrade" button and nothing at all happens. any clues? | 14:10 |
| rogpeppe | hmm, now running do-release-upgrade directly, with crossed fingers | 14:13 |
| sinzui | rogpeppe, make sure ot have go installed from upstream or a PPA. trusty has golang 1.2. I haven't had a successful test run using it. | 14:30 |
| rogpeppe | lunch | 14:38 |
| rogpeppe | sinzui: interesting - i've been using 1.2 for a while and it seems to work ok | 14:47 |
| rogpeppe | sinzui: we'll see how it goes - am currently downloading saucy; hopefully will succeed in upgrading to trusty later on today | 14:47 |
| rogpeppe | hmm, do-release-upgrade has now hung up | 15:17 |
| rogpeppe | last thing it printed was "Setting up adduser (3.113+nmu3ubuntu2) ..." | 15:17 |
| rogpeppe | it seems to be running dpkg | 15:17 |
| rogpeppe | but dpkg is stripped so difficult to find out what it's *actually* trying to do | 15:17 |
| rogpeppe | hmm, looks like it *might* be blocked trying to read from stdin | 15:21 |
| rogpeppe | oh, darn, i broke it | 15:24 |
| * rogpeppe wishes that do-release-upgrade could continue where it left off | 15:24 | |
| rogpeppe | ha ha | 15:35 |
| rogpeppe | diff: \/etc\/cups\/cups\-files\.conf: No such file or directory | 15:35 |
| rogpeppe | diff: \/etc\/cups\/cups\-files\.conf\.dpkg\-new: No such file or directory | 15:35 |
| rogpeppe | rebooting | 15:37 |
| rogpeppe | well, i now seem to be running Trusty Tahr. that was actually quite painless. | 17:02 |
| sinzui | rogpeppe, The only problem I have had with trusty are reports that X/Mir crashed when I didn't see it crash | 17:09 |
| rogpeppe | sinzui: i'm hoping that by running it now, my bug reports might actually be useful to someone before the release... | 17:10 |
| rogpeppe | sinzui: only problem i've seen so far is: | 17:11 |
| rogpeppe | "bzr 2.6.0 is too new for pipeline 1.4" | 17:11 |
| sinzui | rogpeppe, we see spurious failures in CI regarding "unable to connect to environment" most often with azure and canonistack. The revision that just landed has never passed on canonistack, and I cannot tell if cloud or juju is at fault. Do you see anything insightful in this log: http://162.213.35.54:8080/job/canonistack-upgrade/452/consoleFull#-2115410258bc9712e0-ba8d-4476-87e1-593fdc4da857 | 17:13 |
| sinzui | abentley, ^ can rogpeppe use/test the patch to bug 1209131 to get pipelines working again | 17:15 |
| _mup_ | Bug #1209131: Pipeline reports it is too old for newly released bzr 2.6 <bzr-pipeline:New> <https://launchpad.net/bugs/1209131> | 17:15 |
| rogpeppe | sinzui: nothing jumps out at me | 17:22 |
| rogpeppe | sinzui: what is wait_for_agent_update actually doing? | 17:23 |
| rogpeppe | sinzui: and this line is weird - do you know where it's coming from? | 17:23 |
| rogpeppe | <test-release-canonistack> unknown: 1, 0, 2 | 17:23 |
| sinzui | rogpeppe, wait is sleeping, call status, parse for agent state | 17:25 |
| rogpeppe | sinzui: so it succeeds n times, then fails? | 17:26 |
| sinzui | rogpeppe, I am not sure about wht the numbers mean. Each is the status of unit + the state-server | 17:30 |
| rogpeppe | sinzui: i'm trying to see what kind of failure we're observing here | 17:31 |
| rogpeppe | sinzui: is the problem that we're making some change, then can't connect, or is it that we're suddenly unable to connect for some unknown reason? | 17:31 |
| sinzui | ah, well maybe I can do better than read the test is doing. I think I can get the log from the upgrade | 17:32 |
| sinzui | rogpeppe, This is the all machines log of the failed upgrade test: http://162.213.35.54:8080/job/canonistack-upgrade/452/artifact/artifacts/all-machines-test-release-canonistack.log.gz | 17:33 |
| rogpeppe | sinzui: so is this error happening just after an environment has been upgraded? | 17:37 |
| sinzui | rogpeppe, yes | 17:37 |
| rogpeppe | sinzui: i'm not that surprised then | 17:37 |
| rogpeppe | sinzui: when you upgrade an environment, the API server will restart, dropping any existing connections | 17:38 |
| sinzui | rogpeppe, CI just passed r2179. The rev is blessed. | 17:38 |
| * rogpeppe needs a translation of that :-) | 17:38 | |
| rogpeppe | sinzui: what does "blessed" mean in this context? | 17:39 |
| sinzui | rogpeppe, CI passed r2179 after 5 attempts. We could release it as 1.17.1. The log I am showing you is from the fourth failed attempt | 17:40 |
| sinzui | bless == release candidate | 17:40 |
| rogpeppe | sinzui: i see, so 1/5 tests passed... | 17:40 |
| sinzui | yep. | 17:40 |
| rogpeppe | sinzui: i suspect it might pass more often if you waited 20 seconds after upgrading before trying to connect again | 17:41 |
| sinzui | rogpeppe, okay. thank you. I will try that. | 17:41 |
| rogpeppe | sinzui: another possibility is to treat "connection is shut down" as a soft error, and retry again without failing the CI | 17:42 |
| sinzui | rogpeppe, I agree. If you expect the API to drop all connections and let natural error recovery to take over, then the test is too strict | 17:43 |
| rogpeppe | sinzui: there is perhaps an argument that the command line should be more resilient about what happens when an environment is upgraded, but it's not an easy problem to solve well. | 17:44 |
| sinzui | I am always will to retry. Since the test is attempting to behave as a devop, it should retry | 17:45 |
| === gary_poster is now known as gary_poster|away | ||
| === gary_poster|away is now known as gary_poster | ||
| * rogpeppe is done for the day | 19:06 | |
| === gary_poster is now known as gary_poster|away | ||
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!