| === bradm_ is now known as bradm | ||
| axw | hey thumper, have a good week away? | 01:12 |
|---|---|---|
| thumper | axw: yeah, I did | 01:12 |
| thumper | quite relaxing | 01:12 |
| axw | cool | 01:13 |
| axw | good week for it ;) | 01:13 |
| axw | it was a little bit crazy last week, but all turned out well | 01:13 |
| thumper | cool | 01:14 |
| thumper | I'm tyring to get the team's prioritized list | 01:15 |
| thumper | so we can focus on what to do next | 01:15 |
| thumper | I'm finishing off the kvm support | 01:15 |
| axw | thumper: I'm pretty close to having synchronous bootstrap done I think; I've got some updates to do per jam's suggestion of toning down the output | 01:17 |
| * thumper nods | 01:17 | |
| thumper | cool | 01:17 |
| axw | thumper: waiting on some feedback from fwereade regarding state changes for API-based destroy-environment | 01:17 |
| axw | when that's done I can finish off destroy-env for the manual provider | 01:17 |
| thumper | that'll be good to finish off | 01:17 |
| thumper | axw: I think it would be great to create an "ask ubuntu" question around "how do I use juju with digital ocean", and answer yourself demonstrating the manual bootstrap / manual provisioning | 01:18 |
| axw | thumper: I also looked into manual provisioning into non-manual env. There's one particularly difficult bit, which is that agents/units/etc. address each other with private addresses, which isn't likely to work with an external machine | 01:18 |
| thumper | hmm... | 01:19 |
| * thumper nods | 01:19 | |
| axw | thumper: SGTM, tho Mark Baker (I think it was him) wanted me to get in touch before publicising widely | 01:19 |
| thumper | axw: what about crazy shit like running a sshuttle | 01:19 |
| thumper | on the bootstrap node | 01:19 |
| axw | yeah something like that may be necessary | 01:19 |
| axw | tho I'm starting to think that it's a lost cause, and we should just handle it with cross-environment | 01:20 |
| axw | haven't gotten too deep into it yet | 01:20 |
| thumper | hmm... | 01:24 |
| axw | wallyworld_: you can just do "defer os.Chdir(cwd)" - no need to change it now though | 02:06 |
| wallyworld_ | ah bollock | 02:06 |
| wallyworld_ | s | 02:06 |
| wallyworld_ | the tests didn't break regardless, just thought i'd be complete | 02:07 |
| wallyworld_ | i'll fix as a driveby | 02:07 |
| axw | cool | 02:09 |
| wallyworld_ | thumper: initial framework https://code.launchpad.net/~wallyworld/juju-core/ssh-keys-plugin/+merge/197310 | 03:30 |
| thumper | wallyworld_: ack | 03:34 |
| * thumper nips to the supermarket for dinner stuff... | 03:39 | |
| jam1 | axw: "agents address each other with private addresses", in the case of AWS I *think* we use the DNS name, which appropriately resolves private/public based on where you make the request. But I'll agree that you don't guarantee routing. Assuming a flat network for agent communication probably isn't going to change this cycle, but we may get there eventually. | 04:19 |
| axw | jam1: it may not be a difficult problem to solve, but I first came up against this when the agents were deployed with stateserver/apiserver set to the private address | 04:21 |
| axw | that was on EC2 | 04:21 |
| jam1 | axw: yeah, code has been changing in that area. It would appear something is resolving that address and passing around the IP, rather than the DNS name. | 04:26 |
| === jam1 is now known as jam | ||
| axw | jam: I don't think it was the IP - it was an internal DNS name | 04:28 |
| axw | couldn't be resolved outside EC2 | 04:28 |
| axw | bbs, getting lunch | 04:28 |
| axw | jam: still working out the kinks, but synchronous bootstrap will look something like this in the near future: http://paste.ubuntu.com/6507781/ | 05:15 |
| jam | axw_: looks pretty good. | 06:04 |
| === axw_ is now known as axw | ||
| axw | the apt stuff should be at the beginning obviously :) | 06:06 |
| dimitern | morning | 07:11 |
| rogpeppe | mornin' all | 08:43 |
| jam | jamespage: poke about mongo/mongoexport/mongodump/etc | 09:15 |
| jamespage | jam: morning | 09:15 |
| jam | jamespage: I hope you had a good weekned | 09:16 |
| jam | weekend | 09:16 |
| jamespage | yes thanks - how was yours? | 09:16 |
| jam | pretty good. It was Thanksgiving into UAE National Day, and Expo 2020 celebration, so the weekend gaps were all a bit silly. My son will end up with a 5-day weekend | 09:16 |
| jamespage | jam: nice | 09:18 |
| jam | jamespage: so I responded to your questions about what tools we need to include. I'd like to know more from your end what the costs of it are so that we can find a good balance point. | 09:18 |
| jam | morning fwereade, I had a question for you when you're available | 09:19 |
| fwereade | jam, heyhey | 09:19 |
| jam | fwereade: namely, the 14.04 priorities/schedule stuff | 09:19 |
| fwereade | jam, I couldn't see anything by mramm, so I just copied them straight into the old schedule doc | 09:20 |
| fwereade | jam, https://docs.google.com/a/canonical.com/spreadsheet/ccc?key=0At5cjYKYHu9odDJTenFhOGE2OE16SERZajE5XzZlRVE&usp=drive_web#gid=2 | 09:20 |
| jam | fwereade: yay, I was hoping they would end up there | 09:21 |
| jamespage | jam: just reading | 09:25 |
| jam | k | 09:25 |
| jamespage | jam: OK - I see what you are saying - I'll respond on list so everyone can see | 09:26 |
| === axw__ is now known as axw | ||
| axw | fwereade: re https://codereview.appspot.com/28880043/diff/20001/state/state_test.go#newcode567, there's code in AddServce/AddMachine* to assert environment is alive | 09:55 |
| axw | and accompanying tests | 09:55 |
| axw | actually that is the test - maybe I misunderstood you | 09:55 |
| fwereade | axw, yeah -- but Ididn't see a test for what happens if the env was alive as the txn was being calculated, but became not-alive before the txn was applied | 09:56 |
| fwereade | axw, look in export_test.go -- inparticular, SetBeforeHook, I think -- for a mechanism that lets you change state at that point | 09:57 |
| fwereade | axw, there are a few other tests that use it... but not many, the capability is relatively recent | 09:58 |
| axw | fwereade: thanks. just so I understand- you mean to check what happens between the initial environment.Life() check, and when the transaction is executed? | 09:58 |
| fwereade | axw, yeah,exactly | 09:59 |
| axw | okey dokey, I will add that in | 10:00 |
| fwereade | axw, cheers | 10:00 |
| axw | fwereade: and this one: https://codereview.appspot.com/28880043/diff/40001/state/environ.go#newcode84 -- I thought I was following your earlier advice ("I kinda hate this, but we're stuck with it without a schemachange for a total-service-refcount somewhere..."), but I suppose I've misunderstood you | 10:00 |
| axw | jam: I'd appreciate another look at https://codereview.appspot.com/30190043/ if you have some time to spare later | 10:05 |
| fwereade | axw, indeed, I was a bit worried I might have misunderstood myself there | 10:05 |
| axw | heh :) | 10:05 |
| fwereade | axw, I don't *think* I ever said that you'd have to remove everything before an env could be destroyed, but I might have said something else so unclearly that the distinction was minimal | 10:06 |
| fwereade | axw, it's always been manual machines that are the real problem | 10:06 |
| axw | indeed | 10:07 |
| axw | they are not much like what's preexisting | 10:07 |
| fwereade | axw, yeah | 10:07 |
| fwereade | axw, I think we're massaging them into shape though | 10:07 |
| axw | fwereade: how about I just drop that comment and stick with the cleanup. did you have any thoughts on cleaning up machine docs? | 10:07 |
| axw | i.e. destroying machines as environment->dying | 10:08 |
| fwereade | axw, I feel that bit should be reasonably short-circuitable | 10:08 |
| fwereade | axw, ie just kill all the instances and be done with it | 10:09 |
| fwereade | axw, the only reason not to is, again, the manual machines | 10:09 |
| axw | yeah, except for manual machines | 10:09 |
| fwereade | jinx ;p | 10:09 |
| axw | fwereade: so, one option is to have it schedule a new cleanup for machines that *doesn't* imply --force | 10:10 |
| axw | and that will wait for all units to clean up (as a result of services being cleaned up) | 10:10 |
| fwereade | axw, so yeah I could maybe be convinced that *if* there are manual machines we should destroy all the others | 10:10 |
| fwereade | axw, well, there's not actually much point waiting | 10:10 |
| fwereade | axw, I am starting to think that actually destroy-machine should always act like --force anyway | 10:11 |
| axw | fwereade: I was just thinking about leaving units in a weird state, but maybe we don't care? | 10:11 |
| axw | doesn't matter for non-manual of course | 10:11 |
| fwereade | axw, if you don't want a machine, you don't want it, and if the whole thing's being decommissioned there's no point caring about the units in a weird state | 10:11 |
| fwereade | axw, yeah, doesn't apply to manual machines ;p | 10:12 |
| axw | yeah... except if you want to reuse that machine | 10:12 |
| axw | heh | 10:12 |
| axw | ok, I'll play with that some more tomorrow | 10:12 |
| fwereade | axw, destroy-machine implies no reuse, I think | 10:13 |
| fwereade | axw, whereas manual... often implies reuse | 10:13 |
| axw | fwereade: at worst, users can destroy-machine the ones they care about | 10:14 |
| axw | then wait for things to clean up | 10:14 |
| axw | then do destroy-env | 10:14 |
| fwereade | jam, seeding a thought: can we get rudimentary versions into the API by 1.18? can chat about it more after standup | 10:14 |
| axw | assuming destroy-machine doesn't change to always --force | 10:14 |
| fwereade | axw, yeah, indeed | 10:14 |
| fwereade | axw, manual machines again are the reason not to always --force | 10:15 |
| fwereade | blasted things ;) | 10:15 |
| * fwereade has to pop out a mo, bbs | 10:15 | |
| * axw has to go | 10:15 | |
| axw | adios | 10:15 |
| jam | have a good evenning axw | 10:15 |
| axw | thanks jam | 10:17 |
| jam | TheMue: rogpeppe, mgz, standup? https://plus.google.com/hangouts/_/calendar/am9obi5tZWluZWxAY2Fub25pY2FsLmNvbQ.mf0d8r5pfb44m16v9b2n5i29ig | 10:47 |
| jam | wallyworld_: ^^ | 10:48 |
| TheMue | jam: i'm already each day in the qa timeout | 10:48 |
| jam | TheMue: absolutely, just didn't know if you were coming to ours, so you're welcome if you want | 10:49 |
| TheMue | jam: so i think it's better to take part on thursdays, there's more than status | 10:49 |
| TheMue | jam: right now i'm in an interesting testing, maybe tomorrow | 10:49 |
| jam | TheMue: have a good day, then. | 10:52 |
| TheMue | jam: thank, u2 | 10:54 |
| TheMue | jam: short info about current status, built a "Tailer" for filtered tailing of any ReadSeeker and writing to a Writer. looks good, right now testing it. | 10:57 |
| TheMue | jam: it's part of the debug-log api | 10:57 |
| * TheMue => lunch | 11:22 | |
| natefinch | jam: I can try reaching out to the github.com/canonical guy through linked in... we're like 3rd degree relations. Seems like he keeps it up to date there. | 13:00 |
| jam | natefinch: go for it.I'll forward you the email I tried sending earlier | 13:00 |
| natefinch | jam: cool | 13:00 |
| * rogpeppe3 goes for lunch | 13:25 | |
| === gary_pos` is now known as gary_poster | ||
| sinzui | jam I still see my blocking bug stuck in triaged: https://launchpad.net/juju-core/+milestone/1.16.4 Is bug 1253643 a duplicate of bug 1252469 | 14:15 |
| _mup_ | Bug #1253643: juju destroy-machine is incompatible in trunk vs 1.16 <compatibility> <regression> <juju-core:Fix Committed by jameinel> <juju-core 1.16:Fix Committed by jameinel> <https://launchpad.net/bugs/1253643> | 14:15 |
| _mup_ | Bug #1252469: API incompatability: ERROR no such request "DestroyMachines" on Client <terminate-machine> <juju-core:Triaged> <https://launchpad.net/bugs/1252469> | 14:15 |
| jam | sinzui: that gets bumped to 1.16.5 because we moved the DestroyMachines code out of 1.16.4 | 14:16 |
| jam | I'll fix it | 14:16 |
| sinzui | jam, then the bug is closed :) I will start motions for the release of 1.16.4 | 14:16 |
| jam | sinzui: right, the *key* thing is NEC is already using something (1.16.4.1) which we really want to make 1.16.4, and then make the next release 1.16.5 | 14:17 |
| jam | I forgot to check the milestone page, as another of the bugs gets bumped as well | 14:17 |
| sinzui | uhg | 14:17 |
| jam | sinzui: did we end up re-landing bug #1227952 ? | 14:18 |
| _mup_ | Bug #1227952: juju get give a "panic: index out of range" error <regression> <goyaml:Fix Committed by dave-cheney> <juju-core:Fix Committed by dave-cheney> <juju-core 1.16:Fix Committed by sinzui> <https://launchpad.net/bugs/1227952> | 14:18 |
| sinzui | relanding? I don't know. I can check the version | 14:19 |
| sinzui | in deps | 14:19 |
| jam | sinzui: I pivoted to remove everything in lp:juju-core/1.16 back to 1.16.3 so that we could land the things that were critical to NEC, and then get the non-critical stuff out in the next one | 14:20 |
| jam | I think bug #1227952 needs to target 1.16.5 | 14:20 |
| _mup_ | Bug #1227952: juju get give a "panic: index out of range" error <regression> <goyaml:Fix Committed by dave-cheney> <juju-core:Fix Committed by dave-cheney> <juju-core 1.16:Fix Committed by sinzui> <https://launchpad.net/bugs/1227952> | 14:20 |
| sinzui | jam, ah, the missing info from the Wednesday conversation. okay. A fine strategy | 14:21 |
| jam | sinzui: as soon as you give me the go ahead I have lp:~jameinel/juju-core/preparation-for-1.16.5 to bring back all the stuff | 14:21 |
| sinzui | understood | 14:22 |
| === gary_poster is now known as gary_poster|away | ||
| === gary_poster|away is now known as gary_poster | ||
| benji | marcoceppi: It is a work-in-progress, but I would like your initial impressions of lp:~benji/+junk/prooflib-first-cut | 15:51 |
| marcoceppi | benji: I thought proof was already free of all sys.exit calls already? | 15:51 |
| benji | marcoceppi: I haven't looked for sys.exit specifically yet, so far I have concentrated on project structure | 15:52 |
| marcoceppi | benji: why seperate this from charm-tools? | 15:52 |
| marcoceppi | I'm confused as to the goal | 15:53 |
| marcoceppi | benji: care to jump on a hangout? | 15:53 |
| benji | the goal is to create a library that third-parties can consume | 15:53 |
| benji | I have a call in a couple of minutes, but I can do it after that (say 30 minutes or so from now) | 15:53 |
| marcoceppi | benji: I'm confused as to why charm-tools can't be that library? | 15:54 |
| marcoceppi | sounds good! | 15:54 |
| mattyw | stupid question: I'm trying to query the mongodb that gets started when I run make check - but I can't work out where to get the right creds to start a mongo shell connection | 15:58 |
| rogpeppe3 | mattyw: what's "make check"? | 16:05 |
| === rogpeppe3 is now known as rogpeppe | ||
| rogpeppe | mgz: ping | 16:05 |
| mattyw | rogpeppe, running all the tests | 16:14 |
| mgz | rogpeppe: pong sorry, not paying attention | 17:10 |
| rogpeppe | mgz: np, am currently in a call | 17:11 |
| mgz | yell after if you still need me | 17:11 |
| mgz | will be around at least another couple of hours | 17:11 |
| rogpeppe | niemeyer: ping | 17:49 |
| niemeyer | rogpeppe: pongus | 17:49 |
| rogpeppe | niemeyer: i wondered if you might have a moment to join us in a hangout - i'm trying to resolve some issues after restoring a mongodb | 17:50 |
| niemeyer | Sure thing | 17:51 |
| niemeyer | Link? | 17:51 |
| rogpeppe | niemeyer: https://plus.google.com/hangouts/_/calendar/am9obi5tZWluZWxAY2Fub25pY2FsLmNvbQ.mf0d8r5pfb44m16v9b2n5i29ig?authuser=1 | 17:51 |
| natefinch | Fixing tests with <-time.After(x) ...bad or truly terrible? | 18:03 |
| hatch | Hey all, is there a whitelist of characters for service names in core? I need to set up proper validation in the GUI https://bugs.launchpad.net/juju-gui/+bug/1252578 | 18:05 |
| _mup_ | Bug #1252578: GUI shows invalid service name as valid when deploying ghost <juju-gui:Triaged> <https://launchpad.net/bugs/1252578> | 18:05 |
| hatch | found the regex | 18:10 |
| === BradCrittenden is now known as ba | ||
| === ba is now known as bac | ||
| sinzui | abentley, i am have a bad day tearing down envs. Local required manual deletions of lxc dirs and symlinks. I got two timeouts destroying azure | 19:18 |
| abentley | sinzui: Ugh. | 19:18 |
| abentley | sinzui: I am starting a test run on the new instances. (juju-ci-2 env) | 19:19 |
| sinzui | fab | 19:19 |
| abentley | sinzui: Is it possible that you recently destroyed an aws environment? | 19:45 |
| sinzui | abentley, in the last 15 minutes I destroyed test-aws | 19:46 |
| sinzui | and test-hp | 19:46 |
| sinzui | abentley, I don't think these can collide | 19:46 |
| abentley | sinzui: This log is baffling: http://162.213.35.54:8080/job/aws-upgrade-deploy/71/console | 19:47 |
| abentley | sinzui: It bootstraps successfully, and later reports it's not bootstrapped. | 19:47 |
| sinzui | hmm, maybe the control-buckets are the same. | 19:47 |
| abentley | sinzui: Most likely explanation is it was torn down after bootstrapping by one of us or our pet machines. | 19:47 |
| abentley | sinzui: Ends with yakitori? | 19:48 |
| sinzui | abentley, yep | 19:48 |
| abentley | sinzui: I think that's the explanation, then. | 19:49 |
| sinzui | abentley, can you instrument CI destroying aws and undermining my acceptance test? | 19:49 |
| sinzui | well | 19:49 |
| sinzui | I can teardown now and find some more japanese food | 19:49 |
| abentley | I didn't understand the request. But I can certainly change the control bucket. | 19:50 |
| sinzui | abentley, I was wondering if you wanted to make ci destroy aws and then I would see if the env went missing | 19:50 |
| abentley | sinzui: Okay. | 19:51 |
| abentley | sinzui: Done. | 19:51 |
| sinzui | abentley, that was it | 19:52 |
| abentley | sinzui: Okay, changing control bucket here. | 19:52 |
| sinzui | and I will change mine too | 19:52 |
| abentley | sinzui: 1.16.4 on the local provider no longer ooms, but the deploy goes funny: http://162.213.35.54:8080/job/local-upgrade-deploy/75/console | 19:56 |
| abentley | sinzui: So the upgrade went fine, but the deploy seems to use the 1.16.3 tools instead of the 1.16.4. | 19:57 |
| sinzui | abentley, we could use --show-log or --debug to see more info about what was selected | 19:58 |
| marcoceppi | Who knows the most about the openstack provider? | 19:58 |
| abentley | sinzui: Okay. I can say from the existing log that 1.16.3.1 was selected on that run. | 19:58 |
| natefinch | marcoceppi: I wouldn't say I know a lot about the openstack provider, but maybe I can help? | 20:00 |
| sinzui | abentley, looking at before the destroy step, we can see that 1.16.4.1 was selected, but I don't see any agents upgraded during the wait phase | 20:01 |
| abentley | sinzui: As the agents reach the expected value, they disappear from the list. So when 0 disappears from "<local> 1.16.3.1: 1, 2, mysql/0, wordpress/0", we can infer that it was upgraded. | 20:02 |
| abentley | sinzui: When /var/lib/jenkins/ci-cd-scripts2/wait_for_agent_update.py exits without an error, that indicates that all the agents have upgraded. I can change it to print that out. | 20:04 |
| sinzui | hmm, why are the two upgrade commands different in this log | 20:04 |
| sinzui | abentley, this command assume that 1.16.4 is at the location specified in the tools url: juju upgrade-juju -e local --version 1.16.4.1 | 20:05 |
| thumper | morning | 20:05 |
| sinzui | ^ I think that might assume the bucket was populate from the previous effort | 20:06 |
| abentley | sinzui: They are for different reasons. the first tests upgrades. | 20:06 |
| thumper | mramm: back in the land of stars and stripes? | 20:06 |
| mramm | yes | 20:06 |
| abentley | sinzui: The second is to force the agents to match the client. It was intended for the case where the agent is newer than the client. | 20:06 |
| sinzui | abentley, sure, but --version requires a binary at the tools url | 20:07 |
| abentley | sinzui: sure, but shouldn't 1.16.4 deploy its binary to the tools url? | 20:08 |
| sinzui | abentley, only --upload-tools will put the tool in the location. I think --version just pulls from that location | 20:09 |
| natefinch | morning thumper | 20:09 |
| thumper | o/ natefinch | 20:09 |
| thumper | mramm: our 1:1 has moved from 9am to 11am for me due to summer time here and not there | 20:09 |
| thumper | we can move it earlier if you like | 20:09 |
| mramm | ok | 20:09 |
| mramm | right now I have an interview | 20:10 |
| abentley | sinzui: So how does 1.16.3 get into the tools url? Remember we're talking local provider here. | 20:10 |
| mramm | thumper: but I can move it forward for next week | 20:10 |
| thumper | mramm: sure | 20:10 |
| sinzui | abentley, local-bucket > streams > aws | 20:11 |
| abentley | sinzui: So do we need to specify a testing tools-url for local-provider then? | 20:12 |
| sinzui | abentley, I think so, I have in the past with limited success. | 20:12 |
| * sinzui ponders trying lxc with with aws testing | 20:12 | |
| abentley | sinzui: In the old environment, I have http://10.0.3.1:8040/tools | 20:14 |
| thumper | sinzui: fyi, we are going to want to test the local provider with lxc and kvm shortly | 20:18 |
| sinzui | thumper, I was planning that for 1.18, but given the topic of jamespages's reply to the 1.16.4 announcement, I think we need to quickly increment the minor numbers and release 1.18 today | 20:21 |
| * thumper tilts his head | 20:21 | |
| thumper | sinzui: must have been a personal reply | 20:21 |
| thumper | didn't see it | 20:21 |
| sinzui | thumper, do you have a reply to "Adding new modes to the provisioner and new plugins feels outside the scope of stable updates to me" | 20:21 |
| thumper | what is the rationale there? | 20:21 |
| thumper | I guess that makes sense | 20:22 |
| thumper | are we needing a release? | 20:22 |
| sinzui | I learned that jam had planned an alternate 1.16.4 from myself. I made the release with jam's plan, but james thinks we should be doing a 1.18 with these changes | 20:23 |
| sinzui | abentley, thumper I am tempted to create a stable series and branch from 1.16 and just release stables from it with selective merges | 20:24 |
| abentley | sinzui: Does this shed any light? http://162.213.35.54:8080/job/local-upgrade-deploy/76/console | 20:24 |
| abentley | sinzui: That sounds like it could work, but I think it would be better if the dev team was doing that. Or better yet, landing fixes to stable and merging them into devel. | 20:27 |
| sinzui | abentley, I am sure that strategy would avoid the porting pain we have had in the last 30 days | 20:28 |
| abentley | sinzui: agreed. | 20:28 |
| thumper | sinzui: what is the push to get the plugins out? | 20:28 |
| thumper | is it external need? | 20:29 |
| thumper | if that is the driver, | 20:29 |
| thumper | then +1 on a 1.18 from me | 20:29 |
| thumper | if it is to just get testing | 20:29 |
| thumper | why not 1.17? | 20:29 |
| thumper | we haven't done 1.17 yet have we? | 20:29 |
| sinzui | thumper, no way, 1.17 is really broken. we haven't see hp-cloud work in weeks | 20:30 |
| thumper | really? | 20:30 |
| thumper | WTF? | 20:30 |
| thumper | broken how? | 20:30 |
| sinzui | thumper, we can do a dev release with 2071 from Nov 4. | 20:30 |
| thumper | why is it so broken? | 20:31 |
| thumper | shouldn't that be a "stop the line" type issue? | 20:31 |
| sinzui | we cannot tell. exactly. Charms don't deploy on it for testing | 20:31 |
| * thumper shakes his head | 20:31 | |
| thumper | what about canonistack? | 20:31 |
| thumper | is that working? | 20:31 |
| sinzui | 2071 always works, 2072 always fails. canonistack passes | 20:31 |
| sinzui | abentley, about the log. I am still not surprised. I think 1.16.3 was pulled from aws because --upload-tools was not used. | 20:34 |
| thumper | the only think that leaps to mind with that has to do with installing stuff in cloud-init leaving apt in a weird state | 20:34 |
| thumper | that revision is nothing openstack specific | 20:34 |
| thumper | so we are left to look at the differences in the actual cloud images | 20:34 |
| sinzui | abentley, let me see if I can get control of my local and try with aws testing location | 20:34 |
| thumper | sinzui: got logs? | 20:34 |
| sinzui | sure do | 20:35 |
| sinzui | thumper, https://bugs.launchpad.net/juju-core/+bug/1255242 | 20:35 |
| _mup_ | Bug #1255242: upgrade-juju on HP cloud broken in devel <ci> <hp-cloud> <regression> <upgrade-juju> <juju-core:Triaged> <https://launchpad.net/bugs/1255242> | 20:35 |
| sinzui | thumper, abentley ^ about this bug. I ponder if Hp got too fast for juju. I see similar errors on Hp when I am making juju go as fast as possible. When I wait a few minutes for the services to come up before add-relation, I get a clean deployment | 20:37 |
| abentley | sinzui: I can change the script to always supply --upload-tools for the local provider, if that's what we want. | 20:37 |
| thumper | sinzui: weird | 20:38 |
| sinzui | abentley, I think that might be the case. let me finish my test with with local + aws testing | 20:38 |
| abentley | sinzui: Sure. BTW, here's an example of our automatic downgrade to 1.16.3 to match a 1.16.3 client: http://162.213.35.54:8080/job/aws-upgrade-deploy/73/console | 20:39 |
| sinzui | abentley, use use --upload-tools for the second deploy phase. my aws testing hacks didn't help | 20:47 |
| abentley | sinzui: Can I also use it for the first deploy phase? I use the same script for both deploys. | 20:48 |
| sinzui | abentley, if the first juju is stable and the second is proposed, then it is okay | 20:49 |
| abentley | sinzui: Okay, I will use it for both deploys. | 20:50 |
| abentley | sinzui: Okay, I applied --upload-tools to bootstrap, but for some reason, 1.16.3.1 was selected again. Did you mean I should apply it to upgrade-juju? | 21:09 |
| sinzui | abentley, for the second case these is a disaster. 1.16.4 should only upload tools for itself. | 21:10 |
| sinzui | abentley, the test is to verify propose stable can bootstrap itsel | 21:11 |
| sinzui | f | 21:11 |
| sinzui | abentley, The local upgrade and bootstrap just works for me. | 21:16 |
| sinzui | abentley, http://pastebin.ubuntu.com/6511274/ | 21:16 |
| abentley | sinzui: I am not installing the deb, so that I don't have to worry about version conflicts. | 21:17 |
| sinzui | abentley, but is this a case where we are using the extract juju? Since we didn't install juju, the 1.16.3 tools are all that is available | 21:17 |
| abentley | sinzui: Instead, I am just using the binary directly. | 21:17 |
| sinzui | bingo | 21:18 |
| sinzui | this is tricky | 21:19 |
| abentley | sinzui: So there's a ./usr/lib/juju-1.16.4/bin/jujud in a directory in the workspace. Any way to convince juju to use that? | 21:19 |
| sinzui | GOPATH? | 21:19 |
| abentley | I don't know enough about what GOPATH means. Is it for resources as well as executables ? | 21:21 |
| sinzui | GOPATH=./usr/lib/juju-1.16.4/ indicates where to find bins and srcs | 21:22 |
| * sinzui can try this now in fact | 21:22 | |
| sinzui | abentley, I think GOPATH works are in thunderbirds | 21:27 |
| abentley | in thunderbirds? | 21:27 |
| natefinch | that's the signal! Delta team, go go! | 21:28 |
| natefinch | >.> | 21:28 |
| natefinch | <.< | 21:28 |
| sinzui | abentley, "thunders are go" http://pastebin.ubuntu.com/6511330/ | 21:29 |
| thumper | :) | 21:29 |
| * sinzui likes supermarionation | 21:30 | |
| natefinch | this does not surprise me. | 21:30 |
| * abentley liked Team America: World Police, but hasn't seen much of the original stuff. | 21:31 | |
| sinzui | my son has a youtube subscription to follow Captain Scarlet | 21:32 |
| abentley | sinzui: Did I do it wrong? http://162.213.35.54:8080/job/local-upgrade-deploy/81/console | 21:42 |
| natefinch | thumper: my tests for mongo HA require two thread.Sleep() equivalents (<-time.After(time.Second))... it's because I'm starting and stopping mongo servers, and the code that does it is asynchronous.... so sometimes mongo hasn't finished starting yet. Thoughts? I could spend some time making the mongo start function synchronous.. but it's just a test mechanism, so I'm not sure how much time to put into it. | 21:44 |
| thumper | natefinch: I'd suggest making the start synchronous, it shouldn't be too hard no? | 21:45 |
| thumper | we do this now with the upstart start method | 21:45 |
| thumper | so we go: start, are you started? | 21:45 |
| sinzui | abentley, I explicitly call the juju bin instead of PATH resolution | 21:45 |
| thumper | wait a bit, and ask again | 21:45 |
| natefinch | thumper: yeah, that's what I was thinking of doing. Fair enough. | 21:45 |
| thumper | we have short attempts | 21:45 |
| sinzui | abentley, but I see you put the bin as the first element in PATH | 21:46 |
| abentley | sinzui: Also, I run 'which' to make sure I have the right one. | 21:46 |
| thumper | but I think making it synchronous is the most obvious thing, hide the waiting from the caller, make the tests simple to read and understand | 21:46 |
| thumper | I don't think you ever regret making tests better and more understandable | 21:46 |
| sinzui | abentley, is is possible sudo got root's PATH | 21:46 |
| thumper | within reason | 21:46 |
| abentley | sinzui: I use -E to preserve the environment. | 21:47 |
| sinzui | abentley, Doesn't work for me | 21:50 |
| sinzui | GOPATH=~/Work/juju-core_1.16.4 PATH=~/Work/juju-core_1.16.4/bin:$PATH sudo -E juju --version | 21:50 |
| sinzui | 1.16.3-trusty-amd64 | 21:50 |
| sinzui | abentley, but this works because I removed all of the historic PATH: | 21:51 |
| sinzui | $ GOPATH=~/Work/juju-core_1.16.4 PATH=~/Work/juju-core_1.16.4/bin juju --version | 21:51 |
| sinzui | 1.16.4-trusty-amd64 | 21:51 |
| abentley | sinzui: Weird. | 21:51 |
| sinzui | yeah | 21:52 |
| thumper | -E doesn't pass in PATH | 21:52 |
| thumper | IIRC | 21:52 |
| thumper | I use "sudo $(which juju) --version" | 21:53 |
| thumper | mramm: I'm in the hangout... just hanging out | 22:01 |
| bigjools | hello. I have a dead machine but juju still thinks it's there. How can I remove it? terminate-service/destroy-machine all fail to work because they seem to want to contact the agent on the dead machine. | 22:45 |
| davecheney | bigjools: i don't thnk you can at the moment | 22:59 |
| bigjools | davecheney: so my env is fucked? | 23:00 |
| davecheney | you could try building 1.17-trunk from source | 23:00 |
| bigjools | ok | 23:00 |
| davecheney | bigjools: we don't tell the customres their environment is fucked | 23:00 |
| davecheney | but, yes | 23:00 |
| bigjools | :) | 23:00 |
| bigjools | I think that when you see a "hook failed" message it ought to suggest running "juju resolved" | 23:01 |
| bigjools | someone who will remain nameless decided to use "nova delete" instead | 23:01 |
| davecheney | that's a paddlin' | 23:01 |
| === gary_poster is now known as gary_poster|away | ||
| wallyworld_ | thumper: wrt https://codereview.appspot.com/35800044/, william had some issues. i've responded. i feel like there's value in this work. do we need to discuss? | 23:48 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!