mupBug #1450573 changed: HA and backup recovery tests failed <backup-restore> <blocker> <ci> <ec2-provider> <ensure-availability> <regression> <juju-core:Triaged> <https://launchpad.net/bugs/1450573>00:29
wallyworldthumper: any bribe to get you to look at http://reviews.vapour.ws/r/1556/ so that i can get a 1.24 fix running through CI?00:37
mupBug #1451283 was opened: depployer sometimes fails with a unit status not found error <blocker> <ci> <regression> <juju-core:In Progress by wallyworld> <juju-core 1.24:In Progress by wallyworld> <https://launchpad.net/bugs/1451283>00:59
thumperwallyworld: yeah... on it now01:23
thumperwallyworld: actually had that review open already before lunch01:25
* thumper looks in more detail01:25
thumperwallyworld: review done, a few questions01:35
=== menn0 is now known as Guest93369
=== menn0_ is now known as menn0
wallyworldthumper: thank you, i've pushed changes, review board UI messed up my responses and turned them into issues01:50
menn0wallyworld, thumper: I just discovered another critical problem01:50
menn0wallyworld, thumper: an upgrade from 1.18 to 1.23 blows up01:50
menn0wallyworld, thumper: 1.18 to 1.22 works01:51
wallyworldoh joy01:51
menn0wallyworld, thumper: looks like it's JES related01:51
wallyworldmenn0: there was that JES upgrade issue horatio fixed last week01:51
wallyworlddid you pull the latest code?01:52
menn0wallyworld, thumper: the API server can't start because it can't find the machine record01:52
wallyworldah, ok, different, but could be same root cause01:52
menn0wallyworld: I'll make sure I'm using the same code01:52
wallyworldthe issue last week was that an upgrade step was missed01:52
menn0wallyworld: I'm used to fixing these kinds of issues.01:52
wallyworldso the charm collection did nt have the UUID added01:52
thumpermenn0: what machine record?01:52
menn0thumper: I meant document01:53
wallyworldwell, the upgrade step was there but in the wrong place01:53
menn0wallyworld: this is before the upgrades run01:53
wallyworldah, ok01:53
thumperah... why is it different in 1.23 from1.22?01:53
menn0wallyworld: we have a few hacks in the code to deal with code that needs to function before upgrades have run. this looks like another one of those01:54
menn0thumper: not sure yet. will do some more digging now.01:54
wallyworldwaigani_: do you know where i can find the bundles.yaml file used with the aws-deployer-bundle CI tests?01:54
waigani_there was a bug last week where an upgrade step adding UUIDs to charms docs targeted 1.22 instead of 1.2101:54
menn0wallyworld: the good news is that the new approach to dealing with replicaset init looks like it'll work. (that's how I found this bug)01:54
wallyworldgreat :-)01:55
menn0waigani_: yep. wallyworld mentioned that.01:55
waigani_wallyworld: sorry I haven't worked with bundles.yaml or those CI tests yet01:56
davechen1ythumper: ready when you are01:58
menn0grrr... now 1.18.4 won't bootstrap all of a sudden: start: Job is already running: juju-agent-menno-local02:01
menn0there is nothing juju related already running...02:01
menn0time to kick upstart02:01
waigani_wallyworld: from looking at the script - it's passed into the command (http://bazaar.launchpad.net/~juju-qa/juju-ci-tools/trunk/view/head:/deploy_stack.py:613)02:01
wallyworldwaigani_: yes exactly, so the bundle.yaml file has to be stored somewhere, i was wondering if maybe you had come across where in your recent dealings with QA02:04
waigani_wallyworld: no sorry. I haven't needed to run the deployer test yet so I don't know.  best to ask CI.02:05
wallyworldwill do, ta, was hping to avoid waiting 8hrs :-)02:05
rick_h_wallyworld_: looking at the name 'landscape-scalable' there's this one in the store https://jujucharms.com/u/landscape/landscape-scalable/802:08
rick_h_wallyworld_: I can't confirm it's 100% the match but seems likely.02:08
rick_h_wallyworld_: so you'd want the bundles.yaml.orig file https://api.jujucharms.com/charmstore/v4/~landscape/bundle/landscape-scalable-8/archive/bundles.yaml.orig02:10
* rick_h_ bets it's the orig yaml at least. 02:10
wallyworld_rick_h_: ah, yes, that looks right, awesome, tyvm02:10
wallyworld_thumper: you happy with the revised code?02:13
thumperwallyworld_: on a call, haven't looked02:13
wallyworld_ok, np02:15
wallyworld_rick_h_: ah, turns out that wasnt the right bundle. the one i want has django in it02:27
wallyworld_might be a custom CI one /var/lib/jenkins/repository/bundles.yaml02:28
menn0wallyworld_: that 1.18 to 1.23 upgrade problem is definitely occurring with the current 1.23 branch. Bug 1451297 created.02:31
mupBug #1451297: Upgrade from 1.18 to 1.23 fails: password for machine agent can't be set <regression> <juju-core:In Progress by menno.smits> <juju-core 1.23:In Progress by menno.smits> <juju-core 1.24:In Progress by menno.smits> <https://launchpad.net/bugs/1451297>02:31
mupBug #1451297 was opened: Upgrade from 1.18 to 1.23 fails: password for machine agent can't be set <regression> <juju-core:In Progress by menno.smits> <juju-core 1.23:In Progress by menno.smits> <juju-core 1.24:In Progress by menno.smits> <https://launchpad.net/bugs/1451297>02:57
wallyworldthumper: thank you03:45
menn0wallyworld or thumper: here's the fix for that SetPassword upgrade issue. http://reviews.vapour.ws/r/1558/04:47
* thumper looks04:49
thumpermenn0: done04:50
menn0thumper: thanks04:50
menn0axw: do you have a rough idea of when 1.23 will get unblocked? I have another critical bug fix.04:55
axwmenn0: didn't know it was blocked, so nope, sorry04:55
wallyworldmenn0: thanks for fixing04:55
menn0axw: you fixed the blocker!04:56
axwoh that04:56
menn0axw: bug 144943604:56
mupBug #1449436: Environment variables are not propagated to jujud on vivid <blocker> <juju-core:Fix Committed by axwalk> <juju-core 1.23:Fix Committed by axwalk> <juju-core 1.24:Fix Committed by axwalk> <https://launchpad.net/bugs/1449436>04:56
axwmenn0: I'll see if CI has picked it up04:56
menn0axw: np04:56
axwmenn0 wallyworld: do I mark it as Fix Released if it was in a blessed CI version?04:59
axwor lower the importance04:59
axwI think Fix Released is what CI do, but feels odd to say that05:00
axwmenn0: I've marked Fix Released, so should be unblocked now05:02
menn0wallyworld, axw: I thought marking it was Fix Released was only really the done thing for test only fixes?05:08
axwmenn0: hmm yeah, that would make sense. I'll revert that and lower to High, so it's unblocked.05:09
menn0axw: thanks05:09
wallyworldaxw: reviewed, but with a question. maybe it's nothing to worry about09:57
perrito666wallyworld: seriously mate, I think you should install quassel's server portion in another place :p10:05
perrito666morning all10:05
TheMueperrito666: morning10:07
katcosinzui: re: bug 1450919: mgz's comment: a fix was committed to master, but not 1.24 yet. are we sure that PR is contributing?13:36
mupBug #1450919: many window unit tests failures <blocker> <ci> <regression> <windows> <juju-core:Triaged> <juju-core 1.24:Triaged> <https://launchpad.net/bugs/1450919>13:36
sinzuikatco, I cannot be sure. I just read that. I am sure the test failed for 1.2413:37
perrito666wow, this was actually never even remotely implemented https://bugs.launchpad.net/juju-core/+bug/141517613:39
katcosinzui: unless i'm missing something, logically, it cannot be that PR. perhaps it's even because that PR has not been landed into 1.24...13:39
mupBug #1415176: debug-hooks exit 1 , doesn't mark hook as failed <cts> <debug-hooks> <juju-core:Confirmed> <juju-core 1.24:In Progress by hduran-8> <https://launchpad.net/bugs/1415176>13:39
katcoperrito666: if it's not implemented, it's a feature request, yeah?13:39
perrito666I am not sure, apparently our docs says it exists13:40
katcoperrito666: if you feel it is... tag it with "feature" and sinzui, alexisb, and i can decide whether or not it should be in 1.2413:40
sinzuiperrito666, Then maybe the feature was lost?13:42
perrito666sinzui: m?13:42
sinzuiperrito666, "debug-hooks exit 1" was either not implemented or it was and was lost in a version13:43
perrito666reading at debug hooks it would seem as if it never existed, let me peek a bit more13:45
sinzuiperrito666, since the version in the bug is 1.20, we could try 1.18. if 1.18 doesn't have it, then never implemented by be true.13:46
perrito666ill go read 1.1813:46
* perrito666 's life would be easier if 1.18 was in git13:47
katcoperrito666: lp supports git now :D13:47
perrito666katco: does it have the rest of the repo?13:48
katcoperrito666: i doubt it lol13:48
katcoperrito666: but i thought it was pretty neat that lp supports git now13:48
perrito666so, without even going to 1.1813:54
perrito666 Mon Jul 29 17:55:23 2013 <-- this is the first appearance of the command13:54
perrito666and the only change since then is meaningless13:54
* perrito666 is freezing, its 19C14:05
hazmatsinzui: there is trick to it i found14:09
hazmatsinzui: re debug-hooks exit and leave unperturbed, its still an issue that needs resolving.. but the trick is to exit the main window, so the hook window is the only that remains then exit 1 from the hook window.14:10
perrito666hazmat: tell me more14:11
hazmatperrito666: that's all there is to it.. original bug was filed in 2013 to which i added the work around, for some reason it was marked as a dup of one filed in 2015. just added the details to it.14:16
perrito666hazmat: tx, ill keep looking where is this being handled14:17
hazmatperrito666: i suspect its implicit behavior wrt to the workaround, and tmux exit status mirroring last window used. explicit would need a cli/hook cli command to perform the same (actually probably a shell alias).14:18
katcodooferlad: hey, were you looking into landing this into master? https://bugs.launchpad.net/juju-core/+bug/144280115:22
mupBug #1442801: aws containers are broken in 1.23 <blocker> <ci> <deployer> <ec2-provider> <lxc> <regression> <juju-core:Triaged by dooferlad> <juju-core 1.23:Fix Released by dooferlad> <juju-core 1.24:Fix Released by dimitern> <https://launchpad.net/bugs/1442801>15:22
TheMuekatco: UK has a public holiday today, so dooferlad isn't here15:26
katcoTheMue: ah ok. thx15:26
katcoTheMue: anyone else on your team that can take his work and land it? it's blocking15:26
TheMuekatco: 1.23 and 1.24 are released, so I think he's currently porting it15:27
katcoTheMue: yeah, but while he's away no one can land anything =/15:27
katcoTheMue: how can we get it landed earlier?15:27
TheMuekatco: I've got to take a look at it, don't know exactly how big it is15:28
TheMuekatco: will do it15:32
katcoTheMue: you rock, sir!15:33
katcoTheMue: the throngs of juju core developers applaud you :D15:33
TheMuekatco: quick review of http://reviews.vapour.ws/r/1564/?16:26
katcoTheMue: tal16:26
katcoTheMue: is this just a port of work that dooferlad has already done? (i.e. no significant deviations from that branch)?16:27
TheMuekatco: exactly16:27
katcoTheMue: do you have a link handy to the original work so i can compare against it?16:28
TheMuekatco: the original one has been https://github.com/juju/juju/pull/207116:29
katcoTheMue: ty16:29
TheMuekatco: np16:29
katcoTheMue: Ship It16:31
TheMuekatco: thx16:31
katcoTheMue: with my thanks :)16:31
=== kadams54 is now known as kadams54-away
katcosinzui: can you respond to my latest comment on bug 1450919?16:36
mupBug #1450919: many window unit tests failures <blocker> <ci> <regression> <windows> <juju-core:Triaged> <juju-core 1.24:Triaged> <https://launchpad.net/bugs/1450919>16:36
katcosinzui: do we have a tag for customer-reported bugs?16:42
sinzuikatco, yes, many16:42
katcosinzui: which one should i use?16:43
sinzuikatco, cts is the common tag16:43
sinzuikatco, per the bug, I added "gotesttarfile.py is forcing "PATH" and "Path" into the env. We did this to see passed before Core merged their fixes."16:43
TheMuekatco: the fix is merged16:57
katcoTheMue: awesome, ty so much!16:57
TheMuekatco: always a pleasure ;)16:58
mupBug #1451487 was opened: bootstrap is not downloading the disc image, but machine 0 comes up. <bootstrap> <cts> <local-provider> <lxc> <juju-core:Incomplete> <https://launchpad.net/bugs/1451487>17:08
mupBug #1451488 was opened: When deploying a service, the charm is added, but the new instance is remains pending. <community> <juju-core:Incomplete> <https://launchpad.net/bugs/1451488>17:08
TheMueeod o/17:26
katcoTheMue: tc o/17:26
bdxHow do you guys feel about adding a config-flags param for ceph and ceph-osd?17:32
mupBug #1450912 changed: quickstart is universally broken in 1.24 <blocker> <ci> <quickstart> <regression> <juju-core:Fix Released by wallyworld> <juju-core 1.24:Fix Released by wallyworld> <https://launchpad.net/bugs/1450912>17:38
bdxgnuoy, jamespage: How do you guys feel about adding a config-flags param for ceph and ceph-osd?17:38
=== kadams54-away is now known as kadams54
=== kadams54 is now known as kadams54-away
perrito666bbl, bike pause18:32
sinzuikatco, do you have a moment to review http://reviews.vapour.ws/r/1565/19:16
katcosinzui: tal19:16
katcosinzui: ship it19:17
sinzuithank you katco19:17
katcosinzui: ty!19:17
alexisbso folks, do we have a plan of attack on this critical bug: https://bugs.launchpad.net/juju-core/+bug/1450919 ?20:03
mupBug #1450919: many window unit tests failures <blocker> <ci> <regression> <windows> <juju-core:Triaged> <juju-core 1.24:Triaged> <https://launchpad.net/bugs/1450919>20:03
alexisbif not what do we need to move forward20:04
alexisbkatco, ^^20:04
katcoalexisb: hey20:04
alexisbheya katco :)20:05
katcoalexisb: i was going to discuss in the release meeting. i need to understand better how it's not a dupe of bug 144687120:05
mupBug #1446871: Unit hooks fail on windows if PATH is uppercase <ci> <hooks> <windows> <juju-core:Fix Committed by natefinch> <juju-core 1.24:In Progress by natefinch> <https://launchpad.net/bugs/1446871>20:05
alexisbkatco, ack20:05
katcoalexisb: sinzui responded, but i don't understand how his comments are related20:05
katcoalexisb: regardless, nate will probably be fixing that tomorrow. he was already working in that area20:06
alexisbkatco, ok20:06
sinzuikatco, We hacked the test *before* the PATH fixes were merged to help you get a pass.20:06
sinzuikatco, we can remove the PATH fix from CI so that core tests need to do all the setup20:07
katcosinzui: a pass on what? master?20:08
sinzuikatco, everything20:08
katcosinzui: i need more context... i'm not connecting what you're saying at all20:09
katcosinzui: so here's what i know: this critical bug looks a lot like bug 144687120:09
mupBug #1446871: Unit hooks fail on windows if PATH is uppercase <ci> <hooks> <windows> <juju-core:Fix Committed by natefinch> <juju-core 1.24:In Progress by natefinch> <https://launchpad.net/bugs/1446871>20:09
katcosinzui: bug 1450919 calls out 1.24, but nate's fix isn't in 1.24 yet20:09
mupBug #1450919: many window unit tests failures <blocker> <ci> <regression> <windows> <juju-core:Triaged> <juju-core 1.24:Triaged> <https://launchpad.net/bugs/1450919>20:10
sinzuikatco, CI is globally adding PATH and Path to to all windows tests to help them pass. I add the fix in Jauary, mgz, removed it in April, and then put it back when we find Core was still broken20:10
sinzuikatco, Sorry, I cannot speak of that. I didn't do any analysis20:10
katcosinzui: ok so you contend that 1450919 is not about the casing of paths, but about the paths not being there on windows, period?20:11
sinzuikatco, I only contend that PATH and Path are identical. We guarantee that20:11
katcosinzui:  are you ysubg PATH and Path as placeholders for an actual environmental variable?20:12
sinzuikatco, we export PATH and Path to PowerShell when we run the tests. IN the past, somethings in Juju tests (not Juju) were only looking at PATH.20:13
katcosinzui: ok, i'm back to asserting that this new critical bug is just a dupe of an existing, non-critical bug which is not yet in 1.24, but in master20:13
katcosinzui: is there a way to remove CI's PATH Path patch for just master and see if it passes?20:14
sinzuikatco, I honestly don't care how it is classified. CI will tell us when the tests passes20:14
sinzuikatco, I can make a code change to remove it20:14
katcosinzui: and see if master passes?20:15
sinzuikatco, already failed :)20:15
sinzuiBut I am retesting any way because most of the failures are common flakey tests20:15
sinzuikatco, http://juju-ci.vapour.ws:8080/view/Juju%20Revisions/job/run-unit-tests-win2012-amd64/345/console does look better ")20:16
katcothat's a 404 for me20:16
sinzuikatco, my current grief is quickstart and deployer are broken in joyent. I cannot think of how I can help those tests pass20:17
katcoah doh20:17
katcosinzui: one thing at a time :)20:17
katcosinzui: so if this passes, we know the new critical bug is a dupe?20:18
sinzuikatco, IF it passes. I close all bugs about master :). Actually CI will do that for us I think20:19
katcosinzui: so it didn't pass... log looks like the same error. but it also still looks like a dupe bug to me20:29
sinzuikatco, okay. Make it a dupe.20:33
katcosinzui: do you agree?20:33
sinzuikatco, I don't have an opinion.20:33
sinzuikatco, I have had other pressing matters to deal with today20:33
katcosinzui: ok. i'll wait until the release call and get ian's opinion20:34
sinzuimuwa ha ha. A container was left behind from a previous test. Deployer is fine, now I retest quickstart in the same pristine conditions20:38
mupBug #1442801 changed: aws containers are broken in 1.23 <blocker> <ci> <deployer> <ec2-provider> <lxc> <regression> <juju-core:Fix Released by themue> <juju-core 1.23:Fix Released by dooferlad> <juju-core 1.24:Fix Released by dimitern> <https://launchpad.net/bugs/1442801>20:50
mupBug #1450917 changed: afterHook logic skipped if hook is missing <blocker> <ci> <deployer> <maas-provider> <regression> <juju-core:Fix Released by wallyworld> <juju-core 1.24:Fix Released by wallyworld> <https://launchpad.net/bugs/1450917>20:50
wallyworldsinzui: a couple of times, aws-deployer-bundle fails with an "impossible" error. but there's not artifacts like all-machines.log etc that i can find and i can't really debug without these. i'd also like a copy of the bundle.yaml file used in that test. can things be changed so that if it happens again, the logs etc are available? and can I get a copy of the bundle.yaml file?21:05
* perrito666 thinks wallyworld has a very interestin definition of impossible21:05
sinzuiwallyworld, lp:/uju-ci-tools/repository  or just http://bazaar.launchpad.net/~juju-qa/juju-ci-tools/repository/view/head:/bundles.yaml21:06
sinzuiwallyworld, the deployer and quickstart tests don't use the infrastructure to get get logs from an env :(21:07
* thumper afk for a bit21:40
=== kadams54 is now known as kadams54-away
sinzuiwallyworld, ^22:03
wallyworldaxw: perrito666: standup?23:18
mupBug #1451616 was opened: Automate backups of initial environment <juju-core:Triaged> <https://launchpad.net/bugs/1451616>23:32
perrito666swweeeeeeeeeeeeet, I fixed it23:41
perrito666wallyworld: I have the fix for 1415176 shall I propose that to 1.24? or into master already?23:47

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!