[00:05] davecheney: did you find a simple reproduction test case for the dbus issue? [00:07] working on it now [00:07] will time box it til lunch [00:07] then move on to '632 [00:20] menn0: you're on call reviewer ? http://reviews.vapour.ws/r/3228/ [00:20] ta [00:33] davecheney: looking [00:47] davecheney: I don't completely get the fslock code, but ship it I guess [00:47] menn0: IMO the fslock code is broke [00:47] and that fix just made it worse [00:48] yeah the fix didn't make any sense to me [00:48] look at line 111 [00:48] line 125 is a noop [00:48] davecheney: exactly, that's what didn't make any sense [00:49] i want to revert the change so I can reopen the bug [00:49] func (conn *Conn) Signal(ch chan<- *Signal) [00:49] https://bugs.launchpad.net/juju-core/+bug/1519097 [00:49] Bug #1519097: juju/utils/fslock: data race caused by createAliveFile running twice [00:50] thumper: i screwed around with dbus to try to make a repro but it's going to be a lot of work [00:50] waigani: do you have the panic message you got ? [00:50] i'll raise a bug with the dbus project [00:50] thumper: I've created a fake dbus read/write/closer in an attempt to be a message pump that just spews out signal messags [00:51] davecheney: yep, one sec [00:51] but it won't work unless it can emulate all the dbus connection handshaking [00:51] davecheney: here you go: http://pastebin.ubuntu.com/13488000/ [00:55] waigani: ta [01:04] waigani: https://github.com/godbus/dbus/issues/45 [01:04] i'll look at fixing it this week [01:05] davecheney: perfect. Thank you :) [01:05] insane unit test rig, indeed [01:10] it's unreasonable for other people to have to install all the stuff we do to reproduce our bugs [02:04] thumper: cherylj perhaps this is the problem [02:04] lucky(~/src/github.com/juju/juju) % juju set-env agent-stream=devel [02:04] WARNING key "agent-stream" is not defined in the current environment configuration: possible misspelling [02:12] thumper: cherylj I cannot reproduce https://bugs.launchpad.net/juju-core/+bug/1517632 [02:12] Bug #1517632: juju upgrade-juju after upload-tools fails [02:12] works for me [02:13] davecheney: please comment on the bug as non-reproducable with what you tried, and unassign from you [02:16] thumper: done [02:17] davecheney: yeah, you'll see that error if you haven't set agent-stream before. It'll still pick up the correct stream. [02:20] which it did [02:20] and worked fine [02:20] yeah, at this point, I'll need to see if the bootstack team can recreate. [02:21] clucking bell [02:21] I have the fix for a bug [02:21] the hard bit is testing the freaking thing === natefinch-afk is now known as natefinch [03:02] wallyworld: ping? [03:02] hey [03:03] hey wallyworld, I've got a question for you. The people who ran into that weird 1.23 -> 1.24 upgrade error yesterday are asking what they should do now - do a restore to 1.23, or do a file-level backup to 1.20, before they upgraded to 1.23 [03:03] https://bugs.launchpad.net/juju-core/+bug/1517992 [03:04] Bug #1517992: juju-upgrade to 1.24.7 leaves juju state server unreachable [03:04] if you're interested in taking a look [03:04] I'm not sure what problems they could run in to with the file level backup. [03:04] when you say "do a file level backup to 1.20", what do you mean? [03:05] Like restoring through something like crash plan or something [03:05] outside of juju [03:06] so you mean try and go back to 1.20 [03:06] and then upgrade to 1.24 [03:06] yes [03:06] menno has a script to upgrade off 1.23 [03:06] i'd go that path first [03:07] as it has been tested and used by customers [03:07] do you know where I can find it? [03:07] menn0: where's you 1.23 upgrade script? [03:07] there was an email somewhere [03:08] * menn0 looks [03:08] * menn0 remembers [03:08] it's a juju plugin [03:08] in the standard repo [03:09] https://github.com/juju/plugins/blob/master/juju-unstick-upgrade [03:09] cherylj: that that help? ^^^^^ [03:09] hopefully. [03:10] menn0: they're seeing this error on their state server: [03:10] 2015-11-19 17:28:27 DEBUG juju.apiserver.upgrader upgrader.go:185 desired version is 1.24.7, but current version is 1.23.3 and agent is not a manager node [03:10] would that script help in the case where the state server doesn't think it's a manager node? [03:11] that could be a side effect of the condition [03:11] cherylj: oh hang on, the message is from on the state server? [03:12] menn0: yeah [03:16] * thumper headdesks [03:16] cherylj: weird. in the state server's logs does it look like most of the workers have have shut down? [03:16] I don't like this but submitting a fix with no test [03:17] menn0: the state server will start up, then shut down as it's trying to do an upgrade, but then that fails because of that error [03:18] I mean, it will shut down the workers to try and do an upgrade [03:19] cherylj: hmmm, not sure. [03:19] * menn0 looks at that code [03:20] menn0: http://reviews.vapour.ws/r/3230/ [03:20] cherylj: the plugin might still get the state server through it [03:21] cherylj: it'll change the tools symlink for the agent to the new version and restart the agent [03:21] cherylj: which will get it running the new version [03:23] menn0: okay, I will have them give it a try. Thanks! [03:24] thumper: ship it [03:24] menn0: ta [03:25] cherylj: they'll want to follow the instructions for installing juju-plugins. once they have the "juju unstick-upgrade" command will be available. [03:26] thanks, menn0. Hopefully this will work for them! [03:27] just one more question. This was a failed upgrade off of 1.23 (to 1.24.7). Will it still be okay to run the script? [03:28] menn0: ^^ [03:40] cherylj: yes [03:40] cherylj: it addresses specific problems with getting off 1.23 to something newer 1.24 and up [03:40] menn0: cool, thanks. Just wanted to double check :) [03:53] wallyworld: got a quick minute? [03:54] sure [03:54] 1:1 hangout [04:06] [04:06] p [04:07] * thumper is done [04:18] Bug #1438951 changed: destroy-enviroment --force destroy all aws instances

[04:18] Bug #1472009 changed: manual provisioning with juju requires systemd-services

[04:24] Bug #1438951 opened: destroy-enviroment --force destroy all aws instances

[04:24] Bug #1472009 opened: manual provisioning with juju requires systemd-services

[04:27] Bug #1438951 changed: destroy-enviroment --force destroy all aws instances

[04:27] Bug #1472009 changed: manual provisioning with juju requires systemd-services

[04:33] Bug #1438951 opened: destroy-enviroment --force destroy all aws instances

[04:33] Bug #1472009 opened: manual provisioning with juju requires systemd-services

[04:36] Bug #1438951 changed: destroy-enviroment --force destroy all aws instances

[04:36] Bug #1472009 changed: manual provisioning with juju requires systemd-services

[05:04] menn0: http://reviews.vapour.ws/r/3232/ [07:00] mattyw: is there a hangout? [07:00] wallyworld, ashipika grrr, should be one in the meeting thing? [07:01] not that i can see [07:01] I remember giving it a "witty" name [07:01] mattyw: how about https://plus.google.com/hangouts/_/canonical.com/cross-model-relations [07:01] ashipika, heading there now, I can't even add a hangout to the meeting it seems [07:50] urulama: i have run into a problem. the charm store PutArchive function, called via the charmrepo UploadArchive(), complains when a charm is uploaded without a Series in the URL. But with series in metadata, it's perfectly ok to do this. [07:51] so I have a bunch of failing juju tests [07:51] wallyworld: so, yes, if series is in metadata, then you don't need it in the url. if it's not, then it's an old charm and series must be in the url as before [07:52] ok, i need then to look at the test charms used by juju, thanks [07:52] np [07:53] that behaviour is required to keep things backward compatible (and also makes sense, series must be provided somewhere in the uplaod) [08:16] dimitern, ping [08:16] frobware, hey [08:17] dimitern, re: 1519527 - are you on MAAS rc1 or rc2? [08:17] frobware, on rc1 still [08:18] frobware, but it was upgraded one too many times I think it's time for a fresh install [08:19] frobware, I was having issues with the old desktop pc I'm using - 1G RAM only, i386 P4 and occasionally overheating and throttling down - all that slows down machines deployment a lot [08:23] dimitern, I was asking to see if the issue came in rc2; I still have rc1 so will try there first. [08:23] dimitern, good morning btw. ;) [08:25] frobware, good morning :) [08:25] frobware, I'm done with service bindings in state - proposed it late last night [08:26] frobware, I'm doing a quick live test first and then will update the PR's description and ask for reviews [08:27] dimitern, sounds great [08:28] frobware, wait until you see the diff :) [08:29] dimitern, could do with brainstorming session later regarding rendering /e/n/i for bonds. [08:29] frobware, sure [08:39] dooferlad, ping [08:52] fwereade, morning! [08:52] dimitern, o/ [08:52] morning dimitern [08:52] jam, morning :) [08:53] fwereade, I have a review for you, if you can have a look: http://reviews.vapour.ws/r/3223/ [08:53] fwereade, it's a bit large, but mostly due to heavy testing :) [09:27] dooferlad: ping [09:27] voidspace: hi [09:28] dooferlad: did you see my messages about SetNodeNetworkLink about 5:20pm last night? [09:28] voidspace: no [09:28] dooferlad: SetNodeNetworkLink is new, right? [09:28] dooferlad: it would be much more convenient if it took a SystemID rather than a Node [09:29] dooferlad: as getting a node is inconvenient, and constructing one just to pass a system id (all the function actually needs) is a minor burden :-) [09:29] voidspace: OK, can you write what you want in an email / task? I am looking at another problem + have meetings [09:29] dooferlad: ok [09:29] dooferlad: np, see you at standup [09:40] dimitern, reviewed, ping if questions [09:41] fwereade, cheers! [09:55] urulama: there's still a problem. the charmstore.v5 func (h *ReqHandler) serveArchive(id *charm.URL...) method expects id to have series set. but it doesn't. the content which is uploaded has series in metadata, but that handler checks id.Series even before looking at the contents of the request [09:56] so it seems you can't upload a charm with url "~user/wordpress" even if the metadata.yaml has series [09:57] that's using charmrepo.v2 csclient UploadCharmWithRevision() [09:58] did you set the export? [09:58] this is for tests [09:58] hm [09:58] urulama: looking at the code, i can't see how it could possibly work [09:59] which code? [09:59] the code path used does not allow an id with out a series [09:59] charmrepo.v2 csclient UploadCharmWithRevision() [09:59] github.com/juju/juju/testcharms/charm.go:64 [09:59] you cant upload a charm with revision! [09:59] sorry [09:59] gopkg.in/juju/charmrepo.v2-unstable/csclient/csclient.go:191 [10:00] no, there is a revison [10:00] i just left off the params [10:00] the code is [10:00] err = client.UploadCharmWithRevision(id, ch, promulgatedRevision) [10:00] yes, promulgated revision :) [10:00] where promulgatedRevision is 3 or whatever [10:00] id is "wordpress-3" [10:00] ch is a charm with series in metadata [10:01] that the case to support the ingestion, where we have to use that [10:01] i follow the code and i can see how it would fail, the code doesn't allow id to have an emoty series [10:01] uploads should never set revisions in normal cases [10:01] so all this is in existing test code which is failing now tht series is not in the url but in metadata [10:02] voidspace, frobware, jam, fwereade, standup? [10:02] test failing in juju or where? [10:02] test failing in juju [10:02] it is uploading charms to then test bundle deployment [10:02] we used to force a url to have the trusty series [10:03] but now the test allows the url series to be "" because the series is in charm metadata [10:03] if i use the url "trusty/wordpress" then subsequent repo.Resolve("wordpress") calls fail [10:04] but i should just be able to upload to "wordpress" and it should pick the series from the metadata [10:04] dimitern: omw [10:05] but the code errors immediately because it inspects the id and sees series = "" [10:05] wallyworld: so, uploads are of form cs:wordpress-3 then [10:05] rogpeppe: ^ seems core expects the case of name-revision uploads, the one we don't allow anymore [10:05] the tests were written a while back, not by me so i am not sure of their heritage [10:06] wallyworld, urulama: looking [10:06] ty [10:06] testcharms.UploadCharm(c, s.client, "trusty/wordpress-0", "wordpress") is how it used to be [10:07] testcharms.UploadCharm(c, s.client, "wordpress-0", "wordpress") is how i changed it [10:07] yes, this is not allowed, but might be a bug on our side [10:07] because the upload charm was modified to upload a charm with series in metadata [10:07] wallyworld: which commit of charmstore are you using? [10:07] the latest [10:07] wallyworld: 2ce00261132ea5e70753c67d4c39e3f8d6e5f6f0 ? [10:08] a3afbf1 [10:08] from juju core deps.tsv [10:08] wallyworld: that's not the latest [10:09] i also tried pulling tip as well [10:09] wallyworld: but i don't think it matters in this case [10:09] wallyworld: you shouldn't be uploading the charm with a revision number [10:09] ok, we can change core [10:09] wallyworld: the revision number should be chosen by the charmstore itselg [10:09] but the rev number doesn't affect the series issue [10:09] wallyworld: it does [10:09] the charmstore http handler for archive post seems to bail [10:10] when id has series = "" [10:10] wallyworld: because we only check that there's a series if the request is a PUT [10:10] wallyworld: not a POST [10:10] wallyworld: PUT is done when there's a specified revision id [10:10] wallyworld: POST otherwise [10:11] ok, so i change juju core's tests to leave off revision [10:11] let me do that [10:11] already mentioned this, so i think the tests are just wrong. the ones with revision should fail and the ones to be used are without revision [10:11] wallyworld: the reason we still check for series in PUT is that PUT is really there just for the legacy ingestion [10:12] rogpeppe: ok, so i just made that revision param -1 [10:12] we'll see if test works [10:13] wallyworld: presumably it's a new test, otherwise it would have failed anyway, right? [10:13] s/failed anyway/failed beforehand/ [10:14] rogpeppe: i annotaed and tests were written in sept by francesco [10:14] rogpeppe: but using -1 as rev still didn't work [10:14] wallyworld: hmm, but we didn't have multi-series charms back then [10:14] wallyworld: which test is failing? [10:14] no but the tests used to force a series [10:15] ie UploadCharm("trusty/wordpress"...) [10:15] i'm removing the series [10:15] frankban: hey, seems that some tests for bundle are failing in juju now that we have multiseries ... can you check with wallyworld, please [10:15] wallyworld: please tell me which test is failing so i can have a look at what's being tested [10:15] one sec [10:15] github.com/juju/juju/cmd/juju/commands/bundle_test.go:675 [10:16] i'm modifying it because if i let it upload to "trusty/wordpress") then a call tp repo.Resolve("wordpress") fails [10:16] and i should no longer need to specify the series in the upload url anuway [10:16] rogpeppe: i haveto go get my wife, bbiab [10:17] wallyworld: k [10:18] rogpeppe: so a real issue here is that in the core code to process a charm url at deploy time, we were adding in the "default-series" env config propety as the series if the url had an empty series [10:18] this should not be done [10:19] hence we now call repo.Resolve("wordpress") and not Resolve("trusty/wordpress") [10:19] wallyworld: yup, there are definitely a bunch of changes need to be made in core [10:19] well, hm it should be done for legacy charms, right [10:19] unless the user does cs:trusty/wordpress of course [10:19] but this change in part as made these bundle tests fail [10:20] wallyworld: so that test passes for me [10:20] yes because you don't have the above mods [10:20] comment out the code in resolveCharmStoreEntityURL() [10:21] which adds the default-series if no url serie sis set [10:21] urulama: for legacy charms, surely we can still use cs:legacycharm directly without a series [10:21] yes, indeed, it'll use default-series [10:22] rogpeppe: so the issue is: test uploads to "trusty/wordpress", then the bundle code tries to repo.Resolve("wordpress") and boom [10:22] it says no charm found [10:22] wallyworld: hmm, "wordpress" *should* resolve correctly to "trusty/wordpress-0" [10:22] right [10:22] wallyworld: investigating [10:23] rogpeppe: ok, ty, bbiab and i will ping you [10:23] rogpeppe: urulama: btw, i have juju fully functional with new charn store , just getting the tests to pass [10:23] cool! [10:23] wallyworld: great! [10:23] all mltseries stuff works, incl overrides [10:23] ok, really bbiab now [10:23] see you later [10:24] wallyworld: ah! [10:24] wallyworld: i see the problem! [10:24] rogpeppe: what is it? [10:24] urulama: it's that we don't allow wordpress-1 to resolve to trusty/wordpress-1 because that's a silly way to specify a charm [10:25] urulama: because wordpress-1 looks like it's specifying a particular revision of a charm, but that makes no sense when it might resolve to trusty or precise or ... [10:25] right, name-revision is not allowed [10:26] urulama: yeah, and that's what the test is doing [10:26] as written 20min ago :) "seems core expects the case of name-revision uploads, the one we don't allow anymore" [10:26] urulama: trivially fixed (in that test case anyway) [10:26] ah, uploads ... i've meant resolution [10:26] cool! [10:27] urulama: deleting two characters fixes it [10:32] frobware, https://github.com/lxc/lxd says LXD needs 1.3 and there's a PPA I guess you could try on trusty [10:38] frobware: so a fix for the intermittently failing payload/persistence bug has landed on master but isn't yet on maas-spaces [10:38] frobware: could you rebase again please :-) [10:40] voidspace, in progress [10:45] voidspace, dimitern, dooferlad: rebased maas-spaces to de99d4c3da857e478c60a57c806b0d8645078aba [10:46] frobware, great, I'll rebase mine and hopefully drop a commit or two fixing already resolved issues on master [11:18] voidspace, frobware, dooferlad, nice! so in the rebased maas-spaces, I only see the provisioner test failing now [11:20] dimitern: me too [11:21] conflicts when I merge with my branch though :-/ [11:21] ah well, this is why we rebase [11:22] are you actually rebasing? [11:23] rather than merging in trunk? [11:23] so, the previously tested revs of your feature branch are all lost to the aether? [11:23] mgz: no [11:23] mgz: we're rebasing our feature branch [11:23] mgz: and will then merge back onto trunk [11:24] mgz: oh, hmmm... maybe [11:24] mgz: but then the merge back onto trunk will itself be tested [11:24] I mean, it's a choice, old feature branch results are of limited worth if they're red [11:25] but it does screw with the "look it passed, it's ready to merge" process [11:25] mgz: I think it's a worthwhile trade-off [11:25] mgz: it still needs to pass to merge [11:26] mgz, we intend, I think, to stop rebasing and use merging as we're preparing to get it blessed and merge it in master [11:26] that sounds perfectly sane. [11:29] ...don't you need someone with write access to the repo to push --force the feature branch after rebase? [11:29] mgz, btw I saw the mail about the voting status of the run-unit-tests-race job [11:29] mgz, yes, that's what I'm doing. [11:29] mgz, I did file a but a while ago that it won't ever pass if the timeout is not increased - had that happened? [11:29] okay, as long as that works. [11:30] dimitern: the stuff that was timing out has been skipped for now and bugs filed, so dave has some of that covered [11:31] mgz, nevertheless, it still won't pass on a comparable machine with the same timeout as the race detector always slows things down [11:32] mgz, I'll be happy to be proven wrong though, just sayin.. ;) [11:32] yeah, I know, I think some of the work will just need to be fixing some stuff like that in tests. [11:34] rogpeppe: saw backscroll, changing test to testcharms.UploadCharm(c, s.client, "trusty/wordpress", "wordpress") was not all that was needed. charmrepo csclient UploadCharmWithRevision() requires the charm id to have a revision so i had to set that to < -1 [11:34] dimitern, I wonder why we would stop rebasing? We can continue to do that, leaving a final merge into master [11:34] > -1 [11:35] that gets further along till next error, thanks for help, i think i can fix the rest [11:36] wallyworld: you can still do UploadCharm(c, s.client, "trusty/wordpress-1", "wordpress") [11:36] frobware: mostly just adding rebased revs after a bless should really have a ci retest before a merge is proposed [11:36] wallyworld: (as the test did before) [11:36] wallyworld: that worked for me [11:37] rogpeppe: ah, oh, what which string do i remove the -3 from [11:37] * rogpeppe goes back to check [11:38] ah i think i know [11:38] dimitern: thanks for responding on the 1.25.1 lxc address bug, I'll chase that up later today. didn't see anything relevent in the changes to .1 apart from the maas 1.9 pokings from you guys [11:38] rogpeppe: in the bundle [11:38] frobware, AIUI because the CI jobs are configured to test specific revisions and changing them might not trigger a CI run << mgz can clarify [11:38] mysql: [11:38] charm: mysql-1 [11:38] wallyworld: you need to remove the -1 from "mysql-1" [11:38] great, wil try that, ty [11:39] dimitern, mgz: ok, gotcha. but presumably as and when the time comes we can force a retest [11:39] wallyworld: because mysql-1 is ambiguous with respect to non-multi-series charms [11:39] wallyworld: it's always been dodgy, but now we explicitly prohibit it [11:39] rogpeppe: refresh me, what can it be confused with? [11:39] frobware: yeah, the rebase process is just a bit more error prone, and means past tests are non-repeatable. new testing is always fine. [11:39] wallyworld: so if you're specifying a revision number, you want an exact version of a charm [11:40] wallyworld: but if you specify a revision number but no series, you're saying "i want this revision of any one of a number of possible series" [11:40] wallyworld: but there's no link between revision numbers of different series [11:40] wallyworld: so it doesn't really make sense to specify a charm like that [11:40] ah right yes [11:41] rogpeppe: but i now i still get cannot resolve URL \"cs:mysql\": charm or bundle not found") [11:41] wallyworld: and in the new multi-series world, wordpress-1 *is* unambiguous [11:41] because the upload was done for trusty/mysql not mysql. i think i need to add some fake charm metadata to the test [11:41] i had that before but got rid of it [11:41] wallyworld: this passes for me: http://paste.ubuntu.com/13501743/ [11:42] the test now just uploads an empty archive [11:42] wallyworld: "mysql" should resolve to "trusty/mysql-1" [11:42] rogpeppe: yes but your code does would explcitly force juju core to add "trusty" to the requested url [11:42] wallyworld: no it wouldn't [11:43] really? i did in master [11:43] it [11:43] wallyworld: that only applies when you specify a revision number too [11:43] wallyworld: and when the charm is not multi-series [11:43]