/srv/irclogs.ubuntu.com/2015/03/31/#juju-dev.txt

wallyworld_axw_: question, we volumes are listed, i would expect that we'd want to see the unit and storage name to which the volume may be attached, so that we can see this 100GB volume here serves the data-dir storage for mysql/0, right?00:00
wallyworld_at the moment, the PR just shows storage tag00:00
redelmannhi, im having some strange problem after " juju upgrade-juju --version 1.23-beta2"00:22
redelmannall units are loging "unit-mongodb-production-0[25074]: 2015-03-31 00:22:51 ERROR juju.worker runner.go:219 exited "uniter": failed to initialize uniter for "unit-mongodb-production-0": cannot read "/var/lib/juju/agents/unit-mongodb-production-0/state/uniter": invalid operation state: unexpected hook info with Kind Continue"00:23
redelmannok, now stop working :P00:34
axw_wallyworld_: volumes are machine-level, hence we see machine attachment. you could argue that since we know the assigned storage instance, we could also show the unit-storage attachments... but we have "juju storage show" if people really want to see that00:49
wallyworld_axw_: i think that distinction will be lost on the user00:49
wallyworld_storage will also show fs etc00:50
wallyworld_if i list volumes, i'd like to see where they're attached (ie what unit) if i were a user00:51
wallyworld_s/attached/in use00:51
axw_wallyworld_: I'm not opposed to adding an optional UNIT column00:51
wallyworld_ok, let's do it00:52
wallyworld_we could if required (which i'm not sure we will be) add a CLI flag to exclude/include volumes attached to units00:52
wallyworld_redelmann: i'm not sure what's happened with your upgrade, can you raise a bug and attach as many logs as possible from the affected machine00:56
redelmannwallyworld_: ok01:12
wallyworld_redelmann: the error is an internal one that indicates that something got out of sync, but we'd need more info01:13
redelmannwallyworld_: im collecting log from machine-001:14
wallyworld_even if you zip the /var/log/juju dir01:14
wallyworld_but we'd need logs from the machine running the affected unit01:15
redelmannwallyworld_, where i can send logs?02:00
wallyworld_redelmann: attach them to the bug02:00
jw4redelmann: that looks like an upgrade issue where the upgrade steps for 1.23 didn't run.  Let me know when you've created the bug and attached the logs02:24
jw4(one of the upgrade steps)02:25
redelmannjw4: ok, first im trying to remove "some way" upgrade from mongo in machine0, need this running for today :P02:26
jw4thanks redelmann02:26
redelmannjw4, i keep a copy of log for bug02:26
jw4gracias02:26
redelmannjw4: thank to you02:26
axw_wallyworld_ thumper: pyfmt - https://plus.google.com/+EliBenderskyGplus/posts/SzKrvzvUuyc02:27
* thumper looks02:28
axw_I can't see it being as effective as gofmt, since whitespace matters in Python02:28
* thumper guesses yet another python fromatter02:29
axw_indeed :)02:29
thumperyeah, first paragraph02:29
wallyworld_can't have too many02:29
thumperaxw_: interesting02:30
thumperI want a js version too02:30
thumperresharper does a good job with visual studio02:30
thumperand C#02:30
redelmannaxw_, wallyworld_ : https://bugs.launchpad.net/juju-core/+bug/143848902:44
mupBug #1438489: juju stop responding after juju-upgrade <juju-core:New> <https://launchpad.net/bugs/1438489>02:44
redelmannsorry axw_: that no was for you02:45
redelmannjw4: https://bugs.launchpad.net/juju-core/+bug/143848902:45
jw4thanks redelmann :)02:45
axw_np02:45
redelmanntellm if you need more details, and how to get it.02:45
redelmannnow im going to keep trying to remove upgrade from ¿queue? mongo02:46
jw4redelmann: I don't see that error in that log02:47
jw4redelmann: do you have other logs?02:47
redelmannjw4, that error is in "juju debug-log"02:47
jw4redelmann: I see02:47
redelmanncant run debug-log again, it stuck02:47
jw4seems like that text should be in one of the other log files under ~/.juju/02:48
redelmannthat's why im looking in machine0 logs02:48
jw4that error would have come from one of the units; what other machine logs do you have under ~/.juju ?02:49
redelmannjw4, i cant see any log in ~/.juju/02:49
redelmannjw4, all unit say the same02:49
redelmannjw4, actually im deploying to aws02:50
jw4redelmann: ya, I see that environment02:50
redelmannjw4, juju keep log in ~/.juju if deploying to aws?02:50
jw4redelmann: I think it'd be on the machine that's having trouble upgrading02:51
jw4redelmann: how many machines do you have deployed?02:58
redelmannjw4, seven02:59
jw4redelmann: which machine is unit-pgsql-medium-0 on?02:59
jw4redelmann: also, machine-0 should have all-machines.log in the /var/log/juju folder03:00
redelmannjw4, you are right03:01
redelmanngive me a second03:01
jw4redelmann: certainly, thanks!03:02
redelmannim scping it03:02
mupBug #1438489 was opened: juju stop responding after juju-upgrade <juju-core:New> <https://launchpad.net/bugs/1438489>03:02
redelmannjw4, im uploading complete file, this happend today.03:04
jw4redelmann: thanks; I'll review that.  I'm going to be out for 30 minutes or so, but I'll brb03:04
redelmannjw4, ok, i attached it to https://bugs.launchpad.net/juju-core/+bug/143848903:06
mupBug #1438489: juju stop responding after juju-upgrade <juju-core:New> <https://launchpad.net/bugs/1438489>03:06
jw4thanks03:06
davecheneycherylj: do you need some reviews done ?03:10
davecheneyOH MY GOD03:16
davecheneywhy does juju depend on ajstarks/svgo !?!?03:16
davecheneyhow is that possibly necessary for the core operation of juju ?03:16
wallyworld_axw_: only if you feel like it/have time, there's a juju status review which is the first 3 or so branches which would be good to get looked at http://reviews.vapour.ws/r/1334, otherwise i'll bug william when he's net online03:23
axw_wallyworld_: just started looking, it's quite involved. I'd like to get some of my stuff proposed, hope to take a look later on03:24
wallyworld_sure np03:24
wallyworld_i an talk you through anything if needed03:24
wallyworld_i'll have another proposed straight after, and then a 3rd hopefully later today03:25
redelmannjw4, ok, i "fix" it03:27
redelmannjw4, http://paste.ubuntu.com/10710367/03:30
redelmannjw4, after that it upgrade fine03:30
redelmannso, it keep saying cannot read "/var/lib/juju/agents/unit-xxx-0/state/uniter": invalid operation state: unexpected hook info with Kind Continue04:55
jw4redelmann: yeah04:55
redelmannjw4, but now i can juju status ;)04:55
jw4the basic issue is that in 1.23 the uniter does not expect there to be hook information in the uniter state except when it's actually running a hook04:56
redelmannjw4, any workaround? like deleting /var/lib/juju/agents/unit-xxx-0/state/uniter?04:56
jw4redelmann: the upgrade steps were supposed to remove the unnecessary hook info from the uniter state file, but apparently it didn't work04:56
jw4redelmann: if you can find the uniter state file (yaml) and just delete the hook and then restart the uniter, it *might* fix it :)04:57
redelmannjw4, i will try04:58
jw4redelmann: it should just be called uniter I believe05:00
redelmannmh...05:01
redelmannfilter.go:137 cannot retrieve meter status for unit ganglia/0: not found05:01
redelmannmaybe writing a working uniter05:02
jw4redelmann: I'm not sure about that one (possibly upgrade related too, but I'm not familiar with that part)05:02
redelmannjw4, thats happend after deleting uniter file05:02
jw4redelmann: oh. yeah.  I wouldn't delete the uniter file05:02
redelmannjw4, this one: /var/lib/juju/agents/unit-xxx-0/state/uniter05:02
redelmannoh05:03
jw4redelmann: inside that file there should be a line (in yaml format) signifying the hook05:03
jw4if you just update the yaml so there is no hook key/value05:03
jw4that *might* work05:03
jw4redelmann: unfortunately it's past my bedtime, and my wife has been waiting for me to come to bed for 45 minutes now :(05:06
jw4redelmann: :) I'll check in again in the morning05:06
redelmannjw4, thank you05:06
redelmannjw4, have a good night05:06
jw4thanks redelmann you have a great morning/afternoon/night day :)05:07
mattywmorning everyone06:23
tasdomasdimitern, thanks for going through my reviews, much appreciated07:44
dimiterntasdomas, no worries07:45
dimiterntasdomas, aren't you ready to graduate btw ? :)07:45
tasdomasdimitern, well, that's not really up to me to decide ;-]07:46
dimiterntasdomas, I think you are, but I'll have a chat with cmars as well07:46
tasdomasdimitern, but what I find really holding me back in some reviews is lack of knowledge of certain parts of juju07:47
dimiterntasdomas, that will come in time, but when unsure about some part of the code, you should ask questions :)07:48
voidspacemorning all08:00
dimiternmorning voidspace08:00
voidspaceo/08:00
TheMuemorning o/08:00
TheMuedimitern: the described network problem isn't visible in the clients log. sadly I cannot grab the log of the VM08:07
TheMuedimitern: here you can see that downloads from the maas controller fail08:07
TheMuedimitern: also connections to 169.254.169.254 timeout08:09
voidspaceTheMue: o/08:09
TheMuedimitern: so that's why I think the network setup on the node fails after the changes08:09
TheMuevoidspace: o/08:09
dimiternTheMue, hmm - can you ssh into the node after it failed and check /var/log/cloud-init-output.log?08:12
TheMuedimitern: with the written release we've added the gathering of a primary interface out of the lshw data which is used in newCloudinitConfig()08:12
TheMuedimitern: I'll see if I can08:13
dimiternTheMue, yes, in order to determine which interface to bind into the juju-br0 bridge at cloud-init initial boot08:13
TheMuedimitern: yep, this has been with 1.20.1208:14
TheMuedimitern: only checked back where it has been changed08:14
dimiternTheMue, yeah, I remember I did this to fix another bug about not using the correct primary interface for the bridge08:15
TheMueok, while client is still waiting the new node at least gave up08:16
dimiternTheMue, ok, I think this can be faked - if getting the ifaceInfo and the primary interface fails, try using "eth0" as primary interface for newCloudinitConfig()08:17
TheMuenow I only have the problem to log in08:17
dimiternTheMue, maybe the ssh keys where not set?08:17
TheMuedimitern: same setting as yesterday08:17
TheMuedimitern: but don't know how far they are distributed to the node08:18
dimiternTheMue, no, I mean if cloud-init failed to complete for some reason, the ssh keys might not be set on the node08:18
dimiternTheMue, cloud-init sets the keys08:18
TheMuedimitern: yep, that could be the reason08:19
TheMuedimitern: my node dislikes me because I corrupted its data :,(08:19
TheMuedimitern: :)08:20
TheMuedimitern: will do the eth0 approach, should be quickly done08:20
dimitern TheMue, it's not the node - that's a case of juju not generating the userdata correctly so the node cannot boot properly :)08:21
TheMuedimitern: damn security, thought we're building OPEN source *rofl*08:21
dimiternTheMue, try that workaround I suggested - destroy and cleanup to start afresh, then use "eth0" as primaryNIC if lshw cannot be parsed08:21
* TheMue btw loves his MAAS environment08:26
axw_wallyworld_: published some comments on your PR08:28
=== axw_ is now known as axw
TheMuedimitern: no real change, the missing interface information leads to >> 2015-03-31 08:29:49 DEBUG juju.provider.maas environ.go:807 node "/MAAS/api/1.0/nodes/node-4a6120e6-d6ea-11e4-a9b5-000c2975eaae/" network information: []network.Info(nil)08:37
TheMuedimitern: already a bit earlier >> 2015-03-31 08:29:49 DEBUG juju.provider.maas environ.go:760 node "/MAAS/api/1.0/nodes/node-4a6120e6-d6ea-11e4-a9b5-000c2975eaae/" has network interfaces map[]08:38
dimiternTheMue, that could be fine, all it matters is the generated userdata for cloud-init and the node booting ok08:39
dimiternTheMue, did it finish?08:39
TheMuedimitern: looks like before, no action anymore on the node while the client is still "deploying" (repeating this log entry until timeout)08:40
dimiternTheMue, did you use --debug with bootstrap?08:40
dimiternTheMue, it will help if you can paste the output :)08:41
TheMuedimitern: yes, that's where those two lines above are coming from08:41
TheMuedimitern: will create a paste when the timeout is done so that the whole debug log is in08:42
dimiternTheMue, ok08:42
TheMuedimitern: the one I've mailed to you is also debug, see the top line how I'm deploying08:43
TheMuedimitern: fresh-ducks.local is my corrupted node ;)08:43
dimiternTheMue, hmm.. well it seems to me some more debug logging is needed around newCloudinitUserdata to show what's generated08:45
TheMuedimitern: will log it08:47
* TheMue just receive a warning by the national weather service, winds will get really strong today :/ 08:48
voidspaceTheMue: we've had strong winds all night, continuing now08:56
TheMuevoidspace: it will reach its top here between 2 and 6pm, and our neighbor has still two very high trees on her ground which always sway extreme. during the last storm one felt, thankfully not into our direction08:57
mattywdimitern, ping?08:59
dimiternmattyw, hey08:59
dimiternvoidspace, standup?09:01
voidspacedimitern: omw09:01
mupBug #1438590 was opened: juju version tries to make directories and files in my home <landscape> <juju-core:New> <https://launchpad.net/bugs/1438590>09:27
dimiterndooferlad, so just calling c.SpaceCommandBase.SetFlags(f) in c.SetFlags() works09:44
TheMuedimitern: String method sadly had same troubles with references as values in map[string]interface{}, so had to use the renderer way10:24
TheMuedimitern: here is the according output http://paste.ubuntu.com/10711305/10:24
dimiternTheMue, ok, this looks fine to me10:26
dimiternTheMue, does it still fail to boot?10:26
TheMuedimitern: yep, still the same failing behavior10:26
TheMuedimitern: while when using a not corrupted node it works10:27
dimiternTheMue, can you try ssh into it?10:27
dimiternTheMue, while the bootstrap goes on? it should be possible after retrying for a while (might not be possible immediately)10:27
TheMuedimitern: yes, currently rejected10:28
TheMuedimitern: still Permission denied (publickey).10:30
dimiternTheMue, try: ssh -i ~/.juju/ssh/id_rsa ubuntu@IP10:31
TheMuedimitern: still the same10:32
TheMuedimitern: at least it's pingable10:34
dimiternTheMue, can you see the console of the VM ?10:34
TheMuedimitern: if the ubuntu user would have a standard password I could login directly on the console10:34
dimiternTheMue, nope, that won't work unfortunately10:35
TheMuedimitern: I know, sadly10:36
dimiternTheMue, maybe the VM is still booting - do you see any messages on the console?10:36
TheMuedimitern: only the login screen10:37
dimiternTheMue, try with your ssh key configured in maas?10:38
TheMuedimitern: the last interesting part have been messages that downloads from controller and this 169... failed10:38
TheMuedimitern: that's what I'm doing, using that key10:38
TheMuedimitern: I'll bootstrap to a valid node and see what the cloudinit looks there10:44
dimiternTheMue, did you try both the juju_id_rsa key *any* your key?10:45
dimiterns/*any*/*and*/10:46
TheMuedimitern: my key10:46
TheMuedimitern: http://paste.ubuntu.com/10711389/10:47
dimiternTheMue, well, that's why I wrote this earlier: <dimitern> TheMue, try: ssh -i ~/.juju/ssh/id_rsa ubuntu@IP10:47
dimiternTheMue, that's the juju ssh key10:47
TheMuedimitern: eh, sorry, sure, took it too10:48
dimiternTheMue, so both you couldn't ssh with either key10:48
dimiterns/both/both/10:49
dimiternaargh10:49
dimiterns/both//\10:49
TheMuedimitern: hehe, yes10:49
TheMuedimitern: and you're right, the cloudinit looks similar, only the host names10:49
TheMuedimitern: I send you a screenshot of the console. does this help to get in? sadly I cannot cut'n'paste, but enter the key by hand11:01
TheMuewife calls for lunch, bbiab11:03
voidspaceCode comments: https://twitter.com/nzkoz/status/538892801941848064/photo/111:03
dimiternvoidspace, dooferlad, please have a look - juju subnet CLI http://reviews.vapour.ws/r/1339/11:30
dimiternTheMue, that screenshot is too small to read, unfortunately11:37
* fwereade swings by quickly to say: sorry sick yesterday, public holiday today, should be around properly later to catch up a bit11:37
voidspacedimitern: looking after coffee11:45
TheMuedimitern: hmm, it is sent in full size, strange11:57
dimiternvoidspace, I'm in a call with spakiegeek now investingating an issue with maas and juju11:59
dimiternvoidspace, it seems we're not out of the rabbit hole yet12:00
dimiternvoidspace, I'm getting logs to help debugging the issue12:00
dimiternvoidspace, he'll file a bug with details12:01
dimiternvoidspace, it's a separate issue than the one you fixed, but also revolves around parsing lshw properly12:02
dimiternTheMue, ah, so it's just that thunderbird is dump - I managed to zoom in using a copy of the screenshot in gimp12:04
dimiternTheMue, that screen is from too early in the boot process12:05
dimiternTheMue, we should do a screensharing session I think to diagnose the issue12:06
dimiternTheMue, it won't work by just describing what you see :)12:06
TheMuedimitern: just doing some preparations12:14
TheMuedimitern: so, sent you an invitation, we'll see how it works12:19
dimiternTheMue, brt12:19
voidspacedimitern: ah, right :-/12:26
voidspacedimitern: so after #1339 there will be a "subnets" super command, but it will do nothing?12:33
voidspacedimitern: wouldn't it be better to add the functionality first and do the CLI as the last bit - or have I misunderstood?12:33
voidspace"subnet create" anyway12:33
dooferladvoidspace: the idea is to do CLI before API, so we know what the API needs to provide. We know what commands we want already (see Juju Network Model document)12:34
voidspacedooferlad: right, but then we're adding public facing brokenness12:35
voidspacedooferlad: if we know what commands we want, don't we also know what API we need?12:35
dooferladvoidspace: only to a feature branch12:35
voidspaceah yes, of course12:35
voidspaceI still prefer to work the other way round12:35
voidspacethen we don't *need* a long lived feature branch12:35
dooferladvoidspace: We have a pretty good idea about what the API needs to provide, but this way we don't end up designing endpoints that are a bad fit for the code that is using them.12:36
dimiternvoidspace, sorry, in a call again12:37
voidspacedooferlad: ok12:37
voidspacedimitern: np12:37
dimiternvoidspace, the CLI returns an error when you try to use it12:39
voidspacedimitern: well exactly :-)12:40
voidspacedimitern: CLI commands that do nothing but error out shouldn't exist...12:40
voidspacedimitern: why do you need the --private option?12:40
voidspacedimitern: if it's the default it does nothing12:41
voidspacedimitern: flag I mean12:41
voidspacespecifying it does nothing and it's existence is causing you pain in the code12:41
voidspacedimitern: why not just have --public ?12:41
dimiternvoidspace, we always went with the approach "state / api first, cli last" and that bit us on more than one occasion12:42
voidspacedimitern: ok, interesting12:42
voidspaceI'm definitely in favour of *designing* the CLI, just not necessarily implementing it first12:42
dimiternvoidspace, so starting from the CLI with stub API is the new way I'd like to try now12:43
voidspacebut as it's on a feature branch it doesn't matter so much anyway12:43
voidspacedimitern: ok12:43
dimiternvoidspace, so by the time we reach state /api we already know what's needed12:43
dimiternvoidspace, the --private being the default comes as a requirement from mark12:43
voidspacedimitern: fine. In that case the "--private" flag is superfluous, it does nothing.12:44
dimiternvoidspace, it's part of the spec though12:44
voidspacedimitern: just say, "networks are private by default, unless you pass the --public flag"12:44
voidspacedimitern: heh12:44
voidspaceit does *nothing*12:44
dimiternvoidspace, yeah that's correct12:45
dooferladvoidspace: well, it is an exciting way of starting an IRC conversation :-)12:45
voidspaceheh12:46
dooferlad... just like the name "space"12:46
voidspacedimitern: other than that (and feel free to drop my issue - I meant it as a comment) it all looks good to me12:47
voidspacedimitern: mostly boilerplate except for the parameter checking, which all looks fine12:47
dimiternvoidspace, sorry, I'm looking at your review now13:02
dimiternvoidspace, yes, having --private seems redundant, but it's part of the spec13:03
dimiternvoidspace, I'll add a comment13:03
TheMueA large tree crashed our car, shit. I'm off for some time.13:06
dimiternTheMue, oh that's bad :(13:06
jw4TheMue: shoot13:06
jw4TheMue: after all that felling of trees one still got your car!?13:07
dimiternTheMue, I hope the damage is not extensive13:11
wallyworldfwereade: andrew has looked at http://reviews.vapour.ws/r/1334/, it you get time would love it if you could look too13:12
TheMuethe rear is totally crashed13:12
dimiternTheMue, oh dear :(13:12
dimiternTheMue, well, the only thing I could do is offer you to take a day off if you need to sort things out :/13:13
dooferladTheMue: Yikes! I hope your insurer is efficient :-|13:16
mupBug #1438683 was opened: Containers stuck allocating, interface not up <cloud-installer> <landscape> <juju-core:New> <https://launchpad.net/bugs/1438683>13:22
dimiternvoidspace, dooferlad, I've replied to your review comments; thanks!13:32
dooferladdimitern: who knew one little flag would cause so much typing :-)13:33
dimiternvoidspace, I've also triaged that maas bug ^^ and added a comment capturing what me and sparkiegeek discovered so far, which will possibly help finding the issue13:34
voidspacedimitern: yeah, I've been reading it13:34
dimiterndooferlad, indeed :)13:34
voidspacesounds like a mess... :-/13:34
dimiternvoidspace, yep :( workarounds biting us back, as usual13:34
dimiternif only maas had usable api :)13:34
joec1Hi folks and esteemed juju devs, apologies for the impertinent question but are there any known issues getting juju to work in Azure (aside from the bug reports)?13:40
natefinchalexisb: sorry, I had hoped the baby wasn't audible over the headset mic.... guess it was13:45
alexisbyes, and sorry I muted you but the baby cry still triggers the mommy instinct and made it hard to concentrate on what I wanted to say :)13:46
natefinchheh I understand13:46
dimiternvoidspace, you should really do something to start noticing your PMs :)14:03
anastasiamacdimitern: tyvm for reviewing my PR14:25
ericsnowdimitern: PTAL http://reviews.vapour.ws/r/1234/14:25
anastasiamacdimitern: i had to add unit info and change sort order for tabular format14:25
ericsnowdimitern: also http://reviews.vapour.ws/r/1332/ and http://reviews.vapour.ws/r/1333/14:25
dimiternericsnow, ok, I have a few already, will add them to my queue14:26
anastasiamacdimitern: if u have a chance if u could PTAL it again despite ur prev shipit - would be amazing!14:26
ericsnowdimitern: thanks14:26
anastasiamacdimitern: http://reviews.vapour.ws/r/1323/14:26
dimiternanastasiamac, np - remind me which one?14:26
dimiternanastasiamac, ok, cheers14:26
anastasiamacdimitern: thnx ;D m EODing now 00:27 on the clock...14:27
ericsnowdimitern: http://reviews.vapour.ws/r/1332/ shouild be easy and is a blocker for the 1.23 release, so perhaps you could "nice" it up :)14:27
dimiternanastasiamac, oh that's late - have a good evening then :)14:27
dimiternericsnow, ack, will start with it then14:27
ericsnowdimitern: thanks!14:27
anastasiamacdimitern: or early :D ttyl14:28
ericsnowcherylj: it looks like the LICENCE fixes are all done for 1.22 \o/14:28
ericsnowcherylj: is that right?14:28
cheryljericsnow: yes, as long as the patch for dependencies.tsv passes CI :)14:29
ericsnowcherylj: awesome!  Thanks for tackling all that!14:30
cheryljericsnow: np!14:30
ericsnowcherylj: be sure to mark the bug as "fix committed" for 1.2214:30
cheryljthanks for the reminder :)14:31
cheryljhooray!  it passed CI14:32
mupBug #1438721 was opened: sync-tools does not support patch versions <juju-core:New> <https://launchpad.net/bugs/1438721>14:34
natefinchabentley: where can I find the code for the CI tests?14:40
aznashwandimitern, voidspace: ping14:40
dimiternaznashwan, pong14:40
abentleynatefinch: lp:juju-ci-tools14:40
voidspaceaznashwan: pong14:41
natefinchabentley: sigh.. launchpad, huh?14:41
aznashwandimitern: could I bother you guys with what looks like a horrendously huge review but there's actually little to it: http://reviews.vapour.ws/r/1330/ :D14:41
dimiternaznashwan, I'm about 33% done with that monster branch of yours about cloudinit14:41
dimitern:)14:41
abentleynatefinch: of course!14:41
aznashwandimitern: great14:41
aznashwandimitern: as said there, most of it's relocated code14:42
dimiternaznashwan, why did you decide to rename MachineConfig to InstanceConfig?14:42
dimiternaznashwan, that definitely blew up the diff considerably14:42
aznashwandimitern: that was bogdanteleaga's doing at the suggestion of one of you guys14:43
dimiternaznashwan, I see14:44
dimiternok then14:44
aznashwandimitern: he'd be the better half to ask about this14:44
dimiternaznashwan, ok, thanks14:45
bogdanteleagadimitern, it was done after some consulting with william; in hindsight it might not have been the best idea exactly because it ended up making the diff humongous, but there's not much to see outside of /cloudconfig14:47
dimiternbogdanteleaga, right, I agree it could've waited, but well .. :)14:51
TheMuedimitern: dooferlad: voidspace: wow, what an organizational stuff with insurance, fire department, neighbor. will post pics later, car seems to be totally crashed14:51
dooferladTheMue: time for another whisky tasting?14:52
TheMuedooferlad: will these eve definitely need one14:52
dimiternTheMue, :( too bad - hopefully the insurance will cover all14:52
TheMuedimitern: currently all signs are positive, looks like a 100% coverage14:53
TheMuebetter than many of our unit tests *lol*14:54
dimiternTheMue, good!14:54
jw4TheMue: I saw that about your car - You've already had one tree fall this year, and your neighbor was felling more trees a couple weeks ago... what gives?14:55
* natefinch tries to remember how to use bzr14:55
voidspaceTheMue: ouch :-/14:56
natefinchabentley: do we have tests trying out HA?15:02
natefinch(in CI)15:02
abentleynatefinch: Yes.  See http://juju-ci.vapour.ws/job/functional-ha-backup-restore/ and http://juju-ci.vapour.ws/job/functional-ha-recovery/15:03
voidspacedimitern: port is done, but not run manual tests yet15:07
voidspacedimitern: doing a full normal test run first15:07
voidspacedimitern: then can switch to the other bug15:07
dimiternvoidspace, ok, sounds good15:07
cheryljnatefinch, ericsnow:  I'm updating the dependencies for 1.23, and I see that updating juju/names pulls in this change:  https://github.com/juju/names/pull/4315:08
cheryljnatefinch, ericsnow:   It brings in a new tag, but doesn't change any functionality.15:08
natefinchspace tag sounds so much better than freeze tag15:08
cheryljnatefinch, ericsnow:  Will that be okay?  or do we need a new branch?15:08
cheryljhaha15:08
natefinchcherylj: that's expanding the API in a backwards compatible way, so I think it's fine to leave it in the same branch15:09
natefinchcherylj: wait sorry15:10
natefinchcherylj: misunderstood... yes, I think making a new branch there is the right thing to do15:10
natefinchcherylj: it's simple, but doesn't really belong in 1.23, and none of the rest of the code will know what to do with it15:10
cheryljnatefinch: okay, glad I asked :)  Can you create the branch?15:11
natefinchcherylj: do you have the right commit hash handy?15:11
natefinch(and yes)15:11
natefinchis it just the one before that space tag change?15:12
cheryljnatefinch: ce4ecb2967822062fc606e733919c677c584ab7e15:12
natefinchcherylj: thanks15:13
cheryljnp!15:13
natefinchcherylj: there you go15:15
voidspacedimitern: hah, a few calls to network.NewAddress to fix first...15:18
cheryljnatefinch: thanks!15:18
dimiternvoidspace, :) I dread the moment when I have to merge master into net-cli already15:20
voidspacedimitern: heh15:21
voidspacedimitern: it wasn't so many I had to do15:21
mupBug #1438748 was opened: Use of /tmp/discover_init_system.sh is a security vulnerability. <juju-core:In Progress by ericsnowcurrently> <juju-core 1.23:In Progress by ericsnowcurrently> <https://launchpad.net/bugs/1438748>15:28
voidspacedimitern: so, for the "multiple networks with no index" - manually increment the index?15:38
voidspacedimitern: we still risk duplicating an index if there are some with suffix and some without15:38
voidspacedimitern:  no idea if that's possible though15:38
dimiternvoidspace, I guess that's the only thing to do15:38
voidspaceok15:39
dimiternvoidspace, we should add more logging at debug level15:39
voidspacedimitern: ok15:39
dimiternvoidspace, cheers15:39
voidspacedimitern: and I'll look through the code seeing if I can work out why we pick the disabled nic15:39
voidspacedimitern: it's almost certainly due to them having the same subnet (network) *or* due to the index issue15:40
voidspacedimitern: but it would be good to know15:40
voidspacedimitern: running a manual test of my 1.23 branch in parallel to doing this15:40
natefinchabentley: I'm trying to write a new CI test to test using ensure availability to turn existing machines in the environment into state servers.  Is there documentation on how to write CI tests?  It looks fairly straight forward, but docs are always helpful :)15:40
dimiternvoidspace, yeah - it was hard to tell what's going on, but the logs should tell15:42
abentleynatefinch: Sorry, was on a call.15:49
natefinchabentley: np15:49
abentleynatefinch: There isn't documentation.  Basically you need to return 0 on success and 1 on failure, and you should allow the runtime environment name to be customized, and jujupy may be helpful.15:51
natefinchabentley: ok, cool.  It looks pretty easy, except perhaps how to add the extra jenkins job15:52
* natefinch knows jack all about jenkins15:52
abentleynatefinch: when you've got a script that works locally, I can help you with getting it onto Jenkins.15:53
abentleynatefinch: When you say "existing machines", do you mean machines that aren't provisioned by juju?15:54
alexisbnatefinch, can we land in the HA stuff at this stage?15:55
voidspacedimitern: ok, so manually testing this has revealed a problem with the upgrade step...15:57
voidspacedimitern: and I think trunk is screwed too - I just didn't notice before15:57
voidspacedimitern: http://pastebin.ubuntu.com/10712911/16:00
voidspacedimitern: the best fix is to have ReleaseAddress not require an instance iD16:01
dimiternvoidspace, oh sh*t16:01
voidspacedimitern: I'll repeat the test for trunk, but I bet it has the same issue and I just didn't notice it before :-/16:01
dimiternvoidspace, this is because we need the instance id?16:01
voidspacedimitern: yep16:01
dimiternvoidspace, yeah16:01
voidspacedimitern: but I don't think either MaaS or ec2 actually require it16:02
dimiternvoidspace, I should've thought of that16:02
voidspacedimitern: so we should just drop it from the provider method signature16:02
voidspaceif it's really not used16:02
dimiternvoidspace, well, how do you suggest fixing it?16:02
voidspacedimitern: yeah, me too :-)16:02
voidspacedimitern: stop requiring instance ID16:02
dimiternvoidspace, don't you need it to get the machine?16:02
dimiternvoidspace, or we added it only because we needed it for ReleaseAddress?16:03
voidspacedimitern: I think we just thought we might need it16:03
voidspacedimitern: ah no16:04
voidspacedimitern: ec2 does use it16:04
dimiternvoidspace, *whew* ok, then it's easy :)16:04
voidspacenope16:04
dimiternvoidspace, oh?16:04
voidspacedimitern: it actually needs a nic id16:04
voidspacedimitern: but we get the nic id from the instance id16:04
dimiternvoidspace, yeah16:04
dimiternvoidspace, ok, sorry I'm a bit confused16:04
cheryljCan I get a couple license related reviews?  http://reviews.vapour.ws/r/1342/      http://reviews.vapour.ws/r/1343/16:05
dimiternvoidspace, you need instance id in order to call ReleaseAddress in the worker16:05
voidspacedimitern: yes16:05
voidspacedimitern: maas doesn't use the instance id16:05
dimiternvoidspace, you don't need instance id in the provisioner api16:05
voidspacedimitern: but ec2 does16:05
voidspacedimitern: the issue here is specifically when the upgrade runs and there are dead ip addresses16:06
voidspacedimitern: nothing to do with the provisioner api16:06
voidspacedimitern: the worker notices the dead addresses when it starts and tries to remove them16:06
dimiternvoidspace, ok16:06
dimiternvoidspace, so we do need to pass instance id, even if it's unused16:06
voidspacedimitern: the addresses have a MachineId()16:06
voidspacedimitern: the worker tries to get the instance id from the machine - but the machine is already gone16:07
voidspacedimitern: Environ.ReleaseAddress takes an instance id16:07
voidspacedimitern: that is not used by maas, but unfortunately *is* used by ec216:07
natefinchalexisb, abentley: sorry, left for a minute to make lunch....16:08
dimiternvoidspace, but fortunately, in ec2 it doesn't matter16:08
dimiternvoidspace, because if the instance is gone, all its nics and ips are gone with it16:08
natefinchalexisb: I can land it by EOD.  I have to do a few trivial code review cleanup things and write a couple trivial unit tests, and then I can start working on the CI tests16:09
voidspacedimitern: ok, so we need a way to release the address for maas16:09
dimiternvoidspace, so I guess it only matters for maas - we need to release the addresses, but instance id can be empty16:09
dimiternvoidspace, it can be a special value like instance.UnknownId ?16:09
voidspacedimitern: ok, fine16:09
voidspacedimitern: and ec2 will make that a no-op16:09
natefinchabentley: I mean machines that have been added to the juju environment, either through deploy or add-machine.  So, like, you could do juju ensure-availability --to 1,2 and convert machines 1 and 2 into state machines16:09
voidspacedimitern: whereas maas will release the address16:09
voidspacedimitern: fine - thanks16:10
dimiternvoidspace, exactly - the upgrade step should try a best effort, but not fail if instance id cannot be found16:10
abentleynatefinch: Cool.  I was thinking this might be something like the manual provider.  We have a solution for testing the manual provider, but it's not the cleanest.16:11
natefinchabentley: nope, pretty easy, really... just like the other HA tests./.. just bootstrap, juju add-machine -n 2, wait for them to come up, then juju ensure-availability --to 1,2 and then wait for HA.16:12
dimiternericsnow, re http://reviews.vapour.ws/r/1332/ - it seems it just drops upstart-specific confs16:12
abentleynatefinch: great.16:12
ericsnowdimitern: that is correct16:12
dimiternericsnow, shouldn't it only drop them when systemd is used?16:12
ericsnowdimitern: we don't need them for upstart either16:13
dimiternericsnow, i.e. isn't that strictly worse than before?16:13
dimiternericsnow, ok, if they're not used for upstart fair enough16:13
ericsnowdimitern: they are just clutter since we generate a new upstart conf when we bootstrap16:13
dimiternericsnow, I'll take your word for it though :)16:13
dimiternericsnow, ok, that makes sense16:14
dimiternericsnow, thanks!16:14
ericsnowdimitern: if restore were more than replace-the-old-state server then we would have a different story :)16:14
dimiternericsnow, LGTM then16:14
ericsnowdimitern: thanks16:14
natefinchI love it when writing documentation for code reveals a bug.16:17
dimiternnatefinch, it's worse when it reveals a bug in the spec :)16:18
natefinchdimitern: heh16:18
natefinchdimitern: I actually wasn't being sarcastic.  I actually do like it when that happens.  Better than not finding the bug until later :)16:19
natefinchdimitern: btw, are watchers fired off serially, or do I need a lock in my Handle() method in case two get fired off in quick succession?16:19
dimiternnatefinch, you're using the same handler for 2 watchers? brave! :)16:23
natefinchdimitern: no no... but if there's two changes back to back16:23
dimiternnatefinch, the watcher should coalesce events properly16:24
natefinchdimitern: not exactly filling me with confidence ;)16:24
dimiternnatefinch, well, it depends if the watcher is working properly :)16:25
natefinchlol16:25
natefinchok16:25
cheryljnatefinch: we also need to branch juju/utils for 1.23  at commit 3efdaa3f15cc47ee83f6c03f640dc14f5915e28916:34
natefinchcherylj: done16:35
* natefinch is getting the hang of this git thing. ;)16:36
cheryljnatefinch: thanks!  I'm not sure if we need to branch testing too...  It pulls in https://github.com/juju/testing/pull/55  and https://github.com/juju/testing/pull/5616:37
alexisbnatefinch, awesome thank you16:37
natefinchcherylj: to be on the safe side, I think I would16:38
cheryljnatefinch: okay, the commit to branch from there is: c07aef9cb4f2ff1db7ddc9d43a0ea88eca39c9ea16:38
natefinchcherylj: done16:39
cheryljnatefinch: thanks!16:39
natefinchcherylj:  welcome :)16:41
=== natefinch is now known as natefinch-afk
mupBug #1438798 was opened: juju sync-tools requires default environment <juju-core:New> <https://launchpad.net/bugs/1438798>16:58
alexisbvoidspace, you still around?17:11
cheryljnatefinch-afk: I think we can get away with not branching juju/charm for 1.23.  It just pulls in this:  https://github.com/juju/charm/pull/8217:35
cheryljnatefinch-afk: what do you think?17:35
=== natefinch-afk is now known as natefinch
natefinchcherylj: lgtm17:37
cheryljnatefinch: thanks117:37
voidspacealexisb: yep17:39
voidspacealexisb: just, about to go17:39
wwitzel3is there an example of putting a provider behind a feature flag? not exactly sure how to approach this17:39
cheryljnatefinch: could you review a few more of the license changes?  http://reviews.vapour.ws/r/1342/      http://reviews.vapour.ws/r/1343/        http://reviews.vapour.ws/r/1345/            http://reviews.vapour.ws/r/1346/17:39
alexisbvoidspace, looks like you picked up a new bug today for 1.2317:39
voidspacealexisb: yep17:39
alexisbI was just curious the scope of work17:39
voidspacealexisb: two I think :-/17:40
alexisbyay!17:40
voidspacealexisb: should be able to land both in the next two days - one per day, maybe quicker17:40
voidspacealexisb: I need to file the second one17:40
alexisbgreat thank you that is what I needed17:40
voidspacealexisb: the one that is filed is a *little bit* unknown17:40
voidspacealexisb: so it *could* explode :-)17:40
voidspacealexisb: but we don't think so17:40
alexisbvoidspace, ok, keep me posted17:41
voidspacealexisb: will do17:41
natefinchcherylj: I'm kinda slammed getting some critical code into 1.23, maybe someone else could do the review, like voidspace or ericsnow or cmars or wwitzel317:42
cheryljah, okay, thanks17:42
cmarscherylj, about to head out for lunch, can take a look in a little bit17:43
cheryljthanks, cmars17:44
voidspacealexisb: actually 3 bugs17:46
voidspacealexisb: for 1.2317:46
cheryljah, ericsnow got 'em.  Thanks, ericsnow!17:46
ericsnowcherylj: they all LGTM17:46
cheryljthanks!!17:46
voidspacealexisb: bug 143882017:46
mupBug #1438820: IP address life field upgrade step and addresser worker don't play well together <juju-core:In Progress by mfoord> <https://launchpad.net/bugs/1438820>17:46
ericsnowcherylj: good catch on the blobstore dep :)17:46
voidspacebug 143868317:46
mupBug #1438683: Containers stuck allocating, interface not up <add-machine> <cloud-installer> <landscape> <maas-provider> <network> <juju-core:Triaged by mfoord> <https://launchpad.net/bugs/1438683>17:46
voidspacebug 143816817:46
mupBug #1438168: juju 1.23 doesn't release IP addresses for containers <juju-core:In Progress by mfoord> <https://launchpad.net/bugs/1438168>17:46
alexisbvoidspace, are you thinking a day for each?17:47
alexisbany you think could be deferred to a point release?17:47
voidspacealexisb: one of them is done but blocked on the other two17:47
voidspacealexisb: so still two days for all of them is my thinking17:47
voidspacealexisb: bug 1438168 is complete but in testing it I discovered bug 1438820 that also affects trunk17:48
mupBug #1438168: juju 1.23 doesn't release IP addresses for containers <juju-core:In Progress by mfoord> <https://launchpad.net/bugs/1438168>17:48
mupBug #1438820: IP address life field upgrade step and addresser worker don't play well together <juju-core:In Progress by mfoord> <https://launchpad.net/bugs/1438820>17:48
voidspacealexisb: but it is 3 bugs / 3 branches17:48
alexisbvoidspace, ok, now go enjoy your evening17:50
alexisbthanks for the info17:50
cheryljericsnow: I'm glad you commented that you couldn't see the review on RB.  I saw that I created that pull request against the wrong branch!17:52
cheryljericsnow: here's the correct branch:  http://reviews.vapour.ws/r/1347/17:53
ericsnowcherylj: LGTM17:53
cheryljericsnow: thanks again!17:53
ericsnowcherylj: np17:53
mupBug #1438820 was opened: IP address life field upgrade step and addresser worker don't play well together <juju-core:In Progress by mfoord> <https://launchpad.net/bugs/1438820>17:58
natefinchthumper: are you actually on?17:59
=== joec1 is now known as joec1afk
=== joec1afk is now known as joec1
natefinchso, you're supposed to use UnitsWatcher to watch Machines... makes total sense18:50
alexisbnatefinch, fwereade would have lots to say about that (this is one of the areas he wants to clean up really badly)18:57
natefinchalexisb: heh... yeah, this is the first time I've delved into watchers and stuff, and there's some interesting choices being made in there... I'm sure they each made sense at the time19:00
thumpernatefinch: no, I wasn't19:55
thumperobviously forgot to close IRC last night19:55
natefinchthumper: heh, figured19:55
natefinchthumper: 7am isn't your usual start time :)19:55
thumpernope, it isn't19:56
natefinchthumper: I had some questions about watchers and the API, but I think I figured it out from reading other API facades19:56
thumperawesome19:56
ericsnowdavecheney: in case you missed it (re: gccgo in vivid): https://lists.ubuntu.com/archives/ubuntu-devel/2015-March/038740.html19:58
natefinchericsnow: yeah, I saw that, but uh, joining ubuntu-devel to respond is kinda a high barrier of entry.  I think putting gccgo and go as different effective versions is a really bad idea.19:59
ericsnownatefinch: yeah, I was thinking that too :)19:59
thumperericsnow: dave is off now for a while19:59
ericsnowthumper: k20:00
thumperoff until he flies to eurpoe, next week sprinting on arm64 go stuff, then meeting us in Nuremberg20:00
ericsnowthumper: ah20:00
natefinchoh amazon, you're so slow...20:57
natefinchbbl21:02
=== natefinch is now known as natefinch-dinnne
=== natefinch-dinnne is now known as natefinch-dinner
mupBug #1436655 was opened: gce provider should stop using deprecated zone europe-west1-a <gce-provider> <juju-core:Triaged> <juju-core 1.23:Triaged> <https://launchpad.net/bugs/1436655>21:17
ericsnowwwitzel3: would you mind assigning the GCE issues you are working on to yourself21:24
ericsnowwwitzel3: there are 2 right?21:25
* thumper moves to the coffee shop to see if a change of scenery will help with focus21:40
ericsnowcherylj: so all 1.23 changes are wrapped up now, right? :)22:13
cheryljericsnow: yes :)22:13
ericsnowcherylj: lol, too many channels22:14
cheryljhehe :)22:14
cheryljnatefinch-dinner: In updating the dependencies for juju master, I see a new repo, juju/jujusvg which is using the LGPL, but doesn't mention the exception found in other repos.  Should I add it in?22:44
ericsnowcould anyone spare me a review on http://reviews.vapour.ws/r/1349/ and http://reviews.vapour.ws/r/1350/?22:44
redelmann_Hello, is safe to have two enviroments in same AWS account?23:15
redelmann_juju destroy-enviroment --force will kill everithing in aws? or just instances of enviroment?23:16
thumperredelmann_: interesting question...23:25
thumperredelmann_: in theory, it should be fine23:25
thumperredelmann_: in practice, I'd test it first23:25
thumperredelmann_: it should be fine, so if it isn't, please make sure you file a bug23:26
thumpercould just be my paranoia23:26
redelmann_thumper, I tried it a while ago and juju terminate all my instances (juju 1.20 i think)23:28
redelmann_thumper, thats why i asking about this23:28
thumperredelmann_: can you file a bug please?23:29
thumperalso though...23:29
thumperyou shouldn't need to use --force23:30
thumperthat is a very "last resort" mechanism for cleaning up an environment23:30
wallyworld_axw: after school drop off did we need to catch up about storage since we missed the standup?23:38
axwwallyworld_: I don't have much to talk about. I'm going to propose storageprovisioner changes today, assuming I get enough test coverage done23:39
axwcan do if you like tho23:39
wallyworld_all good, maybe we can touch base this arvo if needed23:40
axwwallyworld_: ok. gotta go help get the kids ready, bbl23:40
wallyworld_ttyl23:40
ericsnowwallyworld_: if you have a few minutes, could you take a look at http://reviews.vapour.ws/r/1349/ and http://reviews.vapour.ws/r/1350/?23:49
ericsnowwallyworld_: they are both for 1.2323:49
wallyworld_ok23:50
ericsnowwallyworld_: thanks23:50
redelmann_thumper, ok23:56

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!