[00:00] LiftedKilt: yea, odd. I've not really tried to just add a machine but we've got tests on that that pass. [00:00] LiftedKilt: but that's an odd one, will have to try to replicate [00:00] c0s: good points you are uncovering though [00:00] well, too bad am not making much progress because of them ;) [00:01] ah well [00:01] spring XD looks interesting [00:01] rick_h_: I'll play with it some more tomorrow, but 99% sure all the wily charms fail for me [00:01] but xenial and trusty work just fine [00:01] arosales: it is ... Pivotal seems to be super-hot about this... Althgough, it is more of a developer tool. ... [00:01] c0s: its part of the dev process [00:02] rick_h_: leaving work now - thanks for the help [00:02] c0s: the existence of springXD in itself is a proof point of the complexity at this layer [00:02] But supposedly you can do something like [00:02] xd> stream create kafka-source-test --definition "kafka --zkconnect=localhost:2181 --topic=event-stream | log"--deploy [00:02] and start consuming messages from a kafka topic [00:02] yup, 'tis true sir [00:04] c0s: interesting re springXD. It seems something is going to have to pari with kafka. I am not sure if thats Gobblin, Flume, springXD or something else [00:04] *pair [00:06] rick_h_: would that be an issue with curtain and wiley? [00:06] rick_h_: or was maas putting down a proper wiley that could be ssh'ed into [00:06] arosales: I'm not sure. I'd want to see the logs and see where it fell over from the machine and the juju logs [00:06] arosales: the not able to ssh I thought was a xenial lxd things, not maas/wily [00:07] well juju aside maas should be able to set down a wiley instance that one can ssh into [00:07] at least from a debug perspective [00:07] arosales: yes [00:07] arosales: yes... needs to be defined. Just forcing a user to write their own sinks and consumer code might be too much for most... [00:07] arosales: he didn't have any ssh keys setp so not sure what it must have setup there since juju ssh failed. [00:08] arosales: should have tried juju ssh with --debug to see if anything jumped out [00:08] c0s: agreed even if for a given set [00:09] yup [00:09] ok, I've created a card. [00:09] c0s: thanks [00:09] Switching off... was a long day [00:09] you're welcome arosales [00:09] c0s: have a good evening [00:10] rick_h_: well I am sure LiftedKilt will be back tomorrow, thanks for the help [00:10] TTL === scuttlemonkey is now known as scuttle|afk [02:12] tvansteenburgh: Some basic tests, but I've been blocked from testing 2.0 (an open bug that breaks the PostgreSQL charm) [08:16] hi$ [08:44] xnox, did you hit this - https://bugs.launchpad.net/juju-core/+bug/1564791 ? [08:44] Bug #1564791: 2.0-beta3: LXD provider, jujud architecture mismatch === spammy is now known as Guest42671 [09:21] jamespage, no, because i bootstrap with uploading my own tools. [09:21] jamespage, there are no s390x tools published yet. [09:22] xnox, hmm [09:22] ok [09:23] xnox, ok trying with that now === urulama_ is now known as urulama [09:28] is there any way to have a "private" charm store ? Something that the end-user can use with the GUI to deploy custom charms ? [09:29] jamespage, but the bug is valid.... s390x should not try to download and use amd64 tools. [09:30] deanman: no that I am aware of [09:30] +t [09:35] dosaboy, apologies for not spotting this before but please can we move the context for the internal endpoint config option to charm-helpers please.. [09:44] jamespage: sure [09:51] dosaboy, thanks [10:01] jamespage: https://code.launchpad.net/~hopem/charm-helpers/lp1456876/+merge/290701 [10:02] dosaboy, cough.. [10:02] unit test ? [10:02] jamespage: come on its Friday ;) [10:02] yeah sure [10:16] jamespage: done. === sk_ is now known as Guest90374 [13:12] dosaboy, got anytime for https://code.launchpad.net/~gnuoy/charm-helpers/sysv-service-running/+merge/290729 > [13:12] ? [13:13] gnuoy: otp but yes [13:13] ta === lazyPower is now known as lazy|travel [14:07] Hey cory_fu, kwmonroe, what do you think about changing the default execution mode of spark from yarn-client to standalone? [14:07] Good morning! [14:09] i'm ok with that kjackal, but let's get c0s' +1 [14:11] Is is reasonable since spark can work out of the box, however there is a catch here, our bundles that use spark should add an option to set the old-default execution mode [14:14] That's not a big deal. +1 [14:18] Prabakaran: Hello, you mentioned you want to use juju 2.0 on 14.04 ppc64le. [14:18] It should work, but I think you would have better results with 16.04. [14:18] Prabakaran: have you added the ppa:juju/devel [14:20] ? [14:21] yes i have added .. i am trying to take the error what i am getting [14:21] OK [14:25] Prabakaran: what environment are you using? local (containers), maas, or cloud? [14:32] jamespage: rabbitmq-server and percona-cluster do not need default extra-bindings, correct? I'll land those if that is the case. [14:33] thedac, yeah - relations == network connections in which case extra-bindings are not required! [14:33] thedac, I was also thinking about horizon [14:33] i think the same is true there [14:33] ok [14:34] juju deploy openstack-dashboard --bind "website=public identity-service=internal" [14:34] thedac, I'll raise a README for that... [14:34] cool [14:40] Hello Matt, I was getting "E: Unable to locate package juju2" on the Linux c277-pkvm-vm54 3.16.0-30-generic #40~14.04.1-Ubuntu SMP ppc64le GNU/Linux machine [14:40] I followed the steps referring https://jujucharms.com/docs/devel/getting-started [14:41] Prabakaran: Right but you need to install the ppa:juju/devel instead of the "stable" one. [14:41] k let me try with that [14:41] Prabakaran: https://launchpad.net/~juju/+archive/ubuntu/devel/+packages tells me that there is a package for 14.04.2 [14:41] Prabakaran: a juju2 package [14:42] Prabakaran: after you add the devel ppa you will need to do an apt-get update [14:42] and apt-get upgrade [14:42] ya it is getting installated.. [14:43] The error message you gave me listed 14.04.1, perhaps the system has not been updated in a while [14:43] ? [14:44] i think i got this error because i have used ppa/stable instead of ppa/deve; [14:44] now it is installaed [14:45] Prabakaran: OK great [14:46] Prabakaran: Juju 2.0 is *very* different than 1.25. You will need to set up a different directory. ~/.local/share/juju [14:46] oh k [14:46] can you help me with that? [14:47] Follow the devel docs and report a doc issue if you have problems or suggestions: https://github.com/juju/docs/issues/new [14:47] i think i will have to instal lxd for local cloud right? [14:48] This link is asking me credential for GITHUB [14:48] Yes that is where we keep our issues for documentation [14:48] i dont have github account [14:49] Oh I see [14:49] Prabakaran: If you want to set up LXD you can look here: https://jujucharms.com/docs/devel/config-LXD [14:49] lxd is pretty new and will work best on 16.04 [14:50] Prabakaran: actually reading it now, I see 15.10 or 16.04 is a prerequisite [14:50] https://jujucharms.com/docs/devel/config-LXD#prerequisites [14:50] k [14:51] that is because of the kernel features that are used in LXD [14:52] oh k [14:53] all machine what we are using is of Linux c277-pkvm-vm54 3.16.0-30-generic #40~14.04.1-Ubuntu SMP ppc64le and x86 only.. thats no problem let us swith to ubuntu 16 [14:54] Prabakaran: It is a good idea to get acquainted with 2.0, [14:54] before that let me check this using vm hypervisors [14:54] Prabakaran: you can use your cloud credentials on 2.0 and use AWS or something else, but LXD requires 15.10 or later [14:55] ya k sure.. let me try in ubuntu 15.10 or above [14:56] Thanks for your patience in helping me :) === cos1 is now known as c0s [14:59] jamespage: network spaces meeting? [14:59] thedac, yes [15:07] Prabakaran: not a problem [15:18] arosales: rick_h_: I really appreciate you guys helping with this. Here's a pastebin of the results http://paste.ubuntu.com/15577480/ [15:19] rick_h_: arosales: juju can't ssh into the machine, but I can manually when using juju's private key and the ubuntu user. No idea what's going on there... [15:25] LiftedKilt: can yoilu try with --debug please? [15:30] rick_h_: I did run it with debug in the paste - you have to put --debug before the ssh command otherwise it get's passed to openssh === redelmann is now known as rudi|comida [15:41] LiftedKilt: I am looking at that output and it looks like it successfully sshed in [15:42] stormmore: the bottom part where it is successful is with me manually connecting [15:43] stormmore: the top portion up until the adding to known hosts is when juju tries, and it just hangs there and never connects [15:43] line 20 is where juju hangs, line 27 is me manually connecting [15:45] LiftedKilt: ah ok, I see that, usually when I see problems like that with services it is a network routing issue [15:46] LiftedKilt: what does your network config look like? [15:46] stormmore: I don't think it's a network issue because trusty and xenial images work - it's just wily that fails [15:46] stormmore: maas and juju are on the same box, sitting on a 10.101.0.0/16 network for testing [15:47] LiftedKilt: log in manually and try and ping the maas server and vice versa [15:48] I'm logging in manually from the juju machine === matthelmke is now known as matthelmke-afk [15:50] stormmore: I've narrowed it down to juju not being able to ssh, but on the same machine using the same credentials I am able to manually connect [15:50] LiftedKilt: one different I do see is juju is using the juju_id_rsa-cert and not juju_id_rsa (which it can’t fine) [15:51] can you try ssh ubuntu@10.101.103.1 -i /home/toor/.local/share/juju/ssh/juju_id_rsa-cert [15:51] that file doesn't exist [15:52] in the juju/ssh folder I only have a juju_id_rsa and a juju_id_rsa.pub [15:52] which is what I would expect [15:52] what about the permissions on the files? [15:52] debug1: identity file /home/toor/.local/share/juju/ssh/juju_id_rsa type 1 [15:52] debug1: key_load_public: No such file or directory [15:53] stormmore: 600 on both [15:53] that is what I believe is the problem [15:53] so why is it only failing on wily? [15:54] version of OpenSSH maybe? [16:00] stormmore: openssh client version 1:6.9p1-2 [16:02] do you have a trust/xeniel in the environement that you can get a juju --debug ssh -vvv out of? [16:03] stormmore: spinning up one of each now [16:04] LiftedKilt: idea would be to see what difference we are seeing between wily and them using the same commands [16:04] stormmore: right [16:09] stormmore: here's trusty http://paste.ubuntu.com/15578169/ === urulama is now known as urulama|afk [16:16] so doesn’t show anything different. ok how about doing ssh -vvv ubuntu@10.101.103.1 -i /home/toor/.local/share/juju/ssh/juju_id_rsa? [16:18] stormmore: http://paste.ubuntu.com/15578241/ [16:19] stormmore: also - I rebooted the machine, and now the juju ssh command is returning this: http://paste.ubuntu.com/15578255/ [16:19] rebooted node 0, that is [16:20] just that looks odd and definitely outside my knowledge [16:21] stormmore: apparently the macaroons message isn't related to the reboot, because the other wily nodes I didn't reboot are returning the same message as well [16:32] I am still thinking an openssh version compat issue [16:33] looks like it is getting stuck trying to determine the remote version [16:33] stormmore: but if that was the case, then wouldn't it fail when manually initiated outside juju? [16:34] nope [16:34] also the maas/juju server is running on wily itself [16:35] cmars: ^ any ideas on the "macaroon not found in storage" error? [16:36] cmars: is it a rm ~/.go-cookies kind of thing to try ? [16:36] well this could be a problem [16:36] the "wily" machine isn't wily, it's xenial [16:37] that is what I was wondering [16:37] I was seeing OpenSSH 7 on it [16:37] rick_h_, sorry, i'm looking at the scrollback, what's the context here? [16:38] juju status stopped working - no macaroon found in storage [16:39] LiftedKilt, ah, ok. try rm ~/.go-cookies [16:39] LiftedKilt, what version of juju client are you using? [16:39] I'm running locally on the juju server, which is juju2 beta3 [16:39] clearing go-cookies didn't resolve it [16:41] LiftedKilt, you get this error message from the juju client when running `juju status`? or this is in the logs on the server? [16:41] from the client [16:43] stormmore: it looks like the problem is that juju ssh 0 is trying to ssh to the wrong machine [16:44] node 0 is actually 10.101.103.3, which is running wily. the machine it is connecting to (10.101.103.1) is a separate machine running xenial [16:44] hence the mismatch [16:44] cmars: I've actually been running into the macaroon problem repeatedly, and I have to blow it all away and re-bootstrap to resolve it [16:45] yeah that is really odd [16:45] LiftedKilt, what provider is this? [16:45] maas 1.9 [16:45] LiftedKilt, is the controller HA? [16:46] cmars: No, although when I tried making it HA on a previous deployment, it caused this problem. Had to blow it away and start over === matthelmke-afk is now known as matthelmke [16:49] LiftedKilt, i think this might be an issue with the MAAS provider, I see some bugs open on Juju 2.0-beta3 with MAAS 1.9.1 [16:49] LiftedKilt, my suggestion is to open a bug at https://bugs.launchpad.net/juju-core [16:50] LiftedKilt, include terminal output from running the juju status command, as well as machine logs off the controller machines [16:51] LiftedKilt, also the controllers.yaml and models.yaml, with private keys scrubbed if necessary [16:51] cmars: I wonder if the maas database is getting corrupted, and causing the machine id to stop correlating with the correct node number [16:52] LiftedKilt, it's possible. I know there are some other maas-related bugs right now impacting beta3, this may help provide a clearer picture [16:53] cmars: ok I'll start gathering some info [16:53] LiftedKilt, thanks! [16:54] jamespage: anyt time for a finaly review? https://code.launchpad.net/~thedac/charm-helpers/apparmor/+merge/290096 [16:54] cmars stormmore rick_h_: thanks for the assistance - I'll keep poking at it [16:54] s/anyt/any === rudi|comida is now known as rudi|wfhg === rudi|wfhg is now known as rudi|wfh [18:11] how do you get the JUJU_REMOTE_UNIT in a layers interface? [18:24] cholcombe: the conversation is the JUJU_REMOTE_UNIT typically [18:24] IIRC [18:24] cory_fu: ^? [18:26] marcoceppi, i'm basically trying to recreate this with layers: https://github.com/openstack/charm-ceph-mon/blob/master/hooks/ceph_hooks.py#L540 [18:26] getting the service_name out of the relation in the interface [18:28] cholcombe: The conversation instance will have a "units" property that is a list of the units for which that conversation applies. If the relation scope is UNIT, then it will be a list with only one element. Otherwise, it may contain more [18:29] cory_fu, hmm ok [18:32] marcoceppi: Relying on conv.scope to have a particular value was a mistake on my part and I never should have recommended it. :( === cos1 is now known as c0s [20:29] Can I get a review: https://github.com/juju-solutions/layer-basic/pull/51 [20:46] +1, merged cory_fu === urulama|afk is now known as urulama|eow