/srv/irclogs.ubuntu.com/2016/01/20/#juju.txt

lazypowerblahdeblah o/ hey there00:21
blahdeblah\o lazypower00:22
lazypowerblahdeblah I understand you had some questions about how to get the public-address off every unit related to a subordinate service00:22
blahdeblah:-)00:22
lazypowerCan you re-state the question and I'll do my best to answer it for you00:22
* blahdeblah goes to cut & paste them00:22
blahdeblah I have a subordinate charm which needs to know the public-address of every primary charm with which it and its peers are related during the update-status hook. i.e. Every subordinate needs to have a full list of the primaries.00:22
blahdeblahWhat is the right way to achieve this?00:23
blahdeblahDo I need to gather the public-address from the related primary on each subordinate (during the relation-joined hook), then send that data across the peer relation and store it for use during the update-status hook?00:23
blahdeblahThis feels like a really good way to get inconsistent data on each subordinate, but I couldn't think of another way to do it.00:23
blahdeblahHope that makes sense00:23
lazypowerI think you can break this apart into 2 phases if you think about the communication that needs to happen00:23
lazypower1) all units tha thave this subordinate need the primary address, so thats inherent, but they need somewhere to send it. so on a peer relationship sounds correct00:24
lazypoweryou want to correlate that data,  do subordinates get leaders?00:24
* lazypower deploys one to find out00:24
blahdeblahIf it makes the problem easier, we can say that only the leader of the subordinates needs to do it (at least initially).00:25
lazypoweri dont think subordinates get a leader00:26
lazypowerit wouldn't really make sense for a subordinate to ever declare itself as a leader, it really falls in lign with the parent service its co-locating with00:27
blahdeblahMakes sense I guess00:27
lazypowerjust happens to have its own agent, config, and settings bucket00:27
* lazypower ponders00:27
lazypoweri dont think you've got any amenities to you that really help solve this00:27
blahdeblahIn that case, working out in the subordinate whether it's running on the leader primary probably makes the problem harder, not easier.00:27
lazypoweryou're going to emit massive amounts of data on one side, and need to correlate that on the other.00:28
lazypowerwell no, is-leader is very simple. truthy things in bash are powerful at short circuit operations00:28
blahdeblahI'm not following you00:28
lazypowerwell, if you had a way to know that one is the primary communication node, and it uses leader-set to broadcast its array of public ips00:29
blahdeblahSo the subordinate will still have access to all the leader functionality of its primary?00:29
lazypowerthe cluster could feesibly just emit that list of public-ips from the leader. so the update-status hook has a read-only copy of all the nodes coming from that leader, which by nature of the peer relation has all this data which comes in via relation-set00:29
lazypowerleader controls state of this data, collects, and emits it. The cluster itself uses the data coming from leader-get's dictated data-store00:30
lazypowerfunctionality like this was paramount when mbruzek and I were working on the tls layer bits in a peering fashion00:30
blahdeblahI'm following you even less now00:30
lazypowerhave a look at layer-tls00:30
lazypowerblahdeblah do you follow the TLS key exhcnage pattern? where server generates a CSR, CA responds with signed CSR00:31
blahdeblahSlow down00:31
blahdeblahAre "cluster" and "emit" jargon terms here? Or is "cluster" just generically referring to the group of peer units of the primary charm, and "emit" meaning the data sent on the relation between the primary and subordinate?00:32
lazypoweryeah, i'm throwing in slang, and i apologize for that00:32
lazypowerits late and i'm thinking more about dinner :) but i do want to sort this out.00:34
blahdeblahSo, if those aren't jargon terms, and I'm understanding you correctly, the primary charm would need to be modified to gather the public-address data from its peers and then send it across the relation to the subordinate, right?00:34
lazypoweryep00:34
blahdeblahThis is something I want to avoid, so that I don't have to modify the primary.00:35
blahdeblah(This is the whole reason for it being a subordinate.)00:35
lazypoweri understand, the other side to this is to look at conversation scopes00:35
blahdeblahi.e. So I can use a vanilla cs: charm00:35
blahdeblahlazypower: If you want to head to dinner, go for it; this is totally non-urgent.00:36
lazypowerblahdeblah https://jujucharms.com/docs/devel/developer-layers-interfaces#communication-scopes00:37
lazypowergive that a read, and lets re-convene on this topic00:37
blahdeblahno worries00:38
blahdeblahthanks for your time00:38
lazypowerblahdeblah i think you can do this with peering, and unitdata.kv and some clever comparison00:43
blahdeblahThat certainly matches my current understanding; the comparison need not even be that clever.00:43
lazypowerevery registry of that hook, you db.set() the ip in say a dictionary.00:44
blahdeblahI'll get some test code going next week and see how it goes.00:44
lazypowerthen where you need that data, just build it from whats stored in the unitdata.00:45
lazypowerok, i feel better, i somewhat answered the question. cheers blahdeblah o/00:46
blahdeblahYeah; it just seems a bit wrong to be storing stuff in a non-deterministic way that juju must surely store deterministically.00:46
blahdeblahthanks lazypower00:46
=== CyberJacob is now known as zz_CyberJacob
stubblahdeblah: subordinates do get leaders just like any other charm06:32
stubblahdeblah, lazypower : And the leader is not necessary cohosted with the primary service's leader06:33
blahdeblahInteresting06:33
=== narindergupta_af is now known as narindergupta
=== narindergupta is now known as narindergupta_AF
=== zz_CyberJacob is now known as CyberJacob
=== beisner- is now known as beisner
=== StoneTable is now known as aisrael
lazypowerstub good to know :) thanks14:31
=== narindergupta_AF is now known as narindergupta
hackedbelliniHello! I'm trying to upgrade the charm of my postgresql deployment on juju, but it gives me this: ERROR cannot upgrade service "postgresql-devel" to charm "cs:trusty/postgresql-39": would break relation "postgresql-devel:replication"14:33
hackedbellinithe weird thing is. That relation is the service with itself, and I never created it myself14:34
hackedbelliniI already tried to destroy the relation with "juju destroy-relation postgresql:replication postgresql" but it gives me "ERROR no relations found"14:35
hackedbellinianyone have any light for me?14:36
lazypowerhackedbellini looks like between charm revisions, the peer relationship was removed14:43
lazypowerI think juju is doing the right thing by blocking your upgrade. the charm has become backwords incompatible14:43
=== arturt_ is now known as arturt
hackedbellinilazypower: I see... So, there's no way for me to upgrade the charm on my deploy?15:21
lazypowerhackedbellini not safely, no.15:50
lazypowerhackedbellini i'm fairly certain you can juju upgrade-charm --force --switch {{charm_string_here}} but this would be marked as unsupported as it may exhibit weird behavior15:51
lazypoweri believe the pattern is to deploy the new postgresql charm and migrate the db's, but i may be wrong. stub would have more info here15:51
stubIIRC this was deemed a juju bug. I'm not sure if it is fixed or in what version.15:52
stubI've got tests that confirm upgrade works from r127 to trunk - I'm not sure how that maps to cs: version numbers.15:54
stub(with Juju 1.25)15:54
lazypowerstub so the upgrade-charm --force --switch should work?15:55
lazypoweras in the replica peer relation going away wont have any detrimental effects?15:55
stubJust plain upgrade-charm should work IIRC. upgrade-charm should not block if a relation is removed, or it becomes impossible to ever remove a relation.15:55
lazypoweryeah that behavior was changed in 1.25 i think15:56
stubHow old is cs:39 ? I only trust upgrades at all since bzr r127 (since that is what I have tests for) :)15:57
stubBut that said, no problems at all if you aren't using replication. If you are using replication, it might take me some research on what will happen.15:58
stubIf it is the leadership version, should be no problems. If it is pre leadership, there will be problems.15:58
hackedbellinistub, lazypower: correct me if I'm wrong, but you are saying that on 1.25 I should be able to upgrade-charm, is that correct? When I type 'juju version' it gives me '1.25.0-trusty-amd64'. But the charms are on 'agent-version: 1.20.12.1'. Is that the reason I cannot do that upgrade?16:02
lazypowerhackedbellini thats a big part of it16:02
lazypowerhackedbellini you should be able to juju upgrade-juju and jump those agents up to 1.25, and then your upgrade charm operation *should just work*16:02
lazypowerman 1.20 to 1.25 though? thats a pretty big leap in terms of agent capability16:03
stubYou have to upgrade to at least 1.24 anyway to use the modern PostgreSQL charm - it requires leadership and other features.16:03
lazypoweri'm asking in #juju-dev ot see if we have evidence of 1.20 => 1.25 upgrading without issue16:04
lazypoweri'm a little concerned as i recall 1.22 in that mix being a problem child16:04
lazypowerand it caused some headaches with env upgrades, so before i send you off on that path i want more data :)16:04
stubhackedbellini: Do you have a single PostgreSQL unit in the service, or do you have multiple units in the service?16:05
hackedbellinilazypower: hrm I see. Thank you, that information will help a lot! :)16:05
lazypowerhackedbellini well, they just confirmed that 1.20 is our jumping / reference point for all upgrades16:05
lazypowerso you're g2g on doing a juju upgrade-juju16:05
hackedbellinistub: I have 2 deployments, each one with 1 unit16:05
hackedbellinilazypower: nice! :)16:05
lazypowerwait.. so juju is blocking you on a single node upgrades because a peer relation breaks?16:06
* lazypower facepalms16:06
stubhackedbellini: That will make it easier. I'd still grab a backup just in case since you will have a rather old version, but should work.16:06
lazypowerthank you legacy behavior16:06
stublazypower: I think it was a regression, which is why no problems occurred when the rename actually happened.16:07
lazypowerah16:07
lazypowermakes sense16:07
lazypowerstub man, you're like omnipresent on teh bugs i find16:07
stubI'm not sure though. Can't change history anyway.16:07
lazypoweri must be traveling the same path, a few months behind you16:07
stubDude, I *wrote* half of the bugs.16:07
lazypowerstub will it make you unsettled if i make a t-shirt "Stub Wannabe #1"16:08
lazypowerI think it would be brilliant to follow you around a conference w/ the tee on16:08
stublazypower: You'll need a hippy wig16:08
lazypowerthat can be arranged16:09
hackedbellinilazypower: yes, it was a single node deploy16:09
lazypowerhackedbellini: yeah, i would do as stub suggests. grab backups if you care about the data16:10
lazypowerthen juju upgrade-juju16:11
lazypoweronce that's settled and every agent has reached 1.25, then tackle the pgsql upgrade16:11
hackedbellinistub: yes, I'll make a backup first. Luckily the machine I'm deploying (it is a local juju installation) is using btrfs, so I can just make a snapshot. Do you know if I should snapshot anything else other than /var and the place where '.juju' resides?16:11
stubhackedbellini: I'd go for the pg_dump output (probably cronned to ~postgres/backups)16:12
stubhackedbellini: If you want a filesystem snapshot, ~postgres or ~postgres/9.x/main16:13
stubhackedbellini: But the logical dump from pg_dump is your parachute in case everything else burns completely to the ground.16:14
hackedbellinistub: I mean the "master" machine itself. My deploy is a local one, around 10 charms deployed on 10 different lxcs (forgot to point that). That machine is using btrfs, so if I snapshot /var, all lxcs will also be snapshotted so I can restore everything if anything goes wrong.16:15
stubhackedbellini: Yeah, but I'd still have a logical dump cause I have trust issues ;)16:16
hackedbelliniI know that the lxcs are inside /var and there are important things on the ~/.juju of the user that runs it. But are there other places that I should also snapshot?16:16
cmarsi'm getting an install hook error in my layered charm, https://paste.ubuntu.com/14582988/16:16
hackedbellinistub: hahahaha I know what you mean :)16:16
cmarsanyone seen this?16:16
hackedbelliniwill surely do one also!16:16
stubI like having multiple escape routes dealing with real data ;)16:16
lazypowercmars  > Detected a distutils installed project ('six') which we cannot uninstall.16:17
lazypowerinteresting16:17
cmarssame charm-tools 1.11.0 i've been using16:17
lazypowercmars - clean machine? series, substrate?16:17
cmarslazypower, lxd provider, same trusty image i've been using16:18
=== dannf` is now known as dannf
lazypowerschenanigans. I'm not sure where the copy of six is coming from :/16:18
lazypowercmars have you rm -rf'd your built charm dir, rebuilt, and then deployed?16:19
cmarslazypower, oh yeah, that's my typical workflow16:19
stubcharmhelpers will install the python{,3}-six package if it somehow got imported before charms.reactive's bootstrap got invoked.16:19
stubBut you would have to try hard to do that.16:20
stubsix 1.5 sounds old enough to be coming from the Ubuntu package.16:21
=== tasdomas` is now known as tasdomas
kiko`async@riff:~$ juju sync-tools16:58
kiko`ERROR tools upload failed: 400 ({"Tools":null,"DisableSSLHostnameVerification":false,"Error":{"Message":"cannot get environment config: invalid series \"centos7\"","Code":""}})16:58
kiko`lazypower, any clue on why I'm getting the above?16:58
marcoceppikiko`: what are you trying to do?16:59
kiko`marcoceppi, uhh, juju sync-tools in order to do an upgrade-juju16:59
kiko`juju package is 1.25.016:59
marcoceppikiko`: what kind of environment is this?16:59
kiko`local16:59
kiko`lxc16:59
=== kiko` is now known as kiko
marcoceppiweird, I've not seen that error before17:00
marcoceppinot sure why it's trying to do something with centos717:00
kikohttps://bugs.launchpad.net/juju-core/+bug/151068817:00
mupBug #1510688: sync-tools tries to upload agents that are not permitted by the state server <bug-squad> <sync-tools> <juju-core:Triaged> <https://launchpad.net/bugs/1510688>17:00
kikodamn17:00
kikothis may be a deal-breaker for my upgrade17:01
kikocherylj, rick_h_ ^^17:01
hackedbellinilazypower, stub: unfortunately, we could not upgrade our juju installation, as you can see on what kiko said above17:04
cheryljkiko: hmm, I thought we had done some work around that recently.17:05
cheryljkiko: what series is it complaining about?17:05
lazypowercherylj "centos7"17:05
cheryljbleh17:05
cheryljok17:05
cheryljlazypower, kiko what version are you on now?17:06
kikocherylj, 1.20.1417:06
kikoerr actually17:06
kiko    agent-version: 1.20.12.117:06
cheryljkiko: okay, let me dig around and refresh my memory of that code path17:06
kikocherylj, it really does look like that bug is still alive and well :)17:06
cheryljkiko: well, what I thinking of was very recent, so it wouldn't be in 1.20.17:07
kikocherylj, https://bugs.launchpad.net/juju-core/+bug/151068817:07
mupBug #1510688: sync-tools tries to upload agents that are not permitted by the state server <bug-squad> <sync-tools> <juju-core:Triaged> <https://launchpad.net/bugs/1510688>17:07
kikocherylj, it's basically when you have a newer client installed (I have 1.25.0) and you try and do a sync-tools17:08
kikoit will also break upgrade-juju as soon as 1.26/2.0 is out17:08
kikothis sucks17:10
kikois there a workaround? I need to get off 1.2017:10
cheryljkiko: yeah, it does.  Let me see if I can work out a workaround for you17:10
beisnerthedac, rmq mitaka amulet test enablement MP passing, ready to land if you will:  https://code.launchpad.net/~1chb1n/charms/trusty/rabbitmq-server/next-amulet-mitaka-1601/+merge/28249717:39
thedacbeisner: I'll take a look shortly17:40
=== lifeless_ is now known as lifeless
=== xnox_ is now known as xnox
=== Zetas_ is now known as Zetas
=== med_ is now known as Guest65103
=== marlinc_ is now known as marlinc
cmarsstub, lazypower it looks like python-six is already installed on the 14.04 cloud image: https://paste.ubuntu.com/14583863/18:30
cmarsstub, lazypower so is this breaking the reactive install hook?18:31
=== mbarnett` is now known as mbarnett
aisraelWARNING failed to find 1.25.2 tools, will attempt to use 1.25.020:17
aisraelThis is on aws. Shouldn't amazon have the latest tools?20:17
rick_h_aisrael: try hitting up QA in the other channel?20:22
rick_h_aisrael: you'd think so but not sure how wide it's gone and curious what region/etc?20:22
aisraelrick_h_: ack, #juju-dev or is there a separate channel for q?20:22
aisraelya20:22
aisraelqa*20:22
=== menn0_ is now known as menn0
=== natefinch is now known as natefinch-afk
=== arosales_ is now known as arosales

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!