=== StoneTable is now known as aisrael
mupBug #1645477 changed: Problem deploying openstack-ha with juju deploy  <cpec> <juju:Triaged> <https://launchpad.net/bugs/1645477>00:44
anastasiamac_thumper: ping00:49
mupBug #1645477 opened: Problem deploying openstack-ha with juju deploy  <cpec> <juju:Triaged> <https://launchpad.net/bugs/1645477>00:53
mupBug # changed: 1195040, 1268364, 1457575, 1504272, 1528261, 1544796, 1552237, 1571457, 1613992, 1645477, 1645729, 165388801:05
thumperanastasiamac_: hey01:06
anastasiamac_thumper: it looks like memory/disk issues are prevailent on latest 1.25.x too... could it be related to what we are seeing in juju 2?01:07
anastasiamac_thumper: and hence related to work we did at EOY?01:07
thumperanastasiamac_: all different memory and disk issues01:07
thumperso much changed between 1.25 and 2.001:07
mupBug # opened: 1195040, 1268364, 1457575, 1504272, 1528261, 1544796, 1552237, 1571457, 1613992, 1645729, 165388801:08
blahdeblahanastasiamac_, thumper: We're definitely still seeing memory issues on latest stable versions of both 1.25 and 2.001:08
* thumper sighs... yeah01:08
thumperwe suck01:09
anastasiamac_thumper: we dont!01:09
thumperbut let's just focus on adding features01:09
thumperit'll be fine01:09
anastasiamac_thumper: did u find perfscaling CI collects helpful in triaging 2.x memory issues? don't we track goroutines there?01:11
anastasiamac_veebers: ^^01:11
veebersanastasiamac_: we don't currently track goroutines there01:12
veebers(but that's a possible feature we could add in the near future)01:12
mupBug # changed: 1195040, 1268364, 1457575, 1504272, 1528261, 1544796, 1552237, 1571457, 1613992, 1645729, 165388801:14
blahdeblahanastasiamac_, thumper, veebers: I'm about to do a check of my envs (1.25.9 & 2.0.2) for memory issues; any info I could provide on bug 1645729 or others that would help?01:25
mupBug #1645729: environment unstable after 1.25.8 upgrade <juju:Fix Committed by axwalk> <juju 2.1:Fix Released by axwalk> <juju-core:Fix Released by axwalk> <juju-core 1.25:Fix Released by axwalk> <https://launchpad.net/bugs/1645729>01:25
thumperblahdeblah: not right now... I don't think01:25
veebersblahdeblah: I can't suggest anything useful01:30
mupBug #1650401 changed: Kubernetes-core bundle fails to deploy. Easy-rsa results with failed hook <juju:New> <https://launchpad.net/bugs/1650401>01:35
mupBug #1650405 changed: Juju Embedded - Juju logout/login not working for multiple users connected to same controller   <juju:New> <https://launchpad.net/bugs/1650405>01:35
blahdeblahthumper, veebers: FWIW, I have a 2.0.2 Canonistack controller running 1 env which has up since Dec 27 05:51:53 201601:37
blahdeblahSo it seems a lot better on 2.0.2 than 1.25.9 which has been restarted several times since.01:38
* thumper nods01:38
blahdeblahanastasiamac_ asked for an update on that bug, so I'll add the above there.01:42
anastasiamac_blahdeblah: thank you!01:43
* thumper screams01:50
* thumper headdesks01:50
anastasiamac_thumper: wot?01:50
thumpervery stupid code01:51
* thumper blames ian01:51
anastasiamac_good! u can only blame ppl that r not here to defend themselves :D01:52
anastasiamac_thumper: which part of code?01:52
thumperI'm trying to trace the run away state references...01:53
anastasiamac_awesome \o/ should it not have been behind a feature flag?01:53
thumper75 machines01:53
thumpercall it average 6 units per machine01:54
thumperor even three01:54
thumperthat is 300 api connections01:54
thumpereach api connection creates an extra state because stupid01:55
anastasiamac_wow :(01:55
thumperso... found that01:55
thumperbut it isn't the cause of this leak01:55
thumperi just... don't even...01:56
* redir goes eod02:11
anastasiamac_veebers: i have a q about MM functional tests. r u the person to ask?03:30
thumperanastasiamac_: https://github.com/juju/juju/pull/678903:33
* anastasiamac_ looking03:33
anastasiamac_thumper: Stata? funny :) that alone is an indication of the rush in which this code was produced...03:35
anastasiamac_well, done - i think it should b added to Hall of Fame or at leats 'Treasure Hunt 2017' repository... i suspect there will b a few of these in the months to come03:37
anastasiamac_natefinch: ping... r u really here?03:39
anastasiamac_frobware: ping03:43
veebersanastasiamac_: hey, I probably am. What's the query? :-)03:57
anastasiamac_veebers: was looking at https://bugs.launchpad.net/juju/+bug/164806303:58
mupBug #1648063: kill-controller removes machines from migrated model <gap> <model-migration> <juju:Triaged by alexis-bruemmer> <juju-ci-tools:Triaged by veebers> <https://launchpad.net/bugs/1648063>03:58
anastasiamac_veebers: which looks like it could have a pretty simple functional test... do we have one?03:58
anastasiamac_veebers: i was hoping that we did and could easily verify if the issue has been fixed..03:58
anastasiamac_veebers: there were couple of MM fixes over the last couple of weeks, it seems03:59
veebersanastasiamac_: there is no test for this currently, but it's on the books to be added04:03
anastasiamac_veebers: k. thnx04:04
anastasiamac_thumper: funny: https://bugs.launchpad.net/juju/+bug/1519147... somehow I do not thinkg that Dave's on it04:09
mupBug #1519147: worker/rsyslog: data races <2.0-count> <juju:In Progress by dave-cheney> <https://launchpad.net/bugs/1519147>04:09
thumperyeah... probably not04:09
t0mb0can anyone tell me from https://jujucharms.com/docs/stable/authors-charm-actions#example-schema06:22
t0mb0looking at the example schema06:22
t0mb0what is going to be in the files under actions/06:22
t0mb0ie actions/report06:22
t0mb0from reading that it looks like everything is defined in actions.yaml06:22
t0mb0and then the implementation is handled in the reactive layer06:22
anastasiamac_t0mb0: this looks like a question best suited for #juju channel :)06:31
blahdeblahanastasiamac_: We've got another occurrence of what looks like 1645729 on juju 2.0.2 - do you want any further info gathered?06:52
anastasiamac_blahdeblah: and i was planning to hold my breath for the next 7mins...06:53
anastasiamac_blahdeblah: yes plz ;)06:53
blahdeblahWhat needs gathering?06:53
blahdeblahJust the standard go routines dump?06:54
anastasiamac_blahdeblah: for now, yes.06:54
anastasiamac_blahdeblah: altho m a little confused, the bug is about 1.25.x..06:54
blahdeblahIt seems to be assigned to all current versions, including 2.106:55
blahdeblahpresumably because axw found the same issue in them06:55
anastasiamac_blahdeblah: ah i see... and u have been sayint that equivalent 2.0.2 is fine.. until 5mins away from eod ;D06:55
anastasiamac_blahdeblah: got it..06:55
anastasiamac_blahdeblah: thank you for providing more info!06:56
blahdeblahSo on my canonistack env, 2.0.2 has been good so far, but this is another env on the azure provider06:56
anastasiamac_blahdeblah: \o/ loving it! and yes, update on the bug would be awesome06:58
blahdeblahanastasiamac_: Mind having a 1-minute look over https://wiki.canonical.com/InformationInfrastructure/WebOps/Juju#Gathering_debug_info_with_pprof to make sure I'm gathering the right stuff?06:58
* anastasiamac_ looking06:59
anastasiamac_blahdeblah: looks awesome \o/ someone who worte it up should get a medal07:02
blahdeblahOK - gathering now07:03
blahdeblahTBC, this might be lp:1635311 rather than lp:1645729 - I can't be sure until checking the symptoms07:06
anastasiamac_blahdeblah: ack07:07
frobwareanastasiamac_: pong & HNY08:13
anastasiamac_frobware: HNY to u 2 :D08:13
anastasiamac_frobware: m about to do kids/dinner.. could I pm u later-ish? maybe in 3hrs?08:14
frobwareanastasiamac_: sure08:14
anastasiamac_frobware: \o/08:14
=== rogpeppe is now known as rogpeppe1
=== rogpeppe1 is now known as rogpeppe
SimonKLBwhen using maas as the provider, what is deciding the ip of the machines? for some reason, the machine ip of the node seem to be choosen arbitrary even though im using an external dhcp (not the one maas provides)09:48
SimonKLBwhen i first booted up the maas node i got the correct ip from my dhcp server, but when i run "juju add-machine" the node is created with a different ip09:49
SimonKLBthat ip is even in one of the "reserved ranges" that i have stated in the maas configuration, which makes me even more perplexed09:49
frobwareSimonKLB: what is the network configuration of the node? DHCP? Auto Assign? Static Assign?11:17
frobwareSimonKLB: Correction - "what is the network configuration of the node that was allocated from MAAS" when you did juju add-machine11:18
voidspacenatefinch: hey, you around?11:51
voidspacenatefinch: I'm off to the dentist, I'll be in touch on my return11:59
=== deanman_ is now known as deanman
SimonKLBfrobware: ah, youre correct, it was set to auto-assign, tried static first and then DHCP which worked, still curious though how auto-assigned resulted in a different ip which was reserved12:38
SimonKLBfrobware: does juju have anything to do with how the ip is chosen, or is it purely maas?12:38
frobwareSimonKLB: MAAS12:38
frobwareSimonKLB: consider MAAS as your IP address management source12:38
SimonKLBfrobware: gotcha12:39
SimonKLBfrobware: what about LXD containers, for example in the openstack bundle?12:39
SimonKLBfrobware: from where are they grabbing their ips?12:39
frobwareSimonKLB: if you do `juju add-machine lxd:0` we as MAAS for an IP address. It's static in /e/n/i in the container but is statically allocated from MAAS.12:40
SimonKLBwill it fetch the ip from the same subnet as the host?12:40
SimonKLBfrobware: and if you have multiple nics, which subnet?12:41
frobwareSimonKLB: for Juju 2.0 it will use interfaces that have a subnet and an IP address12:41
frobwareSimonKLB: otp - will answer in a bit12:43
SimonKLBfrobware: yea, just tried it, it has the same behavoir as auto-assign in MAAS it seems12:44
SimonKLBfrobware: it grabs an ip that is reserved, which is problematic :/12:44
frobwareSimonKLB: as in the reserved range? What's the issue?12:46
SimonKLBfrobware: yea, i want to restrict the subnet to a range, from what i understand the reserved ranges are supposed to be "those that are already taken"12:47
SimonKLBfrobware: but when i deploy an LXD container it still grabs an ip from a range that is reserved12:47
SimonKLBfrobware: this is where i read it https://docs.ubuntu.com/maas/2.1/en/intro-concepts12:50
frobwareSimonKLB: you have a reserved range setup in MAAS?12:50
SimonKLB"Reserved range An IP range that MAAS will never use. You can use it for anything you want (e.g. infrastructure systems, network hardware, external DHCP, or the namespace for an OpenStack cloud you will be building)."12:50
SimonKLBfrobware: i have a subnet with a CIDR that includes a number of ips, most of them i have in the reserved range so that they are restricted from being used by MAAS12:51
frobwareSimonKLB: hmm. so the container has been allocated a IP addr from your reserved range.12:51
SimonKLBfrobware: correct12:51
SimonKLBfrobware: and this also happened to a machine before12:51
SimonKLBfrobware: when it had "auto-assign" as the ip mode option12:51
=== deanman is now known as deanman_
frobwareSimonKLB: I have two ranges on my> as reserved and as dynamic.12:57
SimonKLBfrobware: the dynamic ranges are only used when you use the MAAS DHCP right?12:57
SimonKLBfrobware: im running an external DHCP12:57
frobwareSimonKLB: I just added two containers and they ended up as and - which seems to be correct12:57
frobwareSimonKLB: yes to your DHCP question12:58
SimonKLBfrobware: good, with you so far!12:58
frobwareSimonKLB: so, it seems to be allocating out of the correct range. agreed?12:58
SimonKLBfrobware: yea12:59
SimonKLBfrobware: but i wonder if it could be behaving differently with an external DHCP instead of the MAAS one12:59
frobwareSimonKLB: perhaps. difficult for me to repro that quickly.13:00
frobwareSimonKLB: I did try without a DHCP range as well. it was allocated out of the non-reserved range.13:00
SimonKLBfrobware: yea np, but is there a way to simulate the DHCP "IP Mode" option that you have for machines?13:00
SimonKLBfrobware: for lxd containers that is13:01
SimonKLBfrobware: because changing it from auto-assigned to DHCP seemed to have fixed it for me at least for the machines13:01
frobwareSimonKLB: you're using MAAS 2.1.1, correct?13:03
SimonKLBfrobware: maas/xenial,now 2.1.2+bzr5555-0ubuntu1~16.04.113:05
frobwareSimonKLB: I wonder if there is some difference on
frobwareSimonKLB: I just created: reserved range:> and dynamic as>
frobwareSimonKLB: allocated another container and it came back as i.e., before the reserved range13:06
SimonKLBfrobware: what happens when you try to allocate more containers than available IPs?13:07
frobwareSimonKLB: which to me says MAAS is finding the first applicable IP address, given the ranges.13:07
SimonKLBfrobware: yea i wish :)13:07
frobwareSimonKLB: I would expect MAAS to say "no, IPs available", and for Juju to gracefully fail.13:07
* frobware lunches13:18
=== deanman_ is now known as deanman
voidspacenatefinch: bug #163125415:31
mupBug #1631254: [2.0rc3] lxd containers do not autostart <rteam> <juju:In Progress by mfoord> <https://launchpad.net/bugs/1631254>15:31
voidspacenatefinch: https://github.com/tych0/juju/commit/81156dfb3c1d21431cb3bd5047a51e13bd91fc5d15:32
voidspacemacgreagoir: thanks for the email15:46
=== deanman is now known as deanman_
redirthumper: can you have a look at https://github.com/juju/juju/pull/6783 ?21:30
redirwhat's the magic incantation to get cross model relations to work. I assume it is behind a feature flag23:17
redirwell I thikn i have it but still have an error23:20
redirgot it23:33

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!