blahdeblahBlackDex: hi - best to keep charm Qs in the channel here.  It's a team effort and someone else may be able to help you in a more timely fashion due to time zone differences. :-)00:03
=== mskalka is now known as mskalka|afk
stokachubdx, is that the proxy error message again?05:24
Zicre here06:13
Zicall my Ingress are fine since yesterday :)06:13
ZiclazyPower: do I need to disable something to shut the debuf off `juju run-action debug kubernetes-master/0` command of yesterday?06:13
Zichmm, I rebooted all the kubernetes-worker to test the resilience, all Ingress controller are CLBO, kube-dns and kubernetes-dashboard also :/06:57
Zicso I identify the step-to-reproduce in my cluster: rebooting the node which host the default-http-backend-0wt64 pod cause this kind of error06:59
Zichttp://paste.ubuntu.com/23879186/ <= kube-dns logs07:02
ZicKilling container with docker id e5f551e889bc: pod "kube-dns-3216771805-w2853_kube-system(854c0971-e4cc-11e6-b87d-0050569e741e)" container "dnsmasq" is unhealthy, it will be killed and re-created07:02
ZicLiveness probe failed: HTTP probe failed with statuscode: 50307:02
Zicif you have any reviews before I fill the issue on Kubernete's GitHub08:07
marcoceppi_Zic: Most of the k8s team is asleep, unfortunately08:21
marcoceppi_hell, I don't even know why I'm awake08:21
=== marcoceppi_ is now known as marcoceppi
junaidaliHi marcoceppi , you still there? :)09:18
marcoceppijunaidali: o/ yes09:18
junaidalimarcoceppi: have you got time to look into the issue?09:21
marcoceppijunaidali: which issue?09:21
junaidaliaws credentials09:21
marcoceppijunaidali: oh, jeeze, ofc09:22
marcoceppijunaidali: give me 10 mins and I'll be back with some more info09:22
Zicmarcoceppi: no problem, and it's the weekend btw :)09:51
ryebotZic: The debug action is a one-shot thing, no need to shut it off. Did your ingress controllers eventually come back up?14:05
Zicryebot: nope, and it is atm stuck in CLBO since this morning14:13
Zicsame for kube-dns and kubernetes-dashboard14:14
ryebotZic: Sorry about that. Please move forward with your bug. I'll see if I can repro your issue soon.14:15
ryebotZic: In the meantime, you might try destroying them all and seeing if they come back up (our charm should reinstate them automatically).14:17
=== hml_ is now known as hml
Zicryebot: about Ingress controller or also the kube-dns and kubernetes-dashboard pods?14:30
Zicryebot: (I did nothing for now) after 109 restarts (saw in the Restart column of a `kubectl get pods -o wide --all-namespaces`), all my Ingress and kubenertes-dashboard/kube-dns are running14:46
Zicbut I suspect if I reboot again some nodes it will stay in CLBO for hours :/14:46
ZicI'm preparing the issue for GitHub14:47
Zichttps://github.com/kubernetes/kubernetes/issues/40648 (cc ryebot lazyPower)15:02
ryebotZic: Excellent, thanks! We'll be tracking this.15:03
Zicsadly the default-http-backend pod have no logs :/15:05
Zic(kubectl logs on it return nothing)15:05
bdxhaving issues with `juju attach` ... it doesn't seem my resource ever makes it to the charm19:49
bdxeven though uploading to the controller is successful19:50
bdxmy charm just doesn't get the resource19:50
bdxI wonder if it has something to do with the hosted controller19:50
bdxin all honesty .... I can't upload resources to the charmstore, can't upload to the controller .... I guess resources are just broken right now?19:52
bdxthats the impression all my new uses have at least19:52

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!