redirperrito666: I think a little while back I had to kill and recreate my lxd bridge. I don't recall exactly why, but not being able to deploy to lxd seems familiar.00:18
perrito666redir: remember how to do that?00:38
redirperrito666: well that was last year00:42
redirI think something like `sudo ip link set lxdbr0 down && sudo ip link delete lxdbr0 type bridge` followed by `sudo lxd init`00:43
redirbut IIRC I prolly also had to remove all the lxd containers and images I had to init again00:44
redirI don't recall if there's an init only bridge command00:44
redirperrito666: ^00:44
redirperrito666: looks like there's a non-interactive option now -- `lxd init --auto`00:47
rediroh, and you prolly have to kick lxd with sysctl too00:48
redirerm, systemctl00:48
redirbabbageclunk: were you still reviewing my pr from last year?02:28
babbageclunkredir: oh sorry - I haven't picked it back up - have you solved the timing problem you were talking about in the meeting?02:29
rediryep, not really timing, just order of apt installs02:30
redirand updated the QA steps with a note about fixing a kvm breaking network change02:30
redirfor QA at least02:31
redirbut I need 2 +1s since is it over 500 lines02:31
babbageclunkredir: ok, taking another look02:32
redirbabbageclunk: much  obliged02:32
* redir goes afk for a bit. I'll check back later02:44
* redir is unclear about the failures on http://juju-ci.vapour.ws/job/github-check-merge-juju/485/04:15
redirif they look familiar to anyone let me know04:15
babbageclunkjam: could you take another look at https://github.com/juju/juju/pull/6735?04:22
babbageclunkjam: Also, happy new year!04:22
babbageclunkHmm, I'm confused - why is my irc client completing jam when he's not here?04:23
wallyworldbabbageclunk: ah, i just sent an email, thought you were EOD. when you start tomorrow, there's another PR to look at, sorry04:28
babbageclunkwallyworld: ok, will take a look soon - still going through redir's one at the moment!04:29
jambabbageclunk: I'm afk, but I have an IRC forwarder.04:29
jambabbageclunk: so technically I'm always here. :)04:29
wallyworldbabbageclunk: ty. once we get this landed and in the hands of the GUI guys, we may see some nice progress for next week hopefully04:30
* wallyworld needs to do a coffee run, state of emergency here, bbiab04:31
babbageclunkjam: Ah - I didn't see you in the list because you're an op!04:31
jamI am OP :)04:32
jamso technically I'm off this week, I'm only around because I'm sitting here doing my personal budgeting. but I can try and give it a look sometime04:32
jambabbageclunk: so what is DefaultCloudName() vs Cloud.Name? It isn't quite clear from the diff why you need both.04:34
jamis the latter the variable where you store the result of the former?04:34
babbageclunkjam: yes04:35
jambabbageclunk: I will fully admit to not doing as full of a review as last time, but LGTM04:36
babbageclunkjam: good enough for me! ;)04:36
babbageclunkjam: thanks - sorry to interrupt your holiday!04:36
mupBug #1653888 opened: juju-db service using 30GB+ memory <juju-core:New> <https://launchpad.net/bugs/1653888>07:09
=== frankban|afk is now known as frankban
=== deanman_ is now known as deanman
voidspacefrobware: macgreagoir: if you have a chance https://github.com/juju/juju/pull/676111:00
macgreagoirvoidspace: Testing it now.11:12
voidspacemacgreagoir: thanks11:21
macgreagoirvoidspace: LGTM, fwiw11:44
voidspacemacgreagoir: thanks11:44
voidspacemacgreagoir: I'm going to merge then11:45
macgreagoirvoidspace: ianagr :-) but OK by me.11:46
voidspacemacgreagoir: gah, you should be by now11:46
rick_hmacgreagoir: not a 'gr'?12:02
rick_hoh, reviewer12:02
perrito666does anyone know how to nuke the lxd bridge and re-create it?12:07
macgreagoirperrito666: Does `lxd init ...` give you what you need?12:12
perrito666macgreagoir: meh, it says I have containers, which I dont12:23
perrito666macgreagoir: but most likely its what I need12:23
macgreagoir`brctl delbr` et cetera12:26
voidspacerick_h: desparate for coffee - will be 2 mins late!12:27
perrito666macgreagoir: oh I get it, It can not just run init on networking, that is kind of sad :(12:29
perrito666man, apt-get purge has lost some of its touch13:45
natefinchperrito666: I have a vsphere question for you13:48
perrito666natefinch: oh, for me? you shouldn't have bothered13:49
natefinchperrito666: well, just trying to start off the new year on the right foot ;)13:49
natefinchperrito666: I am testing my provider Ping method, which is really just calling govmomi.NewClient with the given URL... but I am getting back 400 bad request for some reason13:50
perrito666natefinch: well shoot, in a moment ill need to go fetch my wife's bd cake13:50
natefinchperrito666: I'll try to be quick13:51
perrito666you might be doing a bad request??13:51
natefinchperrito666: also, happy birthday to your wife :)13:51
natefinchperrito666: is there a specific path I need to give the NewClient function other than https://<someip> ?  I'm running against the vsphere that QA uses, in oil13:51
* perrito666 checks code13:52
perrito666natefinch: first of all, remember you need to close that because they never time out13:53
natefinchperrito666: yeah, doing that13:53
natefinchperrito666: oh, hmm, looks like it might be /sdk13:53
perrito666look for providers/vsphere/client.go line 7913:54
perrito666I wonder if that is going to answer to you without credentials13:54
perrito666natefinch: on second thought, 40X is a good indicator of ping13:55
perrito666it means something is up there and can tell you "no"13:55
natefinchperrito666: well, almost anything will respond with 400 bad request, though13:55
perrito666true, I am not sure how accurate you need your ping to be13:56
natefinchperrito666: as accurate as possible.... with /sdk I get ServerFaultCode: Cannot complete login due to an incorrect user name or password.   which is probably good enough13:57
perrito666k bbl14:11
frobwaremacgreagoir, voidspace: updated https://github.com/juju/juju/pull/6758 with some unit tests. Would appreciate a look so that I can address any issues and try to get a CI run before tonight.14:36
frobwarerick_h: ^^14:36
* frobware bbiab14:37
mupBug #1651260 changed: landscape bundle error when deployed via gui (KeyError in config changed hook in haproxy charm) <matrix> <juju:New> <https://launchpad.net/bugs/1651260>14:55
mupBug #1651260 opened: landscape bundle error when deployed via gui (KeyError in config changed hook in haproxy charm) <matrix> <juju:New> <https://launchpad.net/bugs/1651260>15:07
mupBug #1651260 changed: landscape bundle error when deployed via gui (KeyError in config changed hook in haproxy charm) <matrix> <juju:New> <https://launchpad.net/bugs/1651260>15:10
macgreagoirvoidspace: If you have a wee sec, please: https://github.com/juju/juju/pull/6762/15:32
voidspacemacgreagoir: looks straightforward to me15:35
macgreagoir'Tis :-)15:36
voidspacemacgreagoir: LGTM15:37
* voidspace lunches15:38
natefinchrick_h: lol, so, I was looking at my old change, and I think I know why it's causing a failure now.... because I stopped throwing away the error that we get from lxd init15:58
rick_hnatefinch: heh, one step forward...15:58
natefinchrick_h: so I think we were always hitting this error, we just were logging it and ignoring it before.  The easy fix is to go back to ignoring it.  The harder fix is to try to only run lxd init if it's needed (or handle the "you already ran this, dummy!" error)15:59
natefinchprobably #3 is easiest - still always run lxd init, but check for the "you already ran it" error and ignore just that one.  That way if it fails the first time, we actually fail early, rather than try to continue on even though there's no lxd running16:01
frobwarenatefinch: Logging an error causes too many false positives, IMO. I've seen bug reports / mail / IRC saying, "lxd is broken, it reports an error, see this error in the logs..."16:02
natefinchfrobware: right, my point is, let's not log the error in this known case, since it's basically like the error when you try to create a file that already exists.16:03
natefinchfrobware: but for other unknown errors, let's actually return them, not just log them and effectively ignore them16:04
frobwaremacgreagoir: ping16:06
macgreagoirfrobware: pong16:06
frobwaremacgreagoir: do you have some time to look through https://github.com/juju/juju/pull/675816:07
macgreagoirfrobware: I have it loaded, just kicking a couple of other things first, sorry.16:07
frobwaremacgreagoir: ty16:08
redirhappy wednesday juju-dev16:32
frobwareredir: only 2 to go. :)16:42
macgreagoirG'day redir16:47
macgreagoirfrobware: Review and comment. No show-stopper.16:57
frobwaremacgreagoir: wall clock in tests...16:57
macgreagoirI have one like that and I'd like to see you solve it :-D16:58
frobwaremacgreagoir: ah, you found a bug.16:59
frobwaremacgreagoir: all the tests pass 0 for a timeout, which means indefinite, and therefore no timer/clock is used.17:05
frobwaremacgreagoir: there is one test that checks for timeout and passes 1s.17:06
frobwaremacgreagoir: because of this I chose not to use the testing clock.17:06
macgreagoirfrobware: Maybe worth a comment so we don't scoop it up next time there's a testing sprint?17:06
frobwaremacgreagoir: done17:12
frobwarevoidspace: you about?17:14
voidspacefrobware: yep17:14
macgreagoirfrobware: acked in PR17:14
frobwarevoidspace: could you also take a look over https://github.com/juju/juju/pull/675817:14
voidspacefrobware: yep17:14
voidspacecoffee first though17:15
frobwaremacgreagoir: you've gone all recursive on me.17:15
redirsinzui: yt?17:18
sinzuiredir: yes17:18
redirhappy new year:)17:19
rick_hnatefinch: is it not something that we can check if lxd is configured and fail fast that way? I guess what's fail here? I mean it it's been run you're good and if it's not been run you need to run it.17:19
redirsinzui: you wrote me a while back about a power8 host. Can I use that for live testing kvm? Are the details in the usual place?17:20
natefinchrick_h: maybe there's a lxd command we can run that'll tell us if it is initialized?  I definitely don't want to write any heuristics outside of what lxd itself tells us though17:21
sinzuiredir: I don't think I can help. QA has ppc64el guests. I am not sure we can run kvm on them.17:21
redirsinzui: OK. Good to know, I won't spend time trying17:22
sinzuiredir: well maybe. this can be done because I know a bit about borbein and its vms17:22
* redir backs up17:22
natefinchtych0: is there an lxd command that we can run that'll tell us if we need to run lxd init?17:23
rick_hnatefinch: I guess the error you're getting *is* telling you that in some sense17:23
sinzuiredir: I can get you on to a Juju QA host and on to a charm testing host. brobein has a special kernal to support libvirt. Juju wont just work because the ubuntu kernel needs to be special17:23
natefinchrick_h: yeah, just wondering if there's a better way.  Ideally something that doesn't require string matching on the lxd output for a specific error message17:24
sinzuiredir: So borbein is your best chance to verify that juju can drive kvm, but surely you need a cloud that is ppc64el with the right kernel to test that juju can deploy kvm containers to an instance17:26
* sinzui writes email with instructions.17:27
redirsinzui: yeah that sounds right -- that I need both. I just had a note to ask when it came timet to live test on a different arch. I'll just work with the arm and ppc folks where they can test it.17:28
redirsinzui: the code wants same host/arch to work17:28
sinzuiredir: I understand. QA has been asking for proper ppc64el resources for more than a year17:29
sinzuiredir: arm is a different, but also painful story17:29
redirsinzui: tyvm17:30
sinzuiredir: We have a powerful arm64 host, but have never gotten libvirt working to build a vmaas. I really want to do this, but qemu/libvirt don't like the arch17:31
frobwareThe GIL is gone: https://opensource.googleblog.com/2017/01/grumpy-go-running-python.html17:37
frobwareAnd C extensions too. But...17:37
perrito666frobware: lol, yeah, I think without C extensions its a bit useless though17:43
perrito666isnt ssl in python a C extension?17:43
frobwareperrito666: it cuts out a LOT OF STUFF17:43
perrito666frobware: I presume its a "we just dont want to re-write all this code base in go"17:44
frobwareperrito666: I think it just shows that if you are running your own stuff on your own infra you can do whatever makes sense _for_ _you_17:44
natefinchahh, it's a transpiler... interesting17:45
frobwarevoidspace: any feedback. I need to EOD soon and want to merge before we scream "RC1 - ship it!"17:46
voidspacefrobware: looks good to me I think17:46
frobwarevoidspace: when we rewrite the script in Go (^^) I think a lot of the testing friction will begin to disappear w.r.t. the bridge.py.17:47
voidspaceI like the scriptrunner17:47
voidspacefrobware: ok17:47
voidspacefrobware: I struggle to believe that Go is *really* easier to test ;-)17:47
voidspacefrobware: but possibly17:47
voidspacethe impedance mismatch goes17:47
frobwarevoidspace: I think it's still the fact the script "does it all". Some of the invocation is in python, wrapped by Bash, invoked from Go.17:48
frobwarevoidspace: I would like to not expose dry-run too. :/17:53
voidspacefrobware: yeah, if possible17:53
frobwarevoidspace:  that is purely an implementation detail.17:54
voidspacefrobware: anyway, LGTM17:54
frobwarevoidspace: I'm happy to follow-up, but I think it's important to know get a CI test run on this _feature_ branch.17:54
voidspacefrobware: yep, cool17:54
voidspacefrobware: not sure if I'll get mine in today17:54
frobwarevoidspace: want a review or something else?17:55
voidspacefrobware: no, still trying to find the bug17:55
voidspacefrobware: or at least find whereabouts user config are passed into lxd17:55
voidspacefrobware: just getting back to it really, I'll give it a couple of hours or so before EOD17:56
voidspacebe nice to find itt17:56
voidspacefrobware: not sure17:59
voidspaceI don't think so - or at least I'd like to see all the layers17:59
perrito666we should get  a free day every time we change something in the output of status18:13
=== frankban is now known as frankban|afk
* perrito666 considers hacking his desktop to support a 3rd monitor20:09
natefinchdoes it not?  My laptop supports integrated screen + 2 external20:18
perrito666natefinch: the furniture I meant :p20:22
perrito666the computer card supports like 8 screens20:22
natefinchperrito666: lol,20:30
natefinchrick_h: should this fix go to develop?20:47
natefinchrick_h: for the LXD init thing?20:47
rick_hnatefinch: into the 2.1-dynamic-bridges branch and the 2.1 and develop20:47
rick_hnatefinch: actually just the 2.1-dynamic-bridges and develop20:47
rick_hthe 2.1 will get merged in20:47
natefinchhttps://github.com/juju/juju/pull/6764 and https://github.com/juju/juju/pull/676520:52
alexisbbabbageclunk, perrito666 is one of you around20:53
perrito666alexisb: I am20:53
babbageclunkalexisb: yup20:53
babbageclunkalexisb: perrito666 is.20:53
alexisbbabbageclunk, rick_h has a request20:53
rick_hbabbageclunk: can you please look gat ^20:53
rick_hlook at that is20:53
rick_hbabbageclunk: and help natefinch get his PR landed20:53
perrito666babbageclunk: I can do it so your day start is not so rough20:54
babbageclunkrick_h: yup yup20:54
babbageclunkperrito666: ok, thanks20:54
rick_hnatefinch: can you put a bit more background in that PR description please?20:54
perrito666natefinch: that is 6764 and 6765 right?20:55
rick_hperrito666: correcty20:57
natefinchQA steps and description added20:58
rick_hty natefinch20:59
perrito666natefinch: interesting "sudo dpkg-reconfigure -p medium lxd" <-- that error by lxd is also a lie21:01
perrito666natefinch: ship them21:05
rick_hnatefinch: ty, balloons ^ once that hits we can start a full test run of the feature branch please21:21
perrito666since we are in it, could anyone review this? https://github.com/juju/juju/pull/676321:27
natefinchperrito666: looking21:30
natefinchperrito666: why isn't this using model status21:32
natefinchperrito666: surely controller upgrading should cause the model to become busy until it's finished21:32
perrito666natefinch: you would think21:32
perrito666but no, status actually answers even while controller is upgrading21:33
perrito666I believe this patch will make us discover a whole new world of bugs where juju answers its upgrading when its not21:33
natefinchright but, what I mean is, the model-status is going to still say "available" when it's really not21:34
natefinchi.e. you can't juju deploy something while the controller is upgrading21:35
perrito666natefinch: interesting, the rules gouvering statuses are a bit heavier than this21:37
natefinchperrito666: I'm not sure that that means21:39
perrito666natefinch: there is a logic behind when each status is set21:41
perrito666and we dont have a status that correctly reflects "upgrading" afaik nor think to add it, also adding a status might not be backwards nice21:41
natefinchperrito666: model-status is the correct place to put this.  We only recently created it, so there's no backward compatibility problem21:42
natefinchperrito666: that may be why the powers that be were thinking of just adding a boolean rather than using the status we already have in place.21:42
natefinchrick_h: ^21:42
* rick_h reads back21:43
perrito666natefinch: nope, there was not much thought behind it21:43
natefinchrick_h: basically... we added a new boolean to Model status, but probably should just reuse the relatively new model-status field21:44
rick_hnatefinch: perrito666 +1 to using the field. We don't want to make decisions based on checknig several different fields21:44
natefinchre: https://github.com/juju/juju/pull/676321:44
rick_hnatefinch: perrito666 adding a new "is-controller-doing-X" isn't ideal21:44
perrito666mm, I think I agree with both of you21:47
perrito666rick_h: what do you reckon is the status we should set? I presume I should be adding an "upgrading" message too21:48
alexisbdoes model-status only show up in yaml?21:49
rick_hnatefinch: do you have a link to your PR from the sprint?21:51
natefinchrick_h: I can get it21:51
natefinchrick_h: https://github.com/juju/juju/pull/666121:51
perrito666natefinch: that seems to also work on json right?21:52
natefinchalexisb: it's shown for juju models and juju status --format=yaml/json21:52
rick_hperrito666: ^ shows how the model status was updated to be able to be used generically and might help point to where we're thinking.21:52
rick_hperrito666: right, so the idea is that during an actual upgrade we'd update that as "upgrading" and so it could be "migrating" or "upgrading" or ...21:53
natefinchrick_h, perrito666: gotta run for dinner.... but I recommend Status: status.Busy and StatusInfo: "controller is upgrading"22:01
perrito666sounds like a fair start22:02
rick_hty natefinch22:02
perrito666rick_h: ok there is some room for thought there but I believe that the rule should be something like if status != error && upgrading return  Status: status.Busy and StatusInfo: "controller is upgrading"22:02
rick_hperrito666: k22:03
perrito666I would not Set status for upgrading because we risk having other statuses shadowing that in the future22:03
perrito666sorry for rubber ducking you btw22:03
rick_hhmmm, yea thinking22:03
rick_hso...you're marking it busy though right?22:04
perrito666yes, but I am sort of hijacking that model.Status call and returning a status that is not the one set because the fact that an upgrade is in progress superseeds whatever model think its doing, makes sense?22:07
perrito666rick_h: ^22:07
rick_hperrito666: well, not really. I mean can the model be migrating and upgrading?22:07
rick_hperrito666: I'm concerned that going that route leaves us open to a non-finite state machine of what is going on22:08
perrito666should not22:08
perrito666I mean upgrade a non concurrent task, nothing else should be happening with upgrade22:08
rick_hperrito666: so I'd rather that it was actually set and that other code could rely on that information. e.g. a requested migration call should check that it's available before migrating and it then sets migrating?22:08
rick_hperrito666: and the same is true of upgrading, a requested upgrade would fail if not available and if it starts it updates status and is tracked with "since..." and such22:09
rick_hperrito666: like any other long running activity like that?22:09
perrito666rick_h: if you request anything that makes a change on juju while upgrading juju will say no22:09
perrito666not very politely22:09
rick_hperrito666: right, but by setting a firm status we're allow juju to get more polite?22:09
perrito666rick_h: we are, in this case we should set this status for all models before ugprading22:10
rick_hperrito666: and without tracking the actual change by setting status we don't have real visiblity as to when it started/etc like we do the migration? I guess it feels like we're doing this one different than migration/etc without something I see as really different22:10
perrito666then set.... mm I am not sure what to set after finishing22:10
rick_hperrito666: but you can upgrade a model at a time22:10
rick_hperrito666: only some models may be upgrading22:10
rick_hperrito666: available?22:10
rick_hsince nothing else can take that while you own it22:10
rick_hperrito666: I understand where you're coming from, I'm asking for a bigger promise that Juju works22:11
perrito666rick_h: I am thinking in the not so long term, I mean if the model was in some other status and I mark upgrading I would be loosing that status right?22:11
perrito666also, upgrade cannot happen for a model only, can it?22:11
perrito666In a regular juju I mean22:11
perrito666in a regular juju install upgrading upgrades the controller binary and the mongodb for all models in that controller22:12
rick_hperrito666: yes, because the controller is upgraded on it's own, then the model is migrated at the same version it was, and then can be upgraded I thought22:13
rick_hperrito666: but the whole rule of "the model can't be on a version > than the controller" and migrations and all this comes to play22:13
voidspacerick_h: I'm going to have to rethink my approach to bug #163125423:00
mupBug #1631254: [2.0rc3] lxd containers do not autostart <rteam> <juju:In Progress by mfoord> <https://launchpad.net/bugs/1631254>23:00
voidspacerick_h: however, given that the way to reproduce it is now to *manually* stop the container23:00
voidspacerick_h: as the "hard shutdown" failure mode has been fixed in lxd23:00
voidspacerick_h: I think it can be demoted from critical and need not hold up 2.1.023:00
voidspacerick_h: ok, for me to change it from critical to high?23:00
voidspacerick_h: (basically we need to keep the user namespace and special case the ones that don't need it I think - which is the opposite way to how I was doing it)23:01
voidspacerick_h: I haven't changed it from critical to high myself yet, and I'm now EOD (11pm) - g'night23:03
mupBug #1492237 opened: juju state server mongod uses too much disk space <canonical-bootstack> <mongodb> <oil-2.0> <uosci> <juju:Triaged> <juju-core:New> <https://launchpad.net/bugs/1492237>23:06
mupBug #1634390 opened: jujud services not starting after reboot when /var is on separate partition  <uosci> <juju:Triaged by rharding> <juju-core:New> <https://launchpad.net/bugs/1634390>23:06

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!