[00:21] <axw> wallyworld: https://github.com/juju/docs/issues/1665
[00:39] <perrito666> externalreality: hey, feel free to ping me during the day if you need anything ill be during the day on GMT-3
[01:03] <menn0> axw, wallyworld or thumper: can I bounce some ideas of one or more of you regarding CAAS?
[01:03] <axw> menn0: sure, can you give me 5 mins please?
[01:03] <wallyworld> sure
[01:03] <axw> just finishing a review
[01:03] <menn0> axw, wallyworld: cheers. i'll set up a hangout
[01:05] <menn0> axw, wallyworld: https://hangouts.google.com/hangouts/_/zknvh6oslvc57oppedq6tkfpqae
[01:06] <blahdeblah> Such a cool hangout name
[01:06] <blahdeblah> I should make one like it
[01:06] <blahdeblah> I think I shall name my hangout fee3och3AiwuNoquohvoo9chohhohche
[01:31] <thumper> menn0, babbageclunk: I'm available now
[01:31] <thumper> babbageclunk: https://hangouts.google.com/hangouts/_/canonical.com/bang-bang-chitty-chitty-bang-bang
[01:31] <menn0> thumper: in call above
[01:32] <thumper> menn0: ok, do you want / need me?
[01:32] <externalreality> perrito666 thank you, will reach out if have anything :-D
[02:01] <menn0> thumper: just finished all good
[02:01] <thumper> ack
[03:17] <axw> wallyworld: I'm a bit concerned about the performance of that remote relations watcher. apparently those things block the entire watcher infrastructure, so need to keep it as lean as possible
[03:18] <axw> wallyworld: can we do a join of sorts on the applications/endpoints collections?
[03:18] <wallyworld> axw: yeah agreed. i can't see  way to do it without a db query though
[03:18] <axw> wallyworld: understood, just trying to keep the number of queries down
[03:18] <wallyworld> i got it doen to 2 from 3
[03:19] <axw> can we get it to 1 :)
[03:19] <wallyworld> without a join?
[03:19] <axw> wallyworld: with a join. is there an aggregation operator we can use.. not sure
[03:19] <wallyworld> if it were sql it would be easy
[03:19] <wallyworld> i didn't think mgo supported joins
[03:20] <axw> wallyworld: apparently mongo 3.2 does: https://www.mongodb.com/blog/post/joins-and-other-aggregation-enhancements-coming-in-mongodb-3-2-part-1-of-3-introduction
[03:20] <wallyworld> ooh, ok
[03:21] <axw> wallyworld: not sure if it's worthwhile, maybe not
[03:21] <wallyworld> hopefully mango will play nice then
[03:21] <wallyworld> i'll have a look
[03:25] <axw> wallyworld: another option would be to get all of the relation entries first, and add/delete to that collection in-memory as the watcher fires
[03:26] <axw> wallyworld: that way you only need to query the remote applications collection inside the filter
[03:26] <axw> I think that would be neater
[03:26] <axw> seeing as we're already watching the relations collection
[03:26] <wallyworld> what about scale, if there are 1000 of them. i guess it won't take too much memory
[03:26] <wallyworld> or 10000 etc
[03:27] <wallyworld> also when the watcher fires, we'd still need to look up the relation doc from the id
[03:28] <wallyworld> maybe i'll see if the join works and go from there
[03:31] <axw> wallyworld: don't spend too much time on it, I'm probably worrying about nothing
[03:31] <wallyworld> axw: i'll just finish this gui thing i'm doing and will see if i can get it to one query
[03:32] <axw> ok
[04:15] <axw> thumper: is it known that the depengine report doesn't show model engines? so I can't see the provisioners, for example
[04:15] <axw> unless I'm missing a flag or something...
[04:22] <menn0> thumper: axw: the model engines are different (child) engines. maybe the report on asks the root engine
[04:24] <axw> menn0: ah, it'll be bundled up in the state workers I guess
[04:24] <axw> I think we should probably report on all the model engines too. or at least have a flag to specify a model
[04:25] <menn0> axw: don't think the provisioner is bundled in the state workers
[04:25] <menn0> axw: it's in the per model engines
[04:26] <axw> menn0: the model engine is bundled into it. the provisioner is in the model engine
[04:27] <axw> menn0: model engine runs under the "model worker manager" worker, which gets added to the runner returned by MachineAgent.startStateWorkers
[04:27] <axw> anyway, I think we could weave it in there somehow. a job for another day
[04:30] <menn0> axw: yes, it would certainly be useful to see
[04:55] <axw> jam: can we defer till quarter past please, still eating lunch
[04:55] <jam> axw: sure
[05:13] <axw> jam: I'm there now
[05:18] <wallyworld> axw: after your meeting, here's a GUI related PR https://github.com/juju/juju/pull/6992
[06:09] <axw> wallyworld: looking
[06:09] <wallyworld> ty
[06:17] <axw> wallyworld: why did the base URL path need to change?
[06:17] <wallyworld> because otherwise the gui would construct urls with the model path duplicated
[06:18] <wallyworld> ip:17070/gui/u/admin/controller/u/admin/controller
[06:18] <axw> wallyworld: is that due to a change in the GUI code?
[06:19] <wallyworld> i'm running with stock 2.3.0 but i think there's an upstream change to do with the new url paths
[06:19] <wallyworld> i've asked jeff and co to look at the pr also
[06:24] <axw> wallyworld: reviewed. it looks good but I don't understand why the base URL pat change was necessary
[06:25] <wallyworld> because it seems the gui appends /u/user/model to the base URL. the gui is expecting the base URL to be IP:17070/gui/modeluuid but the gui doesn't realise that juju ignore everything after $uuid
[06:26] <wallyworld> with juju now able to handle gui urls with the model path, any bits after /gui/ are not needed
[06:26] <wallyworld> as the gui appends the model path
[06:27] <anastasiamac> wallyworld: Huw is in out time zone if u need fast turnaround
[06:27] <anastasiamac> our*
[06:27] <wallyworld> he is EOD
[06:27] <wallyworld> but jeff and co aren't :-)
[06:30] <anastasiamac> wallyworld: ^^ here is huwshimi \o/
[06:30] <wallyworld> it's ok, ihave already been talking to others
[06:31] <wallyworld> who are more immediately acros the issue
[07:24] <wallyworld> axw: i've done some tests and sadly mongo joins ($lookup) don't work with dotted field names eg "endpoints.applicationname". i can only get it to work matching simple scalar fields. so looks like it has to be 2 queries for now
[07:24] <axw> wallyworld: no worries, thanks for checking
[07:24] <wallyworld> was interesting to try
[07:27] <wallyworld> axw: i found out those gui changes with regards to the model path in the url are as a result of wanting to use consistent urls for juju and jaas. the gui work landed in gui version 2.3.0. so the fact that juju will now support those paths is actually a very good thing
[07:28] <axw> wallyworld: does it break older GUI versions tho?
[07:28] <axw> wallyworld: if you upgrade  the controller without upgrading the GUI, is the GUI broken all of a sudden?
[07:29] <jam> axw: have you had conversations about how the storage changes for 2.2 will be reflected with bundles?
[07:30] <axw> jam: no. what do the persistent storage changes have to do with bundles?
[07:30] <wallyworld> axw: it suports both old and new clients (both ways), but there's a small glitch that showed up in my testing with old 2.0 cliesnts. the gui guys are looking at it and we'll fix. it could be upstream in the gui, not sure
[07:31] <wallyworld> they'll help root cause it and then we can determine where the fix is needed
[07:31] <axw> wallyworld: I saw the old/new juju clients work. but if the controller is serving GUI version 2.2 (or whatever preceded (2.3.0), will it work?
[07:31] <wallyworld> it's only  a small issue with login
[07:32] <wallyworld> yes because the juju gui CLI probes the controller to see what form of the url to use
[07:32] <axw> wallyworld: I'm not talking about the URL the user plugs into the browser
[07:32] <axw> wallyworld: you changed the baseURLPath from /gui/uuid to /gui
[07:32] <axw> wallyworld: that affects what config gets passed to the GUI
[07:33] <axw> wallyworld: will the old version of the GUI be unhappy with the change?
[07:33] <wallyworld> right but the gui is downloaded with the controller
[07:33] <wallyworld> so new 2.1 controllers will be running gui > 2.3.0
[07:33] <axw> wallyworld: what if you do upgrade-juju?
[07:33] <axw> from 2.0
[07:33] <axw> or 2.1-beta5 for that matter
[07:34] <wallyworld> i'll need to check to be sure
[07:34] <wallyworld> but they can upgrade-gui also
[07:34] <wallyworld> we may need to add some doc to release notes
[07:34] <wallyworld> as it was, gui 2.3.0 was a bit broken with juju 2.1
[07:35] <axw> wallyworld: yeah, I just don't want there to be a surprise
[07:35] <axw> ok
[07:35] <wallyworld> so either way we have potential breakage
[07:35] <wallyworld> agreed, i'll talk to uros and co
[07:36] <jam> axw: so in all my recent discussions, the only way people are really deploying things is via bundles, at least for many people they are "the way". Which just makes me want to raise the question of how storage interacts with it
[07:36] <jam> things like "deploy gets extra flags", sounds like something we may need to introduce into the bundle stanzas
[07:36] <jam> I'm trying to brush up on the topic before bringing it up, and just thinking about edges
[07:37] <axw> jam: you can already do "juju deploy bundle --storage app0:store=foo
[07:37] <axw> jam: is that what you mean?
[07:37] <jam> axw: so conceptually things continue to use that syntax for --storage-filesystems, etc.
[07:38] <jam> I'm a little concerned that it isn't very clear that '--storage ...' means allocate new storage vs "--storage-filesystems" means reuse an existing one.
[07:38] <axw> jam: hmm yeah I guess we'll have to support that too. hadn't considered that
[07:38] <jam> axw: also, it feels like you still need to use '--storage' as part of deploy at least, since that's declaring the shape of the storage for future units
[07:39] <jam> (add-unit with no flags)
[07:39] <axw> jam: yes, that is correct
[07:39] <jam> axw: this sort of thing makes me wonder if we would have been better splitting deploy, or at least introducing a "declare an application but don't give it units yet"
[07:40] <jam> probably not, as what a user really wants is a running application, but it would help with some of the "am I talking about the unit that is being brought up, or about the overall application"
[07:41] <axw> jam: it would make it clearer for the latter case, but yeah, probably don't want to push that onto everyone
[07:41] <jam> axw: is there a reason we have to think about "volumes" vs "filesystems" ? Is that actually interesting for the operator?
[07:42] <axw> jam: I think so, because you can't assign a filesystem to a block-type store
[07:42] <jam> axw: so that's a true statement, but is that a fundamental flag that you have to supply all the time/
[07:42] <jam> ?
[07:43] <jam> vs "juju deploy --attach-storage foo=bar" and getting a "bar is a volume not a filesystem, see "juju storage --filesystems"
[07:43] <jam> or --reuse-storage
[07:44] <axw> jam: the problem is that the volumes and filesystems just have numeric IDs
[07:44] <jam> axw: sure, but we can pull them from the same integer counter, couldn't we?
[07:44] <jam> as in its just 2 vs 3, but it is still just an arbitrary integer
[07:45] <jam> axw: and fwiw, I have a strong feeling that people are really going to want to be naming these things
[07:45] <axw> jam: I suppose so. that doesn't seem like better UX to me. it's not obvious on the command line what the ID is
[07:45] <jam> axw: which is a point for not using just IDs
[07:45] <jam> cause they don't mean anything to anyone
[07:45] <jam> volume 10 isn't particularly more descriptive than just 10
[07:45] <jam> but "production-db-disk" would be
[07:46] <axw> jam: volume-10 tells me what it is, but not what it's used for. it's not nothing, but it's not perfect either
[07:46] <axw> jam: I think people have asked for the ability to name things already
[07:46] <jam> axw: well you're propsed syntax is --storage-volume foo=10
[07:47] <jam> which decouples the "volume" from the 10 by a pretty wide margin
[07:47] <jam> and doing
[07:47] <jam> --attach-storage foo=volume-10
[07:47] <jam> seems better if it is *just* that
[07:48] <jam> axw: on 'list-volumes" (whatever the specific name), how do you figure out what '10' was?
[07:48] <jam> we include information about what the last unit to mount it, and for what named store of that unit?
[07:48] <axw> jam: it gives you the name of the machine attached to, the location, the storage instance
[07:48] <jam> axw: the output of "juju storage --filesystem --format=yaml" seems particularly devoid of information that a human needs to know in order to manage that volume
[07:49] <jam> current: detached
[07:51] <jam> axw: sorry I'm bringing this up now, just trying to page it in if I'm going to be discussing it and it brought up things I hadn't thought of before
[07:53] <axw> jam: there should be more than just "current: detached". that's just the status of the volume. but you're right that we do not show the *last* machine that it was attached to
[07:53] <jam> axw: well last unit+store is probably the most useful
[07:53] <jam> 'this is what the volume was being used for'
[07:54] <axw> jam: that is not available in the model right now. we would need to add it in. might be a bit awkward...
[07:54] <axw> status history would be straight forward, but won't show up in the listing
[07:56] <jam> i think fundamentally 'name' is a useful approximation that also lets users set whatever they want, but I can't help but feel if we just do persistent-by-default
[07:56] <jam> people are going to end up with a lot of volumes that are meaningless to them
[07:56] <jam> sure they're still around
[07:56] <jam> but which one is the critical db
[07:56] <jam> and which ones were the throwaway disks that just didn't get thrown away
[07:57] <jam> $ juju remove-application postgresql
[07:57] <jam> Persistent filesystem “0” remains. You can use “juju remove-filesystem” to remove it.
[07:58] <jam> axw: that is in the right direction, but that line only exists in the terminal for the time that you're looking at it, and then that information is lost.
[07:58] <axw> jam: yep, good point
[07:58] <jam> axw: so I think you and I need a round of talking about this before we bring it to mark. sorry I didn't get a chance to look deeply at it before.
[07:58] <axw> jam: np
[08:16] <wallyworld> axw: i'm off to soccer for a bit - can you see what needs doing to get fallback clouds in 2.1 up to date. you may need to pull in reed's gce changes from develop that landed recently and add to your PR. and then try and land and see if they've removed the block
[08:16] <axw> wallyworld: ok
[08:16] <wallyworld> tyvm
[08:31] <frankban> hey wallyworld thanks for that branch! looking
[09:37] <frankban> wallyworld: https://github.com/juju/juju/pull/6992 reviewed
[11:01] <axw> jam: here's the LXD changes, if you have any time to review: https://github.com/juju/juju/pull/6994
[11:11] <axw> wallyworld: still can't land clouds.yaml changes FYI. I backported the GCE changes, will come back to see if aaron is around later to find out what's the go
[11:16] <jam> axw: thx, looking
[11:42] <wallyworld> frankban: thanks for review, just got back from soccer, will start making the changes
[11:43] <frankban> wallyworld: cool thanks
[11:43] <wallyworld> i didn't reallu know what i was doing in that code :-)
[11:44] <frankban> wallyworld: well, you did incredibly well then ;-)
[11:45] <wallyworld> beginner's luck :-)
[11:45] <wallyworld> i tested it a fair bit
[11:47] <wallyworld> frankban: with the password comment - do you mean the output of $juju gui where it prints the login credential?
[11:47] <wallyworld> i don't quite follow wht the issue is
[11:48] <frankban> wallyworld: yes, after you "juju change-user-password" you'll see that the paswd is empty
[11:48] <wallyworld> ok, i'll comeup with something
[11:57] <axw> wallyworld: I'm thinking we could have "juju gui" generate a single-use, time-limited password, and encode it into the URL
[11:58] <axw> wallyworld: but not for 2.1... too late now
[11:58] <wallyworld> coukd do but i won't have time tonight
[12:06] <frankban> axw: I think, on the long period, we'll want to support macaroon's based auth for that
[12:14] <axw> frankban: how do you get the cookie from the CLI to the browser?
[13:20] <jam> axw: reviewed lxd, overall lgtm
[13:30] <wallyworld> axw: not sure of exact words to put when password is changed and the CLI uses macaroon based login. i don't think the gui even supports that yet
[13:31] <wallyworld> ISTM that is you change password you can no longer access the gui
[13:49] <frankban|afk> wallyworld: no, I do that all the time
[13:49] <wallyworld> frankban|afk: I tried and it gui would not log me in at all
[13:49] <frankban|afk> wallyworld: let me try your branch with the new changes
[13:51] <axw> wallyworld: the only difference is that the password is not on disk any more. you should still be able to log into the GUI (I've also done it before) using the password you gave in change-user-password
[13:51] <axw> jam: thanks
[13:52] <wallyworld> axw: exactly. that's what i tried. i may have just mis remembered what i changed it to. but it was only 3 letters
[13:55] <frankban> wallyworld: I just tried and I was able to log into the GUI, after changing the password, with your branch
[13:55] <wallyworld> frankban: ok, maybe i'm just tired and mistyped :-)
[13:55] <wallyworld> you happy for me to land it?
[13:56] <wallyworld> i tested the old model uuid urls and they work now
[13:56] <frankban> wallyworld: no, the base URL Juju is providing is now wrong: /gui/b5d3e307-41f5-4131-8f71-40a698c4c381
[13:56] <frankban> wallyworld: that should be the one provided only if you manually visit the UUID url
[13:57] <wallyworld> right. i may have made a mistake, let me retest
[13:58] <frankban> wallyworld: maybe that's because the config file is still served by a URL including the uuid
[13:58] <wallyworld> frankban: you talking about the "base" attribute of config.js? that should be correct
[13:58] <wallyworld> yes. i keep serving static content from the model uui url
[13:58] <wallyworld> and config.js
[13:59] <frankban> wallyworld: it's not
[13:59] <wallyworld> ok, let me check
[14:01] <frankban> wallyworld: yes, config.js is still served with the uuid url, but it provides the wrong base
[14:01] <wallyworld> ok, i'll run up a system and look again
[14:04] <frankban> wallyworld: maybe because, even if the index path does not include the uuid, the config.js path always does, so when baseGUIURLPath is calculated it's always the old one, icluding the uuid?
[14:04] <wallyworld> not sure, the TestGUIConfigNewURL unit test seems correct to me
[14:05] <wallyworld> also the logic starting line 147 of apiserver/gui.go seems ok
[14:20] <wallyworld> frankban: yeah, it seems to be getting confused, because of the url used to serve config.js rather than the url used for the gui itself
[14:22] <frankban> wallyworld: yeah, one request for the index, and the one for the config is always a uuid path
[14:22] <wallyworld> and it's the one for the config that is used to set the base attribute
[14:22] <frankban> wallyworld: yes
[14:23] <wallyworld> damn, will need to think of a fix
[14:23] <wallyworld> but it's all stateless
[14:23] <frankban> wallyworld: but the one for the index is used to set the configURL, so maybe the configURL can include a hint?
[14:23] <wallyworld> maybe yeah
[14:23] <frankban> wallyworld: like a query (?uuid=true) in case of old-style index
[14:24] <wallyworld> i'll try
[15:10] <wallyworld> frankban: i think it's god now, seems to test ok for me
[15:10] <wallyworld> good
[15:10] <frankban> wallyworld: trying again
[15:13] <frankban> wallyworld: bootstrapping now
[15:13] <wallyworld> ok
[15:14] <wallyworld> frankban: hope it works cause it's after 1am and i'm tired
[15:14] <frankban> wallyworld: yeah, I was wondering if you were a vampire
[15:14] <wallyworld> i can be if you want me to be
[15:15] <frankban> lol
[15:21] <frankban> wallyworld: ok, new GUI new URL works well, with direct URL with uuid it works well, with the new GUI it's just wonderful
[15:21] <frankban> wallyworld: the I downgraded to GUI 2.2.7
[15:22] <wallyworld> uh oh
[15:23] <frankban> wallyworld: I think we are hitting a pat bug there, after logging in the GUI acts weird and then it redirects to "/gui/:modeluuid/" (literally, not a placeholder)
[15:23] <frankban> wallyworld: IIRC with pat all URLs must end with a /
[15:24] <wallyworld> frankban: so it's just new controller, old gui that is abroken?
[15:24] <frankban> wallyworld: yes, and therefore we could fix that in a follow up 2.1.1 probably
[15:24] <wallyworld> sgtm. i will even fix tomorrow
[15:24] <wallyworld> first up before release is cut
[15:25] <frankban> wallyworld: also, with uuid URL old gui works
[15:25] <frankban> wallyworld: so the only thing slightly broken is: old gui + new URL
[15:25] <wallyworld> yeah ok, that will be an easy fix. that's the one thing i didn't test :-)
[15:26] <wallyworld> you ok for me the $$merge$$ then?
[15:26] <frankban> wallyworld: yes!
[15:26] <wallyworld> and can you watch it for me in case it fails and try again?
[15:26] <frankban> wallyworld: sure, thanks a lot!
[15:26] <wallyworld> thanks for helping test etc
[15:27] <wallyworld> i learnt a bit today
[15:27] <wallyworld> 2.1 will be sweet with these new gui urls
[15:28] <rick_h> wallyworld: is sinzui aware of ^ then for 2.1?
[15:28] <wallyworld> not yet. the old urls still work
[15:28] <sinzui> wallyworld: I sort of know...I read the commit
[15:28] <sinzui> yes, the old urls do work
[15:29] <wallyworld> rick_h: i sort of stumbled into this work not knowing what i was getting myself into :-) it started as a small fix to juju gui command and sort of grew as a result of a required text change that implied juju core was meant to serve different urls
[15:30] <rick_h> wallyworld: like all good deeds :) punishment
[15:30] <wallyworld> indeed
[18:00] <perrito666> people I need to leave for about 1 to 2 hs, you can mail me if anything requires my attention
[18:40] <tych0> hey guys. what branch do i make PRs against these days?
[18:47] <alexisb> tych0, what are you working on?
[18:48] <tych0> alexisb: we have a new storage API in LXD 2.9, i have some patches that handle it
[18:48] <tych0> this will be zesty onward, and doesn't need to be backported to trusty
[18:49] <alexisb> ok, tych0 start with devel
[18:49] <tych0> sounds good
[18:50] <tych0> is that "develop"?
[18:50] <alexisb> you will want to sent a note to thumper and wallyworld to see if they want it in 2.1
[18:50] <alexisb> yes sorry develop
[18:51] <tych0> alexisb: cool, thanks!
[18:51] <alexisb> you bet tych0
[19:07] <perrito666> Alexisb tych0 i doubt they will want that in 2.1 a day before release
[21:30] <anastasiamac> perrito666: thumper: making coffee and coming
[21:31] <perrito666> anastasiamac: nice, bring me one
[21:32]  * anastasiamac brings perrito666 vitual coffee
[23:24] <axw> thumper wallyworld: seen https://github.com/juju/juju/pull/6996?
[23:26] <thumper> axw: no, but in the middle of something
[23:26] <axw> okey dokey
[23:32] <frankban> hey wallyworld thanks for the fix
[23:34] <wallyworld> frankban: no worries. it looks good. the only wrinkle is that you need to click on the canvas after log in. have you seen that?
[23:35] <frankban> wallyworld: yes, I wonder why
[23:35] <wallyworld> not sure, but i didn't see it as a blocker and it is a corner case
[23:35] <wallyworld> we can do that for 2.1.1 if it really is an issue
[23:37] <frankban> wallyworld: +1 it's not a great issue
[23:38] <wallyworld> frankban: and will all that extra effort 2.1 release is delayed a few days :-/
[23:38] <wallyworld> *with
[23:38] <frankban> auch
[23:38] <wallyworld> at least it is all fixed
[23:38] <wallyworld> rc2 will be going out tomorrow
[23:38] <wallyworld> or tonight
[23:38] <frankban> ok
[23:39] <wallyworld> the hosted juju beta is on 2.1rc1 as that is very stable
[23:39] <wallyworld> staleholders wanted more 2.1 final testing before release
[23:40] <frankban> wallyworld: yeah, cool. it will provide new-new GUI 2.4.0 then, for new bootstraps, we should release that tomorrow
[23:41] <wallyworld> great
[23:41] <frankban> well, today...
[23:44] <wallyworld> today, tomorrow, timezones make that very murky :-)