=== bodie__ is now known as bodie_
wwitzel3perrito666: thanks :) glad to hear it00:34
wallyworldaxw: a minute late, just finishing with tim02:00
axwwallyworld: sure, just ping when you're ready02:00
axwI'll go make some tea02:00
wallyworldaxw: bring your tea02:09
menn0thumper: can you have a look at this please? http://reviews.vapour.ws/r/2366/diff/03:01
* thumper looks03:02
menn0thumper: i've realised that there's yet one more PR required: for the api package03:05
menn0thumper: even though Juju won't call WatchAllEnvs itself we should probably have it implemented (WatchAll is)03:06
menn0everything takes too long...03:06
mupBug #1339967 changed: juju: cacheAPIInfo uses the wrong tag when storing the environ UUID <tech-debt> <juju-core:Fix Released> <https://launchpad.net/bugs/1339967>03:34
mupBug #1339967 opened: juju: cacheAPIInfo uses the wrong tag when storing the environ UUID <tech-debt> <juju-core:Fix Released> <https://launchpad.net/bugs/1339967>03:40
* thumper heads out to drop down two laptops to the repair person03:40
mupBug #1235217 changed: support 1.14 environments without .jenv files <jenv> <tech-debt> <juju-core:Won't Fix> <https://launchpad.net/bugs/1235217>03:46
mupBug #1339967 changed: juju: cacheAPIInfo uses the wrong tag when storing the environ UUID <tech-debt> <juju-core:Fix Released> <https://launchpad.net/bugs/1339967>03:46
mupBug #1427505 changed: apiserver/networker: test failure due to map ordering <intermittent-failure> <tech-debt> <test-failure> <juju-core:Fix Released> <https://launchpad.net/bugs/1427505>03:46
mupBug #1235217 opened: support 1.14 environments without .jenv files <jenv> <tech-debt> <juju-core:Won't Fix> <https://launchpad.net/bugs/1235217>03:52
mupBug #1427505 opened: apiserver/networker: test failure due to map ordering <intermittent-failure> <tech-debt> <test-failure> <juju-core:Fix Released> <https://launchpad.net/bugs/1427505>03:52
mupBug #1235217 changed: support 1.14 environments without .jenv files <jenv> <tech-debt> <juju-core:Won't Fix> <https://launchpad.net/bugs/1235217>04:04
mupBug #1427505 changed: apiserver/networker: test failure due to map ordering <intermittent-failure> <tech-debt> <test-failure> <juju-core:Fix Released> <https://launchpad.net/bugs/1427505>04:04
mupBug #1484789 opened: add-machine fails with hosted environments <juju-core:New> <https://launchpad.net/bugs/1484789>05:44
voidspaceTheMue: if you get a chance, could you look at http://reviews.vapour.ws/r/2360/06:52
TheMuevoidspace: morning, will do now07:47
voidspaceTheMue: morning07:48
TheMuevoidspace: you've got a review08:23
mupBug #1484833 opened: juju sync-tools fails to complete as environment is not bootstrapped (catch 22 situation!) <juju-core:New> <https://launchpad.net/bugs/1484833>08:32
dimiternany takers for a trivial review? http://reviews.vapour.ws/r/2363/08:48
dimiternvoidspace, dooferlad, TheMue, axw, wallyworld ^^08:51
TheMuedimitern: *click*08:52
dimiternTheMue, ta!08:56
TheMuedimitern: reviewed08:59
voidspaceTheMue: thanks for the review08:59
TheMuevoidspace: yw08:59
voidspacedimitern: TheMue: dooferlad: be with you in a few minutes08:59
dimiternvoidspace, ok09:00
dimiternTheMue, thanks!09:00
TheMueoh, yes, we're starting now, have to fetch my headset09:00
dimiternvoidspace, ping09:11
voidspacedimitern: ome09:11
=== mup_ is now known as mup
dimiternvoidspace, ping11:55
dimiternvoidspace, looking at what was scheduled and agreed to deliver for the MVP, subnet add (rather than create) is expected to land, FWIW11:59
dimiternvoidspace, and just space create, and both of subnet|space list - that's it12:00
dimitern(along with constraints ofc)12:00
=== MerryChristmas is now known as benonsotfware
=== benonsotfware is now known as benonsoftware
=== dooferlad_ is now known as dooferlad
dooferladdimitern, voidspace, TheMue: I am back this afternoon. Will take a look at the CI test reviews I have got and the board to see the new plan. Anything urgent that should take priority?12:47
TheMuedooferlad: ah, you're back, fine. how do you feel?12:48
mupBug #1484930 opened: Leader's upgrade-charm hook is not run first <juju-core:New> <https://launchpad.net/bugs/1484930>12:50
dimiterndooferlad, awesome!12:52
dooferladTheMue: a bit foggy, but mostly fine12:53
dimiterndooferlad, if you don't feel well though, better not though12:53
dimiterndooferlad, nothing is *that* important ;)12:53
TheMuedimitern: +112:53
dooferladdimitern: I will stop if I think I need to. I can at least get on with low brain effort tasks.12:54
dimiterndooferlad, only you can know that :)12:55
dimiterndooferlad, I'll leave it to your judgment12:56
dooferladdimitern: sure, but once you have run out of paid sick days, it sure feels like the company wants you back :-|12:56
mupBug #1484930 changed: Leader's upgrade-charm hook is not run first <juju-core:New> <https://launchpad.net/bugs/1484930>12:57
dimiterndooferlad, it happens sometimes, but I'm sure we've not gone there yet12:59
dooferladdimitern: oh, I have run out.12:59
dimiterndooferlad, don't worry about it for now13:01
mupBug #1484930 opened: Leader's upgrade-charm hook is not run first <juju-core:New> <https://launchpad.net/bugs/1484930>13:03
dooferladdimitern: why do we want a reference count for state.subnet? We can just perform a simple mgo query to see which spaces are using a subnet (if that is what it is for)13:39
dimiterndooferlad, it's about which subnets have running (etc.) instances13:42
dooferladdimitern: still sounds like a DB query...13:42
voidspacedooferlad: in which case the relationship between instances and subnets needs to be stored somewhere13:43
voidspacedooferlad: in fact that's true anyway13:43
dimiternvoidspace, dooferlad, otp, sorry13:45
dooferladvoidspace: That is what I thought. Seems a shame to add what seems to be a workaround to something that could easily be managed by our single source of truth.13:46
voidspacedooferlad: well, we don't yet have a single source of truth - that's a big part of that card13:46
voidspacedooferlad: references could actually be "list of machine ids" I guess13:47
dooferladvoidspace: that is backwards. Each machine is part of 1+ spaces, so you get the machine document to store the spaces it is in.13:48
voidspacedooferlad: it needs to know subnet, not just space13:48
dooferladvoidspace: then you can query spaces that contain subnets, then machines in those spaces13:48
dooferladActually, you are right, you want to store subnets that a machine is in13:48
voidspacedooferlad: that doesn't tell you which machine is using any specific subnet13:49
voidspaceI suggested putting it on the machine record, dimitern wasn't so kee13:49
dooferladthen if you move a subnet between spaces you don't need to change anything13:49
voidspacehe'd have to explain why13:49
voidspacepart of it maybe transactional13:49
voidspacedooferlad: our transaction simulation doesn't actually work across collections13:50
voidspacedooferlad: so keeping things like reference count in their own collection avoids some problems13:50
voidspacedooferlad: for IPAddresses we store machineId (and instanceId) on the IPAddress13:51
dooferladhttp://blog.labix.org/2012/08/22/multi-doc-transactions-for-mongodb - "Operations may span multiple collections"13:51
voidspacedooferlad: operations can, the transaction guarantees aren't the same I believe13:51
voidspacedooferlad: that's what I was told, although that documentation doesn't seem to agree with it :-)13:52
dooferladI am going with the documentation.13:52
voidspacedooferlad: good luck :-)13:54
voidspacedooferlad: there maybe another (better) reason to prefer reference counts in their own collection13:55
voidspacedooferlad: and it may or may not apply to storing subnet <-> machine relationships13:55
voidspacenot amazingly helpful I realise... but I know fwereade prefers we do that for reference counts at least13:56
dooferladMy problem is this: we have a database. You don't need reference counts if you have the right relations. If you have reference counts you have to work hard to make sure they are right.13:56
voidspacedooferlad: ah, see doc/hacking-state.txt13:57
voidspacedooferlad: the reason for a separate doc is for the sake of watchers13:57
voidspacewatchers don't need to be notified when reference count changes and *watching* only works at the whole doc level13:57
voidspaceI believe...13:58
dimiternvoidspace, dooferlad, instance state != machine life13:58
voidspacedimitern: what does that have to do with it?13:58
TheMuedimitern: quick review of http://reviews.vapour.ws/r/2369/ ? isn't complex13:59
dimiternvoidspace, dooferlad, that's why we need a refcount + txn.Ops on it to avoid removing subnets in use13:59
dimitern(even if no instance are yet/anymore running on it)13:59
voidspacedimitern: we're not proposing anything to do with instance state13:59
voidspacedimitern: we're saying that we need to store machine ids using subnets *anyway*13:59
dimiternTheMue, sure, looking in a moment14:00
voidspacedimitern: in which case a subnet is in use if the number of alive machines using it > 014:00
voidspacedimitern: so you don't need a reference count, you count alive machines14:00
voidspacedimitern: using the db14:00
dimiternvoidspace, you can't store machine ids that don't exist yet though14:00
voidspacedimitern: isn't the machine id created straight away?14:01
voidspacedimitern: it's instance id that is created later14:01
dimiternvoidspace, i.e. as a side-effect or provisioning with space constraints14:01
voidspacedimitern: as soon as you provision with contraints a machine will be created, right? and I don't know what you mean by "side-effects"14:02
voidspacedimitern: picking a subnet is part of provisioning - i.e. part of machine record creation14:02
dimiternvoidspace, I mean a service's space requirements indirectly should keep its subnets in state14:02
voidspacedimitern: a service constraint for undeployed services won't keep any specific subnets in use14:03
dimiternvoidspace, and by requirements I realize constraints are a bad example (except maybe in environment constraints)14:03
katconatefinch: standup14:03
dimiternvoidspace, but for bindings of a service14:03
dimiternvoidspace, once we have them (that was the intent behind --networks)14:03
voidspacedimitern: in which case we add the machine id to all subnets they are "using"14:04
dimiternvoidspace, so even if we avoid adding refcounts and use machineIds-to-subnets, we'll need it pretty soon14:04
voidspace I don't think so14:04
voidspacewe *still* need to do that, even if we use reference counting - so that we could decrement14:04
dimitern(and not having it will keep the subnet up *until* the dead machine doc is in state)14:04
voidspace(by that I mean - store machine id)14:05
voidspacedimitern: in order to decrement properly you have to store somewhere the machine id referencing subnets14:05
voidspacedimitern: agreed?14:05
dimiternvoidspace, ok, first, I absolutely agree we need to be able to link machines to subnets (many-to-many)14:05
voidspaceeven if it's an "implicit" reference as you describe14:05
voidspacedimitern: and if we have that link then the reference count *must* equal the number of links14:06
voidspaceso the reference count is always redundant14:06
dimiternvoidspace, and the original and perhaps not great model was to use machine's NICs (which have subnets) for that14:06
voidspacedimitern: ok, that sounds good - we can do that14:06
voidspaceany reason not to?14:06
voidspacewe have those in state already14:06
dimiternvoidspace, nope - refcount shouldn't stop you from removing a subnet referenced only by non-alive machines14:07
dimiternvoidspace, the only reason not too (not a good one) is we'll need to update (add) NICs too14:07
voidspaceNICs are already added by container address allocation14:08
voidspaceyeah, adding NICs on machine creation is non-zero amount of work :-/14:08
dimiterncomplexity, which can be avoided by a machineSubnetsC for the time being14:08
dooferladwhy do we need subnets --> machines? Again, this is a query.14:08
voidspacedooferlad: in order to determine if the subnet is in use14:09
dimiterndooferlad, that's what we're discussing14:09
alexisbericsnow, I moved our 1x1 to monday so that you can focus on moonstone planing14:09
voidspacedooferlad: so you can tell if it is safe to move the subnet to a different space or remove it altogether (we have a subnet remove command)14:09
dimiterndooferlad, to be able to do that query, you need less data than to ensure it stays like this during an multi-op insert14:09
dooferladdimitern: ah, interesting. *thinks*14:10
voidspacedooferlad: or are you saying "subnets --> machines" is a query. In which case a query of *what*, that information doesn't exist unless we store it.14:10
ericsnowalexisb: thanks!14:10
dooferladvoidspace: I get that, but it feels like you should store in the machine document, what subnets it is part of.14:11
dimiterndooferlad, yeah, mongo makes trivial design discussions so much spicier :D14:11
dooferladthen everything flows from that14:11
dimiterndooferlad, preferably not on the machine document14:11
dooferladwell, wherever the machines network information is. I don't have that in my head yet and need to look it up.14:12
voidspacedimitern: why not? (out of interest)14:13
dimiterndooferlad, voidspace, having it there means any change to it potentially triggers a lot of watchers and workers14:13
dooferladI think my argument is that if you have x is a member of y, you should be able to determine that by x storing that it is a member of y. If you store that y has x, y, z as members then if x goes away you need to update that membership list.14:13
voidspacedimitern: but it doesn't change after machine creation14:13
dimiterndooferlad, voidspace, it changes rarely, but always significantly14:13
voidspacedimitern: when does it change?14:14
dimiterndooferlad, voidspace, e.g. when space membership changes or subsequent deployments14:14
voidspacedimitern: space membership of a *machine*?14:14
voidspacedimitern: how is that possible14:14
dimiterndooferlad, voidspace, you deploy another service from a different space on the same machine14:15
voidspacedimitern: and if it's significant *shouldn't* watchers be triggered...14:15
voidspacedimitern: which would require adding an extra NIC, which we won't support initially14:15
voidspacedimitern: I believe14:15
voidspaceor can you bind multiple subnets to the default NIC on an EC2 instance?14:15
dimiterndooferlad, voidspace, that brings a new NIC on a subnet in that other space, triggers a lot of address and other changes already14:15
voidspacethe *right* thing to do is to use NICs to model this14:16
voidspaceso maybe we should do that14:16
dooferladvoidspace: you don't need physical NICs to have >1 IP address (or addressable containers wouldn't work). I wouldn't worry about physical interfaces (or anything that looks like them)14:16
voidspacedooferlad: EC2 has a concept of a NIC and different instance types have a maximum number of supported NICs14:17
dimiternvoidspace, it's not a change of the machine itself (as juju sees it) - exactly like and why we're not storing what services we deploy to a machine14:17
voidspacedooferlad: and a max number of IP per NIC14:17
dooferladvoidspace: yes, I know14:17
dimitern(only indirectly from their units)14:17
voidspacedooferlad: I think only one subnet per NIC, I may be wrong14:17
voidspacedooferlad: I'm not worrying about physical NICs :-)14:18
dimiterndooferlad, I know - that's yet to be decided (the model expects 1-nic-per-space on machines)14:18
voidspacedooferlad: I'm worrying about modelling machines on multiple spaces in EC214:18
dooferladvoidspace: That EC2 specific logic should be in one tidy place. It is a restriction on what we can do, not an addition.14:18
dimiternwith one-nic-multiple-ips some things will be simpler, but impossible on EC214:18
voidspacedooferlad: well, sure14:19
voidspacewe seem to have sidelined from the main discussion14:19
voidspacewe have a model of NICs in state already14:19
voidspaceNICs hold the relationship between subnets and machines14:19
dooferladindeed. back to the point...14:19
voidspaceso they seem like the right level of abstraction for what we need14:20
voidspaceand they conveniently match what we're trying to model anyway...14:20
dooferladsounds good :-)14:20
dimiterndooferlad, voidspace, it's not just EC2 specific, but AWS is an important target14:20
dooferladdimitern: I completely agree. We just need to make sure that EC2 is a subset, not some blob on the side.14:21
voidspaceyeah, our model needs to *work* on EC2 - just not necessarily be entirely bound by it's limitations14:21
dimiternand NIC in juju terms is more like a linux device than a Elastic Network Interface14:21
dimiterndooferlad, of course, we started from the free-for-all openstack model and AWS restrictions constrained the model in a few ways, but made it better in others14:22
dooferladso, we have IPAddress. That contains a SubnetId and a MachineId. So, we can find all the IPAddress documents that reference a SubnetId before we delete it and use that as the reference count, right?14:26
voidspacedooferlad: IPAddress is only used for "container address allocation"14:27
voidspacedooferlad: not (yet) as a general IPAddress store14:27
dooferladwell, that is horrid.14:27
voidspacebecause it's only for containers that we specifically allocate an address (if the feature flag is on)14:27
voidspacewe get them automatically for machines14:27
dooferladwhat do we use for machines?14:28
voidspaceso no need to request or model them14:28
dooferladto store them that is14:28
voidspacewell, to work out which is the current public/private address for a machine we spit on our finger and stick it in the wind14:28
voidspacewhich is why we have all sorts of bugs around the private/public address of machines appearing to change...14:28
dooferladmachineDoc has two slices of IP addresses.14:29
voidspace(we ask the machine OS and the provider what addresses it thinks it has and report those - holding them on the machineDoc)14:29
dooferladbut no CIDRs14:29
dooferladoh, hang on...14:30
voidspacewe have a bug we have to fix reqarding machine IP addresses apparently changing14:30
voidspacehave to fix  *soon*14:30
voidspaceso moving to a model that actually works would be better14:30
dooferladthe address struct that machine uses could become ipaddressDoc.14:31
dooferladthat would make life easier...14:31
dooferladthen we use that plus a list of machines not yet provisioned that have been allocated a subnet14:32
dooferladwhich I assume is generated from another query looking at constraints14:33
dooferlad(assuming we can't just look at constraints)14:33
natefinchkatco: dang.... somehow my copy of the link failed, can you re-paste?14:53
TheMuedooferlad: dimitern: thx for reviews. the whitebox test does not rely on BaseSuite, so no PatchValue() this time.14:56
mupBug #1484993 opened: TestBootstrapReusesAffinityGroupAndVNet: no tools are available for your environment <blocker> <ci> <i386> <ppc64el> <regression> <unit-tests> <juju-core:Triaged> <https://launchpad.net/bugs/1484993>14:57
mupBug #1484993 changed: TestBootstrapReusesAffinityGroupAndVNet: no tools are available for your environment <blocker> <ci> <i386> <ppc64el> <regression> <unit-tests> <juju-core:Triaged> <https://launchpad.net/bugs/1484993>15:00
mupBug #1484993 opened: TestBootstrapReusesAffinityGroupAndVNet: no tools are available for your environment <blocker> <ci> <i386> <ppc64el> <regression> <unit-tests> <juju-core:Triaged> <https://launchpad.net/bugs/1484993>15:03
dimiternTheMue, you can still call it from juju/testing15:06
dimiternTheMue, at least it does the same thing15:07
TheMuedimitern: will have a look, thought it's a method of an embedded type, which isn't used here15:09
TheMuedimitern: in the moment the patched attempt strategie like you said by PatchAttemptStrategies isn't working15:09
dimiternTheMue, it is a method of BaseSuite via CleanupSuite, which calls PatchValue inside15:09
TheMuedimitern: so I would have to embed the CleanupSuite into the whole suite only for this one patching of a variable I directly have access to15:11
dimiternTheMue, what a moment15:11
TheMuedimitern: will see how the side effects are15:11
dimiternTheMue, s/wait/15:11
dimiternTheMue, environSuite (in whitebox test) uses PatchValue already15:11
TheMuethe whitebox tests are a bit ... different ;)15:11
dimiternTheMue, the other environSuite (non-whitebox one) directly embeds BaseSuite15:12
dimiternlooking provider/maas on master15:12
TheMueso, back, just had phone, sorry15:15
TheMuedimitern: yeah, for the non-wb I've seen, but missed it here15:16
dimiternTheMue, whitebox tests effectively use providerSuite, which already uses PatchValue as it embeds CleanupSuite via Fake(Juju)HomeSuite15:19
TheMuedimitern: yes, deep nested embedding, almost like the "good" old OOP15:20
dimiternTheMue, luckily rogpeppe's godef is amazing for navigating deeply nested types15:21
TheMuedimitern: have to see how I can integrate it into my editor macros15:22
dimiternTheMue, +115:23
voidspaceright EOW15:23
voidspacehappy weekend everyone15:23
TheMuedimitern: hehe, this bad foo == 5, gc.Equals, true is a leftover from some former approaches15:24
TheMuedimitern: PatchValue works fine now15:25
TheMuedimitern: even optimised it using variables for retries and enoughRetries to patch it only once, final problem is now the strategies patching15:28
dimiternTheMue, great!15:28
dimiternTheMue, just use PatchValue for the strategies15:30
TheMuedimitern: yep, thought the same15:30
dimiternTheMue, patching it once and using a retries chan for reporting back15:31
mupBug #1485013 opened: MgoSuite tests must be run with MgoTestPackage <ci> <intermittent-failure> <unit-tests> <juju-core:Triaged> <https://launchpad.net/bugs/1485013>15:33
mupBug #1485017 opened: add the ability to open terminal in hook context <juju-core:New> <https://launchpad.net/bugs/1485017>15:33
TheMuedimitern: hmm, think it's more simple now, will push it so that you can have a final look15:33
TheMuedimitern: so, updated15:35
dimiternTheMue, looking15:35
TheMuedimitern: thx for review, will now merge it16:08
dimiternTheMue, go for it16:09
TheMuedimitern: I added the cards for dummy, maas and ec2. the original one is still there, I grouped them at the end of the lane, so you'll easier find them. I also change the "master card" to a user story, the points are still in there. IMHO these have to be removed due to the points on the individual cards now.16:34
* TheMue grumbles, has branch cannot land due to a fix blocker *sigh*16:35
alexisbTheMue, you could fix it ;)16:35
TheMuealexisb: hehe, sure, but then I have to fix my wife too16:36
alexisbI assigned it to Ian because it is his PR that caused the regression but really it is up for anyone to fix that is affected by the block16:36
alexisbheh understood16:36
alexisbit is friday evening :)16:36
TheMuealexisb: yep, but sill will have a look16:37
dimiternTheMue, got it, thanks!16:43
TheMuedimitern: yw16:43
mupBug #1485071 opened: juju deploy and add-machine not installing agent <juju-core:New> <https://launchpad.net/bugs/1485071>18:27
katcoericsnow: wwitzel3: sorry, also forgot we have to take care of this bug: https://canonical.leankit.com/Boards/View/114568542/11665276719:38
wwitzel3oh right19:40
natefinchericsnow: what do you want to do about the problem with my untrack code and ids that could conflict if they're for different workload types?20:33

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!