| redir | i'll be around for a while | 00:07 |
|---|---|---|
| menn0 | thumper: no right now I'm not. I'm working on this mgo dup key issue | 00:07 |
| menn0 | thumper: I can help if needed though | 00:07 |
| redir | thumper: menn0 I'll be around for another 90 minutes or so | 00:08 |
| redir | so no huge rush | 00:08 |
| menn0 | redir: should I have a quick look at it with you now while you're still around? | 00:08 |
| redir | menn0: if you are at a good stopping point sure. If not let me know when you get to one. | 00:09 |
| menn0 | redir: now is good | 00:09 |
| menn0 | redir: hangout or IRC? | 00:10 |
| redir | HO | 00:10 |
| redir | ? | 00:10 |
| menn0 | ok, let's just use the tanzanite-standup one | 00:10 |
| menn0 | redir: ^ | 00:10 |
| redir | k | 00:11 |
| wallyworld | axw: do you have a moment for a quick chat? | 01:00 |
| axw | wallyworld: sure | 01:00 |
| wallyworld | standup wil do | 01:00 |
| axw | wallyworld: very hard to hear you | 01:03 |
| wallyworld | axw: i'll reconnect | 01:03 |
| thumper | redir: is that race sorted? | 01:45 |
| mup | Bug #1604919 changed: juju-status stuck in pending on win2012hvr2 deployment <oil> <oil-2.0> <juju-core:New> <https://launchpad.net/bugs/1604919> | 01:48 |
| redir | menn0: https://github.com/reedobrien/mgo/tree/fix-race-in-stats | 02:11 |
| redir | menn0: I can make a PR if you want. But if you think it needs more polish... go ahead. | 02:12 |
| === natefinch-afk is now known as natefinch | ||
| redir | menn0: same results on the tests with or without the patch. | 02:12 |
| redir | thumper: ^^ | 02:14 |
| thumper | menn0: http://reviews.vapour.ws/r/5284/diff/# | 02:16 |
| thumper | menn0: I think you fixed the original, this is 1.25 now | 02:16 |
| thumper | axw: oh, you are oncall reviewer | 02:16 |
| thumper | axw: you could look above if you like | 02:17 |
| mup | Bug #1458585 changed: SSHGoCryptoCommandSuite.TestCommand fails <blocker> <ci> <go1.6> <intermittent-failure> <regression> <test-failure> <wily> <xenial> <juju-core:Invalid> <juju-core 1.23:Won't Fix> <juju-core 1.25:In Progress by thumper> <https://launchpad.net/bugs/1458585> | 02:18 |
| axw | thumper: sure, looking | 02:20 |
| axw | thumper: LGTM | 02:21 |
| thumper | axw: ta | 02:21 |
| thumper | axw,wallyworld: isn't status history updated outside of transactions? | 02:24 |
| thumper | so raw access? | 02:25 |
| axw | thumper: I think so? | 02:25 |
| wallyworld | thumper: i thought so | 02:25 |
| thumper | allcollections doesn't mark it as rawAccess | 02:25 |
| thumper | I'm looking at the bug about big transactions to clean up | 02:25 |
| * thumper looks | 02:26 | |
| wallyworld | maybe the collection is accessed directly without txn ops | 02:26 |
| perrito666 | it is accessed without txn ops | 02:26 |
| perrito666 | or was last time I looked | 02:26 |
| axw | st.getRawCollection(statusesHistoryC) | 02:26 |
| thumper | hmm... we probably want to mark it as so otherwise removing the items uses transactions :) | 02:27 |
| thumper | I'll do it | 02:27 |
| menn0 | thumper: sorry, was lunching... looking | 02:31 |
| menn0 | thumper: looks good, although I don't think I fixed this in master (at least I don't remember doing it) | 02:33 |
| thumper | I compared the patch with the utils package | 02:34 |
| thumper | menn0: axw has reviewed anyway | 02:34 |
| natefinch | Is add-cloud intended to work with the static clouds, for example, ec2? (it does, I just am wondering if it's *supposed to*) | 02:36 |
| wallyworld | natefinch: it is supposed to allow you to add your own config | 02:38 |
| wallyworld | thumper: so getRawCollection() - does that not work if collection is not marked as raw? | 02:38 |
| wallyworld | thst sorta sucks a bit | 02:38 |
| thumper | wallyworld: it does | 02:38 |
| natefinch | wallyworld: ok, I had assumed it was only for things like manual, openstack, and maas, where you can actually create your own cloud | 02:39 |
| thumper | wallyworld: but we have code that iterate over the all collections | 02:39 |
| thumper | and do different things for raw | 02:39 |
| wallyworld | ah ok | 02:39 |
| wallyworld | natefinch: private cloud set up is one use case for sure | 02:39 |
| natefinch | wallyworld: just wondering, since I'm working on the interactive add-cloud | 02:40 |
| wallyworld | do the private cloud first, and then the public cloud config stuff can be done later - i'm prerry sure the doc references it | 02:41 |
| wallyworld | one issue is that we need to enumerate valid config items to make user input sensible | 02:41 |
| wallyworld | and there's not really a clear path to do that yet | 02:41 |
| natefinch | wallyworld: yeah... right now I'm just hard-coding it per cloud. | 02:43 |
| wallyworld | the valid config items? | 02:44 |
| natefinch | wallyworld: what values to ask the user for (which map to config values) | 02:44 |
| wallyworld | we should be able to use the config schema definition | 02:44 |
| wallyworld | just as is done when parsing config.yaml | 02:44 |
| wallyworld | but use it to get at the valid item names and types | 02:45 |
| natefinch | I'm not sure I can use the schema to make a user-friendly UI. | 02:46 |
| wallyworld | really? it has the attribute names, their types, whether they are mandatory etc | 02:47 |
| natefinch | what might be a good compromise is to put the interactive add-cloud code inside each provider, so it lives with the code that knows what values to ask about. | 02:47 |
| natefinch | wallyworld: the key word there is user-friendly | 02:48 |
| wallyworld | how else would it be done if not using the schema? | 02:48 |
| natefinch | wallyworld: by having a human write sentences that humans can understand | 02:48 |
| wallyworld | huh? for name - value input? | 02:48 |
| natefinch | wallyworld: this is for the interactive add-cloud. I'm not going to just prompt them with the yaml field names. | 02:49 |
| wallyworld | that's what is used now in config.yaml files, and on the CLI as --config args | 02:50 |
| natefinch | yes, and it's awful, and that's why we're writing an interactive mode. | 02:50 |
| wallyworld | what's wrong with allowing them to enter a value for "http-proxy" say | 02:50 |
| wallyworld | that is entirely readable | 02:51 |
| wallyworld | you'd use tab completion to help too | 02:51 |
| wallyworld | and print the description of each field also | 02:51 |
| natefinch | because if we write it by hand, it'll be clean, crisp, and tasteful. If we auto-generate it, it'll feel auto-generated, it won't be able to do intelligent verification (like checking that a URL is a valid URL)... | 02:55 |
| wallyworld | why can't we check a valid url? of course we can | 02:57 |
| natefinch | the schema doesn't say it's a url, it just says it's a string.... I'm sure we check it later on sometime, but it's one of those things where you'll enter 10 fields, then it'll go off and run through a verification, and spit back that field #7 is incorrect | 02:58 |
| wallyworld | whether it is done off a schema or we hard code prompts, the UX is the same | 02:58 |
| natefinch | I guarantee you, the UX will not be the same if it's hand coded. | 02:59 |
| wallyworld | the schema does need to say it's a url, yes. we can do that | 02:59 |
| natefinch | That's just one example | 02:59 |
| wallyworld | why won't it be the same? | 02:59 |
| wallyworld | it's printing a prompt to enter a value | 02:59 |
| wallyworld | in either case | 02:59 |
| natefinch | yes, but the prompt is quite likely not to be exactly the same as the description of the yaml field | 03:00 |
| wallyworld | so we fix our metadata | 03:00 |
| natefinch | also, there's intelligence that is outside the schema... like the fact that openstack username and password only apply if the auth-mode is user-pass | 03:00 |
| wallyworld | how do we handle the case where the config schema gains a new value? | 03:00 |
| wallyworld | we have already talked about the need to make the schema support grouping and conditional fields | 03:01 |
| natefinch | making the schema more complicated is not the answer | 03:01 |
| natefinch | it's ok to write code | 03:01 |
| wallyworld | it's not ok to create something that is hard to maintain | 03:02 |
| wallyworld | and prone to skew | 03:02 |
| wallyworld | schemas are the common solution for user data entry. our gui uses json schema for example | 03:02 |
| natefinch | we'll spend days, weeks, adding gobs of complexity to the schema just to avoid the simplest of imperative code. | 03:04 |
| wallyworld | it's not gobs of complexity | 03:05 |
| natefinch | every time we have to do an if/else... that's significant added complexity and cognitive overhead to the schema... when it's completely trivial in imperative code. | 03:05 |
| natefinch | how do you signify that openstack user and password only apply if auth-mode is userpass? | 03:06 |
| wallyworld | using a directive in the schema - that's what schemas are for | 03:06 |
| natefinch | how do you do it? | 03:06 |
| wallyworld | http://stackoverflow.com/questions/9029524/json-schema-specify-field-is-required-based-on-value-of-another-field | 03:07 |
| * thumper taps his fingers while the state tests run | 03:07 | |
| thumper | perhaps I should go and clean the coffee machine | 03:07 |
| wallyworld | i haven't got the definitive refence handy | 03:07 |
| wallyworld | but we are talking about bog standard schema features | 03:08 |
| thumper | oh ffs | 03:09 |
| thumper | state tests take over 10 minutes | 03:09 |
| natefinch | that's jsonschema... the provider config schema is our own custom thing, isn't it? gopkg.in/juju/environschema.v1 | 03:09 |
| wallyworld | if we hard code it, who will take the lead on ensuring any config additon or change to the schema will be reflected in add-cloud? that sort of double handing and non-DRY leads to bugs | 03:09 |
| thumper | time go test -check.v -test.timeout=1200s -cover | 03:10 |
| wallyworld | yes it is our own, but trivial to enhance | 03:10 |
| thumper | lets see what that shows us | 03:10 |
| thumper | now I'm definitely going to clean the coffee machine | 03:10 |
| natefinch | wallyworld: I was actually thinking that the interactive code for each provider should live in the provider. Then it lives right next to the configuration, and so it's easy to check that any changes to the config have corresponding changes in the interactive code. | 03:11 |
| wallyworld | but a lot of the base config is defined outisde of the provider in environs/config | 03:11 |
| natefinch | sure, if and when we decide to automate those parts, the common interactive bits for those parts can live next to the common config definitions. The other nice thing about just writing code is that it's easy to pick a sensible subset of things to ask the user, rather than giving them 50 prompts, when 48 of those are optional. | 03:14 |
| wallyworld | the optional schema attributes would not be prompted for - the code can use the is-optional property of the schema to drive that | 03:16 |
| natefinch | we're getting down a rat hole. My initial statement is still the same. We'll have to add a ton of complexity to our schemas (and I *do not* believe that adding conditional dependencies to our schema is a small amount of work), all to save a very small and easily maintained bit of code. And every time we have some other custom field that works in some custom way, we'll have to extend our schema in new and interesting ways and it'll becomes a | 03:20 |
| natefinch | maintenance burden of its own. Also to avoid writing self-contained, very simple imperative code that lives near the configuration values it is intended to fill. | 03:20 |
| natefinch | s/Also/All | 03:20 |
| wallyworld | i disagree, as do every other project that uses a schemma to drive user input | 03:21 |
| thumper | coverage: 85.3% of statements | 03:21 |
| thumper | ok github.com/juju/juju/state617.074s | 03:21 |
| thumper | real11m3.809s | 03:21 |
| natefinch | wallyworld: you can generate a functional UI, you can't generate a *good* one. And I bet I can write all the UIs for all our private clouds in less time than I could write the update to our schema to just support openstack. | 03:24 |
| wallyworld | sure, but at the cost of maintability. it's always easy to bang out bespoke code | 03:25 |
| wallyworld | openstack and aws both have different credential types | 03:26 |
| thumper | axw: http://reviews.vapour.ws/r/5285/ | 03:26 |
| natefinch | it's not complicated code. Print "What auth mode would you like (options..): if authmode == openstack.userpass { Print "What username?" } etc. | 03:27 |
| wallyworld | and that can be trivially driven by a schema | 03:28 |
| wallyworld | to produce the exact same output | 03:28 |
| wallyworld | but will be maintainable | 03:29 |
| natefinch | I already find our usage of schemas to be hard to understand.... adding more complexity is just going to make that worse. | 03:30 |
| natefinch | to be fair, I think that has less to do with the schema itself and more our usage of it | 03:32 |
| wallyworld | seems like you are conflating a schema driven approach with not understanding our schema | 03:32 |
| natefinch | well, no. | 03:34 |
| axw | thumper: reviewed | 03:34 |
| natefinch | wallyworld: I'm trying to avoid writing a framework, and just write what we need. Do the simplest thing that could possibly work. If I'm some newbie juju dev, and I add a config value to some provider, and it needs some custom logic in the prompt, it's trivial to just write that code. If instead I have to go try to figure out how to bend the schema to work, or, more likely, add logic to the schema, it's going to take me an order of magnitude | 03:41 |
| natefinch | longer. We haven't even started and we've already mentioned two additions we would have to make for the schema. | 03:41 |
| wallyworld | we can disagree :-) | 03:43 |
| natefinch | sure, but we gotta pick one :) | 03:44 |
| axw | natefinch: the reason for using a schema is because you may not have acccess to the provider code at all times | 03:44 |
| axw | natefinch: e.g. the GUI, or an older juju client | 03:45 |
| natefinch | well, the older client wouldn't have the new provider code anyway, so it wouldn't know what to do with the information even if it could request it. | 03:50 |
| axw | natefinch: why not? the client doesn't talk to the cloud directly, except for when it's bootstrapping or destroying the controller | 03:51 |
| axw | natefinch: sorry, you're specifically looking at bootstrap aren't you. credentials are bigger than just bootstrap | 03:53 |
| natefinch | axw: currently looking at add-cloud | 03:53 |
| axw | natefinch: it should be possible to add credentials for a cloud after bootstrap also | 03:53 |
| thumper | axw: removed the retry, and cleaned up | 03:55 |
| thumper | axw: the undertaker will restart if there is an error | 03:55 |
| thumper | and the other uses are in migration | 03:55 |
| thumper | and I can work with menn0 to make sure those are retried if necessary | 03:56 |
| axw | thumper: thanks | 03:59 |
| axw | thumper: LGTM with comment fix | 03:59 |
| thumper | axw: ta | 03:59 |
| * thumper waits for test run to complete | 03:59 | |
| thumper | PASS: allwatcher_internal_test.go:940: allWatcherStateSuite.TestStateWatcherTwoModels305.009s | 04:01 |
| thumper | this test seems excessively long | 04:01 |
| natefinch | axw, was that 6 minutes for one test? | 04:01 |
| axw | natefinch: I guess you mean thumper | 04:02 |
| thumper | natefinch: yes | 04:02 |
| natefinch | axw: oops, sorry, yes | 04:02 |
| thumper | 11.5 minutes for the whole package | 04:02 |
| natefinch | thumper: good lord | 04:02 |
| thumper | 5 minutes for that one | 04:03 |
| natefinch | er yes, 5, bad math penalty. | 04:03 |
| * anastasiamac impressed with thumper's machine - my machine panics as off late while running all tests in state package | 04:05 | |
| mup | Bug #1605050 opened: Controller doesn't use tools uploaded with juju bootstrap --upload-tools <juju-core:New> <https://launchpad.net/bugs/1605050> | 04:12 |
| thumper | anastasiamac: just provide the timeout option | 04:15 |
| thumper | mine takes > 10 minutes too | 04:15 |
| wallyworld | thumper: can you recall - what is the error if you try and deploy to a dying model | 04:15 |
| thumper | I don't recall | 04:16 |
| anastasiamac | thumper: \o/ yep, test.timeout is super-useful :D | 04:16 |
| anastasiamac | thumper: tyvm | 04:16 |
| * thumper is looking at this very slow test | 04:29 | |
| thumper | hmm... | 04:33 |
| thumper | menn0: got a minute? | 04:33 |
| menn0 | thumper: give me 30s | 04:33 |
| thumper | k | 04:33 |
| thumper | hmm... | 04:36 |
| thumper | I wonder if this fix is safe | 04:36 |
| thumper | 305s -> 6s | 04:36 |
| menn0 | thumper: ok, what's up? | 04:37 |
| thumper | 1:1 ? | 04:37 |
| menn0 | yep | 04:38 |
| menn0 | thumper: ok back again | 04:49 |
| mup | Bug #1605057 opened: allWatcherStateSuite.TestStateWatcherTwoModels takes > 5 mintues <tech-debt> <unit-tests> <juju-core:In Progress by thumper> <https://launchpad.net/bugs/1605057> | 05:12 |
| thumper | w00t | 05:15 |
| thumper | test win of the day | 05:16 |
| thumper | menn0: http://reviews.vapour.ws/r/5286/ | 05:17 |
| menn0 | thumper: looking | 05:17 |
| mup | Bug #1605057 changed: allWatcherStateSuite.TestStateWatcherTwoModels takes > 5 mintues <tech-debt> <unit-tests> <juju-core:In Progress by thumper> <https://launchpad.net/bugs/1605057> | 05:21 |
| thumper | menn0: if you have a txn.Op with an assertion but no Insert/Remove etc, and it is the only op, what happens if the assertion fails? | 05:22 |
| thumper | I mean as a []txn.Op | 05:22 |
| thumper | but it is the only one | 05:22 |
| menn0 | thumper: then an ErrAborted is returned (I guess) | 05:24 |
| wallyworld | axw: here's a fix for that deleted model bug http://reviews.vapour.ws/r/5287 | 05:24 |
| menn0 | thumper: and we should be careful with this kind of thing. remember it seems to encourage mgo/txn to create runaway txn-queue fields | 05:24 |
| mup | Bug #1605057 opened: allWatcherStateSuite.TestStateWatcherTwoModels takes > 5 mintues <tech-debt> <unit-tests> <juju-core:In Progress by thumper> <https://launchpad.net/bugs/1605057> | 05:24 |
| thumper | menn0: ack, I'll do something else | 05:25 |
| menn0 | thumper: review done. looks good. just a couple of minor suggestions | 05:25 |
| axw | wallyworld: reviewed | 05:34 |
| wallyworld | ta | 05:35 |
| wallyworld | axw: i was using the "Word of God", but yours is better :-) | 05:38 |
| axw | wallyworld: lol | 05:39 |
| wallyworld | axw: also, william has indicated in the past that he hates a generic notfound code used everywhere | 05:39 |
| wallyworld | but i can ask again | 05:39 |
| * thumper is done for the day | 05:40 | |
| thumper | laters folks | 05:40 |
| === akhavr1 is now known as akhavr | ||
| === frankban|afk is now known as frankban | ||
| mup | Bug #1605096 opened: Juju 2.0 Resource Error - sha mismatch error while re-deploying the charm <juju-core:New> <https://launchpad.net/bugs/1605096> | 07:34 |
| babbageclunk | menn0: Nice spotting on the upload tools problem! | 08:10 |
| wallyworld | fwereade: hey, i have a small PR which uses a new bespoke error code for a model which is no longer there (ie model not found). andrew looked but wanted your +1 on using the new error code. i'm sure we've talked in the past and not just using generic notfound everywhere is preferred. could you take a look? http://reviews.vapour.ws/r/5287/ | 08:18 |
| wallyworld | i need to land this for beta13 so if you +1 could you hit $$JFDI$$ for me as i need to head to soccer for a bit | 08:19 |
| menn0 | babbageclunk: just finishing en email to you with a few bits | 08:33 |
| fwereade | wallyworld, LGTM, I left a trivial issue open but I'm happy to land it | 08:33 |
| babbageclunk | menn0: Cool cool | 08:33 |
| menn0 | babbageclunk: sent! | 08:34 |
| * menn0 has to run for now. will check in again later | 08:34 | |
| babbageclunk | menn0: kthxbye | 08:37 |
| mgz | ha... we're not actually using the uploaded tools? well, that explains some of the oddness. | 08:37 |
| === akhavr1 is now known as akhavr | ||
| === akhavr1 is now known as akhavr | ||
| === akhavr1 is now known as akhavr | ||
| * fwereade is trying to arrange for the split water main outside the house to be dealt with, might not make the meeting | 09:57 | |
| babbageclunk | Oh no, I totally missed the meeting! I got the 10-minute reminder but was wrestling with a patch and forgot about it by 11. :( | 10:34 |
| menn0 | babbageclunk: back again for a bit | 10:57 |
| babbageclunk | menn0: oh hai | 10:57 |
| babbageclunk | menn0: Replied to your message. | 10:57 |
| menn0 | babbageclunk: just reading that now | 10:58 |
| menn0 | babbageclunk: your modification LGTM. it's a little more correct that way isn't it. | 10:59 |
| menn0 | babbageclunk: does all look well when you run this under your test? | 11:01 |
| babbageclunk | yup | 11:01 |
| wallyworld | fwereade: thanks for +1, back from soccer, will land after eating | 11:03 |
| babbageclunk | menn0: So, for the logging were you suggesting a sync.Once living at global scope, and then a call to Do from Query.Apply? | 11:23 |
| menn0 | babbageclunk: or newSession or something? you don't want it too early, certainly not at import time, because logging may not be set up properly yet. | 11:25 |
| babbageclunk | menn0: Ok, makes sense - and the key thing is that the once lives at package scope so that it doesn't log once per session. | 11:26 |
| menn0 | babbageclunk: yep exactly | 11:26 |
| stokachu | babbageclunk: menn0: 10 runs in, no errors yet | 11:38 |
| babbageclunk | stokachu: Awesome! | 11:38 |
| menn0 | stokachu: ok great. | 11:38 |
| menn0 | stokachu: please give it a bit more testing if you can | 11:38 |
| stokachu | im going to let them run about 4 hours | 11:38 |
| menn0 | That should do it :) | 11:39 |
| stokachu | :D | 11:39 |
| stokachu | ill post my findings before lunch | 11:39 |
| babbageclunk | menn0: Do we set mgo's logger in juju? I can't see a SetLogger call anywhere. | 11:42 |
| menn0 | babbageclunk: I don't think you want to use mgo's logging infrastructure in this case | 11:42 |
| menn0 | import loggo, create a logger with loggo.GetLogger and use that | 11:42 |
| menn0 | that'll feed straight into juju's logging system | 11:43 |
| babbageclunk | menn0: Ok, sounds good. | 11:43 |
| === akhavr1 is now known as akhavr | ||
| babbageclunk | menn0: Probably doesn't matter, but: logging at info level? Questionable but means we can be sure it we'd see it without the reporter having to change the log level. | 12:06 |
| babbageclunk | menn0: https://github.com/babbageclunk/mgo/commit/17abfbbc91fbb6a37e682946a8d423b1f7c4633d | 12:08 |
| menn0 | babbageclunk: I thought the default log level was WARNING, not sure though | 12:08 |
| babbageclunk | menn0: In Juju? | 12:08 |
| menn0 | yep | 12:08 |
| babbageclunk | huh | 12:09 |
| menn0 | It starts at DEBUG and then at some point during agent startup the model's logging config is applied | 12:09 |
| menn0 | if there hasn't been an explicit one specified a default is used | 12:09 |
| * menn0 checks | 12:09 | |
| menn0 | babbageclunk: TBH this message is almost certainly going to appear before the model's logging config is applied so may it's a moot point | 12:10 |
| babbageclunk | Ok, I'll do it as debug and make sure I can see it in a bootstrap. | 12:10 |
| menn0 | babbageclunk: sounds good | 12:14 |
| menn0 | babbageclunk: i'm off. past bed time here | 12:14 |
| babbageclunk | Can someone who knows about Juju logging help me? I've added the logging I was discussing with menn0 ^^, but I can't see it in logs after bootstrapping. | 12:55 |
| fwereade | babbageclunk, am pretty sure that `juju set-model-config logging-config=INFO` will set the level globally | 12:56 |
| fwereade | babbageclunk, also take note of maybe wanting to log some things in the controller model and some in the hosted model | 12:57 |
| fwereade | babbageclunk, s/log some things/see some logs/ | 12:57 |
| babbageclunk | fwereade: So I should check machine-0.log on the controller and in (say) default? | 12:58 |
| babbageclunk | fwereade: I've changed the place where I log to be a warning but it still doesn't seem to show up either on the controller or a machine that I add. | 12:59 |
| fwereade | babbageclunk, huh | 12:59 |
| babbageclunk | fwereade: Do we log to mongo? | 12:59 |
| babbageclunk | https://github.com/go-mgo/mgo/commit/071b5646959162617d7356b49be8f419e5f8229c | 12:59 |
| fwereade | babbageclunk, we do; I am brainfarting on exactly where the code lives | 13:00 |
| babbageclunk | I'm logging in a sync.Once in session.Dial. If the log writer calls Dial, then maybe that first message gets dropped on the floor (since the writer's being set up) and then the message never gets logged again (because Once)? | 13:01 |
| fwereade | babbageclunk, hmm, I am a little loath to suggest it but I would be inclined to use the mgo logging stuff there | 13:01 |
| mup | Bug #1605241 opened: juju2beta12 lxd instances not starting <conjure> <juju-core:New> <https://launchpad.net/bugs/1605241> | 13:02 |
| fwereade | babbageclunk, and use mgo.SetLogger to give it something that near-enough matches what it expects | 13:02 |
| babbageclunk | fwereade: Where would the logging go? Oh, whereever the logger writes to. | 13:02 |
| fwereade | babbageclunk, which can itself be wrapping a loggo.Logger | 13:02 |
| perrito666 | la | 13:02 |
| * perrito666 wrong window | 13:03 | |
| babbageclunk | fwereade: Wouldn't that mean my patch would need to patch something within juju/juju? | 13:03 |
| fwereade | babbageclunk, as a patch to mgo alone, I think it should use mgo's existing logging facilities | 13:04 |
| fwereade | babbageclunk, if we wanted to make use of it in juju, we can/should separately register a logger for mgo so the output goes where we want it | 13:04 |
| babbageclunk | fwereade: But there's nowhere in juju/juju that already calls mgo.SetLogger | 13:05 |
| fwereade | babbageclunk, this is true -- what's the high level goal here? to get something in juju's logs that tells us that mgo did the right thing? | 13:05 |
| fwereade | babbageclunk, if so, I think we should be using the existing mgo logging | 13:06 |
| fwereade | babbageclunk, if not, sorry I misunderstood :) | 13:06 |
| babbageclunk | fwereade: Just so we can tell that the juju running has the patch without torture testing (like stokachu's doing now) | 13:06 |
| babbageclunk | fwereade: but the existing mgo logging isn't hooked up, so I'd need to make a separate change to do that. | 13:07 |
| fwereade | babbageclunk, and I'm not sure that would be super-helpful either... we probably don't want to risk logging db ops to the db | 13:07 |
| babbageclunk | fwereade: yeah - I think that's kind of the problem I'm hitting. | 13:08 |
| fwereade | babbageclunk, so, how about exposing `mgo.HasAppliedE11000Patch() bool` and logging that explicitly in controller setup? | 13:08 |
| fwereade | babbageclunk, which can just Once the patch and return true? | 13:09 |
| fwereade | that's weird though, logging shouldn't be what triggers it | 13:09 |
| babbageclunk | fwereade: Would I have to do that with reflection in the Juju code? Otherwise no-one would be able to build without applying the patch. | 13:09 |
| babbageclunk | (Which might be a good thing.) | 13:10 |
| fwereade | babbageclunk, ha. dammit again :) | 13:10 |
| fwereade | babbageclunk, although, yeah | 13:10 |
| fwereade | babbageclunk, that may very well be a good thing | 13:10 |
| fwereade | babbageclunk, I am not comfortable developing against arbitrarily-varying dependencies | 13:11 |
| babbageclunk | fwereade: probably means we'd need better (some) tooling to apply/unapply patches. | 13:11 |
| fwereade | babbageclunk, it would, and we were just discussing in the tech board how we probably didn't want to devote effort to it because we wanted this situation to be rare and not encourage it | 13:12 |
| babbageclunk | fwereade: crazy idea - in the once, start a goroutine that waits 2 seconds and then logs the patch message. | 13:12 |
| * babbageclunk felt bad typing that. | 13:13 | |
| * fwereade peers disapprovingly over half-moon spectacles | 13:13 | |
| * babbageclunk is just going to try it for shits and giggles. | 13:13 | |
| mup | Bug #1605241 changed: juju2beta12 lxd instances not starting <conjure> <juju-core:New> <https://launchpad.net/bugs/1605241> | 13:14 |
| * fwereade -- if such a thing lands -- will try to arrange matters such that he gets to hunt you for sport ;p | 13:15 | |
| fwereade | babbageclunk, but, yeah, see if it makes a difference | 13:15 |
| babbageclunk | Would at least confirm that that's the problem. | 13:16 |
| fwereade | babbageclunk, I am generally uncomfortable with our using logging as this sort of data channel, *and* with adding a loggo dependency to mgo... | 13:16 |
| fwereade | babbageclunk, ...but I don't have a better answer offhand, I must admit | 13:18 |
| babbageclunk | fwereade: This logging is only intended to be in the patch - it'll never go upstream, and will be gone when the patch can go. | 13:18 |
| babbageclunk | fwereade: but I know what you mean. | 13:18 |
| fwereade | babbageclunk, there's nothing so permanent as a temporary fix | 13:19 |
| babbageclunk | quite | 13:19 |
| fwereade | babbageclunk, how will we know when we're using a fixed upstream build? ;p | 13:19 |
| fwereade | babbageclunk, unless that also logs... | 13:19 |
| mup | Bug #1605241 opened: juju2beta12 lxd instances not starting <conjure> <juju-core:New> <https://launchpad.net/bugs/1605241> | 13:20 |
| babbageclunk | The problem with the patch is there's no place we can look to determine whether a binary includes it - if the binary is tagged with a hash we can at least look in dependencies.tsv and follow the chain. | 13:21 |
| fwereade | babbageclunk, (can we abuse the flag package to accept `-mgo.ensure-e11000-flag` or did we isolate it well enough that we can't? ...not that that's any better really anyway) | 13:23 |
| fwereade | babbageclunk, wondering about env vars... they're probably the least worst globals I know... | 13:25 |
| * fwereade actually has no idea | 13:27 | |
| fwereade | sorry | 13:27 |
| babbageclunk | Hmm. Am I looking in the wrong place? | 13:28 |
| babbageclunk | Is everything in machine-0.log on the controller machine? | 13:28 |
| fwereade | babbageclunk, I most often just use juju `debug-log --replay` to get it locally so long as the system's generally functioning sanely | 13:30 |
| fwereade | babbageclunk, with appropriate `-m whatever` | 13:30 |
| babbageclunk | ok - that's what I'm doing (with -m controller) | 13:30 |
| babbageclunk | bums, I was sure that would work. | 13:31 |
| fwereade | babbageclunk, hey | 13:32 |
| fwereade | babbageclunk, can you just make it write out a file somewhere..? say, in the standard temp dir, whenever it starts? | 13:32 |
| babbageclunk | Yeah, that seems extremely obvious now. | 13:33 |
| babbageclunk | I can definitely do that. | 13:33 |
| fwereade | babbageclunk, cool | 13:33 |
| babbageclunk | fwereade: Thanks! | 13:33 |
| fwereade | babbageclunk, a pleasure :) | 13:33 |
| * babbageclunk just wasn't expecting this to be so hard! | 13:34 | |
| babbageclunk | I guess that's all of software in a nutshell. | 13:34 |
| fwereade | babbageclunk, never a truer word was spoken | 13:34 |
| fwereade | babbageclunk, that still does have the what-will-a-fixed-upstream-build-do problem, though | 13:37 |
| babbageclunk | I was thinking about putting the time in it, but I guess that's really just the mtime of the file. So an empty file it is! | 13:37 |
| babbageclunk | fwereade: Well, it won't write the file, but we'll be able to see that the PR was merged into the commit. It's just normal change-management stuff, isn't it? | 13:39 |
| babbageclunk | fwereade: (I'm not sure I understand the problem.) | 13:39 |
| fwereade | babbageclunk, yeah, if it's just an internal check that's fine -- but if we expect anyone in the field to need to check it, surely we should always write it forever? | 13:44 |
| fwereade | babbageclunk, i.e. if it's just for us, to have confidence that it was applied in build X, we can trigger the write on an env var and just make that one of the things we do until we merge the new version | 13:45 |
| fwereade | babbageclunk, if it's not just for us it's 1000x harder | 13:45 |
| babbageclunk | fwereade: I think it's just for us. | 13:46 |
| babbageclunk | fwereade: why an env var? | 13:47 |
| babbageclunk | fwereade: (as opposed to unconditionally) | 13:47 |
| babbageclunk | fwereade: Ha ha - what if trying to log from inside dial was triggering a panic due to infinite recursion and then loggo was suppressing that? | 13:49 |
| fwereade | babbageclunk, just because it's nicer not to dump the file wherever we run in production | 13:53 |
| fwereade | babbageclunk, (that would be engagingly horrifying) | 13:53 |
| babbageclunk | fwereade: yeah, makes sense - is it hard to set the env var so that it's set before the controller agent gets bootstrapped? Or would we check the env var each time, and only once.Do the writing when the env var was true | 13:56 |
| babbageclunk | ? | 13:56 |
| fwereade | babbageclunk, I was imagining that patched-mgo could have an init() that checks the var, writes the file if true, and that's all | 13:59 |
| babbageclunk | fwereade: ok, but how would we set the env var on the bootstrapped machine in time for that? I guess we could set the var and bounce jujud. | 14:00 |
| rick_h_ | katco: ping for standup | 14:01 |
| babbageclunk | fwereade: ha ha, I'm lame - all my theorising was for naught. The real problem was that I was logging in Dial, which we never call, and not in DialWithInfo. | 14:22 |
| fwereade | babbageclunk, haha, bad luck | 14:23 |
| babbageclunk | So just logging with loggo works fine. | 14:23 |
| fwereade | babbageclunk, ok, please decorate it with suitably apocalyptic warnings about the terrible practices and I'm happy | 14:23 |
| fwereade | ;p | 14:24 |
| babbageclunk | wilco! (At least it's not the asynchronous logging with a sleep version. I really only suggested that to push the overton window across. :) | 14:24 |
| natefinch | fwereade: I can send an email about what we're doing with interactive commands... gotta run, will be back in an hour and a half, which I know is getting late for you. | 14:25 |
| === natefinch is now known as natefinch-afk | ||
| fwereade | natefinch-afk, I might stop then but will try to be back later | 14:29 |
| frobware | dear lazy-web, anybody know off-hand if Python 3's standard library has YAML support (reading/writing) baked in? | 14:36 |
| rick_h_ | frobware: everyone in python land uses http://pyyaml.org/ | 14:38 |
| frobware | rick_h_: is that part of the standard library? | 14:39 |
| rick_h_ | frobware: no, it's not | 14:39 |
| frobware | rick_h_: asking because the script (bridge) has no 3rd party dependencies | 14:40 |
| frobware | rick_h_: if we introduce dependencies we just need to ensure they get to the node being deployed before the script is run | 14:40 |
| frobware | rick_h_: I just added thar as a "risk" to the doc | 14:41 |
| mgz | frobware: can you just output json? | 14:49 |
| mgz | that's also yaml, and is parsable by python stdlib | 14:50 |
| mgz | we should really only use yaml for the config files we expect users to edit a lot | 14:51 |
| frobware | mgz: need to parse networkd's (YAML) config, and rewrite. | 14:54 |
| rick_h_ | frobware: where does this script come from? | 14:55 |
| mgz | frobware: ah, it's not our yaml? suck | 14:56 |
| frobware | rick_h_: juju - we add it to cloud-init as text which gets dropped into a directory and then we have additional "rules" in our cloud-init to go look for that file and run it. | 14:56 |
| mgz | frobware: wait, don't we always have pyyaml on cloud images? | 14:57 |
| rick_h_ | frobware: and it's python because that's what it needs to be for cloud-init? | 14:57 |
| mgz | frobware: it's a cloud-init dependency | 14:57 |
| mgz | so, we can't run cloud-init if it's not installed | 14:57 |
| frobware | rick_h_: it was the easiest way for it to get to the node being deployed. It's essentially a char* - for those old enough. :) | 14:58 |
| frobware | rick_h_: https://github.com/frobware/juju/blob/1.20/provider/maas/environ.go#L659 | 15:00 |
| frobware | rick_h_: and this time for real: https://github.com/juju/juju/blob/master/provider/maas/environ.go#L1200 | 15:01 |
| alexisb | babbageclunk, I am going to change location sbefore we meet | 15:20 |
| alexisb | will be back online in a few minutes, but may be a little later than half past | 15:20 |
| babbageclunk | hi alexisb - sorry, was afk when you sent that message. | 15:56 |
| fwereade | has anyone else seen cmd/jujud/agent failures with "model is not prepared", and/or deadlocks involving the dummy provider? | 15:57 |
| rick_h_ | frobware: so I just bootstrapped, created a machine with add-machine and from there ran python3 import yaml | 16:02 |
| rick_h_ | frobware: and it's there | 16:03 |
| rick_h_ | frobware: so I don't know if it's the image, the cloud-init, or whatever setup but pyyaml is ootb on a machine in xenial | 16:03 |
| frobware | rick_h_: great & thanks | 16:04 |
| frobware | rick_h_: the only downside given our current logic is that we look for python2 first | 16:05 |
| frobware | rick_h_: do you still have the environment running? can you install python2 and try the same import? | 16:06 |
| === natefinch-afk is now known as natefinch | ||
| rick_h_ | frobware: sure thing | 16:07 |
| natefinch | fwereade: I'm here, but feel free to not be here, since it's dinner time over there | 16:08 |
| rick_h_ | frobware: no luck there, no yaml ootb in python2 | 16:09 |
| frobware | rick_h_: so the issue there is that there's no guarantee somebody doesn't customise their MAAS image and we get python2 by our current search order. | 16:09 |
| rick_h_ | frobware: you have to install python-yaml to get it in py2 | 16:10 |
| rick_h_ | frobware: so can we just change the search order? | 16:10 |
| rick_h_ | frobware: I mean py3 is the default now? | 16:10 |
| frobware | rick_h_: see the comment: https://github.com/juju/juju/blob/master/provider/maas/environ.go#L1200 | 16:10 |
| rick_h_ | frobware: so I see/process that. I guess if we know the script works in py3, and if we find it (they modified so py3 is there in trusty for instance) what's the risk of things going boom? maybe the version of py3? | 16:12 |
| frobware | rick_h_: I would say less so than Feb/March - it was all new at that point. | 16:13 |
| frobware | rick_h_: and the unit tests test both python2 and 3 -- assuming you have both installed (on the SUT) | 16:13 |
| babbageclunk | a quiet stokachu is a happy stokachu? | 16:13 |
| rick_h_ | frobware: ok, just pushing/questioning :) | 16:13 |
| stokachu | babbageclunk: so far so good! | 16:13 |
| * babbageclunk dances | 16:13 | |
| stokachu | babbageclunk: ive been beating the hell out of it with menno's patched binary | 16:14 |
| stokachu | no issues yet | 16:14 |
| frobware | rick_h_: yep, the issue we have is... unless you go and test all this manually we can't just suck it and see against a CI / test rig. | 16:14 |
| frobware | rick_h_: which is true for a lot of what we do | 16:14 |
| babbageclunk | stokachu: (ever seen Archer? I want to say phrasing) | 16:14 |
| babbageclunk | stokachu: great! | 16:14 |
| stokachu | babbageclunk: love archer | 16:15 |
| rick_h_ | frobware: understand, ok...so where does this leave you? | 16:15 |
| rick_h_ | frobware: you get yaml in py3, but not other stuff. But this is all about using the new networking in yakkety stuff which is yaml? | 16:15 |
| rick_h_ | frobware: or is this for something else? | 16:15 |
| frobware | rick_h_: no, you're right this is yakkety (so python3) so maybe ok | 16:15 |
| rick_h_ | frobware: so if they monkey with the maas image we'll still know to use py3 because it's got yakkety only networkd stuff | 16:15 |
| rick_h_ | frobware: so triggering off the OS vs the python search tree seems like a potential path forward? | 16:16 |
| frobware | rick_h_: the monkeying comes from me taking a default Y image and makeing my curtin/custom_pressed have python2 for nefarious reasons | 16:16 |
| rick_h_ | frobware: so you mean to not just add py2, but to remove py3? | 16:17 |
| rick_h_ | I mean, just because py2 is there doesn't mean we have ot use it as long as py3 is there as well. | 16:17 |
| frobware | rick_h_: where "me" is some customer in the wild who knows he wants python2 from the get-go | 16:17 |
| frobware | rick_h_: yep, we get first switch on series | 16:17 |
| frobware | get/can | 16:18 |
| frobware | rick_h_: well, that's likely to break stuff. the issue is really only in adding py2. | 16:18 |
| rick_h_ | frobware: right, but we can be resilient to adding py2 because we can be smarter than just looking at python available | 16:18 |
| frobware | rick_h_: but I don't think we should sweat this. we can first switch on series. | 16:18 |
| rick_h_ | if we want to marshall/unmarshall yaml it's because networkd gave it to us and we know to use py3 in that case | 16:18 |
| frobware | rick_h_: yep | 16:18 |
| rick_h_ | frobware: rgr | 16:18 |
| rick_h_ | sounds like a plan | 16:19 |
| frobware | rick_h_: I'm always wary of the sands shifting underneath my feet. :-D | 16:19 |
| rick_h_ | wheeeee | 16:19 |
| * frobware is out of here before there's a python 4... | 16:20 | |
| rick_h_ | lol | 16:21 |
| mup | Bug #1605313 opened: juju 2.0-beta12 ERROR unable to contact api server after 61 attempts: upgrade in progress (upgrade in progress) <sts> <juju-core:New> <https://launchpad.net/bugs/1605313> | 16:32 |
| redir | brb reboot | 17:11 |
| mup | Bug #1605335 opened: Bootstrap agent initialization timeout too small <juju-core:New> <https://launchpad.net/bugs/1605335> | 17:41 |
| === frankban is now known as frankban|afk | ||
| mup | Bug #1605313 changed: juju 2.0-beta12 ERROR unable to contact api server after 61 attempts: upgrade in progress (upgrade in progress) <sts> <juju-core:New> <https://launchpad.net/bugs/1605313> | 19:36 |
| perrito666 | again? | 19:36 |
| mup | Bug #1605383 opened: Tab complete causes traceback when trying to tab-complete juju commands <juju-core:New> <https://launchpad.net/bugs/1605383> | 20:06 |
| rick_h_ | natefinch: ty for the email, while that shakes out can we work on talking with techboard/etc folks on path forward but in the meantime pick up the windows bug in the todo side for now please? | 20:28 |
| rick_h_ | natefinch: and hold onto the add-cloud while the path forward shakes out | 20:28 |
| natefinch | rick_h_: sounds good | 20:28 |
| rick_h_ | natefinch: cool ty | 20:29 |
| === natefinch is now known as natefinch-afk | ||
| redir | brb stepping away for a minute | 21:39 |
| === redir is now known as redir_afk | ||
| anastasiamac | natefinch-afk: could u plz review this one? http://reviews.vapour.ws/r/5291/ | 21:43 |
| anastasiamac | anyone else is welcome to PTAL :) | 21:43 |
| mup | Bug #1604514 changed: Race in github.com/joyent/gosdc/localservices/cloudapi <ci> <joyent-provider> <race-condition> <juju-core:Fix Released by menno.smits> <juju-core 1.25:Fix Released by menno.smits> <https://launchpad.net/bugs/1604514> | 21:45 |
| mup | Bug #1604559 changed: LockingSuite.TestTestLockingFunctionDetectsDisobeyedLock did not obey <ci> <intermittent-failure> <regression> <unit-tests> <juju-core:Fix Released by thumper> <https://launchpad.net/bugs/1604559> | 21:45 |
| mup | Bug #1605057 changed: allWatcherStateSuite.TestStateWatcherTwoModels takes > 5 mintues <tech-debt> <unit-tests> <juju-core:Fix Released by thumper> <https://launchpad.net/bugs/1605057> | 21:45 |
| mup | Bug #1589635 changed: github.com/juju/juju/state fails on TestMachinePrincipalUnits with an unexpected name <ci> <regression> <test-failure> <unit-tests> <juju-core:Fix Released> <https://launchpad.net/bugs/1589635> | 21:54 |
| === frankban|afk is now known as frankban | ||
| === ses is now known as Guest33830 | ||
| === redir_afk is now known as redir | ||
| thumper | menn0: thoughts on this failure? http://juju-ci.vapour.ws:8080/job/github-merge-juju/8503/artifact/artifacts/windows-out.log | 23:34 |
| mgz | thumper: just looks like bug 1600301 | 23:44 |
| mup | Bug #1600301: cmd/jujud/agent MachineSuite.TestHostedModelWorkers fails because compute-provisioner never really starts <ci> <intermittent-failure> <regression> <unit-tests> <juju-core:In Progress by dooferlad> <https://launchpad.net/bugs/1600301> | 23:44 |
| menn0 | thumper: so I've seen this kind of thing in that package quite a bit recently | 23:58 |
| menn0 | as I said in the call there seems to be a few different failure modes | 23:59 |
| menn0 | I'm guessing they're all the same root cause | 23:59 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!