| wallyworld | thumper: what did we resolve with respect to printing stuff to stderr where the output format was yaml. typically i think we currently have what's printed to screen is just yaml, but in theory if we printed stuff to stderr, the user could still redirect stdout somewhere to get the pure yaml. or do we want to stick with what i think we have now, which is anything printed to the terminal is just yaml | 01:19 |
|---|---|---|
| thumper | I think the answer was complicated... | 01:20 |
| * thumper is in the middle of emails just now | 01:20 | |
| wallyworld | ok, let me know if you have a minute to discuss. calling an action, we might still want to print progress logs to stderr even if format is yaml | 01:22 |
| thumper | I think our general answer was "if you specify a format, whether yaml or json, you get valid yaml or json on stdout" | 01:23 |
| thumper | why print logs to stderr if they are asking for yaml output? | 01:23 |
| thumper | why not have a stderr section in the yaml? | 01:24 |
| wallyworld | we do | 01:28 |
| wallyworld | but | 01:28 |
| wallyworld | we also want to show progress as it happens | 01:28 |
| wallyworld | so the experience is the same waiting for the action to run regardless of format | 01:29 |
| thumper | I can get on board with the progression messages being stderr | 01:29 |
| wallyworld | i am thinking yes also but wanted to get a +1 since it's a bit different | 01:30 |
| wallyworld | i guess it's easy to change it we get -ve feedback | 01:30 |
| * thumper nods | 01:30 | |
| stub | The general use case is people want free form plain text output from actions, so worth optimising for that. YAML is very occasionally more conventient, and rare it actually needs to be machine readable. And even in those cases, its not always suitable as IIRC the existing API only allows simple strings strings as values. | 01:36 |
| stub | (and the keys have limitations too, such as not being able to use filenames because they tend to contain '.' characters) | 01:38 |
| thumper | wallyworld: https://bugs.launchpad.net/juju/+bug/1847084/comments/5 | 01:40 |
| mup | Bug #1847084: Juju k8s controller is not getting configuration parameters correctly <juju:Incomplete by hpidcock> <https://launchpad.net/bugs/1847084> | 01:40 |
| stub | Maybe the action should choose between structured output (action-set), and unstructured (whatever I spit out) | 01:40 |
| thumper | stub: yes, we are optimising for plain freeform output for action execution | 01:40 |
| thumper | but we still need to think of the edge cases | 01:40 |
| wallyworld | my thnking is that if we are providing a feedback mechanism for long running actions via action-log, you'd want to see that regardess of chosen output format | 01:41 |
| wallyworld | we do support structured vs unstructured too | 01:42 |
| wallyworld | stdout and stderr have their own fields in the yaml | 01:42 |
| wallyworld | and we also now print the action stdout and stderr when running with "plain" format | 01:42 |
| wallyworld | as well as the structured | 01:42 |
| stub | Yes, that would be useful. My knee jerk thought would be action-log ends up on stderr with timestamps and maybe severity, and output ends up on stdout or wherever -o filename put it | 01:43 |
| wallyworld | yup | 01:43 |
| wallyworld | that's how it will work | 01:43 |
| wallyworld | this is happenong in 2.7 edge | 01:43 |
| wallyworld | with "juju-v3" feature flag | 01:43 |
| wallyworld | feature flag gets the new "juju call" syntax | 01:44 |
| wallyworld | but run-action also benefits from the improved output | 01:44 |
| stub | If the action was able to choose its mode (via a hook environment tool), you might be able to make the transition seamless without a flag | 01:44 |
| wallyworld | the flag just determines the juju cli syntax | 01:44 |
| wallyworld | all actions benefit from the newer output | 01:45 |
| wallyworld | the yaml is the same but the plain output is improved | 01:45 |
| wallyworld | all actions also get a numeric id rather than a uuid | 01:45 |
| stub | Just thinking it is pointless the operator needing to specify --format=text for actions designed to spit out text or --format=yaml for actions designed to spit out structured data | 01:46 |
| thumper | ugh... this is a breaking change in a point release... | 01:46 |
| thumper | but I can't think of a way to have any compatibility... | 01:47 |
| wallyworld | thumper: breaking change? | 01:47 |
| wallyworld | ids are opaque | 01:47 |
| wallyworld | nothing should depend on their format | 01:47 |
| thumper | the id of the action is part of the interface | 01:47 |
| thumper | but they do | 01:47 |
| wallyworld | they are just a blob | 01:47 |
| thumper | and they have been uuids | 01:47 |
| thumper | and some tests check for uuids | 01:47 |
| thumper | I know there are openstack ones | 01:47 |
| wallyworld | sigh, that's bad behaviour then | 01:47 |
| wallyworld | ids should be opaque | 01:48 |
| thumper | yes | 01:48 |
| thumper | they should | 01:48 |
| wallyworld | so let's go with that then | 01:48 |
| thumper | but the when the docs said "id is a uuid", acceptance tests were written to ensure this | 01:48 |
| wallyworld | easy to change some tests | 01:48 |
| wallyworld | i already updated the juju ones | 01:48 |
| * thumper nods | 01:48 | |
| wallyworld | and the only bad feedback so far has been due to a different bug | 01:49 |
| wallyworld | feedback from openstack folks | 01:49 |
| wallyworld | where we sometimes leave out stdout in the yaml | 01:49 |
| wallyworld | that's just a straight bug unrelated to action ids or such | 01:49 |
| * thumper nods | 01:50 | |
| wallyworld | stub: with --format, we don't really know what an action may or may not write to stdout etc (in addition to producing with action-set) so we display both. for plain output we print nicely, for yaml, it's as fields in the yaml doc | 01:51 |
| stub | Right. The action can tell you what it is going to do though | 01:51 |
| wallyworld | it can but who's to say an action won't do both | 01:52 |
| wallyworld | if there's no stdout we don't print it | 01:52 |
| stub | An action can tell you that too if you let it | 01:52 |
| wallyworld | and it there's no structured data we don't print that | 01:52 |
| wallyworld | but people have raised bugs if a field in the yaml is missing regardless of f it's empty | 01:53 |
| wallyworld | so in the yaml we're thinking well include all fields | 01:53 |
| stub | I'm not too fussed on including all fields, as personally I'd consider it legacy behavior. If I want YAML output, I'll format and spit it out myself, avoiding the limitations JUJU imposes on structured output (keys with a restricted charset, values only being strings) | 01:56 |
| wallyworld | +1 | 01:56 |
| wallyworld | thumper: i like your bug comment, let's see what happens from it. however, their main issue (one of them) is with the controller itself as it can't reach the charm store to download a charm | 01:57 |
| thumper | well doesn't the controller have the proxies set internally? | 01:57 |
| wallyworld | we don't do anything different for k8s vs iaas as far as i know. what do we do for iaas | 01:58 |
| wallyworld | but for k8s, i think we would need to pass in thje proxies via pod env vars | 01:58 |
| wallyworld | for the controller | 01:58 |
| thumper | is the controller in k8s or iaas? | 01:59 |
| wallyworld | in their case? i assume it's all k8s? not sure | 01:59 |
| thumper | does a k8s controller have a proxy updater worker? | 01:59 |
| thumper | as long as it does, the internal proxies are set | 01:59 |
| * wallyworld checks | 02:00 | |
| thumper | as long as there isn't any caas specific behaviour in the proxy updater worker | 02:00 |
| wallyworld | thumper: no, that's currently in the iaas only worker list | 02:01 |
| thumper | well... that seems like a bug | 02:01 |
| wallyworld | yeah, seems so | 02:01 |
| wallyworld | we were initially quite selective about what we ran up in a caas controller | 02:01 |
| thumper | also... k8s controller for the client? | 02:01 |
| wallyworld | and networking/proxies we wanted to be cautious about | 02:02 |
| wallyworld | what do you mean "k8s controller for the client"? | 02:02 |
| thumper | it was an exclamation but with limited info due to public channel :) | 02:02 |
| === si is now known as Guest5009 | ||
| hpidcock | wallyworld: `juju bootstrap --logging-config "<juju.worker.diskmanager>=TRACE"` is that how you enable trace logging on a specific module? | 02:57 |
| wallyworld | --logging-config "<root>=INFO;juju.worker.diskmanager=TRACE" | 02:58 |
| wallyworld | only <root> needs <> | 02:58 |
| hpidcock | awesome thankyou | 02:58 |
| wallyworld | ; separator | 02:58 |
| wallyworld | can add as many packages as you want | 02:59 |
| hpidcock | timClicks: RE: https://bugs.launchpad.net/juju/+bug/1847128 how long did you wait for it to finish initialising? | 03:14 |
| mup | Bug #1847128: [2.7] ceph-osd stuck in "agent initializing" <juju:New for hpidcock> <https://launchpad.net/bugs/1847128> | 03:14 |
| timClicks | hpidcock: I deployed then ran juju-wait for ~30 mins | 03:15 |
| hpidcock | oh wow | 03:15 |
| timClicks | sent SIGINT then checked status | 03:15 |
| hpidcock | yeah mine resolved within 5min | 03:15 |
| hpidcock | really weird | 03:15 |
| timClicks | is it possible that the snap build is somehow different than your env? | 03:20 |
| hpidcock | yeah that's what I'm thinking | 03:21 |
| timClicks | new tutorial up https://discourse.jujucharms.com/t/2189 | 04:36 |
| timClicks | worthy of some peer review if anyone has a minute or 2 | 04:36 |
| babbageclunk | wallyworld: review plz? https://github.com/juju/juju/pull/10704 | 04:51 |
| babbageclunk | I'm just testing it locally | 04:51 |
| wallyworld | babbageclunk: looking | 05:05 |
| babbageclunk | wallyworld: actually, trying it out it's really obnoxious - changing it to log once and then only every 30s | 05:06 |
| wallyworld | ok | 05:06 |
| babbageclunk | wallyworld: I have to drop now, I've sent a status update. Could you babysit the PR and build the snap? | 05:32 |
| babbageclunk | wallyworld: oop, looks like you might be on the phone - ping me on telegram if you need me. | 05:34 |
| wallyworld | babbageclunk: will do, i'll baby sit ty | 05:41 |
| wallyworld | yeah otp | 05:42 |
| === exsdev0 is now known as exsdev | ||
| timClicks | wallyworld: thought you might like to know https://discourse.jujucharms.com/t/new-emoji-available/2190 | 05:55 |
| hpidcock | wallyworld: PR for you to check out https://github.com/juju/juju/pull/10705 | 05:56 |
| wallyworld | hpidcock: will do | 05:56 |
| wallyworld | timClicks: pretty | 05:56 |
| timClicks | wallyworld: I know how much you love social media | 05:57 |
| wallyworld | oh joy | 05:57 |
| hpidcock | it's on 2.6 because I don't know if we want it there or in develop. Let me know. | 05:57 |
| wallyworld | hpidcock: so you found a real issue | 05:57 |
| wallyworld | we can land in 2.6 just in case | 05:57 |
| hpidcock | well 2.6.10 would be broken | 05:58 |
| wallyworld | as we have probs broken 2.6 | 05:58 |
| wallyworld | yep | 05:58 |
| wallyworld | we may need to halt 2.6.10 | 05:58 |
| hpidcock | essentially all the matching logic is weird | 05:58 |
| hpidcock | and it probably never worked as it originally was intended | 05:58 |
| hpidcock | I just happened to break it | 05:59 |
| wallyworld | so it worked by coincidence | 05:59 |
| hpidcock | more | 05:59 |
| hpidcock | yeah | 05:59 |
| wallyworld | joy | 05:59 |
| wallyworld | hpidcock: so the core issue was with dealing with planBlockInfo? | 06:03 |
| hpidcock | yeah, if it didn't match it would just skip checking if any other unique fields match | 06:04 |
| wallyworld | i thought plan info was just for oracle, hmmmm | 06:06 |
| wallyworld | also i just left a comment, see what you thimnk | 06:06 |
| wallyworld | hpidcock: got to go deal with something, i'll check PR again soon. need to digest the changes as it is quite fragile | 06:11 |
| hpidcock | wallyworld: yeah agreed. I don't think it will break unless we are matching with some non-unique values... which would be concerning. | 06:12 |
| babbageclunk | I don't think the golangci-lint people understand what deprecated means. | 06:37 |
| babbageclunk | oh no, I take it back | 06:38 |
| wallyworld | hpidcock: i left another comment on the PR, trying to grok the logic. is there an example of how it failed and how we fixed it? | 07:43 |
| wallyworld | would simply removing the continues be enough, without the loop logic changes? | 07:44 |
| wallyworld | it would be good to see what we were matching on to understand what tripped up the logic | 07:44 |
| stickupkid | babbageclunk, haha | 08:17 |
| nammn_de | is there a fast way to force a bootstrap to a specific juju version? | 08:32 |
| nammn_de | e.g. client is 2.7, is there a way to force 2.6 or do i have to checkout 2.6 first? | 08:32 |
| timClicks | --agent-version | 08:33 |
| nammn_de | timClicks: my client then returns he can only bootstrap 2.7 | 08:33 |
| achilleasa | nammn_de: snap install juju? | 08:38 |
| timClicks | nammn_de: reading the help text more carefully, it looks like this is only possible within the same minor release | 08:38 |
| timClicks | nammn_de: snap refresh --channel=2.6/stable juju | 08:38 |
| nammn_de | timClicks: thanks achilleasa: hmm yeah good idea, is usefull :D | 08:38 |
| achilleasa | nammn_de: just be careful with paths, to be sure '/snap/bin/juju bootstrap...' | 08:39 |
| timClicks | nammn_de: you can see which channels are available via `snap info juju` | 08:40 |
| nammn_de | great thanks guys works like a charm, probably setting an alias for the snap juju to not get confused :D | 08:42 |
| achilleasa | nammn_de: "works like a charm"... I see what you did there ;-) | 08:44 |
| stickupkid | OR just setup go path to take precedence over snap path | 08:46 |
| nammn_de | stickupkid: luckily it already does, puh | 08:46 |
| nammn_de | 😊 | 08:46 |
| stickupkid | :D | 08:47 |
| nammn_de | someone might wanna take a look at this "small change"? https://github.com/juju/juju/pull/10685 | 09:51 |
| Fallenour | hey icey manadart have you guys seen something like this before: https://conn-check.readthedocs.io/en/latest/tutorial-part-3.html | 09:55 |
| Fallenour | $ # all checks on all units | 09:55 |
| Fallenour | $ juju run --service my-service-conn-check 'actions/run-check' | 09:55 |
| Fallenour | $ # all checks on just unit 0 | 09:55 |
| Fallenour | $ juju run --service my-service-conn-check/0 'actions/run-check' | 09:55 |
| Fallenour | $ # nagios (not including no-nagios) checks on all units | 09:55 |
| Fallenour | $ juju run --service my-service-conn-check 'actions/run-nagios-check' | 09:55 |
| Fallenour | $ # nagios (not including no-nagios) checks on just unit 0 | 09:55 |
| Fallenour | $ juju run --service my-service-conn-check/0 'actions/run-nagios-check' | 09:55 |
| Fallenour | Sorry for the wall of text everyone. | 09:55 |
| Fallenour | rick_h, your thoughts? | 09:55 |
| nammn_de | achilleasa stickupkid in upgrade code, do we have to expect someone upgrading from really old juju version (e.g. 2.0) using upstart to new 2.7 with systemd? | 10:31 |
| hpidcock | wallyworld: removing the loops changes the priority intention that was originally there. I'll add a unit test tomorrow that matches what we were seeing. | 10:33 |
| hpidcock | removing the continues* | 10:33 |
| wallyworld | ok | 10:33 |
| wallyworld | i'd be interested in the aws dev info etc as well | 10:33 |
| wallyworld | to see how the bug manifested | 10:33 |
| wallyworld | i need to look at code again. i did tink removing continues was all that was needed | 10:34 |
| nammn_de | achilleasa: what command do I have to run to upgrade from 2.6 to current development juju with the cli? Is it "juju upgrade-controller" will the upgrade steps defined run then? | 10:59 |
| achilleasa | nammn_de: yes, just make sure that you use the correct cli version when upgrading | 11:00 |
| nammn_de | e.g. for correct understanding: "juju2.6 bootstrap ...", followed by "$GOPATH/bin/juju upgrade-controller --build-agent" will trigger the upgradesteps to 2.7, right? | 11:01 |
| achilleasa | I think that should do it... | 11:04 |
| nammn_de | wallyworld: does it still work for you to attach to controllers with delve? For me it returns "could not find debug info", did we change a flag while building? | 11:17 |
| nammn_de | wallyworld: ahh I need to set the debugflag to not omit debug information | 11:19 |
| nammn_de | during trying to running the upgrade steps my model died between 2.6 and 2.7 because `ModelLogfileMaxSize` was added but didn't exist or so. Is that expected? For now I "just" added an upgradeStep to add that value | 12:33 |
| hml | nammn_de: there should never be a failure during upgrade like that | 12:36 |
| hml | nammn_de: does it happen upgrading without your change as well? | 12:37 |
| nammn_de | hml: maybe i did some weird stuff, let me try that | 12:38 |
| nammn_de | hml: yeah seems to be the case Let me get the pastebiun | 12:47 |
| nammn_de | hml: what I did, bootstrap snap juju 2.6, get local code 2.7-beta, run `juju upgrade-controller --build-agent` without my upgrade code. Leads to the pastebin | 12:50 |
| nammn_de | server is down, given status, ssh into and digging into machine log leads to: https://pastebin.canonical.com/p/JpHV2fMSXH/ | 12:50 |
| hml | nammn_de: that’s a bug. was this 2.6.8? | 12:51 |
| hml | or 2.6.9 i guess the released version is | 12:51 |
| nammn_de | 2.6.9 | 12:52 |
| nammn_de | from 2.6.9 -> 2.7-beta1 | 12:52 |
| rick_h | morning juju | 12:53 |
| nammn_de | rick_h: morning | 12:53 |
| nammn_de | hml: trying with the newest develop version, my last one was 9 commits behin or so | 12:54 |
| nammn_de | yeah same, leads to machine down | 12:55 |
| Fallenour | morning rick_h | 13:34 |
| Fallenour | off the wall question, but does anyone have a diagram which can visually represent the place terraform holds in the CI/CD process | 13:34 |
| rick_h | 758196 | 14:04 |
| hml | manadart: i wonder if that defaultSpaceName arg was just poorly named. That the intention was to have a default space on a per machine basis: https://github.com/juju/juju/blob/develop/network/containerizer/bridgepolicy.go#L288 but the code never got there | 14:14 |
| nammn_de | How do I upgrade a local agent from a model? | 15:49 |
| hml | nammn_de: did you run juju upgrade-juju after upgrade-controller? | 15:56 |
| hml | nammn_de: every model is upgraded by the user the after controller is upgraded | 15:56 |
| nammn_de | hml: does upgrade juju copies the jujud from the controller? | 15:57 |
| hml | nammn_de: there is an agent (tools) tar ball which is copied and unpacked | 15:57 |
| hml | on each juju machine when juju upgrade-juju is called on the model | 15:58 |
| pmatulis1 | hml, upgrade-model, upgrade-model :) | 15:58 |
| nammn_de | if i do "juju upgrade-controller --build-agent" my local one is deployed and installed, it also updates the tar ball, which then gets redistributed, right? | 15:58 |
| hml | nammn_de: yes, but you only after you upgrade the model. that line only upgrade the controller model | 15:59 |
| nammn_de | hml: got it, thanks 🦸♀️ | 15:59 |
| nammn_de | someone want to take a look and mind giving some feedback? https://github.com/juju/juju/pull/10696 I am not 100% safe about the upgrade steps. It did seem to work, but cannot guarantee as they are not automated (?) I only added the upgrade step as it did not work otherwise. But it's all written in the commends in the PR | 16:57 |
| stickupkid | nammn_de, send an email to thumper and ask him to review, I wonder why he didn't add them | 17:02 |
| nammn_de | stickupkid: will do! | 17:03 |
| timClicks | Fallenour: it would be really interesting to see whether that subordinate still functions | 19:00 |
| timClicks | weirdly, there are no links in that docs site that point to the source code repository.. which makes the source code hard to inspect :/ | 19:02 |
| === AvondZon_ is now known as AvondZon | ||
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!