| menn0 | woot! the logging-to-mongodb feature branch has landed | 01:07 |
|---|---|---|
| anastasiamac | menn0: wow! congrats :D well done! | 01:08 |
| menn0 | anastasiamac: cheers | 01:08 |
| menn0 | I just checked the size of the diff to master: 3,500 additions and 852 deletions | 01:08 |
| anastasiamac | menn0: would love to know of ur experiences in terms of wirtting ci tests for it as well as user-facing documentation | 01:08 |
| anastasiamac | menn0: when did u create the branch? (for how long have u had it running?) | 01:09 |
| menn0 | anastasiamac: there is no user facing documentation, b/c from the user's perspective nothing changes | 01:09 |
| menn0 | anastasiamac: it's just a necessary change for JES ... and it's a little less fragile than all-machines.log | 01:10 |
| menn0 | anastasiamac: not sure how old the branch is... around mid April | 01:11 |
| wallyworld | except we can't use a text editor when mongo dies :-( | 01:11 |
| anastasiamac | menn0: so there r no changes to configuration | 01:11 |
| anastasiamac | me? | 01:11 |
| menn0 | wallyworld: you still have a text log file of all received logs per state server | 01:11 |
| wallyworld | which we need to accumulate by hand and which are not interleaved | 01:12 |
| menn0 | wallyworld: that's the fallback | 01:12 |
| wallyworld | yeah, i know, i'm just grumpy | 01:12 |
| menn0 | wallyworld: yes, if there are multiple state servers | 01:12 |
| wallyworld | no i mean all machines log also contained logs from nodes | 01:12 |
| wallyworld | all interleaved | 01:12 |
| wallyworld | so you could easily see sequencing | 01:13 |
| menn0 | wallyworld: yeah I understand | 01:13 |
| wallyworld | i often ssh into a state server and grep that file | 01:13 |
| wallyworld | far easier than debug log | 01:13 |
| menn0 | wallyworld: well the next thing is to expand debuglog's capabilities then | 01:13 |
| wallyworld | yeah | 01:13 |
| wallyworld | sorry for grumping, will just make my life harder now | 01:14 |
| wallyworld | i understand the reasons for it | 01:14 |
| menn0 | wallyworld: for example, selecting time ranges is largely implemented on the server side but needs client side support | 01:14 |
| wallyworld | it's the lack of exploring which is the main thing | 01:14 |
| wallyworld | easier to just flitter around in a text file | 01:14 |
| anastasiamac | menn0: weren't u at one stage considering to make log output destination (file, db, etc) configurable? | 01:16 |
| menn0 | anastasiamac: no I don't belive so. it's hard to effectively handle logs for multiple environments in a single file. | 01:17 |
| menn0 | wallyworld, anastasiamac: with the way it's the feature is structured it wouldn't be too hard to generate a all-machines.log or even separate log files per env from the DB. | 01:18 |
| wallyworld | yay | 01:19 |
| menn0 | wallyworld, anastasiamac: with the bonus that it doesn't involve rsyslogd | 01:19 |
| wallyworld | even better :-D | 01:19 |
| menn0 | we'd just need a worker which tails the DB like the debuglog API does and generates the file | 01:19 |
| menn0 | all the pieces are decomposed in such a way that this wouldn't be hard at all | 01:20 |
| wallyworld | friday labs :-) | 01:21 |
| anastasiamac | menn0: this is great! and the feature (as well as its design) would b an awesome demo for team meeting - again tyvm for the feature :) | 01:22 |
| menn0 | anastasiamac: yep good idea | 01:46 |
| * menn0 is having lunch so sorry for the delay | 01:46 | |
| wallyworld | waigani: hey, did you see the maas output on that bug? looks like maas is doing the right thing? | 02:21 |
| waigani | wallyworld: ah really? hmph. well there goes my theory. At least we can test with that as mock output - try to reproduce it in a test. | 02:22 |
| wallyworld | waigani: well, i think it is correct, only took a quick look | 02:23 |
| wallyworld | but we have the output | 02:23 |
| waigani | wallyworld: I'll try to take a look today, but I need to finish off this branch - I'm on leave for the next 4 days | 02:23 |
| waigani | great that we've got the output though, should be helpful | 02:24 |
| wallyworld | waigani: ok, see how you go, let me know where you get to | 02:32 |
| waigani | wallyworld: will do, I'm not far off... | 02:32 |
| waigani | wallyworld: looks like gomaasapi wan't reading the substatus. Here's a PR to fix that: https://code.launchpad.net/~waigani/gomaasapi/substatus/+merge/264371 | 04:09 |
| wallyworld | waigani: ty, looking | 04:09 |
| waigani | wallyworld: I've also added that to the bug as a comment. | 04:09 |
| wallyworld | waigani: so that's a fix for the test service in gomaasapi, we still need to fix juju as well right? | 04:10 |
| waigani | wallyworld: once that gets signed off, it should just be a matter of bumping the dependancies.tsv | 04:11 |
| waigani | wallyworld: and probably adding a test | 04:11 |
| wallyworld | waigani: how? the pr above doesn't fix anything | 04:11 |
| wallyworld | it fixes a test server | 04:11 |
| waigani | wallyworld: ugh, facepalm | 04:12 |
| waigani | wallyworld: let me go back | 04:12 |
| wallyworld | ok | 04:12 |
| waigani | wallyworld: right, yeah sorry so the juju side is still to come | 04:13 |
| wallyworld | sure | 04:13 |
| waigani | but it looks like that's the problem, as you can see from that test | 04:14 |
| waigani | wallyworld: the one thing that makes me nervous is that we are flattening two dimensions into one | 04:14 |
| wallyworld | let me look at the maas output | 04:14 |
| waigani | wallyworld: as juju status doesn't have any concept of a 'substatus' so, we just need to be careful how we decide which status/substatus gets reported | 04:15 |
| wallyworld | waigani: so the maas "deployment_status" api call returns a string doesn't it? | 04:19 |
| wallyworld | "Failed Deployment" or "Deployed" etc | 04:19 |
| waigani | wallyworld: yeah | 04:19 |
| wallyworld | so we don't need to worry about interpretting status vs subststus | 04:19 |
| wallyworld | is it only older maas deployemnts without the new api call? | 04:20 |
| waigani | wallyworld: let me look and I'll get back to you | 04:20 |
| wallyworld | ok | 04:21 |
| waigani | wallyworld: just seeing how we dial up that status from the juju side | 04:21 |
| wallyworld | juju calls deployment_status | 04:21 |
| wallyworld | well, it does while it waits for the node to come up | 04:22 |
| waigani | wallyworld: add this to tip of 1.24: http://pastebin.ubuntu.com/11853823/ | 04:31 |
| waigani | wallyworld: it passes | 04:32 |
| wallyworld | looking | 04:32 |
| waigani | wallyworld: and yeah we call deployment_status in maas/environ.go deploymentStatusCall | 04:32 |
| waigani | wallyworld: so it's reporting the correct error | 04:33 |
| wallyworld | waigani: i'm not sure that's all correct - that test could pass if the status stays "pending" | 04:34 |
| wallyworld | i don't see how we're testing the maas status interpretation | 04:34 |
| wallyworld | and adding a test against a modified test server in gomaas doesn't really prove anything | 04:35 |
| waigani | wallyworld: right, fair point. | 04:36 |
| waigani | wallyworld: how about I checkout 1.24.1 (same as in bug), revert changes to gomaasapi and work on improving this test? | 04:36 |
| wallyworld | try and reproduce a test failure (write a new failing test). then fix juju so the test passes | 04:37 |
| wallyworld | TDD and all that | 04:37 |
| wallyworld | target the test to the specific issue | 04:37 |
| waigani | wallyworld: actually I did run that test on the unmodifed server - and if you comment out the "substatus" line, it's fails - as expected | 04:37 |
| wallyworld | other more general bootstrap tests could also later need changing, but there should be a targetted, specific test done first | 04:37 |
| wallyworld | yes, but that's because the test is relying on the behaviour of hte test server | 04:38 |
| wallyworld | and the test server reports "Deployed" - status 6 | 04:38 |
| wallyworld | in the absense of sub status 11 | 04:38 |
| waigani | wallyworld: I can't get it to fail, I've written a summary on the bug. I've asked to look at the logs. | 06:04 |
| wallyworld | did you test with real maas? | 06:05 |
| wallyworld | or vmaas | 06:05 |
| waigani | wallyworld: I was just about to say, that's the next step | 06:05 |
| wallyworld | it won't fail if you just test with the gomaas test server | 06:06 |
| wallyworld | unless the test server mimics the maas output | 06:06 |
| wallyworld | which it doesn't appear to | 06:06 |
| waigani | wallyworld: yeah, that's what I did - as explained in the bug | 06:06 |
| wallyworld | ok, i'll read the bug :-) | 06:07 |
| wallyworld | waigani: thanks for looking into it, enjoy the time off. i'll see what we can make of it | 06:09 |
| waigani | wallyworld: thanks. I'm keen to know the cause. | 06:11 |
| wallyworld | i'll update the bug | 06:11 |
| wallyworld | when we find it | 06:11 |
| waigani | ah I was just about to | 06:11 |
| waigani | wallyworld: updated | 06:12 |
| wallyworld | waigani: it could be that an old api call to get status is being made, and hence sub status is ignored - that would be my guess | 06:13 |
| wallyworld | anyways, we'll see | 06:13 |
| === liam_ is now known as Guest18768 | ||
| TheMue | fwereade: interested in a quick chat ;) we're alone today | 09:01 |
| perrito666 | fwereade: hi, please dont forget my branch if you have the chance | 09:31 |
| fwereade | perrito666, sorry, the diff looks mangled? would you update and try again please? | 11:53 |
| perrito666 | fwereade: sure | 12:33 |
| perrito666 | mm, says rbt it cannot reach reviewboard, odd | 12:35 |
| perrito666 | fwereade: sorry rbt is playing dumb with me | 13:04 |
| TheMue | voidspace: heya Michael, everything alright with your car? | 13:38 |
| sinzui | katco: perrito666 : there is a new reression in master, bug 1473461 Maybe we want to backout a branch becaue the fix is to write a darwin-specific ./cmd/jujud/util/password/password_darwin.go | 14:02 |
| mup | Bug #1473461: OSX/darwin builds fail: undefined: password.EnsureJujudPassword <ci> <osx> <regression> <juju-core:Triaged> <https://launchpad.net/bugs/1473461> | 14:02 |
| mup | Bug #1473450 opened: upgrade-juju from 1.18 to 1.20.4 leaves some agents down <juju-core:New> <https://launchpad.net/bugs/1473450> | 14:03 |
| mup | Bug #1473461 opened: OSX/darwin builds fail: undefined: password.EnsureJujudPassword <ci> <osx> <regression> <juju-core:Triaged> <https://launchpad.net/bugs/1473461> | 14:03 |
| === kadams54 is now known as kadams54-away | ||
| ericsnow | TheMue: was my review of http://reviews.vapour.ws/r/2129/ helpful? | 14:10 |
| TheMue | ericsnow: very, thanks. I'm working on it. | 14:10 |
| ericsnow | TheMue: cool :) | 14:10 |
| perrito666 | bogdanteleaga: that password thing is yours :) | 14:11 |
| ericsnow | TheMue: I'll try and give you a quick turnaround as soon as you have any comments or updates to the patch | 14:11 |
| TheMue | ericsnow: there has been a longer discussion about the EntityWatcher before. the first Addresser approach used the IP address values. but then Dimiter said it would make sense to use common.LifeGetter and common.Remover. | 14:12 |
| mup | Bug #1473466 opened: When using openstack as provider lxc containers get a hardcoded ip <sts> <juju-core:New> <https://launchpad.net/bugs/1473466> | 14:12 |
| TheMue | ericsnow: so we needed tags for ip addresses, an earlier chnage. | 14:12 |
| TheMue | ericsnow: and this code is mostly based on http://paste.ubuntu.com/11803914/ | 14:12 |
| ericsnow | TheMue: ah | 14:13 |
| TheMue | ericsnow: should have written more into the description *lol* | 14:13 |
| ericsnow | TheMue: it just struck me as odd that the EntityWatcher was part of this patch :) | 14:13 |
| ericsnow | TheMue: yeah, how terrible of you <wink> | 14:14 |
| TheMue | ericsnow: it rushed in as a side-effect. would have been worth an extra change, yes | 14:14 |
| TheMue | hehe | 14:14 |
| ericsnow | TheMue: not really a blocker, but putting multiple effectively distinct changes in a single patch can be a problem every long once in a while (in this case not a big deal) | 14:16 |
| TheMue | ericsnow: what? you need more than one commit to implement <insert favourite complex feature here>? :D | 14:18 |
| ericsnow | TheMue: well, not me personally ;) just this guy I know (okay it's me) | 14:19 |
| TheMue | :,) | 14:19 |
| mup | Bug #1473470 opened: Windows cannot ensurePassword <ci> <regression> <windows> <juju-core:Triaged> <https://launchpad.net/bugs/1473470> | 14:42 |
| alexisb | hello akhavr welcome! | 14:54 |
| akhavr | alexisb: :) | 14:54 |
| perrito666 | ashipika: around? | 14:56 |
| ashipika | perrito666: here | 14:57 |
| perrito666 | just something I found and might bite you in the future | 14:58 |
| ashipika | perrito666: do tell, please | 14:58 |
| perrito666 | ashipika: priv | 14:58 |
| === kadams54 is now known as kadams54-away | ||
| perrito666 | sinzui: around? | 17:20 |
| sinzui | perrito666: I am | 17:22 |
| perrito666 | sinzui: I am trying to use boot_context in my CI test | 17:22 |
| perrito666 | i feel a bit of a lack of clarity regarding what should the arguments be | 17:23 |
| sinzui | perrito666 in deploy_stack.py deploy_job() -> deploy_job_parse_args(), _deploy_job() -> boot_context() | 17:26 |
| sinzui | perrito666: the parse args will have descriptions | 17:27 |
| perrito666 | ah, excelent, thank you | 17:27 |
| sinzui | perrito666: I think we want to add a ricj docstring to boot_context() to encourage it's use | 17:27 |
| sinzui | perrito666: I will take a stab at the docstring in a few minutes, and pass it for review for you and maybe abentley | 17:28 |
| mup | Bug #1473517 opened: juju environment not usable after the upgrade <juju-core:New> <https://launchpad.net/bugs/1473517> | 17:54 |
| perrito666 | sinzui: back, sorry, 2 things, 1) what is bootstrap_host supposed to be and 2) job name can be set to something arbitrary or do I need to add it as an arg? | 18:37 |
| sinzui | job_name is arbitrary yes, but it is alos unique so that the script and base env can be run in parallel | 18:38 |
| perrito666 | and bootstrap_host? | 18:39 |
| sinzui | perrito666: bootstrap_host is not required. it is a placement directive for bootstraping manual or maas | 18:39 |
| perrito666 | None will suffice or you are expecting it to be a given type? | 18:40 |
| sinzui | perrito666: yep | 18:40 |
| perrito666 | ok this function is too long | 18:44 |
| perrito666 | sinzui: ok, almost there: agent_url and agent_stream? | 18:49 |
| sinzui | perrito666: agent_stream == agent-stream in config. None means don't override | 18:51 |
| sinzui | perrito666: agent_url == agent-metadata-url in config. None means don't override | 18:51 |
| sinzui | perrito666: We use these often and will be required to run revision tests in parallel when we need concurent streams | 18:52 |
| sinzui | well we do, which is why the configs force the testing streams | 18:52 |
| perrito666 | sinzui: so you are saying I should make these args | 18:53 |
| sinzui | perrito666: all of those things need to be args becauee we may need to run the test with manual and we certainly will be running with streams created for the test | 18:55 |
| perrito666 | sinzui: I find then odd that those are not part of add_basic_testing_arguments | 18:56 |
| mbruzek | sinzui: Do you know if anyone has an RPM to install juju client? | 18:57 |
| sinzui | perrito666: the only arg from that may be dropped from (job_name, client, bootstrap_host, machines, series, agent_url, agent_stream, log_dir, keep_env, upload_tools) is upload_tools because we discovered this week that we cannot fallback to upload-tools with the new os and arch requirements | 18:57 |
| sinzui | mbruzek: I have not seen one. mbruzek Since client relies on ubuntu-ness I doubt anyone succeeded | 18:58 |
| perrito666 | sinzui: ack | 18:58 |
| sinzui | mbruzek: the centos client will work for centos https://launchpad.net/juju-core/+milestone/1.24.2 | 18:59 |
| sinzui | mbruzek: only centos7 actually. I thnk an error is quickly raise of the host os/series is not recognised | 18:59 |
| mbruzek | sinzui: Marco and I just tried to alien the debian packages, it didn't work... yet | 19:00 |
| sinzui | mbruzek: I am sure the deps wont work becaue mongo is not packaged to meet Juju's needs | 19:01 |
| mbruzek | OK Thanks sinzui | 19:01 |
| abentley | sinzui or jog: Could you please review https://code.launchpad.net/~abentley/juju-ci-tools/s3-download/+merge/264450 ? | 19:15 |
| sinzui | abentley: looking | 19:15 |
| abentley | sinzui: thanks. | 19:16 |
| abentley | sinzui, perrito666: I think that --upload-tools is nice to have for devs who are testing their self-built jujus. | 19:17 |
| sinzui | abentley: I can and maybe should install s3cmd on all slaves | 19:17 |
| sinzui | abentley: My one mac has it | 19:17 |
| abentley | sinzui: Oh. Well, if we do that, we can rip out s3 support for the workspace runner. | 19:17 |
| sinzui | abentley: then I wont because I like what you have written | 19:18 |
| abentley | sinzui: I do like the idea of isolating the resource retrival/publication from the testing step. | 19:19 |
| sinzui | yeah, this is awkwardly coupled. Isn't this because we haven't updated the build job to also upload to s3 yet? | 19:20 |
| abentley | sinzui: It's because we haven't done download support for workspace-runner. | 19:21 |
| abentley | sinzui: We can scp files, which is useful, but the remote client can't download files from s3. | 19:21 |
| sinzui | abentley: This command now works on the osx-slave | 19:32 |
| sinzui | s3cmd -c cloud-city/juju-qa.s3cfg ls s3://juju-qa-data/ | 19:32 |
| abentley | sinzui: Okay, I can switch it over to use s3cmd. | 19:33 |
| sinzui | abentley: I can add s3cmd to windows, but I don't want to give it credentials. I think I put myself in a corner | 19:34 |
| abentley | sinzui: If the plan is to ultimately have our windows jobs uploading everything directly to s3, that would be a problem. | 19:35 |
| abentley | sinzui: Obviously, we can have separate win credentials. | 19:36 |
| sinzui | abentley: The win-client-deploy job need the installer built by build-win-client.The current rule is the ci job procurs the installer under test and delivers it to the win machine. | 19:43 |
| abentley | sinzui: Right. Basically the same pattern I was following with run-osx-client. | 19:46 |
| sinzui | yes. In egnlish, test this <client> using with <script> | 19:46 |
| abentley | sinzui: But then at the end, ci-director uploads the console log to s3. | 19:47 |
| sinzui | yes, and that is also true for windows. | 19:48 |
| abentley | sinzui: I'm imagining that eventually, we'll want workspace-runner to be able to do that. | 19:48 |
| sinzui | r=me abentley | 19:51 |
| abentley | sinzui: I have updated the code so that s3cmd is run remotely. Do you want that version instead? | 19:52 |
| sinzui | abentley: I got scared. | 19:52 |
| sinzui | abentley: I trust the osx machine but not the win machines. I think it is easier to maintain the idea that testing host cannot retrieve the things under test | 19:54 |
| abentley | sinzui: Okay. | 19:54 |
| mbruzek | sinzui: Those Juju binary files in the tar.gz worked on CentOS7. Thank you! | 20:24 |
| mbruzek | marcoceppi: ^ | 20:24 |
| sinzui | mbruzek: I should hope so, we officially distribute them | 20:25 |
| mbruzek | sinzui: Is there a test process for installing Juju on CentOS7 ? | 20:25 |
| mbruzek | to verify they work? | 20:25 |
| sinzui | mbruzek: no. We are still tinkering with what we recommend | 20:26 |
| sinzui | mbruzek: I verified them using a centos machine I provisiioned in aws. I could boostrap an aws env | 20:26 |
| mbruzek | sinzui: thanks. I did not think they would not work, I was just letting you know I was able to get them to work. Thanks. | 20:27 |
| sinzui | mbruzek: this will be tested automatiically later this month. | 20:27 |
| * mbruzek regrets the double negative | 20:27 | |
| mbruzek | sinzui: Do you know any work to get the tools build for centos7? We could not deploy a centos7 charm because there were no tools | 20:29 |
| sinzui | mbruzek: you don't need to build them, they are officially published. The problem is that no one created a way to manually provision or bootstrap them. mgz and and I are still playing with a non-maas way to provision an aws machine with cloud-init | 20:31 |
| sinzui | mbruzek: https://streams.canonical.com/juju/tools/streams/v1/com.ubuntu.juju:released:tools.sjson lists the 1.24.0 and 1.24.2 agents | 20:31 |
| sinzui | mbruzek: This is the most recent centos7 agent https://streams.canonical.com/juju/tools/releases/juju-1.24.0-centos7-amd64.tgz | 20:32 |
| mbruzek | sinzui: marcoceppi downloaded a centos7 image from AWS that appeared to include cloud-init https://aws.amazon.com/marketplace/pp/B00O7WM7QW/ref=srh_res_product_title?ie=UTF8&sr=0-2&qid=1436560475610 | 20:34 |
| mbruzek | Starting with CentOS-7 we now include cloud-init support in all CentOS AMI's, | 20:35 |
| sinzui | mbruzek: yep that is what mgz and I plan to use | 20:35 |
| sinzui | that is also the machine I tested the client with | 20:35 |
| mbruzek | if I downloaded those tools could I use the --upload-tools flag in Juju? | 20:36 |
| sinzui | mbruzek: no you cannot. that simple and practical feature was not built first :( | 20:38 |
| sinzui | mbruzek: I think upload-tools or manual provisioning is the most sensible first level support, instread we got maas. The official way to write charm for centos is to get a maas with handmade centos images first :( | 20:39 |
| mbruzek | sinzui: OK well if you and mgz are working on this I would be interested in helping getting amazon working. | 20:40 |
| sinzui | mbruzek: we will give you an update next week. | 20:40 |
| mbruzek | sinzui: I spoke with a potential customer they like Ubuntu but have some CentOS that they can not work around and specifically asked if Juju would work with CentOS | 20:41 |
| mbruzek | sinzui: thanks for answering my questions | 20:42 |
| sinzui | mbruzek: maybe by 1.25. 1.24 is for cloud engineers who are willing cheat every step to make something work | 20:43 |
| mbruzek | sinzui: understood. Consider me interested and I understand we actually want them using Ubuntu. I would just like to demo CentOS without having to install maas | 20:44 |
| sinzui | mbruzek: me too. My last thought was to bootstrap an ubuntu env, bring up a centos machine, run a prep script to satisfy add-machine, then juju add-machine to let juju really add it. The prep script would provide fake commands or alternates to for the cloud-init to work. For example. Create the ubuntu user. create an lsb_release script that returns centos7 | 20:47 |
| perrito666 | can I mark comments as addressed someway in launchpad? | 21:08 |
| sinzui | perrito666: no, just comment that you are done | 21:37 |
| abentley | sinzui: I've successfully built osx client into (test) s3: https://console.aws.amazon.com/s3/home?region=us-east-1#&bucket=ws-runner-test&prefix=juju-ci/products/version-2873/build-osx-client%20%20/build-17/foo/ | 21:40 |
| sinzui | abentley: \o/ | 21:40 |
| xwwt | abently: good news | 21:40 |
| xwwt | abentley: good news | 21:40 |
| perrito666 | the fact that save produces something called "unsaved comment" in launchpad is a tad confusing | 22:58 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!