| hazmat | wallyworld_, looking at the file dan uploaded .. there's a wierd 5 minute gap http://paste.ubuntu.com/7502559/ | 00:10 |
|---|---|---|
| hazmat | where the agent restarts itself | 00:10 |
| hazmat | oh.. right after it reconfigures logging to WARN.. hence no log | 00:11 |
| sinzui | Is anyone about to help me diagnose a local kvm issue on a ppc64 machine in the lab? | 00:25 |
| sinzui | http://pastebin.ubuntu.com/7503638/ | 00:25 |
| davecheney | sinzui: my gut says you are missing a package | 00:28 |
| davecheney | juju-local does not depend on the entire set of packages required to boot ubuntu images | 00:29 |
| sinzui | thank you davecheney I will investigate | 00:29 |
| davecheney | (based on my experiences 6 weeks ago) | 00:29 |
| davecheney | something like ubuntu-cloud-template | 00:29 |
| davecheney | hang on | 00:29 |
| davecheney | i'll stop guessing | 00:29 |
| sinzui | davecheney, I did install the list of packages it suggested when I tried it first, but I understand the list might no be accurate | 00:29 |
| davecheney | possibly you need lxc-templates | 00:29 |
| sinzui | davecheney, lcv works fine. CI now tests lxc on ppc64el for every rev we change | 00:33 |
| davecheney | ok, i don't know jack about kvm | 00:34 |
| davecheney | it didn't even work on ppc when I was using it | 00:34 |
| davecheney | as all the machines i had access to were themselves inside kvm | 00:34 |
| davecheney | which might be the problem | 00:34 |
| sinzui | davecheney, yeah. I know kvm in kvm works. This is my last hope to get this working. I can see no cloud images were downloaded. Maybe there aren't ppc64el cloud-images suitable for this case | 00:38 |
| davecheney | wallyworld_: axw http://paste.ubuntu.com/7503679/ | 00:50 |
| davecheney | getting closer every day | 00:50 |
| wallyworld_ | farking awesome | 00:50 |
| axw | davecheney: nice | 00:51 |
| axw | hmm, what's that replicaset one | 00:51 |
| davecheney | replica set stuff is always busted | 00:51 |
| davecheney | sorry to whoever wrote it | 00:51 |
| davecheney | its really unreliable | 00:51 |
| axw | yeah I know, I've been working on making it more reliable | 00:51 |
| wallyworld_ | also on golang go :-( | 00:51 |
| axw | apparently still not | 00:51 |
| axw | davecheney: I have a fix for the utils one | 00:51 |
| wallyworld_ | davecheney: TestSecureClientNoAccess failure - that's because we're trying to connect to a non local address | 00:52 |
| axw | just about to approve | 00:52 |
| axw | wallyworld_: it's cos of $http_proxy | 00:52 |
| wallyworld_ | ah right | 00:52 |
| davecheney | https://bugs.launchpad.net/juju-core/+bug/1322392 | 00:52 |
| axw | I've got a CL to clear it | 00:52 |
| _mup_ | Bug #1322392: utils: test failure due to proxy <ppc64el> <juju-core:New> <https://launchpad.net/bugs/1322392> | 00:52 |
| wallyworld_ | great | 00:52 |
| davecheney | ^ sorry, just raised this, it's probably a due | 00:52 |
| davecheney | dup | 00:52 |
| axw | yeah I opened one yesterday | 00:53 |
| wallyworld_ | do we have a bug for TestConstraintsValidator? | 00:53 |
| sinzui | wallyworld_, no | 00:54 |
| wallyworld_ | ok, that's an easy fix anyway | 00:54 |
| axw | wallyworld_: can you please take a look at https://codereview.appspot.com/100730043/ to make sure I haven't buggered up the intent of the test | 00:55 |
| wallyworld_ | sure | 00:55 |
| axw | davecheney: seen this yet? https://code.google.com/p/gosmith/ | 00:57 |
| wallyworld_ | axw: looks fine | 00:57 |
| axw | wallyworld_: thanks | 00:57 |
| davecheney | axw: yeah, dmitry has been ripping the compiler a new one with that | 00:58 |
| davecheney | it's great | 00:58 |
| axw | now it's llgo's turn | 00:58 |
| sinzui | axw, Do you have any insights into this? http://pastebin.ubuntu.com/7503710/ | 01:00 |
| axw | sinzui: 'fraid not | 01:01 |
| sinzui | thank you axw | 01:01 |
| axw | about to get on a meeting, I will take a look at the kvm code later to see if there's somethign else you can run | 01:02 |
| sinzui | axw: I found the issue. | 01:14 |
| sinzui | uvt-kvm: error: no images found that match filters ['release=trusty', 'arc | 01:14 |
| sinzui | h=ppc64']. | 01:14 |
| davecheney | wap, wap, wah | 01:16 |
| davecheney | hang on | 01:18 |
| davecheney | is that 'cos the arch is wrong | 01:18 |
| davecheney | it should be ppc64el, or ppc64le | 01:18 |
| sinzui | davecheney, exactly what I suspect. | 01:19 |
| sinzui | I see ppc64el release | 01:20 |
| sinzui | http://cloud-images.ubuntu.com/releases/streams/v1/com.ubuntu.cloud:released:download.json | 01:20 |
| sinzui | davecheney, but lxc works so WTF | 01:20 |
| davecheney | sinzui: we have several functions inside juju that translate btween the various aliases | 01:20 |
| davecheney | it is possible lxc knows more than kvm | 01:20 |
| davecheney | or even that the lxc images have a different naming scheme | 01:21 |
| sinzui | I could test this by creating an alternate stream, but my weekend has started. I am not keen to start a project at this hour | 01:21 |
| sinzui | davecheney, before the ports were opened of for stilson-06, I hacked up a https server and the hosts file on the machine to help juju find the tools | 01:23 |
| axw | sinzui: any chance I could get direct access to the trusty CI machine that runs the unit tests? there's one in particular that I cannot reproduce anywhere else | 01:28 |
| sinzui | axw, absolutely. one moment while I install your key into one of them | 01:29 |
| sinzui | axw, who else is in tanzanite? | 01:31 |
| axw | sinzui: wallyworld_ and mgz | 01:31 |
| wallyworld_ | yes please | 01:32 |
| axw | davecheney: should be a couple less test failures on trunk now | 01:38 |
| davecheney | axw: i'm working on a mega patch that will remove 90% of the uses of DeepEquals in our tests | 01:40 |
| davecheney | IMO they are almost all wrong | 01:41 |
| axw | mmkay | 01:41 |
| davecheney | for example | 01:41 |
| davecheney | c.Assert(manifest, jc.DeepEquals, set.NewStrings(dummyManifest...)) | 01:41 |
| davecheney | this is a set | 01:41 |
| davecheney | order is unimportant | 01:41 |
| axw | set is a map underneath I think | 01:42 |
| axw | so DeepEquals will work anyway? | 01:42 |
| davecheney | logically your testing for set equality | 01:42 |
| davecheney | not ordering | 01:42 |
| sinzui | wallyworld_, axw, and mgz: I sent an email about accessing all the ppc64 env I manage | 01:48 |
| wallyworld_ | thanks | 01:48 |
| sinzui | wallyworld_, axw. I reported bug 1322401 about the issue. I will work on other tests next week | 01:57 |
| _mup_ | Bug #1322401: kvm on ppc64el cannot fine images <ci> <kvm> <local-provider> <ppc64el> <juju-core:Triaged> <https://launchpad.net/bugs/1322401> | 01:57 |
| axw_ | sinzui: sorry if you've been talking to me, my client didn't realise it was disconnected | 01:58 |
| === axw_ is now known as axw | ||
| sinzui | axw, I wasn't all is fine. and I didn't say the bug had to be fixed this week | 01:58 |
| axw | okey dokey | 01:59 |
| sinzui | oh, axw, or wallyworld_ I was supposed to rebuild 1.19.2 to make ppc64el and arm64 tools. Since ppc64el has so many names, should I always make the tool with a specific name? when the script sees "ppc64el" debs, make a "ppc64" tools? | 02:01 |
| axw | ppc64 is the arch name we use in the juju code | 02:01 |
| sinzui | axw, to restate your brief remark. Juju knows about ppc64. It will look for tools by that name. when the assemble script sees ppc64el, ppc64le, or power8, it will make a ppc64 tools from the first one, and ignore the rest. | 02:10 |
| axw | ah right. sounds reasonable to me | 02:11 |
| sinzui | axw, I think that is right since that is the rule I hacked together to deploy the jenkins slave to stilson-06. I didn't want to put it in production without a dev having a chance to say It was dangerously wrong | 02:13 |
| axw | sinzui: davecheney would probably know better, but AFAIK ppc64el is the right dpkg arch to match against | 02:15 |
| sinzui | axw, that is the common one. | 02:15 |
| sinzui | axw, http://ports.ubuntu.com/pool/universe/j/juju-core/ | 02:16 |
| sinzui | ^ powerpc is still there, there was a ppc64le for a few weeks | 02:16 |
| axw | powerpc eh... fairly sure we don't care about that | 02:18 |
| sinzui | The assembly script makes a tool for each one. There are not tools for ppc64 | 02:20 |
| sinzui | maybe I should make an extra tool each time a ppc64el is found | 02:23 |
| sinzui | davecheney, I /think/ juju is looking for ppc64 tools, not ppc64el tools. Should I be making I make tools named arm64 each time I find a new ppc64el deb? | 02:26 |
| sinzui | davecheney, I think I need to do that because the streams I made to deploy jenkins-slave to stilson-06 didn't work with ppc64el | 02:27 |
| davecheney | axw: I think you've fixed it https://bugs.launchpad.net/juju-core/+bug/1322392 | 02:29 |
| _mup_ | Bug #1322392: utils: test failure due to proxy <ppc64el> <juju-core:New> <https://launchpad.net/bugs/1322392> | 02:29 |
| davecheney | sinzui: I don't know | 02:29 |
| davecheney | this keeps changing | 02:29 |
| axw | davecheney: excellente | 02:29 |
| davecheney | afaik when you use --upload-tools the name of those tools were -ppc64el.tar.gz | 02:30 |
| sinzui | davecheney, users aren't supposes to use it | 02:31 |
| davecheney | sure | 02:31 |
| sinzui | davecheney, since I have an env with tools that were never published, and my computer is amd64, upload-tools is usless | 02:31 |
| davecheney | but that leads me to think that the the published name of the tools should also have the same exension | 02:31 |
| sinzui | so | 02:31 |
| sinzui | https://juju-ci.s3.amazonaws.com/tools/streams/v1/com.ubuntu.juju:released:tools.json | 02:31 |
| sinzui | ^ I created a stream of just the tools we run ci on, including a tools for ppc64. I chose that label because ppc64el didn't work | 02:32 |
| davecheney | sinzui: https://bugs.launchpad.net/juju-core/+bug/1290654 | 02:34 |
| _mup_ | Bug #1290654: juju must not rely on the output of uname -m <ppc64el> <juju-core:Triaged> <https://launchpad.net/bugs/1290654> | 02:34 |
| davecheney | possibly related | 02:34 |
| sinzui | davecheney, yep, suffered that bug with i386 too | 02:35 |
| sinzui | I have a hack in the run-unit-tests jobs to ensure i386 is matched correctly | 02:35 |
| sinzui | I think the safest thing to do is to make two tools for each ppc64el | 02:36 |
| axw | sinzui: do you have a fixed ec2 instance for running the amd64/trusty unit tests? or do you start/stop instances each time? | 02:41 |
| sinzui | axw, neither, we start an instance with an ami, then run apt-get update (and possibly add ctools) | 02:41 |
| axw | sinzui: each run? | 02:42 |
| sinzui | yes | 02:42 |
| axw | k | 02:42 |
| axw | which AMI? I want to try to reproduce this EnsureAdminUser test failure | 02:42 |
| axw | cannot repro anywhere | 02:43 |
| sinzui | axw, http://bazaar.launchpad.net/~juju-qa/juju-ci-tools/trunk/files < run-unit-test is for amd64 and i386 | 02:43 |
| axw | thanks | 02:43 |
| sinzui | axw, run-unit-tests m1.xlarge ami-018c9568 | 02:44 |
| sinzui | axw, some of the tests now have ami's embedded in them. It was a problem when many bit rotten while we were in vega | 02:45 |
| axw | alright, will give that a shot... | 02:45 |
| * sinzui updates each test in http://juju-ci.vapour.ws:8080/ to show what AMIs are used | 02:46 | |
| sinzui | axw, This is the whole test script http://pastebin.ubuntu.com/7503961/ | 02:52 |
| axw | sinzui: thanks | 02:52 |
| wallyworld_ | davecheney: i'm about to fix the TestConstraintsValidator test failure for gccgo in the joyent tests | 03:45 |
| wallyworld_ | are you alreayd doing that one? | 03:45 |
| axw | wallyworld_: can you please take a look at https://codereview.appspot.com/96540047/ | 03:47 |
| wallyworld_ | sure | 03:47 |
| wallyworld_ | axw: take my +1 with a grain of salt since i got it wrong the first time | 03:48 |
| axw | sure :) | 03:48 |
| wallyworld_ | axw: so it seems like you've changed it to how it should have been done first time round when it ws originally written - take use tomb | 03:49 |
| axw | yep | 03:49 |
| axw | nfi why it wasn't to start with | 03:49 |
| wallyworld_ | nate wrote it originlly and roger +1ed :-) | 03:49 |
| wallyworld_ | i think | 03:49 |
| axw | it was fine, but only on gc | 03:49 |
| wallyworld_ | yeah :-( | 03:50 |
| === Ursinha is now known as Ursinha-afk | ||
| wallyworld_ | axw: remind me - we reduced the mgo dial timeout to 30 seconds because it used to be 10 minutes and sometimes tests wouldn't notice a connection had disappeared and so waited 10 minutes for the timeout to expire, right? | 04:23 |
| axw | wallyworld_: hey sorry, otp | 04:30 |
| axw | just a mo | 04:30 |
| === Ursinha-afk is now known as Ursinha | ||
| axw | wallyworld_: it was reduced to 30s because it shouldn't need to be any longer than that; other connections should time out and retry. Yes, some tests were failing to connect for 10m and then the test got killed | 04:32 |
| axw | so we wouldn't get useful output | 04:32 |
| wallyworld_ | axw: so i might have to add a retry look around it because of the bug where a user's slow machine can't start mongo fast enough and bootstrap dies | 04:55 |
| axw | wallyworld_: around what? | 04:56 |
| wallyworld_ | MaybeInitiateMongoServer | 04:56 |
| wallyworld_ | or an upstream caller | 04:56 |
| axw | wallyworld_: ahh.. right, in bootstrap it doesn't retstart | 04:56 |
| axw | restart* | 04:56 |
| axw | bugger, sorry I missed that detail | 04:56 |
| wallyworld_ | in bootstrap if it can't connect after 30s yeah it dies | 04:56 |
| wallyworld_ | np, simple enough fix | 04:57 |
| axw | wallyworld_: you could just set the timeout explicitly in bootstrap | 04:57 |
| axw | rather than use the default | 04:57 |
| wallyworld_ | could do | 04:57 |
| wallyworld_ | i think a loop and smaller timeouts are perhaps better | 04:58 |
| wallyworld_ | maybe not, doesn't really matter for this situation i don't think | 04:58 |
| axw | I don't think it matters. whatever works and is elegant | 04:59 |
| axw | with preference to the former :) | 04:59 |
| wallyworld_ | sure | 04:59 |
| axw | I have nfi about this EnsureAdminUser thing. I can't reproduce on the same instance type as used in CI | 05:00 |
| axw | also trying a faster instance type on ec2 | 05:00 |
| axw | no luck | 05:00 |
| axw | I'm going to try crafting up a separate program soon to cut down the time initialising mongo, speed up the test cycle | 05:01 |
| === vladk|offline is now known as vladk | ||
| wallyworld_ | davecheney: have you done a recent gccgo test run? on my machine, i get compiler errors and seg faults | 05:29 |
| wallyworld_ | lots of tests pass though | 05:29 |
| wallyworld_ | davecheney: when you get a chance could you do another test run and pastebin the results? | 05:33 |
| menn0 | review please (fairly easy one): https://codereview.appspot.com/94710046 | 05:49 |
| === Ursinha is now known as Ursinha-afk | ||
| === Ursinha-afk is now known as Ursinha | ||
| axw | wallyworld_: I may go back on what I said before about longer timeout vs. retry | 06:23 |
| axw | wallyworld_: if we use a retry loop, it may help with these i/o timeout errors | 06:24 |
| wallyworld_ | that's sorta what i was thinking | 06:24 |
| wallyworld_ | i'll add a loop | 06:24 |
| wallyworld_ | bbiab, gotta go get kid from train station | 06:25 |
| axw | k | 06:25 |
| axw | wallyworld_: interesting thing- on CI, the agent and agent/mongo package tests take a long time to run (>15s). on my machine, they take ~1s each | 06:35 |
| axw | on another run, they took closer to 40s each | 06:36 |
| axw | on CI | 06:36 |
| axw | just a guess, but maybe the machine is still busy initialising when the tests start up | 06:37 |
| wallyworld_ | could be | 07:00 |
| wallyworld_ | maybe we delay starting the tests | 07:01 |
| wallyworld_ | axw: do you know how dave has been running his tests? on amd64 using -compiler=gccgo or on ppc? | 07:09 |
| axw | on ppc I think | 07:11 |
| rogpeppe | hi all, | 07:17 |
| axw | hey rogpeppe | 07:18 |
| rogpeppe | axw: yo! | 07:18 |
| rogpeppe | axw: how's tricks? | 07:18 |
| axw | not too shabby | 07:18 |
| axw | how are you? have a good break? | 07:18 |
| TheMue | morning | 07:18 |
| axw | morning TheMue | 07:18 |
| rogpeppe | TheMue: hiya | 07:18 |
| axw | wallyworld_: I don't think it's worth expending any more effort on these i/o timeout bugs until we try adding a delay at the beginning of CI | 07:24 |
| wallyworld_ | agreed | 07:24 |
| axw | wallyworld_: I can extend timeouts and so on, but I think it'll be neverending | 07:24 |
| axw | I'll pick something else up for now | 07:25 |
| wallyworld_ | 1305014? | 07:25 |
| axw | yep | 07:26 |
| wallyworld_ | or 1322281 | 07:26 |
| wallyworld_ | we are sort of expecting more info on those top 2 cards to come in | 07:26 |
| wallyworld_ | the cloud init logs | 07:26 |
| wallyworld_ | davecheney: you around per chance? | 07:27 |
| rogpeppe | axw: yeah, had a great break thanks | 07:29 |
| rogpeppe | axw: two weeks on cyprus | 07:30 |
| axw | nice | 07:31 |
| wallyworld_ | fwereade: hiya, i've done another pass over that doc if you have a chance to look, no rush | 07:50 |
| fwereade | wallyworld_, cheers | 07:51 |
| wallyworld_ | fwereade: i have standup soon and then will be back from soccer in about 4 hours if you needed to chat or whatever | 07:52 |
| fwereade | wallyworld_, nothing hugely springs to mind, but I'm here if you want me :) | 07:53 |
| voidspace | morning all | 07:53 |
| wallyworld_ | fwereade: i meant about the doc | 07:53 |
| wallyworld_ | once you read it | 07:53 |
| fwereade | wallyworld_, I have lots of other stuff I have to figure out today so I might not get there in time, but yu never know your luck | 07:54 |
| wallyworld_ | np, let's tak monday | 07:54 |
| wallyworld_ | mgz: you around for standup? | 09:01 |
| mgz | joining | 09:01 |
| perrito666 | morning | 09:20 |
| mgz | mornin' perrito666 | 09:20 |
| TheMue | mgz, perrito666: morning | 09:21 |
| wwitzel3 | hello | 09:39 |
| === vladk is now known as vladk|offline | ||
| voidspace | my connection is ropey today | 09:52 |
| rogpeppe | anyone know if the bot automatically pulls new deps? i *thought* it did now, but this seems to indicate otherwise: https://code.launchpad.net/~frankban/juju-core/update-testing-package/+merge/220772 | 09:57 |
| mgz | rogpeppe: it doesn't | 10:05 |
| frankban | mgz: what's the process for updating a git repo in tarmac? should we ask Curtis? | 10:07 |
| mgz | frankban: I'll do it | 10:08 |
| frankban | mgz: thanks | 10:08 |
| frankban | mgz: since we want to migrate some packages to github, we will need to update the dependencies file several times. I am also taking notes about the migration process. It would be nice to have the bot to pull changes automatically, but for the time being, is it ok if I include something like "ask mgz or sinzui to pull the last revision of the updated package in the tarmac machine before landing the dependencies.tsv c | 10:13 |
| frankban | hange"? | 10:13 |
| mgz | frankban: this is all changing, this week | 10:14 |
| frankban | mgz: as part of the git migration? | 10:15 |
| mgz | yeah | 10:16 |
| mgz | frankban: for now, bugging me is fine | 10:17 |
| frankban | mgz: cool thanks, please ping me when I can re-submit the branch | 10:24 |
| rogpeppe | anyone have strong feelings about breaking the utils.Apt* functions into a separate package? | 10:27 |
| natefinch | I'm a big fan of small, focused package, so seems like a good idea to me | 10:27 |
| rogpeppe | also potentially moving the proxy-specific functions in osenv into their own package too | 10:28 |
| rogpeppe | fwereade: ^ | 10:28 |
| wwitzel3 | +1 to small, focused packages | 10:29 |
| rogpeppe | i'm focusing more on dependency hygiene than small packages per se | 10:29 |
| rogpeppe | i don't mind larger packages if they're coherent | 10:30 |
| TheMue | coherent packages, small files | 10:30 |
| TheMue | e.g. focussing on individual types | 10:31 |
| rogpeppe | i don't mind large files either, if the contents of each file makes sense as whole | 10:31 |
| rogpeppe | s/as whole/as a whole/ | 10:31 |
| wwitzel3 | I find it tought to build large packages with large files if I am making things coherent | 10:31 |
| fwereade | rogpeppe, +1 to dependency hygiene, small files, and small packages | 10:33 |
| rogpeppe | i think the most important thing is that for every exported thing, it's easy to say "that thing definitely *belongs* in this package" | 10:33 |
| fwereade | rogpeppe, I think you have a greater tolerance for the latter two than many of us | 10:33 |
| rogpeppe | fwereade: possibly. i like to follow the examples demonstrated by the go stdlib | 10:33 |
| rogpeppe | pwd | 10:35 |
| fwereade | rogpeppe, I get antsy past a few hundred lines in a file, and past a few files in a package, tbh | 10:35 |
| fwereade | rogpeppe, mybrain has to start swapping things out at around that point | 10:36 |
| rogpeppe | fwereade: i think there's a lot to be said for intra-package hygiene too. | 10:36 |
| rogpeppe | fwereade: (for avoiding that situation) | 10:37 |
| fwereade | rogpeppe, definitely, but it's *really* hard to get that right in a large team -- package boundaries don't get violated accidentally, but it's easy to miss those sorts of screwups within a package | 10:37 |
| wwitzel3 | rogpeppe: I'm hesitant using the stdlib (of any language) as a guide for how you should craft packages that aren't part of the stdlib. | 10:38 |
| fwereade | rogpeppe, sometimes they still get violated deliberately because someone doesn't understand the situation, but it's usually more obvious that the exposed surface has just changed significantly | 10:38 |
| rogpeppe | fwereade: with too many packages, it's very easy to have packages with no proper raison d'etre - they tend to have fuzzy boundaries and end up having random collections of not-really-related functionality in them | 10:39 |
| voidspace | axw: ping, you still around | 10:39 |
| fwereade | rogpeppe, IME that's more a problem with large packages -- if the bias is to create small packages at the drop of a hat, existing packages keep their focus | 10:41 |
| fwereade | rogpeppe, and if two packages really are so closely related as to be better as one, it's an awful lot easier to merge than to split | 10:41 |
| rogpeppe | fwereade: it's all trade offs as usual :-) | 10:42 |
| fwereade | rogpeppe, yeah :) | 10:42 |
| rogpeppe | lc | 10:42 |
| rogpeppe | fwereade: tbh i've found it pretty easy to split packages in the past | 10:44 |
| === vladk|offline is now known as vladk | ||
| dimitern | TheMue, standup? | 10:45 |
| natefinch | wwitzel3: the standard library for Go is pretty good standard to go by for coding standards. Occasionally there are functions which are hard for mere mortals to follow, but in general, it's a pretty good template. | 10:49 |
| voidspace | natefinch: morning | 10:50 |
| natefinch | voidspace: morning | 10:51 |
| wwitzel3 | natefinch: I guess I've just found that things done for the sake of the "stdlib" are not always the best conventions to be used in your own packages. | 10:52 |
| wwitzel3 | natefinch: that said, my exposure to the Go stdlib at this point is still limited, so I'm still open to the idea | 10:53 |
| wwitzel3 | natefinch: and I've always pissed and moaned that the stdlib for a language wasn't the defacto go to place to learn good practices for building packages, so hopefully as I get more exposure to the Go stdlib, it will be able to promote it as the language that does just that. | 10:55 |
| natefinch | wwitzel3: I hope so :) | 10:57 |
| wwitzel3 | fwereade: you have a moment for a hangout? | 11:05 |
| fwereade | wwitzel3, sure | 11:05 |
| fwereade | wwitzel3, moonstone? | 11:05 |
| wwitzel3 | fwereade: yep | 11:06 |
| === vladk is now known as vladk|offline | ||
| rogpeppe | trivial automatically-made change to review anyone? https://codereview.appspot.com/97670044 | 11:29 |
| rogpeppe | natefinch, fwereade, wwitzel3, voidspace: ^ | 11:30 |
| rogpeppe | (it just sorts all imports standardly) | 11:30 |
| axw | voidspace: sorta here | 11:31 |
| === vladk|offline is now known as vladk | ||
| natefinch | rogpeppe: LGTM | 11:33 |
| rogpeppe | natefinch: ta | 11:33 |
| axw | voidspace: (re rsyslog review) that's why I suggested to remove the upgrade step; the immutability of syslog-port will cause the upgrade-step to fail now | 11:33 |
| axw | voidspace: although... there is a way to open a state connection that doesn't validate environ config changes | 11:33 |
| voidspace | axw: so, when an upgrade is done is the code upgraded first - or the upgrade steps run *then* the code update | 11:35 |
| voidspace | axw: hmmm... the upgrade must be run with the current codebase | 11:35 |
| voidspace | axw: so it can't work | 11:35 |
| axw | voidspace: it runs with the new code | 11:35 |
| voidspace | axw: do we want to *aim* to allow a 1.16 -> 1.20 upgrade | 11:35 |
| voidspace | if so we need to bypass validation | 11:35 |
| voidspace | if not we need to remove those upgrade steps | 11:35 |
| axw | voidspace: I think that was the aim, yeah. so I guess we should leave it in but reopen state without the policy | 11:35 |
| voidspace | axw: how do I do that? :-) | 11:36 |
| fwereade | voidspace, axw: we certainly aim for that to work, yes | 11:36 |
| voidspace | so yes, with my current mp the upgrade tests fail | 11:36 |
| axw | fwereade voidspace: can we just leave a TODO for that for now? | 11:36 |
| voidspace | for the obvious reason | 11:36 |
| axw | ah | 11:36 |
| voidspace | I should have run that before proposing of course | 11:36 |
| fwereade | axw, voidspace: well, we want 1.20 relatively soon, so I'm not sure how good an idea that it | 11:37 |
| voidspace | axw: how hard is it to bypass validation for an upgrade step? | 11:37 |
| axw | voidspace: not sure, just taking a look now | 11:37 |
| voidspace | axw: thanks, I appreciate it | 11:37 |
| axw | voidspace: the upgrade steps get an agent.Config, so can do state.Open | 11:39 |
| axw | voidspace: state.Open has an optional policy argument, which you'd want to pass in as nil | 11:39 |
| axw | voidspace: hrm, but I'm not sure if it'll work, because syslog-port is provider-independent | 11:40 |
| rogpeppe | wow | 11:41 |
| rogpeppe | ok launchpad.net/juju-core/provider/joyent388.185s | 11:41 |
| axw | voidspace: I don't think it's going to work | 11:41 |
| axw | rogpeppe: yeah, wallyworld_ has a pull request to the joyent packages to fix that | 11:41 |
| voidspace | axw: hmm | 11:42 |
| axw | apparently it's due to repeated crypto calls | 11:42 |
| rogpeppe | axw: it wouldn't be the first time... | 11:42 |
| axw | voidspace: so I don't see an easy fix here. it's possible to modify the settings docs in mongo directly, which is a little bit horrible | 11:43 |
| voidspace | axw: instead of adding it to the immutableAttributes, can we have custom validation? | 11:44 |
| voidspace | axw: which will allow setting it to the default but disallow all other values | 11:44 |
| voidspace | axw: so the upgrade will work | 11:44 |
| axw | voidspace: actually I may be wrong, I think it *does* skip all config validation. I'll investigate a bit more... | 11:45 |
| voidspace | axw: let me look at the failing test, it may be failing when it tries to set the value *before* running the upgrade | 11:46 |
| axw | ehh, could do that, but I'd rather not have super special case things | 11:46 |
| voidspace | axw: sure | 11:46 |
| voidspace | nope, it's running UpdateRsyslogPorts that fails | 11:48 |
| rogpeppe | fwereade, natefinch, wwitzel3, voidspace: another fairly trivial review. just a code move. please ignore the dubious import formatting - i wanted to avoid extra diffs from merging the above CL before it landed. https://codereview.appspot.com/100730044 | 11:49 |
| rogpeppe | this is just a code move (plus some associated renaming) | 11:49 |
| voidspace | natefinch: where is your tool for stripping log output from test runs? | 11:50 |
| axw | voidspace: opening without the policy allows you to bypass the validation altogether | 11:50 |
| axw | provider-specific or not | 11:50 |
| axw | just tested it in the state package | 11:51 |
| natefinch | voidspace: go get github.com/natefinch/nolog | 11:51 |
| voidspace | natefinch: thanks | 11:51 |
| axw | voidspace: so, in the upgrade step- state.Open(agentConfig.StateInfo(), state.DefaultDialOpts(), nil) | 11:52 |
| voidspace | awesome | 11:52 |
| voidspace | I was just wondering how to get the Info | 11:52 |
| natefinch | rogpeppe: lgtm | 11:56 |
| rogpeppe | natefinch: thanks! | 11:56 |
| natefinch | I love the way Go namespacing means you don't have to put "Apt" in the name of all the types and functions | 11:56 |
| rogpeppe | fwereade, dimitern, natefinch: ISTM that we should probably allow a single `import "foo"` statement in a file so that we can work nicely with goimports, which will format things that way if it adds a single import. | 11:57 |
| rogpeppe | fwereade, dimitern, natefinch: it would make life easier for ourselves, and it seems a fairly trivial thing | 11:58 |
| natefinch | rogpeppe: yeah, that seems fine. not wrestling with standards that conflict with goimports seems like a good idea in case anyone wants to use it. | 11:58 |
| rogpeppe | natefinch: lots of people run goimports on save | 11:59 |
| natefinch | rogpeppe: yep. Certainly for things like this where it doesn't really matter either way, if one makes goimports happy, that seems a reasonable standard. | 12:00 |
| voidspace | my lunch date is here | 12:02 |
| * voidspace lunches | 12:02 | |
| dimitern | rogpeppe, goimports leaves it alone if you reformat it | 12:08 |
| dimitern | rogpeppe, but i'm fine either way for a single import, as long as it's consistent across the code base | 12:09 |
| rogpeppe | dimitern: yeah, but if it adds a new import when there was none already, it doesn't add the () | 12:09 |
| rogpeppe | dimitern: i'm happy allowing either form, tbh | 12:09 |
| rogpeppe | dimitern: i don't think total consistency matters that much here | 12:09 |
| jcw4 | fwereade, mgz - any chance of getting https://codereview.appspot.com/98260043/ reviewed and landed? | 13:22 |
| === Ursinha is now known as Ursinha-afk | ||
| === Ursinha-afk is now known as Ursinha | ||
| fwereade | jcw4, approved, keep half an eye on it to make sure it lands | 13:51 |
| perrito666 | fwereade: go bot will publicly shame him if it doesnt :p | 13:52 |
| jcw4 | lol... tx fwereade | 13:57 |
| voidspace | natefinch: this is my fix https://codereview.appspot.com/91660043/patch/20001/30010 | 14:14 |
| * perrito666 sees the noise in the standup was generated by a trunk pumping an overflowed sewer... thanks cthulu my nose is a bit clogged today | 14:16 | |
| voidspace | heh | 14:16 |
| mgz | jcw4: sorry, I thought we'd said yesterday to just land it, | 14:22 |
| mgz | jcw4: branch seems to have a text conflict with trunk which will need fixing | 14:23 |
| mgz | jcw4: then it needs a commit message, which the bot should then pick up and merge | 14:23 |
| natefinch | bodie_: you around? | 14:30 |
| bodie_ | hullo | 14:30 |
| natefinch | I was looking at the json schema package | 14:30 |
| bodie_ | aye | 14:30 |
| natefinch | I have some concerns | 14:31 |
| bodie_ | all ears! | 14:31 |
| natefinch | it looks like it could be ok if it were fixed up | 14:31 |
| bodie_ | I was wondering what Dave Cheney meant about 66% coverage -- the author claims 246/248 tests passing on json-schema | 14:32 |
| bodie_ | that's the only reason I thought it would be acceptable | 14:32 |
| natefinch | I think he means code coverage, so like 33% of the code doesn't get touched by the tests | 14:32 |
| bodie_ | ah, is there a way to check that that I'm not aware of? | 14:33 |
| natefinch | bodie_: go test -cover :) | 14:34 |
| jcw4 | mgz: thanks... I saw the text conflict message, I'll try to figure out how to resolve it... should I just merge trunk into my branch? | 14:34 |
| natefinch | looks like 80.6% coverage when I run it right now | 14:34 |
| natefinch | also, failing tests is pretty bad | 14:35 |
| natefinch | bodie_: more concerning for me is when I see stuff like this function: https://github.com/xeipuuv/gojsonreference/blob/master/reference.go#L84 | 14:36 |
| natefinch | bodie_: which says it returns an error, but actually always returns nil | 14:36 |
| bodie_ | oooo | 14:36 |
| bodie_ | hmm | 14:36 |
| bodie_ | okay, I think those concerns make sense | 14:36 |
| bodie_ | ... obviously they do, I mean I agree and understand :) | 14:36 |
| natefinch | bodie_: the last one is the most important.... that's a function that is obviously encountering errors and throwing them away for some reason. That's pretty bad. Who knows what other problems exist. | 14:37 |
| natefinch | The code has a lot of test cases which is great, and it might be a good starting point to make it into something production ready... but it's not production ready as-is, IMO. | 14:38 |
| bodie_ | I'm not sure why I didn't think to look through it -- I think my understanding was that we went with JSON-Schema because there were solid Go implementations already in place, so I made an assumption there | 14:39 |
| bodie_ | Do you think we need to stop and dig in to get this thing shipshape before we continue the Actions implementation, or do you think it makes more sense to chain things together from beginning to end and then attack the pain points? | 14:40 |
| bodie_ | I mean, my original implementation of an Actions spec from YAML was a little clumsy, but conformed the spec to a basic static schema | 14:42 |
| bodie_ | something like actions: {description: ..., params: {name: {description: ..., type: ..., default: ...}, name: ...} | 14:43 |
| bodie_ | oops, insert multiple names of actions as keys to the params values | 14:44 |
| bodie_ | or, we could use the code I have now which validates via gojsonschema, but instead use a generic "Validate" function that, for now, just ensures the spec conforms to the basic actions.yaml format I laid out there ^ | 14:46 |
| mgz | jcw4: yeah, just merge in trunk and resolve/commit | 14:46 |
| bodie_ | then as we move forward, put work into gojsonschema, and once it's in shape, it'll be a drop-in replacement for the Validate we use | 14:46 |
| bodie_ | I'm not trying to be lazy, just expeditious | 14:46 |
| natefinch | I would not feel comfortable using that package, and I think fixing it up is going to take more time than value we'll actually get out of it, in my opinion.... but other people might have differing opinions. fwereade might want to chime in | 14:47 |
| bodie_ | would agree there | 14:48 |
| bodie_ | another option in that case is that we simply Validate against a tiny subset of json-schema, something like the actions.yaml schema I specified | 14:49 |
| fwereade | natefinch, bodie_: heh, yes, I should probably not have taken its quality on trust myself :/ | 14:49 |
| fwereade | natefinch, bodie_: yeah, that method alone is a big ol' WTF-do-not-touch-this-with-a-bargepole | 14:52 |
| * fwereade sighs heavily | 14:52 | |
| bodie_ | lol, fair enough | 14:52 |
| fwereade | bodie_, can you reasonably estimate the effort to get a sane gojsonschema library written? even if it's just a conformant but useful subset? | 14:52 |
| jcw4 | mgz: I committed my merge fix, but I'm afraid I can't see how to update the commit message and land without proposing a new review? | 14:53 |
| fwereade | bodie_, at least we can probably copy the test cases ;p | 14:53 |
| mgz | jcw4: you can just edit the launchpad page | 14:53 |
| mgz | jcw4: copy in the description to the commit message | 14:54 |
| voidspace | jcw4: the commit message you normally set directly on launchpad | 14:54 |
| fwereade | jcw4, I just did your commit message, I should have done that when I approved in the first place :/ | 14:54 |
| jcw4 | I see | 14:54 |
| jcw4 | thx mgz, voidspace, and fwereade | 14:54 |
| jcw4 | fwereade: go bot kicked it back into needs review | 14:56 |
| bodie_ | fwereade, I can't say with any realism right now. do you think the first step should be to think about what our needs are from it, first? it seems very reasonable to me to implement a subset, or even a validator similar to the one in Config. we could pass a json-schema rendering to the frontend | 14:57 |
| * jcw4 will be back in 15 minutes... dropping kids off at school | 14:57 | |
| bodie_ | right now I'm thinking very much in terms of "what could and would work" rather than "what would be best" so forgive me for being less than rigorous | 14:58 |
| bodie_ | just ideating, I expect you to knock these down | 14:58 |
| mgz | sinzui: do you have some time to talk git? | 15:00 |
| natefinch | why can't people just say brainstorming anymore? Why is it ideating now? | 15:00 |
| bodie_ | haha | 15:00 |
| sinzui | mgz, I and going into a meeting. I can talk in about 30 minutes | 15:01 |
| bodie_ | at least I didn't say incepting | 15:01 |
| mgz | sinzui: sounds good | 15:01 |
| perrito666 | natefinch: what does ideating mean? | 15:02 |
| bodie_ | making ideas | 15:02 |
| natefinch | perrito666: brainstorming | 15:02 |
| perrito666 | natefinch: meh, why replace a perfectly working buzzword for another | 15:02 |
| natefinch | see? | 15:02 |
| natefinch | sounds more hip, I guess. | 15:03 |
| bodie_ | that's certainly not the intention... I just like the way it sounds :) | 15:03 |
| * natefinch likes the image of a storm of brains swirling in the dark clouds | 15:04 | |
| perrito666 | sinzui: where can I find the ha b&r test? | 15:04 |
| bodie_ | cleaning up after a brainstorm is the worst. | 15:04 |
| perrito666 | natefinch: http://slurmed.com/fanfics/coldangel_1/blame-it-to-the-brain-part-2/003.jpg | 15:05 |
| natefinch | lol nice | 15:05 |
| bodie_ | let's see here. gojsonschema relies on gojsonpointer and gojsonreference.... | 15:06 |
| bodie_ | gojsonpointer is about 400 lines in total | 15:07 |
| bodie_ | as is gojsonreference | 15:07 |
| bodie_ | gojsonschema itself is probably around 3000 lines, in about 8-12 files | 15:08 |
| bodie_ | that's really not so bad | 15:08 |
| sinzui | perrito666, http://pastebin.ubuntu.com/7505941/ | 15:09 |
| bodie_ | as dave mentioned, the test coverage is low | 15:09 |
| perrito666 | sinzui: tx | 15:09 |
| bodie_ | the question is, how much bad code is there, how much extra is needed to pull it apart into the good bits / organizational concepts, grind out the rot, and how much complexity was the bad stuff glossing over | 15:10 |
| perrito666 | sinzui: just fyi, along with the HA backup/restore patch I added doc on what was the expected behavior | 15:11 |
| natefinch | bodie_: the other option is to take a well-regarded implementation in another language and port it to go. That might be easier | 15:12 |
| natefinch | bodie_: https://github.com/JamesNK/Newtonsoft.Json/tree/master/Src/Newtonsoft.Json/Schema | 15:13 |
| bodie_ | perhaps it would be intelligent simply to harness an existing implementation externally to our code | 15:14 |
| bodie_ | I know it's an ugly option, but it's an option | 15:15 |
| bodie_ | for an obvious example, we didn't implement our own database | 15:16 |
| tasdomas | fwereade, I'm working on separating juju-core/cmd from juju-core. The package cmd depends on loggo - do you think that dependency should be broken (making logging something command implementers need to set up themselves)? | 15:19 |
| jcw4 | woo hoo, branch landed ... yay | 15:20 |
| voidspace | natefinch: ping | 15:21 |
| * bodie_ buys jcw4 a beer | 15:22 | |
| * jcw4 downs it and burps | 15:23 | |
| hazmat | anybody know where the cloudsigma provider stuff lives? | 15:25 |
| perrito666 | brb lunch | 15:25 |
| natefinch | voidspace: how goes? | 15:31 |
| voidspace | natefinch: cool, I need a new task | 15:32 |
| voidspace | natefinch: I've started to look at HA: generate/update StateServingInfo in mongo.EnsureServer | 15:32 |
| voidspace | natefinch: as that sounds plausible that I can do it :-) | 15:32 |
| voidspace | natefinch: would you prefer I work on something else? | 15:33 |
| natefinch | voidspace: haha... I'm not sure exactly what that's buying us... it's an old card. rogpeppe - what was that task about generating StateServingInfo in EnsureMongoServer doing for us? | 15:34 |
| rogpeppe | natefinch: in a call - be with you in a bit | 15:35 |
| voidspace | natefinch: ah, ok | 15:35 |
| natefinch | rogpeppe: sure thing, thanks | 15:35 |
| voidspace | natefinch: which card you prefer I look at | 15:35 |
| natefinch | voidspace: you can do the change mongo to write majority one. I had started on that, but it was causing problems for some reason. The code change is simple, but figuring out why it breaks things may be tricky | 15:36 |
| voidspace | natefinch: ok | 15:36 |
| voidspace | natefinch: do you have a branch you've started on? | 15:37 |
| natefinch | voidspace: lp:~natefinch/juju-core/052-write-majority | 15:38 |
| voidspace | natefinch: ok, I'll take a look | 15:39 |
| voidspace | natefinch: ooh, lots of test failures | 15:41 |
| voidspace | I wonder if it's even an mgo issue | 15:42 |
| === vladk is now known as vladk|offline | ||
| bodie_ | sigh | 15:47 |
| rogpeppe | anyone know what the current status of network routing to containers under ec2 is? | 15:50 |
| rogpeppe | natefinch: hmm, i can't remember about that card. will have a look now. | 15:54 |
| rogpeppe | natefinch, voidspace: i *think* the point is that the StateServingInfo in the agent.conf should not be generated by environs/cloudinit | 16:11 |
| voidspace | natefinch: do we *always* have a replicaset? | 16:19 |
| perrito666 | voidspace: yup | 16:19 |
| perrito666 | learn to love it | 16:19 |
| voidspace | perrito666: thought so | 16:19 |
| voidspace | damn | 16:19 |
| voidspace | I thought maybe write-majority was failing when we didn't | 16:20 |
| voidspace | as the error message says "cannot create database index: norepl" | 16:21 |
| voidspace | and indeed, google confirms that this is possibly the case | 16:25 |
| voidspace | for "norepl" | 16:25 |
| voidspace | This error happens when you try to use the w option without replication enabled. | 16:25 |
| perrito666 | mm, replicaset should be enabled always,that is what broke restore last time | 16:26 |
| voidspace | perrito666: well, except maybe in tests | 16:26 |
| perrito666 | ah | 16:26 |
| natefinch | sorry, was away for lunch. Yes, we always run with a replicaset, except for localhost (which we actually want to fix) | 16:26 |
| voidspace | natefinch: "except for localhost" | 16:26 |
| voidspace | natefinch: I suspect that's what's breaking all the tests | 16:26 |
| perrito666 | breaking is the word of the day >p | 16:26 |
| voidspace | natefinch: setting WMode without replicasets being enabled causes mongo to throw "norepl" errors | 16:26 |
| voidspace | natefinch: and that's what I'm seeing in the tests | 16:27 |
| voidspace | natefinch: using mgo session can we check for this? so we don't unconditionally set WMode? | 16:27 |
| voidspace | session.SetSafe doesn't return anything, so we can't check there for error | 16:29 |
| natefinch | voidspace: I wonder if we shouldn't run the test mongo instances with replicasets, to better replicate our running environments, rather than making them special | 16:29 |
| perrito666 | +1 | 16:30 |
| voidspace | natefinch: that sounds better - are they only used for tests? Or by "localhost" do you mean local provider too. | 16:30 |
| natefinch | voidspace: there's a special testing framework that we use to spin up mongo instances for testing | 16:31 |
| voidspace | natefinch: JuJuConnSuite ? | 16:31 |
| natefinch | voidspace: we do want to fix the local provider too, but that's a separate issue | 16:31 |
| voidspace | natefinch: does the local provider currently *not* use replicasets? | 16:31 |
| natefinch | voidspace: correct | 16:31 |
| voidspace | natefinch: in which case setting WMode in state.Open will be broken for that too | 16:32 |
| voidspace | so we either need to fix *both*, or not set WMode unless we have a replicaset | 16:32 |
| natefinch | voidspace: correct | 16:32 |
| voidspace | do you think fixing both is plausible within the scope of this card? | 16:32 |
| natefinch | voidspace: I can make another card to fix local provider | 16:33 |
| perrito666 | voidspace: you cant fix one and make wmode not set when there is no replicaset? | 16:33 |
| natefinch | we're supposed to do both, so might as well do the prerequisite first | 16:33 |
| voidspace | perrito666: well, the fix is either "always use replicaset" or "don't use WMode when not using a replicaset" | 16:33 |
| voidspace | perrito666: so saying "fix one and don't use WMode sometimes" is like the worst of both worlds :-) | 16:33 |
| voidspace | natefinch: ok, so if you create the card I'll switch to that | 16:34 |
| voidspace | or I can create the card | 16:35 |
| perrito666 | speaking of which can I create a card for what I am working now?, since I split the HA restore story in 2 | 16:35 |
| perrito666 | or should I wait for you to create it? | 16:35 |
| natefinch | voidspace: https://canonical.leankit.com/Boards/View/103148069/109980847 | 16:35 |
| natefinch | perrito666: you can do it | 16:35 |
| voidspace | natefinch: thanks | 16:35 |
| perrito666 | natefinch: going to | 16:36 |
| natefinch | voidspace: there's stuff in agent/mongo/ there's a withHA flag to remove | 16:36 |
| voidspace | natefinch: ok | 16:36 |
| voidspace | so there's a function "shouldEnableHA", that returns literally "providerType != provider.Local" | 16:45 |
| voidspace | I've tried changing that to "true" and am running tests... | 16:45 |
| voidspace | natefinch: *why* didn't we use replicasets for local - for perf reasons? | 16:45 |
| voidspace | rogpeppe: ^^ | 16:45 |
| natefinch | voidspace: a couple people experienced problems | 16:45 |
| voidspace | ah | 16:46 |
| voidspace | natefinch: do you recall the nature of those problems? | 16:46 |
| * perrito666 did some clean up of his cards | 16:46 | |
| natefinch | voidspace: nope. There's a bug somewhere I think. | 16:47 |
| natefinch | voidspace: I believe ian was one of the people who reported having problems | 16:47 |
| voidspace | no failures on my machine so far | 16:47 |
| voidspace | job done ;-) | 16:47 |
| natefinch | voidspace: I believe the nature of the problem was long the lines of "bootstrap never completes" or something critical like that | 16:47 |
| natefinch | voidspace: nothing too subtle | 16:48 |
| voidspace | well, so long as it's only for Ian I'm not too worried | 16:48 |
| voidspace | heh, I'll talk to him on Tuesday | 16:48 |
| voidspace | and look for the bug | 16:48 |
| natefinch | cool | 16:48 |
| voidspace | so, apiserver/client tests fail | 16:49 |
| voidspace | digging in | 16:49 |
| voidspace | ah, but pass on their own | 16:50 |
| natefinch | voidspace: I think it was less of a tests problem and more of a production problem. But most of us could run it without a problem. | 16:50 |
| voidspace | sure, but getting the code right is the *first* step - then I can try and *find* the production problems | 16:50 |
| voidspace | it's ten to six on Friday night before a bank holiday though | 16:52 |
| voidspace | gonna look for this bug then call it a night I think | 16:52 |
| rogpeppe | voidspace: try looking in the bugs that were closed by the change that made juju not used replicaset on the local provider | 16:52 |
| bodie_ | separately from the issue of code quality, gojsonschema's deps are prepended with Apache2, but there's no LICENSE in their root directories | 16:52 |
| voidspace | rogpeppe: ok, thanks | 16:52 |
| bodie_ | I'm not sure where that puts us | 16:53 |
| bodie_ | any input welcome | 16:53 |
| rogpeppe | bodie_: hi | 16:53 |
| natefinch | bodie_: I think licenses in the code is fine even if there's no license file. | 16:53 |
| rogpeppe | bodie_: you pinged me last night, but i wasn't around | 16:53 |
| bodie_ | hullo, just was going to ask about the gojsonschema code quality and what my next steps ought to be. | 16:53 |
| rogpeppe | bodie_: at first glance, it's not great at all | 16:54 |
| rogpeppe | bodie_: i could see lots of opportunities for simplification and making the code more Go-idiomatic | 16:54 |
| bodie_ | yeah, my mistake there -- I completely assumed we'd vetted it for some reason, since I was under the impression we chose JSON-Schema for use with Juju due to the existence of a quality implementation for Go | 16:55 |
| bodie_ | since xeipuuv's implementation appears to be the go-to for Go, I wrongly made that connection | 16:55 |
| rogpeppe | bodie_: i expect it was chosen because people know about it | 16:55 |
| bodie_ | i'll be back in a minute, kid is melting down | 16:56 |
| rogpeppe | bodie_: personally, i think the standard is much more complex than it needs to be, but that's just me reacting to most of the modern computing world probably :-) | 16:56 |
| rick_h_ | bodie_: it was mainly chosen due to the JS side and the idea it could be supported in Go | 16:56 |
| rick_h_ | bodie_: and that it's got a definition that's standard | 16:56 |
| rogpeppe | rick_h_: that's definitely a plus point | 16:57 |
| rick_h_ | jeremy has written a very good JS side of it and making it work down to Go is part of the work in making this work out | 16:57 |
| rogpeppe | rick_h_: jeremy's written the JS schema verification engine? | 16:59 |
| rick_h_ | rogpeppe: yes, it generates UI and does validatio on it and such. mark S found it and brought him into the project because of it. https://github.com/jdorn/json-editor/ | 17:00 |
| rogpeppe | rick_h_: cool | 17:00 |
| rogpeppe | rick_h_: i think it would be worth moving towards having our own high quality Go implementation, whether that entails from-scratch or mutating gojsonschema | 17:01 |
| rick_h_ | rogpeppe: yea, I think there was hope there was some good starting points out there for Go | 17:02 |
| rogpeppe | rick_h_: i'd probably be tempted to start from scratch but with some close contemplation of existing implementations. it's very useful that there's an existing comprehensive test suite. | 17:03 |
| bodie_ | and back -- | 17:04 |
| natefinch | rogpeppe: yeah, the test suite was what I was most interested from that package. | 17:05 |
| rogpeppe | natefinch: i was thinking mostly of the set of examples provided by json schema itself | 17:05 |
| natefinch | rogpeppe: ahh | 17:06 |
| natefinch | rogpeppe: my thought was to port an implementation from another language, since that's generally pretty easy, and gets us ahead of the game, rather than having to sit down and start from scratch. Maybe we could fix up that go repo, but I honestly haven't given its approach any thought after I looked at some specific code samples | 17:07 |
| bodie_ | I'm hesitating to move forward in a specific direction since I already made a bad call here, but I chatted with fwereade and I think my next step is to start digging through the existing implementation to see whether it can be salvaged and how much rot there is to it | 17:07 |
| rogpeppe | natefinch: the actual test suite in gojsonschema looks as if it could easily be more driven by the data inside json_schema_test_suote | 17:08 |
| rogpeppe | bodie_: that's a good idea | 17:08 |
| rogpeppe | bodie_: also perhaps study the standard and see if you can think of a nice obvious way to implement it simply and efficienctly. | 17:09 |
| rogpeppe | bodie_: BTW i wouldn't say you made a bad call - grabbing the only json schema implementation is the obvious thing to do. it's a pity there's not a really nice implementation already out there. | 17:11 |
| bodie_ | I did some digging through the standard last night and I think "simple" and "efficient" are not the choicest words I would select to describe it | 17:12 |
| rick_h_ | lol | 17:12 |
| bodie_ | rogpeppe, thanks for that -- definitely feeling pretty embarrassed here | 17:12 |
| bodie_ | I wonder -- | 17:12 |
| bodie_ | a functional implementation might actually make it much simpler. | 17:13 |
| rogpeppe | bodie_: yeah, it's the usual completely bloated web standard | 17:13 |
| perrito666 | bodie_: cmon, there is no reason to feel embarrassed | 17:13 |
| rogpeppe | bodie_: and fairly impenetrable, also as usual | 17:13 |
| bodie_ | I'm reading an article about how a J-schema validator was implemented in Clojure (my pet language) and it's really quite beautifully done | 17:13 |
| bodie_ | perhaps could rip off -- er, find inspiration in some of the ways this deals with it | 17:14 |
| bodie_ | me likey fp | 17:14 |
| rogpeppe | bodie_: yeah, that might work nicely | 17:14 |
| bodie_ | rick_h_, rogpeppe -- I think there's a lot to be said for the value of a high quality json schema utility in Go | 17:15 |
| bodie_ | brb again -- | 17:15 |
| rogpeppe | bodie_: one thing i'd like to know is if it might be possible to define the actual json schema syntax as go structs | 17:15 |
| voidspace | EOD, g'night all | 17:25 |
| perrito666 | voidspace: gnight | 17:25 |
| natefinch | night voidspace | 17:25 |
| voidspace | thanks all | 17:26 |
| voidspace | see you on Tuesday | 17:26 |
| wwitzel3 | voidspace: have a good weekend | 17:26 |
| perrito666 | why tue? | 17:26 |
| natefinch | perrito666: UK has one of their poorly named bank holidays on monday | 17:27 |
| natefinch | perrito666: where the name of the holiday is "Bank Holiday" | 17:28 |
| perrito666 | according to wp is "Spring Bank Holiday" | 17:28 |
| perrito666 | lol and its determined by Last Monday in May | 17:29 |
| natefinch | perrito666: btw, I and presumably wwitzel3 will also be out monday, since it's a holiday for the US, too - Memorial Day | 17:29 |
| perrito666 | well Ill then be a bit late since sunday its my birthday and I presume I might get a zip or two of wine | 17:29 |
| natefinch | perrito666: happy birthday, and you can be as late as you want, because the rest of us will be not working :) | 17:30 |
| perrito666 | lol | 17:30 |
| perrito666 | tx | 17:30 |
| bodie_ | back, sorry. michelle is sick and i'm watching the babe -- she's down for a nap now | 17:32 |
| bodie_ | rogpeppe, thinking about what you said | 17:32 |
| bodie_ | rogpeppe, right now we're parsing it out into a map[string]interface{} | 17:35 |
| rogpeppe | bodie_: going down some of the schema document, it *might* be possible to parse into something like: http://paste.ubuntu.com/7506473/ | 17:36 |
| bodie_ | then dumping that into a JsonSchemaDoc | 17:36 |
| bodie_ | if it complains, we reject it as invalid jsonschema | 17:36 |
| rogpeppe | bodie_: obviously i've omitted most of the properties there | 17:36 |
| bodie_ | so, in the state, our schema will be defined as a nested map | 17:36 |
| rogpeppe | bodie_: and there might be some kind of meta-attribute thing going on which makes the approach infeasible | 17:37 |
| bodie_ | to validate a json argument map from the frontend, they'll just GetJsonSchema or whatever | 17:37 |
| bodie_ | taking a look | 17:37 |
| bodie_ | ah, I see what you mean | 17:37 |
| bodie_ | interesting | 17:37 |
| bodie_ | an actual Go implementation of JSON-Schema, basically, rather than a parser / validator | 17:38 |
| rogpeppe | omg, there's the whole email address syntax in there | 17:38 |
| natefinch | lol | 17:38 |
| bodie_ | lol | 17:38 |
| bodie_ | and much more! | 17:38 |
| bodie_ | yours today for only 19.99 man-months... | 17:38 |
| rogpeppe | bodie_: the point that we *might* be able to leverage encoding/json to do a lot of the initial verification for us | 17:39 |
| natefinch | rogpeppe, bodie_: A struct that might look like this (if you don't know C# bool? is a nullable bool, sorta like *bool in Go) | 17:39 |
| natefinch | ^^^ https://github.com/JamesNK/Newtonsoft.Json/blob/master/Src/Newtonsoft.Json/Schema/JsonSchema.cs#L43 | 17:39 |
| rogpeppe | bodie_: then you wouldn't need to do nearly so much dynamic-value mangling to parse the schema | 17:41 |
| bodie_ | my eyes!!! | 17:41 |
| * rogpeppe needs to go | 17:41 | |
| natefinch | haha... C# is pretty good. And it's easy to translate to Go. | 17:41 |
| sinzui | perrito666, I am not really at work today, but I have good news for you. http://juju-ci.vapour.ws:8080/view/Functions/ | 17:41 |
| natefinch | see ya rogpeppe | 17:41 |
| rogpeppe | natefinch, bodie_: see ya | 17:41 |
| bodie_ | have a good weekend! | 17:42 |
| sinzui | C# is nicer than java | 17:42 |
| bodie_ | fair enough | 17:42 |
| bodie_ | I don't personally find either very pleasant :) | 17:42 |
| bodie_ | but maybe I just haven't had enough exposure to C# | 17:42 |
| perrito666 | sinzui: sweet, pretty good for not your best day off | 17:43 |
| perrito666 | sinzui: why are there several runs for the same r? | 17:43 |
| sinzui | perrito666, The failed test runs related to resource contention with other tests I adjusted the envs they run it. | 17:43 |
| sinzui | perrito666, CI will run tests up to 5 times before cursing them. While juju might be at fault, clouds, network, and resource contention by be the real cause | 17:44 |
| natefinch | bodie_: the reason I went with C# is that people generally don't try to do too much magicky stuff in it, which means it might have a valid implementation that would port well to Go. It might be a little class-heavy, but probably a lot less so than a java implementation | 17:44 |
| sinzui | perrito666, and in the case that azure is just dead to juju and cloud-init, its not juju's fault that azure tests fail | 17:45 |
| perrito666 | sinzui: well It would have made a pretty heavy weekend for me if I was responsible for the testing cursing after my commit | 17:45 |
| sinzui | perrito666, I named the new test -devel, telling CI that the test does not get a vote to curse. The test certainly was at fault for the first few runs | 17:46 |
| jcw4 | I like the idea of patterning a Go json schema after NewtonSoft's implementation | 17:46 |
| sinzui | perrito666, If the test reliably runs over the next few days, I will promote it by removing the -devel | 17:46 |
| natefinch | bodie_: I've used the Newtonsoft stuff in C#, it's basically the go-to implementation for Json, since there isn't built-in support in the .net framework (or at least there wasn't as of about a year ago) | 17:46 |
| jcw4 | I believe that library has become the standard JSON impl in .NET vNext | 17:47 |
| natefinch | jcw4: oh, cool | 17:47 |
| jcw4 | natefinch: looking for the link I read recently... | 17:48 |
| === vladk|offline is now known as vladk | ||
| jcw4 | natefinch: https://twitter.com/JamesNK/status/466375482850963458 | 17:55 |
| jcw4 | asp.net v next | 17:55 |
| bodie_ | lol | 17:55 |
| natefinch | nice chart | 17:56 |
| jcw4 | :) | 17:56 |
| === vladk is now known as vladk|offline | ||
| === Ursinha is now known as Ursinha-afk | ||
| === Ursinha-afk is now known as Ursinha | ||
| bodie_ | I need to have a look over this jsonschema stuff over the weekend. my family is sick and I got very little sleep. I hope to have something better to show for my investigation over the next couple of days. | 20:34 |
| bodie_ | enjoy your weekends :) | 20:35 |
| natefinch | bodie_: good luck | 20:36 |
| === Ursinha is now known as Ursinha-afk | ||
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!
