[00:15] wallyworld: verbose test output https://github.com/juju/juju/pull/6514 [00:29] looking [00:30] menn0: i would even love a make arg to add gocheck.vv [00:30] so that we can diagnose better hung individual tests [00:30] but the change as submitted looks good [00:45] wallyworld: agreed but -check.vv makes things a bit hard to follow though [00:46] menn0: yeah, i ws thinking of it as a maek arg so that it was off by default [00:59] wallyworld: interestingly, we already have per package test timings [00:59] wallyworld: it varies a lot. for run-unit-tests-xenial-amd64 I've seen the state package tests take anywhere from 11mins to 19.2 mins. [01:00] wallyworld: maybe we just need to bump the timeout [01:00] maybe yeah [01:04] bumping timeout is kind of like sweeping under the rag... is there really no way we can shorten the time the tests run? [01:04] rug* [01:11] anastasiamac: thumper fixed a bunch of unnecessarily long running tests in state recently [01:11] anastasiamac: I don't think there's much low hanging fruit left [01:12] anastasiamac: we keep adding functionality and tests so the test time keeps increasing [01:12] menn0: FWIW, in the prometheus metrics I'm gathering from my azure env I can see the effects of periodic txnpruner in terms of deletion ops going up every 2 hours, but there's no corresponding spike in CPU in jujud [01:12] anastasiamac: it also seems like the test run time is highly variable in the virtualized environments in which we run the tests [01:12] menn0: i c... so based on the times u've put ^^, what timeout r u thinking? 25min? [01:13] but then I'm not seeing CPU spikes in my env at all [01:13] axw: ok, so it's probably not that [01:14] menn0: I'm going to try hammering the disk to see if it triggers [01:14] axw: did you manage to have a look at the logs for the system that is seeing the problem? i.e. is the pruner running for long periods there? [01:14] menn0: I think the log level was set to WARNING, I'll have another look [01:15] menn0: yeah, set to the default of =WARNING [01:16] menn0: i have a tiny question but it may need context, could u HO or prefer IRC? [01:17] anastasiamac: i'm easy [01:18] in standup? [01:18] anastasiamac: ok [01:39] damn, roofing guys caused my power to drop out. hopefully back now for bit [01:40] anastasiamac: it's not necessarily sweeping under rug. if there are X tests and each test takes Y time, then the tests will take X*Y to run. there's not much we can do about mgo startup time etc. of couse we can tune tests, but there's only so much that can be done [01:43] IIRC, Gustavo said we were doing something stupid with mongo - restarting it for every test or something. Probably unavoidable per package, but maybe worth looking into to make sure that we're not restarting it for every test? [01:45] i *think* someone did look at that. not retarting has other issues, eg test isolation [01:45] not sure of the final outcome there [02:20] heh, I'm running the tests serially to get nicely ordered output and whoo boy, they take a lot longer that way [02:21] 35 minutes so far.... 35 minutes so far, and usually it's like... I dunno, 15ish? [02:34] heh: 44m31.434s [02:42] menn0: wallyworld how relevant is this to us now, with new logging/rsyslog in 2.0? https://bugs.launchpad.net/juju/+bug/1562088 [02:42] Bug #1562088: flexible /etc/rsyslogd.d/25-juju.conf configuration [02:47] anastasiamac: it's not. 1.25 only [02:54] natefinch: \(ˆ˚ˆ)/ [02:54] natefinch: or even better ヽ(´o｀； [02:56] menn0: tyvm [03:10] PASS: bakerystorage_test.go:72: BakeryStorageSuite.TestExpiryTime 58.037s [03:10] that's one test [03:10] PASS: syslog_test.go:220: syslogSuite.TestConfigChange 31.715s [03:10] PASS: syslog_test.go:193: syslogSuite.TestLogRecordForwarded 31.731s [03:10] PASS: local_test.go:547: localServerSuite.TestStopInstanceSecurityGroupNotDeleted 29.046s [03:17] natefinch: ouch [03:17] natefinch: we should get that one fixed :) [03:17] * menn0 is EOW === Guest91977 is now known as ahasenack === ahasenack is now known as Guest60864 === Guest60864 is now known as ahasenack === ahasenack is now known as Guest42345 === Spads_ is now known as Spads [13:01] rick_h_: ping [13:01] dimitern: pong [13:01] rick_h_: do you have a few minutes for a quick HO? [13:01] dimitern: otp atm, will have time possiblly before standup or right after [13:02] rick_h_: sure [13:10] balloons: So our candidate for 2.02 is 8fcc982d ? [13:11] balloons: I mean 2.0.1 [13:33] dimitern: free if you want to chat [13:34] rick_h_: ok, joining standup HO [13:34] dimitern: rgr omw [13:47] mgz: ping [13:47] voidspace: yo [14:00] dimitern: voidspace natefinch mgz dooferlad ping for standup [14:00] omw [14:01] rick_h_: omw [14:06] natefinch: funny you should send that email out re. tests. i had a test fail bc "`exec /usr/bin/mongod` file not found". i was sad for a bit [14:06] and begrudgingly installed mongo =| [14:07] haha [14:07] so sad that our "unit" tests depend on installing mongo [14:08] natefinch: well beyond that... why are we trying to *start* mongo by just running the binary if it's not running? just fail and let the user maintain a semblance of control over their machine [14:08] katco: good point [14:22] natefinch: basically, sent message to list just now about ssh from windows client [14:23] atm my fix uses github.com/andybalholm/crlf to wrap stdin [14:24] which is probably not enough code to be bothered importing (not even sure I want the golang.org/x/text dep) [14:25] wondered if you'd come across a modern go idiomatic method of translating newlines [14:28] mgz: you can do it pretty easily with just a io.reader wrapper... the naive implementation might be slightly worse performance-wise than that implementation, but this isn't something we're using to read hundreds of megabytes either [14:29] I'll make an example, one sec [14:29] natefinch: ta [14:38] mgz: https://play.golang.org/p/aS3IFXBgdl [14:40] natefinch: thanks! [15:27] * rick_h_ goes to family lunch === lazyPowe_ is now known as lazyPower [15:45] alexisb: ping [16:14] heya redir [16:14] I am going to go drop off kiddo [16:15] but will ping when I am back === Guest42345 is now known as ahasenack === ahasenack is now known as Guest79971 === freyes__ is now known as freyes [17:33] redir, ping [17:33] pong [19:57] * redir heads out for a bit.