| tumbleweed | cjwatson: and the primary archive? | 00:03 |
|---|---|---|
| cjwatson | tumbleweed: There were probably-similar-cause problems with the arm64, ppc64el, and some powerpc non-virtualised builders earlier today, but those have been resolved. | 00:04 |
| cjwatson | tumbleweed: I'm in the middle of retrying all the builds I can see. | 00:04 |
| tumbleweed | cjwatson: that sounds about right, thanks | 00:05 |
| cjwatson | Something to do with an ntp upgrade, we think. | 00:05 |
| tumbleweed | lol | 00:06 |
| cjwatson | I know, right | 00:06 |
| HeOS_ | Hi to all! | 10:06 |
| HeOS_ | What about staging? | 10:06 |
| HeOS_ | He is not working. :( | 10:07 |
| wgrant | HeOS_: Should be working now. | 10:53 |
| HeOS_ | wgrant: yeah, thanks! | 10:53 |
| HeOS | Is staging not working again? | 14:33 |
| HeOS | Could anyone help with staging? | 14:55 |
| cjwatson | Moment | 14:57 |
| HeOS | cjwatson, okay. | 15:13 |
| wgrant | HeOS: What is your API script trying to do? | 15:23 |
| wgrant | It's making an awful lot of requests. | 15:23 |
| cjwatson | Nothing anomalous in monitoring, it's not in swapdeath or anything | 15:25 |
| cjwatson | We should probably arrange for nagios to actually check the staging appservers. | 15:26 |
| wgrant | It's probably just the 2.7 hang bug that I've hit the test suite a couple of times. | 15:26 |
| wgrant | only shows up in reasonably concurrent situations, which we aren't exposed to on the single-threaded prod appservers. | 15:26 |
| cjwatson | We shouldn't have to wait for user reports of a hang. (OTOH staging is down for restores for fairly long periods of time ...) | 15:27 |
| HeOS | wgrant, my script get a bugs and create a new series for this bugs. | 15:29 |
| cjwatson | I guess atemoya is unlikely to be able to cope with running more appservers. | 15:29 |
| cjwatson | It's got about 5GB free at the moment, but that'd be eaten up quickly enough ... | 15:30 |
| wgrant | We should probably attempt to track down the hang, but it's a bit late for that for me, unless you want to try gdbing it. | 15:34 |
| cjwatson | I'm inclined to restart the appserver and leave this on our backlog with some notes. I have other things to do this afternoon :) | 15:35 |
| wgrant | Indeed. | 15:36 |
| wgrant | Hopefully it will happen again at a more reasonable time forme. | 15:36 |
| wgrant | It's probably the same issue that occasionally hangs buildbot and my local test suite, but only since 2.7 | 15:37 |
| HeOS | Does my script load on staging? | 15:43 |
| cjwatson | HeOS: Staging isn't totally reliable in various ways. You might find you spend less time waiting for us to deal with hangs if you slow your script down a bit. | 15:44 |
| cjwatson | HeOS: But surely whatever testing you're doing can be done in smaller-scale ways. | 15:44 |
| wgrant | The script could also be optimised a little. More than half of the requests are for duplicated series and milestones -- you'll find things are a few times faster if you cache those objects locally rather than re-requesting them every time. | 15:45 |
| cjwatson | HeOS: Anyway, it's back up now. | 15:45 |
| HeOS | I use a staging for debug and development. :) | 15:47 |
| HeOS | At this time my script should connect to staging and get data. I can't work without this. | 15:48 |
| cjwatson | Then I suggest you work on making it more efficient to reduce the probability of failures. | 15:53 |
| dobey | deploy a local instance of launchpad.dev for testing | 16:08 |
| dobey | if you really want to abuse it | 16:08 |
| HeOS | I don't touch a staging, but he is unavailable again. | 20:59 |
| cjwatson | HeOS: The last incomplete request was just before 18:00 and was from your IP address. | 21:30 |
| cjwatson | Sorry, just before 16:00. | 21:30 |
| cjwatson | I've asked for another appserver restart. | 21:32 |
| cjwatson | It's up again now. | 21:50 |
| === MasterPiece| is now known as MasterPiece | ||
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!