| === dkarakasilis1 is now known as dkarakasilis | ||
| === guruprasad changed the topic of #launchpad to: Short Launchpad downtime for upgrading the database server OS coming up | Help contact: guruprasad (05:00-13:00 UTC Mon-Fri) | Launchpad is an open source project: https://dev.launchpad.net/ | This channel is logged: http://irclogs.ubuntu.com/ | User Guide https://help.launchpad.net/ | Support and spam reporting: https://answers.launchpad.net/launchpad | Contribute to our documentation: | ||
| guruprasad | I have stopped some Launchpad services in preparation for upgrading the OS of the Launchpad PostgreSQL database primary via a failover. | 03:19 |
|---|---|---|
| tsimonq2 | guruprasad: Thanks. I'm sure you know, but process-upload is sad. | 03:47 |
| guruprasad | tsimonq2, that is expected because we are preparing for a database primary failover for the OS upgrade. | 03:48 |
| guruprasad | as mentioned in the topic | 03:48 |
| tsimonq2 | No problem, thanks. | 03:48 |
| guruprasad | The outage has started now. | 04:26 |
| guruprasad | The outage lasted ~4 minutes from the time I posted about its start. | 05:04 |
| guruprasad | Things should be back up and running. | 05:04 |
| guruprasad | Will start the stopped Launchpad services soon. | 05:05 |
| guruprasad | And done | 05:13 |
| === guruprasad changed the topic of #launchpad to: Help contact: guruprasad (05:00-13:00 UTC Mon-Fri) | Launchpad is an open source project: https://dev.launchpad.net/ | This channel is logged: http://irclogs.ubuntu.com/ | User Guide https://help.launchpad.net/ | Support and spam reporting: https://answers.launchpad.net/launchpad | Contribute to our documentation: | ||
| tsimonq2 | \o/ | 05:37 |
| RikMills | guruprasad: getting errors from the lp api | 05:42 |
| === JanC is now known as Guest7955 | ||
| === JanC_ is now known as JanC | ||
| RikMills | guruprasad: https://dpaste.org/k1OTc | 05:47 |
| guruprasad | RikMills, thanks for the report. We received some reports just before yours and are investigating. | 05:47 |
| === guruprasad changed the topic of #launchpad to: Launchpad database is in read-only mode and we are investigating | Help contact: guruprasad (05:00-13:00 UTC Mon-Fri) | Launchpad is an open source project: https://dev.launchpad.net/ | This channel is logged: http://irclogs.ubuntu.com/ | User Guide https://help.launchpad.net/ | Support and spam reporting: https://answers.launchpad.net/launchpad | Contribute to our documentation: | ||
| tsimonq2 | Can confirm the same errors. | 05:52 |
| guruprasad | RikMills, tsimonq2, can you retry and check if you are still running into errors? | 05:57 |
| tsimonq2 | guruprasad: Works for me. | 06:00 |
| RikMills | so far so good :) | 06:00 |
| === guruprasad changed the topic of #launchpad to: Help contact: guruprasad (05:00-13:00 UTC Mon-Fri) | Launchpad is an open source project: https://dev.launchpad.net/ | This channel is logged: http://irclogs.ubuntu.com/ | User Guide https://help.launchpad.net/ | Support and spam reporting: https://answers.launchpad.net/launchpad | Contribute to our documentation: | ||
| === JanC_ is now known as JanC | ||
| === graber1 is now known as graber | ||
| === JanC is now known as Guest4504 | ||
| === JanC_ is now known as JanC | ||
| guruprasad | Stopping some Launchpad services for a Launchpad database failover | 07:43 |
| === guruprasad changed the topic of #launchpad to: Some downtime is expected due to an upcoming Launchpad DB failover | Help contact: guruprasad (05:00-13:00 UTC Mon-Fri) | Launchpad is an open source project: https://dev.launchpad.net/ | This channel is logged: http://irclogs.ubuntu.com/ | User Guide https://help.launchpad.net/ | Support and spam reporting: https://answers.launchpad.net/launchpad | Contribute to our documentation: | ||
| guruprasad | There will be an outage now for performing a failover. | 08:44 |
| guruprasad | The outage is over and the web application and API should be up and running now. | 08:56 |
| guruprasad | Will start the stopped background services after verifying that everything is okay. | 08:56 |
| guruprasad | I have started the background services now. | 09:06 |
| === guruprasad changed the topic of #launchpad to: Help contact: guruprasad (05:00-13:00 UTC Mon-Fri) | Launchpad is an open source project: https://dev.launchpad.net/ | This channel is logged: http://irclogs.ubuntu.com/ | User Guide https://help.launchpad.net/ | Support and spam reporting: https://answers.launchpad.net/launchpad | Contribute to our documentation: | ||
| === gurupras- is now known as guruprasad | ||
| === graber1 is now known as graber | ||
| robinhood2021 | I cannot log in to Launchpad. I get the error ID OOPS-c40a1575d3c3dd0cf1bc8cb01c655846 | 17:06 |
| cjwatson | robinhood2021: if you don't get a reply from an LP staff member here, then email feedback@launchpad.net with that OOPS ID and they should be able to fix your account | 17:13 |
| tsimonq2 | The script that updates PPA quotas after deleting items, has that ran recently? | 18:21 |
| tsimonq2 | My PPA is above quota and has been for about 9 hours now. I deleted a bunch of items about 10 hours ago that should have completely resolved that. | 18:22 |
| tsimonq2 | riscv64 builders look sad. Half a dozen idle nodes that aren't picking up new jobs. | 18:38 |
| cjwatson | Seems like a buildd-manager problem rather than just riscv64, since I see a similar problem on other architectures. | 18:42 |
| cjwatson | Maybe a consequence of the DB outage earlier? But I'm just guessing. | 18:42 |
| cjwatson | Given the failover it might have fallen off a perf cliff. | 18:43 |
| tsimonq2 | That sounds about right. | 18:44 |
| tsimonq2 | Looks like there was just a blip, maybe that's somewhat related. | 18:45 |
| tsimonq2 | \o/ I have much more space in my PPA again | 19:23 |
| tsimonq2 | Looks like a fair share of the arm64 builders are stuck at "Cleaning for 5 hours 10 minutes" | 21:58 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!