guruprasad | cjwatson, knock knock! Do you have a few minutes for helping me understand some finer lp-archive details? | 13:33 |
---|---|---|
guruprasad | Thanks in advance! | 13:34 |
cjwatson | guruprasad: what's up? | 17:50 |
guruprasad | cjwatson, is there a limit configured somewhere in Launchpad about the retention period of the history for PPAs? Asking because one user complained that their PPA only has the history for the last 2 weeks instead of all the way back to March 2023. | 18:03 |
guruprasad | I tried digging through the code but had no success in finding anything relevant. | 18:04 |
guruprasad | I was also trying to understand if there were some scenarios where the PPA's history timespan got reduced by things getting deleted from the librarian due to various conditions. | 18:05 |
cjwatson | guruprasad: lib/lp/soyuz/scripts/expire_archive_files.py | 18:05 |
cjwatson | guruprasad: Pretty sure I advertised the caveats about PPA retention when we launched the service | 18:06 |
guruprasad | I keep going back to your announcement post on the internal discourse but didn't find anything regarding this. | 18:08 |
guruprasad | And for example, the Launchpad team PPA has history going back much farther than the PPA for which this issue was reported and it is not in the allow-list in the expire_archive_files.py script. | 18:09 |
cjwatson | Expiry only happens to packages that have been superseded/deleted though | 18:11 |
cjwatson | Pretty sure you'll find that those are gone from the LP team PPA too | 18:11 |
cjwatson | I can't check the internal announcement any more of course, but I mentioned it in https://www.youtube.com/watch?v=hxDPqRDY5Lk (13:10 ish) | 18:11 |
guruprasad | Thanks! This is useful. | 18:13 |
cjwatson | (specifically, to publications that have dateremoved set) | 18:13 |
cjwatson | Librarian deletion is not going to be a useful avenue of investigation here - librarian-gc will never delete anything that's referenced from another table | 18:14 |
guruprasad | I think I got the answer to my question - unless allowlisted, PPAs tend to have a short history. Private PPAs have full history and have to be blocklisted to expire things earlier than otherwise. | 18:16 |
guruprasad | And I am assuming the criteria to be considered by the LP management on adding more PPAs to the allowlist has to be done on a case-to-case basis, due to the increased storage cost associated with retaining more history for public PPAs. | 18:17 |
cjwatson | guruprasad: It's less of an issue now than when it was when expiry was first put in place - available space for the librarian is much bigger now. But I suspect it's still the case that indefinite or even very much longer default expiry for PPAs would be impractical - there's a lot of throughput there. | 18:24 |
guruprasad | Yeah, that makes sense. | 18:26 |
guruprasad | And this (the retention period limitation) is something I need to consider and mention at the time of advertising the snapshot service for the ESM archives. | 18:27 |
cjwatson | guruprasad: How come? https://git.launchpad.net/launchpad/commit/?id=a22da89fe72eabfb53250cd82a9857f6be7bcd8a | 18:37 |
guruprasad | There are other archives under esm.ubuntu.com that are not covered by the allowlist changes here and I will have to add any additional PPAs backing those to the list. | 18:42 |
guruprasad | But you're right - the archives in that allowlist are retained indefinitely. | 18:43 |
guruprasad | and not expired by that script. | 18:43 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!