| sandbag | hello, are there any upstream https mirrors? | 08:15 |
|---|---|---|
| sandbag | apart from archive.ubuntu.com | 08:18 |
| Berge | sandbag: What do you mean by upstream? | 08:30 |
| Berge | There's a big network of country level mirrors, on the form $CC.archive.ubuntu.com an $CC.releases.ubuntu.com | 08:31 |
| sandbag | Berge: like the one which is the first source of truth | 08:33 |
| sandbag | are those country level mirrors mirroring anything else or not? | 08:33 |
| Berge | sandbag: Mirroring other things than Ubuntu? | 08:34 |
| Berge | Many of them would be, yes | 08:34 |
| Berge | There is a system with trigger-based updating of country level mirrors, so they're fairly up to date | 08:35 |
| Berge | Typically | 08:35 |
| Berge | …minutes or hours behind | 08:35 |
| sandbag | > Many of them would be, yes. where can i find that? | 08:36 |
| sandbag | Berge: no, not other things than ubuntu. what i meant is are all those mirros building from scratch or just mirroring? | 08:37 |
| Berge | sandbag: https://launchpad.net/ubuntu/+archivemirrors | 08:38 |
| sandbag | i saw that, it doesn't mention which mirrors strictly use https | 08:38 |
| Berge | But it will say if a mirror is up to date | 08:38 |
| Berge | HTTPS isn't a requirement for mirror operators. Why do you want that? | 08:39 |
| sandbag | because of security? | 08:39 |
| sandbag | i want to mirror https only | 08:39 |
| Berge | The security of apt does not rely on transport level integrity | 08:39 |
| Berge | Packages are signed and verified | 08:39 |
| sandbag | yes im aware. but i still dont want to be snooped by my ISP | 08:39 |
| sandbag | why is https not a requirement? i fail to see that. it doesn't cost anythig | 08:40 |
| Berge | The ISP will be able to ascertain _that_ you're downloading something from a Ubuntu mirror, mostly | 08:40 |
| Berge | HTTPS absolutely costs something | 08:40 |
| Berge | Orders of magnitude more CPU load on the mirror, for instance | 08:40 |
| sandbag | there are other things as well, it opens up the possibility of ssl striping attacks | 08:40 |
| Berge | No, it doesn't | 08:40 |
| Berge | Or, not in a way that's relevant here | 08:41 |
| Berge | Anyway, I don't know of a list of HTTPS enabled mirrors | 08:41 |
| Berge | The one I operate happens to be | 08:41 |
| Berge | But then you have to trust me instead of your ISP (-: | 08:41 |
| sandbag | where does your mirror mirror from ? | 08:41 |
| Berge | sandbag: I didn't quite understand your question about building from scratch | 08:41 |
| Berge | You can easily build your own mirror from scratch | 08:41 |
| Berge | The one I operate, no.{releases,archive}.ubuntu.com, is using update triggers | 08:42 |
| sandbag | yes that's a lot of work which is why i wanted mirrors who serve https without mirroring from http mirrors | 08:42 |
| Berge | It gets triggered when there are updates, and then syncs those updates from a upstream | 08:42 |
| Berge | It's immaterial where it syncs from, security wise | 08:42 |
| sandbag | Berge: so ur building the whole repo? | 08:42 |
| Berge | Since you ask, it's using rsync, not HTTP | 08:42 |
| Berge | And without any encryption on the wire | 08:43 |
| Berge | sandbag: Not sure what you mean by building | 08:43 |
| Berge | It's two rsync commands | 08:43 |
| sandbag | the ubuntu packages, etc from source | 08:43 |
| Berge | I'm certainly not building the packages | 08:43 |
| Berge | It's just downloading binary and source packages from a different mirror | 08:43 |
| sandbag | and which mirror is that | 08:44 |
| sandbag | ur understanding what im asking? | 08:44 |
| Berge | Depends a bit, but typically from the excellent people at https://www.accum.se/ | 08:44 |
| Berge | sandbag: I don't quite understand what you want to know, no | 08:45 |
| Berge | Nor why it's relevant | 08:45 |
| sandbag | i wanna setup an https mirror but the mirror from which i will mirror has to mirror from https exclusively | 08:45 |
| sandbag | all the way to the top | 08:45 |
| Berge | I don't think any mirror does that | 08:45 |
| Berge | All mirrors sync between themselves with rsync | 08:45 |
| sandbag | where u will find a mirror which does building | 08:45 |
| Berge | HTTP would be prohibitively expensive bandwidth wise | 08:45 |
| Berge | sandbag: That _builds_ the packages? | 08:46 |
| Berge | That's not a mirror's task | 08:46 |
| Berge | Canonical operates build farms that builds binary packages | 08:46 |
| sandbag | so where do u get those packages from? | 08:46 |
| Berge | As I said, a different mirror | 08:46 |
| sandbag | man i know that. im talking about at top | 08:47 |
| Berge | Again, the security of apt packages does _not_ rely on transport level mechanisms, such as HTTPS | 08:47 |
| sandbag | some mirror will eventually have to take thos packages from directly canonical | 08:47 |
| Berge | Packages are signed with a completely different system | 08:47 |
| Berge | sandbag: yes | 08:47 |
| sandbag | so how does canonical gives it | 08:47 |
| sandbag | to those few mirrors | 08:47 |
| Berge | rsync | 08:47 |
| sandbag | is it https | 08:47 |
| Berge | No, it's rsync | 08:47 |
| sandbag | is there doc for that? | 08:47 |
| Berge | For the program rsync? | 08:47 |
| sandbag | no for distribution of canonical via rsync | 08:48 |
| Berge | I don't, no | 08:49 |
| Berge | It's a trivial thing | 08:49 |
| sandbag | can i create the infrastructure for building those packages and not take from canonical | 08:49 |
| Berge | I don't understand the question | 08:50 |
| Berge | Do you want to build the packages from scratch, and not download the binary packages from a mirror? | 08:50 |
| sandbag | yes. i dont wanna mirror another mirror | 08:50 |
| Berge | But you have to download the source packages from a mirror anyway | 08:51 |
| sandbag | because there arent any upstream https mirror | 08:51 |
| Berge | Why do you want that? | 08:51 |
| sandbag | because https + pgp is better than just pgp | 08:51 |
| Berge | That is not how the security model works | 08:51 |
| sandbag | defense in depth? | 08:52 |
| Berge | https://archive.ubuntu.com/ is probably the closes you can come to that | 08:52 |
| sandbag | does it mirror from anywhere else? | 08:52 |
| Berge | I don't know how it works internally | 08:53 |
| sandbag | it would be honestly better to have a column in mirror list to see who mirrors whom | 08:53 |
| Berge | That's completely irrelevant | 08:53 |
| Berge | And changes | 08:53 |
| sandbag | why? | 08:53 |
| sandbag | what if someone wants to know the true source | 08:53 |
| Berge | Planned and unplanned maintenance, for instanc | 08:53 |
| Berge | I'll move the DNS pointers for the mirror we operate here when it's undergoing maintenance | 08:54 |
| Berge | sandbag: I don't think I have more to add here | 08:56 |
| Berge | The security model for package integrity is not based on transport level security in any way, and thus mirror operations behind the scenes optimise for things like bandwidth and CPU | 08:57 |
| Berge | This is what allows randoms (like me) to operate Ubuntu mirrors, without Canonical having to rely on my integrity, or the integrity of the server, its physical security, my security practices, and so on | 08:58 |
| sandbag | ok | 09:09 |
| maswan | the project/trace directory will tell you the mirror hierarchy | 09:10 |
| maswan | but again, it is all over rsync and not http/https anyhow | 09:11 |
| maswan | and the main reason why cctld.archive.ubuntu.com doesn't use https is that it'll get repointed to other mirrors as needed so that it'll always work | 09:11 |
| Berge | They left | 09:17 |
| kotodama | ah yeah they left | 09:18 |
| kotodama | strange question | 09:18 |
| kotodama | there is no source mirror, Canonical archive servers are the source... | 09:18 |
| kotodama | I can understand the desire to not be snooped on by their ISP, but then I don't really get what they were aiming for here | 09:19 |
| Berge | Their ISP will trivially see that you're downloading packages from an Ubuntu mirror anyway, in most cases | 09:20 |
| kotodama | indeed | 09:21 |
| kotodama | and they can still deep inspect / check for sizes to determine the package | 09:22 |
| kotodama | makes it more difficult though | 09:22 |
| kotodama | we enabled HTTPS on our archive servers because well... it's 2024 :) | 09:22 |
| maswan | kotodama: we have https on our mirror, but not for the cctld stuff since that'll break when it gets repointed. same for debian | 09:23 |
| kotodama | heh indeed | 09:24 |
| mgedmin | I think interest in https for apt archives grew after that CVE where apt would process http response headers badly, leading to remote code execution or something like that? | 10:03 |
| mgedmin | suddenly all the "why care about mitm, the release file is validated via gpg" responses seemed quieter | 10:04 |
| Berge | It's a thin layer of extra security, though | 10:09 |
| Berge | You'd have to trust the randoms operating the mirrors | 10:09 |
| grove | Currently the CPU requirements to mirror a lot of stuff (we mirror ~90 projects on our mirror server) are fairly minor, if we had to build all that stuff, that would increase many many times. We would probably have to consider cutting back on the number of projects we mirror | 10:22 |
| Berge | _Building_ the packages would be absolutely unreasonable | 10:23 |
| Berge | And without reproducible builds, it'd be actively harmful to security | 10:23 |
| asomerset | wait don't all the cctld mirror operators not sync direct from canonical "CDN" anyway? | 11:29 |
| Berge | asomerset: no, there are tiers | 11:35 |
| Berge | Our mirror (no.[ar].u.c) has excellent connectivity to maswan's crowd's mirror | 11:36 |
| asomerset | UBUARC_MIRROR=archive.ubuntu.com::ubuntu/ | 11:36 |
| asomerset | i seem to be drinking the koolaid straight from the fountain | 11:36 |
| asomerset | for most of Africa | 11:37 |
| Berge | Firehose fountain | 11:38 |
| Berge | asomerset: You run an Africa mirror? | 11:38 |
| Berge | African, even | 11:38 |
| asomerset | yep its officially in Kenya, but serves most of East and Southern Africa | 11:53 |
| Berge | cool | 11:54 |
| Berge | What bandwidth speeds to you get to a.u.c while syncing? | 11:55 |
| asomerset | its hard to tell exactly given the otherstuff we are mirroring and potential overlaps i'd be guessing we sustain 100-200mbps | 11:56 |
| asomerset | we have peaks of 500mbps and sometimes even 1gbps on download | 11:56 |
| asomerset | we cruise at 1-1.5gbps serving throughout the day with peaks at 2gbps | 11:57 |
| asomerset | if you feel like being nosy - https://stats.mirror.liquidtelecom.net/ | 11:59 |
| asomerset | i am sad that releases and cdimages doesn't push more traffic | 12:00 |
| Berge | yeah, it's on decline in general | 12:06 |
| Berge | awstats is a blast from the past (-: | 12:07 |
| maswan | We use analog :) | 12:21 |
| === amurray_ is now known as amurray | ||
| sarnold | wow, 2.17 GB on 404s alone https://stats.mirror.liquidtelecom.net/cgi-bin/awstats.pl?config=ubuntu-archive.mirror.liquidtelecom.com | 21:48 |
| tomreyn | 33.2 % of accessing browsers used the "CloudFlare" user agent | 21:54 |
| tomreyn | i wonder why that would be desirable - wouldn't you want it to be all or nothing? | 21:55 |
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!