I just installed apt cacher ng for catching my apt upgrade packages and saw a huge time improvement even though I have a good internet connection. It act as a proxy and caches the response packages.
Do you run something similar? Or maybe even run a local repo mirror? Warning, they are really big so I don’t think it is recommended unless you really need almost everything.
I run the same one and it works great. I’ve had to adjust the https something or other blocker on it occasionally as I add additional sources.