Close
This topic is way too long, and I cannot answer everything.

The bandwidth is a fixed pipe, 1Gbps, or 100MB/s, which translates to a maximum of 330TB/month at 100% utilization. Since that is not possible, a 300TB utilization is already considered full. It's like the concept of water, everyone needs water at around the same time, and we only have so much pipe capacity to carry them through to everyone.

As for the crash, it wasn't a crash. It was a scheduled server upgrade. The restart failure causes the crash because of an error in fstab config.

However, I believe the kernel was having some issues. Updating the kernel somewhat fixes the problem a bit. That or people stop their crawler for now.