Hi Raghu,
The next one is on June 10th, which you're more than welcome to attend.
Bot scraping / abuse has led to many instances of the server node crashing / freezing completely.
So yes, it does put "so much more load" on the website, even on GET requests.
Keep in mind that many pages / endpoints are quite old and do heavy computation on every request.
They are being rewritten to leverage caching and pre-computing, but this will take time.
Nick
On Sun, Jun 1, 2025 at 11:05 PM Raghu Saxena <poiasdpoiasd@xxxxxxxx> wrote:
Dear Robert,
On 5/30/25 8:48 PM, Robert Sparks wrote:
> <snipped>
> Its needed to slow the bot scraping (which is very different now than
> it was in past years). You can avoid it by logging into the
> mailarchive with your datatracker credentials.
Has there been public or documented discussion regarding this topic? I
ask out of curiosity, since I can't imagine the "bot scraping" has led
to so much more load that the website now requires captchas. Just
wondering how much more the traffic / resource consumption increased
that the IETF decided to implement captchas even for GET requests.
Regards,
Raghu Saxena
