Strava

AI companies have evolved into data-hungry organisations that require large datasets to train their models. To meet that need, many AI startups disregard long-standing Internet conventions – e.g., Respecting robots.txt files. This signals to automated crawlers which parts of the website are off-limits and allows them to aggressively scrape the data. This trend has forced…