Tea@programming.dev to Technology@lemmy.worldEnglish · edit-23 days agoAI crawlers cause Wikimedia(The umbrella organization of Wikipedia and a dozen or so other crowdsourced knowledge projects) Commons bandwidth demands to surge 50%.diff.wikimedia.orgexternal-linkmessage-square80fedilinkarrow-up1708arrow-down17cross-posted to: technology@beehaw.orgtechnology@lemmy.zip
arrow-up1701arrow-down1external-linkAI crawlers cause Wikimedia(The umbrella organization of Wikipedia and a dozen or so other crowdsourced knowledge projects) Commons bandwidth demands to surge 50%.diff.wikimedia.orgTea@programming.dev to Technology@lemmy.worldEnglish · edit-23 days agomessage-square80fedilinkcross-posted to: technology@beehaw.orgtechnology@lemmy.zip
minus-squaregrysbok@lemmy.sdf.orglinkfedilinkEnglisharrow-up2·2 days agoBots lie about who they are, ignore robots.txt, and come from a gazillion different IPs.
minus-squarecatloaf@lemm.eelinkfedilinkEnglisharrow-up2·2 days agoThat’s what ddos protection is for.
Bots lie about who they are, ignore robots.txt, and come from a gazillion different IPs.
That’s what ddos protection is for.