Hubi@feddit.de to Reddit@lemmy.worldEnglish · 7 months agoReddit shares plunge almost 25% in two days, finish the week below first day closewww.cnbc.comexternal-linkmessage-square173fedilinkarrow-up1850arrow-down115cross-posted to: reddit@lemmy.mlmodcoord@lemmit.onlinetechnology@lemmit.onlinetechnology@lemmit.online
arrow-up1835arrow-down1external-linkReddit shares plunge almost 25% in two days, finish the week below first day closewww.cnbc.comHubi@feddit.de to Reddit@lemmy.worldEnglish · 7 months agomessage-square173fedilinkcross-posted to: reddit@lemmy.mlmodcoord@lemmit.onlinetechnology@lemmit.onlinetechnology@lemmit.online
minus-squareSacrificedBeans@lemmy.worldlinkfedilinkarrow-up12·7 months agoRobots.txt is a compliance standard afaik, more like an advisory guide. It cannot enforce bans on scrapers, it sets some polite boundaries for the website.
Robots.txt is a compliance standard afaik, more like an advisory guide. It cannot enforce bans on scrapers, it sets some polite boundaries for the website.