How crawlers impact the operations of the Wikimedia projects
diff.wikimedia.org
Since the beginning of 2024, the demand for the content created by the Wikimedia volunteer community – especially for the 144 million images, videos, and other files on Wikimedia Commons – has grow…
  • Strawberry@lemmy.blahaj.zoneEnglish
    2·
    2 days ago

    Sure but we’re in the comments section of an article about wikipedia being crawled, which is silly because they could just download a snapshot of wikipedia

    • TXL@sopuli.xyzEnglish
      1·
      6 hours ago

      That’s right. It’s not humans making careful decisions about what to download. It’s a program that follows links and saves files.