lemmyreader@lemmy.ml to Security@lemmy.mlEnglish · 2年前Amazon storing classified US government documents improperlylemmy.mlimagemessage-square72fedilinkarrow-up1536arrow-down123file-text
arrow-up1513arrow-down1imageAmazon storing classified US government documents improperlylemmy.mllemmyreader@lemmy.ml to Security@lemmy.mlEnglish · 2年前message-square72fedilinkfile-text
minus-squareAmbiguousProps@lemmy.todaylinkfedilinkEnglisharrow-up7·2年前robots.txt doesn’t have to be followed. It doesn’t block crawling.
robots.txt doesn’t have to be followed. It doesn’t block crawling.