User-agent: Sogou Spider Allow: / User-agent: Bingbot Allow: / User-agent: Baiduspider Allow: / User-agent: 360Spider Allow: / User-agent: Yahoo Slurp Disallow: / User-agent: Bytespider Disallow: / User-agent: YandexBot Disallow: / User-agent: DuckDuckBot Disallow: / User-agent: Qwantify Disallow: / User-agent: MSNBot Disallow: / User-agent: Teoma Disallow: / User-agent: ia_archiver Disallow: / User-agent: twiceler Disallow: / User-agent: YisouSpider Disallow: / # 只放行谷歌网页正文蜘蛛 User-agent: Googlebot Allow: / # 屏蔽谷歌图片、谷歌移动端爬虫 User-agent: googlebot-image Disallow: / User-agent: googlebot-mobile Disallow: / User-agent: gptbot Disallow: / User-agent: amazonbot Disallow: / User-agent: SemrushBot Disallow: / User-agent: ImagesiftBot Disallow: / User-agent: applebot Disallow: / User-agent: GPT4Bot Disallow: / User-agent: ClaudeBot Disallow: / User-agent: MetaBot Disallow: / User-agent: AhrefsBot Disallow: / User-agent: MajesticBot Disallow: / User-agent: MJ12bot Disallow: / User-agent: BLEXBot Disallow: / User-agent: PetalBot Disallow: / User-agent: DotBot Disallow: / User-agent: Rogerbot Disallow: / User-agent: LinkpadBot Disallow: / User-agent: SeekportBot Disallow: / User-agent: SpeedySpider Disallow: / User-agent: SurveyBot Disallow: / User-agent: Spbot Disallow: / User-agent: Curl Disallow: / User-agent: Wget Disallow: / User-agent: Scrapy Disallow: / User-agent: Python-urllib Disallow: / User-agent: Go-http-client Disallow: / User-agent: HttpClient Disallow: / User-agent: Java Disallow: / User-agent: Lua Disallow: / User-agent: Node.js Disallow: / User-agent: Discordbot Disallow: / User-agent: TelegramBot Disallow: / User-agent: TikTokBot Disallow: / User-agent: Pinterestbot Disallow: / User-agent: Redditbot Disallow: / User-agent: Mediumbot Disallow: / User-agent: LingueeBot Disallow: / User-agent: CoccocBot Disallow: / User-agent: NaverBot Disallow: / User-agent: NateBot Disallow: / User-agent: Inktomi Disallow: / User-agent: LookSmart Disallow: / User-agent: InfoSeek Disallow: / User-agent: ClaudeBot Disallow: / # 所有未单独声明的蜘蛛 全部禁止 User-agent: * Disallow: /