User-agent: * Disallow: Disallow: /search Disallow: /download/ Crawl-delay: 10 User-agent: Googlebot Disallow: /*.ris Disallow: /*.bib Allow: /search$ Disallow: /search? Disallow: /download/ User-agent: Amazonbot Disallow: / User-agent: SemrushBot Disallow: / User-agent: MJ12bot Disallow: / User-agent: trendkite-akashic-crawler Disallow: / User-agent: Seekport Crawler Disallow: / User-agent: Linespider Disallow: / Sitemap: https://elifesciences.org/sitemap.xml