User-agent: * Disallow: /pricing User-agent: Slurp Crawl-delay: 900 User-Agent: *