# robots.txt User-agent: rdfbot/1.0 Crawl-Delay: 1800 User-agent: Yandex Crawl-Delay: 599 User-agent: * Disallow: