Crawl Delay Directive in robots.txt
The Crawl-delay directive tells bots to wait a certain number of seconds between crawl requests. While not supported by all search engines (notably ignored by Googlebot), it can be useful to reduce strain on servers from aggressive crawlers. Use with caution and test impact on crawl rates before applying it broadly.