Robots Directives

Robots directives are instructions placed in a website's robots.txt file or in the HTML header to control how search engine crawlers should interact with and index the site's content.