Generate a robots.txt file for your website for free. Robots.txt Generator is an SEO tool used to create a text file that is uploaded to your website's root directory. When web spiders and robots crawl your site for information, they read your robots.txt file as directives to identify parts of your website that may be blocked. Using our Robots.txt Generator allows you to customize your robots.txt file to keep web crawlers from accessing all or certain parts of your website.
What is a Robots.txt File? "The robots exclusion protocol (REP), or robots.txt is a text file webmasters create to instruct robots (typically search engine robots) how to crawl and index pages on their website." Source - MOZ
See also: Wordpress Robots.txt Default File