Robots.txt
A robots.txt file tells search engine crawlers which URLs the crawler can access on your site. This file exists primarily to avoid overloading your site with requests.
Easy Eatery generates a standard robots.txt file for your website by default. You can view the contents of your robots file by visiting your {YOUR DOMAIN}/robots.txt.
You can use the robots.txt editor to add directives to the robots.txt file to change how specific crawlers access your website.
Add content here as you would to a standard robots.txt file, and upon saving, the robots.txt file will be refreshed with the updated content.
This feature is for advanced users when additional directives need to be added to the file. Any mistakes or wrong directives could adversely affect your website's performance in organic search or prevent it from being indexed entirely.
