Where to Find This Tab
Go to SEO Rank Genius → Tools & Analytics → robots.txt (first tab under “Crawl Intelligence”).
The Editor
- Shows current robots.txt content
- Validation for syntax errors
- Live preview before saving
- Auto-includes sitemap URL
Common Rules
| Rule | Effect |
|---|---|
Disallow: /wp-admin/ | Blocks admin pages (standard) |
Allow: /wp-admin/admin-ajax.php | Required for AJAX content |
Sitemap: /sitemap.xml | Points crawlers to sitemap |
Never use
Disallow: / unless you want to de-index your entire site.Troubleshooting
Changes Not Taking Effect
- robots.txt is cached by search engines. May take hours to days.
- Use Google’s robots.txt Tester to verify.