Here is a quick tip how you can filter out some of the core files of WordPress that can get indexed by search engines. This method has been tested with Google.
- Create Robots.txt
- Add the following directives:
Disallow: /cgi-bin/
Disallow: /wp-admin/
Disallow: /wp-content/
Disallow: /wp-content/themes/K2RC6/comments-ajax.php
Disallow: /wp-includes/
Disallow: /wp-login.php
Disallow: /wp-login.php?action=register
Disallow: /wp-login.php?action=lostpassword
Disallow: /wp-trackback.php
- Generate a Sitemap
- Add your sitemap to the Robots.txt
sitemap: http://money.tradeo.biz/sitemap.xml.gz
Now you will avoid having system pages (like mysql queries/error and logins) indexed by search engines.
If you use affiliate links to monetize your site, add the URL’s to your Robots.txt file. This will keep them usable, but search engine spiders will not be able to follow them, thus, increasing your page value (yes, because you have less outgoing links).
If you find any good indexing optimisations for WordPress, drop a comment here or maybe a mail to let me know. Thanks!
See also: WordPress SEO 101