CUNY Academic Commons

User Diane Krauthamer writes:
"I am doing some SEO cleanup for newlaborforum.cuny.edu and have found that Google crawls a number of URLs that point to the main login page. Because of this I would like to make a change to the robots.txt file, which should be on the server. The change I would like to make is to add the following line:

This should prevent Google from crawling those specific pages.

Disallow: /wp-login.php

I can provide an example of the crawl error report if need be, or documentation to support making this fix, but essentially pages that appear to have the following are being crawled when they shouldn't be:

. Thanks again for your help.

Please let me know if you have any questions, and thank you for your help."