Support #9684
closedSEO cleanup for newlaborforum.cuny.edu
0%
Description
User Diane Krauthamer writes:
"I am doing some SEO cleanup for newlaborforum.cuny.edu and have found that Google crawls a number of URLs that point to the main login page. Because of this I would like to make a change to the robots.txt file, which should be on the server. The change I would like to make is to add the following line:
This should prevent Google from crawling those specific pages.
Disallow: /wp-login.php
I can provide an example of the crawl error report if need be, or documentation to support making this fix, but essentially pages that appear to have the following are being crawled when they shouldn't be:
. Thanks again for your help.
Please let me know if you have any questions, and thank you for your help."
Updated by Marilyn Weber almost 6 years ago
Diane is the Web Designer for the New Labor Forum.
She's had a few requests, and each of them require a bit of back and forth here on Redmine. I'm not complaining here, but I've suggested to her that she really should join. She was invited a few months ago - https://redmine.gc.cuny.edu/issues/8906. Can you tell if she has joined? Can the invite be sent again?
Or you can tell me if there's an objection to that. Thanks!
Updated by Boone Gorges almost 6 years ago
- Status changed from New to Rejected
Thanks for the report. I'm afraid the robots.txt file in use on the Commons is shared between all sites, so that a change will affect all sites. And I don't think that we want to disable indexing of the login page across the Commons, since there may well be people who specifically look for the Commons login page via search engines.
I see that Diane does have an account: dkrauthamer, with an sps.cuny.edu email address. She should be able to go through the Lost Password routine. https://redmine.gc.cuny.edu/account/lost_password