Recent problems with Google search of MrExcel.com

[Jerry Sullivan]

Have other people been experiencing problems using Google to find specific threads they have posted to or read?

For the last 12-18 months, I've been using Google to help track down code examples that I've previously read or written.

If I want to find a snippet from a post I participated in earlier this year that shows to open files using Shell, I enter these parameters and 2-3 hits might be returned making it very easy to track down the one I had in mind...

js411 shell site:mrexcel.com

It's been a very effective resource up until about 1-2 months ago.
Since then, many searches have come up empty or missing specific threads that in fact exist matching those search parameters.

Are others experiencing the same, and any ideas as to the cause?

 

[Robert Mika]

Yep...
I got the same.
I used to track my own post and the serach was quite accurate, now I'm getting almost nothing.
I think(I am maybe wrong) the problem lays that after forum was rebulid lots of links to old threads is not working anymore so the google robot is not pick them up....

 

[Aladin Akyurek]

Yep...
I got the same.
I used to track my own post and the serach was quite accurate, now I'm getting almost nothing.
I think(I am maybe wrong) the problem lays that after forum was rebulid lots of links to old threads is not working anymore so the google robot is not pick them up....

Old links which Google already has stored in its DB do not work because they became inaccessible due to the recent board update.

 

[Jerry Sullivan]

Old links which Google already has stored in its DB do not work because they became inaccessible due to the recent board update.

Thanks Robert and Aladin for your replies.

The observed results aren't consistent with that potential cause. The problem isn't that the returned links don't work, it's that the search isn't finding pages that match the search parameters.

A specific example might help. I went searching for a thread in which Aladin posted a graphic that visually explained how Binary Search works with the Lookup function.

aladin "binary search" site:mrexcel.com

There's 56 Hits. Dates include 2003, 2004, 2005, 2007, 2009, so if the recent problem with Google search is due to Old links in the DB becoming inacessible, why were these links returned and not others? Perhaps these were added more recently by a search bot or due to being referenced in updated pages?

Drilling deeper to try to find that elusive thread, I recalled that the discussion included how Vlookup works when there are multiple matches.

aladin vlookup multiple "binary search" site:mrexcel.com

This returns 6 hits. None of them are the thread I want to find, but several of them have links to the elusive thread:
http://www.mrexcel.com/forum/excel-questions/310278-vlookup-multiple-matches-match-returned.html

So there it is, but why didn't Google return a direct link to that thread as a result of my last search?
Luckily this is an often referenced thread, if it were not, would it be orphaned from the search world?

Now that we have the thread, we can experiment further.
Searching for the thread title:
"vlookup with multiple matches - which match is returned" site:mrexcel.com

This yields 52 results, the first of which is the desired thread.
So the Google DB seems to have the title, but not content of the thread.

A note below the result provides a link to a possible explanation:
A description for this result is not available because of this site's robots.txt – learn more.
https://support.google.com/webmasters/bin/answer.py?hl=en&answer=156449

Is the problem that the MrExcel site has a robot.txt file that is (perhaps unintentionally) blocking search engines from accessing some pages?

 

[Aladin Akyurek]

Jerry,

This is very inquisitive note. I think Tracy and Bill should read this...

Thanks a million,

Aladin

 

[shg]

Curious timing, Jerry. I was going to vector someone at excelforum (That Travelling Salesman/Sat-Nav style problem....) to the thread where you posted some branch-and-bound code for the TSP problem. I tried both a local search and Google site search for several words that I know appeared in the thread and was unable to locate it.

 

[Jerry Sullivan]

Jerry,

This is very inquisitive note. I think Tracy and Bill should read this...

Thanks a million,

Aladin

My thanks to you Aladin. Half of my searches begin with...
site:mrexcel.com aladin <insert topic here>

I hope the solution is as simple as changing the settings in a robots.txt file.

If so, it shouldn't take long for Google's search bots to re-index the entire site's content.

 

[Jerry Sullivan]

Curious timing, Jerry. I was going to vector someone at excelforum (That Travelling Salesman/Sat-Nav style problem....) to the thread where you posted some branch-and-bound code for the TSP problem. I tried both a local search and Google site search for several words that I know appeared in the thread and was unable to locate it.

Hi shg,

Here is a link to that thread...
http://www.mrexcel.com/forum/excel-questions/633361-looking-some-script-help.html#post3173665

I couldn't find it through Google or this site's search. Got lucky in that I hadn't deleted a subscription email message from this thread.

 

[shg]

Thanks, Jerry.

 

[Jerry Sullivan]

I think Tracy and Bill should read this...

It's been a couple days and I don't know if Tracy or Bill has seen this.

Could one of the Moderators please bring it to their attention to find out if this site might have a Robots.txt file or some setting that is unintentionally blocking content from searches?

It's likely that if helpers can't find threads that they know exist; then the web community as a whole will have a harder time finding relevant hits.

This data from alexa.com shows a sharp decline in MrExcel.com traffic rank over the last month (approximately the time the problems with searches appeared to start).