More in this section

Forums / Bugs & Issues / Google bot picking up over 5000 server errors (500 internal server errors) on our site

Google bot picking up over 5000 server errors (500 internal server errors) on our site

2 posts, 0 answered
  1. Jacques
    Jacques avatar
    427 posts
    Registered:
    28 Jun 2007
    14 Aug 2012
    Link to this post
    Hi there,

    We've recently upgraded our client's site to Sitefinity 4.4 and for brief while these server errors seemed to have gone done to only 33 found. Then we deployed a new feature and suddenly the 500 internal server errors went up exponentially. (over 5000 to be exact)

    Environment
    - Sitefinity 4.x
    - Intelligencia for URL rewriting (which comes from urlrewriter.net)

    Scenario
    - It is a property website.
    - The user enters a search for a location and picks one from a list (a hyperlink list). The URL takes the form http://www.propertycompany.com/property-for-sale/city/state/country
    - The rewriter has the following rules:

      <rewriter>

        <rewrite url="/property-for-sale(.*)" to="/property-search-results?location=$1&amp;searchType=sale"/>

        <rewrite url="/property-to-rent(.*)" to="/property-search-results?location=$1&amp;searchType=lease"/>

        <rewrite url="/properties-on-show(.*)" to="/property-search-results?location=$1&amp;searchType=onshow"/>

        <rewrite url="/property\/(.*)" to="/property-details?propertyId=$1"/>

           <if url="/About-Us/Property-News/11-09-22/The_Big_Disconnect_in_the_Housing_Market.aspx">

          <redirect to="/about-us/Property-News/" permanent="true" />

        </if>

           <if url="/home.asp">

          <redirect to="/home" permanent="true" />

        </if>

      </rewriter>


    It works by redirecting any URL with location data to the /property-search-results page and tacks on the location data as a query string.
    - The user can also select filters which will add to the URL by adding a final segment to the URL structure which would look like this http://www.propertycompany.com/property-for-sale/city/state/country/f-pr1-gr2-fj4

    Issue

    The problems being reported by Google are where the URL is messed up and the server throws an exception (correctly so), because it shouldn't be a natural 404. We actually check this page and if the correct URL format isn't found we throw an exception.

    The failing URLs take many forms of which this one is an example http://www.propertycompany.com/property-for-sale/arbitrary-incorrect-page/f-pr1-gr2-fj4
    So it's as if Goolge is seeing these links. But we've tried a number of scnearios to replicate the problem and we can't do it. We go onto the individual pages and try and see where Google sees these badly constructed links, but we can't find them anywhere?

    Anyone out there have any idea how we can replicate what Google is seeing? Or perhaps someone recognizes this scenario with URL Rewriting and knows where we've gone wrong?

    Thanks,
    jacques
  2. Jacques
    Jacques avatar
    427 posts
    Registered:
    28 Jun 2007
    16 Aug 2012
    Link to this post
    Anyone?
2 posts, 0 answered