More in this section

Forums / General Discussions / Bots still indexing files not in sitemap

Bots still indexing files not in sitemap

2 posts, 0 answered
  1. joe jeanjaquet
    joe jeanjaquet avatar
    2 posts
    Registered:
    17 May 2010
    15 Feb 2011
    Link to this post
    The SEO people upstairs are using an SEO tool to give suggestions about pages.  The software is finding issues with pages that aren't found in the sitemap.xml file.
    My thought is that there is a directory in Sitefinity that is referencing these pages, and isn't being dis-allowed in the robots.txt file.  Is this true, and if so, what is the directory?
    Otherwise, what could be the reason for this?

    Note:  The robots.txt isn't dis-allowing anything at the moment, it just references the sitemap.xml.

    Thanks, Joe
  2. Georgi
    Georgi avatar
    3583 posts
    Registered:
    28 Oct 2016
    17 Feb 2011
    Link to this post
    Hi Joe,

    The fastest way for solving the issue would be to disable the access for bots to those directories. Otherwise we will have to go through every template and check if there are links pointing to the dialogs.

    Greetings,
    Georgi
    the Telerik team
2 posts, 0 answered