1-888-365-2779
+1-888-365-2779
Try Now
More in this section

Forums / Developing with Sitefinity / Combine two Search Indexs

Combine two Search Indexs

8 posts, 2 answered
  1. David
    David  avatar
    78 posts
    Registered:
    29 Dec 2008
    10 Feb 2010
    Link to this post
    Hi all,

      I am running into a problem combining a custom search index provider with the builtin indexing services of Sitefinity.   If I set up a service with the custom index provider alone; the indexing works fine.

      When I set up a service that indexes the custom and pageindex;  Lucene errors out with a System.IO.FileNotFoundException.  I look and sure enough the file is not there in the index folder. 

    [FileNotFoundException: Could not find file 'C:\Users\dpearson\Web\AndersonSitefinity\App_Data\Search\MainSearch\Index\_a.cfs'.]
       System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath) +305
       System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy) +1162
       System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share) +66
       Telerik.Lucene.Net.Store.Descriptor..ctor(FSIndexInput enclosingInstance, FileInfo file, FileAccess mode) +43
       Telerik.Lucene.Net.Store.FSIndexInput..ctor(FileInfo path) +46
       Telerik.Lucene.Net.Store.FSDirectory.OpenInput(String name) +69
       Telerik.Lucene.Net.Index.CompoundFileReader..ctor(Directory dir, String name) +114
       Telerik.Lucene.Net.Index.SegmentReader.Initialize(SegmentInfo si) +128
       Telerik.Lucene.Net.Index.SegmentReader.Get(Directory dir, SegmentInfo si, SegmentInfos sis, Boolean closeDir, Boolean ownDir) +163
       Telerik.Lucene.Net.Index.IndexWriter.MergeSegments(SegmentInfos sourceSegments, Int32 minSegment, Int32 end) +433
       Telerik.Lucene.Net.Index.IndexWriter.Optimize() +108
       Telerik.Search.Engine.BaseIndexer.Close() +22
       Telerik.Search.Engine.Crawler.Index(String provider, String[] urls, LinkedList`1 data, Boolean appendToIndex) +243
       Telerik.Search.Engine.IndexingManager.StartIndexing(IIndexingService service, Boolean appendToIndex) +86
       Telerik.Search.Engine.IndexingService.Index(Boolean appendToIndex) +11
       Telerik.Search.WebControls.Admin.ControlPanel.Service_Command(Object sender, CommandEventArgs e) +287
       System.Web.UI.WebControls.LinkButton.OnCommand(CommandEventArgs e) +108
       System.Web.UI.WebControls.LinkButton.RaisePostBackEvent(String eventArgument) +135
       System.Web.UI.WebControls.LinkButton.System.Web.UI.IPostBackEventHandler.RaisePostBackEvent(String eventArgument) +10
       System.Web.UI.Page.RaisePostBackEvent(IPostBackEventHandler sourceControl, String eventArgument) +13
       System.Web.UI.Page.RaisePostBackEvent(NameValueCollection postData) +175
       System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint) +1565


    Here is my Custom Index Provider which works by itself when not combine with other index providers.

    01.using System;
    02.using System.Collections.Generic;
    03.using Telerik.Framework.Search;
    04.using System.Linq;
    05.  
    06.  
    07./// <summary>
    08./// Summary description for CustomIndexProvider
    09./// </summary>
    10./// 
    11.namespace CustomIndex
    12.{
    13.    public class CustomIndexProvider : IIndexingServiceClient
    14.    {
    15.        public CustomIndexProvider()
    16.        {
    17.            //
    18.            // TODO: Add constructor logic here
    19.            //
    20.        }
    21.  
    22.        #region IIndexingServiceClient Members
    23.  
    24.        public string Description
    25.        {
    26.            get { return "This is my description"; }
    27.        }
    28.  
    29.        public IIndexerInfo[] GetContentToIndex()
    30.        {
    31.            return new IIndexerInfo[0];
    32.        }
    33.  
    34.        public string[] GetUrlsToIndex()
    35.        {
    36.             return GetUrlArray();
    37.        }
    38.  
    39.        public string[] GetUrlArray()
    40.        {
    41.            var db = new Products.ProductsDataContext();
    42.            var records = (from c in db.Floorings
    43.                           select c);
    44.  
    45.            List<string> mysku = new List<string>();
    46.            string myurl = "http://andersonlocal/Flooring.aspx?sku=";
    47.  
    48.            foreach (var item in records)
    49.            {
    50.                mysku.Add(myurl + item);
    51.            }
    52.  
    53.            return mysku.ToArray();
    54.  
    55.        }
    56.  
    57.        public event EventHandler<IndexEventArgs> Index;
    58.  
    59.        public void Initialize(IDictionary<string, string> settings)
    60.        {
    61.             
    62.        }
    63.  
    64.        public string Name
    65.        {
    66.            get { return "CustomIndex"; }
    67.        }
    68.  
    69.        #endregion
    70.    }
    71.}


    Another Note:  I think is similiar to this post: http://www.sitefinity.com/support/forums/sitefinity-3-x/developing-with-sitefinity/pdf-searching-error.aspx  also, I am testing this on Windows 7 with the IIS set up for local website, not a live server.

    Let me know if this is the same problem, or maybe it is my code, or maybe a timeout.  My custom index is 600 pages.

    Thanks,
    David
  2. Radoslav Georgiev
    Radoslav Georgiev avatar
    3370 posts
    Registered:
    01 Feb 2016
    12 Feb 2010
    Link to this post
    Hello David,

    Thank you for using our services.

    I have tried to reproduce this issue with a custom index built like the sample in this article:http://www.sitefinitywatch.com/notes/09-06-24/Working_with_Sitefinity_Search.aspx . However instead of getting the Urls to index I am getting content items using the GetContentToIndex method implementation and the indexer works both when alone and combined with other search index type. I have tested this with a set of about 700 content items. Are you indexing content from a custom module? If this module is generic content based maybe you can try an approach similar to the one in the article. 

    Unfortunately we have not been able to fix the issue related to the one in the forum thread you are referring to. I am afraid that if the method mentioned above does not work you will have to use the solution in the forum thread - make the search results control use two separate indexes.

    Sincerely yours,
    Radoslav Georgiev
    the Telerik team

    Do you want to have your say when we set our development plans? Do you want to know when a feature you care about is added or when a bug fixed? Explore the Telerik Public Issue Tracking system and vote to affect the priority of the items.
  3. David
    David  avatar
    78 posts
    Registered:
    29 Dec 2008
    12 Feb 2010
    Link to this post
    It does look like I will need to do two different search indexes.  
     I am searching a products table from an outside database. 

    The product page (flooring.aspx?sku=xxx) page is a control that queries the outside database to pull in the flooring info data. (example: Brand, color, Thickness, width, Species, etc....)

    If I am thinking correctly, the CustomIndex should parse each page (flooring.aspx?sku=001 to flooring.aspx?sku=633) and index that data for search function.  So I am loading the page URLs using GetUrlsToIndex() like Gabe example.  I created a method that would load string List with the 633 Urls then converts it to a string array for GetUrlsToIndex.

    public string[] GetUrlsToIndex()
    {
         return GetUrlArray();
    }
     
    public string[] GetUrlArray()
    {
        var db = new Products.ProductsDataContext();
        var records = (from c in db.Floorings
                       select c);
     
        List<string> mysku = new List<string>();
        //string myurl = "http://andersonlocal/Flooring.aspx?sku=";
        string myurl = "/Flooring.aspx?sku=";
     
        foreach (var item in records)
        {
            mysku.Add("[url]/Flooring.aspx?sku=" + item.SKU);
        }
     
        return mysku.ToArray();
     
    }


    I have been debating if the myurl string needs to be a full url like http://andersonlocal/flooring.aspx?sku or /flooring.aspx?sku=.   I will go back over Gabe's post on custom index to see if I miss anything.

    Another item I notice, some of my SKU have a value like this:  H46.3.. I am wondering if this could cause problems with a url?

    Thanks,
    David
  4. Radoslav Georgiev
    Radoslav Georgiev avatar
    3370 posts
    Registered:
    01 Feb 2016
    15 Feb 2010
    Link to this post
    Hello David ,

    Thank you for getting back to me.

    You are guessing correctly. What the index does is that it loads all the Urls in the string array and parses each page to index relevant data. If the product pages are in the same website as the search is running I think that using the relative Url will be a better approach. If you utilize the full path you might run in to problems if you change domain names for example and you still have hardcoded Url. If you wish to get the full path consider using the HttpContext.Current.Request.Url.Host which will give you the domain name.

    I do not think that having dot characters in the Url will be a problem. However if it turns out that it is you might consider encoding the Urls so that those characters are represented by their codes.

    Regards,
    Radoslav Georgiev
    the Telerik team

    Do you want to have your say when we set our development plans? Do you want to know when a feature you care about is added or when a bug fixed? Explore the Telerik Public Issue Tracking system and vote to affect the priority of the items.
    Answered
  5. David
    David  avatar
    78 posts
    Registered:
    29 Dec 2008
    16 Feb 2010
    Link to this post
    Hi Radoslav,
        I got my code working today.  I ended up changing myurl variable to http://andersonlocal/Flooring/xxxx.aspx.  The xxxx is the SKU number.  I also did a URL rewrite in the web.config file.   Instead of just indexing one page, it indexes all of the products.   It now appears to be working great.  I did not see any problems with . in the sku number, that I know of.  I think I saw some duplicates in the results, I will investigate more on Tuesday.

       I completely forgot about the Request.URL.Host.   I will add that to my class in the morning.  We should have the site live Wed, if our new servers works good.   

    We will be submitting a case study to Sitefinity.

    David
  6. David
    David  avatar
    78 posts
    Registered:
    29 Dec 2008
    16 Feb 2010
    Link to this post
    I have one quick knowledge question.  Does the search engine use the fieldsInfoProvider.xml when indexing or is it the results control that uses the filter?

    Thanks,
    David
  7. Ivan Dimitrov
    Ivan Dimitrov avatar
    16072 posts
    Registered:
    16 Jun 2017
    16 Feb 2010
    Link to this post
    Hi David ,

    The fieldsInfoProvider.xml is used during the indexing. It specifies which fields should be included in the snippet of the ResultItem. All other data is located in the segments files generated by Lucene search engine.

    Greetings,
    Ivan Dimitrov
    the Telerik team

    Do you want to have your say when we set our development plans? Do you want to know when a feature you care about is added or when a bug fixed? Explore the Telerik Public Issue Tracking system and vote to affect the priority of the items.
    Answered
  8. David
    David  avatar
    78 posts
    Registered:
    29 Dec 2008
    16 Feb 2010
    Link to this post
    Thanks all,  I know custom search index now... :)

    David
Register for webinar
8 posts, 2 answered