More in this section

Forums / Developing with Sitefinity / Sitefinity 5.1 - Adding Search Scope for Custom Module

Sitefinity 5.1 - Adding Search Scope for Custom Module

7 posts, 0 answered
  1. Nathan
    Nathan avatar
    3 posts
    Registered:
    15 Aug 2011
    23 Jul 2012
    Link to this post
    I've just spent a whole afternoon looking for a solution to this and haven't found anything but dead ends. My goal is the following: I have several custom modules (coded from scratch, not from the module builder) that have items that I would like to integrate into the Sitefinity search engine. 

    My understanding is that I need to do something (whatever that is) that will make my custom modules show up as additional options on the Create/Edit Search Index page. 

    I've read comments from Telerik staff from 2011 that roughly say "don't try this now, because it will be much easier in the Q2 release," but there don't seem to be any answers after the Q2 release stating what the much easier way is (at least not that I've found. It's entirely possible I'm just looking for the wrong thing).

    I've unsuccessfully tried a number of suggestions, and even tried to create a custom Pipe based on the ContentInboundPipe (yay, ILSpy!) and tried registering it like the NewsModule is registering its ContentInboundPipe (as much as possible), but without effect. My module is implementing the IPublishingEnabledModule interface, and I can see my ConfigurePublishing method being called, but nothing seems to happen. 

    I'm not even sure where to start looking at this point. Heck, I'm not even sure at this point that I'm supposed to be implementing an Inbound Pipe vs. an Outbound one. 

    Can someone at least give me a starting point? 

    Thanks, 

    - Nathan Frank
  2. Nathan
    Nathan avatar
    3 posts
    Registered:
    15 Aug 2011
    24 Jul 2012
    Link to this post
    The one caveat that I discovered this morning that I think was blocking my figuring out efforts yesterday is that new pipes do not appear to show up in existing search indexes. That may have something to do with how I'm creating things, but I'll take things working if that's the caveat. 

    The following sources finally got me where I'm currently at: 
    - Documentation on the publishing system: http://www.sitefinity.com/documentation/documentationarticles/developers-guide/deep-dive/publishing-system
    - Creating a custom pipe walkthrough: http://www.sitefinity.com/documentation/documentationarticles/developers-guide/deep-dive/publishing-system/creating-a-custom-pipe
    - Looking at the code for ContentInboundPipe using ILSpy


    So here's what ended up working for me...


    Module Changes: 

    1. Implement IPublishingEnabledModule
    public class StaffProfilesModule : ContentModuleBase, IPublishingEnabledModule
    {

    2. Add ConfigurePublishing method
    public void ConfigurePublishing()
    {
        PublishingSystemFactory.RegisterPipe(StaffProfilesPipe.PipeName, typeof(StaffProfilesPipe));
     
        var pipeSettings = StaffProfilesPipe.GetDefaultInboundPipeSettings();
        PublishingSystemFactory.RegisterPipeSettings(StaffProfilesPipe.PipeName, pipeSettings);
     
        var mappingsList = StaffProfilesPipe.GetDefaultMappings();
        PublishingSystemFactory.RegisterPipeMappings(StaffProfilesPipe.PipeName, true, mappingsList);
     
        var definitions = StaffProfilesPipe.CreateDefaultPipeDefinitions();
        PublishingSystemFactory.RegisterPipeDefinitions(StaffProfilesPipe.PipeName, definitions);
     
        var templatePipeSettings = PublishingSystemFactory.GetPipeSettings(StaffProfilesPipe.PipeName);
        PublishingSystemFactory.RegisterTemplatePipe("SearchItemTemplate", templatePipeSettings);
    }

    New Pipe class: 
    public class StaffProfilesPipe : IPipe, IPushPipe, IPullPipe, IInboundPipe
    {
        public const string PipeName = "StaffProfilesPipe";
        private SitefinityContentPipeSettings _pipeSettings;
        private IPublishingPointBusinessObject _publishingPoint;
        private IDefinitionField[] _definitionFields;
     
        private string _publishingProviderName;
     
     
        public virtual string PublishingProviderName
        {
            get
            {
                if (String.IsNullOrEmpty(_publishingProviderName))
                {
                    _publishingProviderName = Config.Get<PublishingConfig>().DefaultProvider;
                }
                return _publishingProviderName;
            }
            set { _publishingProviderName = value; }
        }
     
        public string Name
        {
            get { return PipeName; }
        }
     
        public virtual IDefinitionField[] Definition
        {
            get
            {
                if (_definitionFields == null)
                {
                    string text = null;
                    if (_pipeSettings != null)
                    {
                        text = _pipeSettings.ContentTypeName;
                    }
                    if (string.IsNullOrEmpty(text) ||
                        !PublishingSystemFactory.ContentPipeDefinitionsRegistered(Name, text))
                    {
                        _definitionFields = PublishingSystemFactory.GetPipeDefinitions(Name);
                    }
                    else
                    {
                        _definitionFields = PublishingSystemFactory.GetContentPipeDefinitions(Name, text);
                    }
                }
                return _definitionFields;
            }
        }
     
        public virtual PipeSettings PipeSettings
        {
            get { return _pipeSettings; }
            set { Initialize(value); }
        }
     
        public virtual Type PipeSettingsType
        {
            get { return typeof (SitefinityContentPipeSettings); }
        }
     
     
        public static List<Mapping> GetDefaultMappings()
        {
            var mappingsList = new List<Mapping>
                                   {
                                       PublishingSystemFactory.CreateMapping(PublishingConstants.FieldTitle,
                                                                             ConcatenationTranslator.TranslatorName,
                                                                             true,
                                                                             PublishingConstants.FieldTitle),
                                       PublishingSystemFactory.CreateMapping(PublishingConstants.FieldContent,
                                                                             ConcatenationTranslator.TranslatorName,
                                                                             false,
                                                                             PublishingConstants.FieldDescription),
                                       PublishingSystemFactory.CreateMapping(PublishingConstants.FieldItemHash,
                                                                             TransparentTranslator.TranslatorName,
                                                                             false,
                                                                             PublishingConstants.FieldItemHash),
                                       PublishingSystemFactory.CreateMapping(PublishingConstants.FieldPipeId,
                                                                             TransparentTranslator.TranslatorName,
                                                                             false,
                                                                             PublishingConstants.FieldPipeId),
                                       PublishingSystemFactory.CreateMapping(PublishingConstants.FieldLink,
                                                                             TransparentTranslator.TranslatorName,
                                                                             false,
                                                                             PublishingConstants.FieldPipeId),
                                       PublishingSystemFactory.CreateMapping(PublishingConstants.FieldContentType,
                                                                             TransparentTranslator.TranslatorName,
                                                                             false,
                                                                             PublishingConstants.FieldContentType)
                                   };
     
            return mappingsList;
        }
     
        public static IList<IDefinitionField> CreateDefaultPipeDefinitions()
        {
            return new IDefinitionField[]
                       {
                           new SimpleDefinitionField(PublishingConstants.FieldTitle,
                                                     Res.Get<PublishingMessages>().ContentTitle),
                           new SimpleDefinitionField(PublishingConstants.FieldContent,
                                                     Res.Get<PublishingMessages>().ContentContent),
                           new SimpleDefinitionField(PublishingConstants.FieldLink,
                                                     Res.Get<PublishingMessages>().ContentLink),
                           new SimpleDefinitionField(PublishingConstants.FieldItemHash,
                                                     Res.Get<PublishingMessages>().ItemHash),
                           new SimpleDefinitionField(PublishingConstants.FieldPipeId,
                                                     Res.Get<PublishingMessages>().PipeId)
                       };
        }
     
        public static SitefinityContentPipeSettings GetDefaultInboundPipeSettings()
        {
            return new SitefinityContentPipeSettings
                       {
                           ContentTypeName = typeof (StaffProfileItem).FullName,
                           IsInbound = true,
                           PipeName = PipeName,
                           IsActive = true,
                           MaxItems = 0,
                           InvocationMode = PipeInvokationMode.Push
                       };
        }
     
     
        public void Initialize(PipeSettings pipeSettings)
        {
            if (!(pipeSettings is SitefinityContentPipeSettings))
            {
                throw new ArgumentException("Expected pipe settings of type SitefinityContentPipeSettings");
            }
     
            _pipeSettings = (pipeSettings as SitefinityContentPipeSettings);
            _publishingPoint = PublishingSystemFactory.GetPublishingPoint(_pipeSettings.PublishingPoint);
            _definitionFields = null;
        }
     
        public virtual IEnumerable<WrapperObject> GetConvertedItemsForMapping(params object[] items)
        {
            throw new NotImplementedException();
        }
     
        public virtual bool CanProcessItem(object item)
        {
            var publishingSystemEventInfo = item as PublishingSystemEventInfo;
            if (publishingSystemEventInfo != null)
            {
                return (publishingSystemEventInfo).ItemAction == "SystemObjectDeleted" ||
                       CanProcessItem((publishingSystemEventInfo).Item);
            }
            if (item == null)
            {
                return false;
            }
     
            var wrapperObject = item as WrapperObject;
            if (wrapperObject != null)
            {
                return CanProcessItem((wrapperObject).WrappedObject);
            }
     
            var content = item as IContent;
            if (content != null)
            {
                Type contentType =
                    TypeResolutionService.ResolveType(((SitefinityContentPipeSettings) PipeSettings).ContentTypeName);
     
                if (contentType.IsInstanceOfType(item))
                {
                    return true;
                }
            }
            return false;
        }
     
        public virtual PipeSettings GetDefaultSettings()
        {
            return PublishingSystemFactory.CreatePipeSettings(Name, PublishingManager.GetManager(PublishingProviderName));
        }
     
        public virtual string GetPipeSettingsShortDescription(PipeSettings initSettings)
        {
            return "Case Studies Inbound Pipe";
        }
     
        public virtual void CleanUp(string transactionName)
        {
            throw new NotImplementedException();
        }
     
        public virtual void PushData(IList<PublishingSystemEventInfo> items)
        {
            var addList = new List<WrapperObject>();
            var removeList = new List<WrapperObject>();
     
            foreach (PublishingSystemEventInfo current in items)
            {
                try
                {
                    string contentTypeName = ((SitefinityContentPipeSettings) PipeSettings).ContentTypeName;
                    if (!string.IsNullOrEmpty(contentTypeName) && current.ItemType != contentTypeName)
                    {
                        Type contentType = TypeResolutionService.ResolveType(contentTypeName);
                        Type itemType = TypeResolutionService.ResolveType(current.ItemType);
                        if (!contentType.IsAssignableFrom(itemType))
                        {
                            continue;
                        }
                    }
                    var wrapperObject = new WrapperObject(PipeSettings, current.Item)
                                            {
                                                MappingSettings = PipeSettings.Mappings,
                                                Language = current.Language
                                            };
     
                    if (_pipeSettings.LanguageIds.Count <= 0 ||
                        _pipeSettings.LanguageIds.Contains(wrapperObject.Language))
                    {
                        string itemAction;
                        if ((itemAction = current.ItemAction) != null)
                        {
                            if (itemAction != "SystemObjectDeleted")
                            {
                                if (itemAction != "SystemObjectAdded")
                                {
                                    if (itemAction == "SystemObjectModified")
                                    {
                                        StaffProfileItem contentItem = GetContentItem(current);
                                        SetProperties(wrapperObject, contentItem);
                                        removeList.Add(wrapperObject);
                                        addList.Add(wrapperObject);
                                    }
                                }
                                else
                                {
                                    StaffProfileItem contentItem2 = GetContentItem(current);
                                    SetProperties(wrapperObject, contentItem2);
                                    addList.Add(wrapperObject);
                                }
                            }
                            else
                            {
                                removeList.Add(wrapperObject);
                            }
                        }
                    }
                }
                catch (Exception ex)
                {
                    this.HandleError("Error when push data for item action {0} for item {1}."
                                         .Arrange(new[] {current.ItemAction, current.Item}), ex);
                }
            }
            if (removeList.Count > 0)
            {
                _publishingPoint.RemoveItems(removeList);
            }
            if (addList.Count > 0)
            {
                _publishingPoint.AddItems(addList);
            }
        }
     
        public virtual IList<WrapperObject> GetData()
        {
            var wrapperObjects = LoadWrapperObjectItems();
            return wrapperObjects;
        }
     
        public virtual void ToPublishingPoint()
        {
            var wrapperObjects = LoadWrapperObjectItems();
     
            var items = wrapperObjects
                .Select(item => new PublishingSystemEventInfo
                                    {
                                        Item = item,
                                        ItemAction = RelatedActionsConstants.SystemObjectModified,
                                        Language = item.Language
                                    })
                .ToList();
     
            PushData(items);
        }
     
        protected virtual StaffProfileItem GetContentItem(WrapperObject wrappedObject)
        {
            if (wrappedObject == null)
            {
                return null;
            }
            if (wrappedObject.WrappedObject == null)
            {
                return null;
            }
            var contentItem = wrappedObject.WrappedObject as StaffProfileItem;
            if (contentItem != null)
            {
                return contentItem;
            }
            return GetContentItem((WrapperObject) wrappedObject.WrappedObject);
        }
     
        protected virtual StaffProfileItem GetContentItem(PublishingSystemEventInfo item)
        {
            var wrappedObject = item.Item as WrapperObject;
            if (wrappedObject != null)
            {
                return GetContentItem(wrappedObject);
            }
            return (StaffProfileItem) item.Item;
        }
     
     
        protected virtual IList<WrapperObject> LoadWrapperObjectItems()
        {
            IEnumerable<StaffProfileItem> items = GetContentItems();
            return items.Select(ConvertToWraperObject).ToList();
        }
     
        protected IList<StaffProfileItem> GetContentItems()
        {
            if (_pipeSettings == null)
            {
                throw new InvalidOperationException("Pipe settings are required!");
            }
     
            StaffProfilesManager manager = StaffProfilesManager.GetManager(_pipeSettings.ProviderName);
     
            var query = manager.GetStaffProfiles();
     
            if (_pipeSettings.MaxItems > 0)
            {
                query = query.Take(_pipeSettings.MaxItems);
            }
     
            var list = query.OrderByDescending(x => x.PublicationDate).ToList();
     
            return list;
        }
     
        protected virtual WrapperObject ConvertToWraperObject(StaffProfileItem item)
        {
            var obj = new WrapperObject(PipeSettings, item)
                          {
                              MappingSettings = PipeSettings.Mappings,
                              Language = PipeSettings.LanguageIds.FirstOrDefault()
                          };
     
            SetProperties(obj, item);
     
            return obj;
        }
     
        protected virtual void SetProperties(WrapperObject wrapperObject, StaffProfileItem item)
        {
            wrapperObject.AddProperty(PublishingConstants.FieldTitle, item.Title);
            wrapperObject.AddProperty(PublishingConstants.FieldContent, item.Content);
            wrapperObject.AddProperty(PublishingConstants.FieldItemHash, GenerateItemHash(item));
            wrapperObject.AddProperty(PublishingConstants.FieldIdentifier, item.Id);
            wrapperObject.AddProperty(PublishingConstants.FieldPipeId, _pipeSettings.Id);
            wrapperObject.AddProperty(PublishingConstants.FieldLink, GetUrl(item));
            wrapperObject.AddProperty(PublishingConstants.FieldContentType, _pipeSettings.ContentTypeName);
        }
     
        protected virtual string GenerateItemHash(StaffProfileItem item)
        {
            var builder = new StringBuilder();
     
            builder.Append(item.Id);
            builder.Append("|");
            builder.Append(item.Title);
     
            var hasher = new SHA1CryptoServiceProvider();
     
            byte[] originalBytes = Encoding.UTF8.GetBytes(builder.ToString());
            byte[] encodedBytes = hasher.ComputeHash(originalBytes);
            return Convert.ToBase64String(encodedBytes);
        }
     
        protected string GetUrl(StaffProfileItem item)
        {
            var url = item.Urls.FirstOrDefault();
     
            if (url != null)
            {
                return url.Url;
            }
     
            return null;
        }
    }

    This is a merging of the XmlInboundPipe from the walkthrough and the code from ContentInboundPipe. There were some things XmlInboundPipe was doing (i.e., populating ItemHash) that ContentInboundPipe wasn't, so I left those in. I also changed how the Link property was getting populated. I'm not sure how it happened in ContentInboundPipe (it seemed to be returning null if there was a default page configured in the search index). 

    The methods that contain customization based on the item type are: 
    - GetDefaultMappings
    - CreateDefaultPipeDefinitions
    - SetProperties

    These basically define and then populate the item-specific fields. 

    Re-install the module, create a new search index, and the new search scope appears. What I have is not showing a friendly name for the search scope and there's a lot of code here that I don't understand (e.g, how is ItemHash used? is Identifier needed in this case? etc.), but I'll take it if it works. 
  3. Nathan
    Nathan avatar
    3 posts
    Registered:
    15 Aug 2011
    01 Aug 2012
    Link to this post
    Following up with the results of a support ticket. 


    I had used that code at one point, but it was before I realized new search scopes don't show up on existing search indexes. 

    Note that I put the code in my module's implementation of the IPublishingEnabledModule.ConfigurePublishing method. 
  4. Jenna Brenn
    Jenna Brenn avatar
    5 posts
    Registered:
    07 May 2010
    05 Sep 2012
    Link to this post
    Your code worked great - THANK YOU!!  I had been trying to make the code in the documentation work for a very long time.  You have made my day, if not my week.  :)
  5. Michael
    Michael avatar
    67 posts
    Registered:
    16 Mar 2012
    11 Oct 2012 in reply to Jenna Brenn
    Link to this post
    You can also do your registration code for the publishing system stuff in the Initialize (not Install) method of the module I've found... but I like this method better.

    Did anyone figure out how to get new search scopes to show up in existing indexes though? It's annoying having to delete and completely recreate my index every time a new type is added, etc.
  6. Pedro
    Pedro avatar
    5 posts
    Registered:
    03 Jan 2013
    03 Jan 2013 in reply to Nathan
    Link to this post

    Dear Nathan,

    First, I would like to thank you for having the time to post this solution online, it is being very helpful.

    Second, I have some doubts about your code that I would like to ask if you could help me with. Here they go:

    Imagine that your module has 3 fields: a title that is metadata, a summary that is a localized string, and contacts, that is also a localized string.

    How would this fields be used in the GetdefaultMappings, CreateDeafultPipeDefinitions, SetProperties and how would I go about to use them there

    And what will the "Additional fields for indexing" in the Search & indexes in the UI be for these fields the module has? 

    Thank you very much for your time and patience.

    Regards,

    Pedro


  7. Pedro
    Pedro avatar
    5 posts
    Registered:
    03 Jan 2013
    04 Jan 2013
    Link to this post
    I tried to search only in Title, using "Title" as the "Additional fields for indexing" but for some reason the search returned 0 results although I have items for my custom module created with the title I searched for. Anybody has any idea why this doesn't work?
    Thank you very much
7 posts, 0 answered