+1-888-365-2779
Try Now
More in this section

Forums / Deployment / Migrating data between environments

Migrating data between environments

13 posts, 1 answered
  1. Doug Shontz
    Doug Shontz avatar
    6 posts
    Registered:
    10 Feb 2010
    05 Jul 2010
    Link to this post
    I am working on my first (of hopefully many) Sitefinity sites and we are getting close to having the site go live.  I fear I may have backed myself unknowingly into a corner that seems difficult to get out of (at least for a newbie).  I have been developing on my laptop localhost, and I also have a "live" site that will ultimately be production.  One of the people at my client's site has been working to set up their user base, which now totals more than 100 users.  That has been happening on the "live" database.  Locally, I have been working through the kinks of dealing with their images and documents using the API.  Once I got that working the way they wanted, I tested and uploaded over 50 libraries.

    Now, it seems I'm faced with a "merging" issue.  I have the image and document libraries on my local SQL Server, and the user base is on the remote "live" database.  Having been a SQL Server DBA for 10+ years now, I made the assumption that all is data (like most CMS systems) and all can be migrated together.  In looking through the forums, however, this does not seem to be a menial task.  I don't want to end up with two databases, but I also don't want to lose the extensive amount of work either side has done.  I couldn't find any schema documentation (the first thing a DBA goes for), so I struck out there.  The export doesn't seem to do images and documents, and it threw a System.OutOfMemoryException when I tried to run it, anyway.

    I've tried to find ways to do this, and a number of the forums simply say to back up the db and restore it into production.  Others say to use Red Gate (which I have), but I don't want to overwrite everything.

    Have I accidentally created a monster or can some generous mind lend a hand?  It is really this cumbersome to move CMS-level data around to different environments?

    Regards,
    Doug Shontz, Jr.
  2. Ivan Dimitrov
    Ivan Dimitrov avatar
    16072 posts
    Registered:
    19 Sep 2016
    05 Jul 2010
    Link to this post
    Hi Doug Shontz,

    We use data layer ( in 3.x editions it is Nolics, but since 4.0 CTP we use our ORM - Open Access)There are two options that will ensure you will not loose any data

    1. Using database replication tool

    2. Using the API

    You can add an additional provider that points to the online website. You can use the additional provider to show the data on your local website. Here is a useful blog post that shows how to add additional providers ( it explains how to do it for a single database, but the case is the same if you use separate database server).Note that the provider names should be different.

    - in the online  website add a new connectionString that points to the local sever database

    Copy Code
    <dataAccess defaultConnection="DefaultConnection">
            <connections>
      
        <add name="LocalServerConnection" driver="Telerik.DataAccess.Providers.ExtendedSqlServer2005Provider, Telerik.DataAccess" connectionStringName="LocalServerConnectionString" />
        <add name="DefaultConnection" driver="Telerik.DataAccess.Providers.ExtendedSqlServer2005Provider, Telerik.DataAccess" connectionStringName="Sitefinity" />
                <add name="GenericContentConnection" driver="Telerik.Cms.Engine.Data.Providers.GCSql2005Provider, Telerik.Cms.Engine.Data" connectionStringName="Sitefinity" />
            </connections>
        </dataAccess>


    - in online website create an additional provider for Libraries module

    <cmsEngine defaultProvider="Generic_Content">
        <providers>                       
            <clear />
    <add name="Libraries" urlRewriteFormat="~/{Provider}/{LibraryName}/[Name].sflb.ashx" urlDateTimeFormat="yy-MM-dd" urlWhitespaceChar="_" visible="False" defaultMetaField="Name" applicationName="/Libraries" allowVersioning="False" allowLocalization="False" localizationProviderName="" allowWorkflow="False" securityProviderName="" versioningProviderName="" connectionStringName="GenericContentConnection" type="Telerik.Libraries.Data.DefaultProvider, Telerik.Libraries.Data" tagEditorTemplate="~/Sitefinity/Admin/ControlTemplates/Libraries/BatchTagsEditor.ascx"/>
        <add name="LibrariesLocal" urlRewriteFormat="~/{Provider}/{LibraryName}/[Name].sflb.ashx" urlDateTimeFormat="yy-MM-dd" urlWhitespaceChar="_" visible="False" defaultMetaField="Name" applicationName="/LibrariesLocal" allowVersioning="False" allowLocalization="False" localizationProviderName="" allowWorkflow="False" securityProviderName="" versioningProviderName="" connectionStringName="LocalServerConnection" type="Telerik.Libraries.Data.DefaultProvider, Telerik.Libraries.Data" tagEditorTemplate="~/Sitefinity/Admin/ControlTemplates/Libraries/BatchTagsEditor.ascx"/>



    Then implement a logic that will get the items from the LibrariesLocal provider and create new items underLibraries provider. The problem here is that you should decide how to identify the items so that you will not create duplicated IContent items. You cannot use the ID of the items, because they are different in Test and Main databases. In this case you could use the title.

    below is a sample code

    Copy Code
    ContentManager managerTest = new ContentManager("LibrariesLocal");
    ContentManager managerMain = new ContentManager("Libraries");
    IList listofTestItems = managerTest.GetContent();
    for (int i = 0; i< listofTestItems .Count; i++)
       {
        // here I am not making the comparison
        IContent copyContent = (IContent)listofTestItems[i];
        managerMain .SaveContent(copyContent);
       }

    You could just copy the content items from the LibrariesLocal " to Libraries provider - creating all metakeys and setting the content by using Generic Content API

    The way requires custom coding and if you want to cover all cms parts you have to create "wrappers" for all modules, pages, users, etc.


    The last option that I could come up right now is using web services to get the data from your local website and publish it on the online website or feed reader.

    Best wishes,
    Ivan Dimitrov
    the Telerik team
    Do you want to have your say when we set our development plans? Do you want to know when a feature you care about is added or when a bug fixed? Explore the Telerik Public Issue Tracking system and vote to affect the priority of the items
    Answered
  3. Doug Shontz
    Doug Shontz avatar
    6 posts
    Registered:
    10 Feb 2010
    06 Jul 2010
    Link to this post
    Wow...I DID accidentally create a monster.  Or at least a bigger issue than I thought.  First, thank you for the very detailed reply.  I will gladly take that as the answer.  Second, let me voice a request/concern.  I was really hoping to find some documentation on the schema of the database.  I'm coming from being a database guy, so I am much more proficient at SQL than C# (at least for now).  On any enterprise level system I work on, I spend much more time in Management Studio than Visual Studio.  Using a database replication tool (like the Red Gate tools) is only a good option if you know what you are copying over.  I'm using Sitefinity both for my "side" projects and as our content management system at my primary job.  I can think of a ton of scenarios where migrating data back and forth between iterations of systems (DEV, QA, UAT, etc) might be necessary.  I realize the API is very strong, but it worries me that the database is a bit of a black box.  I know that the API is standardized and that the schema could change from version to version, but this is literally the only system I'm dealing with at my job where I can't seemingly push the data back and forth using data tools.  We have e-commerce systems, this CMS, and a lot of customized applications and solutions, and all of them have schema documentation which allow me to do something with the database other than "replicate" them.

    Instead of me just complaining (like the first paragraph), I would like to know if there is a way to get involved with helping with the database documentation?  I'm rather vested in Sitefinity on all fronts at this point.  I'm literally going to start with two clean dbs, start adding items, and doing data diffs to see what was added so I can make my own "map".  Is there any plans to do database documentation and can I help in any way?

    Regards,
    Doug
  4. Ivan Dimitrov
    Ivan Dimitrov avatar
    16072 posts
    Registered:
    19 Sep 2016
    06 Jul 2010
    Link to this post
    Hi Doug Shontz,

    The relations in Sitefinity database is quite complex and they are done through data layer - Nolics. Nolics has been discontinued and we will not use it anymore for further implementations.  I mentioned that we will use Open Access which is our product and it is  documented here.
    Anyway, thank you for your interest to help with this. If you want I could send you the dbclasses which contains some general data about the database tables - primary key, table type etc.

    Best wishes,
    Ivan Dimitrov
    the Telerik team
    Do you want to have your say when we set our development plans? Do you want to know when a feature you care about is added or when a bug fixed? Explore the Telerik Public Issue Tracking system and vote to affect the priority of the items
  5. Doug Shontz
    Doug Shontz avatar
    6 posts
    Registered:
    10 Feb 2010
    06 Jul 2010
    Link to this post
    Ivan,

    I would definitely appreciate having any information about the db possible.  Database complexity doesn't scare me.  Thirteen years of being a DBA and working on multi-terabyte databases and data warehouses in Oracle and SQL Server have made me immune to the fear reverse-engineering of a crazy schema.  If you could send me those classes, they might be able to tell me something other than what the database already has.  I will definitely learn the API layer and make that a priority for me, but there is a greater comfort for me knowing what data is being stored where.

    I appreciate you quick and thorough responses!

    Regards,
    Doug
  6. Ivan Dimitrov
    Ivan Dimitrov avatar
    16072 posts
    Registered:
    19 Sep 2016
    08 Jul 2010
    Link to this post
    Hi Doug Shontz,

    I attached the dbclasses. I am not sure how helpful they will be for you, but there is no other resource that I can send you now.

    Sincerely yours,
    Ivan Dimitrov
    the Telerik team
    Do you want to have your say when we set our development plans? Do you want to know when a feature you care about is added or when a bug fixed? Explore the Telerik Public Issue Tracking system and vote to affect the priority of the items
  7. Adriano
    Adriano avatar
    4 posts
    Registered:
    23 Dec 2010
    01 Feb 2011
    Link to this post
    Hi,

    I'm working with Sitefinity 4.0.
    Is there any improvement to import and (or) export content from one environment to the other?
  8. Ivan Dimitrov
    Ivan Dimitrov avatar
    16072 posts
    Registered:
    19 Sep 2016
    02 Feb 2011
    Link to this post
    Hi Adriano,

    In Sitefinity 4.0 you can use Sitefinity WCF RESTful Services. To migrate data from Sitefinity 3.x to Sitefinity 4.0 you can use the migration tool.

    Kind regards,
    Ivan Dimitrov
    the Telerik team
    Do you want to have your say when we set our development plans? Do you want to know when a feature you care about is added or when a bug fixed? Explore the Telerik Public Issue Tracking system and vote to affect the priority of the items
  9. Bryan
    Bryan avatar
    6 posts
    Registered:
    04 Nov 2010
    18 Feb 2011
    Link to this post
    Is there a way to import a 3.7 site from one environment to another, e.g. test to staging or staging to live?
  10. Ivan Dimitrov
    Ivan Dimitrov avatar
    16072 posts
    Registered:
    19 Sep 2016
    21 Feb 2011
    Link to this post
    Hello Bryan,

    Migrating website from one environment to another should be done with third party database and file replication tools. Another option is using custom tool that works with our API, but this will be more complicated ( we have migration too for 3.7 to 4.0 which shows similar scenario).

    We are going to provider migration option between staging and production in one of the minor versions of Sitefinity 4.0.

    Best wishes,
    Ivan Dimitrov
    the Telerik team
  11. Omar
    Omar avatar
    11 posts
    Registered:
    23 Mar 2011
    31 Mar 2011
    Link to this post
    Hi, 
          I'm facing the same trouble that Doug explain in the first post, we are already working with the last version of the product, i have a development team working on dynamic content in one environment (DEV) and a designers team working on static content in another environment (PRE), and i'm creating the production environment with client access, where i'm  defining the security schemas (users configuration), the next step is move the dynamic content to PRE to join it with static content, and both contents move it to PROD environment (Every environment has their own DB)
    The separation in environments is because is already working in this project at least 25 persons (Developers, Content Creators & QA), and we need to separate the work in stages.
         So my question is, Doug had finally been able to migrate the data from one DB to another?, was included in this last version any deployment process that allow us to support this scenario? exist any documentation beside the dbclasses to evaluate the availability of data migration?

    regards,

    Omar
  12. Doug Shontz
    Doug Shontz avatar
    6 posts
    Registered:
    10 Feb 2010
    31 Mar 2011
    Link to this post
    Omar,

    From my end, the answer is no.  Unfortunately, I was never allowed the time or resources to figure this out correctly.  When I detailed to my management the issues I was facing and after much wailing and gnashing of teeth, we ultimately had to abandon Sitefinity for this very reason.  We just couldn't afford not to be able to effectively support multiple instances in a "standard" way.  The API is good, but it just doesn't handle moving mountains of data.  If you are willing to work with it, I'm sure it can be done, but that wasn't a risk we were willing to take.

    I still love and support Sitefinity from a side-project standpoint where the requirements are less harsh and you do not need a true SDLC, but I no longer use Sitefinity at my primary location.  I just keep in mind now to always be working against one environment (which side projects allow).

    For the record of this, I have not worked with 4.0 yet.  I am only supporting two clients right now with Sitefinity, and they are fine on 3.7 and have a requirement of needing wikis, so I won't be looking at 4.0 until it has all the same functionality as 3.7.  I also may not be able to get the client to move up due to the price increase, but we'll see.

    Sorry I could not be of more help!
    Doug
  13. Omar
    Omar avatar
    11 posts
    Registered:
    23 Mar 2011
    31 Mar 2011
    Link to this post
    Thanks Doug for your answer, 

    I think that the support of multiple environments is essential in the development/maintenance of large sites or for multiple clients (that represent a large development team & SDLC).

    An iterative development and the management multiple versions of controls or templates before be implemented in production environment is a very common scenario for us.

    Somebody know if this scenario will be supported in the future?, 
    someone already solved this issue in another way?

    I'll appreciate any suggestion.

    regards,

    Omar
Register for webinar
13 posts, 1 answered