1-888-365-2779
+1-888-365-2779
Try Now
More in this section

Forums / Developing with Sitefinity / Upload very large files into library with api

Upload very large files into library with api

10 posts, 0 answered
  1. Christian
    Christian avatar
    20 posts
    Registered:
    12 Mar 2008
    01 Oct 2009
    Link to this post
    Hi,

    we are trying to upload a 1GB file via Manager.UploadFile wich takes a byte array as parameter.

    The phsysical file comes from filesystem.

    Could you give us any tip how to read the large file and overgive it as a parameter? With small files it works nice, but not with large ones.

    In the moment we fill a memorystream via a filestream with a given buffer size. Finally we fill a byte array via memorystream.toarray() and there we get outofmemory. we have 4 gb of ram (2GB are free / 32bit system).

    While deleting entries from the library that are large, we get outofmemory, too. It seems maybe there is a problem with nolis or mapper and handling things ?! It seems memory get not released after doing any sql query with large byte arrays...

    Maybe one chance is to save the file other ways / database query (insert) manualy. If this is true, could you give as a tip thanks where to insert all sql stuff / relations... But maybe you have an idea hoe to handle this with sityfinity api core functions.
    chris!
  2. Ivan Dimitrov
    Ivan Dimitrov avatar
    16072 posts
    Registered:
    16 Jun 2017
    02 Oct 2009
    Link to this post
    Hi Christian,

    It is not possible to save such large file into the database as a binary content. The problem is that we create staged content before the item is uploaded which results in the server performance which causes a memory leak. We use the following method UploadFile where we pass the file content as a buffer. Then we use FileStream and BinaryWriter classes to write the data. You could try increasing the Minimum and Maximum of SQL server memory and Index creation memory ( 0 - means unlimited), configure the timeout of the server ( 0 means unlimited) and use at least 1024 KB for Minimum memory per query. Note that this chamges could be done only over dedicated SQL server. Another option is storing the large files in your file system.

    We are working on fix that will be available from Sitefinity 4.0.

    Regards,
    Ivan Dimitrov
    the Telerik team

    Instantly find answers to your questions on the new Telerik Support Portal.
    Watch a video on how to optimize your support resource searches and check out more tips on the blogs.
  3. Christian
    Christian avatar
    20 posts
    Registered:
    12 Mar 2008
    02 Oct 2009
    Link to this post
    Do you provide a provider / setting for storing files in the filesystem instead of database by default?
  4. Ivan Dimitrov
    Ivan Dimitrov avatar
    16072 posts
    Registered:
    16 Jun 2017
    02 Oct 2009
    Link to this post
    Hello Christian,

    By default there is not file system provider. You can take a look at the following link - physical upload of files. Note that thumbnails are stored in local database no matter what kind of provider you are going to use. In an extended provider, the idea is setting the content of an IContent object to null, but there is no way for doing that with thumbnails with 3x versions.

    Best wishes,
    Ivan Dimitrov
    the Telerik team

    Instantly find answers to your questions on the new Telerik Support Portal.
    Watch a video on how to optimize your support resource searches and check out more tips on the blogs.
  5. Christian
    Christian avatar
    20 posts
    Registered:
    12 Mar 2008
    02 Oct 2009
    Link to this post
    hi,

    I created a custom provider.

    One problem: It semms if a list of items inside a library gets opened it calls

    IContent

     

    GetContent(Guid id)

     

     

     

     

     


    in the provider. That doesn´t make sense to me, because there my provider will fill content with binary data from IO. In the list I only need some metadata of the items in the library, but not the content field. Loading of the list is extremly slow because it loads alls content from all library items ?!

    Could you tell me what method is the correct one to override for showing the list without loading the full object and what method to override to load a single item with full content?
  6. Ivan Dimitrov
    Ivan Dimitrov avatar
    16072 posts
    Registered:
    16 Jun 2017
    02 Oct 2009
    Link to this post
    Hello Christian,

    The methods that our API Expose for getting single content is GetContent( Guid ID) and GetContent( Guid ID, itemInfo ). For getting the List you can use the other 19 overloads of GetContent method.

    "Note that thumbnails are stored in local database no matter what kind of provider you are going to use" Even you avoid this you will not be able to make the process of getting the images faster because you need to make a call to another folder or domain.

    I hope this helps.

    Sincerely yours,
    Ivan Dimitrov
    the Telerik team

    Instantly find answers to your questions on the new Telerik Support Portal.
    Watch a video on how to optimize your support resource searches and check out more tips on the blogs.
  7. Christian
    Christian avatar
    20 posts
    Registered:
    12 Mar 2008
    02 Oct 2009
    Link to this post
    Hi, i am not talking about images. We are saving any content there, but its not important. In fact the list inside a library seems not to load on demand the data, the list itself loads the full content of every item inside the list. I need to know which method gets called by the list by your standard library list to show up the item list inside a library because there is no need to load the content field inside the list ?!
  8. Ivan Dimitrov
    Ivan Dimitrov avatar
    16072 posts
    Registered:
    16 Jun 2017
    03 Oct 2009
    Link to this post
    Hi Christian,

    For list mode we use WebServices and client side binding of RadGrid control calling GetContentItems method.

    Best wishes,
    Ivan Dimitrov
    the Telerik team

    Instantly find answers to your questions on the new Telerik Support Portal.
    Watch a video on how to optimize your support resource searches and check out more tips on the blogs.
  9. Christian
    Christian avatar
    20 posts
    Registered:
    12 Mar 2008
    03 Oct 2009
    Link to this post
    thanks, but how can we load the liste without fetching the content field? Why do you load the full content for a list (master / detail) ?
  10. Ivan Dimitrov
    Ivan Dimitrov avatar
    16072 posts
    Registered:
    16 Jun 2017
    04 Oct 2009
    Link to this post
    Hi Christian,

    The following method GetContent does not support partial loading of data, it calls single or filtered list of items.

    Regards,
    Ivan Dimitrov
    the Telerik team

    Instantly find answers to your questions on the new Telerik Support Portal.
    Watch a video on how to optimize your support resource searches and check out more tips on the blogs.
Register for webinar
10 posts, 0 answered