Sitecore media streaming issue after publishing!

Recently we came across a strange behaviour of Sitecore Media streaming in MediaCache that "Overwritten media files are not getting reflected after publish". Just to have clear idea, this is not a browser caching issue mentioned in Sitecore KB article.

What's the issue?

We created a media item on CM and published it, and was visible on live site. Now we overwritten a new media file to the same media item and published it again. (Either using Detach/Attach from Content Editor or using Overwrite existing media from Page Editor.) Surprisingly, we were still getting older media file! We published again, again and again, but newly published media not getting updated.

And yet, this is a very random issue and occurs very rarely.

How we tried to troubleshoot?

1. We have multiple servers in cluster with 2 target databases. We found that few servers of both target databases are serving older media file and rest of them serving latest one.
2. Then we thought there might be some Item Path Cache or Item Cache clearing issue (Which happens on Sitecore some times). So, we cleared both these caches for this media item using Sitecore API. But result was same.
3. Then we cleared whole Item Cache and Data Cache using Sitecore API. The result was same.
4. Then we cleared All Sitecore Caches using http:///sitecore/admin/cache.aspx page. The result was same.
5. Final option we had to clear all media cache physical files (Website\App_Data\MediaCache) so that Sitecore will create new media cache from database and can serve latest one. Even after deleting all files and folder from it, new files got generated but still were older one.

So, no solution at all after applying these many tricks!

How we fixed?

We had no other option but recycling the Application Pool. Finally, the master key worked for us. :)

What we concluded and what's the solution?

The only conclusion we had that Sitecore is storing media files somewhere in Server memory as well. Strange, right?

We raised to Sitecore Support for further investigation. Many thanks to Andrey Krupskiy from support who investigated and confirmed that Sitecore is really storing media files in RAM as well that might have caused this and provided below solution.

There is an internal media cache in RAM. This cache is used when media is not yet saved to the filesystem. Even, if you check code of Sitecore.Resources.Media.MediaCache class, in Reflector, it says the same. Sitecore serves media file RAM before its actual file cache gets generated on disk (might be to serve media faster), which is default behaviour of Sitecore. We can disable this behaviour by changing below configuration in Web.Config.

<setting name="Media.StreamPartiallyCachedFiles" value="false" />

We disabled the Media.StreamPartiallyCachedFiles setting as shown above on CM and CD servers.

Now it has more than a month now, we haven't faced the issue again.

Dealing with duplicate item names in Sitecore publishing

A very common challenge in Sitecore is to deal with duplicate item name inside a single parent. Many times our content authors complaint that they are not able to see latest content for an item even after publishing it many times. Let's see why users were getting older content.

What they actually do?

1. We have one item say Test under Home and published it.

On CM Server

On CD Server

2. Now, deleted the Test item, and created another item with Test name itself and published
(Or created new item named Test and deleted older one.)

On CM Server

On CD Server

Uff, how can we get latest content, when live database contains two items with same name under a single parent? Now, when you request for http://sitecoretactics/test/, it will surely going to render any random item. But due to increasing number of such cases, we thought to have a permanent solution to have unique item

Solution to have unique item on live sites

We will create a custom processor in <publishItem> pipeline that will take care for this. When the item Test gets published, it will check any other item with same name exist on target database or not. If exist with different ItemId, then it will delete that old item.

Create a PublishItemProcessor class as below.

namespace SitecoreTactics.Publishing
{
    public class RemoveDuplicateItems : PublishItemProcessor
    {
        Item sourceItem = context.PublishHelper.GetSourceItem(context.ItemId);
        if (sourceItem != null)
        {
            Item targetItem = context.PublishOptions.TargetDatabase.GetItem(sourceItem.Paths.Path);

            if (targetItem != null && targetItem.ID != sourceItem.ID)
            {
                context.PublishHelper.DeleteTargetItem(targetItem.ID);
            }
        }
    }
}

Using a patch config, add this processor inside pipeline as below. You may need to change processor's order if you have any customizations done in the pipeline.

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <pipelines>
      <publishItem>
        <processor type="SitecoreTactics.Publishing.RemoveDuplicateItems, SitecoreTactics" />
      </publishItem>
    </pipelines>
  </sitecore>
</configuration>

Now repeat the same steps above, you will find older item gets deleted!