Sitecore media streaming issue after publishing!

Recently we came across a strange behaviour of Sitecore Media streaming in MediaCache that "Overwritten media files are not getting reflected after publish". Just to have clear idea, this is not a browser caching issue mentioned in Sitecore KB article.

What's the issue?

We created a media item on CM and published it, and was visible on live site. Now we overwritten a new media file to the same media item and published it again. (Either using Detach/Attach from Content Editor or using Overwrite existing media from Page Editor.) Surprisingly, we were still getting older media file! We published again, again and again, but newly published media not getting updated.

And yet, this is a very random issue and occurs very rarely.

How we tried to troubleshoot?

1. We have multiple servers in cluster with 2 target databases. We found that few servers of both target databases are serving older media file and rest of them serving latest one.
2. Then we thought there might be some Item Path Cache or Item Cache clearing issue (Which happens on Sitecore some times). So, we cleared both these caches for this media item using Sitecore API. But result was same.
3. Then we cleared whole Item Cache and Data Cache using Sitecore API. The result was same.
4. Then we cleared All Sitecore Caches using http:///sitecore/admin/cache.aspx page. The result was same.
5. Final option we had to clear all media cache physical files (Website\App_Data\MediaCache) so that Sitecore will create new media cache from database and can serve latest one. Even after deleting all files and folder from it, new files got generated but still were older one.

So, no solution at all after applying these many tricks!

How we fixed?

We had no other option but recycling the Application Pool. Finally, the master key worked for us. :)

What we concluded and what's the solution?

The only conclusion we had that Sitecore is storing media files somewhere in Server memory as well. Strange, right?

We raised to Sitecore Support for further investigation. Many thanks to Andrey Krupskiy from support who investigated and confirmed that Sitecore is really storing media files in RAM as well that might have caused this and provided below solution.

There is an internal media cache in RAM. This cache is used when media is not yet saved to the filesystem. Even, if you check code of Sitecore.Resources.Media.MediaCache class, in Reflector, it says the same. Sitecore serves media file RAM before its actual file cache gets generated on disk (might be to serve media faster), which is default behaviour of Sitecore. We can disable this behaviour by changing below configuration in Web.Config.

<setting name="Media.StreamPartiallyCachedFiles" value="false" />

We disabled the Media.StreamPartiallyCachedFiles setting as shown above on CM and CD servers.

Now it has more than a month now, we haven't faced the issue again.

Dot or Space in URL gives 404 Error - Fixed

Our application security certification program raised a security issue on our Sitecore Staging and Live environment. "Any URL where folder or file name ends with Space or Dot gives 404 - The resource cannot be found error."

For example, accessing URLs like
- http://sitecore72/sitecore%20/login
or
- http://sitecore72/sitecore./login

Both gives below error. Same, if we do same kind of request on any live site, it gives same error like for URL: http://sitecore/CMS /Sitecore/

Tricks we already applied, what failed:

• Tried to catch this error from Application_Error event from Global.asax, but no luck found. 
• We also tried to handle it from web.config using CustomErrors, but again no luck.

Solution we found at last:

After checking many ASP.NET sites, including Microsoft website, this issue was found on majority of sites. For example, http://www.microsoft.com/security /default.aspx, it gives same error. So finally we concluded that this bug is not from Sitecore side, but it's something what .NET is playing with.

After reading more on net, found below setting. The value of relaxedUrlToFileSystemMapping attribute should be true to solve this issue, by default its value is false in ASP.NET and Sitecore:

    <httpruntime relaxedurltofilesystemmapping="true" />

Now, after applying this fix, the above URL on Staging environment sends user to NotFound error page and on live environment, it opens a valid page by ignoring Dot or Space.

What relaxedUrlToFileSystemMapping attribute does?

It indicates whether the URL in an HTTP request is required to be a valid Windows file path. It determines how the URL in an incoming HTTP request will be validated.

If this property is false, the URL is validated by using the same rules that determine whether a Windows file system path is valid.
If it is true, it will not validate any folder or file name.


Note: Later on we found that Sitecore has solved this bug from Sitecore 7.2 Update-2 by changing same attribute, they have not mentioned this bugfix in their release notes.

Solved Sitecore PDF Loading Issue

Are getting problem in viewing PDF files in your newly upgraded Sitecore environment? We came across the same bug while upgrading Sitecore to version 7.2 that in some browsers while opening PDF files, they get stuck. Surprisingly after that, pressing Ctrl+F5 was allowing to view PDF file. See below screenshot when the downloading stuck:

Workaround from Sitecore

In Sitecore 7.2, they gave workaround to solve the issue by forcing download of PDFs by doing below setting:

        <mediaType name="PDF file" extensions="pdf">
          <mimeType>application/pdf</mimeType>
          <forceDownload>true</forceDownload>
        </mediaType>

But, this is not the solution for our problem. We can't say our clients to download and view the PDF, it should be viewed in browser itself. We applied many tricks, and finally we solved it.

So, what was the issue?

After reverse engineering MediaRequestHandler, we came to know that Sitecore is downloading PDF content partially in range using HTTP 206 Status Code, which might be creating problems in different browser. In Sitecore versions earlier than before 6.5, Sitecore was downloading PDF or any other media in one go, which never failed.

So, one thing was sure that this bug was just because of Range Retrieval mechanism provided by Sitecore or the plug-in our browser does not support it. See below screenshot which shows how Sitecore downloads PDF content in range.

 

Final Solution

We just tried disabling the Range Retrieval Mechanism as below and found the bug resolved.

    <setting name="Media.EnableRangeRetrievalRequest" value="false" />

Many thanks to our colleague Sandeep Gupta who helped in the investigation.

Enjoy accurate media download now!!

Other Sitecore 7.2 Bugs Solved!

• Nasty Bug Found in Sitecore 7.2 Publish Related Items
• Sitecore Media Attach Detach Getting Slower

Bug Fix in Sitecore 7.2 Publish Related Items

While testing our newly upgraded Sitecore 7.2 solution, we faced a nasty bug in newly released feature of Publish Related Items

When we ticked Publish Related Items checkbox to publish all references, it publishes all the reference items three times. You are not believing, right? Even we too when our QA raised this to us. Such a nasty bug this is!!

If you have enabled traceToLog like below for UpdateStatistics processor,

    <processor type="Sitecore.Publishing.Pipelines.PublishItem.UpdateStatistics, Sitecore.Kernel" runIfAborted="true">
        <traceToLog>true</traceToLog>
    </processor>

you will find below logs which tells the story of the bug that item named Images gets published three times.

17268 16:13:23 INFO  ##Publish Item: Name=Images, Uri=sitecore://master/{15451229-7534-44EF-815D-D93D6170BFCB}?lang=en&ver=1, Operation=Updated, ChildAction=Allow, Explanation=Version 'en_1' was published.
17268 16:13:23 INFO  ##Publish Item: Name=Images, Uri=sitecore://master/{15451229-7534-44EF-815D-D93D6170BFCB}?lang=en&ver=1, Operation=Updated, ChildAction=Allow, Explanation=Version 'en_1' was published.
17268 16:13:23 INFO  ##Publish Item: Name=Images, Uri=sitecore://master/{15451229-7534-44EF-815D-D93D6170BFCB}?lang=en&ver=1, Operation=Updated, ChildAction=Allow, Explanation=Version 'en_1' was published.

How we solved this bug:

While investigating and spending few hours we found that Sitecore is not able to remove duplicate items which are added as reference items, which we can solve by overriding RemoveDuplicateReferrers method of ProcessQueue processor as below:

Step 1: Replace below line from web.config

    <processor type="Sitecore.Publishing.Pipelines.Publish.ProcessQueue, Sitecore.Kernel"/>

with:

    <processor type="SitecoreTactics.Publishing.Pipelines.Publish.ProcessQueue, SitecoreTactics"/>

Step 2: Override the ProcessQueue processor

We have two alternatives to solve this, one is given by Ivan Sheyenko from Sitecore Support and one solved by our colleague Muktesh Mehta.

Below is the solution provided by Sitecore Support, which still needs improvements:

public class ProcessQueue : ProcessQueue
{
    // Methods
    protected override IEnumerable<publishingcandidate> RemoveDuplicateReferrers(IEnumerable<publishingcandidate> referredItems, PublishContext context)
    {
        Assert.ArgumentNotNull(referredItems, "referredItems");
        Assert.ArgumentNotNull(context, "context");
        List<publishingcandidate> list = new List<publishingcandidate>();
        foreach (IEnumerable<publishingcandidate> enumerable in context.Queue)
        {
            foreach (PublishingCandidate candidate in enumerable)
            {
                list.Add(candidate);
            }
        }
        List<publishingcandidate> list2 = new List<publishingcandidate>();
        foreach (PublishingCandidate candidate2 in referredItems)
        {
            if (!(list.Contains(candidate2) || list2.Contains(candidate2)))
            {
                list2.Add(candidate2);
            }
        }
        return list2;
    }
}

Below is the solution done by Muktesh Mehta, which is more useful in our architecture:

public class ProcessQueue : ProcessQueue
{
    // Methods
    protected virtual System.Collections.Generic.IEnumerable<publishingcandidate> RemoveDuplicateReferrers(System.Collections.Generic.IEnumerable<publishingcandidate> referredItems, PublishContext context)
        {
            Assert.ArgumentNotNull(referredItems, "referredItems");
            Assert.ArgumentNotNull(context, "context");
            List<id> idCollection = new List<id>();
            System.Collections.Generic.List<publishingcandidate> finalReferredItems = new System.Collections.Generic.List<publishingcandidate>();
            foreach (var referred in referredItems)
            {
                if(!idCollection.Contains(referred.ItemId))
                {
                    idCollection.Add(referred.ItemId);
                    finalReferredItems.Add(referred);
                }
            }
            return finalReferredItems.AsEnumerable();
        }
}

Now, enjoy bug-free Sitecore Related Item Publishing.. Njoy!

Other Sitecore 7.2 Bugs Solved!

• Sitecore Media Attach Detach Getting Slower
• Solved Sitecore PDF Loading Issue