Things to remember while using CDN for Sitecore websites

We had many learnings while using CDN for our different types of Sitecore websites, so thought to share here, if get useful to others! Before that I want to share one of few interesting incidents that left a message to us for configuring CDN carefully.

We believe that Sitecore resolves the Site using host name from the request URL, so for serving media requests we do not need to forward cookies or any other parameters from CDN to Sitecore CD servers. That's very true, but partially. If you are thinking for providing the best user experience to end-users, you need to take care more than this.

Now, consider a case that a user is publishing a new Content Page that have few images related to it also got published on your one more Publihing Target Databases. First request of this page came to one of many clustered servers on the same time when your publishing got finished. What are the chances that all those images will be visible to that first view of the page, 100%? No, not at all. Let's see why.

You know that CDN manages sticky session using a cookie (i.e, AWSELB for AWS CloudFront). We normally need sticky session for content pages. So, we never forget forwarding Cookies from CDN to CD servers and don't do the same for media files as media files has nothing to do with Cookies. In such cases, the first request of content page went to a server i.e, A. But, the images might get served from different servers say B, C, etc. (as we set them not to carry cookies) and think the images are yet to get reflected on any of these servers due to publishing or caching delay of a second. So, content of the page will be served properly but images will return 404 and CDN will cache the response for few minutes. It means we are still leaving with chances that end-users will get disturbed page layout for few minutes. This also gets applied to Stylesheet or Javascript files as well if they are served from Sitecore items. We can fix the issue if we apply sticky session forwarding for media requests as well.

CDN Configurations for caching media files

  1. Forward Cookies those play role in maintaining sticky session (i.e. AWSELB cookiefor all media items (To fix above explained issue)
  2. Forward Querystrings (To support Media Querystring parameters explained here for responsive websites)
  3. Never cache such media files those are protected i.e, those have disclaimers. You can keep them in a separate media folder and apply rule not to cache such URL patterns.
  4. If your media items are getting changed rarely, keep bigger caching duration i.e, 1 hour, otherwise keep it little as 5 minutes. Or to get more accurate results, Instead of all above rules, you can also get benefits of 304 if-modified-since header to serve media requests.

CDN Configurations for caching content pages

  1. If it's a pure static site without any user logins or protections, you can serve the site without forwarding any parameter.
  2. If your site is developed for multi device support, you must forward Referrer and User-Agent request headers.
  3. If your site is having any kind of login facility or requires session or has cookie-oriented responsive or adaptive architecture, you must forward Cookies header.
  4. If you have implemented security based on IP Addresses, you must forward X-Forwarded-For header.
  5. If you have implemented Browser Based Content Negotiation, you must forward Accept, Accept-Language, etc. parameters.
  6. If you are using functionalities like personalization, secured content, etc. you can avoid content caching on CDN.
  7. Never cache HTML content served through other than GET request.

So, for getting best usage of CDN with best user experience, you must have knowledge how your website are developed and behaves.

Avoid UNC file share to prevent File Change Notification issues

Recently we were experienced many unexpected application pool recycling with File Change Notifications (FCN) on our newly created Sandbox Sitecore CM instances. Every few hours (ranging from 20-40 hours) only one of two CM instances was getting restarted automatically with below error.

Change Notification for critical directories. File Change Notification Error in App_LocalResources HostingEnvironment initiated shutdown CONFIG change HostingEnvironment caused shutdown

What we tried

After spending a good amount of time on it, we hypothetical thought for few workarounds to do like
  • Stop Anti-virus on server
  • Use Process Monitor to check a File System for Web-root and Temporary Internet Files, etc. folders to know what's actually causing this
  • Check there's no access given on the web-root to any unauthorized user. 
But all looked good for us, even not getting much help from Google as well. So final option we had to debug the crash dumps.

One more thought came in mind that recycling is happening only on one instance, which is using Sublayouts from other Sitecore instance using UNC file share, and that other instance never got recycled! We know that DFS was the best option here, but we were not able to digest that UNC share can really cause this issue. At the same time we found a nice post where someone already faced the same issue and they fixed it by stopping UNC path sharing for sublayouts. And yes, it worked for us too.
(Note: Recycling is not happening because of number of re-compilations - "numRecompilesBeforeAppRestart")

- http://www.dnnsoftware.com/forums/threadid/318762/scope/posts/file-change-notification-issues-web-farm-over-unc-share
- http://blogs.msdn.com/b/tess/archive/2006/08/02/686373.aspx

What we Learned and Investigated:

- We learnt one more reason that can recycle the application pool
- Avoid using UNC sharing for ASPX, ASCX, RESX, App_Code files, etc. compilable files and Use DFS for them, what Sitecore recommends in Sitecore Scalability Guide.