How Sitecore caching work

This post describes different levels of Sitecore caches. Cache plays very important role in website performance. So, understanding of all Sitecore caches is really important. If we understand all of them, then it would be easy to do performance tuning using cache settings.

This post contains just theoretical overview of cache, will be posting about practical usage and performance tuning of caches soon :)

We can check how different cache are allocated and cleared, we have a tool given by Sitecore: http://mysite.com/sitecore/admin/cache.aspx. Even a great tool available to Sitecore Market Place - Sitecore Cache Admin, which describes how actual cache is managed by Sitecore.

Different Database Cache:

Prefetch Cache

Prefetch caches contain items that Sitecore accesses during and immediately after initialization and items with children that Sitecore often accesses as a group. Sitecore maintains those caches over the life of the application.

Each database prefetch cache entry represents an item in a database. Database prefetch cache entries include all field values for all versions of that item, and information about the parent and children of the item.

Read How to configure prefetch cache and how it affects application startup.

Data Cache

Data caches are dependent on database prefetch caches, which operate at a lower level. Like database prefetch caches, each entry in a database data cache represents a single item in a database, including parent/child relationships and field values for all versions in all languages of that item. Sitecore does not pre-populate database data caches.

The Caching.DefaultDataCacheSize setting in the web.config file specifies the default size for database data caches.

The purpose of this cache is to minimize the amount of requests to the database. This is extremely important for performance, as requesting items from the database is rather expensive.

Item Cache

Item caches store items. Database item caches are dependent on database data caches, which operate at a lower level. Each entry in a database item cache represents a single version of an item in a single language. Sitecore does not pre-populate database item caches.

The Caching.DefaultDataCacheSize setting in the web.config file specifies the default size for database data caches.

Database item caches contain objects of type Sitecore.Data.Items.Item.

The Caching.DefaultltemCacheSize setting in the web.config file specifies the default size for database item caches.

It would be best to have the average size of an item in Caching.AverageltemSize configuration attribute.

Standard Value Cache

Standard values caches contain standard values for data templates in the database. Sitecore does not pre-populate database standard values caches. Database standard values caches do not depend on any other caches.

The Caching.StandardValues.DefaultCacheSize setting in the web.config file specifies the default size for database standard values caches.

Sitecore uses the Caching.StandardValues.AverageValueSize setting in the web.config file to estimate the amount of memory consumed by the database standard values cache.


Different Website Cache

If you don't specify cache at a level then it gets its values from default website cache in the web.config. We can customize all these cases site-wise.
<cacheSizes>
 <sites>
         <website>
            <html>10MB</html>
            <registry>0</registry>
            <viewState>0</viewState>
            <xsl>5MB</xsl>
         </website>
       </sites>
</cacheSizes>

HTML Cache

The HTML cache (also known as the output cache) associated with each managed Web site contains the output generated by individual renderings under different conditions.

Sitecore provides caching options which allow the rendered data to be retrieved from cache if the data source, device, authentication status, user, rendering parameters and/or query string parameters are the same as the previous request.

Sitecore allows developers to define output cache criteria in three places:
  • In the Caching section of the sublayout and rendering definition item. (Global)
  • In the properties of the presentation component when you statically bind it to a layout or sublayout. (Static Controls)
  • In the Caching section of the Control Properties dialog when you bind a presentation component to a placeholder in layout details. (Dynamic)
HTML cache is disabled in the preview, webedit and debug modes.

Filtered Item Cache

The filtered items cache associated with each managed Web site contains information about versions of items relevant to different users.

The filteredItemsCacheSize attribute of each /configuration/Sitecore/sites/siteelement in the web.config file specifies the size of the filtered items cache for that managed Web site.

The Caching.DefaultFilteredItemsCacheSize setting in the web.config file specifies the default size of the size filtered items caches.

Registry Cache

The registry cache associated with each managed Web site contains data used primarily by the Sitecore user interfaces.

The registryCacheSize attribute of each /configuration/Sitecore/sites/site element in the web.config file specifies the size of the registry cache for that managed Web site.

The Caching.DedaultRegistryCacheSize setting in the web.config file specifies the default size for the registry caches.

Media Cache

Sitecore stores all media files to physical file system. All other cache are stored in RAM actually.

When publishing is done, Sitecore does not clear Media Cache like it does for other caches. Sitecore clears these media cache periodically.

User Cache

The client data store cache stores information about each authenticated user, such as the username or other user properties.

The Caching.DefaultClientDataCacheSize setting in the web.config file specifies the size of the client data store cache.

The disableClientData attribute of each /configuration/Sitecore/sites/site element in the web.config file enables or disables client data caching for that managed Web site.

Proxy Cache

Sitecore has a proxy item feature that allows items in one area of the content tree to appear in another.

These proxy items behave in the same way as normal items but have unique IDs to distinguish them from the original items.

The proxy cache keeps track of these IDs and how they map back to the original items.

How cache clearing works?

HTML cache

On publishing of any item, HTML cache is cleared. If we are using multisite module, the we can rewrite HTML cache module to clear HTML cache for item's related site. HTML cache for each page is built from multiple items, so publishing a single item, we cannot judge at how many items it would affect. That's why we have to clear full site HTML cache.

Item cache

Whenever any item is published, its Item Cache is also updated. If you publish an item which is linked to other items, then these items are cleared as well.

If you publish standard values or a template, it will clear all items based on that template.

If you delete/recycle/restore any item, its parent item's cache also updated, similarly while doing sorting, their siblings cache might get updated.

Data cache

Data cache is updated incrementally when changes take effect after a publish. It is rebuild incrementally when the items are requested again.

Prefetch cache

By default, items/templates specified in the config file are cached in Prefetch Cache when application initiated. Prefetch cache are updated same way of Data Cache.

Note: An ASP.NET application server restarts effectively removes all entries from all caches, except media cache. If we are using ASP.NET caching based on some items' values, we must clear ASP.NET cache too on publishing.

Sitecore Image Parameters and Image Control

This post is for those who are still:

- Creating duplicate Image Items on Sitecore for achieving Responsive Web Design or Image Gallery.
- Creating separate images for desktops, phones and tablets to achieve responsive web design.
- Creating Thumbnails, Preview and original images for image galleries.
- Resize images on-the-flyas per requirement.

Tired from maintaining multiple items(create, update, delete, publish) of each image?

There is a short and sweet solution within Sitecore itself to give freedom from above headache, that is Sitecore Image Parameters and Sitecore Image Control.

How it is beneficial?

- Using it, each requested image is created/scaled and cached on disk by Sitecore itself, so it does not impact on performance.
- This gives freedom from multiple uploads/updates/publish of same image and multiple Item Cache
- Saves lots of human efforts and time
- Freedom from, reduction in database size, beneficial from maintenance and point-of-view too.

Different Image Parameters:

w

Width of image

h

Height of image

mw

Maximum width of image

mh

Maximum height of image

iar

Ignore Aspect Ratio. Value should be 1 or 0.

as

Allow Stretch. Value should be 1 or 0.

thn

Create Thumbnail. Value should be 1 or 0.

bc

Background Color (When there is no aspect ratio set)

sc

Scale Image. 1 is default value.

Few Samples:

Expected Result Sitecore Image Control Image URL
Original <sc:Image Field="My Image" /> http://com.com/~/media/myimage.jpg
Width=150 <sc:image field="My Image" width="150" /> http://com.com/~/media/myimage.jpg?w=150
Height=200 <sc:image field="My Image" height="200" /> http://com.com/~/media/myimage.jpg?h=200
Height=200
Width=200
Ignore Aspect Ratio
<sc:image field="My Image" width="200" height="200" iar="true" /> http://com.com/~/media/myimage.jpg?h=200&w=200&iar=true


You can findout all parameters for image control from:
http://sdn.sitecore.net/Articles/XSL/5%203%20Enhancements/Image%20Enhancements.aspx

Now, you will think that how can we achieve adaptive images using theImage control. Well, for that we have two approaches:

  1. Use Sitecore Adaptive Images module.
  2. Use JS plugins like responsejs. This is more preferable approach.

    This plugin needs all images to be rendered as below:
    
    

    Means, the the control we create, should render image tag supporting different size attribute and value urls.

    Here, if plugin finds the device screen is suitable to 330 width, it will update image's source to data-330's value. Similarly to data-961 when it finds device's screen suitable to 961 width.

    For this, we can extend Sitecore's Image Control to achieve this image rendering.

Enjoy!!

Sitecore 404 Page Not Found handler without 302

When your requested page not found and while handling 404 error page, are you getting 200 or 302 HTTP Status Error Code? Then there is something to do more to achieve 404. For SEO purpose, our 404 error page should return a 404 response header. By default Sitecore shows a Page Not Found page but it's not 404 in real.

There can be many approaches to achieve 404 in Sitecore depending on our requirement. Here, two of them are shown.
  1. Show a specific Sitecore page on 404.
    This will show 404-Page Not Found page with current language, device and layout requested. Means, if you request any page from mobile and desktop, both will show 404 but with different rendering/output set in both device/layouts.
  2. Simple 404 - Page Not Found text.
    This will simply show text with 404 status code.

1. Show a specific Sitecore page on 404.

Step - 1:

Create a new processor which will be used before ExecuteRequest in httpRequestBegin pipeline in web.config. If context item is null, then set an Sitecore Item (errorpage) as Context and set a flag in Request Cache that this request is of 404.
namespace SitecoreTactics.Pipelines.HttpRequest
{
    public class Page404Resolver : Sitecore.Pipelines.HttpRequest.HttpRequestProcessor
    {
        public override void Process(Sitecore.Pipelines.HttpRequest.HttpRequestArgs args)
        {
            // If current item not available in Sitecore, then
            if(Sitecore.Context.Item == null)
            {
                // Find an error-page item and set it to context Item
                Item item404 = Sitecore.Context.Database.GetItem("Path of error-page Item");
                Sitecore.Context.Item = item404;

                // Set a flag in request cache to say this request is of 404.
                Sitecore.Context.Items["is404"] = "true";
            }
        }
    }
}
//else, continue pipeline... So no need to write else block

Step - 2:

In web.config, do below settings in HttpRequestBegin pipeline.
Add above processor Page404Resolver below ItemResolver as below.
<httpRequestBegin>
  <-- few processors -->
  <processor type="Sitecore.Pipelines.HttpRequest.ItemResolver, Sitecore.Kernel" />
  <processor type="SitecoreTaxtics.Pipelines.HttpRequest.Page404Resolver, SitecoreTactics.Pipelines" />
  <-- few processors -->
</httpRequestBegin>

Step - 3:

In layouts, check whether this request Cache has flag to true (means 404), then set the Status Code to 404 and description
Change in Layout:
protected override void Render(HtmlTextWriter writer)
{
    base.Render(writer);

    // If current request is for 404-errorpage...
    if (Sitecore.Context.Items["is404"] == "true")
    {
        try
        {
            Response.StatusCode = 404;
            Response.TrySkipIisCustomErrors = true;
            Response.StatusDescription = "File not found";
            Response.End();
        }
        catch (ThreadAbortException)
        {
             // Log error
        }
    }
}

2. Simple 404 - Page Not Found text

To just write down some 404 message, use above processor itself, but change code as below. In this approach, there is no need to write any code in layout as above.

// If current item not available in Sitecore, then go for 404
if (Sitecore.Context.Item == null)
{
  System.Web.HttpContext context = System.Web.HttpContext.Current;
  
  // Apply Response Headers
  context.Response.TrySkipIisCustomErrors = true;
  context.Response.StatusCode = 404;
  context.Response.StatusDescription = "Page not found";
  
  try
  {
    // Write 404 - description to response
    string str404 = "The page you requested does not exist.";
    context.Response.Write(str404);
    context.Response.End();
  }
  catch (System.Threading.ThreadAbortException ex)
  {
    // Log error
  }
}

That's it, you have your 404!!

Sitecore Pipelines and Processors in web request

Sitecore is such a flexible CMS, you can do any customizations so quickly. Sitecore has customized ASP.NET's framework to provide more flexibility and power for itself and Sitecore developers.

Pipelines are nothing but to perform a sequential opterations/process, which is defined in web.config. Which gives better flexibility by adding new or overriding existing processors.

Below are the pipelines covered while any page is requested. These pipelines would be different on Staging and Live environment, also it depends on different modules installed on Sitecore environment.

initialize


This pipeline initiate Sitecore application

preProcessRequest


It is invoked for each HTTP request managed by ASP.NET. It is used to prevent Sitecore from processing requests with specific extensions other than .aspx. And yet, it is not used for request processing logic.

Below are processors in this pipeline:

NormalizeRawUrl

If Sitecore not getting proper raw URL, then it normalize (rewrite)  it and set into current context.

IIS404Handler

It handles 404 error page and rewrite to proper URL.

FilterUrlExtensions

It decides which extensions to allow or reject. i.e., aspx, html, asmx, ics, etc. are allowed (Allowed/Blocked extensions are specified within this processor configurations in web.config file)

StripLanguage

It rewrite the URL by embedding language.

httpRequestBegin


This pipeline contains processors which are used for request processing logic. It sequential finds Site, Device, Language, Item, Layout, etc. details for the requested URL and then allow Sitecore to continue rendering it.

CheckIgnoreFlag

This is where Sitecore realises the html file doesn’t exists and redirects it to it’s 404 page.

StartMeasurements

It starts timer to record performance counters/measurements. The timer is stopped and performance counters/measurements are retrieved in the httpRequestEnd pipeline’s StopMeasurements processor. If any counter exceeds the defined threashold, it's logged in log file.

IgnoreList

Decides whether handle the current request or not depending on the ignoreUrlPrefix setting value. It checks if the requested page is in the ignore URL list in the web.config.

Mostly the URLs which Sitecore does not render itself are set in IgnoreList. Like, RichTextEditor, Telerik Dialogs, axd files.
Set IgnoreUrlPrefixes to a '|' separated list of url prefixes that should not be regarded and processed as friendly urls (ie. forms etc.)
If such page is defined, the pipeline terminates

SiteResolver

It parses the incoming URL and Determines the current site defined in /configuration/sitecore/sites. It identifies Site either by hostname of the current request or by passed querystring parameter: sc_site.It resolves Sitecore Site object. This object is now added to the current Context (Sitecore.Context.Site)

UserResolver

Identifies current user and add it to current request context (Sitecore.Context.User). In backend, Sitecore uses standard ASP.NET membership.

DatabaseResolver

Identifies current database and add it to current request context.(Sitecore.Context.Database. It also resovles database by passing querystring parameter:sc_content.

BeginDiagnostics

If debugging is on, it starts diagnosis of sitecore request.

DeviceResolver

Resolves the current Sitecore device either by querystring parameter sc_device or then knowing BrowserAgent and add it to current context (Sitecore.Context.Site.Device)

LanguageResolver

Resolves the Sitecore context language and add it to current context (Sitecore.Context.Language). Language can be identified either by querystring parameter sc_lang or by cookie (SiteName#Lang) value set to the browser.

CustomHandlers

It triggers custom handlers defiend in customHandlers/handler in web.config. This fulfills use of .ashx requests. Like, for url starts with ~/media/, the hander is set to sitecore_media.ashx. So, it will proceed for all requests of media. Similarly we have different handlers for api, xaml, icon, feed, etc.

FilterUrlExtensions

This processor is actually deprecated, same processor is taking care in preprocessRequest pipeline.

QueryStringResolver

It analyzes such query string as “sc_itemid”. It checks this item in context database with context language. If this item is valid, then it sets it to current context.

DynamicLinkResolver

It parses dynamic links (Links served using ~link.aspx) and gives itemid, language and database details by using a media prefix syntax (specified in the configuration/sitecore/mediaLibrary/mediaPrefixes section of web.config).
Depending on it, it sets this item as Context Site item.

AliasResolver

One Sitecore item can have multiple aliases, means one item can be accessed using different URLs. It checks the requested URL to see if it matches an alias that exists on the system.
An Alias is a an alternate path for an item when browsing the web site.
For this "AliasesActive" setting should be enabled in web.config

DefaultResolver

Resolves the start path item for the context site. Sitecore instantiates this item using the “RootPath” and “StartItem” attributes of the context site by simply concatenating these values when
  1. Context.Item hasn’t been set yet
  2. Context.Database has been set (usually from Context.Site.Database)
  3. Context.Site has been set and Context.Site.StartPath is non-empty (i.e., positive length)
  4. The LocalPath is empty or equals /default

FileResolver

It checks whether this request is a physical file or not. If it is physical file, then will be served as is, otherwise Sitecore will use its filepath as default.aspx (Default Page) and and continue request with Sitecore rendering process. If it finds same Item Name and a physical file, then physical file will be getting more priority.

ItemResolver

Resolves the Sitecore context item from the incoming URL.
The item will be null when a request is requested, which does not find any Sitecore item.

LayoutResolver

It determins layout for the request from querystring parameter: sc_layout or by getting layout assigned to the Sitecore.Context.Item (Set from the presentation details). Sitecore assigns Layout to a request by updating Sitecore.Context.Page.FilePath

ExecuteRequest

Rewrites the context path and handles the “item not found” or ”layout not found” errors

renderLayout


PageHandlers

Custom page handlers are executed here defined in pageHandlers/handler in web.config. We are not having any custom page handlers.

SecurityCheck

Checks security of all items in request for current user. It also checks security depending on Current Site requested and Preview Mode.

InsertRenderings

Adds renderings to the current page, which are assigned.

PageExtenders

We can add page extenders here defined in pageextenders/pageextender. For example, PreviewPageExtender, WebEditPageExtender, DebuggerPageExtender

BuildTree

Builds full page, and expands all controls and placeholders.

InsertSystemControls

All controls are inserted to the page as System Controls.

InsertUnusedControls

All unused controls are inserted to the page as System Controls.

BrowserCaching

Sets browser caching headers. Also add last modified header (Item updated DateTime)

My first ever blog post!

This is my first ever blog post for Sitecore on my website and I am not sure what to write! 

I have lots of hopes and dreams associated with having a new website that allows me to do so many things that I have wanted to do for so long and yet haven’t had the ability (or time) to do properly.

I would be posting here about my learning, experiments and other tactics on Sitecore. Will see how this go ahead. :)