Things to remember while using CDN for Sitecore websites

We had many learnings while using CDN for our different types of Sitecore websites, so thought to share here, if get useful to others! Before that I want to share one of few interesting incidents that left a message to us for configuring CDN carefully.

We believe that Sitecore resolves the Site using host name from the request URL, so for serving media requests we do not need to forward cookies or any other parameters from CDN to Sitecore CD servers. That's very true, but partially. If you are thinking for providing the best user experience to end-users, you need to take care more than this.

Now, consider a case that a user is publishing a new Content Page that have few images related to it also got published on your one more Publihing Target Databases. First request of this page came to one of many clustered servers on the same time when your publishing got finished. What are the chances that all those images will be visible to that first view of the page, 100%? No, not at all. Let's see why.

You know that CDN manages sticky session using a cookie (i.e, AWSELB for AWS CloudFront). We normally need sticky session for content pages. So, we never forget forwarding Cookies from CDN to CD servers and don't do the same for media files as media files has nothing to do with Cookies. In such cases, the first request of content page went to a server i.e, A. But, the images might get served from different servers say B, C, etc. (as we set them not to carry cookies) and think the images are yet to get reflected on any of these servers due to publishing or caching delay of a second. So, content of the page will be served properly but images will return 404 and CDN will cache the response for few minutes. It means we are still leaving with chances that end-users will get disturbed page layout for few minutes. This also gets applied to Stylesheet or Javascript files as well if they are served from Sitecore items. We can fix the issue if we apply sticky session forwarding for media requests as well.

CDN Configurations for caching media files

1. Forward Cookies those play role in maintaining sticky session (i.e. AWSELB cookie) for all media items (To fix above explained issue)
2. Forward Querystrings (To support Media Querystring parameters explained here for responsive websites)
3. Never cache such media files those are protected i.e, those have disclaimers. You can keep them in a separate media folder and apply rule not to cache such URL patterns.
4. If your media items are getting changed rarely, keep bigger caching duration i.e, 1 hour, otherwise keep it little as 5 minutes. Or to get more accurate results, Instead of all above rules, you can also get benefits of 304 if-modified-since header to serve media requests.

CDN Configurations for caching content pages

1. If it's a pure static site without any user logins or protections, you can serve the site without forwarding any parameter.
2. If your site is developed for multi device support, you must forward Referrer and User-Agent request headers.
3. If your site is having any kind of login facility or requires session or has cookie-oriented responsive or adaptive architecture, you must forward Cookies header.
4. If you have implemented security based on IP Addresses, you must forward X-Forwarded-For header.
5. If you have implemented Browser Based Content Negotiation, you must forward Accept, Accept-Language, etc. parameters.
6. If you are using functionalities like personalization, secured content, etc. you can avoid content caching on CDN.
7. Never cache HTML content served through other than GET request.

So, for getting best usage of CDN with best user experience, you must have knowledge how your website are developed and behaves.

Sitecore media streaming issue after publishing!

Recently we came across a strange behaviour of Sitecore Media streaming in MediaCache that "Overwritten media files are not getting reflected after publish". Just to have clear idea, this is not a browser caching issue mentioned in Sitecore KB article.

What's the issue?

We created a media item on CM and published it, and was visible on live site. Now we overwritten a new media file to the same media item and published it again. (Either using Detach/Attach from Content Editor or using Overwrite existing media from Page Editor.) Surprisingly, we were still getting older media file! We published again, again and again, but newly published media not getting updated.

And yet, this is a very random issue and occurs very rarely.

How we tried to troubleshoot?

1. We have multiple servers in cluster with 2 target databases. We found that few servers of both target databases are serving older media file and rest of them serving latest one.
2. Then we thought there might be some Item Path Cache or Item Cache clearing issue (Which happens on Sitecore some times). So, we cleared both these caches for this media item using Sitecore API. But result was same.
3. Then we cleared whole Item Cache and Data Cache using Sitecore API. The result was same.
4. Then we cleared All Sitecore Caches using http:///sitecore/admin/cache.aspx page. The result was same.
5. Final option we had to clear all media cache physical files (Website\App_Data\MediaCache) so that Sitecore will create new media cache from database and can serve latest one. Even after deleting all files and folder from it, new files got generated but still were older one.

So, no solution at all after applying these many tricks!

How we fixed?

We had no other option but recycling the Application Pool. Finally, the master key worked for us. :)

What we concluded and what's the solution?

The only conclusion we had that Sitecore is storing media files somewhere in Server memory as well. Strange, right?

We raised to Sitecore Support for further investigation. Many thanks to Andrey Krupskiy from support who investigated and confirmed that Sitecore is really storing media files in RAM as well that might have caused this and provided below solution.

There is an internal media cache in RAM. This cache is used when media is not yet saved to the filesystem. Even, if you check code of Sitecore.Resources.Media.MediaCache class, in Reflector, it says the same. Sitecore serves media file RAM before its actual file cache gets generated on disk (might be to serve media faster), which is default behaviour of Sitecore. We can disable this behaviour by changing below configuration in Web.Config.

<setting name="Media.StreamPartiallyCachedFiles" value="false" />

We disabled the Media.StreamPartiallyCachedFiles setting as shown above on CM and CD servers.

Now it has more than a month now, we haven't faced the issue again.

Our Learning on Publishing Sitecore Sublayouts

After we started using Web Deploy for publishing sublayouts on our multisite environment, we were getting random caching issues on our CD servers. We have multiple publishing target databases and load balanced multiple CD servers. So, as I mentioned in my previous blog, when a sublayout is published, it is published with help of Web Deploy to one server and then the sublayout is replicated to all other servers using DFS.

What kind of issues we found?

Sometimes we found that the published sublayout gets reflected on few servers, on few servers we still get older content. But surprisingly, the published sublayout was replicated on all servers and still we were getting different output from different servers. So, when this happens, we used to clear caches on servers where such issues occurred assuming this might be because of Sitecore caching issues, later on we found workaround to publishing those sublayouts again after 1 or 2 minutes.

Now you will understand how critical it would be to publish sublayouts and getting them reflected on live servers quickly to make go-lives, re-brandings, news releases or press releases successful in one go.

Note: We found this only for those Cacheable sublayouts.

Our learnings to fix such issues

1. Web Deploy publishing should be synchronous.

We analyzed below sequence happened rarely.

1. First we started publishing, so as per my previous blog, Web Deploy will start deploying sublayout in async mode (By default Web Deploy is configured on Sitecore is asynchronous). So, sublayout item publishing and sublayout physical file deploying are done in parallel.
2. So, chances that item gets published before sublayout file is copied.
3. Now, item is published, so CD servers will invoke "publish:end:remote" event and clear HTML (sublayout) cache.
4. Now, before the new published sublayout file gets deployed to CD server, end-user requested a page which uses the same sublayout. So, the HTML cache will be generated again for older Sublayout.
5. Now, Web Deploy sent a new Sublayout. (So, we have HTML cache of older sublayout)

Learning: Sublayout publishing should always be synchronous, so publishing will get on hold till the sublayout is not deployed to the CD server, which can be configured in WebDeploy.config as below.

<event name="publish:begin">
 <handler type="SitecoreTactics.Publishing.BeginWebDeploy, SitecoreTactics" method="PublishSublayouts">
  <synchronous>true</synchronous>
  <tasks hint="list:AddTask">
   ......
  </tasks>
 </handler>
</event>

2. Target Database should be the first publishing target.

We have 3 publishing targets say, web, web-2, web-3. Means, any publishing will be done in this sequence itself. Earlier we found the target database inside the WebDeploy.config was set as last publishing target, means web-3. We might have done this in past to make sure the item is published before sublayout getting deployed.

But, as per our recent experiences and findings, we should keep it as first publishing target. So, in our case, it should be web. This will help us when there are multiple servers in DFS, so when DFS is taking some more time in replicating to other servers. So,  for reducing such chances of delayed Sublayout deploy, we should keep web as the publishing target in WebDeploy.config as below.

<tasks hint="list:AddTask">  
 <default type="Sitecore.Publishing.WebDeploy.Task">  
   <!-- It should be the first Publishing target database -->  
   <targetDatabase>web</targetDatabase> 
 </default>
</tasks>

3. We can make few seconds delay in clearing HTML Cache.

If we have done above cases, there are no chances that the CD server where Web Deploy is sending Sublayout file will get any issues. But just consider a worst case where DFS is taking more time to replicate sublayouts to other servers. So, chances that HTML Cache will get cleared before the sublayout replication is done.

To avoid such cases, we can add a delay of few seconds, say 2 or 3 seconds before clearing HTML Cache (Only when the sublayouts are getting published)

Finally, we have hassle free one-time Sublayout publishing working without any caching issues very well! I'm sure this will be helpful to others who are facing same kind of issues.

Get Optimized Images For Sitecore Responsive Websites

If you are having Responsive or Adaptive websites built in Sitecore and using Sitecore Image Parameters to resize images on-the-fly, this post is helpful to you!

Recently while working for responsive websites, we found that while resizing image with less dimensions, Sitecore produces image with more file size than its original one. It's really not expected because it also increases page load time.

Example

Below is the image I found from Sitecore Website's Homepage, which is having dimensions of 660px × 441px and having size of 48.45 kB (49,614 bytes). Image: http://dijaxps1e29ue.cloudfront.net/~/media/Redesign/Common/Heros/600x441/Homepage_hero_600x441.ashx?ts=121514021455931&la=en

Now if I request Sitecore to produce image with less resolution. i.e., with width of 600px. So, it generates an image of dimensions of 600px × 401px and having size of 57.4 kB (58,778 bytes). Image: http://dijaxps1e29ue.cloudfront.net/~/media/Redesign/Common/Heros/600x441/Homepage_hero_600x441.ashx?ts=121514021455931&la=en&w=600

Is it a bug from Sitecore?

No. But Sitecore by default uses "Lossy Compression Algorithm" to resize images, so reducing image dimensions will not reduce file size. Also, Sitecore uses 95% of image quality by default, that will generate image with bigger file size. To know more about it, you can check code from Sitecore.Resources.Media.ImageEffectsResize class ResizeImageStream() function.

<setting name="Media.UseLegacyResizing" value="false" />
<setting name="Media.Resizing.Quality" value="95" />

Reducing above quality setting may give us image with less file size but should we compromize with quality of image?

Then how to solve this?

There is an alternate way Sitecore gives which was the default behavior in Sitecore in earlier versions that is by enabling Sitecore's ImageLegacyResizing. You can know more about it from Sitecore.Resources.Media.ImageEffectsResize class ResizeLegacy() function. You can do below settings for getting reduced file size. Thank to Sitecore Support guy (Paul Kravchenko) for guiding me in this direction.

<setting name="Media.UseLegacyResizing" value="true" />
<setting name="Media.InterpolationMode" value="Low" />

Media.UseLegacyResizing
This setting controls whether to use legacy resizing (ie. bypass the Sitecore.ImageLib library).

Possible values can be:
true
false (Sitecore's default value)

Media.InterpolationMode
The interpolation mode to use when resizing images, which are available on System.Drawing.Drawing2D.InterpolationMode enum. Read more about InterpolationMode. We can any of below values as per our need.

Possible values can be:
Bicubic
Bilinear
Default
High (Sitecore's default value)
HighQualityBicubic
HighQualityBilinear
Low
NearestNeighbor

Again, Sitecore defines these settings in configuration file so values of the setting remains same for each image resize, so not that much useful to get a generic solution. Eager to know if someone has such generic solution to resize any kind of image, by maintaining quality with reduced file size what Photoshop or Paint.Net gives.

Solved Sitecore PDF Loading Issue

Are getting problem in viewing PDF files in your newly upgraded Sitecore environment? We came across the same bug while upgrading Sitecore to version 7.2 that in some browsers while opening PDF files, they get stuck. Surprisingly after that, pressing Ctrl+F5 was allowing to view PDF file. See below screenshot when the downloading stuck:

Workaround from Sitecore

In Sitecore 7.2, they gave workaround to solve the issue by forcing download of PDFs by doing below setting:

        <mediaType name="PDF file" extensions="pdf">
          <mimeType>application/pdf</mimeType>
          <forceDownload>true</forceDownload>
        </mediaType>

But, this is not the solution for our problem. We can't say our clients to download and view the PDF, it should be viewed in browser itself. We applied many tricks, and finally we solved it.

So, what was the issue?

After reverse engineering MediaRequestHandler, we came to know that Sitecore is downloading PDF content partially in range using HTTP 206 Status Code, which might be creating problems in different browser. In Sitecore versions earlier than before 6.5, Sitecore was downloading PDF or any other media in one go, which never failed.

So, one thing was sure that this bug was just because of Range Retrieval mechanism provided by Sitecore or the plug-in our browser does not support it. See below screenshot which shows how Sitecore downloads PDF content in range.

 

Final Solution

We just tried disabling the Range Retrieval Mechanism as below and found the bug resolved.

    <setting name="Media.EnableRangeRetrievalRequest" value="false" />

Many thanks to our colleague Sandeep Gupta who helped in the investigation.

Enjoy accurate media download now!!

Other Sitecore 7.2 Bugs Solved!

• Nasty Bug Found in Sitecore 7.2 Publish Related Items
• Sitecore Media Attach Detach Getting Slower

Sitecore Bug - Media Attach Detach Getting Slower

While upgrading to Sitecore 7.2, we faced another big bug regarding slowness. When we attach or detach media file or edit image file online, we found heavy load and slowness on DB server which subsides subsequently, but the browser becomes unresponsive during this time. And slowness is observed across CM platform. (We have not checked but this issue is found from Sitecore 6.5 onwards)

We have New Relic configured on CM server, which shows slow transactions. The stacktrace in all cases has same portion mentioned below:

Sitecore.Data.DataProviders.Sql.SqlDataApi.CreateReader(:0)
Sitecore.Data.DataProviders.Sql.SqlDataProvider.Exists(:0)
Sitecore.Data.DataProviders.Sql.SqlDataProvider.CheckIfBlobShouldBeDeleted(:0)
Sitecore.Data.DataProviders.Sql.SqlDataProvider.RemoveOldBlobs(:0)
Sitecore.Data.DataProviders.Sql.SqlDataProvider.SaveItem(:0)

Alongwith above stacktrace, we also found following as slowest SQL operation which is implemented in CheckIfBlobShouldBeDeleted method, which tells whether blob is used or not:

SELECT tmp.[Id] FROM 
   (SELECT sf.[Id] FROM [SharedFields] sf 
      WHERE sf.[Value] LIKE @blobId 
    UNION 
    SELECT uf.[Id] FROM [UnversionedFields] uf 
      WHERE uf.[Value] LIKE @blobId 
    UNION 
    SELECT vf.[Id] FROM [VersionedFields] vf 
      WHERE vf.[Value] LIKE @blobId 
    UNION 
    SELECT af.[FieldId] [Id] FROM [ArchivedFields] af 
      WHERE af.[Value] LIKE @blobId 
    ) tmp

Credit goes to Muktesh Mehta for drilling down the issue and Sergey Kravchenko from Sitecore Support who gave us solution for it.

We raised this to Sitecore with all above information, Sitecore provided a new config file and assembly for overriding SqlServerDataProvider, which solved our issue. You can get the assembly and config from Sitecore Support using Ticket# 412563.

Hope, this will help you if you are facing the same issue!

Media Performance Related Posts:
- IRequiresSessionState slowing down Sitecore Media Requests
- Improve Sitecore Media Performance using Reverse Proxy
- Upload Sitecore media items in selected languages

Other Sitecore 7.2 Bugs Solved!

• Nasty Bug Found in Sitecore 7.2 Publish Related Items
• Solved Sitecore PDF Loading Issue

IRequiresSessionState slowing down Sitecore Media Requests

For last few weeks we faced a performance issue on our Sitecore production servers. Page response time was substantially increased by 50% and getting slowness in full page loading. While investigation we found that this was just because of our customized MediaRequestHandler.

Our requirement was to display some Disclaimer Page when a user request for some Sitecore media files (not for any image), after accepting disclaimer we can continue to download requested media. All other media items should be served directly.

We used session for managing this disclaimer. For using session in the MediaRequestHandler, we implemented iRequiresSessionState interface in it. See below code snippet

    class CustomRequestHandler : Sitecore.Resources.Media.MediaRequestHandler, IRequiresSessionState
    {
        protected override bool DoProcessRequest(HttpContext context)
        {
            if (context.Request.Url.ToString().ToLower().Contains("/~/media/files/"))
            {
                // Check session, if session does not available, then send it to disclaimer
                // Once user accepts Disclaimer, generate a session with user details.
            }
            else
            {
                // Proceed all image requests
                base.DoProcessRequest(...);
            }
        }
    }

So, what the reason for the slower page rendering? Yes, that's IRequiresSessionState which was a halts in performance.

Can IRequiresSessionState really make my page slower?

Below image shows how ASP.NET Session state works:

Now we can definitely say YES, it can slow down. State access requires that the aspx pages or handler, run sequentially. Each page/handler may read from and write to the session state. To handle this without creating any errors in the session state, .NET will only run one at a time using locking mechanism, so no handlers will run when the aspx page itself is running.

Tactics:

• Can we customize handler not to use session state?
  - No, this will disable session so we cannot achieve disclaimer functionality
• We can decide the request needs session or not and use session dynamically.
  - Is it possible? Yes, it is, using HttpModules

Solved by Dynamically deciding Session State Behavior programmatically

Changing ASP.NET session state behavior dynamically is possible only using HttpModules or Global.asax. It is not possible in HttpHandlers or other code behind. This solution is possible only in .NET 4.0 and later. Yes, we can define it at Page Directives but that's static.

As per our requirement, we have to show Dislaimer for selected Sitecore Media Files only (Not for any Media Images), we thought to create a HttpModule, which will check whether the request is for media files or images. If it is a file, then enable session state and if it is an image, disable session state.

Step-1:

We removed IRequiresSessionState from our CustomMediaRequestHandler, so it will not use Session State directly.

Step-2:

Created HttpModule, which will decide whether to use Session State or not.

    public class CustomMediaRequestSessionModule : IHttpModule
    {
        public void Init(HttpApplication application)
        {
            application.BeginRequest += (new EventHandler(this.Application_BeginRequest));
        }

        private void Application_BeginRequest(Object source, EventArgs e)
        {
            HttpApplication application = (HttpApplication)source;
            var CurrentContext = HttpContext.Current;

            if (CurrentContext.Request.Url.ToString().ToLower().Contains("/~/media/files/"))
            {
                // Files need Disclaimer functionality, so session needed
                CurrentContext.SetSessionStateBehavior(System.Web.SessionState.SessionStateBehavior.Required);
            }
            else if (CurrentContext.Request.Url.ToString().ToLower().Contains("/~/media/images/"))
            {
                // Images do not need Disclaimer, so no session needed
                CurrentContext.SetSessionStateBehavior(System.Web.SessionState.SessionStateBehavior.Disabled);
            }
        }
        
        public void Dispose()
        {
        }
    }

Step-3:

Now, in web.config, register this module under system.webServer/modules section:

 <add type="SitecoreTactics.Resources.Media.CustomMediaRequestSessionModule, SitecoreTactics" name="CustomMediaRequestSessionModule" /> 

And now we checked Page Response time, now feeling heavenly. All problem gone, Page Response Time now dropped down to earlier stage.

Good reads for improving Sitecore Media Item Performance

• Improve media item performance by applying Browser Caching
• Improve media item performance by using Reverse Proxy

Improve Sitecore Media Performance using Reverse Proxy

- Are you facing slowness in serving media library items?
- Are you getting increased response time due to media requests?
- Is your Sitecore instance serves media slower even if you have applied output (response) cache?

Then this post will surely help you, which describe how we can improve Sitecore Media Library performance by implementing Reverse Proxy Server.

How Reverse Proxy will help to improve media performance?

The Reverse Proxy will play a role being a proxy between the client and Sitecore web server. Reverse Proxy provides caching mechanism, which caches all media items. So, once any user has requested any media file from server, will be get cached on Reverse Proxy itself. So, from second time onwards, Reverse Proxy will not get media from Sitecore Web Server but will serve it from its own cache.

We can use URL Rewrite Module and Application Request Routing (ARR) to implement a Reverse Proxy Server.

Step - 1 : Install ARR and Url Rewrite on IIS?

1. Setup IIS 7.0+ on your Server which will work as Proxy.
2. Install URL Rewrite module. You can download it from here.
3. Install ARR module. You can download it from here.

Step - 2 : Configure URL Rewrite module:

1. Create a new Website in IIS or use Default website, and click on Website, then click on URL Rewrite option under IIS section
   
2. Edit inbound rules as below:
   
   
   
   
   
   Above configuration shows that if requested host is www.patelyogesh.in, then this rule will be applied for its all requests (*).
3. Configure Rewrite URL for above configurations as below. This will make sure that all requests coming from http://www.patelyogesh.in/ will be rewrited to http://rp.patelyogesh.in/*.
   
   
   
   When we apply above configurations, it will generate a web.config file under the website directory, which will look like below:
   
   

   <system.webServer>
       <rewrite>
         <rules>
            <rule name="Sitecore-Production" stopProcessing="true">
                <match url="(.*)" />
                <action type="Rewrite" url="http://rp.patelyogesh.in/{R:1}" />
                <conditions>
                    <add input="{HTTP_HOST}" pattern="www.patelyogesh.in" />
                </conditions>
            </rule>
         </rules>
       </rewrite>
     </system.webServer>
   

   
   We can also configure multiple domain's URL rewrites in the same way.
   
   

Step - 3 : Configure ARR (Application Request Routing) Cache

1. Select the Server Node, now select Application Request Routing Cache option.
   
   
2. Add Drive where the caching will be stored by ARR Module. The below image shows how we can configure ARR module and how it will look.
3. Make sure the Identity user of the Application Pool should have read/write access of the drives configured here. So, ARR will store all cache files here only.
   
   
   
4. Enable Proxy As per below image, click on Proxy Settings on the right side bar and enable Proxy.
   
   
   

Finally, Reverse Proxy setup finished, took just 15 minutes only!!

Now, Request to http://www.patelyogesh.in/. This will serve you content from http://rp.patelyogesh.in itself by traversing through Reverse Proxy. It's really easy and simple, isn't it?

How to confirm Reverse Proxy working fine?

We can provide our Custom Response Headers using Proxy Settings from ARR Cache option as shown in above image. So, if Reverse Proxy is working well, we will get those headers in response.

How to confirm ARR Caching working fine?

In File Explorer, open the Drive folder we configured in ARR Cache settings. We will get all files cached by ARR.

Cache for the URL: http://www.patelyogesh.in/~/media/Images/yogi.png (Rewrited URL: http://rp.patelyogesh.in/~/media/Images/yogi.png) will be stored at : <website root>\rp.patelyogesh.in\~\media\Images\yogi.png.full.

How to delete ARR cache programmatically?

In ARR Cache settings, you will find a button Delete Specific Cached Objects which can clear specific URL cache. It also supports wild cards for clearing cache.

We can create a web service on the ARR website, which will get a URL as input and will clear cache accordingly using below code. Now, on each media item publish from Sitecore, we will clear the ARR cache. We can decice How and when to make a call to the ARR web service to cache clear according to our architecture.

Source code to clear ARR cache programmatically:

[WebMethod]
public static ClearCache(string urlToCacheClear)
{
      var url = urlToCacheClear == "ALL" ? string.Empty : urlToCacheClear;

      var m = new ServerManager();
      var x = m.GetApplicationHostConfiguration().GetSection("system.webServer/diskCache");
      var method = x.Methods["FlushUrl"].CreateInstance();
      method.Input.SetAttributeValue("url", url);
      method.Execute();
}

Enjoy Reverse Proxy! Enjoy improved Sitecore Media Library performance!!

Good reads on Reverse Proxy:
- http://www.agarwalnishant.com/2013/04/improve-sitecore-media-library.html

- http://www.iis.net/learn/extensions/url-rewrite-module/reverse-proxy-with-url-rewrite-v2-and-application-request-routing

How Sitecore media cache Works?

Sitecore stores all media cache to file system, unlike all other caches, stored in RAM. Media items are stored in database, so media cache is required to reduce database calls and serve media files faster to end-user. Let's understand Sitecore media cache mechanism.

How Media Cache Created?

- When we upload a new media file to Sitecore, its media cache is created in Website\App_Data\MediaCache\<sitename>\<Hashcode of MediaId> folder. Sitecore assigns unique MediaId to each media item, which gets changed on each modification of media item.

- For each media item, Sitecore creates an INI file with name of MediaId in the respective folder, which stores different attributes of the media file inside it.

See below image as a reference:

In above case, this media's MediaId is "8c683332453741038b8876bf5915d188", so the ini file (8c683332453741038b8876bf5915d188.ini) is generated with name of MediaId. On right side, all information is stored in same file. dataFile shows physical media file name 7b4a5e3934914d57a390bedcab67380c.jpg.

Different attributes in INI file:

Attribute	Description
Key	height, width, thumbnail, background color, etc. image parameters passed by query string. You can get better idea by reading my earlier Blog Post on Sitecore Image Control Parameters
extension	Extension of  media file.
headers	Content Type, etc.
datafile	Physical file name stored as media cache in same folder.

How Media Cache Deleted?

Sitecore provides a cleanup agent to clear older media files, which clears media files every specified interval of time. By default it clears all media cache files created 90 days ago. See below settings in web.config:

    
     <agent type="Sitecore.Tasks.CleanupAgent" method="Run" interval="06:00:00">
        <!-- Specifies files to be cleaned up.
             If rolling="true", [minCount] and [maxCount] will be ignored.
             [minAge] and [maxAge] must be specified as [days.]hh:mm:ss. The default value
             of [minAge] is 30 minutes.
             [strategy]: number of files within hour, day, week, month, year
             [recursive=true|false]: descend folders?
        -->
        <files hint="raw:AddCommand">
          <remove folder="/App_Data/MediaCache" pattern="*.*" maxAge="90.00:00:00" recursive="true" />
        </files>
      </agent> 

Media Cache Hidden Secrets

• If a user has requested a media file with different querystring parameters, then Sitecore creates different media files runtime and stores all those files in same folder where original media file is stored. Also, all these combinations are stored in the same INI file itself.
  
  You can try accessing your media image with different parameters like below and check media cache:
  -http://sitecoreblog.patelyogesh.in/~/media/Images/myimage.jpg
  -http://sitecoreblog.patelyogesh.in/~/media/Images/myimage.jpg?h=100
  -http://sitecoreblog.patelyogesh.in/~/media/Images/myimage.jpg?w=200
• When a media item is updated, Sitecore creates a new media cache with new INI file and a new media file even though the media file(blob) remains same or item already has same media cache. So, there will be duplicate media cache but Sitecore will refer to latest file only.
• As we know Sitecore creates media cache in folder with name of Context Site. So, if one media file is accessed by two different Sites (SiteContext), then media cache will be generated for both sites, means in both sites' folders. For example, When media item is accessed by Content Editor, cache will be created for Shell site and when accessed by Website, then it will be created for Website.

How to read/create media cache programatically?

Yes, it's possible to create or read media cache by Sitecore API:

    
    ///////////////////////////////////
    // Read Media Cache
    ///////////////////////////////////

    MediaRequest request; // media parameters like height, width, etc.
    Sitecore.Resources.Media.Media media // item for which you want to read media cache.

    // set request.Options
    // set media

    // This is the media cache stored in MediaStream
    MediaStream mStream = MediaManager.Cache.GetStream(media, request.Options);


    ///////////////////////////////////
    // Write Media Stream to Media Cache
    ///////////////////////////////////

    // Manipulate your media file and set it into MediaStream

    // store the media stream to media cache.
    MediaManager.Cache.AddStream(media, request.Options, mStream, out outStream);

We can know more about Media Cache understanding below classes using Reflector:
- Sitecore.Resources.Media.MediaCache
- Sitecore.Resources.Media.MediaCacheRecord

Block files by MIME content type while Sitecore upload

As per Web Security best practices, while media upload in Sitecore, we should block upload of EXE, DLL, BAT, ASPX, ASP, etc. files on server. Do you think, it is enough? I think, No.

We should also block files by checking their MIME content types because someone can also upload Exe/Dll files by renaming them as Jpg or any other extension is allowed. So, this can be a serious threat too.

So, checking MIME content type is equal important as checking file extensions.

Why checking only file extension is not enough?

We implemented a module to restrict certain extensions, provided by Yuriy Yurkovsky from Sitecore Support, Prevent files from being uploaded which is working absolutely fine. Michael Reynolds also nicely presented restricting file extensions on his post Restrict Certain Extensions From Being Uploaded.

Later on, while testing for security threats, we found two issues while implementing blocking extensions.Thanks to our QA Analyst Chirag Patel for finding such nice scenarios and also shown us how it is harmful.

1. What if I upload file as "setup. EXE" instead of "setup.EXE"? (Just add a space after dot)
2. What if I upload file my EXE file by renaming as JPG? (Setup.JPG instead of Setup.EXE)

Yes, in both cases we were able to upload EXE contents which should be blocked by us. See below image, how EXE file uploaded as JPG behaves when client requests. This can be a serious threat to our application.

For case 1, we updated the code given in above module by removing the space between dot and file extension.
For case 2, we can use below approach.

How to restrict upload of certain MIME content types

As per the case 2, users can upload EXE  files by renaming them as JPG file. So, we can block them by their content type. Let's see how we can block content types, which is equal important as blocking files by extensions.

Below can be the patch configuration file, for better understanding, I used same format as Michael Reynolds' post to restrict extensions:

Here, two kind of content types are blocked:
- application/octet-stream (Used for bin, dms, lha, lzh, exe, dll contents)
- application/zip (Used for zip content)
 

<?xml version="1.0" encoding="utf-8"?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <processors>
      <uiUpload>
        <processor mode="on" type="SitecoreTactics.Pipelines.Upload.CheckForRestrictedContentType, SitecoreTactics" patch:before="processor[@type='Sitecore.Pipelines.Upload.CheckSize, Sitecore.Kernel']">
          <restrictedcontentTypes hint="raw:AddRestrictedContentType">
            <!-- content types to restrict -->
            <contentType>application/octet-stream</contentType>
            <contentType>application/zip</contentType>
        </restrictedcontentTypes>
        </processor>
      </uiUpload>
    </processors>
  </sitecore>
</configuration>

You can get more content types from:
http://www.freeformatter.com/mime-types-list.html
http://www.dailycoding.com/Posts/mime_contenttypes_with_file_extension.aspx


Below can be the source code to block certain content types defined in above config file.

namespace SitecoreTactics.Pipelines.Upload
{
    public class CheckForRestrictedContentType : UploadProcessor
    {
        private List<string> _RestrictedContentType;
        private List<string> RestrictedContentType
        {
            get
            {
                if (_RestrictedContentType == null)
                {
                    _RestrictedContentType = new List<string>();
                }

                return _RestrictedContentType;
            }
        }

        public void Process(UploadArgs args)
        {
            foreach (string fileKey in args.Files)
            {
                string fileName = args.Files[fileKey].FileName;
                string contentType = args.Files[fileKey].ContentType;

                if (IsRestrictedContentType(contentType))
                {
                    args.ErrorText = Translate.Text(string.Format("The file \"{0}\" cannot be uploaded. Files with an content Type of {1} are not allowed.", fileName, contentType));
                    Log.Warn(args.ErrorText, this);
                    args.AbortPipeline();
                }
            }
        }


        private bool IsRestrictedContentType(string contentType)
        {
            return RestrictedContentType.Exists(restrictedContentType => string.Equals(restrictedContentType, contentType, StringComparison.CurrentCultureIgnoreCase));
        }

        protected virtual void AddRestrictedContentType(XmlNode configNode)
        {
            if (configNode == null || string.IsNullOrEmpty(configNode.InnerText))
            {
                return;
            }

            RestrictedContentType.Add(configNode.InnerText);
        }
    }
}

I feel, now my Sitecore application is more secured!

Show PDF Thumbnail as Icon in Content Editor

Sitecore shows PDF icon as a thumbnail, so it becomes very difficult to find out a PDF file from a big list of uploaded files. Just imagine, life would be so easy when Sitecore provides PDF thumbnails as the icons just like images!!

It is quite possible and easy to show PDF thumbnails in different dimensions just by overriding the MediaRequestHandler of Sitecore. See my earlier post, PDF Thumbnail Handler blog. You can also find PDF Thumbnail Handler on Sitecore MarketPlace.

Use of PDF Thumbnails Handler

Once the concept of PDF Thumbnail Handler is understood, we can achieve this easily. Do following:

1. Install PDF Thumbnail Handler to your Sitecore and make it up and running.
2. Update PDF item's Icon field. Replace ~/media to ~/mediathumb
3. Now, check Sitecore Content Editor will show PDF thumbnails as icons.

By default PDF icons are available as below image:


The Icon has value: ~/media/36C02213E38441D9BA1AA82DB86A80E0.ashx?h=16&thn=1&w=16, which will load icon of PDF which is defined in the sitecore itself.

As per PDF Thumbnail Creation Handler, by using ~/mediathumb handler by updating its value to: ~/mediathumb/36C02213E38441D9BA1AA82DB86A80E0.ashx?h=16&thn=1&w=16. See below image which shows how PDF thumbnail is shown as icon.

 Let's make PDF thumbnails working in Content Editor

Our requirement is to show thumbnails like below image:

Override MediaProvider of Sitecore, for that you need to do changes in web.config file.

   <!-- override Sitecore MediaProvider -->
   <mediaProvider type="SitecoreTactics.MediaProvider, SitecoreTactics"/>

Below is the code required in MediaProvider class. In the GetMediaUrl function, when the request of any PDF file is there, then replace existing ~/media/ handler with ~/mediathumb/.

namespace SitecoreTactics
{
    public class MediaProvider: Sitecore.Resources.Media.MediaProvider
    {
        public override string GetMediaUrl(MediaItem item, MediaUrlOptions options)
        {
            string mediaUrl;
            mediaUrl = base.GetMediaUrl(item, options);

            // When item is PDF and Thumbnail is requested
            if (item.Extension == "pdf" && options.Thumbnail)
                mediaUrl = mediaUrl.Replace(Config.MediaLinkPrefix, "~/mediathumb/");

            return mediaUrl;
        }
    }
}

Wow, let's enjoy easier life with PDF thumbnails in Content Editor!!

Related Posts:
- PDF Thumbnail Handler
- Sitecore HTTP Custom Handler

PDF Thumbnail Creation Handler in Sitecore

I recently published a Sitecore Marketplace module PDF Thumbnail Creater Handler. Basically it allows to generate thumbnail on-the-fly (dynamically) for the uploaded PDF in sitecore by passing width and/or height. This will allow user to request thumbnail for any height or width and the thumbnail will be stored as a media cache in Sitecore.

Requirement

Suppose user uploaded PDF in Sitecore.

• User want to generate thumbnails of the fist page of uploaded PDF. 
• User can choose height and/or width of thumbnails without any configurations.
• If user replaces a new file instead of that PDF, it should serve thumbnail of newly uploaded PDF. 
• Similarly the thumbnail URL should work if user moves/copies/deletes the PDF item.
• Finally, the conversion process should be scalable and quick enough so it does not affect performance and the thumbnails should be cached as media cache too

Below are the outputs of same PDF but different size thumbnails: 

PDF Thumb Original http://sitecore/~/mediathumb/Files/MyPDF.pdf	PDF Thumb Width=300 http://sitecore/~/mediathumb/Files/MyPDF.pdf?w=300	PDF Thumb Width=150 http://sitecore/~/mediathumb/Files/MyPDF.pdf?w=150

Theoretical Concept

I just thought to achieve this overriding Sitecore MediaRequestHandler. Sitecore allows to generate different sized thumbnails of uploaded image files, See how. Why should not I use the same concept to generate thumbnails from PDF? Only concern I looked was to convert PDF to JPG only, but was not that much easy. 

So, what I wanted to achieve:
 

PDF / Thumbnail Details	PDF / Thumbnail Path/URL
PDF Sitecore Path	/sitecore/media library/Files/MyPDF
PDF URL	http://sitecore/~/media/Files/MyPDF.pdf
PDF Thumbnail URL	http://sitecore/~/mediathumb/Files/MyPDF.pdf or http://sitecore/~/mediathumb/Files/MyPDF.jpg
PDF Thumbnail URL Width = 100 px	http://sitecore/~/mediathumb/Files/MyPDF.pdf?w=100
PDF Thumbnail URL Height = 200 px	http://sitecore/~/mediathumb/Files/MyPDF.pdf?h=200
PDF Thumbnail URL Width = 100 px Height = 200 px	http://sitecore/~/mediathumb/Files/MyPDF.pdf?w=100&h=200

How this achieved

PDF to JPG conversion can be done using GhostScript (With GPL License, which is free), which is very efficient and gives flexibility with many other options.

You can read my older post regarding Sitecore Custom HTTP Handler, I have described there in detail.

I created own Sitecore Custom Handlers (SitecoreTactics.ThumbnailManager.PDFThumbnailRequestHandler) to generate thumbnails of media(PDF) items. See below config changes this requires:

    
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>

    <!-- Define Custom Handler -->
    <customHandlers>
      <handler trigger="~/mediathumb/" handler="sitecore_media_thumb.ashx"  />
    </customHandlers>

    <!-- Define Media Prefix -->
    <mediaLibrary>
      <mediaPrefixes>
        <prefix value="~/mediathumb" />
      </mediaPrefixes>
    </mediaLibrary>
  </sitecore>


  <!-- Define Web Handler -->
  <system.webServer>
 <handlers>
     <add verb="*" path="sitecore_media_thumb.ashx" type="SitecoreTactics.ThumbnailManager.PDFThumbnailRequestHandler, SitecoreTactics.ThumbnailManager" name="SitecoreTactics.PDFThumbnailRequestHandler"/>
 </handlers>
  </system.webServer>
</configuration>    

Handler's source code to process thumbnail and use media cache as below.

namespace SitecoreTactics.ThumbnailManager
{
    public class PDFThumbnailRequestHandler : Sitecore.Resources.Media.MediaRequestHandler
    {
        protected override bool DoProcessRequest(HttpContext context)
        {
            Assert.ArgumentNotNull(context, "context");
            MediaRequest request = MediaManager.ParseMediaRequest(context.Request);

            if (request == null)
                return false;

            Sitecore.Resources.Media.Media media = null;
            try
            {
                media = MediaManager.GetMedia(request.MediaUri);
                if (media != null)
                    return this.DoProcessRequest(context, request, media);
            }
            catch (Exception ex)
            {
                Log.Error("PDF Thumbnail Generator error - URL:" + context.Request.Url.ToString() + ". Exception:" + ex.ToString(), this);
            }

            if (media == null)
            {
                context.Response.Write("404 - File not found");
                context.Response.End();
            }
            else
            {
                string itemNotFoundUrl = (Context.Site.LoginPage != string.Empty) ? Context.Site.LoginPage : Settings.NoAccessUrl;

                if (Settings.RequestErrors.UseServerSideRedirect)
                    HttpContext.Current.Server.Transfer(itemNotFoundUrl);
                else
                    HttpContext.Current.Response.Redirect(itemNotFoundUrl);
            }
            return true;
        }

        protected override bool DoProcessRequest(HttpContext context, MediaRequest request, Sitecore.Resources.Media.Media media)
        {
            Assert.ArgumentNotNull(context, "context");
            Assert.ArgumentNotNull(request, "request");
            Assert.ArgumentNotNull(media, "media");

            if (this.Modified(context, media, request.Options) == Sitecore.Tristate.False)
            {
                Event.RaiseEvent("media:request", new object[] { request });
                this.SendMediaHeaders(media, context);
                context.Response.StatusCode = 0x130;
                return true;
            }

            // Gets media stream for the requested media item thumbnail
            MediaStream stream = ProcessThumbnail(request, media);
            if (stream == null)
            {
                return false;
            }
            Event.RaiseEvent("media:request", new object[] { request });
            this.SendMediaHeaders(media, context);
            this.SendStreamHeaders(stream, context);
            using (stream)
            {
                context.Response.AddHeader("Content-Length", stream.Stream.Length.ToString());
                WebUtil.TransmitStream(stream.Stream, context.Response, Settings.Media.StreamBufferSize);
            }
            return true;
        }

        private MediaStream ProcessThumbnail(MediaRequest request, Sitecore.Resources.Media.Media media)
        {
            MediaStream mStream = null;
            
            ParseQueryString(request);

            mStream = MediaManager.Cache.GetStream(media, request.Options);

            if (mStream == null)
            {
                string tempPath = Settings.TempFolderPath + "/PDF-Thumbnails/";

                tempPath = MainUtil.MapPath(tempPath);

                if (!Directory.Exists(tempPath))
                    Directory.CreateDirectory(tempPath);

                // Prepare filenames
                string pdfFile = tempPath + media.MediaData.MediaId + ".pdf";
                string jpgFile = tempPath + media.MediaData.MediaId + ".jpg";

                string resizedJpgFile = tempPath + media.MediaData.MediaId + "_" + request.Options.Width.ToString() + "_" + request.Options.Height.ToString();

                if (!File.Exists(jpgFile))
                {
                    // Save BLOB media file to disk
                    MediaConverter.ConvertMediaItemToFile(media.MediaData.MediaItem, pdfFile);

                    // Convert PDF to Jpeg - First Pager
                    MediaConverter.ConvertPDFtoJPG(pdfFile, 1, jpgFile);

                }

                // Resize Image
                MediaConverter.ReSizeJPG(jpgFile, resizedJpgFile, request.Options.Width, request.Options.Height, true);

                // Convert resized image to stream
                MediaStream resizedStream = new MediaStream(File.Open(resizedJpgFile, FileMode.Open, FileAccess.Read, FileShare.Read), "jpg", media.MediaData.MediaItem);

                // Add the requested thumbnail to Media Cache
                MediaStream outStream = null;
                MediaManager.Cache.AddStream(media, request.Options, resizedStream, out outStream);

                if (outStream != null)
                {
                    // If Media cache is enabled
                    return outStream;
                }

            }

            // If Media cache is disabled
            return mStream;
        }

        public void ParseQueryString(MediaRequest mediaRequest)
        {
            HttpRequest httpRequest = mediaRequest.InnerRequest;

            Assert.ArgumentNotNull((object)httpRequest, "httpRequest");
            string str1 = httpRequest.QueryString["as"];
            if (!string.IsNullOrEmpty(str1))
                mediaRequest.Options.AllowStretch = MainUtil.GetBool(str1, false);
            string color = httpRequest.QueryString["bc"];
            if (!string.IsNullOrEmpty(color))
                mediaRequest.Options.BackgroundColor = MainUtil.StringToColor(color);

            string str2 = httpRequest.QueryString["dmc"];

            mediaRequest.Options.Height = MainUtil.GetInt(httpRequest.QueryString["h"], 0);
            string str3 = httpRequest.QueryString["iar"];
            if (!string.IsNullOrEmpty(str3))
                mediaRequest.Options.IgnoreAspectRatio = MainUtil.GetBool(str3, false);

            mediaRequest.Options.MaxHeight = MainUtil.GetInt(httpRequest.QueryString["mh"], 0);
            mediaRequest.Options.MaxWidth = MainUtil.GetInt(httpRequest.QueryString["mw"], 0);
            mediaRequest.Options.Scale = MainUtil.GetFloat(httpRequest.QueryString["sc"], 0.0f);
            string str4 = httpRequest.QueryString["thn"];
            if (!string.IsNullOrEmpty(str4))
                mediaRequest.Options.Thumbnail = MainUtil.GetBool(str4, false);

            mediaRequest.Options.Width = MainUtil.GetInt(httpRequest.QueryString["w"], 0);
        }
    }
}

You can get full source code (of older version) of this module from Sitecore Marketplace.

Update:
Module available on Sitecore Marketplace contains older code, having a bug on media cache that when someone overwrite media files (using detach/attach), it was serving older thumbnail. This bug has been fixed in above code, and will be available on marketplace very soon. Meanwhile, you can download the source code (excluding DLLs) from https://drive.google.com/file/d/0B1otw7vE3rGTQmU1U1l2TTJHQTA/view?usp=sharing:

Benefits of this approach

1. Dynamic conversion of PDF to Thumbnail when requested
2. Allows to convert different size thumbnails
3. Repeated thumbnails will be served from media cache.
4. Conversion is fast using GhostScript and media cache adds more power.

Related Posts:
- Show PDF Thumbnail Icons in Content Editor
- Sitecore HTTP Custom Handler

Save Sitecore Media Item to Disk file

Once we required to convert the Sitecore Media Item to a disk file (Save media item as a physical file on server). Sitecore does not provide any API to do this directly.

Below is the code to do it, thought to post it if can help others..

    string mediaItemPath = "/sitecore/media library/Images/myimage";
    string diskFolderPath = @"D:\Sitecore-Media\";

    MediaItem mediaItem = (MediaItem)Sitecore.Context.Database.GetItem(mediaItemPath);
    ConvertMediaItemToFile(mediaItem, diskFolderPath);


    public static void ConvertMediaItemToFile(MediaItem mediaItem, string folderName)
    {
        if (mediaItem.InnerItem["file path"].Length > 0)
            return;

        string fileName = folderName + mediaItem.Name + "." + mediaItem.Extension;

        var blobField = mediaItem.InnerItem.Fields["blob"];
        Stream stream = blobField.GetBlobStream();
        if (stream == null)
        {
            return;
        }

        string relativePath = Sitecore.IO.FileUtil.UnmapPath(fileName);
        try
        {
            SaveToFile(stream, fileName);
            stream.Flush();
            stream.Close();
        }
        catch (Exception ex)
        {
            Log.Error(string.Format("Cannot convert BLOB stream of '{0}' media item to '{1}' file", mediaItem.MediaPath, relativePath));
        }
    }

    private static void SaveToFile(Stream stream, string fileName)
    {
        byte[] buffer = new byte[8192];
        using (FileStream fs = File.Create(fileName))
        {
            int length;
            do
            {
                length = stream.Read(buffer, 0, buffer.Length);
                fs.Write(buffer, 0, length);
            }
            while (length > 0);

            fs.Flush();
            fs.Close();
        }
    }

Sitecore media and browser cache

Have you ever faced issues like your media items are not getting reflected to your page or you are still referring to older media files after media publish? Or your media files are not getting cached when accessing through revere proxy? Or your media files are not getting cached on browser level? Here is the solution in Sitecore itself, that is using Media Response Cacheability.

Media response cacheability is served using cache-control header, read more on topic 14.9 regarding cache-control header.

In web.config, you can define media response cacheability options in settings section like below:

    <!--  MEDIA RESPONSE - CACHEABILITY
    The HttpCacheability is used to set media response headers.
    Possible values: NoCache, Private, Public, Server, ServerAndNoCache, ServerAndPrivate
    Default value: public-->

    <setting name="MediaResponse.Cacheability" value="public" />

Here are six different settings to define media response headers, using which Sitecore manages media on client or browser level caching:

Media Cacheability Option	Description
NoCache	Browser cache is not created while using this option, so, every time media is served from server to device. This is not a good idea, when you want to improve performance by serving media files faster. This will slow down page speed.
Private	This option allows browsers to store media cache. But, the response is cacheable only on the client and not by shared (proxy server) caches. Suppose, the ISP is having a invisible proxy between user and internet, then the user can not get benefit of media caching.
Public	On step ahead than Private, using this option, response is cacheable by clients and shared (proxy) caches. So, anybody can use its caching mechanism. This option is mostly preferred to get optimum performance gain.
Server	The response is cached only at the origin server. Similar to the NoCache option. Clients receive a Cache-Control: no-cache directive but the document is cached on the origin server. Equivalent to ServerAndNoCache.
ServerAndNoCache	Applies the settings of both Server and NoCache to indicate that the content is cached at the server but all others are explicitly denied the ability to cache the response.
ServerAndPrivate	Indicates that the response is cached at the server and at the client but nowhere else. Proxy servers are not allowed to cache the response.

Let's go back to solve these problems.
1. Media files are not getting cached?
    - Use public or private depending on your need, described above.

2. Caching is on from Sitecore, still cache is not getting generated on device or browser.
    - Chances are use of any proxy before reaching to you. Check, your settings might be set as Private. Set it as Public.

Upload Sitecore media items in selected languages

One day, one idea came in my mind to create media items in specific languages only, which is very useful thought. By default, Sitecore media item gets created in many languages, where we need actually one or two out of them. Want to know why? Then read why sitecore media item gets created in multiple languages.

NOTE: This issues was recreated in Sitecore 7.2 and older versions. Sitecore 8 and later creates media items only in context language.

Specific languages for Unversioned Media Item

How it's benefitial?

In Unversioned media item, item gets created in all languages exist under /sitecore/system/languages/. Creating its version in selected languages is benefitial:

1. Less insert queries to DB tables, less records fetched while fetching media item details.
2. DataCache and ItemCache will be created in selected languages only instead of all languages.
3. Reduced Database size, where millions of media are stored.
4. So, overall performance improvement while uploading media item and while using/fetching media item.

How to achieve it?

1. Create your custom class inherited from the Sitecore.Resources.Media.MediaCreator one, say SitecoreTactics.Resources.Media.MediaCreator. In this class you should override CreateItem method to retrieve there the set of the languages from the web.config setting. This set should be used as the languageArray in the foreach cycle. Thus the unversioned mediaitem with certain language versions will be created.
   
   See how it looks:

   namespace SitecoreTactics.Resources.Media
   {
    using System.Reflection;
    public class MediaCreator : Sitecore.Resources.Media.MediaCreator
    {
     protected override Item CreateItem(string itemPath, string filePath, MediaCreatorOptions options)
     {
      // Override the CreateItem function method and write custom code to create selected language versions
     }
     }
   }
   
   

2. Create your custom class inherited from Sitecore.Resources.Media.MediaProvider one. In the constructor of this class you should assign an instance of our custom MediaCreator class (from Step#1) to the MediaProvider "creator" private field using the Reflection. So, the MediaProvider will use our own code of MediaCreator.
   
   How the code looks:

    namespace SitecoreTactics.Resources.Media
    {
     using System.Reflection;
     public class MediaProvider : Sitecore.Resources.Media.MediaProvider
     {
      public MediaProvider() : base()
      {
       typeof(Sitecore.Resources.Media.MediaProvider).GetField("creator", BindingFlags.NonPublic | BindingFlags.Instance).SetValue(this, new SitecoreTactics.Resources.Media.MediaCreator());
      }
     }
    }
   
   

3. Change in config, register our custom mediaprovider class as below:

   <mediaLibrary>
       ...
       <mediaProvider type="SitecoreTactics.Resources.Media.CustomMediaProvider,SitecoreTactics.Resources" />
       ...
   </mediaLibrary>
   
   

Wow, it worked well!

Specific languages for Versioned Media Item

Versioned Media Item does not need an automated way to upload media in selected languages. The reason is, this kind of media does not share Media File. So, it is used only when we need to upload different files for different languages of same media item. If still someone need to achieve it, here is the way:

1. Add a settings in the web.config file containing set of the pipe separated needed languages. For example:

   <setting name="SetOfLanguages" value="en|ru-RU|en-GB" />
   

2. Add an extra checkbox "Versionable For Certain Languages" on the Media Folder Upload Form using this file: \Website\sitecore\shell\Applications\Media\MediaFolder\MediaFolder.xml
3. Create a custom class inherited from Sitecore.Pipelines.Upload.UploadArgs and add here an extra "SpecialVersioned" property. This property will indicate to Sitecore.Pipelines.Upload.Save processor to execute the custom actions.
4. Create a custom class based on Sitecore.Shell.Applications.FlashUpload.Advanced.UploadTarget one and define it in \Website\sitecore\shell\Applications\FlashUpload\Advanced\UploadTarget.aspx file instead of the default one.
5. In this class you should assign a value to your custom UploadArgs.SpecialVersioned property similar to the UploadArgs.Versioned one.
6. Create your custom class based on the Sitecore.Pipelines.Upload.Save one and register it in the web.config <uiUpload> pipeline instead of the default one.
7. An example of its rewritten Process method is here: Expand Code

   public void Process(Sitecore.Support.Pipelines.Upload.UploadArgs args)
   {
    Assert.ArgumentNotNull(args, "args");
    for (int i = 0; i < args.Files.Count; i++)
    {
     HttpPostedFile file = args.Files[i];
     if (!string.IsNullOrEmpty(file.FileName))
     {
      try
      {
       bool flag = UploadProcessor.IsUnpack(args, file);
       if (args.FileOnly)
       {
        if (flag)
        {
         UnpackToFile(args, file);
        }
        else
        {
         string str = this.UploadToFile(args, file);
         if (i == 0)
         {
          args.Properties["filename"] = str;
         }
        }
       }
       else
       {
        MediaUploader uploader2 = new MediaUploader
        {
         File = file,
         Unpack = flag,
         Folder = args.Folder,
         Versioned = args.Versioned,
         Language = args.Language,
         AlternateText = args.GetFileParameter(file.FileName, "alt"),
         Overwrite = args.Overwrite,
         FileBased = args.Destination == UploadDestination.File
        };
        if (args.SpecialVersioned)
        {
         uploader2.Versioned = true;
        }
        List<MediaUploadResult> list = uploader2.Upload();
        Log.Audit(this, "Upload: {0}", new string[] { file.FileName });
        if (args.SpecialVersioned)
        {
         string[] setOfLanguages = MainUtil.RemoveEmptyStrings(Settings.GetSetting("SetOfLanguages", "en").ToLowerInvariant().Split(new char[] { '|' }));
         var languagesForProcessing = new List<Globalization.Language>();
         foreach (var language in setOfLanguages)
         {
          var lang = Globalization.Language.Parse(language);
          languagesForProcessing.Add(lang);
         }
         foreach (MediaUploadResult result in list)
         {
          foreach (var language in languagesForProcessing)
          {
           Item langItem = result.Item.Database.GetItem(result.Item.ID, language);
           if (langItem.Versions.Count < 1)
           {
            langItem = langItem.Versions.AddVersion();
            var mediaItem = new MediaItem(langItem);
            using (new EditContext(mediaItem, SecurityCheck.Disable))
            {
             mediaItem.InnerItem["extension"] = FileUtil.GetExtension(file.FileName);
            }
            Media media = MediaManager.GetMedia(mediaItem);
            Assert.IsNotNull(media, typeof(Sitecore.Resources.Media.Media));
            media.SetStream(file.InputStream, FileUtil.GetExtension(file.FileName));
           }
          }
         }
        }
        foreach (MediaUploadResult result in list)
        {
         this.ProcessItem(args, result.Item, result.Path);
        }
       }
      }
      catch (Exception exception)
      {
       Log.Error("Could not save posted file: " + file.FileName, exception, this);
       throw;
      }
     }
    }
   }
   
   

It's working great, isn't it?
And yes, many thanks to Yuriy Zakharov from Sitecore Support to provide such a nice solution!!

Sitecore Custom HTTP Handlers

We recently had a requirement that we need thumbnail for PDFs stored in Sitecore, so whenever we need to find PDFs or other media files, we can find out them by its thumbnail. In starting we thought to create a thumbnail for each media file and then store it to another location in Sitecore tree itself while uploading media. It was a good idea, but the problem in this approach would come when a user updates the media? We need to create thumbnail again... Same thing when user deletes/moves/publishes, etc. it needed to generate thumbnail all the time.

Finally we came to a perfect solution of using Custom HTTP Handler, using which we can fulfill this requirement easily.

Overview of Sitecore Custom Handlers?

Search for in your web.config. These are the custom handlers called by Sitecore to serve requests of media, xaml, icon, feed, etc. See below block in web.config:

    
      <handler handler="sitecore_media.ashx" trigger="~/media/" />
      <handler handler="sitecore_api.ashx" trigger="~/api/" />
      <handler handler="sitecore_xaml.ashx" trigger="~/xaml/" />
      <handler handler="sitecore_icon.ashx" trigger="~/icon/" />
      <handler handler="sitecore_feed.ashx" trigger="~/feed/" />
    

Sitecore.Pipelines.HttpRequest.CustomHandlers processor from httpBeginRequest dispatches requests to specialized handlers that are required. These handlers are defined in web.config under configuration/sitecore/customHandlers.

If the current request ends with the a handler’s file (e.g., sitecore_media.ashx), then Sitecore aborts the current pipeline, since this request is specifically targeting a handler directly, which are defined in system.webServer/handlers section as below, which shows that the media request will be served through Sitecore.Resources.Media.MediaRequestHandler. See below block of web.config:

   <handlers>
      <add name="Sitecore.MediaRequestHandler" path="sitecore_media.ashx"  type="Sitecore.Resources.Media.MediaRequestHandler, Sitecore.Kernel" verb="*" />
   </handlers>

Create own Sitecore Custom Handlers to generate thumbnails of media items

Now, we created a new custom handler for creating thumbnail and registered in handlers as below:

    
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>

    <!-- Define Custom Handler -->
    <customHandlers>
      <handler trigger="~/mediathumb/" handler="sitecore_media_thumb.ashx"  />
    </customHandlers>

    <!-- Define Media Prefix -->
    <mediaLibrary>
      <mediaPrefixes>
        <prefix value="~/mediathumb" />
      </mediaPrefixes>
    </mediaLibrary>
  </sitecore>


  <!-- Define Web Handler -->
  <system.webServer>
 <handlers>
     <add verb="*" path="sitecore_media_thumb.ashx" type="SitecoreTactics.ThumbnailManager.PDFThumbnailRequestHandler, SitecoreTactics.ThumbnailManager" name="SitecoreTactics.PDFThumbnailRequestHandler"/>
 </handlers>
  </system.webServer>
</configuration>    

This class will create a thumbnail of the request PDF on-the-fly using GhostScript. We will save this thumbnail as Media Cache, so on the next request, the thumbnail will not be created, but will be served from Media Cache itself, same as Sitecore serves media items.

In this case suppose, the PDF we uploaded has URL:
http://mydomain.com/~/media/files/Docs/mypdf.pdf.
Now, we can get its thumbnail as JPG file using URL, which will be served by our new handler because it contains /~/mediathumb trigger.
http://mydomain.com/~/mediathumb/files/Docs/mypdf.jpg.

Finally, we do not need to take care if media item is moved, deleted, published or updated. I will attach its source code very soon.... :)

What else we can do using Custom Handlers?

- In above approach, we can also generate thumbnail of different size, dimensions passed through querystring, same way Sitecore provides it for images. Read how we can Resize Sitecore Media Images on the fly.
- Even Custom Handlers can be useful to serve different kind of resources like creating sitemap (/~/sitemap/), RSS for page or its children (/~/rss/), stylesheets (/~/css/), javascripts (/~/js/), etc.

Related Posts:
- PDF Thumbnail Handler
- Show PDF Thumbnail Icons in Content Editor

Sitecore media url - remove ashx extension

Have you ever marked all your media items are rendered in our page output as .ashx extension? By default, Sitecore renders all media files or images (.jpg, .gif, .pdf, .xls, etc.) as .ashx extension.

The same thing you will find while getting mediaurl for your media items.

You can render media with its own extension using Media.RequestExtension setting in web.config.
It defines the extension to use in media request URLs. It's default its value is ashx, so all media url are .ashx.
So, media's URL will be: http://mydomain.com/~/media/Images/Banner/image.ashx

<setting name="Media.RequestExtension" value="" />

Setting its value to blank will render media url with relevant extension. Now, all your media items will be rendered in page output with its valid extension like, http://mydomain.com/~/media/Images/Banner/image.jpg
Still, your existing URL http://mydomain.com/~/media/Images/Banner/image.ashx will also work.

Image url can have different querystring parameters. Read more for Image Control and QueryString parameters.

Why my Sitecore Media Item created in multiple languages?

Have you ever faced that you upload a single media file to Sitecore and it gets created in multiple languages? This is not strange but expected behavior in Sitecore. Let's see how.

Check Media.UploadAsVersionableByDefault setting in your web.config. If it is false, then uploaded media item will be created in all those languages which exist under /sitecore/system/languages. This media item will use unversioned template /sitecore/templates/System/Media/Unversioned/File template, so the Blob(Media) and its common details would be shared across all language versions. See below image.



This option is really helpful when you want to use common media files across multiple languages content pages as the media is shaerd across all languages.


Where, if you have this setting as true, then each media item will be generated in default language only. This media item will use versioned template /sitecore/templates/System/Media/Versioned/File template, so the Blob(Media) and its all details would be distinct across languages. See below image.



This option is really helpful when you want to use different media files for different language content pages.

We can choose whether we need versioned or unversioned media. While doing Upload File (Advanced), Sitecore asks to select Make Uploaded Media Items Versionable checkbox. If the checkbox is selected, it will create versioned media, otherwise will create unversioned media items. See below image:



You want to upload media items in Sitecore with your selected languages only? Refer my new post: Upload sitecore media items in selected languages.

Sitecore Image Parameters and Image Control

This post is for those who are still:

- Creating duplicate Image Items on Sitecore for achieving Responsive Web Design or Image Gallery.
- Creating separate images for desktops, phones and tablets to achieve responsive web design.
- Creating Thumbnails, Preview and original images for image galleries.
- Resize images on-the-flyas per requirement.

Tired from maintaining multiple items(create, update, delete, publish) of each image?

There is a short and sweet solution within Sitecore itself to give freedom from above headache, that is Sitecore Image Parameters and Sitecore Image Control.

How it is beneficial?

- Using it, each requested image is created/scaled and cached on disk by Sitecore itself, so it does not impact on performance.
- This gives freedom from multiple uploads/updates/publish of same image and multiple Item Cache
- Saves lots of human efforts and time
- Freedom from, reduction in database size, beneficial from maintenance and point-of-view too.

Different Image Parameters:

w

Width of image

h

Height of image

mw

Maximum width of image

mh

Maximum height of image

iar

Ignore Aspect Ratio. Value should be 1 or 0.

as

Allow Stretch. Value should be 1 or 0.

thn

Create Thumbnail. Value should be 1 or 0.

bc

Background Color (When there is no aspect ratio set)

sc

Scale Image. 1 is default value.

Few Samples:

Expected Result	Sitecore Image Control	Image URL
Original	<sc:Image Field="My Image" />	http://com.com/~/media/myimage.jpg
Width=150	<sc:image field="My Image" width="150" />	http://com.com/~/media/myimage.jpg?w=150
Height=200	<sc:image field="My Image" height="200" />	http://com.com/~/media/myimage.jpg?h=200
Height=200 Width=200 Ignore Aspect Ratio	<sc:image field="My Image" width="200" height="200" iar="true" />	http://com.com/~/media/myimage.jpg?h=200&w=200&iar=true

You can findout all parameters for image control from:
http://sdn.sitecore.net/Articles/XSL/5%203%20Enhancements/Image%20Enhancements.aspx

Now, you will think that how can we achieve adaptive images using theImage control. Well, for that we have two approaches:

1. Use Sitecore Adaptive Images module.
2. Use JS plugins like responsejs. This is more preferable approach.
   
   This plugin needs all images to be rendered as below:
   
   
   Means, the the control we create, should render image tag supporting different size attribute and value urls.
   
   Here, if plugin finds the device screen is suitable to 330 width, it will update image's source to data-330's value. Similarly to data-961 when it finds device's screen suitable to 961 width.
   
   For this, we can extend Sitecore's Image Control to achieve this image rendering.

Enjoy!!