Dot or Space in URL gives 404 Error - Fixed

Our application security certification program raised a security issue on our Sitecore Staging and Live environment. "Any URL where folder or file name ends with Space or Dot gives 404 - The resource cannot be found error."

For example, accessing URLs like
- http://sitecore72/sitecore%20/login
or
- http://sitecore72/sitecore./login

Both gives below error. Same, if we do same kind of request on any live site, it gives same error like for URL: http://sitecore/CMS /Sitecore/

Tricks we already applied, what failed:

• Tried to catch this error from Application_Error event from Global.asax, but no luck found. 
• We also tried to handle it from web.config using CustomErrors, but again no luck.

Solution we found at last:

After checking many ASP.NET sites, including Microsoft website, this issue was found on majority of sites. For example, http://www.microsoft.com/security /default.aspx, it gives same error. So finally we concluded that this bug is not from Sitecore side, but it's something what .NET is playing with.

After reading more on net, found below setting. The value of relaxedUrlToFileSystemMapping attribute should be true to solve this issue, by default its value is false in ASP.NET and Sitecore:

    <httpruntime relaxedurltofilesystemmapping="true" />

Now, after applying this fix, the above URL on Staging environment sends user to NotFound error page and on live environment, it opens a valid page by ignoring Dot or Space.

What relaxedUrlToFileSystemMapping attribute does?

It indicates whether the URL in an HTTP request is required to be a valid Windows file path. It determines how the URL in an incoming HTTP request will be validated.

If this property is false, the URL is validated by using the same rules that determine whether a Windows file system path is valid.
If it is true, it will not validate any folder or file name.


Note: Later on we found that Sitecore has solved this bug from Sitecore 7.2 Update-2 by changing same attribute, they have not mentioned this bugfix in their release notes.

Block files by MIME content type while Sitecore upload

As per Web Security best practices, while media upload in Sitecore, we should block upload of EXE, DLL, BAT, ASPX, ASP, etc. files on server. Do you think, it is enough? I think, No.

We should also block files by checking their MIME content types because someone can also upload Exe/Dll files by renaming them as Jpg or any other extension is allowed. So, this can be a serious threat too.

So, checking MIME content type is equal important as checking file extensions.

Why checking only file extension is not enough?

We implemented a module to restrict certain extensions, provided by Yuriy Yurkovsky from Sitecore Support, Prevent files from being uploaded which is working absolutely fine. Michael Reynolds also nicely presented restricting file extensions on his post Restrict Certain Extensions From Being Uploaded.

Later on, while testing for security threats, we found two issues while implementing blocking extensions.Thanks to our QA Analyst Chirag Patel for finding such nice scenarios and also shown us how it is harmful.

1. What if I upload file as "setup. EXE" instead of "setup.EXE"? (Just add a space after dot)
2. What if I upload file my EXE file by renaming as JPG? (Setup.JPG instead of Setup.EXE)

Yes, in both cases we were able to upload EXE contents which should be blocked by us. See below image, how EXE file uploaded as JPG behaves when client requests. This can be a serious threat to our application.

For case 1, we updated the code given in above module by removing the space between dot and file extension.
For case 2, we can use below approach.

How to restrict upload of certain MIME content types

As per the case 2, users can upload EXE  files by renaming them as JPG file. So, we can block them by their content type. Let's see how we can block content types, which is equal important as blocking files by extensions.

Below can be the patch configuration file, for better understanding, I used same format as Michael Reynolds' post to restrict extensions:

Here, two kind of content types are blocked:
- application/octet-stream (Used for bin, dms, lha, lzh, exe, dll contents)
- application/zip (Used for zip content)
 

<?xml version="1.0" encoding="utf-8"?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <processors>
      <uiUpload>
        <processor mode="on" type="SitecoreTactics.Pipelines.Upload.CheckForRestrictedContentType, SitecoreTactics" patch:before="processor[@type='Sitecore.Pipelines.Upload.CheckSize, Sitecore.Kernel']">
          <restrictedcontentTypes hint="raw:AddRestrictedContentType">
            <!-- content types to restrict -->
            <contentType>application/octet-stream</contentType>
            <contentType>application/zip</contentType>
        </restrictedcontentTypes>
        </processor>
      </uiUpload>
    </processors>
  </sitecore>
</configuration>

You can get more content types from:
http://www.freeformatter.com/mime-types-list.html
http://www.dailycoding.com/Posts/mime_contenttypes_with_file_extension.aspx


Below can be the source code to block certain content types defined in above config file.

namespace SitecoreTactics.Pipelines.Upload
{
    public class CheckForRestrictedContentType : UploadProcessor
    {
        private List<string> _RestrictedContentType;
        private List<string> RestrictedContentType
        {
            get
            {
                if (_RestrictedContentType == null)
                {
                    _RestrictedContentType = new List<string>();
                }

                return _RestrictedContentType;
            }
        }

        public void Process(UploadArgs args)
        {
            foreach (string fileKey in args.Files)
            {
                string fileName = args.Files[fileKey].FileName;
                string contentType = args.Files[fileKey].ContentType;

                if (IsRestrictedContentType(contentType))
                {
                    args.ErrorText = Translate.Text(string.Format("The file \"{0}\" cannot be uploaded. Files with an content Type of {1} are not allowed.", fileName, contentType));
                    Log.Warn(args.ErrorText, this);
                    args.AbortPipeline();
                }
            }
        }


        private bool IsRestrictedContentType(string contentType)
        {
            return RestrictedContentType.Exists(restrictedContentType => string.Equals(restrictedContentType, contentType, StringComparison.CurrentCultureIgnoreCase));
        }

        protected virtual void AddRestrictedContentType(XmlNode configNode)
        {
            if (configNode == null || string.IsNullOrEmpty(configNode.InnerText))
            {
                return;
            }

            RestrictedContentType.Add(configNode.InnerText);
        }
    }
}

I feel, now my Sitecore application is more secured!