Sitecore Parallel Publishing Implementation

Many readers of my blogs asked me the practical way to implementing parallel publishing using multiple publish queue on Sitecore. Earlier I have posted a blog on it explaining theoretical approach for Parallel Publishing in Sitecore using custom EventQueue. It is really difficult to explain that approach on a blog. So here, I am going to explain easiest and quickest implementation of multiple publish queue practically.

Why Parallel Publishing from different instances?

In our multisite environment, hundreds of users work at a time, add/update thousands of items and publish them frequently. Daily 15,000+ items get published. Sometimes this number crosses 50,000 per day. So, mostly we see long publishing queue and users have to wait for minutes to get their items published. To make publishing more scalable, we thought to have multiple publishing instances in our CM environment and we dedicated them for different types of users based on business need.

What we wanted to achieve:

We wanted to have three different Publishing Instances for our users.
Publish Instance - 1(PI-1) Dedicated to Client Content Authors.
Publish Instance - 2(PI-2) Dedicated to Internal Content Authors.
Publish Instance - 3(CM-1) Dedicated to all other users. This is the Content Management Server, where users use Page Editor or Content Editor.

For this, we created three different roles named Publish Instance One, Publish Instance Two and Content Management Instance. So, depending on role assigned, we will allocate Publishing Instance to the user.

How we hooked Publishing Process

Below image shows how we hooked and modified startPublishing event to choose Publish Instance depending on user publishes.

Let's see practical implementation of Parallel Publish in Sitecore?

You can see How Parallel Publishing works on Sitecore in below video.
In this video, below is the allocation done for users:

User	Role	Allocated Publishing Instance
sitecore\admin	Publish Instance One	Machine-PI-1
sitecore\yogesh	Publish Instance Two	Machine-PI-2
All other users	Content Management Instance	Machine-CM

So, we have three different Publishing Queues available.

Now see how we can achieve this in simple steps below:

1. Setup three different CM instances (in a cluster) with same master, web and core databases.
Parallel publishing requires 1+ Sitecore Instances So indirectly we can say parallel publishing is possible on n number of CM instances.


2. Enable EventQueue on each instance
On all the Sitecore Instances, it is must to enable EventQueue from web.config or App_config\Include\ScalabilitySettings.config file.

     <setting name="EnableEventQueues" value="true" />
   

3. Configure allocation of Roles & Publishing Instances
On all the Sitecore Instances, create a config file: App_config\Include\ParallelPublish.config file. This file should remain same on each instance.

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <parallelpublishRoles>
      <!-- Roles needed for Parallel Publising -->
      <role name="sitecore\Publish Instance One" publishinstance="Machine-Sitecore-PI-1" />
      <role name="sitecore\Publish Instance Two" publishinstance="Machine-Sitecore-PI-2" />
      <role name="sitecore\Content Management Instance" publishinstance="Machine-Sitecore-CM" />
    </parallelpublishRoles>
  </sitecore>
</configuration>
   

4. Override publish:startPublishing Event with your custom code as below

namespace SitecoreTactics.Publishing
{
    public class RemotePublishingEventHandler : Sitecore.Publishing.RemotePublishingEventHandler
    {
        public override void OnStartPublishing(object sender, EventArgs args)
        {
            RemoteEventArgs<Sitecore.Publishing.StartPublishingRemoteEvent> args2 = args as RemoteEventArgs<Sitecore.Publishing.StartPublishingRemoteEvent>;
            if (args2 == null)
            {
                throw new InvalidOperationException("Unexpected event args: " + args.GetType().FullName);
            }

            Sitecore.Globalization.Language lang = Sitecore.Globalization.Language.Parse(args2.Event.ClientLanguage);
            StartPublishingRemoteEvent myArgs = new StartPublishingRemoteEvent(args2.Event.Options, args2.Event.StatusHandle, args2.Event.UserName, lang);

            // Findout Dedicated Publishing Instance for current user
            string allocatedPublishInstance;
            using (new SecurityDisabler())
            {
                User user = User.FromName(args2.Event.UserName, true);

                allocatedPublishInstance = GetPublishInstanceForUser(user);
            }

            // Allocate Publishing Instance to current publishing
            myArgs.SetPublishInstance(allocatedPublishInstance);

            // Allocated Publishing Instance will pick up the publishing
            if (this.IsPublishingServer(myArgs))
            {
                Sitecore.Publishing.DistributedPublishingManager.StartPublishing(myArgs);
            }
        }

        protected string GetPublishInstanceForUser(User user)
        {
            // Find role and allocated publishing instance
            foreach (XmlNode node in Factory.GetConfigNodes("parallelpublishRoles/role"))
            {
                string role = XmlUtil.GetAttribute("name", node);
                string publishInstance = XmlUtil.GetAttribute("publishinstance", node);
                if (user.IsInRole(role))
                    return publishInstance;
            }

            // Return default Publish Instance
            return Settings.Publishing.PublishingInstance;
        }
    }


    public class StartPublishingRemoteEvent : Sitecore.Publishing.StartPublishingRemoteEvent
    {
        public StartPublishingRemoteEvent(Sitecore.Publishing.DistributedPublishOptions[] options, Sitecore.Handle statusHandle, string userName, Sitecore.Globalization.Language clientLanguage)
            : base(options, statusHandle, userName, clientLanguage)
        {
        }

        // Methods
        [Obsolete("Use StartPublishingRemoteEvent(DistributedPublishOptions[] options, Handle statusHandle, string userName, Language clientLanguage) instead.")]
        public StartPublishingRemoteEvent(Sitecore.Publishing.DistributedPublishOptions[] options, Sitecore.Handle statusHandle, string userName) :
            base(options, statusHandle, userName, Sitecore.Data.Managers.LanguageManager.DefaultLanguage)
        {
        }
        
        public void SetPublishInstance(string piName)
        {
            base.PublishingServer = piName;
        }
    }
}

5. Update your custom publish:startPublishing event in web.config with below changes

<event name="publish:startPublishing">
        <!-- handler type="Sitecore.Publishing.RemotePublishingEventHandler, Sitecore.Kernel" method="OnStartPublishing" / -->

      <handler type="SitecoreTactics.Publishing.RemotePublishingEventHandler, SitecoreTactics.Publishing" method="OnStartPublishing" />
        
      </event>

   

Now try in your environment, parallel publishing is surely working for you too. If it's not working, check

• EventQueue is disabled, enable it
• All CM instance should have exact time. If two instances have time difference of 5 minutes, then there the instance running late will get updates of other instance after 5 minutes, so sync will never be done between them.
• Give proper Roles Name and Publishing Instance Name in the ParallelPublish.config file.
• Check all your Sitecore instances are pointing to same databases - master, web, core.
• Configure the publish:startPublishing event and ParallelPublish.config on each Sitecore Instance.
• ParallelPublish.config file should have same contents on all the instances.
• Instance which needs to do publish should be up and running.

Now enjoy parallel publishing without waiting time, without queue stuck!