Sitecore Multiple Publish Queue (Parallel Publishing)


Are you facing a long long queue stuck up with Sitecore publishing and clients are complaining a lot? Then surely you might have think about having multiple publish queue.

See my other post regarding Practical Implementation of Sitecore Parallel or simultaneous Publishing using multiple Publish Queue.

Benefits of Parallel Publishing:

  1. More than one user can do publishing simultaneously, so no more "Publishing Queued".
  2. Less load on CM environment, where actually users are editing or previewing pages as publishing load is now shared with Publish Instances.
  3. Less chances of Publish Queue stuck-up as Publish Jobs threads are now divided between multiple instances.
  4. If one publish instance is stuck-up, then we can move publishing traffic to another instance.

What is expected?

The expectation is to do parallel publishing on two or more Sitecore Instances. Say, our Content Authors' publish should go to PI-1 and clients' publish should go to PI-2. Or, Content Editor publish should go to PI-1 and Page Editor publish should go to PI-2.




This idea is quite possible and easier too. Let's see how can we use multiple publish queue. For that, you might need to understand How Sitecore Publish Instance works.

What is the Challenge?

As we see from the above link, Sitecore allows to add only one PublishInstance name, as per below setting so publishing will be done from that server itself.
        <setting name="Publishing.PublishingInstance">
           <patch:attribute name="value">CM-PI-1</patch:attribute>
        </setting>
So, the problem is well described in below image. See the red balloon points.
Ohhhh.... how to come out from this challenges, as Sitecore does not support this? Well, we have an alternate to overcome this issue.

Solution to Setup Multiple Publish Instances

As we seen, when Sitecore CM sets publish, it is recorded in EventQueue and then the related PI will pickup that publish and start publishing. Can't we do the same process by bypassing EventQueue? I mean we can prepare our own Event Queue.

So, that's really possible now. Also, don't worry about caching updates, means items updated/published on CM or PI-1 or PI-2 will get reflected on other servers immediately using Event Queue. So, our main concern should be publishing only.

There can be two different approaches to achieve this:
  1. Creating own table same as EventQueue
  2. By Invoking Web Service to start Publish

1. Creating own table same as EventQueue

  • We can create own table, which can have fields like below:

    Fields in MyPublishQueue
    ID
    ItemId
    SubItems
    Languages
    PublishType
    PublishInstance

    So, when user sets publish from CM, we will not start publish. We will just make entries in above table, with appropriate Publish Instance Name.
  • Now, both PI will have a our new Sitecore scheduler running. This scheduler should be triggered every few time interval as per our need, same as EventQueue checks. Write down some code to find any publishes for their own instance, and starts publish there itself. Refer Alex Shyba's blog to Create Sitecore Scheduler.

Below image shows how to achieve this approach. Here, as what we expect, Content Editor publish should go to PI-1 and Page Editor publish should go to PI-2, which is maintained by our Custom EventQueue table.

Parallel Sitecore Publish using custom EventQueue

This approach is very useful when we use for Scheduled Publish.

2. By Invoking Web Service to Start Publish

We can create a web service on both PIs, with input parameters as below:
  • ItemId
  • SubItems
  • Languages
  • PublishType (Smart/Republish)
This web service will do publish according to the passed items. So, when user sets publish from CM for PI-1 or PI-2, web service of respective PI should be invoked. So, publishing will be started by the webservice itself.
Below image shows how to achieve this approach.

Parallel Publish using WebService


This approach is very useful when we are not using Scheduled Publish.

Finally, you might be knowing how to start publishing using Sitecore. If No? See the code below:
public void PublishItem(Sitecore.Data.Items.Item item, User user)
{
 using(UserSwitcher(user))
 {
  // The publishOptions determine the source and target database,
  // the publish mode and language, and the publish date
  Sitecore.Publishing.PublishOptions publishOptions =
  new Sitecore.Publishing.PublishOptions(item.Database,
              Database.GetDatabase("web"),
              Sitecore.Publishing.PublishMode.SingleItem,
              item.Language,
              System.DateTime.Now);  // Create a publisher with the publishoptions
  Sitecore.Publishing.Publisher publisher = new Sitecore.Publishing.Publisher(publishOptions);

  // Choose where to publish from
  publisher.Options.RootItem = item;

  // Publish children as well?
  publisher.Options.Deep = true;

  // Do the publish!
  publisher.Publish();
 }
}

Note: Using multiple Publish Instance can be done with 3 Sitecore instances CM, PI-1 & PI-2. We can also do parallel publish with total two instances too CM & PI-1.

Sitecore does not recommend to have Multiple Publishing Instances. But, using above approaches, we implemented Parallel Publishing in 2012 with Sitecore 6.4.1, published millions of items with parallel publish, and still counting.

No need to say but having multiple Sitecore instances requires license accordingly.


No words to say about great Sitecore Architecture!!

Must read posts for Sitecore Publish

- Sitecore Publishing Facts
- Intelligent Publish in Sitecore - The most optimized approach
- Sitecore Setup Separate Publish Instance

Site-wise languages in Sitecore Ribbon for multisite environment

Are you working in Sitecore multisite environment and different sites are built in different languages? Then one idea should come in your mind that can't we set site-wise languages to the Page Editor's Webedit Ribbon and Content Editor's Language Gallery Form.

By default, the ribbon shows all languages defined in /sitecore/system/Languages. In our Sitecore instance, there are 40+ languages and many different culture sites. So, this idea came in our mind and implemented.

See below snap, there are total 5 languages in System.

Default Languages in Content Editor


Default Languages in Page Editor


Expected Behavior

Now, suppose, your site A uses 2 languages and site B uses other 3 languages. Then selecting item of site A, it should show 2 languages and same for site B, it should show 3 languages.

In this example, the site is built in 3 languages: En, hi-IN, de-DE. So, the ribbon should not show other languages en-GB and fr-FR. See below snap what is expected.

Solution

Prepare template to choose site-wise languages

We can have a multilist field to the Home Item of each site, where we can choose site's languages from the whole list from /sitecore/system/languages. See below screen.


Change Languages in Content Editor Gallery

Open file: \Website\sitecore\shell\Applications\Content Manager\Galleries\Languages\Gallery Languages.xml Change the code beside to your custom code. SO, update the file as below:
     <!-- comment below line -->
     <!--<CodeBeside Type="Sitecore.Shell.Applications.ContentManager.Galleries.Languages.GalleryLanguagesForm,Sitecore.Client"/>-->

     <!-- add below line -->
     <CodeBeside Type="SitecoreTactics.GalleryLanguagesForm, SitecoreTactics"/>
Code in the CodeBeside file: Create a custom class, which will be a repilca of the Sitecore.Shell.Applications.ContentManager.Galleries.Languages.GalleryLanguagesForm class. We just need to override OnLoad event in this class. So, this will be look like..
namespace SitecoreTactics
{
 public class GalleryLanguagesForm : Sitecore.Shell.Applications.ContentManager.Galleries.GalleryForm
 {
  // Other methods

   protected override void OnLoad(EventArgs e)
  {
   Assert.ArgumentNotNull(e, "e");
   base.OnLoad(e);
   if (!Context.ClientPage.IsEvent)
   {
    Item currentItem = GetCurrentItem();
    if (currentItem != null)
    {
     // Comment below line from the code
     //foreach (Language language in currentItem.Languages)

     // Instead of all languages, we will pass only selected languages
     LanguageCollection languages  = GetMySelectedLanguages();
     foreach (Language language in languages)
     {
      // Same code which is in foreach block
     }
    }
   }
  }
  
  // Other methods

 }
}

Download above sourcecode

Here is the expected output:

Change languages in Page Editor

For this, we need to update the command from Commands.config file as below:
     <!-- comment below line -->
     <!--<command name="webedit:changelanguage" type="Sitecore.Shell.Applications.WebEdit.Commands.ChangeLanguage,Sitecore.Client"/>-->


     <!-- add below line -->
     


Code in the class file: Create a custom class, which will be a repilca of the Sitecore.Shell.Applications.WebEdit.Commands.ChangeLanguage class. We just need to overrideExecute event in this class. So, this will be look like..
namespace SitecoreTactics
{
 public class ChangeLanguage : WebEditCommand
 {
  // Other methods
 
  public override void Execute(CommandContext context)
  {
   Assert.ArgumentNotNull(context, "context");
   if (context.Items.Length == 1)
   {
    Item item = context.Items[0];
    
    // Comment below line
    //LanguageCollection languages = LanguageManager.GetLanguages(item.Database);
    
    // Instead of all languages, we will pass only selected languages
    LanguageCollection languages  = GetMySelectedLanguages();
    
    SheerResponse.DisableOutput();
    Menu control = new Menu();
    foreach (Language language in languages)
    {
     string id = "L" + ShortID.NewId();
     string languageName = GetLanguageName(language.CultureInfo);
     string icon = LanguageService.GetIcon(language, item.Database);
     string click = "webedit:setlanguage(language=" + language.Name + ")";
     control.Add(id, languageName, icon, string.Empty, click, false, string.Empty, MenuItemType.Normal);
    }
    SheerResponse.EnableOutput();
    SheerResponse.ShowPopup("ChangeLanguageButton", "below", control);
   }
  }
  
  // Other methods
 }
}
Here is the expected output:


Cheers, finally we achieved site-wise selected languages for each site for our multisite Sitecore environment.

Other Language specific customizations - used for multisite environment:
Why my Sitecore Media Item created in multiple languages?
Upload Sitecore media items in selected languages

Hostname Resolution without editing Host File

Have you ever thought to map hostname without changing host file? If your Sitecore or any other website is working with Load Balanced environment with multiple servers, you surely have thought to point your request to a particular server (IP Address). When you need to check your functionality working/not working on any of these servers, you are surely going to make host entry for individual servers one by one and test the site.

Uffff... it's so boring and lengthy task and chances of human mistakes. Let's see how we can automate this.

Traditional way for Host Name Resolution Using a Hosts File

Find Hosts file in the %SystemRoot%\system32\drivers\etc directory. Read more about Host Name Resolution. Following is an example of the contents of the Hosts file:
#
# Table of IP addresses and host names
#

127.0.0.1 localhost
192.168.220.111 www.patelyogesh.in # Server-1
#192.168.220.112 www.patelyogesh.in # Server-2

How we can bypass hosts file for host name resolution

Traditional way to get response using WebRequest is requesting to http://www.patelyogesh.in but this will get content from any of the servers which Load Balancer choose for our request.

We can get content from particular server by requesting to: http://<IP Address>/<Page> and passing host/domain name as request.Host as below:
      
var request = (HttpWebRequest)WebRequest.Create("http://<IP Address>/<Page>");
request.Host = HostName;
Let us see how whole source code / GetResult function will be to achieve this:
      string strIPAddress = "192.168.220.111";
      string strHostName = "www.patelyogesh.in";

      string result = GetResult(strIPAddress, strHostName);

      public string GetResult(string strIPAddress, string strHostName)
      {
            var request = (HttpWebRequest)WebRequest.Create("http://" + strIPAddress);
            request.Host = strHostName;

            request.Credentials = CredentialCache.DefaultCredentials;
            HttpWebResponse response = (HttpWebResponse)request.GetResponse();

            // Get the stream containing content returned by the server.
            Stream dataStream = response.GetResponseStream();

            // Open the stream using a StreamReader for easy access.
            StreamReader reader = new StreamReader(dataStream);

            // Read the content. 
            string responseFromServer = reader.ReadToEnd();

            // Cleanup the streams and the response.
            reader.Close();
            dataStream.Close();
            response.Close();
            return responseFromServer;
      }
Again, www.patelyogesh.in is mapping to 192.168.220.111. Now suppose we want to request to http://www.patelyogesh.in/AboutUs.html page, we can pass parameters as:

var request = (HttpWebRequest)WebRequest.Create("http://192.168.220.111/AboutUs.html");
request.Host = "www.patelyogesh.in";
Thus we can access all the pages using C# .NET 4.5 bypassing host file using above source code.

Hope you like this idea! Cheers!!

Sitecore bypass login without password

This blog explains how to bypass login for any Sitecore user or how to implement Single Sign-On on Sitecore to automate user login without knowing password.

Practical Use

Suppose, you have a non-Sitecore application, which has different users and all its users also access Sitecore by login into it. Now, switching between these two sites is really repetitive and time-consuming task, need to remember passwords for two websites, etc. If we provide SSO (Single Sign-On) here so that the user logs in only once and access both Sitecore and non-Sitecore websites with single login.

How to achieve

Create a web page in Sitecore website suppose sso.aspx, that will do login in Sitecore with passed userame via querystring. After login, we have to redirect user to welcome.aspx.

So, from the non-sitecore application, we will call url: http://mysitecore.com/sso.aspx?username=user_sitecore. On load of the page, we will login using user_sitecore and redirect user to welcome.aspx.

But make sure you encrypt username before passing in querystring. You can also pass a unique token number (A random generated number/string) along with username and encrypt it to make it more secured.

Login without password is possible using Sitecore.Security.Authentication.AuthenticationManager.Login(username) function. See below source code:

public partial class SSO : System.Web.UI.Page
{
    protected void Page_Load(object sender, EventArgs e)
    {
         string strUserName = Request.QueryString["username"];
         if (Sitecore.Security.Authentication.AuthenticationManager.Login(strUserName))
         {
               Response.Redirect("welcome.aspx", true);
         }
    }
}

Hurrey.... we did login without giving user password.

Sitecore Publish Instance

Are you not getting any direction to improve your Content Management server performance due to publishing? Are you facing slowness while doing publishing or having publishing queue stuck-up issues? Separate Publish Instance is one of the solution.

How to setup Sitecore Publish Instance?

  1. Sitecore PI is nothing but a clone instance of Sitecore CM environment.
  2. On CM server, open \Website\App_Config\Include\Scalability.config file. Update Publish Instance Name as below.
            <setting name="Publishing.PublishingInstance">
               <patch:attribute name="value">PI-Sitecore</patch:attribute>
            </setting>
    
    If the value is empty Sitecore will use the instance name by combining the machine name and the IIS Site name. So for the IIS Site Sitecore on the server PI the instance name would become PI-Sitecore.
    Just to remember, do not configure above setting on Publish Instance.
    Note: Sitecore allows to add only one publish instance name in this setting.
  3. Enable EventQueue on both CM and PublishInstance servers. To enable EventQueues, from web.config, find EnableEventQueues setting. Set its value to true. This setting can also be set from \App_Config\Include\ScalabilitySettings.config, which will be given more precedence over web.config settings.
          <setting name="EnableEventQueues">
            <patch:attribute name="value">true</patch:attribute>
          </setting>
    
  4. Disable any scheduled tasks or agents running if not required.

How Publishing works on Publish Instance?

Below image shows the flow of publishing from CM to PI server.
Sitecore Publish Instance Architecture


Suppose, Sitecore CM Application has Instance named Staging-Sitecore (In image: CM1), and Publish Instance is named PI-Sitecore (In image: CM-PI-1). Now, as per above settings, Sitecore CM1 knows that it's publishing should be done by Publish Instance: PI-Sitecore.

Now, when a user sets a publish job, Sitecore adds its entry in EventQueue table with a flag/column of PublishInstance. It simply adds PI-Name in that column's value. So, every publish job is assigned its Publish Instance Name. Now, as per EventQueue architecture every Sitecore Instance checks for EventQueue every few seconds to fetch which events/jobs it has to perform. Here, when CM1 server checks EventQueue, it will not find any publish job, where PI will find records with its name. So, it will fetch those publish details and start publishing those publish jobs.

Now, as per architecture, all other instances will start updating cache of those published and related items, to get updated.

Can we utilize Publish Instance for other tasks?

- PI is not only PI, which can still be used as CM. So, we can have two Content Management servers. So, we can make less used CM as PI.

- If we are using it only for Publishing, then we can reduce load of CM server by moving all backend tasks or jobs(except Sitecore default) to PI.

How many instances Sitecore supports?

Logically, unlimited. We can have multiple Sitecore Instances working as CM with/without load-balanced environment with one PI.

As per Sitecore architecture, only one PI is recommended. But after reviewing publishing code and my experiements done locally, I concluded that we can have two PIs too. But, that will work with minor architecture change and only one limitation that an item can not be published on both PIs simultaneously.

Must read posts for Sitecore Publish

- Sitecore Publishing Facts
- Intelligent Publish in Sitecore - The most optimized approach
- Sitecore Parallel Publishing using Multiple Publish Instances