Coveo For Sitecore VS Meta Fields

Conflicts Between Sitecore Data And The Actual Web Page

Coveo has two different methods of getting content to index. The newer one is the FetchPageContentProcessor, and the older on is the BasicHtmlContentInBodyProcessor.

Lots of older Coveo installations, for instance versions 4.0 and 4.1, use the older one. This one generates a bunch of html using the Sitecore fields in the page item and feeds that to the Coveo indexer. The newer one renders out the entire page as the browser sees it, and feeds that page to the indexer.

What Are The Implications?

The first implication is that the newer version calculates relevancy a lot better. Got accordions on your page that use their own data source items? Now Coveo can see them and feed that data into the relevancy calculator. Since so many Sitecore sites use a component based build for pages, this is in general a huge improvement.

The second implication is that, well, all of the page is now available to Coveo. This means things like the menu, footer, and other general content need to be excluded. Coveo makes that easy with their no-index comment syntax, so we won't cover that today.

That makes a new problem that's more subtle, however. Coveo can now see the <meta> tags. Great, you say? Well, yes, they are very useful for relevance and searching. But not so fast; they are commonly named things that conflict with Sitecore fields. Title, Description, and so on. See Search Result Title Selection for an example from Coveo.

Why Does That Matter And How Do We Work Around It?

Before, your search result titles said nice things like "Our Article To Improve Your Business" or "Five Amazing Tips". Now they say things like "Five Amazing Tips | My Awesome Company". While your company is awesome, you don't want to see that name thirty times on your search page. It doesn't help your users find what they want. What happened?

The title meta tag is overriding the "Title" field in your Sitecore item. Or possibly the Sitecore display name, if you were using that before. Since you likely have more than a few fields like this, I'm going to detail a generic method to apply this easily to many. The first thing you need is a basic text field to Coveo data converter, using a computed field. Various guard clauses in the code have been left out for brevity.

public class BasicTextField : IComputedIndexField
{
	public string FieldName { get; set; }
	public string ReturnType { get; set; }
	public string SourceField { get; set; }
	
	public BasicTextField(XmlNode configNode)
	: base()
	{ 
		this.FieldName = XmlUtil.GetAttribute("fieldName", configNode);
		this.ReturnType = XmlUtil.GetAttribute("returnType", configNode);
		this.SourceField = XmlUtil.GetAttribute("sourceField", configNode);
	}
	
	public object ComputeFieldValue(IIndexable indexable)
	{
		Item item = (Item)(indexable as SitecoreIndexableItem);
		String value = item[SourceField];

		if (!String.IsNullOrEmpty(value))
		{
			return (object)value;
		}

		return (object)null;
	}
}

This is adapted from here: Create a Computed Field with some additions to make configuration easier. This would go in whichever Foundation dll you are using to handle indexing or search. So that's the C# end of it. Now we need the actual configuration.

The first entry is for the FieldMap/Fieldnames section of your custom Coveo fields. I've named it ActualTitle since it will replace the Coveo-generated title. The second entry uses that code we just created and goes in the DocumentOptions/fields config section.

<fieldType fieldName="ActualTitle" isSortable="true" type="System.String" settingType="Coveo.Framework.Configuration.FieldConfiguration, Coveo.Framework" returnType="System.String" />

<field fieldName="ActualTitle" sourceField="Title">YourAwesomeCompany.Foundation.Indexing.BasicTextField,YourAwesomeCompany.Foundation.Indexing</field>

What Next?

Now you can make similar config entries for description, or any other field you need. No need to make more C# classes, do additional deploys, etc. Most importantly, now you don't have to go through all of your Sitecore content and change the field names. This means your other code stays as is, any special processing for meta tag fields is fine, and so on.

The only downside is that you do need to change your Coveo result templates to have "ActualTitle" or "ActualDescription" in them, but this is much easier than changing a whole bunch of Sitecore items.