The Future of Marketing with Sitecore: Data, Intelligence and Machine Learning with Cortex

It’s an exciting step closer to the “self-optimizing digital journey.”

We love finding smart ways to help marketing teams with both the data Sitecore collects as well as its marketing features. 

Here are two exciting opportunities with Sitecore 9 and beyond.

Collection and analysis of behaviour with Sitecore xConnect

With Sitecore 7 and 8 we had the introduction of the xDB, a flexible and powerful database collecting behaviour across the customer journey.

The ideal state was to tag your journey with as many “digital sensors” as possible, similar to covering a car or ship with physical sensors, to record the richest set of behaviour and inputs. Then this data could be used for real-time personalization or exported to a business intelligence platform, among other things.

That said, it was a bit more intensive in the older versions to extend the model to include CRM data, for example, or to export the data into a warehouse. (For the technically oriented folks, there was typically some heavy lifting required for ETL from a NoSQL to a relational structure).

With xConnect, we have a much easier way to pull in data from all channels and external business systems, as well as exporting data externally. Put simply, xConnect is a web API-based service layer that sits in between the xDB and any trusted system that wants to read, write, or search xDB data. It abstracts the underlying database and offers a consistent, standard, structured way to interact with the data.

This means we have a much easier time doing the following very desirable marketing activities!

• Pulling CRM data, such as name, email, preferences or transactional history into xDB and using it to personalize content, promote specific calls to action or segment email lists
• Capturing physical as well as digital behaviour; for example integrating beacon technology to capture someone’s physical presence in a store or near a product
• Exporting behavioural data into a centralized data warehouse and joining to data from other business systems, making it easier to build effective marketing dashboards in PowerBI, Tableau or other BI tools
• Exploring applications of data science to boost behavioural insights

Sitecore Cortex: Embedded Machine Learning

This is one of the most exciting areas of development in Sitecore 9. The Sitecore community including product-side, customer-side and partner-side, being the smart innovators they are, had already seen the potential for some extremely cool applications of machine learning (ML) within the platform.

A few early pioneering examples included:

• Building a predictive model with the xDB to determine which behaviours strongly influence conversion and engagement
• Integrating Sitecore and Microsoft Cognitive Services for enhanced content author experience, including image search and assistance via chatbot
• Applying a clustering algorithm to understand groupings of content in Sitecore by keyword and similarity

But let’s back up for a minute and understand how machine learning can help the marketing function. (It’s safe to say machine learning is firmly climbing the peak of inflated expectations in the hype cycle).

What does machine learning help us do?

Put simply, machine learning lets us replace human judgement with prediction. There are very specific things that computers do better than humans, and those include learning from and finding patterns within large datasets to make “good enough” predictions.

So any obscure, tedious, difficult, data-crunching processes that exist – in marketing, or in any discipline – are candidates for automation with machine learning. A classic marketing example would be the automation of image tagging via image recognition techniques – instead of a human looking at each image and tagging “flower” or “car”, a computer could do that instead, and get smarter and smarter about it.

So in a nutshell, machine learning can:

1. Automate tedious processes by replacing human judgement with prediction or pattern recognition
2. Discover new patterns, groupings, segments and behaviours we didn’t know existed

So what does Sitecore Cortex do?

Cortex is the machine learning module inside Sitecore 9, and it seeks to take good advantage of the wealth of data collected by the xDB (facilitated of course by xConnect!), directly within the product.

There are so many places in the content authoring and marketing workflows where tasks can be automated – and made more intelligent – with machine learning and rich behavioural data.

The idea behind Sitecore Cortex, your “personal data scientist”, is a framework that makes it easy to embed machine learning models into these tasks, replacing previous human configuration or pipelines. The first areas of application that have been released fall under:

• Probabilistic scoring on a Sitecore Contact’s likelihood to convert/purchase
  • How could this make our lives easier? To personalize on a score threshold, to segment “hot leads” from other contacts.
• Segment/Audience discovery
  • How could this make our lives easier? To understand unknown clusters of visitor behaviour that may be worth targeting. Consider this a complementary opposite to persona tagging and pattern matching.
• Automated personalization
  • How could this make our lives easier? Instead of marketers manually configuring specific personalization rules and variants on the website, Cortex could find the best spots and serve content automatically targeted to individual visitors.

Making it easy to scale for performance with Cortex

The underlying technology in Cortex leverages Azure Web Jobs, R and Microsoft Cognitive Toolkit. The architectural concept is that models built in R can be scaled across, and executed locally with, Sitecore Content Delivery nodes.

This is one of the chief concerns in replacing “normal” processing pipelines with predictive models – how do we ensure no compromise on content delivery performance? Sitecore’s approach will greatly facilitate real-time, or near real-time queries and data feeding to the model.

One additional thing I’d love to see is Sitecore expanding the Cortex scaling to include models and pipelines on other platforms such as Azure ML or RapidMiner.

Making it easy for your team to pull in the right skillsets

The other advantage of this architectural approach is that it will be easy to segment skillsets to build on and extend Cortex. Prior experiments with Sitecore and machine learning taught us that this skillset mix works well:

• Data scientist with platform or R programming expertise and ability to calibrate models for best performance
• Sitecore analyst who deeply understands xDB and xConnect data structures
• Sitecore developer who understands the pipelines and solution architecture
• Business/marketing stakeholder who understands marketing objectives, audience segments and KPIs for the organization (helps if they are a Sitecore power user too)

We’ve found that it is not strictly necessary for the data scientist to deeply understand the marketing context, as long as they are guided by the other three team members. This of course may vary depending on complexity of objectives and solution, and we always advocate framing any experiments with a solid business problem to solve.