What is real-time analytics?

Gartner defines real-time analytics as:

the discipline that applies logic and mathematics to data to provide insights for making better decisions quickly. For some use cases, real-time simply means the analytics are completed within a few seconds or minutes after the arrival of new data.

The process of preparing and measuring data as soon as it enters the system is referred to as real-time analytics.

To put it another way, organisations can gather live streams of data, analyse them rapidly, and extract insights or conduct operations on the data in real-time, allowing them to respond swiftly.

They enable firms to take advantage of chances to intervene or avoid issues before they arise. Real-time processing necessitates continuous data input, constant processing, and consistent data output.

Real-time analytics can be classified into the following:

Event Tracking or Streaming Analytics: process event data on a continuous basis and stream the process result to downstream apps or users in real-time with Event Tracking or Streaming Analytics.

Use cases include online sales trends of specific products for eCommerce company and real-time forex or stock price movements of banks or investment companies.

On-demand Analytics: process the event data in real-time and store processed data in a data store for end users and downstream application consumption.

This includes total web page error messages in the last 1 hour, web page visits versus sale conversation rates since sales launch, total subscriptions of an IPO for the last 5 hours, etc.

The Event Streaming Data Architecture

Event streaming data architecture comprises three fundamental components: Message Broker, ETL Process, and Data Storage & Analysis.

Message Broker

Message broker collects streaming data from so-called producers.

Producers may keep track of everything, including analytics, clickstreams, and searches. All of these activities are then saved in Apache Kafka's atomic building components (topics and partitions).

Kafka's ability to parallelise partitions in order to enhance overall throughput is one of its main strengths. Consumers are then given access to information.

Because Kafka is one of the most prominent message brokers, it is utilised as a message broker in the following design (Figure 1). Amazon Kinesis, Google Cloud, Pub/Sub, RabbitMQ are other message broker examples.

Figure 1. Streaming data processing

Extract, Transform, and Load (ETL) Process

These are three database functions that are combined into one tool to extract data from a database, modify it, and place it into another database.

The raw data is then transformed and cleaned so that it can be then used by SQL-based systems for further analysis.

Data Storage & Analysis

After the data is prepared by data stream processors, the next would be to store and analyse it in order to extract valuable insights.

Such a vast amount of data can be stored in data platforms, such as ByteDance's ByteHouse.

Retail e-commerce in Event Tracking

COVID-19 caused a significant rise in e-commerce, especially in retail.

Consumers often shop online for home entertainment and essentials like apparel, groceries, household supplies, and personal-care products. The rise in online shopping is projected to continue even after the COVID-19 situation dies down.

Categories where expected growth in online shoppers exceeds current trends include essentials such as apparel, groceries, household supplies, and personal-care products. Even discretionary categories such as skin care and makeup, fashion wear, and jewellery and accessories show expected customer growth of more than current per cent.

Real-time analysis is valuable in our ever-changing world, where we need to stay on top of trends and make concrete decisions. It is even more important when the online presence and spending of consumers are constantly growing, even as we speak.

Real-time data analytics has become a crucial competitor for corporate growth and one of the most essential elements of any organisation, thanks to improvements in technology and AI.

There has been a shift in consuming data and critical insights on consumer behaviour can be gained in real-time.

The Business Operations team constantly monitors platform activities and transactions to uncover any hidden issues and identify trends in orders, inventory management, and security. The company has a limited amount of time to respond to any situations within a narrow timeframe.

Types of Events To Be Tracked

Event tracking is a highly targeted and effective means of granular measurement. The following are the key events to consider in e-commerce.

These events are often tracked by e-commerce companies. Through targeted analysis, companies can measure and improve user experience. This boosts the revenue brought in.

Figure 2. Streaming data processing with ByteHouse

"Add to Cart" Event

Tracking the goods added to the shopping cart allows you to not only see how many times they've been added but also to calculate the product's conversion rate and make comparisons with other products using the same metrics.

"Add to Cart" Errors

This event will aid in the identification of the most common mistakes clients make when adding goods to their shopping carts. Keep an eye out for recurring issues so they can be fixed and the user experience may be enhanced.

View Product

This event provides information on which items are the most popular. It can assist in determining conversion rates and other queries such as: What proportion of goods are added to the basket once they are viewed? What proportion of the total is really purchased?

Proceed to Checkout

Collecting checkout traffic data is beneficial since it aids in the study of the percentage of users who made the transition from checkout to purchase.

Checkout Error

This event aids in the monitoring of mistakes made by customers throughout the checkout process.

The ratio of checkout error events to unique checkout error events is a highly valuable insight to look into and fix. The higher the ratio of a specific type of error, the higher users struggle with this field (or step) during the checkout process.

"Page not Found" Errors

This event is key in ensuring all pages are appearing correctly. This alerts the IT Operations Team and ensures that it is fixed immediately.

About the author: Sudarsan is a customer solution architect at ByteDance; and, he is a CDMP (DAMA) certified data architect with prior experiences with National University of Singapore, Development Bank of Singapore, Standard Chartered Bank and BNP Paribas.