A Big Data Solution Providing Insights into Customer Behavior Across 30+ Dimensions
About the Client
The Client is a US-based telecom company participating in the federal Lifeline Support Program and providing prepaid cell phones and service packages to low-income individuals.
Turning Multi-Source Telemetry Into Customer-Behavior Insights
As part of the project, INNERLUXES's analytics team was to design and implement a data management and analytics platform to let the Client collect data from multiple sources and gain insights into customer behavior. The Client wanted the platform to analyze historical data and enable forecasting. Access rights were another issue to solve, as the Client planned to provide its tenants with access to tenant-related analytics.
An AWS Big Data Platform With Kafka Streaming and Redshift
The data analytics platform gathered raw data (such as users' impressions and click-throughs, tariff plans, device models, apps installed, and more) from 10+ sources. To collect this telemetry data and move it into Apache Kafka, INNERLUXES's big data team suggested the MQTT protocol.
The team also suggested using Amazon Spot Instances to reduce the cost of AWS computing resources, and used AWS Application Load Balancers to ensure the analytical system's scalability.
Apache Kafka acted as the data streaming platform, where the raw data was organized for further offload into the landing zone running on Amazon Simple Storage Service. Amazon Redshift was chosen for data storage and warehousing, supplied with telemetry data from Android mobile phones as well as information from the Enterprise Resource Planning system and the Home Location Register (HLR).
To enable regular and ad hoc reporting, INNERLUXES developed ROLAP cubes with 30+ dimensions and 10+ facts. For instance, the analytical system measured a particular user's advertising impressions and click-throughs to calculate the reward points earned. Another example: based on an increased number of calls to support, the Client could expect that a user was likely dissatisfied with the service — which, if no action were taken, could lead to customer churn.
Not only the Client but also its tenants (also telecom companies with their own customers and HLRs) were granted access to the platform for valuable insights. For example, a tenant can access the part of analytics related to its company. To make this possible, INNERLUXES introduced two approaches: shared access (organized at the data warehouse level) and dedicated access (involving a separate AWS account).
Customer Insights and an 80% Cut in AWS Compute Cost
With INNERLUXES's big data services, the Client was able to:
- Measure the engagement and identify the preferences of a particular user.
- Spot trends in users' behavior.
- Make predictions about how users would behave.
- Invoice advertisers based on their calculated share.
- Benefit from insightful data analytics (for example, daily earnings, number of new users, customer service data, and more).
The use of Amazon Spot Instances allowed the Client to reduce the cost of AWS computing resources by 80%.
Technologies and Tools
Amazon Web Services (Amazon cloud), Apache Kafka (data streaming), the Message Queuing Telemetry Transport (MQTT) protocol, Amazon Simple Storage Service (persistent storage used for the data landing zone), Amazon Redshift (data warehouse), Apache Airflow, and Python (ETL).