NinthDecimal is a marketing platform powered by location data. This data is harnessed for building a precise understanding of consumers’ physical world behavior.

NinthDecimal was looking to improve its current legacy data platform which has a lot of drawbacks.

Squadex has designed and built a robust data and machine learning platform that processes 5 billions of events per day and supports 10 different data products. The platform scales data analytics operations and allows daily iterations for building, testing, deploying and monitoring predictive algorithms and models.

As a result of implementation, NinthDecimal achieved a robust data and machine learning platform enabled the solution to perform faster and effective analysis of huge amounts RTB logs. As a result, analysts can gain insights to measure, attribute, and optimize marketing strategies with the most actionable understanding of campaign ROI available.

Increased by 10x

cluster throughput

99%

job completion rate

Improved by 50%

team’s productivity

Squadex Big Data Services

Advanced Analytics & Data Science

Big Data Platform & Next Generation BI

About NinthDecimal

NinthDecimal helps Fortune 500 companies and marquee brands activate this customer intelligence through audience targeting, measurement, insights and data licensing solutions. By filling the gap in understanding how consumers spend the majority of their time, NinthDecimal provides marketers a new model of the customer journey for impactful engagement.

The NinthDecimal platform processes trillions of data points from over 230M unique devices every month to power its product suite. NinthDecimal’s unique household to device matching capability bridges online and offline data, powering omni-channel offerings that include partnerships with leading companies. NinthDecimal has been recognized as one of the most rapidly growing, innovative companies.

PROBLEM

The client needed a measurement and analytics platform to analyze real-time data from multiple sources for actionable insights.  The scale of data processing operations was 5 billions of events per day and a hybrid cluster of 50 AWS nodes and 400 bare metal nodes managed by Apache Mesos.

Their existing legacy platform had the following challenges:

  • DevOps estimated a twelve-month timeline to hand off a data pipeline from a data scientist to a data engineer and then to operations for deployment into production environment
  • Developmental Delays caused implementation bugs and inaccurate timeline projections thus hampering revenue generation
  • Only a 40% job success rate, while 60% of Apache Spark jobs were randomly failing
  • Data scientists were wasting time and clustering resources by re-running dozens of Spark jobs

SOLUTION

Squadex designed and built a robust data and machine learning platform that processes 5 billions of events per day and supports 10 different data products. The platform scales data analytics operations and allows daily iterations for building, testing, deploying and monitoring of predictive algorithms and models.

Real-time and interactive production clusters, as well as all dev-QA operations, were built in Microservices architecture with Apache Kafka as a core messaging bus and durable distributes events store. The solution has been deployed across 4 main clusters:

  • AWS/EMR/Lambda data cluster used for initial data ingestion and filtering
  • On-premise Kubernetes cluster with stateful microservices powered by Apache Kafka
  • Batch data processing cluster with Apache Spark and Apache Mesos used for large scale ad campaign analytics jobs
  • Snowflake cluster used for custom research and analytics as well as for storing final data shapes for partners

AWS Products Used

Results

Solution’s success has been measured using the following KPIs:

  • Apache Spark Job completion rate has reached 99%
  • Cluster throughput has increased by 10x, resulting in savings of $100K/Mo.
  • Teams’ productivity has improved by 50%, resulting in an ROI increase by $1M/Yr.
  • Squadex provided complete coverage of DevOps operation with minimal TCO, while the client’s involvement in the release process has been completely eliminated, thereby providing an instant ROI

Testimonials

The legacy platform that we had just wasn’t getting it done. Not only was our data pipeline way too long, but it was costing us a lot of money since we were wasting so much time on re-running the work we did in Spark. Squadex came in and optimized everything. They saved us a lot of money, provided us with and instant ROI and drastically reduced our release time. They really hit it out of the park.

Lenny Blyukher, VP of Engineering at NinthDecimal

Looking to Improve Your Big Data Pipeline?Schedule Free Consulting Session with our Subject Matter Expert