Session: Building a modern data lake to optimize digital offers for banking partners

Shravana Krishnamurthy, Director of Engineering @ Cardlytics

Cardlytics empowers advertisers with industry-leading purchase insights, enabling them to launch and optimize digital offers. By leveraging extensive purchase data from over 200 million bank customers, we identify opportunities, target real individuals within their banking environments, and measure the actual sales impact of our ads. Partnering with financial institutions, we run rewards programs that drive customer loyalty and deepen bank relationships. With a data scale covering $3.5 trillion in spend and 1 in 2 U.S. transactions, Cardlytics provides unmatched precision in Return on Ad Spend (ROAS) metrics, helping brands drive incremental sales and grow market share.

In this talk, Shravana Krishnamurthy will share insights on building a modern datalake architecture at Cardlytics using Hudi, Airflow, Spark, Lake Formation, Athena, and EMR. The discussion will cover key learnings on Hudi concepts, including indexing strategies, file sizing, and the development of streaming pipelines that ensure efficient data processing. Additionally, Shravana will highlight the use of Superset for Data quality and monitoring.

Want to learn more? Check out these resources from our co-chair Onehouse:

Where & when?

Open Source Data Summit 2024 was held on October 2nd, 2024.

What is the cost of access to the live virtual sessions?

OSDS is always free and open for all to attend.

What is Open Source Data Summit?

OSDS is a peer-to-peer gathering of data industry professionals, experts, and enthusiasts to explore the dynamic landscape of open source data tools and storage.

The central theme of OSDS revolves around the advantages of open source data products and their pivotal role in modern data ecosystems.

OSDS is the annual peer hub for knowledge exchange that fosters a deeper understanding of open source options and their role in shaping the data-driven future.

Who attends OSDS?

OSDS is attended by data engineers, data architects, developers, DevOps practitioners and managers, and data leadership.

Anyone who is looking for enriched perspectives on open source data tools and practical insights to navigate the evolving data landscape should attend this event.

On October 2nd, 2024 we convened for discussions about:
  • Benefits of open source data tools
  • Cost/performance trade-offs
  • Building data storage solutions
  • Challenges surrounding open source data tool integration
  • Solutions for the cost of storing, accessing, and managing data
  • Data streams and ingestion
  • Hub-and-spoke data integration models
  • Choosing the right engine for your workload
Are you interested in speaking or sponsoring the next Open Source Data Summit?

Submit a talk proposal here or reach out to astronaut@solutionmonday.com.

Register for on-demand access to the OSDS 2024 sessions and announcements about OSDS 2025

"*" indicates required fields

Privacy Policy*
This field is for validation purposes and should be left unchanged.