In the process of building a monitoring and analytics platform that ingests trillions of data points a day, Datadog has learned many lessons about scalable, distributed systems in the cloud. We'd like to share those experiences with our community in this series: "Datadog on..."
Each episode will offer a conversation with the engineers who build Datadog. They'll share real-world experiences architecting, building, operating, and monitoring modern systems giving you actionable information you can apply at your organization. With plenty of time left for Q&A, we'd like you to join the discussion.
Datadog on Apache Iceberg
November 20, 2025
Alessandro Nori, Leopold Boudard and Ara Pulido
Historically, Datadog has relied on technologies like Snowflake and Apache Spark on raw parquet files (lacking consistent table structure) to power internal analytics and data science at scale. As usage grew across product teams, more features depended on data science teams, and our datasets grew to include more telemetry data, these s...