direkt zum Inhalt springen

direkt zum Hauptnavigationsmenü

Sie sind hier

TU Berlin

Inhalt des Dokuments

Talks DIMA Research Seminar

Talks  WS21/22
Talk/Location
Lecturer/Subject
Do 18.11.2021 13:00 - 14:00
tu-berlin.zoom.us/j/67220107820
"Storage and Indexing in Cloud Data Warehouses" by Benjamin Wagner (Firebolt)
 

DIMA/IAM presentation: "Storage and Indexing in Cloud Data Warehouses" by Benjamin Wagner (Firebolt)

Join Zoom Meeting

 

Meeting ID: 672 2010 7820

Passcode: 043725

 

Title: "Storage and Indexing in Cloud Data Warehouses".

 

Abstract: Like so many things in life, database system research goes through phases. In the 2000s, the advent of large and comparably cheap main-memory systems seemed to eliminate the need for traditional research topics like buffer managers, on-disk compression, or spilling operators. In recent years, however, improvements in main memory capacity and pricing have stagnated. At the same time, the amount of data has kept growing exponentially. As such, many of the traditional questions facing database systems have re-emerged as exciting research topics.

This can be best seen in modern cloud data warehouses like Firebolt. Disaggregated storage and compute have added another layer to the already deep memory hierarchy. AWS S3 as the backing storage medium suffers from both very high latency and low throughput. This talk shows how to build a modern storage and indexing layer into a cloud data warehouse. We provide an in-depth examination of F3, Firebolt's custom file format. A columnar data layout, aggressive compression of ordered data, and sparse indexing enable high query performance in a challenging cloud environment.

 

Bio: Benjamin leads Firebolt's query processing team. After studying at TU Munich, he decided to join Firebolt to work on real-world data challenges facing the world's most data-driven companies.

He is passionate about all things data and excited by the possibilities arising from academia and industry collaborating to solve some of the hardest problems in data management.


Zusatzinformationen / Extras

Quick Access:

Schnellnavigation zur Seite über Nummerneingabe