[Photo: Databricks]

Databricks on Monday unveiled a new data architecture that integrates transactional databases and analytics systems.

At its annual Data+AI Summit conference in San Francisco, Databricks unveiled its lake transactional/analytical processing (LTAP) architecture, which integrates transactional databases and analytics. It also unveiled a real-time analytics engine designed to deliver millisecond responses without separate systems.

Databricks said LTAP handles operational and analytics workloads at the same time using a single dataset stored in a data lake. Applications, analytics systems and AI agents can access the same data, eliminating the need for CDC (change data capture) pipelines linking operational and analytics environments, ETL (extract, transform, load) processing to move operational data into analytics systems, and replica databases, the company explained.

AI agents read, analyse and execute data in near real time, and the company says existing architectures are not suited to the AI-agent era.

SiliconANGLE reported that Shanku Niyogi (샹쿠 니요기), Databricks' vice president of product management, said, "Agents need to analyse and execute data faster than humans. This is causing the data stack to become a bottleneck."

Companies have long run separate systems for transaction processing and analytics. Operational applications write data to transactional databases, while analytics systems use copies brought in through ETL and CDC pipelines. Databricks said this structure causes latency, complexity and governance issues, and that the problem grows as AI applications increase.

Niyogi said a large bank customer is currently operating hundreds of thousands of Postgre databases and brings data into the lake through a CDC pipeline for each one.

LTAP is based on Lakebase, the Databricks database platform introduced last year. It writes transactional data directly into open columnar formats such as Delta Lake and Apache Iceberg while maintaining compatibility with Postgre databases.

Databricks also unveiled Lakehouse//RT, a real-time analytics engine. Based on the new execution engine, Raiden, it delivers response times as low as 10 milliseconds for small jobs and under 100 milliseconds for large jobs, the company said.

Databricks will provide LTAP as an upgrade to Lakebase customers. Lakehouse//RT has entered beta testing, and existing Lakehouse customers can use it with their current subscriptions.

Keyword

#Databricks #LTAP #Data+AI Summit #Lakebase #Lakehouse//RT
Copyright © DigitalToday. All rights reserved. Unauthorized reproduction and redistribution are prohibited.