AivexaNewsSearch
AI news for builders and product teamsChecked every hour

Reducing the noise: How Atlassian built an OCSF security lakehouse on Databricks

Collected Oct 15, 2026

Atlassian has built a security lakehouse on Databricks organized around OCSF, the Open Cybersecurity Schema Framework, according to a Databricks post framing the work as a customer case study. The stated problem is scale: the company's logging footprint spans roughly two decades, and the volume and variety of that telemetry create noise that gets in the way of security analysis.

OCSF is an open, vendor-neutral schema for security events, originally shepherded by AWS and now developed under a broader industry umbrella. Its premise is that logs from endpoints, identity systems, cloud control planes and network gear should land in one shared event shape so that detections, correlations and dashboards can be written once rather than per source. A lakehouse, in Databricks terms, combines object storage with table formats and a query engine so the same data serves both engineering pipelines and SQL analytics.

Put together, the pattern is: normalize incoming security telemetry into OCSF, store it in open tables, and run detection and investigation workloads over it. A likely trade-off is that schema mapping is real, ongoing engineering work, since every source has its own field quirks. The payoff Atlassian is pursuing is a single query surface over years of logs instead of a patchwork of per-tool silos.

Why it matters: security engineering teams weighing a lakehouse against a traditional SIEM get a concrete reference point for schema-first ingestion at very large log volumes. For platform teams, the practical lesson is to treat log normalization as a durable data-engineering problem, not a one-off integration.

Read at Databricks

Based on reporting from the original publisher. Visit the source for full context and later updates.

Publisher excerpt

The challenge: Two decades of logging noise at global scaleAtlassian is a global...