@OpenLineage

An open framework for data lineage collection and analysis

openlineage.io
Joined October 2020
  • Tweets278
  • Following18
  • Followers2.3K
38Media
Must-read article by @confluentinc's Kai Waehner on cross-platform lineage using OpenLineage and Marquez! kai-waehner.de/blog/2024/05/…

Open Standards for Data Lineage: OpenLineage for Batch AND Streaming

Enterprise-Wide Data Lineage for Data Streaming (Kafka, Flink) with Cloud (Confluent, Collibra, Purview) and Open Standards (OpenLineage).

kai-waehner.de
2
6
1,602
This month's Marquez Community Meeting is THIS Thursday, April 25th, at 11am PT. On the agenda: an update on dataset schema versioning, a new dataset tagging experience, and a preview of job tagging. bit.ly/MarquezMeet
2
548
Replying to @AICouncilConf @DolanRyanne @LinkedIn @MichelleUfford @eBay @KyleJWeller @Onehousehq @TobikoData @captainnobody1 @mX @okta @andrewlamb1111 @InfluxDB @mrocklin @erichansander @twirldata @QuestDb
and a panel hosted by Eric Veleker, Prod & Eng, @AtlanHQ Panel: Data Lineage We’ve Come a Long Way Julien Le Dem @J_ @OpenLineage Proj Lead @LFAIDataFdn @HarelShein-Eng Mgr, Data Lineage @datadoghq @dsrealtime01-C. Evg @Manta Sheeri Cabral -Product Mgr @collibra (7 of 7) 👆
2
5
1
4
1,779
OpenLineage retweeted
Julien Le Dem@J_1 Mar 2024
My blog post recollecting a decade of building open source standards is complete! From the birth of @ApacheParquet and the creation of @ApacheArrow to @OpenLineage. Come sit around the campfire with me as I comment on what I learned through the process. sympathetic.ink/2024/01/24/T…

Ten years of Building Open Source Standards: From Parquet to Arrow to OpenLineage

Over the last decade, I have been lucky enough to contribute to a few successful open source projects in the data ecosystem.

sympathetic.ink
8
36
4,118
OpenLineage retweeted
Julien Le Dem@J_8 Dec 2023
The recording of my latest talk is out! At SBTB, I shared the lessons learned from the last 10 years (ish) of building open source standards from #Parquet to #Arrow to #OpenLineage. youtube.com/watch?v=Xz6HiLa1…

SBTB 2023: Julien Le Dem, Ten years of building open source standards.

This talk video is a node in devreal.ai, the developer community gr...

youtube.com
6
9
901
OpenLineage retweeted
Julien Le Dem@J_5 Oct 2023
See you tomorrow at the Marquez meetup in San Francisco to talk about lineage! There will be pizza and good company! Please sign up at Marquez Meetup @ Astronomer meetu.ps/e/Mp9Vp/1SJ7M/i

Marquez Meetup @ Astronomer, Thu, Oct 5, 2023, 5:30 PM | Meetup

Join us on Thursday, October 5th, from 5:30-8:30 pm to learn about the Marquez project. Meet other members of the community, get tips on making the most of the latest impro

meetup.com
1
2
469
OpenLineage retweeted
Mars Lan@mars_lan3 Oct 2023
Proud to announce our @OpenLineage integration! Thanks to @J_ and the team for the support.
Metaphor Data@MetaphorData3 Oct 2023
📣 Missed #AirflowSummit Toronto? Dive into our integration post by @alyiwang & @mars_lan , our CTO. 🚀 Learn how our tie-up with @OpenLineage is reshaping data governance, making it accessible for all. Stay updated on data's future! 📊🔗 #DataGovernance openlineage.io/blog/metaphor…

Metaphor's Integration with OpenLineage Enhances Data Governance and Collaboration | OpenLineage

Metaphor’s journey towards data governance excellence led to the adoption of OpenLineage.

openlineage.io
1
2
3
823
Come join us Thur Oct 5th for our first-ever meetup! Bring your questions, ideas and vision for Marquez! 🙌 marquezproject.ai/blog/first…

Join Us in October for Our First Meetup | Marquez Project

The first Marquez meetup will be held on 10/5/23 in San Francisco at Astronomer's HQ.

marquezproject.ai
1
1
440
OpenLineage retweeted
Julien Le Dem@J_30 Jun 2023
Congratulations to long time @OpenLineage maintainer Maciej Obuchowski (github.com/mobuchowski) for becoming an @ApacheAirflow Committer, recognizing his work in making OpenLineage support built-in. lists.apache.org/thread/0vdt…
3
11
917
OpenLineage retweeted
Julien Le Dem@J_28 Jun 2023
Replying to @wslulciuc
@wslulciuc and I talking about @OpenLineage and @MarquezProject at the @Data_AI_Summit #DataAISummit
1
3
1
18
1,604
OpenLineage retweeted
Atlan
@AtlanHQ5 Jun 2023
Do more of what brings you joy! 🥰 P.S. Waitlist for Atlan AI is now open. Join here to receive weekly updates about what we're building and an exclusive invite to our launch event: atln.cm/waitlist-atlan-ai
3
4
556
Imagine how better a DE’s life would be if every @ApacheAirflow operator returned lineage info too! @OpenLineage project is making it possible. A great talk by @J_ on building Parquet, Arrow and OpenLineage now!
1
5
25
2,708
OpenLineage retweeted
Julien Le Dem@J_29 Mar 2023
See you at my talk at @DataCouncilAI tomorrow: Ten years of building open source standards: from @ApacheParquet to @ApacheArrow to @OpenLineage datacouncil.ai/talks/ten-yea… – at Austin, TX

Ten Years of Building Open Source Standards

Explore the journey of building open-source standards with insights from Julien Le Dem, covering the birth of Parquet, transition to low-latency systems and…

aicouncil.com
5
14
1,330
The first Providence Data Lineage meetup is just one week away! RSVP here: meetup.com/providence-data-l…
1
2
164
LF AI & Data hosted project @OpenLineage is now an Incubation project! Congrats to the entire OpenLineage team on their continued growth & success. Learn more: hubs.la/Q01zcKVp0 #OpenLineage #LFAIDataFdn #AI #Data
10
1
10
852
My latest post on the @OpenLineage blog is now available. "What's in a Namespace?" is an intro to namespaces with a simple use case and resources for learning more about this under-documented key to the spec. Find it at openlineage.io/blog/whats-in…
3
1
364
OpenLineage retweeted
Julien Le Dem@J_25 Oct 2022
We are looking for feedback on the @OpenLineage @ApacheFlink integration. openlineage.io/docs/integrat… If you are using Flink and are interested in collecting lineage from your streaming jobs, give it a try. (did you check out @MarquezProject ?)
1
4
3
Adaptability is essential when it comes to lineage, and @OpenLineage's support for custom extractors makes it one of the most adaptable tools out there. openlineage.io/blog/extracto…
1
2
Thinking about pursuing column-level lineage capture in your pipelines? Read about the latest developments in column-level lineage at @OpenLineage here: openlineage.io/blog/column-l…
2
1
OpenLineage retweeted
Airbyte
@AirbyteHQ1 Sep 2022
How do you make data quality an easier problem to solve? airbyte.com/blog/data-qualit…

Why is data quality harder than code quality? | Airbyte

Data quality issues arise when data fails to meet expectations. Best practices inspired by code quality make data quality an easier problem to solve.

airbyte.com
1
8
11
Load more