@lakeFS

OSS Data Version Control - git for data. Tweets about data engineering best practices, mlops and dataops 🌐 Join us: https://xitter.cf/t.co/vbXfSRLdCv

github.com/treeverse/lakeFS
Joined June 2020
  • Tweets760
  • Following106
  • Followers1.3K
228Media
Superb piece by @alex_woodie of @BigDATAwireNews (aka @datanami) on the impact #dataversioncontrol is having on production #AI and #ML. Includes the details on our most recent round. Worth your time: bigdatawire.com/2025/07/29/l…
1
7
439
S3 Express One-Zone is a game changer. Object store scale with single digit millisecond latency. More exciting- @lakeFS worked with the S3 team as design partners to support this out of the gate!! lakeFS + S3 Express = 🤯 Read my post on the AWS blog: aws.amazon.com/blogs/storage…

lakeFS and Amazon S3 Express One Zone: Highly performant data version control for ML/AI | Amazon...

Machine learning presents a number of new challenges to data teams, calling for technology solutions that can support training and fine-tuning performance-critical workloads with high performance....

aws.amazon.com
1
16
2
62
8,858
Stop wondering. Why did my workflows fail and what data went through them? Don't operate your pipelines with a blindfold on. Learn how to separate dev and prod for more trust in your data team. Join Prefect and @lakeFS in a webinar Wed 11/15 at 12pm ET: vist.ly/ibx8
3
1
11
1,306
September is right around the corner. Who's joining us where? 🤝💬✈️🌍🤝 #meetups #lakeFS links to events in our comments
2
1
885
Meet us here! meetup.com/nyc-data-science-… meetup.com/seattle-data-scie… bigdataldn.com/seminar-sched… airflowsummit.org/sessions/2… confluent.io/events/current/
1
449
one more meetup added to the roster! meetup.com/dataops-for-big-d…
382
Is #throwbackthursday still a thing? Is RUST the future of analytics? We're going 𝐯𝐞𝐫𝐤𝐥𝐞𝐦𝐩𝐭 here. 𝐃𝐢𝐬𝐜𝐮𝐬𝐬 ☕️💬hubs.la/Q01ZW8G40
1
1
455
Happy Birthday to the team @lakeFS We're thrilled to celebrate our lakeFSary 🎈 🎉
4
553
Just sayin'
1
1
339
hubs.la/Q01ZpZZh0

Data Engineering Patterns: Write-Audit-Publish (WAP)

This blog explains the concept of Write-Audit-Publish, which is a pattern in data engineering to enforce data quality in data pipelines.

lakefs.io
214
Let's delve into the power of parallel ML - running experimentation in parallel with different parameters and explore how @lakeFS and @Minio can supercharge your #MachineLearning experiments and streamline your development pipeline hubs.la/Q01Z5VV20
4
387
Will #Rust dethrone JVM or Python? most likely not. But it will be playing a major role in the data stack of tomorrow. @ozkatz100 makes the case for why in this recap of his talk from @Data_AI_Summit hubs.la/Q01YLXvD0
2
3
510
Couldn't resist 🦖🤣🌊 courtesy of @TasteIessTrex
1
4
293
🥉🥈🥇AUGUST WORKSHOP: Master data pipeline version control using the Medallion Architecture with @lakeFS against your Cloud. Save your spot today hubs.la/Q01Y7Zzn0
1
234
🇷 you ready? This is how you version control in R with @lakeFS It's also how you test code in isolation, remove a subset of the data, write it in a new format and apply changes to the main dataset. All using an open-source tool @rmoff will blow you away! hubs.la/Q01XWsZc0
1
235
Find out how to use @dagster with @lakeFS is to create an isolated environment of the data to run production pipelines, in a way that ensures only quality data is exposed to production. hubs.la/Q01XBBk40
3
4
693
📖Good read!📖 What's the difference between 🔎data discovery and 👀data observability? @shinjikim at @selectstarhq breaks down in detail the key differences. selectstar.com/blog/data-dis…
2
2
365
🚘 Drag 'n' drop files or folders 🚘 Upload entire directory structures into the upload pane of the @lakeFS pane in the UI. Uploads are parallelized and will use pre-signed URLs to upload directly to the object store (meaning they won't have to go through the lakeFS server 😮)
238
💃 🕺It's time to #SlackeFS!💃 🕺 Our growing Slack #community is a great hub for you to share best practices, learn from one another and discuss your own experiences. Join the conversation here 👇 hubs.la/Q01X9dt30
1
216
🤿 Let's dive into the latest enhancements in the @lakeFS & @ApacheAirflow Integration! Join this live session on Wednesday, July 12 and find out how to enable seamless bi-directional data management between #lakeFS and #apacheairflow hubs.la/Q01WVyPH0
1
170
Load more