Matsui-lab (@utokyo_bunny) keeps code, models and shared datasets in sync across laptops, lab servers and cloud: Git for code, S3 for storage, DVC tracking dataset versions with MD5 hashes in Git. Full step-by-step setup ๐ก
In Japanese!
hubs.la/Q04z1FdJ0
What does the data architecture for production AI agents actually look like?
Join MinIO and @lakeFS to dive into agent memory, data isolation, reproducibility, and governance + how to build those capabilities into the infrastructure layer without slowing development down.
๐
10/22 | 12 PM ET
๐: bit.ly/3WbJB15
Phat Vo from @fintech24h_com treats data lineage as AI security infrastructure: DVC keeps an audit trail of what went into every training run, alongside layered defenses and continuous monitoring. Provenance first ๐ช๐ผ
hubs.la/Q04z1rMS0
Evgeny Levashov's guide to ML storage on S3: naming schemes for datasets, features and model weights, DVC for dataset versioning, MLflow for artifacts, Feast for features, and lifecycle rules to control costs โจ
hubs.la/Q04z1pXk0
Tomorrow, 11:30 AM ET: AWS's own solutions architects walk through the DVC + SageMaker AI + MLflow lineage pattern from their blog, live.
๐ฃLast call: hubs.la/Q04wzs_d0
๐ฏ DevOps, MLOps, DataOps, AIOps, AgentOps... confused?
Rabiya Abbasi's beginner's guide covers ALL the 'Ops' with tool recommendations for each discipline + a decision-making framework.
Your 2026 roadmap starts here!
hubs.la/Q04wyVcB0
How do you prove a specific record was excluded from training after an opt-out? Next week AWS's own solutions architects show the pattern live ๐๐ผ DVC + SageMaker AI + MLflow. Sept 22.
Register here: hubs.la/Q04wzjXr0
๐ฆDVC retweeted
Every ML failure I have debugged traces back to unversioned data.
Code has Git.
ML data has DVC, lakeFS, Delta Lake, and Iceberg.
Most teams use none of them.
Here is the data versioning landscape and how to pick,
grouped into 3 capability areas:
๐๐ก๐ฒ ๐๐๐ซ๐ฌ๐ข๐จ๐ง ๐๐จ๐ฎ๐ซ ๐๐๐ญ๐
โ Reproduce a model months later without guessing
โก Debug regressions: was it the code or the data
โข A/B test with different data snapshots
โฃ Roll back bad training data before it hits production
๐๐จ๐จ๐ฅ ๐๐จ๐ฆ๐ฉ๐๐ซ๐ข๐ฌ๐จ๐ง
โค DVC: Git-like, works with any storage, best for file-level ML datasets
โฅ LakeFS: Git for object storage, branch/merge on S3, best for lakehouse workflows
โฆ Delta Lake: table-level time travel + versioning, best for Databricks/lakehouse
โง Iceberg: open table format with snapshots, best for multi-engine environments
๐๐ก๐ข๐๐ก ๐ญ๐จ ๐๐ข๐๐ค
โจ File-based ML datasets (images, audio, docs): DVC
โฉ Lakehouse-native workflows on S3: LakeFS or Iceberg
โช Databricks or Delta-committed shops: Delta Lake native
โซ Snowflake shops adopting open formats: Iceberg via Polaris
Unversioned data is the #1 hidden cause of "the model was working last week."
Real senior data engineers make data versioning the default,
Not an afterthought triggered by a production incident.
The gap between "our model is broken and we do not know why"
And "we rolled back to yesterday's data snapshot in 10 minutes"
Is where data versioning earns its keep.
Which tool are you using today? ๐
Follow for more.
If you are looking for a Data Engineering Roadmap, Resume Template or Interview questions with answers do check out my Product โ lnkd.in/gpcJ5rxj
hashtag#DataEngineering hashtag#ML hashtag#MLOps hashtag#Reproducibility
Sysdai builds a production data pipeline end-to-end: Kafka streaming, Airflow batch orchestration, Great Expectations validation, and DVC for dataset versioning and lineage. Handles Lambda and Kappa patterns, batch and streaming both. Check it out!
๐๐ฝ hubs.la/Q04tJY5P0
2 weeks out! AWS solutions architects walk through the SageMaker + DVC + MLflow lineage pattern from their own blog post, live. Sept 22.
Join us!
hubs.la/Q04wyX9n0
Balajee Asish Brahmandam containerizes an MLOps pipeline end to end: multi-stage training/serving containers, MLflow tracking, DVC data versioning, and GPU passthrough, all in one Compose file. A year of hard-won lessons.
๐๐ฝ hubs.la/Q04tMCFR0
Dr. Aleena Baby turns the classic 47-files-in-one-folder ML mess into a professional project layout: configs, src, tests, CI/CD workflows, MLflow tracking, and .dvc files for data. A one-weekend cleanup for reproducibility.
๐๐ฝ hubs.la/Q04tN6wj0
New research: Buhayh, McKinnie, Canel & Burke simulate 4 profile-portability scenarios across Amazon, Goodreads & MovieLens, using DVC for reproducibility. What happens to your recommender data when you switch algorithms?
๐๐ฝ hubs.la/Q04tKBmf0
AWS featured DVC in an official ML lineage pattern alongside SageMaker AI and MLflow.
Join the team who built it for a live walkthrough on Sept 22!
hubs.la/Q04v-GN80
New research: Madugula & Esteva de la Rosa invert the Kelly Criterion to price DeFi loan risk, with MLflow tracking and DVC + DagsHub versioning RWA data for reproducibility. A Reverse Kelly AMM for risk-adjusted lending.
๐๐ฝ hubs.la/Q04tMBfG0
๐ฆDVC retweeted
DevOps to MLOPS ๐
How is MLOPS different from usual DevOps workflows?
MLOps is different from the usual DevOps workflows, just like machine learning is different from traditional software development.
For example,
In MLOps, you need to track three components: code, data, and models
While in DevOps, you only track code.
Additionally, MLOps introduces the following.
- ๐๐ฎ๐๐ฎ ๐๐ฒ๐ฟ๐๐ถ๐ผ๐ป๐ถ๐ป๐ด: To track changes in datasets using tools like DVC (Data Version Control) or LakeFS
- ๐๐
๐ฝ๐ฒ๐ฟ๐ถ๐บ๐ฒ๐ป๐ ๐๐ฟ๐ฎ๐ฐ๐ธ๐ถ๐ป๐ด: To log hyperparameters, metrics, and model versions for each training run
- ๐ ๐ผ๐ฑ๐ฒ๐น ๐ฟ๐ฒ๐ด๐ถ๐๐๐ฟ๐: To version and manage models separately from application artifacts
- ๐๐ผ๐ป๐๐ถ๐ป๐๐ผ๐๐ ๐ง๐ฟ๐ฎ๐ถ๐ป๐ถ๐ป๐ด (๐๐ง): Beyond CI/CD, models need regular retraining as data changes
In the latest newsletter edition,
We break down everything about what MLOps means for DevOps engineers. It covers,
- What is MLOPS
- Skills Required For DevOps Engineers
- Key MLOPS Tools
- Do you need GPU-based clusters to learn MLOps?
So if you are ready to go โall inโ with MLOPS, this edition is for you.
๐ฅ๐ฒ๐ฎ๐ฑ ๐ถ๐ ๐ต๐ฒ๐ฟ๐ฒ: newsletter.devopscube.com/p/โฆ
Are you working on any MLOps projects right now?
Or are you planning to upskill in MLOps?
Either way, share your thoughts in the comments.
#devops #MLOPS
Three pillars of modern MLOps, one clear framework: data versioning with DVC as the gold standard, experiment tracking (MLflow, W&B, ClearML), and metadata/lineage management. The shift from ad-hoc ML to deterministic engineering.
๐๐ฝ hubs.la/Q04tM9FT0
New research: ContextNest adds a governance layer beneath RAG for AI agents, using hash-chained version histories and audit traces. Governed selection beats BM25 (97% vs 93% pass rate) at a third the token cost. hubs.la/Q04tJ-fr0
Proud to be a part of @lakeFS being named a Cool Vendor in Data Management. The world is catching up to the importance of data version control. ๐
Get the report: hubs.la/Q04sX88H0



















