Today, we are excited to introduce a very powerful new framework to The Synthetic Data Vault : 𝗰𝗼𝗻𝘀𝘁𝗿𝗮𝗶𝗻𝘁 𝗮𝘂𝗴𝗺𝗲𝗻𝘁𝗲𝗱 𝗴𝗲𝗻𝗲𝗿𝗮𝘁𝗶𝗼𝗻 (#CAG for short). CAG addresses the shortcomings of generative models in capturing the context buried in enterprise data stores - with human input. (Link to the announcement: datacebo.com/announcements/i…) ❎ 𝗚𝗲𝗻𝗲𝗿𝗮𝘁𝗶𝘃𝗲 𝗔𝗜 𝗺𝗼𝗱𝗲𝗹𝘀 𝗳𝗮𝗶𝗹 𝘁𝗼 𝗰𝗮𝗽𝘁𝘂𝗿𝗲 𝗱𝗲𝘁𝗲𝗿𝗺𝗶𝗻𝗶𝘀𝘁𝗶𝗰 𝗿𝗲𝗹𝗮𝘁𝗶𝗼𝗻𝘀𝗵𝗶𝗽𝘀 𝗯𝗲𝘁𝘄𝗲𝗲𝗻 𝗰𝗼𝗹𝘂𝗺𝗻𝘀, 𝗿𝗼𝘄𝘀, 𝗮𝗻𝗱 𝘁𝗮𝗯𝗹𝗲𝘀. We call such relationships database context. Database context describes hard and fast rules under which data is created and stored. What is even harder is that usually, this context is not explicitly stored within the database schema itself – but data teams know that it exists. Downstream applications process this data based on the context using logic within the application software. When the generative models are used to create #syntheticdata the expectation is that the #syntheticdata will also follow the database context. ✅ When we launched The Synthetic Data Vault — a system to enable enterprises to build generative models for their own #multitable data — we provided the ability to include context via what we called #𝗰𝗼𝗻𝘀𝘁𝗿𝗮𝗶𝗻𝘁𝘀. 🔥 Over the years, 𝗰𝗼𝗻𝘀𝘁𝗿𝗮𝗶𝗻𝘁𝘀 𝗵𝗮𝘀 𝗯𝗲𝗰𝗼𝗺𝗲 𝗼𝗻𝗲 𝗼𝗳 𝘁𝗵𝗲 𝗺𝗼𝘀𝘁 𝗽𝗼𝗽𝘂𝗹𝗮𝗿 𝗳𝗲𝗮𝘁𝘂𝗿𝗲𝘀 𝗼𝗳 𝗼𝘂𝗿 𝗦𝗗𝗩 𝗘𝗻𝘁𝗲𝗿𝗽𝗿𝗶𝘀𝗲 𝗽𝗿𝗼𝗱𝘂𝗰𝘁. 💪 𝗪𝗶𝘁𝗵 𝗖𝗔𝗚 𝘄𝗲 𝗮𝗿𝗲 𝗱𝗼𝘂𝗯𝗹𝗶𝗻𝗴 𝗱𝗼𝘄𝗻 𝗼𝗻 𝘁𝗵𝗶𝘀 𝗳𝗼𝗰𝘂𝘀. To use this new and powerful framework, users can just pick the pre-defined pattern that corresponds to their database context and tell SDV where to apply it. It will then augment your synthesizer directly with this information. And 100% valid #syntheticdata 𝗥𝗲𝗮𝗱 𝗺𝗼𝗿𝗲 𝗮𝗯𝗼𝘂𝘁 𝗖𝗔𝗚, 𝘄𝗵𝗮𝘁 𝗶𝘁 𝗺𝗲𝗮𝗻𝘀 𝗳𝗼𝗿 𝘆𝗼𝘂, 𝗮𝗻𝗱 𝗵𝗼𝘄 𝘁𝗼 𝗮𝗰𝗰𝗲𝘀𝘀 𝗶𝘁 𝗶𝗻 𝗼𝘂𝗿 𝗹𝗮𝘁𝗲𝘀𝘁 𝗽𝗿𝗼𝗱𝘂𝗰𝘁 𝗮𝗻𝗻𝗼𝘂𝗻𝗰𝗲𝗺𝗲𝗻𝘁 𝗵𝗲𝗿𝗲: datacebo.com/announcements/i… 𝗛𝗮𝗽𝗽𝘆 𝗵𝗼𝗹𝗶𝗱𝗮𝘆𝘀 𝗮𝗻𝗱 𝗲𝗻𝗷𝗼𝘆 𝘀𝘆𝗻𝘁𝗵𝗲𝘀𝗶𝘇𝗶𝗻𝗴! - from all of DataCebo Team #syntheticdata #generativeai #data #machinelearning #ml #ai

Working with customers all over the world has taught us about one important, but often overlooked benefit of using #syntheticdata: increased data diversity. Data diversity refers to the overall variety of data that is accessible for a project. While it's a simple concept, increasing data diversity can deliver enormous value to an organization — allowing for more robust testing, better predictions, and even higher creativity. In our latest blog post, (link: lnkd.in/eumZtB38) Neha Patki elaborates this concept using some concrete examples. In the blog we cover: ✳️ How #generativeai models in #SDV (@sdv_dev ) can create #syntheticdata that is diverse ? ✅ How generative models maintain the delicate balance between creating novel (diverse) data and preserving high degree of statistical resemblance to real data ❎ While we can create really diverse data using random data generators, they won’t resemble the real data. Also in this blog are examples of how data diversity as generated by #syntheticdata impacts the outcomes. (Links to each of these case studies are in the blog as well) Diverse synthetic data enables creation of a more robust product. ING creates synthetic financial transactions with 100x the combinations present in their real data. This allows them to thoroughly test all aspects of a complicated payments service, and keep their payments systems working. Diverse #syntheticdata can help make better predictions. A research team at UCLA (@UCLA ) created synthetic credit card fraud events, which combined different characteristics of real fraud events into rarer occurrences. The synthetic data allowed them to better predict future credit card fraud by nearly 20x. It allows product development teams to navigate new ideas. One promising new direction for research and development teams involves using synthetic data to invent brand new products by combining attributes of existing ones. Link to the blog: datacebo.com/blog/synthesize…

Excellent article in @Forbes today calling #syntheticdata “an all-too-rare example of…genuinely useful” generative AI, for the particular application of software testing. Read @jpwarren profile of @datacebo and @kveeramac : forbes.com/sites/justinwarre… #bigdata #syntheticdata #generativeai #data #datascience #enterprisedata #tabulardata #predictiveAI #machinelearning #ML #generativemodels #MLmodels #productiontesting #softwaretesting #cybersecurity #hacking #infosec #security

🎉2022 was an exciting year for us, our #opensource & enterprise SDV, and for #syntheticdata overall. 🥳⭐ Here’s a recap of our 2022 + some innovations to come in 2023👉 ow.ly/i8KC50MTc3T #machinelearning #artificialintelligence

Our co-founder & VP of Product @n4atki outlines 3 strategies that have directly impacted how far & wide our #opensource Synthetic Data Vault ecosystem has been adopted 💡Hint: they all start with #userexperience ow.ly/iSWB50MMsYQ #syntheticdata #machinelearning #AI

What non-contributors bring to the #opensource table 👉 indispensable feedback ⭐ finding use cases💡& helping us to improve our #OS 💪+ the product that relies on it. Read our latest blog post 📝 ow.ly/CJZk50MGOvY #ML #AI #SyntheticData #datascience

Our Copulas Python library hit a million downloads this weekend! More background on how it's used: github.com/sdv-dev/Copulas/b… #datascience #stats #bigdata #syntheticdata

4 ways to compare synthetic & real data with SDMetrics. Explore the open-source library: github.com/sdv-dev/SDMetrics #SyntheticData #DS #DataScience #BigData