@RyanMarcus

Assistant prof @CIS_Penn. Machine learning for systems, databases. Also https://nitter.cf/t.co/VQvaqvHixu

 Philadelphia, PA
ryanmarc.us
Joined March 2009
  • Tweets687
  • Following1.1K
  • Followers1.7K
107Media
Query optimization benchmarks too easy? Gremlin generates simple select-project-join queries that are up to 800x slower on DuckDB and PostgreSQL than a known plan. Gremlin simultaneously searches for both an adversarial input and matching fast plan to reveal perf bugs in QOs.
2
1
14
846
Check out the paper here: rm.cab/gremlin Generated queries for PG and DuckDB: github.com/Speculative/adver… Jeff Tao will present this work at the AIDB workshop today at VLDB! Check out Jeff's other work: speculative.tech/
4
225
Our group is part of two @VLDBconf papers on query opt. and execution: 1) Ultron, a system for building history-based optimization rules, deployed at @databricks 2) Global hash tables strike back!(?), an experimental analysis of fully concurrent group by aggregation. ⬇️
2
8
62
5,007
Ultron, in collaboration with @databricks , is an async push-style history engine used by the Databricks fleet. An example Ultron application -- selecting shuffle vs. braodcast hash joins -- reduced the median latency of applicable queries by 25%. rm.cab/ultron
1
4
438
"Global Hash Tables Strike Back!" analyzes partitioned vs. fully concurrent GROUP BY implementation strategies, finding that a simple, custom-tailored hash table may be the missing ingredient to effective fully concurrent GROUP BY implementations. vldb.org/pvldb/vol19/p523-ma…
1
266
Most database teams optimize what they see in workload logs. But those very optimizations change what users choose to run! In our CIDR paper, we argue that industrial workloads exhibit 𝐬𝐮𝐫𝐯𝐢𝐯𝐨𝐫𝐬𝐡𝐢𝐩 𝐛𝐢𝐚𝐬: logs reflect a negotiation between users and the platform.
A four-step cycle diagram showing feedback between database users and engineers. ① User submits their workload to the system: A square grid of colored squares represents the workload (3 green, 2 purple). ② Engineers observe properties of the workload: A database cylinder leads to a chart showing workload composition: purple 30%, green 60%. A speech bubble from an engineer character says, “Most queries are green. I’ll trade lower purple performance for higher green performance.” ③ Engineers identify hotspots and optimize their system. ④ Users optimize their workloads based on their platform: A speech bubble from a user character says, “Our platform is good at green, but bad at purple — send more green!” An arrow shows users adjusting the workload and feeding it back into the system. Overall, the diagram illustrates a feedback loop where system optimizations influence user behavior, which then shapes the workload engineers observe.

ALT A four-step cycle diagram showing feedback between database users and engineers. ① User submits their workload to the system: A square grid of colored squares represents the workload (3 green, 2 purple). ② Engineers observe properties of the workload: A database cylinder leads to a chart showing workload composition: purple 30%, green 60%. A speech bubble from an engineer character says, “Most queries are green. I’ll trade lower purple performance for higher green performance.” ③ Engineers identify hotspots and optimize their system. ④ Users optimize their workloads based on their platform: A speech bubble from a user character says, “Our platform is good at green, but bad at purple — send more green!” An arrow shows users adjusting the workload and feeding it back into the system. Overall, the diagram illustrates a feedback loop where system optimizations influence user behavior, which then shapes the workload engineers observe.

1
6
465
more replies
For researchers, databases traces are a MAJOR upgrade compared to synthetic benchmarks (or simply making something up, which is shockingly common). We argue we need more of these workload traces to build a complete picture, and, perhaps more importantly, see what is missing.
1
2
210
We conclude with a discussion about how database researchers should use industrial traces, and how we might begin to build systems that optimize for "the query the user never sends." 📄Paper: rm.cab/survivorshipbias
1
170
OLAP workloads are dominated by repetitive queries -- how can we optimize them? A promising direction is to do 𝗼𝗳𝗳𝗹𝗶𝗻𝗲 query optimization, allowing for a much more thorough plan search. Two new SIGMOD papers! 🧵
1
10
664
LimeQO (by @yi_zixuan), a 𝑤𝑜𝑟𝑘𝑙𝑜𝑎𝑑-𝑙𝑒𝑣𝑒𝑙 approach to query optimization, can use neural networks or simple linear methods to find good query hints significantly faster than a random or brute force search. 📄rm.cab/limeqo
Infographic describing LimeQO, a workload-level, offline, learned query optimizer. On the left, it shows a workload consisting of multiple queries (q₁ to q₄), each with a default execution time (3s, 9s, 12s, 22s respectively). On the right, alternate plans (h₁, h₂, h₃) show varying execution times for each query, with some entries missing (represented by question marks). For example, q₁ takes 1s under h₂, much faster than the 3s default. A specific callout highlights that for q₃, plan h₃ reduced the time from 12s to 3s, but took 18s to find, resulting in a benefit of 9s gained / 18s search. The image poses the question: “Where should we explore next to maximize benefit?” The image credits Zixuan Yi et al., SIGMOD '25, and provides a link: https://rm.cab/limeqo

ALT Infographic describing LimeQO, a workload-level, offline, learned query optimizer. On the left, it shows a workload consisting of multiple queries (q₁ to q₄), each with a default execution time (3s, 9s, 12s, 22s respectively). On the right, alternate plans (h₁, h₂, h₃) show varying execution times for each query, with some entries missing (represented by question marks). For example, q₁ takes 1s under h₂, much faster than the 3s default. A specific callout highlights that for q₃, plan h₃ reduced the time from 12s to 3s, but took 18s to find, resulting in a benefit of 9s gained / 18s search. The image poses the question: “Where should we explore next to maximize benefit?” The image credits Zixuan Yi et al., SIGMOD '25, and provides a link: https://rm.cab/limeqo

1
7
453
For that one query that must go 𝑟𝑒𝑎𝑙𝑙𝑦 𝑓𝑎𝑠𝑡, BayesQO (by Jeff Tao) finds superoptimized plans using Bayesian optimization in a learned plan space. It’s costly, but the results can train an LLM to speed things up next time. 📄rm.cab/bayesqo
Infographic describing BayesQO, an offline, multi-iteration learned query optimizer. On the left, it shows a Variational Autoencoder (VAE) being pretrained to reconstruct query plans from vectors, using orange-colored plan diagrams. The decoder part of the VAE is retained. In the center and right, the image shows Bayesian optimization being performed in the learned vector space: new vectors are decoded into query plans, tested for latency, and refined iteratively. At the bottom, a library of optimized query plans is used to train a robot labeled “LLM,” which can then generate new plans directly. The caption reads: "We get a fast query, but also a library of high-quality plans. We can train an LLM to speed up the process for next time!" The image credits Jeff Tao et al., SIGMOD '25, and links to https://rm.cab/bayesqo.

ALT Infographic describing BayesQO, an offline, multi-iteration learned query optimizer. On the left, it shows a Variational Autoencoder (VAE) being pretrained to reconstruct query plans from vectors, using orange-colored plan diagrams. The decoder part of the VAE is retained. In the center and right, the image shows Bayesian optimization being performed in the learned vector space: new vectors are decoded into query plans, tested for latency, and refined iteratively. At the bottom, a library of optimized query plans is used to train a robot labeled “LLM,” which can then generate new plans directly. The caption reads: "We get a fast query, but also a library of high-quality plans. We can train an LLM to speed up the process for next time!" The image credits Jeff Tao et al., SIGMOD '25, and links to https://rm.cab/bayesqo.

6
329
Pair(akeet) programming.
1
13
843
At aiDM@SIGMOD, PhD student Zixuan Yi will present LimeQO, the first *workload-level* learned query optimizer: simultaneously learning to optimize an entire query workload at once! By casting the problem as low rank matrix completion, we show that linear methods are all you need.
A workload matrix. Each row represents a query, and each column represents a hint. Each entry is the latency of that query using that hint. Some values are unobserved, others are observed.

ALT A workload matrix. Each row represents a query, and each column represents a hint. Each entry is the latency of that query using that hint. Some values are unobserved, others are observed.

The LimeQO system model. A workload matrix is continuously approximated. Then, offline exploration is used to verify which query plans represent improvements.

ALT The LimeQO system model. A workload matrix is continuously approximated. Then, offline exploration is used to verify which query plans represent improvements.

2
2
22
1,828
📝 Check out the paper: rm.cab/limeqo 📄 Zixuan's NEDB poster: bu-disc.github.io/nedbday/20… 🌐 Zixuan's website: zixy17.github.io/

Zixuan (Eve) Yi

Zixuan (Eve) Yi is a PhD student in computer science at the University of Pennsylvania, working at the intersection of machine learning and systems on efficient and reliable AI systems.

zixy17.github.io
3
576
Greatly enjoyed talking with Jack! We discussed the "research journey," what it means for DB research to be impactful, and new work from our lab about query optimization!
Disseminate: The Computer Science Research Podcast@DisseminatePod
20 May 2024
🚨 The first episode in our #HighImpact series with Ryan Marcus (@RyanMarcus) is available now! 🎧 Listen on Spotify ➡️ open.spotify.com/show/6IQIF9… 🎧 Listen on Apple ➡️ podcasts.apple.com/us/podcas…

Disseminate: The Computer Science Research Podcast

Podcast · Jack Waudby · This podcast features interviews with Computer Science researchers. Hosted by Dr. Jack Waudby researchers are interviewed, highlighting the problem(s) they tackled, solutions...

open.spotify.com
1
16
1,908
How much has everyone's favorite open source query optimizer, PostgreSQL, improved over the last 10 years? Turns out, quite a lot! Blog post: rmarcus.info/blog/2024/04/12…
A plot of P90 tail latency for PostgreSQL major versions 8 through 16. Tail latency is cut approximately in half from version 8 to version 16.

ALT A plot of P90 tail latency for PostgreSQL major versions 8 through 16. Tail latency is cut approximately in half from version 8 to version 16.

2
11
61
6,316
I'm recruiting PhD students for Fall 2024 @CIS_Penn! Our lab is using ML to build the next generation of data systems. Come build systems that automatically invent new algorithms, adapt to changing environments, and understand user intention! rm.cab/phd
The three pillars of machine programming: invention, adaption, and intention. Inventive and adaptive systems are called "instance optimized" systems. Inventive and intentional systems are "program synthesis." Adaptive and intentional systems are "auto tuning."

ALT The three pillars of machine programming: invention, adaption, and intention. Inventive and adaptive systems are called "instance optimized" systems. Inventive and intentional systems are "program synthesis." Adaptive and intentional systems are "auto tuning."

1
18
3
70
13,605
Excited to be involved in 3 VLDB papers, a demo, and 2 workshop papers! Collaborations between UPenn and Meta, Intel, MIT, TUM, and Stony Brook. Check them all out in this thread 👇 or on my website rm.cab/vldb23
A list of the six papers described below. AdaChain is presented in session H4 on Wednesday at 1:30pm in Orca AutoSteer is presented in session A4 on Wednesday at 1:30pm in Grand Ballroom Robust Cardinality is presented in session C5 at 3:30pm in Ballroom AB QO-Insight is presented in demo group B on Tuesday at 3:30pm Superoptimization is presented at AIDB on Friday at 2:30pm RLShard is presented at AIDB on Friday at 2:45pm

ALT A list of the six papers described below. AdaChain is presented in session H4 on Wednesday at 1:30pm in Orca AutoSteer is presented in session A4 on Wednesday at 1:30pm in Grand Ballroom Robust Cardinality is presented in session C5 at 3:30pm in Ballroom AB QO-Insight is presented in demo group B on Tuesday at 3:30pm Superoptimization is presented at AIDB on Friday at 2:30pm RLShard is presented at AIDB on Friday at 2:45pm

1
6
31
3,331
more replies
SageDB: a prototype instance optimized analytics DB, is the culmination of several years of research into instance optimized systems -- make sure to check out @jialin_ding's presentation on Wednesday at 10:30am in Gulf Islands. rm.cab/sagedb
1
2
1,006
(technically, SageDB was in VLDB volume 15, last year, but it is being presented this year!)
820
Load more