Expert in tackling complex software challenges, with deep knowledge of TCP, MySQL and PostgreSQL kernels, and a passion for AI, history, math, and physics.
Beijing, China
Joined October 2012
- Tweets3.3K
- Following136
- Followers17.8K
- Likes1.4K
wangbin579 retweeted
Comparing the read-only performance of MySQL and PostgreSQL with the same 128 hash partitions.
wangbin579 retweeted
The MySQL optimizer team has been quiet about their achievements (I also know why 馃榾) and have made substantial progress in fixing the core architecture that they inherited according to my sources.
I was chatting with Norvald (lead) at FOSDEM and he told me that the refactoring has now been completed and they are in a position to make significant improvements over plan generation.
I haven鈥檛 looked at the code but I trust Norvald.
Will these or future changes make it into MySQL or remain in Heatwave? That鈥檚 an open question.
wangbin579 retweeted
The execution plan definitely has issues, not making good use of cached plans, which causes serious LWLock conflicts. But it's not just a fast path and execution plan problem鈥攖here are other performance issues too, so it's a multi-bottleneck problem.
wangbin579 retweeted
On my machine, even with the fast path applied, the improvement is limited. The main issue is that the execution plan doesn鈥檛 cache plans effectively. Once that鈥檚 fixed, throughput will improve a lot, but there鈥檚 still a 2x gap compared to my MySQL open-source version.
wangbin579 retweeted
You can try the performance of hash partitioning. I've further optimized it, and the gap has become much smaller.
wangbin579 retweeted
A truly good thread pool is based on transaction scheduling and resource isolation (e.g., isolating OLAP from OLTP). That's the thread pool that truly understands databases.
wangbin579 retweeted
Poolers mainly aim to keep performance stable, not to boost throughput peaks. Whether it's a thread pool or a user-space connection pool, it often has a negative impact with a truly efficient database. I only realized this after solving MySQL's scalability issues.
wangbin579 retweeted
Yes, this can work for most applications, but for many high-demand scenarios, such as distributed file systems, where millions of requests per second are required, improving concurrency capabilities is essential to solving the problem of high throughput.
wangbin579 retweeted
There are many areas that still need improvement, and you can check my previous twitter posts for more details. Here's an additional example: for better cache friendliness, the '_bt_binsrch' function could learn from MySQL's 'page_cur_search_with_match_bytes' function.
wangbin579 retweeted
Some notable books include: PostgreSQL 9.6 High Performance, The Internals of PostgreSQL, PostgreSQL 14 Internals, and PostgreSQL Database Kernel Analysis (in Chinese).
wangbin579 retweeted
Some notable papers include: Greenplum: A Hybrid Database for Transactional and Analytical Workloads, Orca: A Modular Query Optimizer Architecture for Big Data, Distributed PostgreSQL for Data-Intensive Applications, and Serializable Snapshot Isolation in PostgreSQL.