The Stata Center was designed by Frank Gehry and was completed in 2005. The Stata Center houses CSAIL (The MIT Computer Science and Artifical Intelligence Laboratory) and LIDS (The MIT Laboratory for Information and Decision Systems). Some of my favorite pictures of the Stata Center were taken during construction. (I’m a member of CSAIL)

The OpenSQL Camp will be held on the first floor of the Stata Center where we’ve reserved four classrooms (with capacities 318, 90, 60, and 60 respectively). We expect to have access also to the social areas along the interior “student street”.

The OpenSQL Camp has been running since 2008 in an “unconference format” with a blend of spontaneity and planning, no registration fees, and is open for all essentially on a first-to-sign-up basis (the sign up sheet isn’t up yet, however). The unconference typically includes seminars, hackathons, freeform discussions, and socializing with discussion topics including MySQL, PostgresQL, NoSQL, Maria, Drizzle, Storage Engines, the implications of new hardware and data structures technologies, and “soft” topics such as the formation of the Open Database Alliance, as well as anything else people want to talk about.

Mid October is near the peak of the fall foliage season. We are trying to get the conference organized early so that people will be able to find reasonable accomodation and travel arrangements.

It looks like Sheeri and I will be doing a lot of the legwork to organize this event, but this event can succeed only if many people pitch in to make it work.

I hope to see you there!

You can find this talk and other mostly technical material at http://tokutek.com/technology/.

You can find this talk and other mostly technical material at http://tokutek.com/technology/.

This research was funded in part by the National Science Foundation.

This is a whiteboard presentation and is less well developed than the talk I gave a the MySQL conference (I posted those slides two days ago. But since it includes audio it may be easier to understand.

This talk presents the data structure underlying the TokuDB storage engine for MySQL.

You can find this talk and other mostly technical material at http://tokutek.com/technology/.

The ignite talk Wednesday at 7pm is “What Is a Performance Model for SSDs?“

The ignite talk is a 5-minute talk at tonight’s Ignite MySQL session organized by Brian Aker. I’ll present some performance measurements on the Intel X25E SSD. The bottom line is that although I can get the 3,300 random 4KB writes per second, as the spec sheet advertises, I cannot seem to get more than about 11,000 reads per second, although the spec sheet says I should get 35,000.

The 1-hour talk Thursday at 10:50am is “How TokuDB Fractal Tree Indexes Work.“

This talk about Tokutek’s Fractal Trees. I’ll explain how Fractal Trees work, and show why they can get one to two orders of magnitude speedup on insertions compared to B-tree indexes. The talk is about data structures and algorithms, but I think it should be easy for everyone to understand. If you want to know why Fractal Trees are fast, this is the talk for you.

Tokutek’s GA product uses fractal trees to deliver high-performance while maintaining rich indexes.

The first talk is a 5-minute talk at tonight’s Ignite MySQL session organized by Brian Aker. I’ll present some performance measurements on the Intel X25E SSD. The bottom line is that although I can get the 3,300 random 4KB writes per second, as the spec sheet advertises, I cannot seem to get more than about 11,000 reads per second, although the spec sheet says I should get 35,000.

My second talk is tomorrow (Thursday) at 10:50am, where I’ll talk about Fractal Trees. I’ll explain how Fractal Trees work, and show why they can get one to two orders of magnitude speedup on insertions compared to B-tree indexes. The talk is about data structures and algorithms, but I think it should be easy for everyone to understand. If you want to know why Fractal Trees are fast, this is the talk for you. Thanks to Mark Callaghan and his facebook fans for supporting my talk. The talk was finally accepted yesterday!

• A linear model where the miss rate is (1-C/D)/50, and
• A inverse-proportional model where the miss rate is D/(1000C).

He seemed happy (and maybe surprised) that that the linear model is a bad match and that inverse-proportional model is a good match. The linear model is the one that would make sense if every page were equally likely to have a hit.

I’ll argue here that it’s not so surprising. Suppose that miss rate has a heavy-tailed distribution, such as Zipf’s law. An example of a Zipf’s-law distribution would be if the most frequently accessed cache block accounts for X of the accesses, the second most frequent accounts for X/2 of the accesses, the third most frequent accounts for X/3 of the accesses, and so forth with the ith most frequently accessed block accountig for X/i of the accesses

What miss rate should we expect? Essentially in this distribution, if you look at the number of misses in the first N blocks, it’s half the number of misses found in the next N blocks. Thus, we would expect the miss rate to be proportional to 1/C, where C is the cache size. That matches Mark’s experiment.

This simple heavy-tailed distribution shows up all over the place, and is often a good model for this kind of system. For example, I would expect one were to collect the top page returned for every google query, the frequency of page hits follows this distribution. A more frequently cited example is the frequency distribution of words in a natural language. Such a distribution probably controls the query cache too.

One shortcoming of iiBench is that iiBench assumes that all inserted index values are equally likely, leading to a linear miss-rate model. Since TokuDB’s advantage on insertions is related to the cache miss rate, reducing the miss rate will tend to make InnoDB look better, reducing TokuDB’s advantage. Thus InnoDB’s miss rate probably isn’t as bad on real data as iiBench suggests. That probably explains why iiBench shows a 200x advantage for TokuDB, but when talking to customers the advantage is often more like 10x or 20x. It seems clear to me that iiBench would serve us better if it had the option of generating data according to Zipf’s law. (Since I helped design iiBench, I have no qualms about criticizing it.)

Here’s a little game that helps show why heavy-tailed distributions are fun to think about. This game comes from the St. Petersberg Paradox. The game is a lottery in which I’m going to pay you some money, and the amount of money is a function of a random variable. The payoff schedule is that I’ll pay you one dollar half the time, I’ll pay you two dollars a quarter of the time, I’ll pay you $4 with probability 1/8, and in general I’ll pay 2ⁱ dollars with probability 2^-i-1. How much would you pay to buy one of these lottery tickets? (For an analysis, see Wikipedia).

One of my objectives for the camp is to make progress on a universal storage engine API, to make it possible to use the same storage engines in MySQL, PostgreSQL, Ingres, or any other database. I’m also looking forward to hearing other people’s great ideas.

After OpenSQLcamp, I’ll be attending Supercomputing’09. Supercomputing and database hardware technology seems to be converging. Many of the fastest databases today look like a supercomputer with disks attached. Will there be other kinds of convergence? For example, what kind of convergence will we see between multicore computing and cluster computing? Today we program multicore machines very differently from clusters. I think in the future that difference will vanish.