New in TokuDB v4.1 includes important improvements, most notably support for SAVEPOINT and improved Fast Loader performance (introduced in v4.0).

This new release builds on TokuDB’s unique combination of capabilities:

• 10x-50x faster indexing for faster querying
• Full support for ACID transactions
• Short recovery time (seconds or minutes, not hours or days)
• Immunity to database aging to eliminate performance degradation and maintenance headaches
• 5x-15x data compression for reduced disk use and lower storage costs

Because of its high indexing performance and transaction support, TokuDB is well suited to Web applications that must simultaneously store and query large volumes of rapidly arriving data, including:

• Logfile Analysis
• High-speed Webcrawling
• Real-time clickstream analysis
• Social Networking
• eCommerce Personalization

• Faster indexing of sequential keys.
• Faster bulk loads on tables with auto-increment fields.
• Faster range queries in some circumstances.
• Added support for InnoDB.
• Upgraded from MySQL 5.1.30 to 5.1.36.
• Fixed all known bugs.

About TokuDB

TokuDB for MySQL is a storage engine built with Tokutek’s Fractal Tree™ technology. TokuDB provides near seamless compatibility for MySQL applications. Tables can be individually defined to use TokuDB, MyISAM, InnoDB® or other MySQL-compliant storage engines. Data is loaded, inserted, and queried using standard MySQL commands, with no restrictions or special requirements. Fractal Trees can index data up to 50 times faster than traditional database technologies, enabling near real time analysis on large volumes of rapidly arriving data.

Notice: Tokutek and Fractal Tree are trademarks or registered trademarks of Tokutek, Inc. MySQL is a registered trademark of Sun Microsystems, Inc. InnoDB is a registered trademark of Oracle Corporation.

Please see Schlomi’s post for details, but I can summarize a few points, partly because I need some vocabulary for the points I’d like to make below. Scholmi notes that there are two main features determining the depth of a B-tree (or B+-tree):

1. The number of rows in the database. We’ll call that N.
2. The size of the indexed key. Let’s call B the number of key that fit in a B-tree node. (Sometimes B is used to refer to the node size itself, rather than the number of keys it holds, but I hope my choice will make sense directly.)

Given these quantities, the depth of a B-tree is log_B N, give or take a little. That’s just (log N)/log B. Now we can rephrase Scholmi’s point as noting that small keys means a bigger B, which reduces (log N)/log B. If we cut the key size in half, then the depth of the B-tree goes from (log N)/log B to (log N)/log 2B (twice as many keys fit in the tree nodes), and that’s just (log N)/(1+log B).

Let’s put some numbers in there. Say you have a billion rows, and you can currently fit 64 keys in a node. Then the depth of the tree is (log 10⁹)/ log 64 ≈ 30/6 = 5. Now you rebuild the tree with keys half the size and you get log 10⁹ / log 128 ≈ 30/7 = 4.3. Assuming the top 3 levels of the tree are in memory, then you go from 2 disk seeks on average to 1.3 disk seeks on average, for a 35% speedup.

That’s a nice savings, assuming, of course, that the new, smaller key you used is as useful for queries. And the time for an insertion into a B-tree enjoys the same savings. An insertion is O((log N)/log B) — about the same as point queries, up to a constant, but in any case, you’d still get a similar speedup.

What about range queries? Here things aren’t so sensitive to the depth of the tree. In a range query, you seek to a leaf that has your first row, and then you iterate throw all rows, jumping to sibling leaves as necessary, until you reach your ending row. The initial time to seak to the first leaf — the part that depends on the depth of the tree — is typically in the noise.

There is, however, a more subtle effect going on here. If you have fast insertions (say, by using smaller keys), you can afford to keep more indexes. And having the the right index around can make a huge difference to a range query: not having an index can mean that a range query over a small number of rows can become a full table scan. I’ve seen a 5-order-of-magnitude speedup in such queries once the right index is added.

Here’s a simple example of of what I’m talking about, using iiBench. [Details available http://tokutek.com/products/iibench/.]

mysql> show variables like 'query_cache_type';
+------------------------------+---------+
| Variable_name                | Value   |
+------------------------------+---------+
| query_cache_type             | OFF     |
+------------------------------+---------+
7 rows in set (0.00 sec)

mysql> select count(*) from tokudb_1B_noindex where customerid = 50000;
+----------+
| count(*) |
+----------+
|    10014 |
+----------+
1 row in set (11 min 17.55 sec)

mysql> alter table tokudb_1B_noindex add index cust_idx (customerid);

mysql> select count(*) from tokudb_1B_noindex where customerid = 50000;
+----------+
| count(*) |
+----------+
|    10014 |
+----------+
1 row in set (0.31 sec)

We issue a query on a table with no secondary indexes, and then again after building an appropriate secondary index — with query caching off, of course. The speedup is from more than 11 minutes, to 0.31 seconds, in this case a factor 2185x faster.

So for range queries, it all boils down to having the right set of indexes. Faster insertions means you can keep more indexes, so the savings on insertion times mentioned above can help: if you can afford to keep 4 indexes on your live data using big keys, you can afford around 6 indexes if you use keys half the size. [Well, it's more complicated than that, because this assumes you are keeping 3 levels from each for the index in memory. Caveat indexor.]

Once again, this assumes that you can find small keys to use for all your indexes. And the fastest indexes for range quires are covering indexes, which tend to have large keys. [Stay tuned for a tokuview posting on covering indexes.] For non-covering indexes, we’d have to do lots of point queries into the main table. Yikes!

So we are caught in a bind. We’d like build secondary indexes to make range queries fast, and we’d like the secondary indexes to be covering for our important queries, but covering indexes are slow to build using B-trees, partly because the keys are large and the trees are deep, but for a bunch of other reasons as well, reasons I’ll cover in another blog post.

What to do? Of course, my solution is to use TokuDB for MySQL, which has much faster insertions. TokuDB isn’t built on B-trees, but rather on Fractal Trees, so the whole large-key question goes away. See the Technology Brief on Tokutek’s Technology Page for a discussion of the insertion performance of Fractal Trees and TokuDB.

We are focusing on the storage engine (a la MySQL) level, and we are looking at a database on a single disk — the one we are using for illustration is the 1TB Hitachi Deskstar 7K1000. It has a disk seek time 14ms and transfer rate of around 69MB/s. [See tomshardware.com] We insert random pairs, each 8 bytes. So that’s 62.5 billion pairs to fill the disk, and at 4KB-size blocks, that 2^28 leaves (= 2^40 bytes / 2^12 bytes/leaf).

Now, my analysis requires each insertion to induce a disk seek. Suppose we do something clever with main memory. After all, we have this main memory hanging around. It should be possible to buffer up some insertions, and once we fill up main memory, insert key/value pairs that belong on the same leaf. Thus, fetching a leaf will be amortized over the number of rows that we expect to insert into a leaf.

So what’s the best we could do? If we could somehow figure it out, the best we could do would be to take the leaf that’s going to receive the most insertions, fetch that one in, and do all the insertions from memory into that one. This assume we won’t overflow that leaf and have to rebalance, etc. We’re just going to count the cost of fetching in a leaf.

At first, you’d get a lot of advantage, since you start with an empty database with no leaves, so the first set of insertions (256 of them, in fact) will all go to the first leaf. So let’s consider the situation where we’ve filled up the database half way. We’ll discount this time and just look at the time to fill up the second half of the disk. At that point, we have 2^27 leaves.

Now, let’s suppose we have 1GB = 2^30 bytes of memory. It really won’t make an appreciable difference, it turns out, if we have as much or half as much memory. With this much memory, we can get 2^27 rows into memory.

Now we have what is know as a ball and bins problem. Think of the leaves as a set of 2^27 bins, and the in-memory rows as 2^27 balls. Since the insertion keys are random, we can think of each row/ball as picking a random leaf/bin where it goes. Now, what is the expected maximum number of balls in any single bin? The answer is well known — ln (2^27) / lnln (2^27) = 18.7/2.9 = 6.4. So we expect the very best leaf to typically get 6.4 rows inserted.

So now the maximum speedup we could expect is 14 years/6.4 = 2.1875 years. Now, that’s a lot better, but in fact, it’d be hard (and probably impossible) to get anywhere near that performance. Why? First, there’s the question of how you know which rows go on the same leaf without actually having the leaves. You could keep a B-tree in memory of the minimum key in each leaf, but this would fill half of memory. Second, we didn’t count the time to fill the first half of memory. We could do a finer analysis of the time needed to load the last half of the database, plus the preceding 1/4 rows, plus the preceding 1/8, etc., until you have a small enough database to fit in memory, and that ends up at around 4 years.

Finally, and most importantly, the factor of 6.4 is far too generous. The average number of balls in each bin is 1, and we only expect a single bin to get above 6 balls. Then we immediately empty that bin (by inserting the rows into the database). Now we throw some more rows in memory (balls into bins), and they are very unlikely to produce yet another 6-ball bin. That are almost certainly going to land in bins that are either empty or have a single ball. Not to get too technical, the number of bins with a large number of balls drops exponentially as the number of balls increases, until we get down to single bin with more than 6 balls. As you empty these very full bins, it takes a large number of new balls to get more bins that have a lot of balls.

Now, if I were still an academic, I’d take the time to study this recycling balls-and-bins problem, but instead I have a company now, and theorems of this time are not really on any critical path! Still, one of my partners, Michael Bender, suspects that the answer is going to be lnln 2^27 = 2.9. I’d be very surprised if the answer is any more than this.

So a realistic number for going through all this work is that it’ll take 28 years/2.9 = 9.7 years. Your disk will still die before you fill it.

My recommendation: don’t use this heuristic!

Sorting before inserting?

What about another heuristic? How about, when memory fills, sort the in-memory rows, then merge the sorted order of the database with the sorted order of the in-memory inserted rows. To compute the time, we need to figure out how good a job we do of keeping the leaves of the tree in order on the disk. If we keep them all nicely in order and packed — which is to say, if every time we merge, we rebuild the index from scratch, we get the following: The bandwidth time for one batch of memory is 1GB/69MB/s = 14.5s. Discounting sort time, the first memory-full of rows takes 14.5s, the second batch takes 14.5s * 2, then 14.5s * 3, up until 14.5s * 1000, for the last batch before filling memory. That gives a total time of 14.5s * 1000 * 1001/2 = 7.3 million seconds = 84 days. What an improvement! Still, it’s almost 3 months to index the data, compared to the 10 hours it takes to log the data. So your yield is 0.2% of bandwidth.

Let me point out that we’ve made some pretty big assumptions here. For one thing, we’ve assumed that you are rebuilding the index from scratch each time. It’s much faster to build an index on data given in sequential order, but surely it’s not free! So the 3 months to build the index this way is a gross underestimate. If you don’t rebuild the index each time, your database ages. It’s pretty tricky to estimate the time it takes to build the index this way, but a very nasty bit of estimating later, I get that it takes at least 41 years (my main assumption is that once the database is half full, it is fully aged). Oy!

Bigger Batches

By now you’re saying to yourself that you can get a 1 TB database built in much less than 3 months. If you have all the data ahead of time, you can sort the data in roughly 40 hours, plus another at least 10 hours to build the index (assuming our magical no-overhead indexer for sorted data). So you get a tradeoff between

• Waiting for all the data to come in (and get stale), then 2 days to build the index.
• Insert as you go at a limited rate, so that the entire project takes 3 months. Throughout the process, all inserted data is available for querying.
• Anything in between.

Conclusions

And we’re back to the old tradeoff: you can get faster insertion if you are willing to let enough of your data get stale. I want a storage engine that’s fast for insertions, fast for range queries and doesn’t age. So does everyone else. This would end 90% of the kludges of how DBAs deal with database (ok, so I made up the 90%, but it’s probably not far off, is it?).

Next, we’re going to look at Cache Obliviousness, and see if we can get any closer to this impossible dream.

So now, for a bit of math:

Brodal and Fagerberg showed in 2003 that there’s a tradeoff between insertions and queries. The insertions they consider are lenient. Well, any lower bound for lenient is a lower bound for strict, but they also gave upper bounds, so it matters. Also, they don’t know from lenient, but if you look at their upper bound, they are implementing lenient insertions. The queries they consider are, unfortunately, point queries. That’s too bad for us, because we’ve already seen that point queries are just too slow to be of interest on hard disks.

Still, they have matching upper and lower bounds, so let’s see what they say:

They show that, for any p between 0 and 1, if Query time < 1/p * log_B N then Insertion time > (1/pB^{1-p} log_B N). And visa versa, that is, we get a lower envelope of Query, Insertion times that are possible. Let try p=1. This says that if Query time is at most log_B N (which a B-tree gets), then insertion time is at least log_B N (also what a B-tree gets). So a B-tree is optimal for one point of this tradeoff.

More interestingly, when p=1/2, the query time doubles to 2log_B N. The insertion time is then no less than 2/sqrt{B} * log_B N. Wow. Taking B=4K, sqrt{B} = 64, and we get something that’s 32 times faster than a B-tree. But we’re taking about a lower bound. Maybe it’s not a very good lower bound. Maybe there’s no data structure that does this well.

It turns out there is (please see their paper for more details). Consider what that means: you can speed up insertions by a factor of 32 by slowing down queries by a factor of 2. That’s a pretty favorable tradeoff! Unfortunately, their solution doesn’t do range queries any better than B-trees — in fact, it’s somewhat more cumbersome, though no worse asymptotically than B-trees.

Conclusions

The Brodal & Fagerberg is quite nice. They’ve really captured something interesting about storing data on disk. However, it’s not the result we need, because the tradeoff that I think is interesting “in the field” is one between insertions and range queries, the kind of thing that translates to a MySQL database. But range queries depend on locality on secondary storage, and the typical mathematical models in use for thinking about external memory (the Disk Access Model and the Cache-Oblivious Model, about which more later) don’t capture this sort of locality. They focus on intra-block locality, not inter-block locality. I don’t know of any theoretical results that address the problem we have been considering.

So if you can’t do theory, you do hacks. Next time, we’ll start looking at some of the hacks that people use (or claim to use) to speed up their databases. Which of them play nicely with disks?

So how do we match the bandwidth bound for queries and updates? I already mentioned in passing how to do this, but let’s look more closely.

Fast Updates

The way to get fast updates is to log them. You can easily saturate disk bandwidth by writing out the insertion, deletion and update requests with no index.

A query now will typically start by sorting the data. Even a point query requires looking at all the data, but a range query requires looking at all the data log times (in order to sort it), or using a large amount of extra storage. Let’s focus on sorting for range queries.

For queries that range over all the data, sorting may be tolerable (after all, you’d be doing a lot better in terms of bandwidth than a B-tree gets over the same query!). This solution does, however, have some drawbacks. First, for smaller queries, where you only look at x% of the data, the fraction of bandwidth you are getting is x/log N, for appropriate base of the log. You can beat this somewhat if you put a lot of work into it. But you’re still going to get very little of your bandwidth.

[2 points if you've figured out that some queries don't actually require sorting the data or using lots of space. 2 more points if you figured out that end users don't want this kind of noise in their database, where some queries turn out to be unpredictably slow. A final 2 points for figuring out that almost no one wants to learn enough about a data base to be able to predict which queries will be slow.]

Fast Queries

The way to get fast queries is to lay out the data on disk carefully. That is, the best you could do would be to take all your data, sort it, and insert it in order into a sensible database (MySQL + InnoDB will do nicely) that will lay out the leaves more or less in order. Then range queries become linear scans on the disk, and you can get close to bandwidth rates. This is one of the main tricks for getting performance out of a data warehouse. Dumping and reloading data for this purpose is also one of the time-consuming things that DBAs do.

Inserting data now requires batching data, sorting it and inserting it. The time per insertion might not be so bad, but the latency is killer. It is not uncommon for data to become available for query in the data base a day or two after it comes into the system.

Conclusion

As we try to tease apart what’s fast and what’s inherently slow in a database, we get the idea that there’s some basic tradeoff between updates (even lenient ones) and range queries. Is this tradeoff inherent, or is there some way to get bandwidth speeds on both lenient updates and range queries?

Next time we’ll look at some tradeoff lower bounds and try to get closer to some answer.

Just to rehash, we are focusing on the storage engine (a la MySQL) level, and we are looking at a database on a single disk — the one we are using for illustration is the 1TB Hitachi Deskstar 7K1000. It has a disk seek time 14ms and transfer rate of around 69MB/s [See tomshardware.com] We will insert and delete random pairs, each 8 bytes. So that’s 62.5 billion pairs to fill the disk.

Strict Updates

These are the easier update types to analyze. Please review the definition of strict updates from the last blog entry. Now notice that each insertion or deletion requires a point query. For example, during an insertion, in order to determine if there’s already a row with a particular key value in the database, one must look up that key. In order to tell how many rows are removed during a delete operation (if any), one must look up the key being deleted. There’s no getting around the fact that the disk head needs to move to the location on disk of the information being returned.

Since these point queries are not the main point of what’s going on, and since many people might not realize that they are even happening, I’m going to call them cryptogets (the get because a point query is often referred to as a get at the storage engine level).

Since no strict update can be faster than a point query, filling up the disk with data takes at least 28 years. And once again, this is independent of the data structure used.

Lenient Updates

What about lenient updates? The only lower bound is the 10 hour bandwidth lower bound we’ve already talked about. Ten hours is the time it takes to write the data out sequentially on disk. And certainly, if you don’t need to return any information at the time of an update, you could just log the updates, which is to say, you could write them out sequentially to disk.

That makes queries slow, but the point is that a log is a (very simple) data structure that matches the disk-bandwidth lower bound.

What about B-trees? A B-tree index will do (at least) one disk seek per update. That means that it will take at least 28 years to finish filling the disk. This ratio is even worse than B-trees achieved for range queries.

[Yes, yes, I know, what about presorting the data before inserting? Won't that make things much faster? Yes, it will, but that's the subject of another blog entry.]

Conclusions

B-trees do a disk seek per insertion during which they swap in all the information needed to implement a strict update. They don’t get any performance benefit from implementing lenient updates. I believe that this is the reason that people don’t make the distinction between different classes of B-trees.

So far I’ve focused on lower bounds. In future blog entries, I’ll talk about algorithms for actually implementing lenient updates. Ok, here comes the punch line: lenient updates stomp all over strict updates, performancewise.

But first, we’ll look at the tradeoff between updates and range queries. Stay tuned.

Depending on the exact settings of your database, the updates give a varying amount of feedback. For example, when a key is deleted, all rows with that key are deleted (assuming the database allows duplicate keys). The normal behavior is to return the number of rows deleted. The normal behavior when deleting a key that has no corresponding rows in the database is to return an error message. On insertion, one can allow duplicate or not. In the latter case, the storage engine returns an error message if a duplication insertion is attempted.

We’ll see that the details of error messages have a profound influence on the lower-bound arguments I’ve been making (and we’ll see a bit later that they have a profound influence on how quickly you can actually implement these operations). Here, we define two classes of update operations:

Strict Updates

These are the insertions and deletions that come by default in MySQL, and many other databases.

• Insertions let you know if there’s already a row with that key in the database.
• Unique-key indices: Returns an error on the insertion of a repeated key.
• Deleting a key will tell you how many rows had that key.
• Deletion of a non-existing key will return an error.

As I say, these are the types of updates that we are all used to.

Lenient Updates

The alternative to strict updates is lenient updates. In a sense, they are equivalent, in that there’s nothing you can do with one that you can’t do with the other. It’s just going to turn out that lenient updates can be made to run much faster.

• Insertions do not let you know if there’s already a row with that key in the database.
• Unique-key indices: Overwrites with new value on the insertion of a repeated key. Does not return an error.
• Deleting a key will not tell you how many rows had that key.
• Deletion of a non-existing key will not return an error.

Conclusions

What’s the upshot? I’ve been presenting operations in terms of lower bounds that are dominated by disk seeks (slow!) versus those dominated by disk bandwidth (much faster). See if you can use this analysis to understand the behavior of these two types of updates.

Recall that the focus is on the storage engine (a la MySQL) level, and a database on a single disk — the one we are using for illustration is the 1TB Hitachi Deskstar 7K1000. It has a disk seek time 14ms and transfer rate of around 69MB/s [See tomshardware.com] Now imagine filling the disk with random pairs, each 8 bytes. So that’s 62.5 billion pairs.

Range Queries

Suppose the above data is stored in a B-tree, and that you’d like to iterate over all the data in order by key. Further suppose that the B-tree has 4KB blocks. Thus each leaf has 256 key-value pairs. The tree has around 244 million leaves. Before we figure out how long it’s going to take to look at those leaves, we have to understand how aged the B-tree is.

A B-tree that you get from loading up the data in sorted order in one fell swoop has all the leaves laid out in a relatively nice order, so that sibling leaves tend to be on the same track, and you get lots of leaves for each disk seek. As the B-tree ages (either from doing insertions & deletions, or from inserting things in a more random order to begin with), the leaves tend to be scattered all over the place. Let’s assume that the tree is well aged to see how bad things can get.

A range query over the whole data set in our scenario will give a disk seek per leaf, which would take 3.4 x 10^6 seconds (= 244 x 10^6 x 14 x 10^-3 seconds) or 39 days. For one query.

Now the question is, can you do better? The first thing to note is that I have made my argument as to how long a B-tree would take, and in particular one that’s aged and has a specific block size. In the point query case, I made an argument about *any* data structure. In this case, I can’t. I can’t say that there isn’t some other data structure that ages well, for example, and thus does better. For example, in the extreme case where the data has been sorted on disk, it can be read at full bandwidth, in which case it would take 10 hours to do a range query. That’s still a longish time, but it’s not 39 days, and it means that the B-tree example is using about 1% of available disk bandwidth.

So there are certainly ways to fix this. Use a bigger block size for the B-tree (and kill insertion performance). Sort the data before insertion into the B-tree (and kill insertion performance). The point is that unlike point queries, the natural lower bound is band width, not disk seeks, and the textbook data structure for storing data on disk has poor performance for this operation.

As I’ve already argued, range queries are the natural query type for large data sets, not least because of the argument above, that point queries can’t even be implemented on disks, but also more positively because large data is useful for large-scale market analysis, where aggregate statistics about the data set are mined. And that’s implemented by range queries.

Conclusion

Unlike point queries, where performance will only be gained by (expensive!) hardware changes, data structural changes are key for range queries. In a way, we already know this. What is a data warehouse if not a database that’s been optimized for range queries at the expense of insertion times.

In future postings, we’ll see if there are any options to data warehouse for achieving fast range queries (hint: there are). In the next posting, we’ll look at insertion rates.