Secondary indexes act the same way in TokuDB as they do in InnoDB. They store the defined secondary key, and the primary key as a pointer to the rest of the row. So, say the table foo has the following schema:

create table (a int, b int, c int, primary key (a), key(b));

And we did:

insert into foo values (1,10,100),(2,20,200);

Logically, there is one dictionary that stores all the data (this is the clustered primary key). Let us call it the main dictionary:

key  value
1    10,100
2    20,200

And there is another dictionary for the secondary key that stores the column ‘b’ and the primary key, ‘a’:

key  value
10   1
20   2

For secondary indexes to work properly, there must be a one to one correspondence between elements in the secondary index and in the primary index. If this correspondence is broken, then the table is corrupt.

Now suppose we were to execute:

replace into foo values (1,1000,1000);

This does:

• in main dictionary, overwrite the value of key ’1′ and value ’10,100′ with key ’1′ and value ’1000,1000′.
  
• in secondary dictionary, remove the key ’10′ with value ’1′.
  
• in secondary dictionary, insert the key ’1000′ and key ’1′.
  

Notice that we cannot perform the second step unless we know the content of the existing row that is being replaced. Learning the content of the existing row requires a lookup in the main dictionary, which incurs a disk seek.

So, when executing “replace into” or “insert ignore” on tables with secondary keys, all engines must still incur a disk seek on the primary dictionary to learn where associated elements are in a secondary index, whereas if no secondary keys exist, then TokuDB’s fractal trees can avoid this disk seek.

Even with secondary indexes, fractal tree indexes are preferred. B-trees still incur additional disk seeks on insertions into secondary indexes that fractal trees do not. However, with no secondary indexes, fractal trees can do away with the mandatory disk seek whereas B-trees do not.

The reason “insert ignore” and “replace into” can be made fast with TokuDB’s fractal trees is that the semantics of what to do in case a duplicate key is found is simple. In one case, you ignore, and in the other, you overwrite. With specific tombstone messages defined for these simple semantics, we defer the uniqueness check to a more opportune time.

The semantics of “insert … on duplicate key update” are not simple:

• if the primary (or unique) key does not exist, insert the new row
  
• if the primary key does exist, perform some update as defined in the SQL statement
  

The problem is we do not have a way of encoding the SQL update function into a message, the way we are able to encode “replace into” as an ‘i’ and “insert ignore” as an ‘ii’. If we did, we could similarly make “insert … on duplicate key update” fast.

I am not claiming that this is not theoretically possible, just that the storage engine API in MySQL does not allow for the encoding of updates as messages. Instead, what MySQL does is the following:

• call handler::write_row to attempt an insertion, if it succeeds, we are done
  
• if handler::write_row returns an error indicating a duplicate key, outside of the handler, apply the necessary update to the row
  
• call handler::update_row to apply the update
  

The storage engine API does not have any access to the function that applies an update to the existing row. This is why the storage engine has no way of encoding any SQL update function (even some simple ones, such as “increment column a”).

So, in the meantime, to implement these semantics, B-trees and Fractal Tree data structures both:

• look up the primary (or unique) key to verify existence
  
• take the appropriate action based on whether the primary (or unique) key exists
  

The first step incurs a disk seek on large data sets with an ad-hoc primary (or unique key). And that is why it is slow.

So, the moral of the story is this. In MySQL, “insert … on duplicate key update” is slower than “replace into”. Although the sematics are slightly different in the case where the primary key is found (the former is defined as an update, whereas the latter is defined as a delete followed by an insert), if possible, the simpler semantics of “replace into” allow it to be faster than “insert … on duplicate key update”.

The semantics of “insert ignore” are similar to that of “replace into”:

• if the primary (or unique) key does not exist: insert the new row
  
• if the primary (or unique) key does exist: do nothing
  

B-trees have the same problem with “insert ignore” that they have with “replace into”. They perform a lookup of the primary key, incurring a disk seek. We have already shown how fractal trees do not incur this disk seek for “replace into”, so let’s see how we can avoid disk seeks with “insert ignore”.

The only difference with “replace into” is when the primary (or unique) key exists, instead of overwriting the old row with the new row, we disregard the new row. So, all we need to do is tweak our tombstone messaging scheme (that we use for deletes and “replace into”) so that when “insert ignore” commands do not overwrite old rows with new rows. Similar to deletes and replace into, with this scheme, “insert ignore” can be two orders of magnitude faster than insertions into a B-tree.

Here is what we do. We insert a message into the fractal tree, with a new message “ii”, to signify that we are doing an “insert ignore”. The only difference between this message and the normal “i” message for insertions is what we do on queries and merges. On queries, if the message is an “ii”, then the value in the LOWER node is read, and not the higher node. On merges, if the higher node has a message of “ii”, the value in the LOWER node takes precedence over the value in the higher node.

Let’s look at an example that is similar to what we looked at for “replace into”:

create table foo (a int, b int, primary key (a));

Suppose the fractal tree for this table looks as follows:

- 

- -

- - - -

....

(i (1,1)) (i (2,2)) (i (3,3)) (i (4,4)) ... (i (1000,1000)) ... (i (2^32, 2^32))

The ‘i’ stands for insertion message. Now suppose we do:

insert ignore into foo values (1000, 1001).

With fractal trees, we insert (ii (1000,1001)) into the top node. The tree then looks as such:

(ii (1000,1001)) 

- -

- - - -

....

(i (1,1)) (i (2,2)) (i (3,3)) (i (4,4)) ... (i (2^32, 2^32))

So upon querying the key ’1000′, a cursor notices that (1000,1001) has a message of “ii”. If it finds another value for the key 1000 in a lower node, it reads that value, otherwise, it reads (1000,1001). Because (1000,1000) is located in a lower node, the cursor returns (1000,1000) to the user. On merges, the message in the lower node, (1000,1000) overwrites the message in the higher node, (1000,1001).

While “insert ignore” can be fast, there are caveats (indexes, triggers, replication), just as there are with “replace into”. In a future posting, I will get into some of them.

The semantics of “replace into” are as follows:

• if the primary (or unique) key does not exist, insert the new row
  
• if the primary (or unique) key does exist, overwrite the existing row with the new row
  

The slow, expensive way B-trees use to implement these semantics are:

• look up the primary (or unique key), to verify its existence
  
• if it does not exist, insert the new row, otherwise overwrite the existing row
  

The first step incurs a disk seek. That slows down performance considerably.

Instead, with TokuDB’s fractal tree data structure, we can follow a similar strategy to what we use for deletes. Recall that for deletes, we do not look up the key we wish to delete, and physically remove its data from the tree. Instead, we use tombstone messaging, or tombstone deletions, to defer the physical removal of the data to a better time. The same idea applies here. Instead of searching for the primary key, as we are forced to do with B-trees, we insert an insertion message into the fractal tree, and defer the existence check for later.

Let us look at an example. Take the following table:

create table foo (a int, b int, primary key (a));

Suppose the fractal tree for this table looks as follows:

- 

- -

- - - -

....

(i (1,1)) (i (2,2)) (i (3,3)) (i (4,4)) ... (i (1000,1000)) ... (i (2^32, 2^32))

The ‘i’ stands for insertion message. Now suppose we do:

replace into foo values (1000, 1001).

With fractal trees, we simply insert (i (1000,1001)) into the top node. The tree then looks as such:

(i (1000,1001)) 

- -

- - - -

....

(i (1,1)) (i (2,2)) (i (3,3)) (i (4,4)) ... (i (2^32, 2^32))

Similar to deletes, with this scheme, “replace into” can be two orders of magnitude faster than insertions into a B-tree.

On queries, a message in a higher node overrides messages in lower nodes. So upon querying the key ’1000′, a cursor notices that (1000,1001) is located higher than (1000,1000), and therefore returns (1000,1001) to the user. On merges, the message in the higher node overwrites the message in the lower node.

So, by using messages, fractal trees can achieve the same performance boost for “replace into” as it does for insertions. In fact, using “replace into” in the manner above is how a customer has achieved an 80x speedup under actual field conditions. The details can be found here.

Next week, I explain how “insert ignore” can similarly be fast. Also, while this shows how “replace into” can be fast (and IS fast for a lot of scenarios we see with TokuDB), there are some caveats (with indexes, triggers, and replication). I will get into those in a couple of weeks.

update foo set price=price+1 where product=toy;

Executing this statement has two steps:

• a query to find where product=toy
  
• a combination of insertions and deletions to change old rows to new rows
  

The analysis is identical to that for deletions. Just like for deletes, clustering keys make the query go fast, as explained here. In this case, the appropriate clustering key would be on ‘product’. And fractal tree data structures make the insertions and deletions go fast, as explained here and here.

So, the same story applies. Updates are a combination of queries and value changes. To make updates fast, both the query and the value changes need to be fast. With B-tree based storage engines, users may be hesitant to add indexes to speed up the query, due to the added cost of value changes in the index. With TokuDB, this tradeoff does not exist, because TokuDB uses fractal tree data structures. Using a clustering key to drive down the cost of the query, and using fractal tree indexes to keep the cost of the data changes down, leads to very fast updates.

• B-trees do insertion disk seeks. While they’re at it, they piggyback some other work on the disk seeks. This piggyback work requires disk seeks regardless.
  
• TokuDB’s Fractal Tree indexes don’t do insertion disk seeks. If we also get rid of the piggyback work, we end up with no disk seeks, and a two order of magnitude improvement.
  

So it’s all about finding out which piggyback work is important (important enough to pay a huge performance penalty for), and which isn’t.

This blog post is about one of the most straightforward operations: ad-hoc insertions with a primary key. Since the difference I’ve identified between B-tree indexes and Fractal tree indexes is the disk seeks on insertions, it may seem that there’s little to look at in this case. But the semantics of insertion into a primary key is as follows: if the primary key being inserted does not already exist in the table, insert the element, otherwise return an error notifying the user that a duplicate key exists.

So, if we have a table:

create table foo (a int, b int, primary key (a))

and we run:

insert into foo values (1000,1);

Before inserting (1000,1) into the table, the storage engine must first check to see if any row in the table where a=1000 exists. If so, return an error stating that there is a duplicate key. The requirement to return an error is VERY expensive: it requires a disk seek.

The only way a disk based storage engine can verify if a key exists is to look up the element. This is the same reasoning used for unique secondary keys here. If the primary key is ad-hoc, the lookup requires disk seeks. And disk seeks are evil.

Once again, B-trees already pay for the disk seek just for the insertion. Because the leaf node must be in memory to do the
insertion, verifying uniqueness essentially comes for free. So B-trees suffer from disk seeks, whether you do duplicate checking or not.

So how can users achieve fast performance with a Fractal tree based storage engine — as compared to B-trees, which will be slow regardless? Remove this requirement of returning an error if a duplicate is key found. It is way too expensive. MySQL has syntax to perform other actions instead of returning an error:

• replace into
  
• insert ignore
  
• insert … on duplicate key update
  

The question is, do these commands do piggyback work that requires disk seeks? If they do, performance will be slow for ANY disk based storage engine. If not, then fractal tree data structures will be achieve up a speedup of up to two orders of magnitude. In next week’s post, I examine these cases.

Let’s take a step back and analyze the work needed to be done to execute a MySQL delete statement. Suppose we have the table:

create table foo (
	id auto_increment
	a int,
	b int,
	primary key (id)
)

Say we wish to perform the following operation that deletes 100,000 rows:

delete from foo where a=1;

In MySQL, this statement executes in two steps. First, MySQL finds all the rows where a=1, via a query. The query is equivalent to “select * from foo where a=1;” Then for each row found, MySQL deletes the row by calling handler::delete_row(row). So the time to execute deletions is equivalent to the time to find the rows (T_query) plus the time to delete the found rows (T_change). Or, stated another way:

T_delete = T_query + T_change

The previous post shows why T_change is faster for fractal tree data structures than for B-trees. Let’s use that fact to see how we can make deletions fast under MySQL.

Back to the original problem at hand:

delete from foo where a=1;

For this statement, T_query is a table scan. Because no index on ‘a’ exists, every element must be processed to find where a=1. For large tables, this can be expensive. The (minor) advantage here is T_change, the cost of removing the 100,000 rows, requires no disk seeks, because rows where a=1 stay in memory as MySQL deletes them. The problem remains, we process too much data in T_query to delete some rows.

So how can we speed up a query? Indexing! Or specifically, add an index on ‘a’. The schema is now:

create table foo (
	id auto_increment
	a int,
	b int,
	primary key (id),
	key (a)
)

Now what is the cost of “delete from foo where a=1;”? Well, for TokuDB, InnoDB, and MyISAM, T_query requires roughly 100,000 disk seeks, because point queries are needed to retrieve the entire row. The key (a) is not a covering index. The (minor) advantage here (again) is that T_change has no additional disk seeks, because the query does the disk seeks necessary to bring the rows where a=1 into memory, both in the primary index (or .MYD file for MyISAM) and secondary index.

The problem remains that we are still doing about 100,000 disk seeks!

So how can we further speed up T_query? Remember, we want to make the query “select * from foo where a=1;” faster. Clustering indexes! The schema looks like:

create table foo (
	id auto_increment
	a int,
	b int,
	primary key (id),
	clustering key (a)
)

Suppose MyISAM and InnoDB supported clustering indexes (they don’t). What is their cost for performing the deletion? Well, T_query becomes much faster, because it would be a range query as opposed to 100,000 point queries. But T_change is still expensive. As explained in my last post, B-trees require disk seeks for deletion when the row is not in memory. The rows that need to be deleted in the primary key are not in memory and require disk seeks. So, for MyISAM and InnoDB, we are still stuck with at least 100,000 disk seeks. For this reason, some erroneously think “clustering keys do not help here”.

Now what about TokuDB, a storage engine that uses fractal tree data structures? T_query is also fast, because a range query is performed. But T_change is ALSO fast, because the deletions in the primary index do NOT always require disk seeks. This is where the advantage of fractal tree data structures over B-trees comes into play, allowing clustering keys to speed up a deletion procedure.

So, the moral of the story is this. Deletions are a combination of queries and value changes. To make deletions fast, both the query and the value changes need to be fast. With B-tree based storage engines, users may be hesitant to add indexes to speed up the query, due to the added cost of value changes in the index. With TokuDB, this tradeoff does not exist, because TokuDB uses fractal tree data structures. Using a clustering key to drive down the cost of T_query, and using fractal tree indexes to keep the cost of T_change down, leads to very fast deletes.

In this post, let’s examine deletions, and get an intuitive understanding for why fractal-tree data structures exhibit the same two orders of magnitude faster deletions than B-trees. In MySQL 5.1, this advantage is really eye-popping for TokuDB v. InnoDB, because InnoDB does not use its insert buffer for deletions. I understand there is a delete buffer in 5.5, which I haven’t experimented with yet.

B-trees exhibit the same weakness on deletions as they do on insertions: they need to have the appropriate leaf node in memory. For large tables, bringing the leaf node into memory often requires a disk seek. Fractal tree data structures do not have this requirement.

Before going on, a clarification. In MySQL, delete statements have two steps: queries and value changes. For instance, the statement:

delete from foo where a=1;

must first query all rows where a=1 (the first step), and then proceed to remove the rows that are found (the second step). In this post, we focus on the second step. For storage engine developers, this is the function handler::delete_row. In a future post, I will analyze the first step, tie it together with this post, and show how deletions can be fast in MySQL with TokuDB.

Back to deletions. Let’s analyze value changes. We know the contents of the row being deleted. So how can fractal tree data structures avoid an unnecessary disk seek? The answer: deletion messages (sometimes called tombstone deletes).

Suppose we have a fractal tree data structure with the following elements inserted: (1), (3),…(999). Up until now, we shown the fractal tree as such.

-

- -

- - - -

...

1 3 5 7 9 ... 999

In reality, the elements stored are not just keys, but rather (message, key) pairs. The message may be one of two operations: insertion or deletion. We represent an insertion with the message ‘i’. So, the fractal tree looks more like this:

-

- -

- - - -

...

(i,1) (i,3) (i,5)... (i,999)

To delete an element, for example (5), we insert a deletion message into the tree, marking it with a ‘d’. So, after deleting (5), the fractal tree data structure looks like this:

(d,5)

- -

- - - -

...

(i,1) (i,3) (i,5)... (i,999)

With this scheme, deletions are as fast as insertions, which is to say two orders of magnitude faster than insertions or deletions into a B-tree.

On queries, a message in a higher node overrides messages in lower nodes. So upon querying (5), a cursor notices that (d,5) is located higher than (i,5), and therefore the key (5) does not exist in the fractal tree data structure. On merges, the deletion message and insertion message cancel each other out, and space is reclaimed.

So, by using deletion messages and treating deletions like insertions, fractal tree data structures can achieve the same performance boost (two orders of magnitude) over B-trees.

Now that insertion disk seeks are out of the way (and I don’t want to shortchange the importance of getting rid of these seeks!), let’s look at other places where databases perform seeks, and see if we can get rid of them. Over my next couple of posts, I will look at several use cases and analyze whether disk seeks are required. If disk seeks are required, then performance will suffer on large amounts of data, for TokuDB and any other disk-based storage engines.

If disk seeks are not required, things get interesting. Removing these unnecessary disk can speed up a database as long as all disk seeks in a command execution are removed. Since TokuDB eliminates seeks on insertions, we should avoid disk seeks altogether. Since B-trees induce disk seeks on ad-hoc insertions, cleaning up the remaining disk seeks has limited utility.

For today, let’s look at a simple use case that may be obvious: insertions on secondary indexes v. unique secondary indexes. Take the following table:

Create Table: CREATE TABLE `t` (
  `a` int(11) NOT NULL AUTO_INCREMENT,
  `b` int(11) DEFAULT NULL,
  `c` int(11) DEFAULT NULL,
  PRIMARY KEY (`a`),
  UNIQUE KEY `b_unique` (`b`),
  KEY `b_norm` (`b`)
)

Suppose most of the table resides on disk.

Now I run:

insert into t (b) values (1000);

Are there any mandatory disk seeks involved?

When inserting into the fractal tree for the primary dictionary, we use an auto increment value, so insertions are sequential. Insertions run really fast, because a disk seek is usually not mandatory (disk seeks eventually happen when blocks get full, but they do not occur on EACH insertion).

Let’s look at inserting into b_unique and b_norm visually. Take the following identical fractal trees for ‘b_unique’ and ‘b_norm’:

-

- -

- - - -

...

1, 3, 5, 7, ..., 999, 1001, 1003, ...

To insert into ‘b_norm’, the fractal tree can simply insert (1000) in the top node. To insert into ‘b_unique’, the fractal tree must first search for 1000, verify that it is not between 999 and 1001, and then insert into the top node. This lookup causes a disk seek and slows down the insertion.

Note that because B-trees require disk seeks to do insertions anyway, some operations come with no additional cost in B-trees. Uniqueness checks are one such example. As a result, some B-tree users may not think twice about making secondary keys unique (after all, unique keys can help the query optimizer). Fractal tree data structures, on the other hand, incur a huge cost for a uniqueness check.

So, the moral of this story is if you care about insertion performance and avoiding disk seeks, try to avoid unique secondary keys, and go with normal secondary keys. Otherwise, fractal tree data structures will be just as slow as B-trees, and not two orders of magnitude faster.

TokuDB fractal tree data structures deliver insertion performance benefits over traditional B-trees by performing fewer disk seeks on random insertions (in effect, turning random I/O into sequential I/O). This is why TokuDB typically outperforms InnoDB on insertion workloads, because TokuDB’s random insertions into secondary indexes is much faster than InnoDB’s insertions — up to two orders of magnitude faster.

So let’s consider the first place where TokuDB avoids a disk seek as opposed to a B-tree. On an insertion, a B-tree seeks need to have the appropriate leaf node in memory. For large tables, this requires a disk seek. A detailed view of what Fractal Tree indexes do is available in
this video. Here is an intuitive way to understand why fractal tree indexes are fast at insertions (an sample fractal tree is in the figure below). Seven out of eight (87.5%) insertions will be in the top three nodes, which will always be in memory. So, at least 87.5% of insertions will be strictly in-memory, doing no seeks, and in practice this percent is even higher. Now, when the fractal tree does write to disk, it will write nodes of greater depth. These nodes are written at disk bandwidth, and so the performance is not limited by disk seek time. This is why fractal trees indexed insertions are so fast.

Simple Fractal Tree

Insertions, therefore, are an operation that do not require a disk seek. There are some data structures that perform a disk seek (B-trees) and others that don’t (Fractal Tree indexes). Other operations require a disk seek, no matter what data structure you use, e.g. point queries, uniqueness checks, etc. In a previous post, I talked about how to avoid disk seeks by replacing point queries from secondary indexes to the primary table by using a clustering index.

So now that TokuDB gets rid of insertion disk seeks, it’s time to get rid of as many as possible. In the coming weeks, I’ll be posting a series of blogs about other cases where disk seeks can be avoided.