Integración de subsistemas

You now have a table design that can answer two out of the four questions on the ear- lier list. You haven’t yet ascertained whether it answers the other two questions. Also, you haven’t defined your write patterns. The questions so far define the read patterns for the table.

From the perspective of TwitBase, you expect data to be written to HBase when the following things happen:

■ A user follows someone

■ A user unfollows someone they were following

Let’s look at the table and try to find places you can optimize based on these write pat- terns. One thing that jumps out is the work the client needs to do when a user follows someone new. This requires making an addition to the list of users the user is already following. When TheFakeMT follows one more user, you need to know that the user is number 5 in the list of users TheFakeMT follows. That information isn’t available to your client code without asking the HBase table. Also, there is no concept of asking

HBase to add a cell to an existing row without specifying the column qualifier. To solve that problem, you have to maintain a counter somewhere. The best place to do that is the same row. In that case, the table now looks like figure 4.3.

The count column gives you the ability to quickly display the number of users any- one is following. You can answer the question “How many people does TheFakeMT

follow?” by getting the count column without having to iterate over the entire list. This is good progress! Also notice that you haven’t needed to change the table defini- tion so far. That’s HBase’s schema-less data model at work.

Adding a new user to the list of followed users involves a few steps, as outlined in figure 4.4.

2:Olivia count:2

count:4 TheFakeMT 1:TheRealMT 2:MTFanBoy 3:Olivia 4:HRogers

TheRealMT 1:HRogers

follows

Figure 4.3 The follows table with a counter in each row to keep track of the number of users any given user is following at the moment

Code for adding a new user to the list of followed users looks like this:

Get g = new Get(Bytes.toBytes("TheFakeMT"));

g.addColumn(Bytes.toBytes("follows"), Bytes.toBytes("count");

Result r = followsTable.get(g);

byte[] count_bytes = r.getValue(Bytes.toBytes("follows"), Bytes.toBytes("count"));

int count = Bytes.toInteger(count_bytes); count++;

String newUserFolowed = "MTFanBoy2";

Put p = new Put(Bytes.toBytes("TheFakeMT"));

p.add(Bytes.toBytes("follows"), Bytes.toBytes(count), Bytes.toBytes(newUserFollowed)); p.add(Bytes.toBytes("follows"), Bytes.toBytes("count"), Bytes.toBytes(count)); followsTable.put(p);

As you can see, keeping a count makes the client code complicated. Every time you have to add a user to the list of users A is following, you have to first read back the count from the HBase table, add the next user, and update the count. This process smells a lot like a transaction from the relational database systems you’ve likely worked with.

2:Olivia count:2

count:4 TheFakeMT 1:TheRealMT 2:MTFanBoy 3:Olivia 4:HRogers

TheRealMT 1:HRogers

follows

Client code: Get current count.

1 2

TheFakeMT : follows: {count -> 4} TheFakeMT : follows: {count -> 5}

3

TheFakeMT : follows: {5 -> MTFanBoy2, count -> 5}

count:5 2:Olivia count:2

5:MTFanBoy2 TheFakeMT 1:TheRealMT 2:MTFanBoy 3:Olivia 4:HRogers

TheRealMT 1:HRogers

follows

4 Row that needs to be updated

Increment count Add new entry 1

Update count. Add new entry.

Write the new data to HBase. 2

3 4

Figure 4.4 Steps required to add a new user to the list of followed users, based on the current table design

Get current count from table

Increment count and put in new entry

91

How to approach schema design

Given that HBase doesn’t have the concept of transactions, this process has a couple of issues you should be aware of. First, it isn’t thread-safe. What if the user decides to fol- low two different users at the same time, maybe using two different browser windows or different devices? That’s not a common occurrence, but a similar effect can hap- pen when the user clicks the Follow button for two different users quickly, one after the other: the threads processing those requests may read back the same count, and one may overwrite the other’s work. Second, what if the client thread dies halfway through that process? You’ll have to build logic to roll back or repeat the write opera- tion in your client code. That’s a complication you’d rather avoid.

The only way you can solve this problem without making the client code complicated is to remove the counter. Again, you can use the schema-less data model to your advan- tage. One way to do it is to move the followed user ID into the column qualifier. Remem- ber, HBase stores only byte[], and you can have an arbitrary number of columns within a column family. Let’s use those properties to your advantage and change the table to look like figure 4.5. You put the followed user’s username in the column qualifier instead of their position on the list of followed users. The cell value now can be anything. You need something to put there because cells can’t be empty, so you can enter the num- ber 1. This is different from how you would design tables in relational systems.

TIP Column qualifiers can be treated as data, just like values. This is differ- ent from relational systems, where column names are fixed and need to be defined up front at the time of table creation. The word column can be con- sidered a misnomer. HBase tables are multidimensional maps.

The simplicity and flexibility of HBase’s schema allows you to make such optimizations without a lot of work but gain significant simplicity in client code or achieve greater performance.

With this new table design, you’re back to not having to keep a count, and the cli- ent code can use the followed user ID in the column qualifier. That value is always unique, so you’ll never run into the problem of overwriting existing information. The code for adding new users to the followed list becomes much simpler:

String newUserFollowed = "MTFanBoy2";

Put p = new Put(Bytes.toBytes("TheFakeMT")); p.add(Bytes.toBytes("follows"), Bytes.toBytes(newUserFollowed), Bytes.toBytes(1)); followsTable.put(p); Olivia:1 MTFanBoy:1 TheFakeMT Olivia:1 HRogers:1 HRogers:1 TheRealMT:1 TheRealMT follows

Figure 4.5 Cells now have the followed user’s username as the column qualifier and an arbitrary string as the cell value.

New user ID in column qualifier; cell value is 1

The code for reading back the list changes a little. Instead of reading back the cell val- ues, you now read back the column qualifiers. With this change in the design, you lose the count that was available earlier. Don’t worry about it right now; we’ll teach you how to implement that functionality in the next chapter.

TIP HBase doesn’t have the concept of cross-row transactions. Avoid designs that require transactional logic in client code, because it leads to a complex client that you have to maintain.