Does the CAP Theorem have a Second Order?

A couple of years ago, we decided at Central 1 that our services should fall on the Availability-Partition Tolerance (AP) side of the CAP Theorem. The assertion at the time was that, at a business level, it is reasonable to accept eventual consistency if we can be always available and partition tolerant. With our old systems, we made that tradeoff all the time, and sorted out the reconciliation issues the next day.

Recently, we were working on implementing Interac Online Payments, which has a fairly complex message flow that includes the POS switching network. The details aren’t important here, but the net result was that we needed to handle a scenario where the first part of a transaction might come to one data center, and the second part would come to the other. Conceptually, it was a bit like the propose and commit in 2-phase commit coming to different data centers.

The system is based on an Active-Active database server pair with two-way replication between them. Unfortunately, we were seeing the commit message come to the remote data center before the propose message was replicated there. Our solution is to try to route the commit message to the same data center as the original propose message. The result is that if the service is unavailable at the location that received the propose message, (even if the propose was replicated) we respond negatively to the commit: we answer inconsistently. Having said that, we can always receive a message, and our system continues to function if the network gets partitioned.

This leads me to wonder if the CAP Theorem has a second order. That is, if I have a data service that is AP, is it impossible for me to create a service on top of it that is Available-Consistent or Consistent-Partition Tolerant?


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: