Welcome back to another weekly Tech Tip, Mitch here. So last week, Brett gave us a really great high-level overview of the different ways in which Ceph can be configured in a multisite configuration. So, what I thought we would do this week is go a little more in-depth on Ceph object multisite in particular. It's really well fleshed out. It's really well supported and it's really cool. So, let's jump in.

Ceph object multisite allows anyone to build their own cloud that can be accessed from all over the world, by putting new clusters online in different geographic locations to out serve clients in that area. The data that is then being generated by clients will be written to the respective cluster through a RADOS gateway endpoint. That data then replicates with all other sites within the zone group to become eventually consistent. So, I just want to very quickly talk about the two different types of multisite that Brett talked about in his last video, synchronous and asynchronous.

Ceph by its own design is synchronous replication. And so that means from a client's level, write is not acknowledged until all replication copies are written into the cluster. So, for example, if you have a two-way replication cluster with your failure domain at the host level, and your client writes into the cluster, that client does not receive acknowledgment until that second copy has been written to any other node than where the primary write happened. So, you can very quickly see where this would become unusable at the multisite level, especially when the cluster spanned vast geographic distances to one another.

This is where asynchronous replication is used. This allows each cluster in the multisite to become eventually consistent with each other. So first, I want to get through some nomenclature that you should be aware of about Ceph multisite before we continue.

So, from the top down, we have realm, Zone Group and zones and I'm going to explain all those right now. So, a realm represents a globally unique namespace and represents the highest level of the Ceph multisite cluster. Each realm can then contain either one or multiple zone groups and multiple zones. So next we have zone groups. These used to be called regions and they're essentially made to define one or more multiple geographic areas. So next we have zones. These are the lowest level of the Ceph multisite configuration and they're represented by one or more object gateways underneath one single Ceph cluster.

I'm going to go through a very straightforward multisite Ceph cluster configuration. Let's say you have a cluster in data center A, and a cluster in data center B, and you want these to be able to replicate to one another. And you also want to be able to have one failure and still serve clients for your multisite configuration.

So Ceph Cluster in Data Center A will be located in Zone one, and Ceph cluster in data center B will be located in Zone two. Next, we choose which cluster will be our Masters zone cluster. This cluster is used to create users and buckets for the entire zone group. Now both clusters will make up zone group 1. In this configuration, there's only one zone group and zone group 1 will be the master's own group. Next, this zone group must be part of a realm. We would then choose a realm named for our multisite. This is a global namespace for the entire multisite.

The realm itself works on periods. Each period of time represents the state of the zone group and zone configuration at a specific time. Each time there is a change to the configuration, the period must be updated and committed.

Once these two sites are configured with all this information, we will create a user that will be used to create the synchronization link between the two clusters. If you take a look at the dashboard of each cluster, you will see that the RGW pool names will contain the name of the zone the respective cluster belongs to. As you can see, the name of the zones are called yin and yang. So, this configuration will actually replicate all data between each zone and can act as a failover for each other in the case of an outage. The only thing to note here is that your master zone, if that is the cluster that drops out, for example, you will not be able to create users in buckets until that zone comes back up or you promote the secondary zone to the master zone.

So, I consider this probably one of the simplest configurations for Ceph multisite, as there are so many finely granular ways to tune your multisite Ceph clusters. For example, you can select only certain buckets to replicate to the secondary cluster, or you can set only one way syncing for certain buckets or certain users. It's very, very finely granular.

This is essentially just a failover type disaster recovery scenario where each side can serve clients of the cluster. So, it's really cool.

So, I haven't done a fun fact in a while and this one's probably pretty obvious if you're paying attention. But fun fact is, when I was configuring our two Ceph clusters in a multisite configuration and I was naming them, it was supposed to be yang, not yan. I made a spelling error and decided not to correct it. So, forever now our clusters will be called yin and yan.

Right. Guys, thanks for watching. Hopefully you found the deeper dive into Ceph Multisite interesting. I'm going to leave some links in the description below. If you want to read further into the different ways in which you can configure this. As I said, there's so many different ways to set this up for your own specific environment. So if you have any ideas, leave in the comments section below and we'll definitely check it out. We'll see you on the next one.