Chapter 5
Synchronization in Empress Replication

5.2 Replication Master Server and Synchronization Client

A Replication Master Server needs to access and control the replication operations of any RMT that wants to participate in a replication relation. This control is done by Replication Master Server residing in RMT-Side and accessing that RMT. In a Replication World, only the Replication Tables that are not RMTs, do not need to be accessed by a Replication Master Server.

A Synchronization Client is a client for a Replication Master Server. It accesses the RRT and sends synchronization requests to a Replication Master Server. The relation between a Replication Master Server and a Synchronization Client are shown in [Figure: Replication Master Server and Synchronization Client].

Figure 5-1
Replication Master Server and Synchronization Client

5.2.1 Connection between an RMT-Side and an RRT-Side

1. A Synchronization Client residing at RRT-Side, and
2. A Replication Master Server residing at RMT-Side.

In order to establish this connection, the following conditions must satisfy:

• In RMT-Side, a Replication Master Server must be started. and must be accessing the RMT.
• Synchronization Client must be configured to access the Replication Master Server on the RMT-Side.
• User of the Replicate Table shall be authorized to access Replication Master Server residing at RMT-Side. This is an authorization given by the Administrator of the Replication Master Server to control the access of Synchronization Clients to this Replication Master Server. This control might require Synchronization Clients to identify themselves by sending their login name and password. (See [User Authorization])
• A network connection between Synchronization Utility and Replication Master Server must be existent.
• Synchronization Client must have privilege to access RRT

1. Setting required environment variables (optional)

   These environment variables are to be used by Empress Server Administration Utility. Generally four environment variables MSUSERAUTHCONFIGFILE, MSNETSERVERCONFIGFILE, MSNETTYPECONFIGFILE and MSCONFIGFILEPATH should be set, before starting a Replication Master Server. The default values of these environment variables are given in $EMPRESSPATH/config/initfile. These environment variables are explained in [Configuring Empress Server and its Clients]. If there is no need to change contents of Network Configuration Files or setting User Authorization security, go directly to step 3.

2. Network Configuration and User Authorization Configuration (optional, depends on Step 1)

   Users can change the contents of network server configuration file and network type configuration file, such as server name, host name, port number etc. Refer to [Network Configuration] for setting Network Configurations. If increased security is required by checking username and password of administrators or users, the corresponding password file and user authorization should be created. Refer to [User Authorization] for increasing the security and user authorization.

3. Creating an Empress Server Start Configuration File (optional)

   This is to optionally specify the database and RMTs to be handled by a Replication Master Server. This is explained in [Starting an Empress Server].

4. Starting an Empress Server

   In this step an Empress Server is started, using Empress Server Administration Utility. This is explained in [Starting an Empress Server].

1. Setting required environment variables (optional)

   Similar to setting up a Replication Master Server, four environment variables should be set for Empress Server Utility and Synchronization Utility on the client side. Usually the client of an Empress Server needs to set environment variable MSNETSERVERCONFIGFILE to point to a new network server configuration file. These environment variables are explained in [Configuring Empress Server and its Clients].

2. Network Configuration and User Authorization Configuration (optional, depends on Step 1)

   Usually the client of an Empress Server needs to change contents of a network server configuration file, so that the Synchronization Utility can access the Empress Server.

3. Using Synchronization Utility or Empress Server Administration Utility

   In this step [Synchronization Utility] sends requests to a running Empress Server. The functions of [Empress Server Administration Utility] can also be used to perform administrative and non-administrative operations on Empress Servers.

5.3 Synchronization Utility

   $ emprepsync replicate_database replicate_table

The synchronization request can be made manually, or can be automated through some other features like "cron" in Unix environment. The Synchronization Client then updates the replicate table based on this data. The RRT-Side of a Replication Relation does not have to be always connected to the host machine running Replication Master Server, but only for the duration of the synchronization.

Note that Empress Replication is an asynchronous process. At any certain time, the contents of an RRT might not be the same as its RMT. Only at the end of a "successful" synchronization, the contents of RRT and its RMT are synchronized.

5.4.1 Table Timestamp and Recovery Timestamp

1. Current Master Table Start Timestamp
2. Table Timestamp
3. Recovery Timestamp

Table Timestamp is the timestamp that a replicate table has the snapshot of its Master Table. A replicate table with Table Timestamp TTS means that it is consistent with master table at TTS. Only a successful synchronization can change table timestamp of a replicate table. For a master table, current timestamp is it's own timestamp, which is increasing with time.

When a replicate table is switched to master table, the original table timestamp is defined as Recovery Timestamp (RTS), and current timestamp becomes the table timestamp of current master table. When a replicate table is explicitly or implicitly synchronized with the new master table, it automatically inherits the recovery timestamp and broadcasts to its RRTs for next synchronization. Generally, Recovery Timestamp of a replication table is less than its Table Timestamp.

5.4.1 Choosing Replication Master

The process of choosing a Replication Master is explained here. Synchronization utility opens the assigned replicate table, and gets all of its enabled Replication Master Entries. Then following the Replication Master Order, Synchronization utility tries to establish a connection to the replication master server assigned to the chosen replication master entry.

If the connection is established, synchronization utility sends the server the following information:

1. Database name and table name of RMT
2. Host name, database name and table name of itself (RRT)
3. Original master table information, current master table start timestamp, table timestamp of RRT and synchronization mode

1. Whether the assigned RMT exists
2. Whether RMT is in the list of replication tables of the Replication Master Server Start Configuration File, if the Replication Master Server is started with a [Replication Master Server Start Configuration File]
3. Whether RRT is authorized to synchronize from the RMT
4. Whether RRT and RMT are in the same replication world

Note that the list of Replication Master Entries can be altered by:

• Adding and subtracting replication masters (see [Creating Replication Master] and [Dropping Replication Master]),
• Enabling and disabling replication masters (see [Enabling and Disabling Replication Masters] , and
• Changing Replication Master Order (see [Changing Replication Master Order].

Start of a Synchronization Request updates Last Request Time information of the RRT. Arrival of Synchronization Request sent to the RMT updates Last Request Time information of the RMT. Successful Synchronization updates Last Successful Request Time and Last Request Status of both RMT-Side and RRT-Side.

5.4.2 Synchronization Requirements

1. "Choosing" an RMT, as described in [Choosing Replication Master].
2. Establishing a connection between the Synchronization Client in the RRT-Side where the RRT resides, and the Replication Master Server in the RMT-Side where the "chosen" RMT resides, (as explained in [Connection between an RMT-Side and an RRT-Side].)
3. Synchronizing the RRT data, with the changed, up-to-date data of the RMT, since the last Synchronization.

In this section, synchronization algorithms on subset replication are not discussed. They are similar to full set Synchronization algorithms, only that SRSC must be applied for synchronization.

5.4.3.1 Algorithm 1: Forward algorithm

1. CMTS(RRT) == CMTS(RMT) and TTS(RRT) < TTS(RMT)
2. CMTS(RRT) < CMTS(RMT) and TTS(RRT) < RTS(RMT)

select * from RMT 
   where EMPRESS_TIMESTAMP > TTS (RRT) 
     and EMPRESS_TIMESTAMP <= TTS (RMT)

After synchronization client gets the changed data, synchronization utility will automatically update the replicate table, and changes its TTS, and synchronization status. If CMTS of RRT is less than CMTS of its RMT, then its RTS and CMTS are also automatically modified.

5.4.3.2 Algorithm 2: Backward algorithm

CMTS (RRT) == CMTS (RMT), TTS (RRT) > TTS (RMT)

1. if "force synchronization" option is not applied:
   then no synchronization is done

2. if "force synchronization" option is applied:
   Replication Master Server first asks the client to send back the change list between TTS(RMT) to TTS (RRT) on RRT, i.e. it performs the following pseudo-query:

   select * from RMT
      where EMPRESS_RECORD_NUMBER (or Primary key) in changed list

   then collects the data depending on the change list on RMT, and sends it to client. After synchronization client gets the data, it first physically deletes all data with EMPRESS_TIMESTAMP between TTS(RMT) to TTS(RRT) on RRT:

   delete from RRT 
     where EMPRESS_TIMESTAMP > TTS(RMT)
     and   EMPRESS_TIMESTAMP < TTS(RRT)
   

   then applies the changed data to RRT:

   insert into RRT
      (select * from RMT 
         where EMPRESS_TIMESTAMP > TTS(RMT)
         and   EMPRESS_TIMESTAMP < TTS(RRT))
   

   then rolls back RRT to TTS(RMT) status, and it finally updates TTS(RRT) and synchronization status.

5.4.3.3 Algorithm 3: Recovery algorithm

• CMTS (RRT) < CMTS (RMT) and TTS (RRT) > RTS (RMT)

select * from RMT 
   where EMPRESS_RECORD_NUMBER (or Primary key) in changed list 
   or (EMPRESS_TIMESTAMP > RTS(RMT) and EMPRESS_TIMESTAMP <= TTS(RMT))

delete from RRT 
  where EMPRESS_TIMESTAMP > RTS(RMT)
  and   EMPRESS_TIMESTAMP < TTS(RRT)

insert into RRT
   (select * from RMT 
      where EMPRESS_TIMESTAMP > RTS(RMT)
      and   EMPRESS_TIMESTAMP < TTS(RMT))

Figure 5-2
Flowchart for Synchronization Algorithms