2.1.3 Replication Server

Data replication is now one of the most common ways to distribute data to remote sites. If a large number of remote users and remote applications need to perform ad hoc queries and transaction processing or a large volume data transfer is involved, the network can bog down. In particular, for the SMA, we have a primary database server at the Mauna Kea site, most of the users will remotely access the data. Based on their location, the remote users can be divided into two groups, namely Cambridge (Massachusetts) based or Nankang (Taipei) based. For a large volume of the SMA visibility data, users and applications may have to wait unacceptable lengths of time for receiving a complete data set. This will also generate a large amount of network traffic. Instead, we can replicate the data to local servers, and users can access the data locally.

 

Figure: 2. A concept of the replication sites for SMA Astronomical DATA Archive. Hilo is considered as the primary site; the secondary sites are in Cambridge and Nankang. The data in SMADB can be replicated through normal network link. However, the large FITS-IDI files need either a dedicated link or portable medium with an express air delivery service. For the ASIAA, it is not necessary to follow this design. They can develop a database system based on their resources. The FITS-IDI files can be duplicated from the primary site.

Sybase Replication Server provides an excellent solution for replication of a database at a remote secondary site. As illustrated in Fig. 2, Replication Server transmits data from the primary database server in Hilo (where the data originates) to secondary site (the Cambridge site and/or the Nankang site) on the network. Rather than requiring each user to access the remote database server over a long-distance network, this Replication Server handles data transmission and ensures that each local server has an up-to-date copy of the data.

We note that in this on-line archive system, the RDBM (Sybase) handles only the header information stored in SMADB (the SMA Astronomical Database, see 2.2) while the visibility data in the massive storage system is handled separately. Since the volume of the header information data is small, it can be easily replicated through the network. During the testing period, the visibility data might be small enough to allow us to duplicate all the data including the data in SMADB and the FITS-IDI files at the remote sites via the existing network link. As the full correlator operation comes on line, the duplication through network will run into some problems due to the limited bandwidth in network transfer. Either a special network link or other portable medium combined with some commercial express delivery service is required.

The approach proposed here is a remedy for the problems associated with the distribution of and direct access to the SMA data. Using Replication Server is efficient, because Replication Server only replicates original data that is added, modified, or deleted. It also is fast, because Replication Server copies all of the data to the remote server, so the remote users can access it over the Local Area Network (LAN). Replication Server provides an additional important advantage. If the local data server or local network is down and transactions need to be replicated, Replication Server performs all of the necessary synchronization when the local network is available to each other. This is an excellent function for disaster recovery.

Replication Server will also effectively reduce the manpower required for the maintenance of the SMA Astronomical Database.

However, Replication Server has not been created in our system yet. We strongly propose to implement the Sybase Replication Server to our database server system in the next software upgrade. The cost would be minor as compared with the returns.