SalesLogix Technical Support Knowledge Base 

Synchronization

What is it?

Synchronization is a method of sharing data between the main office and individuals in other locations called remotes. SalesLogix utilizes off-line synchronization rather than the longer, more cumbersome method of on-line database comparison. Therefore, each remote has their own database containing a subset of the information in the main database residing at the main office.

When broken down to the bare minimum, SalesLogix synchronization is simply an exchange of files between the remotes and the main office. These files contain database changes and this exchange of files is possible since all changes made against the remote and main database by users are logged in Transaction Exchange Files (TEFs). In addition, SalesLogix synchronization is at the field level. This means that when a specific value is changed, the entire record does not have to be sent up to the main database or down to the remotes. Instead, only the field change will be sent back and forth.

The Synchronization Process

An Overview

The figure below serves as an overview of the synchronization process involving remotes and the main office. The main office contains the main database, the central set of shared synchronization folders and the Sync Server. It supports workgroup users and remotes.

SalesLogix remotes include both remote users and remote offices. Remote users connect to their individual databases on their own machines. Remote offices can support workgroup users who connect directly to the remote office database via a network. All remotes (users and offices) have a subset of synchronization folders and either the Sync Client or the Remote Office Sync Server.

In the figure above, remotes use SalesLogix applications such as the SalesLogix Client to make changes on their remote databases. These changes are also logged as Transaction Exchange Files (TEFs) in the remote Sync folders as indicated by the black arrows on the Remote side. SalesLogix applications at the main office such as the Workgroup Administrator make changes directly on the main database. These changes are also logged as TEFs in the shared Sync Logging folders as indicated by the black arrows on the Main Office side.

The Sync Client (or Remote Office Sync Server) on the remote then transfers these TEFs to the shared Sync Logging folders at the Main Office. This process is indicated by the blue arrows in the figure above. The Sync Client is also responsible for moving files waiting at the Main Office in the shared Sync Logging folders to the remote Sync folders. Then, these files are applied to the remote database so that the remote database now contains the same information as the main database.

In turn, the Sync Server at the main office processes the files sent up from the remote and applies these to the main database. It also generates files for the other remotes containing these changes. This process is indicated by the red arrow on the main office side. In addition, the Sync Server is responsible for processing the TEFs generated by the workgroup users at the main office to create files for the multiple remotes who require these changes so that all distributed databases eventually contain the same information as the main database.

This entire process of synchronization is broken down in the upcoming sections. They detail what occurs when the Sync Server at the Main Office runs through a sync cycle. Also outlined is what specifically occurs when a Remote User or Remote Office synchronizes with the Main Office.

How Sync Files are Named

SalesLogix uses the SalesLogix.key file to generate unique keys for data entered into the database. A very similar design is used to generate the name of closed Transaction Exchange Files used in sync. Therefore, they have the following general form of A2EK-0000000123456789.Q4XB where:
A2EK = the site code of the machine creating the file 
(i.e. the FROM site code)
0000000123456789 = the key value from the SalesLogix.key file of the machine creating the file
Q4XB = the site code of the machine designated to receive the file 
(i.e. the TO site code)
In all cases, either the FROM site code or the TO site code will be that of the Sync Server machine. In the case that the Sync Server is installed on the same machine as either the Workgroup Administrator or SalesLogix Client, it is possible to generate sync files where the FROM site code and the TO site code are the same.

Sync files that are waiting to go out to remotes or that have just been received from remotes will have various file prefixes. For example, files containing database changes will have a prefix of ZIP while files containing documents will have a prefix of FILE. Therefore, the file name will have the form of ZIP-A2EK-0000000123456789.Q4XB. These various prefixes are explained as they are encountered for processing by the Sync Server in the section titled Details of the Sync Server Cycle .

The Sync Server Cycle

The Different Files & Folders

Before laying out the Sync Server's cycle in detail, it is necessary to introduce the various files and folders involved.

Synchronization Folders

Sync Server requires access to the Shared Sync Logging folder set up on the network since this is where the system's TEFs are received and posted. It is critical that this logging folder be shared correctly as outlined within the SalesLogix Implementor's Guide since all components of SalesLogix, including remotes when they are syncing, need to connect to this location.

The Shared Sync Logging folder contains six sub-folders that SalesLogix automatically creates the first time Sync Server is cycled. These six folders are In Files, WG Logs, Shared Logs, Server Logs, Archives and Out Files.
In Files: Remotes (Users and Offices) post their sync files in this folder when they sync with the main office.
WG Logs: Users at the main office post sync files to this folder as they use the various SalesLogix applications to make changes directly on the main database.
Shared Logs: This folder is only utilized if more than one Sync Server is configured for use at the main office. If multiple sync servers are involved, this folder contains copies of In Files and WG Logs processed by the other sync servers. NOTE: All Workgroup Users at the main office must be assigned to one Sync Server. Therefore, only this Sync Server will have files in WG Logs to share. All other Sync Servers at the main office will only have Remotes assigned to them and therefore only have Infiles to share.
Server Logs: Sync Server uses this folder to maintain a copy of all incoming files that it processes. Therefore, Server Logs includes copies of all Infiles, WG Logs, and Shared Logs processed by Sync Server.
Archives: Sync Server uses this folder to maintain a copy of all files being sent down to the remotes with the exception of any documents that are being sent. There is no real need to maintain a copy of documents being sent down as Archives since they exist in the shared location on the network. These Archives are deleted once a confirmation is received from the remote that the file was successfully transferred.
Outfiles: All files being sent to remotes are placed in this folder. Remotes syncing in pull files down from this location.

A few other folders that play a role in the sync process are the shared folders for Sales Library and Attachments. Sync Server pulls copies of the necessary documents from these folders in order to deliver them to the remotes. In addition, documents sent from the remotes are placed in the shared Attachments folder so they can be accessed by users at the main office. 

Sync Server also utilizes a transient folder called Sync Temp. This folder is created and deleted during the course of the sync cycle and should therefore not exist if the sync cycle was completed successfully. This folder is used to house files being sent to the remotes prior to compressing them so that they only need to be written across the network once and in compressed form. Network traffic as the result of a sync cycle is therefore minimized.

Synchronization Files

There are two main files involved in the sync process, SendTran.gdb and ConfTran.gdb. They are both InterBase database files that perform different functions in the sync process. 

Sendtran.gdb is a transient file that is used to temporarily store all database changes that must go out to remotes. Therefore, all sync files in the In Files, WG Logs and Shared Logs folders get applied to SendTran.gdb. Since it is a transient file, it also means that SendTran.gdb is created and then deleted during the course of a successful sync cycle.

ConfTran.gdb is used for the purpose of conflict resolution when a change is made to the same information by two different users in a given time interval. This time interval is specified within the SalesLogix Workgroup Administrator along with the rules that will govern conflict resolution. It is best to limit this time interval to something relatively small such as 7 days since a longer time interval will result in a larger ConfTran.gdb file and therefore hurt the performance of the Sync Server. All changes processed by Sync Server that update existing information are copied to ConfTran.gdb during the course of the Sync Cycle.

Details of the Sync Server Cycle

The diagram below details the various parts of the sync server cycle. This cycle can be broken down into four major phases: the Apply process, the Send process, Subscription and the Cleaning Process. At the end of a complete sync cycle, any other programs that have been queued to run such as Agents or Remote Tasks are executed.

One important item to note is that the sync server used in this discussion is the main sync server at the main office. It is possible to set up multiple sync servers at the main office. In fact, this may be a necessity when dealing with more than 100 total (Workgroup and Remote) users or when dealing with more than 10 remotes syncing via POP3 e-mail. These additional sync servers are secondary sync servers and it is possible to assign remotes to be supported by these secondary sync servers. This helps balance each sync server's processing load. Workgroup users at the main office are always assigned to and supported by the main sync server.

The Apply Process

• Processing files from the WG Logs Folder
When the sync cycle begins, the first step is to process the files that are in the WG Logs folder of the shared sync logging path. This is denoted by the navy and dark grey arrows in the diagram above. These WG Logs are generated by any SalesLogix application that connects to the main office database. Sync Server can only process any WG Logs that have been closed. WG Logs are closed when one of three following things occur:
1) The user closes the SalesLogix application normally - not via
Ctrl+Alt+Del.
2) The user refreshes the SalesLogix application.
3) The time interval (ex: every 15 min.) set up in the Workgroup
Administrator for closing WG Logs is reached.
Since these WG Logs reflect changes that have already been made to the main database, there is no need to apply them to the main database. Therefore, Sync Server simply ports them into SendTran.gdb for sending out to remotes.

• Processing files from the Shared Logs Folder
After the WG Logs have been processed, Sync Server looks for any files in the Shared Logs folder of the shared sync logging path. This is also denoted by the navy and dark grey arrows in the diagram above. These Shared Logs are only created if more than one Sync Server is set up within the system for the Main Office. These Shared Logs contain copies of the WG Logs and In Files processed by the other Sync Server(s) so that all remotes are made aware of changes made to the main database regardless of which Sync Server processed these changes. Since these Shared Logs reflect changes that have already been made to the main database, there is no need to apply them to the main database. Therefore, Sync Server simply ports them into SendTran.gdb for sending out to remotes.

• Decompressing and Moving Documents from the In Files Folder
Once the WG Logs and Shared Logs have been processed, Sync Server looks in the In Files folder of the shared sync logging path for any documents that have been sent by remotes. In Files of this type are identified by a FILE prefix and are decompressed and placed in the shared Attachments folder so they may be accessed by users at the main office.

• Processing Confirmed and Requested files from the In Files Folder
The next step in the sync cycle also involves the In Files folder. Sync Server looks for any files that have been confirmed or requested by remotes. These files will only exist if the Sequence Files Before Apply option is selected on the Sync Server and each remote's profile within the SalesLogix Workgroup Administrator. In Files of these types are identified by a TEF prefix and are created by the remotes as they process sync files on their machines.
Confirmation of a file tells Sync Server that the file referenced was successfully applied to the remote's database. This confirmation causes the referenced file to be deleted from the Archives folder of the shared sync logging path since it is no longer needed. Note that these files will not be deleted periodically if the Sequence Files Before Apply option is not selected on the Sync Server and each remote's profile within the SalesLogix Workgroup Administrator.
Requesting of a file tells Sync Server that a gap was detected in the file sequence on the remote when processing the sync files. Therefore, the files that fill that sequence gap must be re-sent to the remote. These missing files are copied from the Archives folder to the Outfiles folder in the shared sync logging path for re-sending to the remote. If for some reason the requested file(s) can not be found in the Archives folder, a zero byte placeholder file is generated and placed in the Outfiles folder for sending to the remote.

• Processing All Other files from the In Files Folder and Conflict Resolution
After the documents, confirmations and requests in In Files have been processed, Sync Server looks at any other files in the In Files folder of the shared sync logging path. This is denoted by the blue and dark grey arrows in the sync server diagram above. These In Files represent changes that the remotes have dropped off during their sync and must now be processed at the main office. As a result, In Files reflect changes that have not yet been made on the main database and must therefore be applied to the main database. Once they are applied, Sync Server ports them into SendTran.gdb for sending out to all remotes.
Conflict Resolution: If these changes being applied to the main database update existing information, it is possible that Conflict Resolution rules will need to be enforced prior to applying the change. Each change that is updating information in the database contains the information that used to be in that field (Original Field Data) in addition to the changed information. If this Original Field Data does not match the information in the main database, Sync Server scans ConfTran.gdb to find the transaction that first changed the field in question. If this transaction is not located, the new change is processed normally (i.e applied to the main database and then ported to SendTran.gdb). If this transaction is located in ConfTran.gdb a conflict is detected and this conflict is resolved via the Conflict Resolution rules that are selected within the SalesLogix Workgroup Administrator. For example, if the Conflict Resolution rule dictates that the last change wins, then the new change will simply be applied to the main database and processed normally. However, if the Conflict Resolution rule dictates that any change made by a Workgroup User wins, then the change coming from the remote will not be applied and the remote will be notified along with the winning change contained in the main database.

• Generating Shared Logs for the Other Sync Server(s)
If there is more than one Sync Server configured at the Main Office with remotes assigned to it, Shared Logs must be generated. To accomplish this, the sync process places a copy of all WG Logs and In Files into the Shared Logs folder of this other Sync Server(s) shared sync logging path, as it is set up within the SalesLogix Workgroup Administrator. This is done so that all remotes receive all necessary changes regardless of the Sync Server that processed the change.

• Deleting WG Logs, In Files and Shared Logs
The last step in the apply process of the Sync Server cycle is deleting the files in WG Logs, In Files and Shared Logs since they are all now contained within SendTran.gdb and are therefore no longer necessary.

The Send Process

• Processing SendTran.gdb
The send process of the Sync Server cycle begins when all the changes contained within SendTran.gdb are processed. This is denoted by the black and maroon arrows in the sync server diagram above. This processing involves determining which changes are sent out to which remotes and then generating a sync file in the Sync Temp folder. These changes from SendTran.gdb are processed in batches of 5000. After one set of 5000 has been written out as sync files to the Sync Temp folder locally, these sync files are compressed and copied across the network to the Out Files folder so they are ready for remotes to pick up as they sync in to the Main Office. A copy of these Out Files is also placed into the Archives folder for the purpose of file confirmation and requesting. This processing continues until all the changes contained within SendTran.gdb have been written as Out Files and Archives via the Sync Temp folder.

• Queuing Documents to be sent
It is possible that some transactions in SendTran.gdb designate an attached document that must be sent to remotes. This is denoted by the teal arrow in the sync server diagram above. If this is the case, this document is copied from the shared Attachments folder, compressed and placed in the Out Files folder. A copy of this document is not placed into Archives since it exists in the shared location and can always be re-copied from there if necessary.

• Queuing Sales Library items to be sent
The last step of the send process of the Sync Server cycle involves the SalesLogix Sales Library. It is possible that some transactions in SendTran.gdb designate a Sales Library item that must be sent to remotes. This is also denoted by the teal arrow in the sync server diagram above. If this is the case, this item is copied from the shared Sales Library folder, compressed and placed in the Out Files folder. A copy of this item is not placed into Archives since it exists in the shared location and can always be re-copied from there if necessary.

Subscription

The SalesLogix subscription system allows the SalesLogix Administrator and Remote(s) to minimize the number of accounts that are continuously sent down to the remote(s). This sending of accounts is controlled via subscription rules that can be based upon any field in the database. Since it can involve extracting a potentially large number of records from the database, the subscription process can be the most time consuming and resource intensive portion of the Sync Server cycle.

• Running Subscription Rules
Running subscription rules involves identifying any accounts that need to be sent to a remote or must be removed from a remote. There are four instances that identify an account for sending to remotes:
1) The remote is the account owner (Force Subscribed).
2) The remote is the Primary Team Member of the Team owning the account (Force Subscribed).
3) There is a subscription rule in place that the account meets the criteria for (Subscribed).
4) The account has been manually subscribed to by the remote (Requested).
The only instance that identifies an account to be removed from a remote is a change in account ownership that no longer allows the remote to have access to the account. This would occur if the ownership was changed to a team that the remote did not belong to or to a different individual remote.

• Sending Accounts to Remotes
Once accounts have been flagged, they are sent out to the remote(s). This is denoted by the red arrow in the sync server diagram above. The number of accounts sent to an individual remote per Sync Server cycle must be limited to a reasonable number depending on the server hardware so that this process does not bog down the entire sync cycle. This limit can be set within the Sync Server profile found in the SalesLogix Workgroup Administrator. These accounts being sent to remotes are written to sync files that are placed directly in the Sync Temp folder.

• Removing Accounts from Remotes
After accounts have been sent to the remote(s), the accounts flagged for removal from the remotes are processed. This involves writing the accounts to be removed to sync files that are placed directly in the Sync Temp folder. This process is also denoted by the red arrow in the sync server diagram above. There is no limit to the number of accounts that can be removed from a remote during a given sync cycle.

• Generating Out Files
Once changes as a result of subscription have been processed and written to the Sync Temp folder locally, these files are compressed and copied across the network to the Out Files folder so they are ready for remotes to pick up as they sync in to the Main Office. A copy of these Out Files is also placed into the Archives folder for the purpose of file confirmation and requesting. This entire process is denoted by the maroon arrows in the sync server diagram above.

The Cleaning Process

Once subscription is complete, Sync Server begins the process of cleaning up. This includes maintaining ConfTran.gdb and SendTran.gdb. 

• Maintaining ConfTran.gdb
As previously mentioned, ConfTran.gdb contains changes that have been made to the database within a given period of time. This time period is specified within the SalesLogix Administrator. Therefore, Sync Server ensures that changes that are older than this time period are deleted from ConfTran.gdb as part of the clean up process of a sync cycle.

• Generating Server Logs and Maintaining SendTran.gdb
As a fail safe measure, Sync Server maintains the contents of SendTran.gdb as Server Logs. This is done by exporting SendTran.gdb to a flat file that is then placed in the Server Logs folder. This process is denoted by the green arrow in the sync server diagram above. Once Server Logs have been generated, SendTran.gdb is deleted and the sync cycle is in essence complete.

Running Other Programs

Once the sync cycle is complete, there are a few additional tasks that are co-ordinated via the Sync Server. These include executing SalesLogix Agents and running any scheduled Remote Tasks. Any sync files resulting from these activities are then copied into the Out Files folder so they are ready for remotes to pick up as they sync in to the Main Office.