Alston Grove

SymmetricDS is web-enabled, database independent, data synchronization software. It supports near real time data synchronization between remote databases that do not have to be of the same type. The software was designed to scale for a large number of databases, work across low-bandwidth connections, and withstand periods of network outage.

If you’ve been wondering what I’ve been up to lately, I’m proud to have spent my time co-authoring SymmetricDS with Chris Henson and others.

It all started while implementing a commercial Point of Sale (POS) system for a large retailer in the area. The POS came with software to “trickle back” transactions to a central office for financial posting and reports. We found that the software wouldn’t handle the load generated by the 1400 stores in the chain. It also didn’t solve other problems we had, such as:

• Loading a store with an initial set of item, tax, and employee data
• Synchronizing data changes down to stores
• Conditional synchronization and filtering for subsets of data
• Routing sensitive credit card and customer data to a separate, protected database
• Supporting one database platform at the store and another at general office

Since we couldn’t find software that did everything we needed, we wrote our own. It was deployed to half the chain the first year and made the project a success. We learned a lot and gained experience with synchronization issues.

Some of the developers were unsatisfied with the implementation and wanted to modify the architecture of the program to gain improvements. The POS project had already moved on to other areas of the system, so there was not enough momentum to support such large modifications. Instead, the developers used their spare time and created an open source project to implement a new architecture from the ground up, still leveraging the knowledge and experience of the original system. We named the software “Symmetric,” which later became “SymmetricDS” (short for data synchronization) to help identify it from other uses of the term.

Meanwhile, the POS project continued to maintain the old code base for synchronizing data, but some infrequent bugs hinted at larger problems in the code, raising concerns for rolling out the rest of the chain. The issue got visibility and became part of the next release for the POS. The development group proposed bringing in SymemtricDS as a replacement, which would further refine the software and put it through rigorous quality assurance testing. After putting in place an “Open Source Policy,” the business agreed to use the software and contribute back improvements made during implementation. This move gave credibility and “polish” to the product as production-ready software.

After the retailer deployed SymmetricDS to stores in production, we announced a 1.0 release of the software publicly. It made a lot of hard work feel worthwhile and satisfying. Currently, we’re planning the next release and completing the User Guide documentation. We’re very happy and excited that we’ve delivered something unique and useful to the community, so we hope you take the time to check it out.

This entry was posted on Saturday, November 10th, 2007 at 3:41 pm and is filed under Software. You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.