The Cloudant/CouchDB dashboard-development team has just landed some substantial improvements to data-replication, particularly around triggering, monitoring, and troubleshooting replication jobs.
The project is the result of a great effort from a number of people, including the numerous Cloudant and Apache CouchDB™ users. I’m very thankful to all the people who provided comments on the previous replication section, and expressed hopes for future versions of our continuously improving dashboard. We have more in-plan, but you should check out the current drop — I’m excited for the problems that have been solved, and functionality we’ve added.
The Good Stuff
We’ve redesigned the activity section for increased display area and greater information density, and have added numerous new tools to help you track, organize, and work with your current and past replication jobs.
We’ve also streamlined the process for triggering new replication jobs. The Replication form now follows a progressive-disclosure model, where relevant options and information are displayed as you make selections in each step.
Filter/Sort/Selection Improvements for the Activity Section
We’ve added a filter into the activity section, and have made each column sortable, allowing you to organize and quickly search replication information to find what you need. No more scanning long lists to find a particular job; sort on the new timestamp column and then start entering the source or target, or the replication document ID into the filter and see just how quickly that job appears. Filter or sort to display whatever you’re looking for, whether it’s errored jobs, continuous-replication jobs, or jobs timestamped on a particular day. It now takes few keystrokes and minimal navigation.
We’ve also added a bulk-selector to the activity section so that you can easily clear history and cancel jobs without repetitious clicking. Use the filter to display only the jobs you want to clear, and then select-all-delete to quickly clear them, or pick and choose the specific jobs that you’d like to delete all at once.
Your local source and target databases as well as the replication documents associated with each job are all just single clicks away, meaning that you have direct access to replicated data and troubleshooting information.
From the editors: In addition to UX improvements, the Cloudant engineering team has optimized the underlying processes supporting replication. We’ll have the full story in a forthcoming article that covers replication performance improvements and future changes to the API. For now, here’s a list of the most commonly encountered bugs the replication team has squashed as of their latest updates (these fixes are now live in Cloudant’s current release):
- Hang of HTTP worker when streaming target has died
- Situation where changes reader gets stuck in cleanup process by capping max number of messages
- Un-triggered replications
- Deadlock condition with remote source
- Stuck changes feed
- Race condition in replicator rescan logic
- Excessive frequency of checkpoint attempts
- Leaking replication changes feed readers
Better Management of Replication Jobs
Full information from previous replication jobs (successful or failed ones) can now repopulate the replication form, meaning that we’ve made it much easier for you to:
- Re-run completed jobs
- Troubleshoot and correct failed jobs
- Duplicate jobs, but with adjustments to anything you’d like to adjust (new target/source/type/ID)
Similar to the replication-form repopulation, triggering a new replication job no longer redirects you away from the replication form, meaning that you can trigger numerous jobs in succession by re-clicking “Start Replication” after making whatever changes you’d like to the existing replication details.
We’ve improved conflict-handling for your replication documents. In addition to validating new replication-document IDs against existing IDs, the dashboard now gives you the option to overwrite a conflicting replication document when you trigger a job. So, if your first job fails, you can quickly recreate it, using the same replication document ID. Name your replication documents as you’d prefer, and keep the failed ones from cluttering.
Our Team and Your Feedback
Ben Keen started the development effort, which Garren Smith later took over and expanded in scope. We received valuable code reviews from Robert Kowalski and Michelle Phung, and testing by a good number of other developers. As a designer for IBM Cloudant and CouchDB, I’m thrilled to be able to work with such a talented development community on projects such as this.
You can sign-in to your account to give the new Replication section a try. There’s a lot to see and play around with. And please let us know what you think in the comments below, or send feedback directly to me, at email@example.com. I love the engagement in our community, and look forward to your thoughts.