The Cloudant/CouchDB dashboard-development team has just landed some substantial improvements to data-replication, particularly around triggering, monitoring, and troubleshooting replication jobs.

The project is the result of a great effort from a number of people, including the numerous Cloudant and Apache CouchDB™ users. I’m very thankful to all the people who provided comments on the previous replication section, and expressed hopes for future versions of our continuously improving dashboard. We have more in-plan, but you should check out the current drop — I’m excited for the problems that have been solved, and functionality we’ve added.

The Good Stuff

We’ve redesigned the activity section for increased display area and greater information density, and have added numerous new tools to help you track, organize, and work with your current and past replication jobs.

Redesigned replication activity section in the Cloudant dashboard.

We’ve also streamlined the process for triggering new replication jobs. The Replication form now follows a progressive-disclosure model, where relevant options and information are displayed as you make selections in each step.

The new form for replication jobs can now be repopulated with previous information

Filter/Sort/Selection Improvements for the Activity Section

We’ve added a filter into the activity section, and have made each column sortable, allowing you to organize and quickly search replication information to find what you need. No more scanning long lists to find a particular job; sort on the new timestamp column and then start entering the source or target, or the replication document ID into the filter and see just how quickly that job appears. Filter or sort to display whatever you’re looking for, whether it’s errored jobs, continuous-replication jobs, or jobs timestamped on a particular day. It now takes few keystrokes and minimal navigation.

We’ve also added a bulk-selector to the activity section so that you can easily clear history and cancel jobs without repetitious clicking. Use the filter to display only the jobs you want to clear, and then select-all-delete to quickly clear them, or pick and choose the specific jobs that you’d like to delete all at once.

Fiilter, sort and selection improvements in the replication activity section of the Cloudant dashboard.

Your local source and target databases as well as the replication documents associated with each job are all just single clicks away, meaning that you have direct access to replicated data and troubleshooting information.

From the editors: In addition to UX improvements, the Cloudant engineering team has optimized the underlying processes supporting replication. We’ll have the full story in a forthcoming article that covers replication performance improvements and future changes to the API. For now, here’s a list of the most commonly encountered bugs the replication team has squashed as of their latest updates (these fixes are now live in Cloudant’s current release):

  • Hang of HTTP worker when streaming target has died
  • Situation where changes reader gets stuck in cleanup process by capping max number of messages
  • Un-triggered replications
  • Deadlock condition with remote source
  • Stuck changes feed
  • Race condition in replicator rescan logic
  • Excessive frequency of checkpoint attempts
  • Leaking replication changes feed readers

Better Management of Replication Jobs

Full information from previous replication jobs (successful or failed ones) can now repopulate the replication form, meaning that we’ve made it much easier for you to:

  • Re-run completed jobs
  • Troubleshoot and correct failed jobs
  • Duplicate jobs, but with adjustments to anything you’d like to adjust (new target/source/type/ID)

Similar to the replication-form repopulation, triggering a new replication job no longer redirects you away from the replication form, meaning that you can trigger numerous jobs in succession by re-clicking “Start Replication” after making whatever changes you’d like to the existing replication details.

Hitting "start replication" can trigger numerous jobs in succession

We’ve improved conflict-handling for your replication documents. In addition to validating new replication-document IDs against existing IDs, the dashboard now gives you the option to overwrite a conflicting replication document when you trigger a job. So, if your first job fails, you can quickly recreate it, using the same replication document ID. Name your replication documents as you’d prefer, and keep the failed ones from cluttering.

Better conflict handling for replication documents.

Our Team and Your Feedback

Ben Keen started the development effort, which Garren Smith later took over and expanded in scope. We received valuable code reviews from Robert Kowalski and Michelle Phung, and testing by a good number of other developers. As a designer for IBM Cloudant and CouchDB, I’m thrilled to be able to work with such a talented development community on projects such as this.

You can sign-in to your account to give the new Replication section a try. There’s a lot to see and play around with. And please let us know what you think in the comments below, or send feedback directly to me, at jmmcdavi@us.ibm.com. I love the engagement in our community, and look forward to your thoughts.

Join The Discussion

Your email address will not be published. Required fields are marked *