Rep is the interactive tool bench component in CouchDyno and is probably the most fun one to play with.

Initially, Rep started with an assortment of Python scripts to create replication scenarios. Some functions created databases, others filled them with data, some replicated documents, while others compared the contents of source and target databases.

We came up with the idea later to group the scripts into higher-level functions, refactor some common library code, and ultimately save the command line history to quickly recall the scenarios during testing. Often when debugging, you spend a lot of time in the Edit > Build > Run > Setup > Test cycle. Rep helps shorten the time wasted in the Run > Setup > Test part.

Let’s look at an example of how to use Rep.

Prerequisites

Rep assumes the user has:

  • Set up and configured the ability to build and run a local CouchDB 2.0 cluster. See the README for more details
  • Python 2.7
  • virtualenv
  • pip
  • Setting up a local database cluster

    The start command might look like:

     $ cd $couchdb_source_dir
     $ ./configure ...
     $ make
     $ ./dev/run --admin=adm:pass
    

    After running the start command, a three-node cluster in development mode should be running. The first node listens on port 15984, with adm as the admin user and pass as the password.

    Launching Rep

    Then after cloning or downloading CouchDyno, run the ./build.sh script in its top directory. build.sh is just a wrapper around virtualenv and pip requirements install.

    Now, let’s launch Rep.

    $ ./venv/bin/rep 
    Interactive replication toolbox
    rep, rep.getsrv, rep.getrdb and couchdb modules are auto-imported
    Assumes the cluster runs on http://adm:pass@localhost:5984
    Type rep. and press  to auto-complete available functions
    ...
    In[1]:
    

    Notice this prompt looks remarkably like a Jupyter / IPython prompt. That’s because it uses Jupyter/IPython underneath and adds a few extra auto-imports and other nice things. Also, a lot of the module-level functions can be discovered by pressing the [Tab] key after typing “rep.” and the prompt should auto-complete with a set of choices.

    Here is how to set up one source to two targets replication scenario. The source will contain 10 documents. These 10 documents should be replicated to the two targets. Rep will then verify that is the case. And will also repeat the whole thing thing times in a row:

     rep.replicate_1_to_n_and_compare(n=2, cycles=3, num=10)
    
     configuration:
       cluster_branch = None
       cluster_repo = None
       cluster_reset_data = True
       cluster_settings = None
       cluster_tmpdir = None
       connection_timeout = 30000
       create_target = False
       http_connections = 20
       prefix = cdyno
       replicator_url = None
       server_url = http://adm:pass@localhost:15984
       source_url = None
       target_url = None
       timeout = 0
       worker_processes = 4
       ...
    

    Summary

    Hopefully this blog post gave you a quick intro with how to use Rep, one of the CouchDyno projects to test, benchmark, and experiment with CouchDB and IBM Cloudant.

Join The Discussion

Your email address will not be published. Required fields are marked *