Today (July 10, 2018), I am very excited to announce the the GA of IBM Big Replicate for Hadoop 2.12, IBM Big Replicate for Object Stores 2.12, IBM Big Replicate for Hive 2.0, and IBM Big Replicate for Security 2.0 coming on July 13th, 2018!
WHAT IS IBM BIG REPLICATE?
Big Replicate is a replication technology that gives you LIVE DATA — consistent data everywhere, spanning platforms and locations, even for changing data at petabyte scale. Big Replicate enables data replication for Hadoop and cloud object stores with continuous data availability and consistency exceeding the most demanding enterprise SLAs.
WHY BIG REPLICATE?
Moving big data operations from lab to production environments, across multiple hadoop distributions, and on-premise to cloud governed by data availability agreements is no easy task. Big Replicate helps customers solve these problems with the following benefits:
– 100% Availability of data with virtually zero RTO/RPO
– Reduced costs by doubling capacity with no increase in hardware costs
– Migrate to cloud easily with no problem and no downtime
– Fast and real-time data replication
– No vendor lock-in
WHAT’S NEW IN THIS RELEASE?
1. New packaging and licensing options
Big Replicate has expanded so much over the last few years that the original way to buy no longer makes sense for all its new capabilities. This release provides new ways to purchase Big Replicate by offering 4 separate purchase-able products:
–Big Replicate for Hadoop, which enables active-active data replication between Hadoop clusters, including between cross-distribution clusters – available on a per node (or virtual server) basis
–Big Replicate for Object Stores, which enables the active replication of data from Hadoop data clusters to object storage targets – available on a per TB basis
–Big Replicate for Hive, an add-on to Big Replicate for Hadoop that enables the active-active replication of the Hive metastore – available on a per Install (or cluster) basis
–Big Replicate for Security, an add-on to Big Replicate for Hadoop that coordinates and replicates security policies between separate DB-based security policy provider instances to maintain common policy enforcement in each – available on a per Install (or cluster) basis
2. Hive Replication
Big Replicate for Hive, IBM’s version of WANdisco’s Live Hive Plugin for Fusion, was developed to overcome the limitations of the WD Hive Metastore, which has been deprecated. It replicates Hive metadata as it changes in any cluster, regardless of where and when changes are made. All applications have access to the same Hive tables wherever they are required. Live Hive also allows replication between different versions of Hive, meaning the distributions at the target and endpoints can be different. Furthermore, Live Hive can coexist with other plugins
3. Security Policy Replication
Big Replicate for Security includes WANdisco’s Live Sentry Plugin and Live Ranger Plugin for Fusion. It extends Big Replicate to information managed and used by Apache Ranger and Sentry. Use it to keep your security policies consistent among Hadoop deployments.
4. Tooling for Major Upgrades
This release introduces a tool that assists in bootstrapping a new deployment from the configuration and settings of a previous installation. Use it to capture and store the prior installation’s configuration so that a new deployment can be brought up quickly and automatically.
5. Non-Blocking Repair
Big Replicate supports a feature called “Repair”, which provides a mechanism to correct inconsistencies between zones by restoring file system metadata and replicating content. Prior versions provide this capability by blocking other operations that may be attempted against a location under repair while that task is in process. Big Replicate 2.12 introduces a non-blocking variant of Repair that can operate without affecting application operation during the task execution.
Non-blocking repair is a variant of regular repair with enhanced concurrency controls to prevent lost updates and file corruption. It performs a best-effort repair of a replicated location, reducing the scope of the locks applied to individual files. During execution of a non-blocking repair, only individual files are locked, rather than the root of the replicated location. This makes non-blocking repair suitable for use during regular application operation against replicated directories.
6. Heterogenous Plugins
Deploy more complex environments where not every Fusion server requires the same set of plugins to be installed. Big Replicate 2.12 will operate safely under conditions where Fusion servers have different plugins.
7. Authorization Roles for User Interface
The user interface now allows individual users to be assigned to one of a collection of roles that restrict their capabilities.
SOME USEFUL LINKS
Technical documentation can be found in IBM Knowledge Center.
This new release of Big Replicate offerings is available for download from Passport Advantage and Passport Advantage Express website.
Learn more by visiting US
Read announcement