In Streams v4.0, we have introduced a new feature called Consistent Region in the product. This feature enables applications to guarantee processing of all tuples. This article discusses how the checkpoint process affects applications performance.
Reading from an external source—such as the network or filesystem—is often a performance bottleneck. When source operators are the performance bottleneck for a streaming application, we have a tendency to blame the reading from the external source. But, that is not always the case. Particularly for large tuples which have many attributes, the actual performance bottleneck can be parsing.
Pointing ZooKeeper’s transaction log location to a disk with fast storage can dramatically increase your Streams performance from an administrative perspective . The effect on running applications is minimal. Learn how to set up ZooKeeper’s transaction log location to improve Streams performance.
InfoSphere Streams Version 4 is a major new release with significant advances in high availability and ease of use. Leveraging these requires careful consideration of performance related configuration options.