IBM Streams 4.3.0
Resource allocation for application services
A resource for application services can be static (added explicitly to an instance) or dynamic (added to or removed from an instance dynamically based upon the job or processing element requirements). A static resource is specified manually by name or automatically by a resource specification and remains allocated to the instance until it is explicitly removed. A dynamic resource is released from the instance when the instance is stopped or when the resource is no longer needed.
Resource allocation mode
The resource allocation mode (applicationResourceAllocationMode) determines
whether an application resources is scoped to an instance or to a job.
- Instance: Only the statically defined resources in the instance are considered for placing the job onto, and the complete set of them are considered in the placement calculation. The number of resources available to the job is determined by the number of static resources in the instance. Resources are not allocated dynamically. Resources can be shared between jobs within the instance.
- Job: Resources are allocated to a job as needed, and are sourced either from the existing instance static and dynamic resources (if dynamic resource allocation is enabled with the dynamicResourceAllocationEnabled property) or from new dynamically allocated resources. The scheduler determines the required number of resources for the job.
The mode can be set at the instance or domain level. An instance cannot contain both instance-scoped and job-scoped resources.
How job-scoped resources are allocated
An instance can have both static and dynamic resources at the same time. When the resource
allocation mode is job-scoped
(applicationResourceAllocationMode=job), IBM®
Streams allocates static resources first, and then dynamic resources as needed.
- When a job is submitted, a resource sharing mode (jobResourceSharing) can
be specified at either the domain or instance level. The resource sharing mode controls whether the
job's resources can be shared by other jobs or are isolated only to this job's usage.
- sameJob: Job resources can be shared only within this job.
- sameUser: If unused resources aren't available, job resources can be shared by other jobs from the same user that are also specified as sameUser sharing.
- sameInstance: If unused resources aren't available, job resources can be shared by other jobs from the same instance that are also specified as sameInstance sharing.
- When a job is submitted, IBM Streams analyzes the application to determine the job's resource requirements. It attempts to satisfy these requirements by first considering the set of static application resources that are in the instance and are not allocated to a job. If this initial set of resources can't satisfy the job's requirements, additional application resources are requested. If the resource manager can't fulfill the job requirements with dynamic resources, existing resources that were previously allocated to other jobs are considered. In this case, the sharing mode of the current job and the previous jobs determine whether sharing is allowed. The resulting set of resources are allocated to this job and are used for placement purposes for the job's PEs.
- The scheduler can request dynamic application resources for the instance as jobs are submitted. If additional resources are required, the scheduler can request these resources. All dynamic application resources that are returned to the scheduler are considered to be exclusive to the requesting instance. The instance has exclusive use of the resources and no other instances are given these resources for application use.