faust.assignor.copartitioned_assignor

Copartitioned Assignor.

class faust.assignor.copartitioned_assignor.CopartitionedAssignor(topics: Iterable[str], cluster_asgn: MutableMapping[str, CopartitionedAssignment], num_partitions: int, replicas: int, capacity: Optional[int] = None)[source]

Copartitioned Assignor.

All copartitioned topics must have the same number of partitions

The assignment is sticky which uses the following heuristics:

  • Maintain existing assignments as long as within max_capacity for each client

  • Assign actives to standbys when possible (within max_capacity)

  • Assign in order to fill max_capacity of the clients

We optimize for not over utilizing resources instead of under-utilizing resources. This results in a balanced assignment when max_capacity is the default value which is ceil(num partitions / num clients)

Notes

Currently we raise an exception if number of clients is not enough for the desired replication.

num_partitions: int
replicas: int
max_capacity: int
min_capacity: int
topics: Set[str]
get_assignment() → MutableMapping[str, CopartitionedAssignment][source]
Return type:

_GenericAlias[str, CopartitionedAssignment]