Guidelines for Partitioning Elements on Log Stores

Guidelines for sizing a log store when partitioning elements that are within a log store.

The rules below assume that the log store is dedicated to a partitioned stream. For non-join cases, the rules are:
  • When there is no retention specified, the sizing calculation is 2 * N where N is the size allocated for the stream in the non-partitioned case.
  • When there is retention on the target, the sizing calculation is N * (K + 1) where N is the size allocated for the non-partitioned case and K is the number of partitions. This is because each partition honors the retention clause.
In the case of joins, also include the space required to store the input to the join in the log store. This means that the sizing calculation is N when you are partitioning the input, and N * K when you are broadcasting the input where N is the size required to store the input.