Packages

c

com.here.platform.data.processing.java.spark.partitioner

AdaptiveLevelingPartitioner

final class AdaptiveLevelingPartitioner extends PartitionNamePartitioner with ScalaPartitionNamePartitionerWrapper

A PartitionerOfKey that uses a precalculated leveling com.here.platform.data.processing.java.leveling.Pattern.

Keys are distributed to Spark partitions strictly following the leveling points that the pattern specifies. Keys left not aggregated by the pattern are distributed among a disjoint set of Spark partitions using a fallback partitioner, if specified. Otherwise they are uniformly distributed over the existing partitions.

The number of partitions used for aggregated keys is fixed and matches the number of leveling points of the pattern.

Linear Supertypes
ScalaPartitionNamePartitionerWrapper, Wrapper[spark.partitioner.PartitionNamePartitioner], PartitionNamePartitioner, PartitionerOfKey, Serializable, Serializable, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. AdaptiveLevelingPartitioner
  2. ScalaPartitionNamePartitionerWrapper
  3. Wrapper
  4. PartitionNamePartitioner
  5. PartitionerOfKey
  6. Serializable
  7. Serializable
  8. AnyRef
  9. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new AdaptiveLevelingPartitioner(pattern: AdaptivePattern, fallbackPartitioner: PartitionNamePartitioner)

    pattern

    The adaptive leveling pattern that controls the partitioning.

    fallbackPartitioner

    The partitioner used for non-aggregated keys.

  2. new AdaptiveLevelingPartitioner(pattern: AdaptivePattern)

    pattern

    The adaptive leveling pattern that controls the partitioning.

    Note

    With this constructor, non aggregated keys are distributed over the existing Spark partitions of the pattern.

  3. new AdaptiveLevelingPartitioner(impl: spark.partitioner.AdaptiveLevelingPartitioner)

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  6. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  7. def equals(o: Any): Boolean
    Definition Classes
    Wrapper → AnyRef → Any
  8. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  9. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  10. final def getPartitionForKey(key: Key): Int

    Gets the partition for a given key.

    Gets the partition for a given key.

    key

    the key for which the partition must be calculated

    returns

    the partition, identified by one scala.Int, in which the key should be located

    Definition Classes
    PartitionNamePartitionerPartitionerOfKey
  11. def getPartitionForName(name: Name): Int
    Definition Classes
    ScalaPartitionNamePartitionerWrapper → PartitionNamePartitioner
  12. def hashCode(): Int
    Definition Classes
    Wrapper → AnyRef → Any
  13. val impl: spark.partitioner.AdaptiveLevelingPartitioner
    Definition Classes
    AdaptiveLevelingPartitionerWrapper
  14. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  15. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  16. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  17. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  18. def numPartitions: Int

    Gets the total number of partitions defined by the partitioner.

    Gets the total number of partitions defined by the partitioner.

    returns

    the total number of partitions defined

    Definition Classes
    ScalaPartitionNamePartitionerWrapper → PartitionerOfKey
  19. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  20. def toSpark: Partitioner

    Gets a standard Spark org.apache.spark.Partitioner wrapping this one.

    Gets a standard Spark org.apache.spark.Partitioner wrapping this one.

    returns

    this partitioner wrapped inside a standard Spark interface

    Definition Classes
    ScalaPartitionNamePartitionerWrapper → PartitionerOfKey
  21. def toString(): String
    Definition Classes
    Wrapper → AnyRef → Any
  22. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  23. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  24. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()

Inherited from ScalaPartitionNamePartitionerWrapper

Inherited from PartitionNamePartitioner

Inherited from PartitionerOfKey

Inherited from Serializable

Inherited from Serializable

Inherited from AnyRef

Inherited from Any

Ungrouped