Packages

t

com.here.platform.data.processing.java.leveling

AdaptivePatternEstimateFn

trait AdaptivePatternEstimateFn extends InputLayers with InputOptPartitioner with Serializable

Main interface the user has to implement to calculate an com.here.platform.data.processing.leveling.AdaptivePattern.

The users estimate the contribution of input partitions to different tile, by returning a weight for each tile involved. An com.here.platform.data.processing.java.leveling.AdaptivePattern is then calculated by accumulating weights to find out leveling points whose total weights (the sum of the weights of their children) does not exceed a given threshold. Users are free to give any custom meaning to weights and thresholds.

Used by AdaptivePatternEstimator, it is invoked in a distributed fashion on every input partition of the layers mentioned as input. com.here.platform.data.processing.java.blobstore.Retriever may be passed in constructor and used internally, however this pattern is discouraged.

AdaptivePatternEstimator applies this function to the whole input layers at every run non-incrementally so, unless the input layers are few and small, using a com.here.platform.data.processing.java.blobstore.Retriever would download the whole input data every time.

Estimates don't have to be precise, so the suggested pattern is to use the payload size present in the metadata as indication of the data size, without retrieving the payload.

Linear Supertypes
Serializable, Serializable, InputOptPartitioner, InputLayers, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. AdaptivePatternEstimateFn
  2. Serializable
  3. Serializable
  4. InputOptPartitioner
  5. InputLayers
  6. AnyRef
  7. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Abstract Value Members

  1. abstract def estimateFn(src: Java.Pair[InKey, InMeta]): Iterable[(HereTile, Long)]

    Estimate the weights of zero, one or more output HereTiles

    Estimate the weights of zero, one or more output HereTiles

    src

    Input key and metadata, it can be used to retrieve the data with a com.here.platform.data.processing.java.blobstore.Retriever, although this method is not suggested.

    returns

    Output HereTiles with their estimated weights. Weights must be positive.

  2. abstract def inLayers: Map[String, Set[String]]

    Layers of the input catalogs that should be queried and provided to the compiler, grouped by input catalog and identified by catalog id and layer ID.

    Layers of the input catalogs that should be queried and provided to the compiler, grouped by input catalog and identified by catalog id and layer ID.

    Definition Classes
    InputLayers
  3. abstract def inPartitioner(parallelism: Int): Option[PartitionerOfKey]

    If the returned com.here.platform.data.processing.java.Java.Option is Empty, the Executor will use a default partitioner, if it is defined then the given partitioner will be applied when querying the input catalogs.

    If the returned com.here.platform.data.processing.java.Java.Option is Empty, the Executor will use a default partitioner, if it is defined then the given partitioner will be applied when querying the input catalogs.

    parallelism

    the parallelism of the partitioner

    returns

    the input optional partitioner with the given parallelism

    Definition Classes
    InputOptPartitioner

Concrete Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  6. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  7. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  8. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  9. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  10. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  11. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  12. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  13. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  14. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  15. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  16. def toString(): String
    Definition Classes
    AnyRef → Any
  17. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  18. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  19. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()

Inherited from Serializable

Inherited from Serializable

Inherited from InputOptPartitioner

Inherited from InputLayers

Inherited from AnyRef

Inherited from Any

Ungrouped