trait AdaptivePatternEstimateFn extends InputLayers with InputOptPartitioner with Serializable
Main interface the user has to implement to calculate an com.here.platform.data.processing.leveling.AdaptivePattern.
The users estimate the contribution of input partitions to different tile, by returning a weight for each tile involved. An com.here.platform.data.processing.java.leveling.AdaptivePattern is then calculated by accumulating weights to find out leveling points whose total weights (the sum of the weights of their children) does not exceed a given threshold. Users are free to give any custom meaning to weights and thresholds.
Used by AdaptivePatternEstimator, it is invoked in a distributed fashion on every input partition of the layers mentioned as input. com.here.platform.data.processing.java.blobstore.Retriever may be passed in constructor and used internally, however this pattern is discouraged.
AdaptivePatternEstimator applies this function to the whole input layers at every run non-incrementally so, unless the input layers are few and small, using a com.here.platform.data.processing.java.blobstore.Retriever would download the whole input data every time.
Estimates don't have to be precise, so the suggested pattern is to use the payload size present in the metadata as indication of the data size, without retrieving the payload.
- Alphabetic
- By Inheritance
- AdaptivePatternEstimateFn
- Serializable
- Serializable
- InputOptPartitioner
- InputLayers
- AnyRef
- Any
- Hide All
- Show All
- Public
- All
Abstract Value Members
-
abstract
def
estimateFn(src: Java.Pair[InKey, InMeta]): Iterable[(HereTile, Long)]
Estimate the weights of zero, one or more output HereTiles
Estimate the weights of zero, one or more output HereTiles
- src
Input key and metadata, it can be used to retrieve the data with a com.here.platform.data.processing.java.blobstore.Retriever, although this method is not suggested.
- returns
Output HereTiles with their estimated weights. Weights must be positive.
-
abstract
def
inLayers: Map[String, Set[String]]
Layers of the input catalogs that should be queried and provided to the compiler, grouped by input catalog and identified by catalog id and layer ID.
Layers of the input catalogs that should be queried and provided to the compiler, grouped by input catalog and identified by catalog id and layer ID.
- Definition Classes
- InputLayers
-
abstract
def
inPartitioner(parallelism: Int): Option[PartitionerOfKey]
If the returned com.here.platform.data.processing.java.Java.Option is Empty, the Executor will use a default partitioner, if it is defined then the given partitioner will be applied when querying the input catalogs.
If the returned com.here.platform.data.processing.java.Java.Option is Empty, the Executor will use a default partitioner, if it is defined then the given partitioner will be applied when querying the input catalogs.
- parallelism
the parallelism of the partitioner
- returns
the input optional partitioner with the given parallelism
- Definition Classes
- InputOptPartitioner
Concrete Value Members
-
final
def
!=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
##(): Int
- Definition Classes
- AnyRef → Any
-
final
def
==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
asInstanceOf[T0]: T0
- Definition Classes
- Any
-
def
clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( ... ) @native()
-
final
def
eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
def
equals(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
def
finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( classOf[java.lang.Throwable] )
-
final
def
getClass(): Class[_]
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
-
def
hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
-
final
def
isInstanceOf[T0]: Boolean
- Definition Classes
- Any
-
final
def
ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
final
def
notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
-
final
def
notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
-
final
def
synchronized[T0](arg0: ⇒ T0): T0
- Definition Classes
- AnyRef
-
def
toString(): String
- Definition Classes
- AnyRef → Any
-
final
def
wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... ) @native()