package publisher
- Alphabetic
- Public
- All
Type Members
-
sealed abstract
class
NonEmptyPublishingAction extends PublishingAction
A non empty publishing action, meaning any action other than PublishingAction.Skip.
-
class
Publisher extends AnyRef
Publishes payloads, in the form of an RDD of com.here.platform.data.processing.catalog.Partition.Key and com.here.platform.data.processing.blobstore.Payload, to the Blob API and returns RDD of com.here.platform.data.processing.catalog.Partition.Commit to be committed to the Metadata API.
Publishes payloads, in the form of an RDD of com.here.platform.data.processing.catalog.Partition.Key and com.here.platform.data.processing.blobstore.Payload, to the Blob API and returns RDD of com.here.platform.data.processing.catalog.Partition.Commit to be committed to the Metadata API.
- Note
both the full snapshot and the incremental one are implemented here the difference is mainly in the joining strategy adopted
,metadata related to the published payloads need to be committed in a subsequent operation
-
sealed abstract
class
PublishingAction extends Product with Serializable
A publishing action, result of the correlation of a candidate payload, the existing metadata and the change in the base catalog (in case of composite layers).
-
trait
PublishingMode extends AnyRef
Represent a policy to publish payloads to a catalog.
Value Members
-
object
DataPublisher
This companion object defines some utility function to be used by the publisher.
This companion object defines some utility function to be used by the publisher. It is mainly devoted at controlling which objects are serialized towards workers
-
object
FullPublishingMode extends PublishingMode
Publishes the candidates interpreting them as a new complete snapshot of the catalog.
Publishes the candidates interpreting them as a new complete snapshot of the catalog. This removes all the data in the catalog that is not included in the candidates.
-
object
IncrementalPublishingMode extends PublishingMode
Publishes the candidates interpreting them as what should be changed in the catalog.
Publishes the candidates interpreting them as what should be changed in the catalog. What is added or modified is uploaded, what is marked for deletion is deleted. Other keys present in the catalog are left unchanged.
- object PublishingAction extends Serializable
- object PublishingMode