Package com.here.platform.data.client.flink.javadsl

Class FlinkWriteEngineImpl

java.lang.Object
com.here.platform.data.client.flink.javadsl.FlinkWriteEngineImpl
All Implemented Interfaces:
FlinkWriteEngine
public class FlinkWriteEngineImpl extends Object implements FlinkWriteEngine

  • Constructor Details

    • FlinkWriteEngineImpl

      public FlinkWriteEngineImpl(HRN hrn, Settings settings, scala.Function0<com.here.platform.data.client.engine.scaladsl.WriteEngine> toCreateWriteEngine, DataClientFlinkContext context)

  • Method Details

    • blobChecksum

      public Optional<String> blobChecksum(String layer, com.here.platform.data.client.scaladsl.NewPartition.BufferedBlob blob)
      Description copied from interface: FlinkWriteEngine
      Computes a blob's checksum based on its layer configuration.

      The algorithm used is defined accordingly to the layer's configuration. If the layer doesn't have the respective field set, then an empty optional is returned.

      Specified by:
      blobChecksum in interface FlinkWriteEngine
      Parameters:
      layer - the layer that the specified blob is associated to.
      blob - the blob that will have its checksum computed.
      Returns:
      a checksum or empty if the layer's configuration field is not set.

    • blobCrc

      public Optional<String> blobCrc(String layer, com.here.platform.data.client.scaladsl.NewPartition.BufferedBlob blob)
      Description copied from interface: FlinkWriteEngine
      Computes a blob's crc based on its layer configuration.

      The algorithm used is defined accordingly to the layer's configuration. If the layer doesn't have the respective field set, then an empty optional is returned.

      Specified by:
      blobCrc in interface FlinkWriteEngine
      Parameters:
      layer - the layer that the specified blob is associated to.
      blob - the blob that will have its crc computed.
      Returns:
      a crc or empty if the layer's configuration field is not set.

    • copyObject

      public void copyObject(String layerId, String key, String sourceKey)
      Description copied from interface: FlinkWriteEngine
      Copies object from one location to other

      Specified by:
      copyObject in interface FlinkWriteEngine
      Parameters:
      layerId - the id of the layer
      key - the key of the destination object
      sourceKey - the key of the source object

    • deleteObject

      public void deleteObject(String layerId, String key)
      Description copied from interface: FlinkWriteEngine
      Deletes the specified object from object store.

      Specified by:
      deleteObject in interface FlinkWriteEngine
      Parameters:
      layerId - the id of the layer
      key - the key of the object to be deleted

    • publish

      public org.apache.flink.api.connector.sink2.Sink<PendingPartition> publish()
      Description copied from interface: FlinkWriteEngine
      Publish new partitions to a catalog.

      This method can be used to publish new partitions to stream layer.

      Specified by:
      publish in interface FlinkWriteEngine
      Returns:
      a sink which all partitions are published to server.

    • put

      public CommitPartition put(PendingPartition partition)
      Description copied from interface: FlinkWriteEngine
      Adds the selected partition to the underlying blobstore

      Specified by:
      put in interface FlinkWriteEngine
      Parameters:
      partition - the partition to add
      Returns:
      a partition that could be committed

    • updateObjectTtl

      public void updateObjectTtl(String layerId, String key, long ttlSeconds)
      Description copied from interface: FlinkWriteEngine
      Updates the TTL for the object stored in Volatile Layer

      Specified by:
      updateObjectTtl in interface FlinkWriteEngine
      Parameters:
      layerId - the id of the layer
      key - the key of the target object
      ttlSeconds - new value of ttl for the object

    • uploadIMLFeatureCollection

      public void uploadIMLFeatureCollection(String layer, FeatureCollection featureCollection, OptionalInt batchSize)
      Description copied from interface: FlinkWriteEngine
      Upload a large FeatureCollection to the Interactive Map Layer

      Specified by:
      uploadIMLFeatureCollection in interface FlinkWriteEngine
      Parameters:
      layer - the id of the layer
      featureCollection - the FeatureCollection to upload
      batchSize - the number of features in an upload batch

    • uploadObject2

      public void uploadObject2(String layerId, String key, com.here.platform.data.client.scaladsl.NewPartition.Blob blob, Optional<String> contentType, Optional<ContentEncoding> contentEncoding)
      Description copied from interface: FlinkWriteEngine
      Uploads the specified object to object store

      Specified by:
      uploadObject2 in interface FlinkWriteEngine
      Parameters:
      layerId - the id of the layer
      key - the key of the object to be uploaded
      blob - the blob of data that is to be uploaded
      contentType - the optional content-type of the object
      contentEncoding - the optional content-encoding of the object