object Partition
Groups common types to represent partitions, both generic and tiles, as well as various types of metadata that are used in the various stages of processing. The information includes metadata returned by querying the catalog for a particular version, metadata returned by querying the differences between two versions, and metadata provided when committing a new version of a catalog.
- Alphabetic
- By Inheritance
- Partition
- AnyRef
- Any
- Hide All
- Show All
- Public
- All
Type Members
-
sealed
trait
Change extends Serializable
Changes returned by querying DataStore.
Changes returned by querying DataStore.
Queries can result in retrieving the latest metadata as well the metadata at a given version, or metadata that was changed between two versions.
-
implicit final
class
ChangeWrapper extends AnyVal
Provides a key/meta semantic to a pair of Key and Meta.
-
final
class
Checksum extends Serializable
Part of Partition metadata, the checksum of the actual payload.
-
sealed
trait
Commit extends Serializable
Represents metadata being published to the Data API as part of a new commit.
Represents metadata being published to the Data API as part of a new commit.
A commit is always incremental: partitions can be added, changed or deleted. To publish a snapshot, all the added and changed partitions must be published as such while all the partitions not mentioned in the snapshot but present in the last version of the catalog must be deleted.
-
implicit final
class
CommitWrapper[T <: Commit] extends AnyVal
Provides a key/commit semantic to a pair of Key and Commit.
-
type
DataHandle = String
Part of Partition metadata, the dataHandle identifying of the payload.
-
case class
Deleted(version: Version) extends Change with Product with Serializable
Represents a partition that was deleted in a given version.
Represents a partition that was deleted in a given version. This can only be returned by querying the metadata changed between two versions.
-
case class
Generic(name: String) extends Name with Product with Serializable
Name of generic partition.
-
case class
HereTile(tileId: Long) extends Name with Product with Serializable
Name of heretile partition.
-
case class
Key(catalog: Id, layer: Id, partition: Name) extends KeyOrName with Product with Serializable
A key to uniquely identify a partition in compilers, driver tasks and, in general RDDs.
A key to uniquely identify a partition in compilers, driver tasks and, in general RDDs.
As it contains the identifier of the corresponding catalog, partition keys of multiple catalogs can be processed in the same RDD without collisions. Output catalogs have also their unique ID, so it is in theory possible to mix input and output partitions in the same RDD.
- catalog
The ID of the catalog.
- layer
The ID of the layer.
- partition
The name of the partition, which is a tile ID for tiled layers and a generic string for generic layers.
-
sealed
trait
KeyOrName extends AnyRef
A common supertype for Key and Name.
A common supertype for Key and Name. Used to define partitioners that work for both Key and Name, but not on other types.
-
sealed
class
Meta extends Change
Represents a partition present in the given version of the catalog or a partition added or changed when querying changes between two versions.
Represents a partition present in the given version of the catalog or a partition added or changed when querying changes between two versions. In this case the version indicates at which intermediate version between the two the partition was added or changed.
-
implicit final
class
MetaWrapper extends AnyVal
Provides a key/meta semantic to a pair of Key and Meta.
-
sealed
trait
Name extends Serializable with KeyOrName
Name of the partition unique inside a layer.
Name of the partition unique inside a layer.
Partition names may form a hierarchy. A name may have a parent name, that may also recursively have a parent name, until a name with no parent is reached.
-
case class
PullFromBase(meta: Meta) extends Commit with Product with Serializable
Represents a partition that is being published in the base catalog and it is inherited as a "virtual" commit.
Represents a partition that is being published in the base catalog and it is inherited as a "virtual" commit. Only applicable in case of composite layers and when the base catalog version is updated.
From a publishing perspective this is a "virtual" commit, meaning it is not actually published. It is logically relevant when "read back" within the same commit, as it does represent a change with respect to the previous version of the catalog. But the change is inherited from the base catalog rather than being a direct change of the extension layer.
-
class
Put extends Commit
Represents the addition of a new partition or the change of an existing partition.
-
case class
RebasePut(put: Put) extends RebaseCommit with Product with Serializable
Represents a partition that is being re-published to override a base change during a rebase.
Represents a partition that is being re-published to override a base change during a rebase. Only applicable in case of composite layers and when the base catalog version is updated.
While technically identical to a simple Put commit, it is logically different when "read back" within the same commit, as it does not represent a change with respect to the previous version of the catalog.
- put
The commit used to restore a partition in the composite layer that would be otherwise overridden by a base change.
-
implicit final
class
ValueWrapper[T] extends AnyVal
Provides a key semantic to a pair of Key and something else.
Value Members
-
final
def
!=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
##(): Int
- Definition Classes
- AnyRef → Any
-
final
def
==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
asInstanceOf[T0]: T0
- Definition Classes
- Any
-
def
clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( ... ) @native() @IntrinsicCandidate()
-
final
def
eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
def
equals(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
getClass(): Class[_]
- Definition Classes
- AnyRef → Any
- Annotations
- @native() @IntrinsicCandidate()
-
def
hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native() @IntrinsicCandidate()
-
final
def
isInstanceOf[T0]: Boolean
- Definition Classes
- Any
-
final
def
ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
final
def
notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native() @IntrinsicCandidate()
-
final
def
notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native() @IntrinsicCandidate()
-
final
def
synchronized[T0](arg0: ⇒ T0): T0
- Definition Classes
- AnyRef
-
def
toString(): String
- Definition Classes
- AnyRef → Any
-
final
def
wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... ) @native()
-
final
def
wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
- object Checksum extends Serializable
-
object
Delete extends Commit with Product with Serializable
Represents the deletion of a partition.
-
object
DeleteFromBase extends Commit with Product with Serializable
Represents a partition that is being deleted in the base catalog and it is inherited as a "virtual" commit.
Represents a partition that is being deleted in the base catalog and it is inherited as a "virtual" commit. Only applicable in case of composite layers and when the base catalog version is updated.
From a publishing perspective this is a "virtual" commit, meaning it is not actually published. It is logically relevant when "read back" within the same commit, as it does represent a change with respect to the previous version of the catalog. But the change is inherited from the base catalog rather than being a direct change of the extension layer.
- object Generic extends Serializable
- object HereTile extends Serializable
- object Key extends Serializable
-
object
Meta extends Serializable
Provides the construction/extractor interface with nominal parameters.
- object Name extends Serializable
-
object
Put extends Serializable
Provides the construction/extractor interface with nominal parameters.
-
object
RebaseDelete extends RebaseCommit with Product with Serializable
Represents a partition that is being re-deleted when the same partition is being added or changed in the base catalog during a rebase.
Represents a partition that is being re-deleted when the same partition is being added or changed in the base catalog during a rebase. Only applicable in case of composite layers and when the base catalog version is updated.
While technically identical to a simple Delete commit, it is logically different when "read back" within the same commit, as it does not represent a change with respect to the previous version of the catalog.
Deprecated Value Members
-
def
finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( classOf[java.lang.Throwable] ) @Deprecated
- Deprecated