Type Members

1.  case class BackwardResolution[K, V](references: DeltaSet[Key, Meta], referencesToSubjects: (Key, Meta) => Set[K])(implicit evidence$4: Ordering[K], evidence$5: ClassTag[K], evidence$6: ClassTag[V]) extends ResolutionStrategy[K, V] with Product with Serializable

   Given a DeltaSet containing references and a function that maps each reference to a set of subject partitions, BackwardResolution makes the mapped references available to the resolver when processing the subject partition, without any further network query.

   Given a DeltaSet containing references and a function that maps each reference to a set of subject partitions, BackwardResolution makes the mapped references available to the resolver when processing the subject partition, without any further network query.

   references

   The DeltaSet containing the referenced partition key-meta pairs.

   referencesToSubjects

   The function that maps each reference to a set of referenced partition keys.

2.  case class Broadcast(references: DeltaSet[Key, Meta]) extends ResolutionStrategy[Any, Any] with Product with Serializable

   Sends a full copy of the metadata contained in a given DeltaSet to all Spark executors in the cluster.

   Sends a full copy of the metadata contained in a given DeltaSet to all Spark executors in the cluster. The metadata will be completely available to mapValuesWithResolver; no further network requests are required for resolving an existing key.

   references

   The DeltaSet containing the metadata.

   Note

   The references DeltaSet is evaluated non-incrementally by this strategy.

3.  case class DirectQuery(referenceCatalog: Id, referenceLayers: Set[Id], cacheSize: Long = 10000) extends ResolutionStrategy[Any, Any] with Product with Serializable

   Resolves a reference by directly querying the Data API.

   Resolves a reference by directly querying the Data API.

   referenceCatalog

   The catalog to query.

   referenceLayers

   The layers to query.

   cacheSize

   The number of key-meta pairs to keep in a local cache to avoid querying the service several times for the same key. Defaults to 10000.

4.  case class ForwardResolution[K, V](references: DeltaSet[Key, Meta], subjectToReferences: (K, V) => Set[Key])(implicit evidence$1: Ordering[K], evidence$2: ClassTag[K], evidence$3: ClassTag[V]) extends ResolutionStrategy[K, V] with Product with Serializable

   Given a DeltaSet containing references and a function that maps each subject to a set of reference partitions makes the mapped references available to the resolver when processing the subject partition, without any further network query.

   Given a DeltaSet containing references and a function that maps each subject to a set of reference partitions makes the mapped references available to the resolver when processing the subject partition, without any further network query.

   references

   The DeltaSet containing the referenced partition key-meta pairs.

   subjectToReferences

   The function that maps each subject to a set of referenced partition keys.