class MultiExecutorsNonIncrementalMode extends MultiExecutorsBaseMode
Runs a sequence of com.here.platform.data.processing.driver.Executors in non-incremental mode.
This driver task allows the running of a sequence of compilers where every subsequent executor is provided with data produced by the previous one. This allows the execution of multiple compilers without the need of creating a filter for each of them.
- Note
input layers list of subsequent compilers have to include the needed output layers from the previous ones
,no output layer overwriting is allowed in the current implementation
,users of this class are strongly encouraged to build the related driver task via the com.here.platform.data.processing.driver.MultiCompilerTaskBuilder. This guarantees a type safe approach in the construction. Moreover, the mentioned builders assigns to each executor a unique id based on the driver task id set by the user.
- Alphabetic
- By Inheritance
- MultiExecutorsNonIncrementalMode
- MultiExecutorsBaseMode
- ContextAwareMode
- TaskMode
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Instance Constructors
- new MultiExecutorsNonIncrementalMode(context: DriverContext, id: Id, executors: Seq[Executor])
- context
the com.here.platform.data.processing.driver.DriverContext relative to this mode
- id
the identifier of the com.here.platform.data.processing.driver.DriverTask the mode belongs to, for logging reasons
- executors
the executor that should sequentially run
Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- def actualRun(): RDD[(Key, Commit)]
- Attributes
- protected
- def applyChanges(existingData: RDD[(OutKey, OutMeta)], outCommits: RDD[(OutKey, OutCommit)], version: Version): (RDD[(OutKey, InMeta)], RDD[(OutKey, Change)])
Propagates changes into the input metadata to simulate a commit.
Propagates changes into the input metadata to simulate a commit.
In multi-compiler full compilation intermediate results are only published to the blobstore, while metadata are kept and committed at the end of the executor sequence execution. This means that for a subsequent executor changes done by the previous one has to appear in the input metadata.
- existingData
the status of the out data before compilation
- outCommits
the just completed executor changes
- version
the version to be committed
- returns
the new state of the input metadata for the incoming execution
- Attributes
- protected
- Definition Classes
- MultiExecutorsBaseMode
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @IntrinsicCandidate() @native()
- val context: DriverContext
- Definition Classes
- ContextAwareMode
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @IntrinsicCandidate() @native()
- def getNextVersion(currentBase: BaseVersion): Version
Increments the base version of the output catalog.
Increments the base version of the output catalog.
In multi-compiler driver tasks we need to simulate a publishing and the related commit. For this reason we need to increase the version of some input data.
- currentBase
the current base version
- Attributes
- protected
- Definition Classes
- MultiExecutorsBaseMode
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @IntrinsicCandidate() @native()
- def isApplicable: Boolean
This mode is always applicable, as it's always possible to compile the input catalogs from scratch.
This mode is always applicable, as it's always possible to compile the input catalogs from scratch.
- returns
if the mode is applicable, so if the mode should run and its result evaluated
- Definition Classes
- MultiExecutorsNonIncrementalMode → TaskMode
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @IntrinsicCandidate() @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @IntrinsicCandidate() @native()
- val parallelism: Int
- Attributes
- protected
- Definition Classes
- ContextAwareMode
- def run(): TaskResult
Runs the compiler ignoring changes and compiling always the whole input catalogs.
Runs the compiler ignoring changes and compiling always the whole input catalogs.
- returns
the result of the compilation
- Definition Classes
- MultiExecutorsNonIncrementalMode → TaskMode
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def toString(): String
- Definition Classes
- AnyRef → Any
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
Deprecated Value Members
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable]) @Deprecated
- Deprecated
(Since version 9)