Package com.here.platform.data.client.spark

Class LayerDataFrameReader.SparkSessionExt

java.lang.Object

com.here.platform.data.client.spark.LayerDataFrameReader.SparkSessionExt

Enclosing interface:

LayerDataFrameReader

public static class LayerDataFrameReader.SparkSessionExt
extends Object

Implicit class to simplify the creation of custom Spark DataFrameReaders.

For example:

 import com.here.platform.data.client.spark.LayerDataFrameReader.SparkSessionExt

 val spark: SparkSession = ???
 val dataFrameReader: LayerDataFrameReader = spark.readLayer(catalogHrn, layer)
 val dataFrame: DataFrame = spark.readLayer(catalogHrn, layer).load()
 

param: sparkSession SparkSession to use

Constructor Summary

Constructors

Constructor	Description
SparkSessionExt(org.apache.spark.sql.SparkSession sparkSession)

Method Summary

Modifier and Type	Method	Description
LayerDataFrameReader	readLayer(HRN catalogHrn, String layerId)
LayerUpdater	updateLayer(HRN catalogHrn, String layerId)

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

SparkSessionExt

public SparkSessionExt(org.apache.spark.sql.SparkSession sparkSession)

Method Details

readLayer

public LayerDataFrameReader readLayer(HRN catalogHrn,
 String layerId)

updateLayer

public LayerUpdater updateLayer(HRN catalogHrn,
 String layerId)