DataFrameAsset

Signature

class great_expectations.datasource.fluent.spark_datasource.DataFrameAsset(&42;, name: str, type: Literal['dataframe'] = 'dataframe', id: Optional[uuid.UUID] = None, order_by: List[great_expectations.datasource.fluent.interfaces.Sorter] = None, batch_metadata: Dict[str, Any] = None, batch_definitions: List[great_expectations.core.batch_definition.BatchDefinition] = None)

A DataAsset that represents a Spark DataFrame.

Methods

Signature

add_batch_definition_whole_dataframe(name: str) → BatchDefinition

Add a BatchDefinition that represents the entire DataFrame.

Parameters

name – The name of the Batch Definition.

Returns

A BatchDefinition object that represents the entire DataFrame.

Signature

delete_batch_definition(name: str) → None

Delete a batch definition.

Parameters

name (str) – Name of the BatchDefinition to delete.

Signature

get_batch_definition(name: str) → great_expectations.core.batch_definition.BatchDefinition

Get a batch definition.

Parameters

name (str) – Name of the BatchDefinition to get.

Raises

KeyError – If the BatchDefinition does not exist.