org.apache.datasketches.hive.tuple

Class DataToSketchUDAF.DataToSketchEvaluator<U,S extends org.apache.datasketches.tuple.UpdatableSummary<U>>

java.lang.Object

org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator

org.apache.datasketches.hive.tuple.DataToSketchUDAF.DataToSketchEvaluator<U,S>

All Implemented Interfaces:

Closeable, AutoCloseable

Enclosing class:

DataToSketchUDAF

public abstract static class DataToSketchUDAF.DataToSketchEvaluator<U,S extends org.apache.datasketches.tuple.UpdatableSummary<U>>
extends org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator

org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.AbstractAggregationBuffer, org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.AggregationBuffer, org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.AggregationType, org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.Mode

Field Summary

Fields

Modifier and Type	Field and Description
protected org.apache.hadoop.hive.serde2.objectinspector.StructObjectInspector	intermediateInspector_
protected static String	NOMINAL_NUM_ENTRIES_FIELD
protected org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector	nominalNumEntriesInspector_
protected static String	SKETCH_FIELD

Fields inherited from class org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator

partitionEvaluator

Constructor Summary

Constructors

Constructor and Description
DataToSketchEvaluator()

Method Summary

Modifier and Type	Method and Description
U	extractValue(Object data, org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector valueInspector) Override this if it takes more than a cast to convert Hive value into the sketch update type U
org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.AggregationBuffer	getNewAggregationBuffer()
protected abstract org.apache.datasketches.tuple.SummaryDeserializer<S>	getSummaryDeserializer() Get an instance of SummaryDeserializer
protected abstract org.apache.datasketches.tuple.SummaryFactory<S>	getSummaryFactory(Object[] data) Get an instance of SummaryFactory possibly parameterized based on the original input array of objects.
protected abstract org.apache.datasketches.tuple.SummarySetOperations<S>	getSummarySetOperationsForIterate(Object[] data) Get an instance of SummarySetOperations possibly parameterized based on the original input array of objects.
protected abstract org.apache.datasketches.tuple.SummarySetOperations<S>	getSummarySetOperationsForMerge(Object data) Get an instance of SummarySetOperations possibly parameterized based on the intermediate data object.
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector	init(org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.Mode mode, org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector[] inspectors)
protected void	initializeState(org.apache.datasketches.hive.tuple.UnionState<S> state, Object data)
void	iterate(org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.AggregationBuffer buf, Object[] data)
void	merge(org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.AggregationBuffer buf, Object data)
void	reset(org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.AggregationBuffer buf)
Object	terminate(org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.AggregationBuffer buf)
Object	terminatePartial(org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.AggregationBuffer buf)

Methods inherited from class org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator

aggregate, close, configure, createPartitionEvaluator, evaluate, getExprString, getPartitionWindowingEvaluator, getWindowingEvaluator, isEstimable

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

NOMINAL_NUM_ENTRIES_FIELD

protected static final String NOMINAL_NUM_ENTRIES_FIELD

See Also:

Constant Field Values

SKETCH_FIELD

protected static final String SKETCH_FIELD

See Also:

Constant Field Values

nominalNumEntriesInspector_

protected org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector nominalNumEntriesInspector_

intermediateInspector_

protected org.apache.hadoop.hive.serde2.objectinspector.StructObjectInspector intermediateInspector_

Constructor Detail

DataToSketchEvaluator

public DataToSketchEvaluator()

Method Detail

init

public org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector init(org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.Mode mode,
                                                                          org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector[] inspectors)
                                                                   throws org.apache.hadoop.hive.ql.metadata.HiveException

Overrides:

init in class org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator

Throws:

org.apache.hadoop.hive.ql.metadata.HiveException

iterate

public void iterate(org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.AggregationBuffer buf,
                    Object[] data)
             throws org.apache.hadoop.hive.ql.metadata.HiveException

Specified by:

iterate in class org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator

Throws:

org.apache.hadoop.hive.ql.metadata.HiveException

getNewAggregationBuffer

public org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.AggregationBuffer getNewAggregationBuffer()
                                                                                                     throws org.apache.hadoop.hive.ql.metadata.HiveException

Specified by:

getNewAggregationBuffer in class org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator

Throws:

org.apache.hadoop.hive.ql.metadata.HiveException

extractValue

public U extractValue(Object data,
                      org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector valueInspector)
               throws org.apache.hadoop.hive.ql.metadata.HiveException

Override this if it takes more than a cast to convert Hive value into the sketch update type U

Parameters:

data - Hive value object

valueInspector - PrimitiveObjectInspector for the value

Returns:

extracted value

Throws:

org.apache.hadoop.hive.ql.metadata.HiveException - if anything goes wrong

getSummaryDeserializer

protected abstract org.apache.datasketches.tuple.SummaryDeserializer<S> getSummaryDeserializer()

Get an instance of SummaryDeserializer

Returns:

SummaryDeserializer

getSummaryFactory

protected abstract org.apache.datasketches.tuple.SummaryFactory<S> getSummaryFactory(Object[] data)

Get an instance of SummaryFactory possibly parameterized based on the original input array of objects. Might be called once during the first call to iterate.

Parameters:

data - original input array of objects

Returns:

an instance of SummaryFactory

getSummarySetOperationsForIterate

protected abstract org.apache.datasketches.tuple.SummarySetOperations<S> getSummarySetOperationsForIterate(Object[] data)

Get an instance of SummarySetOperations possibly parameterized based on the original input array of objects. Might be called once during the first call to iterate.

Parameters:

data - original input array of objects

Returns:

an instance of SummarySetOperations

terminatePartial

public Object terminatePartial(org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.AggregationBuffer buf)
                        throws org.apache.hadoop.hive.ql.metadata.HiveException

Specified by:

terminatePartial in class org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator

Throws:

org.apache.hadoop.hive.ql.metadata.HiveException

merge

public void merge(org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.AggregationBuffer buf,
                  Object data)
           throws org.apache.hadoop.hive.ql.metadata.HiveException

Specified by:

merge in class org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator

Throws:

org.apache.hadoop.hive.ql.metadata.HiveException

initializeState

protected void initializeState(org.apache.datasketches.hive.tuple.UnionState<S> state,
                               Object data)

getSummarySetOperationsForMerge

protected abstract org.apache.datasketches.tuple.SummarySetOperations<S> getSummarySetOperationsForMerge(Object data)

Get an instance of SummarySetOperations possibly parameterized based on the intermediate data object. Might be called once during the first call to merge.

Parameters:

data - intermediate data object

Returns:

an instance of SummarySetOperations

terminate

public Object terminate(org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.AggregationBuffer buf)
                 throws org.apache.hadoop.hive.ql.metadata.HiveException

Specified by:

terminate in class org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator

Throws:

org.apache.hadoop.hive.ql.metadata.HiveException

reset

public void reset(org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.AggregationBuffer buf)
           throws org.apache.hadoop.hive.ql.metadata.HiveException

Specified by:

reset in class org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator

Throws:

org.apache.hadoop.hive.ql.metadata.HiveException