org.apache.datasketches.pig.tuple

Class DataToSketchAlgebraicIntermediateFinal<U,S extends org.apache.datasketches.tuple.UpdatableSummary<U>>

java.lang.Object

org.apache.pig.EvalFunc<org.apache.pig.data.Tuple>

org.apache.datasketches.pig.tuple.DataToSketchAlgebraicIntermediateFinal<U,S>

Type Parameters:

U - Type of the update value

S - Type of the summary

Direct Known Subclasses:

DataToDoubleSummarySketch.IntermediateFinal

public abstract class DataToSketchAlgebraicIntermediateFinal<U,S extends org.apache.datasketches.tuple.UpdatableSummary<U>>
extends org.apache.pig.EvalFunc<org.apache.pig.data.Tuple>

Class used to calculate the intermediate pass (combiner) or the final pass (reducer) of an Algebraic sketch operation. This may be called multiple times (from the mapper and from the reducer). It will receive a bag of values returned by either the Intermediate stage or the Initial stages, so it needs to be able to differentiate between and interpret both types.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.pig.EvalFunc

org.apache.pig.EvalFunc.SchemaType

Field Summary

Fields inherited from class org.apache.pig.EvalFunc

log, pigLogger, reporter, returnType

Constructor Summary

Constructors

Constructor and Description
DataToSketchAlgebraicIntermediateFinal(int sketchSize, float samplingProbability, org.apache.datasketches.tuple.SummaryFactory<S> summaryFactory, org.apache.datasketches.tuple.SummarySetOperations<S> summarySetOps, org.apache.datasketches.tuple.SummaryDeserializer<S> summaryDeserializer) Constructs a function given a sketch size, sampling probability, summary factory, summary set operations and summary deserializer
DataToSketchAlgebraicIntermediateFinal(int sketchSize, org.apache.datasketches.tuple.SummaryFactory<S> summaryFactory, org.apache.datasketches.tuple.SummarySetOperations<S> summarySetOps, org.apache.datasketches.tuple.SummaryDeserializer<S> summaryDeserializer) Constructs a function given a sketch size, summary factory, summary set operations, summary deserializer and default sampling probability of 1.
DataToSketchAlgebraicIntermediateFinal(org.apache.datasketches.tuple.SummaryFactory<S> summaryFactory, org.apache.datasketches.tuple.SummarySetOperations<S> summarySetOps, org.apache.datasketches.tuple.SummaryDeserializer<S> summaryDeserializer) Constructs a function given a summary factory, summary set operations, summary deserializer, default sketch size and default sampling probability of 1.

Method Summary

Modifier and Type	Method and Description
org.apache.pig.data.Tuple	exec(org.apache.pig.data.Tuple inputTuple)

Methods inherited from class org.apache.pig.EvalFunc

allowCompileTimeCalculation, finish, getArgToFuncMapping, getCacheFiles, getInputSchema, getLoadCaster, getLogger, getPigLogger, getReporter, getReturnType, getSchemaName, getSchemaType, getShipFiles, isAsynchronous, needEndOfAllInputProcessing, outputSchema, progress, setEndOfAllInput, setInputSchema, setPigLogger, setReporter, setUDFContextSignature, warn

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DataToSketchAlgebraicIntermediateFinal

public DataToSketchAlgebraicIntermediateFinal(org.apache.datasketches.tuple.SummaryFactory<S> summaryFactory,
                                              org.apache.datasketches.tuple.SummarySetOperations<S> summarySetOps,
                                              org.apache.datasketches.tuple.SummaryDeserializer<S> summaryDeserializer)

Constructs a function given a summary factory, summary set operations, summary deserializer, default sketch size and default sampling probability of 1.

Parameters:

summaryFactory - an instance of SummaryFactory

summarySetOps - an instance of SummarySetOperaions

summaryDeserializer - an instance of SummaryDeserializer

DataToSketchAlgebraicIntermediateFinal

public DataToSketchAlgebraicIntermediateFinal(int sketchSize,
                                              org.apache.datasketches.tuple.SummaryFactory<S> summaryFactory,
                                              org.apache.datasketches.tuple.SummarySetOperations<S> summarySetOps,
                                              org.apache.datasketches.tuple.SummaryDeserializer<S> summaryDeserializer)

Constructs a function given a sketch size, summary factory, summary set operations, summary deserializer and default sampling probability of 1.

Parameters:

sketchSize - parameter controlling the size of the sketch and the accuracy. It represents nominal number of entries in the sketch. Forced to the nearest power of 2 greater than given value.

summaryFactory - an instance of SummaryFactory

summarySetOps - an instance of SummarySetOperaions

summaryDeserializer - an instance of SummaryDeserializer

DataToSketchAlgebraicIntermediateFinal

public DataToSketchAlgebraicIntermediateFinal(int sketchSize,
                                              float samplingProbability,
                                              org.apache.datasketches.tuple.SummaryFactory<S> summaryFactory,
                                              org.apache.datasketches.tuple.SummarySetOperations<S> summarySetOps,
                                              org.apache.datasketches.tuple.SummaryDeserializer<S> summaryDeserializer)

Constructs a function given a sketch size, sampling probability, summary factory, summary set operations and summary deserializer

Parameters:

sketchSize - parameter controlling the size of the sketch and the accuracy. It represents nominal number of entries in the sketch. Forced to the nearest power of 2 greater than given value.

samplingProbability - parameter from 0 to 1 inclusive

summaryFactory - an instance of SummaryFactory

summarySetOps - an instance of SummarySetOperaions

summaryDeserializer - an instance of SummaryDeserializer

Method Detail

exec

public org.apache.pig.data.Tuple exec(org.apache.pig.data.Tuple inputTuple)
                               throws IOException

Specified by:

exec in class org.apache.pig.EvalFunc<org.apache.pig.data.Tuple>

Throws:

IOException