Package org.apache.datasketches.pig.tuple

Pig UDFs for Tuple sketches.

See: Description

Class Summary

Class	Description
AlgebraicInitial	This is a common pass-through implementation for initial step of an Algebraic operation
ArrayOfDoublesSketchesToPValueEstimates	Calculate p-values given two ArrayOfDoublesSketch.
ArrayOfDoublesSketchToEstimateAndErrorBounds	This is a User Defined Function (UDF) for obtaining the unique count estimate along with a lower and upper bound from an ArrayOfDoublesSketch.
ArrayOfDoublesSketchToEstimates	This UDF converts an ArrayOfDoubles sketch to estimates.
ArrayOfDoublesSketchToMeans	This UDF converts an ArrayOfDoubles sketch to mean values.
ArrayOfDoublesSketchToNumberOfRetainedEntries	This is a User Defined Function (UDF) for obtaining the number of retained entries from an ArrayOfDoublesSketch.
ArrayOfDoublesSketchToQuantilesSketch	This UDF converts a given column of double values from an ArrayOfDoubles sketch to a quantiles DoublesSketch to further analyze the distribution of these values.
ArrayOfDoublesSketchToVariances	This UDF converts an ArrayOfDoubles sketch to variance values.
DataToArrayOfDoublesSketch	This UDF creates an ArrayOfDoublesSketch from raw data.
DataToArrayOfDoublesSketch.Initial
DataToArrayOfDoublesSketch.IntermediateFinal
DataToDoubleSummarySketch	This UDF creates a Sketch<DoubleSummary> from raw data.
DataToDoubleSummarySketch.Initial
DataToDoubleSummarySketch.IntermediateFinal
DataToSketch<U,S extends org.apache.datasketches.tuple.UpdatableSummary<U>>	This is a generic implementation to be specialized in concrete UDFs
DataToSketchAlgebraicIntermediateFinal<U,S extends org.apache.datasketches.tuple.UpdatableSummary<U>>	Class used to calculate the intermediate pass (combiner) or the final pass (reducer) of an Algebraic sketch operation.
DoubleSummarySketchToEstimates	This UDF converts a Sketch<DoubleSummary> to estimates.
DoubleSummarySketchToPercentile	This UDF is to get a percentile value from a Sketch<DoubleSummary>.
UnionArrayOfDoublesSketch	This is to union ArrayOfDoublesSketches.
UnionArrayOfDoublesSketch.Initial
UnionArrayOfDoublesSketch.IntermediateFinal
UnionDoubleSummarySketch	This is to union Sketch<DoubleSummary>.
UnionDoubleSummarySketch.Initial
UnionDoubleSummarySketch.IntermediateFinal
UnionSketch<S extends org.apache.datasketches.tuple.Summary>	This is a generic implementation to be specialized in concrete UDFs
UnionSketchAlgebraicIntermediateFinal<S extends org.apache.datasketches.tuple.Summary>	This is to calculate the intermediate pass (combiner) or the final pass (reducer) of an Algebraic sketch operation.

Package org.apache.datasketches.pig.tuple Description

Pig UDFs for Tuple sketches. Tuple sketches are based on the idea of Theta sketches with the addition of values associated with unique keys. Two sets of tuple sketch classes are available at the moment: generic Tuple sketches with user-defined Summary, and a faster specialized implementation with an array of double values.

There are two sets of Pig UDFs: one for generic Tuple sketch with an example implementation for DoubleSummay, and another one for a specialized ArrayOfDoublesSketch.

The generic implementation is in the form of abstract classes DataToSketch and UnionSketch to be specialized for particular types of Summary. An example implementation for DoubleSumamry is provided: DataToDoubleSummarySketch and UnionDoubleSummarySketch, as well as UDFs to obtain the results from sketches: DoubleSumamrySketchToEstimates and DoubleSummarySketchToPercentile.

UDFs for ArrayOfDoublesSketch: DataToArrayOfDoublesSketch, UnionArrayOfDoublesSketch, ArrayOfDoublesSketchToEstimates.

Author:

Alexander Saydakov