org.apache.datasketches.pig.sampling

Class ReservoirUnion

java.lang.Object

org.apache.pig.EvalFunc<T>

org.apache.pig.AccumulatorEvalFunc<org.apache.pig.data.Tuple>

org.apache.datasketches.pig.sampling.ReservoirUnion

All Implemented Interfaces:

org.apache.pig.Accumulator<org.apache.pig.data.Tuple>

public class ReservoirUnion
extends org.apache.pig.AccumulatorEvalFunc<org.apache.pig.data.Tuple>

This is a Pig UDF that unions reservoir samples. It implements the Accumulator interface for more efficient performance. Input is assumed to come from the reservoir sampling UDF or to be in a compatible format: (n, k, {(samples)} where n is the total number of items presented to the sketch and k is the maximum size of the sketch.

Author:

Jon Malkin

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.pig.EvalFunc

org.apache.pig.EvalFunc.SchemaType

Field Summary

Fields inherited from class org.apache.pig.EvalFunc

log, pigLogger, reporter, returnType

Constructor Summary

Constructors

Constructor and Description
ReservoirUnion(String kStr) Reservoir sampling constructor.

Method Summary

Modifier and Type	Method and Description
void	accumulate(org.apache.pig.data.Tuple inputTuple)
void	cleanup()
org.apache.pig.data.Tuple	getValue()
org.apache.pig.impl.logicalLayer.schema.Schema	outputSchema(org.apache.pig.impl.logicalLayer.schema.Schema input) Validates format of input schema and returns a matching schema

Methods inherited from class org.apache.pig.AccumulatorEvalFunc

exec

Methods inherited from class org.apache.pig.EvalFunc

allowCompileTimeCalculation, finish, getArgToFuncMapping, getCacheFiles, getInputSchema, getLoadCaster, getLogger, getPigLogger, getReporter, getReturnType, getSchemaName, getSchemaType, getShipFiles, isAsynchronous, needEndOfAllInputProcessing, progress, setEndOfAllInput, setInputSchema, setPigLogger, setReporter, setUDFContextSignature, warn

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ReservoirUnion

public ReservoirUnion(String kStr)

Reservoir sampling constructor.

Parameters:

kStr - String indicating the maximum number of desired entries in the reservoir.

Method Detail

accumulate

public void accumulate(org.apache.pig.data.Tuple inputTuple)
                throws IOException

Specified by:

accumulate in interface org.apache.pig.Accumulator<org.apache.pig.data.Tuple>

Specified by:

accumulate in class org.apache.pig.AccumulatorEvalFunc<org.apache.pig.data.Tuple>

Throws:

IOException

getValue

public org.apache.pig.data.Tuple getValue()

Specified by:

getValue in interface org.apache.pig.Accumulator<org.apache.pig.data.Tuple>

Specified by:

getValue in class org.apache.pig.AccumulatorEvalFunc<org.apache.pig.data.Tuple>

cleanup

public void cleanup()

Specified by:

cleanup in interface org.apache.pig.Accumulator<org.apache.pig.data.Tuple>

Specified by:

cleanup in class org.apache.pig.AccumulatorEvalFunc<org.apache.pig.data.Tuple>

outputSchema

public org.apache.pig.impl.logicalLayer.schema.Schema outputSchema(org.apache.pig.impl.logicalLayer.schema.Schema input)

Validates format of input schema and returns a matching schema

Overrides:

outputSchema in class org.apache.pig.EvalFunc<org.apache.pig.data.Tuple>

Parameters:

input - Expects input to be a bag of sketches: (n, k, {(samples...)})

Returns:

Schema based on the