org.apache.datasketches.pig.sampling

Class ReservoirSampling

java.lang.Object

org.apache.pig.EvalFunc<T>

org.apache.pig.AccumulatorEvalFunc<org.apache.pig.data.Tuple>

org.apache.datasketches.pig.sampling.ReservoirSampling

All Implemented Interfaces:

org.apache.pig.Accumulator<org.apache.pig.data.Tuple>, org.apache.pig.Algebraic

public class ReservoirSampling
extends org.apache.pig.AccumulatorEvalFunc<org.apache.pig.data.Tuple>
implements org.apache.pig.Algebraic

This is a Pig UDF that applies reservoir sampling to input tuples. It implements both the Accumulator and Algebraic interfaces for efficient performance.

Author:

Jon Malkin

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	ReservoirSampling.Initial
static class	ReservoirSampling.IntermediateFinal

Nested classes/interfaces inherited from class org.apache.pig.EvalFunc

org.apache.pig.EvalFunc.SchemaType

Field Summary

Fields inherited from class org.apache.pig.EvalFunc

log, pigLogger, reporter, returnType

Constructor Summary

Constructors

Constructor and Description
ReservoirSampling(String kStr) Reservoir sampling constructor.

Method Summary

Modifier and Type	Method and Description
void	accumulate(org.apache.pig.data.Tuple inputTuple)
void	cleanup()
org.apache.pig.data.Tuple	exec(org.apache.pig.data.Tuple inputTuple)
String	getFinal()
String	getInitial()
String	getIntermed()
org.apache.pig.data.Tuple	getValue()
org.apache.pig.impl.logicalLayer.schema.Schema	outputSchema(org.apache.pig.impl.logicalLayer.schema.Schema input)

Methods inherited from class org.apache.pig.EvalFunc

allowCompileTimeCalculation, finish, getArgToFuncMapping, getCacheFiles, getInputSchema, getLoadCaster, getLogger, getPigLogger, getReporter, getReturnType, getSchemaName, getSchemaType, getShipFiles, isAsynchronous, needEndOfAllInputProcessing, progress, setEndOfAllInput, setInputSchema, setPigLogger, setReporter, setUDFContextSignature, warn

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ReservoirSampling

public ReservoirSampling(String kStr)

Reservoir sampling constructor.

Parameters:

kStr - String indicating the maximum number of desired entries in the reservoir.

Method Detail

exec

public org.apache.pig.data.Tuple exec(org.apache.pig.data.Tuple inputTuple)
                               throws IOException

Overrides:

exec in class org.apache.pig.AccumulatorEvalFunc<org.apache.pig.data.Tuple>

Throws:

IOException

accumulate

public void accumulate(org.apache.pig.data.Tuple inputTuple)
                throws IOException

Specified by:

accumulate in interface org.apache.pig.Accumulator<org.apache.pig.data.Tuple>

Specified by:

accumulate in class org.apache.pig.AccumulatorEvalFunc<org.apache.pig.data.Tuple>

Throws:

IOException

getValue

public org.apache.pig.data.Tuple getValue()

Specified by:

getValue in interface org.apache.pig.Accumulator<org.apache.pig.data.Tuple>

Specified by:

getValue in class org.apache.pig.AccumulatorEvalFunc<org.apache.pig.data.Tuple>

cleanup

public void cleanup()

Specified by:

cleanup in interface org.apache.pig.Accumulator<org.apache.pig.data.Tuple>

Specified by:

cleanup in class org.apache.pig.AccumulatorEvalFunc<org.apache.pig.data.Tuple>

outputSchema

public org.apache.pig.impl.logicalLayer.schema.Schema outputSchema(org.apache.pig.impl.logicalLayer.schema.Schema input)

Overrides:

outputSchema in class org.apache.pig.EvalFunc<org.apache.pig.data.Tuple>

getInitial

public String getInitial()

Specified by:

getInitial in interface org.apache.pig.Algebraic

getIntermed

public String getIntermed()

Specified by:

getIntermed in interface org.apache.pig.Algebraic

getFinal

public String getFinal()

Specified by:

getFinal in interface org.apache.pig.Algebraic