org.apache.datasketches.pig.cpc

Class UnionSketch

java.lang.Object

org.apache.pig.EvalFunc<org.apache.pig.data.DataByteArray>

org.apache.datasketches.pig.cpc.UnionSketch

All Implemented Interfaces:

org.apache.pig.Accumulator<org.apache.pig.data.DataByteArray>, org.apache.pig.Algebraic

public class UnionSketch
extends org.apache.pig.EvalFunc<org.apache.pig.data.DataByteArray>
implements org.apache.pig.Accumulator<org.apache.pig.data.DataByteArray>, org.apache.pig.Algebraic

This is a Pig UDF that performs the Union operation on CpcSketches. This class implements both the Accumulator and Algebraic interfaces.

Author:

Alexander Saydakov

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.pig.EvalFunc

org.apache.pig.EvalFunc.SchemaType

Field Summary

Fields inherited from class org.apache.pig.EvalFunc

log, pigLogger, reporter, returnType

Constructor Summary

Constructors

Constructor and Description
UnionSketch() Constructor with default lgK and target HLL type
UnionSketch(int lgK, long seed) Base constructor.
UnionSketch(String lgK) Constructor with given lgK as string and default seed.
UnionSketch(String lgK, String seed) Constructor with given lgK and seed as strings

Method Summary

Modifier and Type	Method and Description
void	accumulate(org.apache.pig.data.Tuple inputTuple) An Accumulator version of the standard exec() method.
void	cleanup() Cleans up the UDF state after being called using the Accumulator interface.
org.apache.pig.data.DataByteArray	exec(org.apache.pig.data.Tuple inputTuple) Top-level exec function.
String	getFinal()
String	getInitial()
String	getIntermed()
org.apache.pig.data.DataByteArray	getValue() Returns the sketch that has been built up by multiple calls to accumulate(org.apache.pig.data.Tuple).

Methods inherited from class org.apache.pig.EvalFunc

allowCompileTimeCalculation, finish, getArgToFuncMapping, getCacheFiles, getInputSchema, getLoadCaster, getLogger, getPigLogger, getReporter, getReturnType, getSchemaName, getSchemaType, getShipFiles, isAsynchronous, needEndOfAllInputProcessing, outputSchema, progress, setEndOfAllInput, setInputSchema, setPigLogger, setReporter, setUDFContextSignature, warn

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

UnionSketch

public UnionSketch()

Constructor with default lgK and target HLL type

UnionSketch

public UnionSketch(String lgK)

Constructor with given lgK as string and default seed.

Parameters:

lgK - in a form of a String

UnionSketch

public UnionSketch(String lgK,
                   String seed)

Constructor with given lgK and seed as strings

Parameters:

lgK - in a form of a String

seed - in a form of a String

UnionSketch

public UnionSketch(int lgK,
                   long seed)

Base constructor.

Parameters:

lgK - parameter controlling the sketch size and accuracy

seed - for the hash function

Method Detail

exec

public org.apache.pig.data.DataByteArray exec(org.apache.pig.data.Tuple inputTuple)
                                       throws IOException

Top-level exec function. This method accepts an input Tuple containing a Bag of one or more inner Sketch Tuples and returns a single serialized CpcSketch as a DataByteArray. Sketch Tuple is a Tuple containing a single DataByteArray (BYTEARRAY in Pig), which is a serialized HllSketch.

Specified by:

exec in class org.apache.pig.EvalFunc<org.apache.pig.data.DataByteArray>

Parameters:

inputTuple - A tuple containing a single bag, containing Sketch Tuples.

Returns:

serialized CpcSketch

Throws:

IOException - from Pig.

See Also:

"org.apache.pig.EvalFunc.exec(org.apache.pig.data.Tuple)"

accumulate

public void accumulate(org.apache.pig.data.Tuple inputTuple)
                throws IOException

An Accumulator version of the standard exec() method. Like exec(), accumulator is called with a bag of Sketch Tuples. Unlike exec(), it doesn't serialize the result at the end. Instead, it can be called multiple times, each time with another bag of Sketch Tuples to be input to the union.

Specified by:

accumulate in interface org.apache.pig.Accumulator<org.apache.pig.data.DataByteArray>

Parameters:

inputTuple - A tuple containing a single bag, containing Sketch Tuples.

Throws:

IOException - by Pig

See Also:

exec(org.apache.pig.data.Tuple), "org.apache.pig.Accumulator.accumulate(org.apache.pig.data.Tuple)"

getValue

public org.apache.pig.data.DataByteArray getValue()

Returns the sketch that has been built up by multiple calls to accumulate(org.apache.pig.data.Tuple).

Specified by:

getValue in interface org.apache.pig.Accumulator<org.apache.pig.data.DataByteArray>

Returns:

Sketch Tuple. (see exec(org.apache.pig.data.Tuple) for return tuple format)

See Also:

"org.apache.pig.Accumulator.getValue()"

cleanup

public void cleanup()

Cleans up the UDF state after being called using the Accumulator interface.

Specified by:

cleanup in interface org.apache.pig.Accumulator<org.apache.pig.data.DataByteArray>

See Also:

"org.apache.pig.Accumulator.cleanup()"

getInitial

public String getInitial()

Specified by:

getInitial in interface org.apache.pig.Algebraic

getIntermed

public String getIntermed()

Specified by:

getIntermed in interface org.apache.pig.Algebraic

getFinal

public String getFinal()

Specified by:

getFinal in interface org.apache.pig.Algebraic