Reductions
delimiter is configurable), determine the geographic midpoint. See “geographic midpoint” at: http://www.geomidpoint.com/methods.html For implementation algorithm, see: http://www.geomidpoint.com/calculation.html
transform
public Schema transform(Schema inputSchema)
- param delim Delimiter for the coordinates in text format. For example, if format is “lat,long” use “,”
A StringReducer is used to take a set of examples and reduce them. The idea: suppose you have a large number of columns, and you want to combine/reduce the values in each column.
StringReducer allows you to specify different reductions for differently for different columns: min, max, sum, mean etc.
Uses are: (1) Reducing examples by a key (2) Reduction operations in time series (windowing ops, etc)
transform
public Schema transform(Schema schema)
Get the output schema, given the input schema
outputColumnName
public Builder outputColumnName(String outputColumnName)
Create a StringReducer builder, and set the default column reduction operation. For any columns that aren’t specified explicitly, they will use the default reduction operation. If a column does have a reduction operation explicitly specified, then it will override the default specified here.
- param defaultOp Default reduction operation to perform
appendColumns
public Builder appendColumns(String... columns)
Reduce the specified columns by taking the minimum value
prependColumns
public Builder prependColumns(String... columns)
Reduce the specified columns by taking the maximum value
mergeColumns
public Builder mergeColumns(String... columns)
Reduce the specified columns by taking the sum of values
replaceColumn
public Builder replaceColumn(String... columns)
Reduce the specified columns by taking the mean of the values
customReduction
public Builder customReduction(String column, ColumnReduction columnReduction)
Reduce the specified column using a custom column reduction functionality.
- param column Column to execute the custom reduction functionality on
- param columnReduction Column reduction to execute on that column
setIgnoreInvalid
public Builder setIgnoreInvalid(String... columns)
When doing the reduction: set the specified columns to ignore any invalid values. Invalid: defined as being not valid according to the ColumnMetaData: {- link ColumnMetaData#isValid(Writable)}. For numerical columns, this typically means being unable to parse the Writable. For example, Writable.toLong() failing for a Long column. If the column has any restrictions (min/max values, regex for Strings etc) these will also be taken into account.
- param columns Columns to set ‘ignore invalid’ for
Last modified 3yr ago