weka.classifiers.meta
Class LogitBoost

java.lang.Object
  extended byweka.classifiers.Classifier
      extended byweka.classifiers.DistributionClassifier
          extended byweka.classifiers.meta.LogitBoost
All Implemented Interfaces:
java.lang.Cloneable, OptionHandler, java.io.Serializable, Sourcable, WeightedInstancesHandler

public class LogitBoost
extends DistributionClassifier
implements OptionHandler, Sourcable, WeightedInstancesHandler

Class for boosting any classifier that can handle weighted instances. This class performs classification using a regression scheme as the base learner, and can handle multi-class problems. For more information, see

Friedman, J., T. Hastie and R. Tibshirani (1998) Additive Logistic Regression: a Statistical View of Boosting download postscript.

Valid options are:

-D
Turn on debugging output.

-W classname
Specify the full class name of a weak learner as the basis for boosting (required).

-I num
Set the number of boost iterations (default 10).

-Q
Use resampling instead of reweighting.

-S seed
Random number seed for resampling (default 1).

-P num
Set the percentage of weight mass used to build classifiers (default 100).

-F num
Set number of folds for the internal cross-validation (default 0 -- no cross-validation).

-R num
Set number of runs for the internal cross-validation (default 1).

-L num
Set the threshold for the improvement of the average loglikelihood (default -Double.MAX_VALUE).

-H num
Set the value of the shrinkage parameter (default 1).

Options after -- are passed to the designated learner.

See Also:
Serialized Form

Field Summary
protected  Attribute m_ClassAttribute
          The actual class attribute (for getting class names)
protected  Classifier m_Classifier
          An instantiated base classifier used for getting and testing options
protected  Classifier[][] m_Classifiers
          Array for storing the generated base classifiers.
protected  boolean m_Debug
          Debugging mode, gives extra output if true
protected  int m_MaxIterations
          The maximum number of boost iterations
protected  int m_NumClasses
          The number of classes
protected  Instances m_NumericClassData
          Dummy dataset with a numeric class
protected  int m_NumFolds
          The number of folds for the internal cross-validation.
protected  int m_NumIterations
          The number of successfully generated base classifiers.
protected  int m_NumRuns
          The number of runs for the internal cross-validation.
protected  double m_Offset
          The value by which the actual target value for the true class is offset.
protected  double m_Precision
          The threshold on the improvement of the likelihood
protected  java.util.Random m_RandomInstance
          The random number generator used
protected  int m_Seed
          Seed for boosting with resampling.
protected  double m_Shrinkage
          The value of the shrinkage parameter
protected  boolean m_UseResampling
          Use boosting with reweighting?
protected  int m_WeightThreshold
          Weight thresholding.
protected static double Z_MAX
          A threshold for responses (Friedman suggests between 2 and 4)
 
Constructor Summary
LogitBoost()
           
 
Method Summary
 void buildClassifier(Instances data)
          Builds the boosted classifier
 Classifier[][] classifiers()
          Returns the array of classifiers that have been built.
 double[] distributionForInstance(Instance instance)
          Calculates the class membership probabilities for the given test instance.
 Classifier getClassifier()
          Get the classifier used as the classifier
 boolean getDebug()
          Get whether debugging is turned on
 double getLikelihoodThreshold()
          Get the value of Precision.
 int getMaxIterations()
          Get the maximum number of boost iterations
 int getNumFolds()
          Get the value of NumFolds.
 int getNumRuns()
          Get the value of NumRuns.
 java.lang.String[] getOptions()
          Gets the current settings of the Classifier.
 int getSeed()
          Get seed for resampling.
 double getShrinkage()
          Get the value of Shrinkage.
 boolean getUseResampling()
          Get whether resampling is turned on
 int getWeightThreshold()
          Get the degree of weight thresholding
 java.util.Enumeration listOptions()
          Returns an enumeration describing the available options.
static void main(java.lang.String[] argv)
          Main method for testing this class.
protected  Instances selectWeightQuantile(Instances data, double quantile)
          Select only instances with weights that contribute to the specified quantile of the weight distribution
 void setClassifier(Classifier newClassifier)
          Set the classifier for boosting.
 void setDebug(boolean debug)
          Set debugging mode
 void setLikelihoodThreshold(double newPrecision)
          Set the value of Precision.
 void setMaxIterations(int maxIterations)
          Set the maximum number of boost iterations
 void setNumFolds(int newNumFolds)
          Set the value of NumFolds.
 void setNumRuns(int newNumRuns)
          Set the value of NumRuns.
 void setOptions(java.lang.String[] options)
          Parses a given list of options.
 void setSeed(int seed)
          Set seed for resampling.
 void setShrinkage(double newShrinkage)
          Set the value of Shrinkage.
 void setUseResampling(boolean r)
          Set resampling mode
 void setWeightThreshold(int threshold)
          Set weight thresholding
 java.lang.String toSource(java.lang.String className)
          Returns the boosted model as Java source code.
 java.lang.String toString()
          Returns description of the boosted classifier.
 
Methods inherited from class weka.classifiers.DistributionClassifier
calculateEntropy, calculateLabeledInstanceMargin, calculateMargin, classifyInstance
 
Methods inherited from class weka.classifiers.Classifier
forName, makeCopies
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
 

Field Detail

m_Classifiers

protected Classifier[][] m_Classifiers
Array for storing the generated base classifiers.


m_Classifier

protected Classifier m_Classifier
An instantiated base classifier used for getting and testing options


m_MaxIterations

protected int m_MaxIterations
The maximum number of boost iterations


m_NumClasses

protected int m_NumClasses
The number of classes


m_NumIterations

protected int m_NumIterations
The number of successfully generated base classifiers.


m_NumFolds

protected int m_NumFolds
The number of folds for the internal cross-validation.


m_NumRuns

protected int m_NumRuns
The number of runs for the internal cross-validation.


m_WeightThreshold

protected int m_WeightThreshold
Weight thresholding. The percentage of weight mass used in training


m_Debug

protected boolean m_Debug
Debugging mode, gives extra output if true


Z_MAX

protected static final double Z_MAX
A threshold for responses (Friedman suggests between 2 and 4)

See Also:
Constant Field Values

m_NumericClassData

protected Instances m_NumericClassData
Dummy dataset with a numeric class


m_ClassAttribute

protected Attribute m_ClassAttribute
The actual class attribute (for getting class names)


m_UseResampling

protected boolean m_UseResampling
Use boosting with reweighting?


m_Seed

protected int m_Seed
Seed for boosting with resampling.


m_Precision

protected double m_Precision
The threshold on the improvement of the likelihood


m_Shrinkage

protected double m_Shrinkage
The value of the shrinkage parameter


m_RandomInstance

protected java.util.Random m_RandomInstance
The random number generator used


m_Offset

protected double m_Offset
The value by which the actual target value for the true class is offset.

Constructor Detail

LogitBoost

public LogitBoost()
Method Detail

selectWeightQuantile

protected Instances selectWeightQuantile(Instances data,
                                         double quantile)
Select only instances with weights that contribute to the specified quantile of the weight distribution

Parameters:
data - the input instances
quantile - the specified quantile eg 0.9 to select 90% of the weight mass
Returns:
the selected instances

listOptions

public java.util.Enumeration listOptions()
Returns an enumeration describing the available options.

Specified by:
listOptions in interface OptionHandler
Returns:
an enumeration of all the available options.

setOptions

public void setOptions(java.lang.String[] options)
                throws java.lang.Exception
Parses a given list of options. Valid options are:

-D
Turn on debugging output.

-W classname
Specify the full class name of a weak learner as the basis for boosting (required).

-I num
Set the number of boost iterations (default 10).

-Q
Use resampling instead of reweighting.

-S seed
Random number seed for resampling (default 1).

-P num
Set the percentage of weight mass used to build classifiers (default 100).

-F num
Set number of folds for the internal cross-validation (default 0 -- no cross-validation).

-R num
Set number of runs for the internal cross-validation (default 1.

-L num
Set the threshold for the improvement of the average loglikelihood (default -Double.MAX_VALUE).

-H num
Set the value of the shrinkage parameter (default 1).

Options after -- are passed to the designated learner.

Specified by:
setOptions in interface OptionHandler
Parameters:
options - the list of options as an array of strings
Throws:
java.lang.Exception - if an option is not supported

getOptions

public java.lang.String[] getOptions()
Gets the current settings of the Classifier.

Specified by:
getOptions in interface OptionHandler
Returns:
an array of strings suitable for passing to setOptions

getShrinkage

public double getShrinkage()
Get the value of Shrinkage.

Returns:
Value of Shrinkage.

setShrinkage

public void setShrinkage(double newShrinkage)
Set the value of Shrinkage.

Parameters:
newShrinkage - Value to assign to Shrinkage.

getLikelihoodThreshold

public double getLikelihoodThreshold()
Get the value of Precision.

Returns:
Value of Precision.

setLikelihoodThreshold

public void setLikelihoodThreshold(double newPrecision)
Set the value of Precision.

Parameters:
newPrecision - Value to assign to Precision.

getNumRuns

public int getNumRuns()
Get the value of NumRuns.

Returns:
Value of NumRuns.

setNumRuns

public void setNumRuns(int newNumRuns)
Set the value of NumRuns.

Parameters:
newNumRuns - Value to assign to NumRuns.

getNumFolds

public int getNumFolds()
Get the value of NumFolds.

Returns:
Value of NumFolds.

setNumFolds

public void setNumFolds(int newNumFolds)
Set the value of NumFolds.

Parameters:
newNumFolds - Value to assign to NumFolds.

setUseResampling

public void setUseResampling(boolean r)
Set resampling mode


getUseResampling

public boolean getUseResampling()
Get whether resampling is turned on

Returns:
true if resampling output is on

setSeed

public void setSeed(int seed)
Set seed for resampling.

Parameters:
seed - the seed for resampling

getSeed

public int getSeed()
Get seed for resampling.

Returns:
the seed for resampling

setClassifier

public void setClassifier(Classifier newClassifier)
Set the classifier for boosting. The learner should be able to handle numeric class attributes.

Parameters:
newClassifier - the Classifier to use.

getClassifier

public Classifier getClassifier()
Get the classifier used as the classifier

Returns:
the classifier used as the classifier

setMaxIterations

public void setMaxIterations(int maxIterations)
Set the maximum number of boost iterations

Parameters:
maxIterations - the maximum number of boost iterations

getMaxIterations

public int getMaxIterations()
Get the maximum number of boost iterations

Returns:
the maximum number of boost iterations

setWeightThreshold

public void setWeightThreshold(int threshold)
Set weight thresholding


getWeightThreshold

public int getWeightThreshold()
Get the degree of weight thresholding

Returns:
the percentage of weight mass used for training

setDebug

public void setDebug(boolean debug)
Set debugging mode

Parameters:
debug - true if debug output should be printed

getDebug

public boolean getDebug()
Get whether debugging is turned on

Returns:
true if debugging output is on

buildClassifier

public void buildClassifier(Instances data)
                     throws java.lang.Exception
Builds the boosted classifier

Specified by:
buildClassifier in class Classifier
Parameters:
data - set of instances serving as training data
Throws:
java.lang.Exception - if the classifier has not been generated successfully

classifiers

public Classifier[][] classifiers()
Returns the array of classifiers that have been built.


distributionForInstance

public double[] distributionForInstance(Instance instance)
                                 throws java.lang.Exception
Calculates the class membership probabilities for the given test instance.

Specified by:
distributionForInstance in class DistributionClassifier
Parameters:
instance - the instance to be classified
Returns:
predicted class probability distribution
Throws:
java.lang.Exception - if instance could not be classified successfully

toSource

public java.lang.String toSource(java.lang.String className)
                          throws java.lang.Exception
Returns the boosted model as Java source code.

Specified by:
toSource in interface Sourcable
Parameters:
className - the name that should be given to the source class.
Returns:
the tree as Java source code
Throws:
java.lang.Exception - if something goes wrong

toString

public java.lang.String toString()
Returns description of the boosted classifier.

Returns:
description of the boosted classifier as a string

main

public static void main(java.lang.String[] argv)
Main method for testing this class.

Parameters:
argv - the options