| Title: | Yet Another Random Forests Package |
|---|---|
| Description: | A customizable, asynchronous, and parallelized random forest implementation. Supports user-defined splitting and aggregation behavior, missingness incorporated in attributes, and random-forest-based imputation. |
| Authors: | Adam Kapelner [aut, cre, cph] (ORCID: <https://orcid.org/0000-0001-5985-6792>), Matt Olson [aut], Abhinav Patil [aut] |
| Maintainer: | Adam Kapelner <[email protected]> |
| License: | GPL-3 |
| Version: | 1.2.0 |
| Built: | 2026-10-16 10:14:51 UTC |
| Source: | https://github.com/kapelner/YARF |
A 201-row, 25-variable automobile dataset distributed with YARF for examples and demonstrations.
A data frame with 201 rows and 25 variables.
Distributed with the YARF project.
Nine benchmark data frames distributed with YARF: 'ankara', 'baseball', 'boston', 'compactiv', 'ozone', 'pole', 'triazine', 'wine.red', and 'wine.white'. They are retained for reproducibility of package benchmarks.
Data frames whose final column is the benchmark response. Dataset dimensions range from 186 to 8,192 rows and from 9 to 59 variables.
Distributed with the YARF project.
Calculates the area under the curve
calcAUC(roc_results, plot = TRUE)calcAUC(roc_results, plot = TRUE)
roc_results |
The results from the function |
plot |
Make a plot of the ROC with the AUC shaded in? Default is |
The area under the curve approximated via numerical integration.
Computes information about the "proximity" of observations within the YARF model. Given two datasets, information is computed for all pairs of observations. Information returned is the prediction nodes (for all trees) for both objects and their common node in the tree structure plus much information about the nodes.
compute_raw_proximity_info( yarf_mod, X1 = NULL, X2 = NULL, oob_only_1 = TRUE, oob_only_2 = TRUE, verbose = TRUE )compute_raw_proximity_info( yarf_mod, X1 = NULL, X2 = NULL, oob_only_1 = TRUE, oob_only_2 = TRUE, verbose = TRUE )
yarf_mod |
A YARF model object. |
X1 |
A n* x p matrix where each row is an observation. If |
X2 |
A n x p matrix where each row is an observation. If |
oob_only_1 |
|
oob_only_2 |
|
verbose |
Print out periodic messages on progress of the computation of the proximity information.
The default is |
A list indexed by the row number of X1 whose elements are a list indexed by
the row number of X2 whose elements are a list indexed by tree number whose
elements are a list consisting of the following information: (a) X1 prediction node
(b) X2 prediction node, (c) shared node, (d) locations of all three nodes indicating
left/right directions, (e) depth differences to shared node where 0 indicates they
are in the same node, (f) parent node's cost for all three (where the parent cost
of the root is the null model cost)
Adam Kapelner
num_permutation_samples times.
The p-val is determined by a permutation-like test.Tests the effect of H0: one, or two, ... or all covariates are not predictive (out of sample) via a permutation test.
We permute the covariates column(s) and build YARF models num_permutation_samples times.
The p-val is determined by a permutation-like test.
cov_importance_test( yarf_mod, covariates = NULL, num_permutation_samples = 100, plot = TRUE )cov_importance_test( yarf_mod, covariates = NULL, num_permutation_samples = 100, plot = TRUE )
yarf_mod |
The fit model to test against permuted training data. |
covariates |
The covariates to permute as a vector of indices or names. If multiple covariates
they are permuted as a single block to not break collinearity. If this parameter is
specified as |
num_permutation_samples |
How many different YARF models should be built - one for each different
permutation of the variables tests. The is the resolution of the test. The
default is |
plot |
Plot a histogram of the permutation samples with the original sample displayed?
Default is |
A list with the following components: permutation_samples_of_error whose value
is a vector with entries being the error values for each permutation sample,
observed_error_estimate is the error value of the original model built with
unpermuted data and pval is the estimated significance level of the permutation test.
Adam Kapelner
This function investigates all two-way interactions in the forest by measuring how many times splits of one variable is nested in splits of another variable.
first_order_interaction_investigator( yarf_mod, plot = TRUE, only_non_self_interactions = FALSE, num_var_plot = 50 )first_order_interaction_investigator( yarf_mod, plot = TRUE, only_non_self_interactions = FALSE, num_var_plot = 50 )
yarf_mod |
The YARF model in which to query |
plot |
This controls whether or not the function plots the interaction strengths. If so, it plots
in order of strength from highest to lowest. Default is |
only_non_self_interactions |
Should we only show interactions between different variables? Default is |
num_var_plot |
How many first order interactions to plot? All would be p-choose-2 which
can be large. The default is |
A list with two components: (1) interaction_counts is a p x p sized matrix
of interaction counts between the variables and (2) sorted_counts is a vector of
the top interactions in order. Returns invisibly.
Adam Kapelner
Gets information about each tree: the number of nodes, number of leaves and maximum depth.
get_tree_num_nodes_leaves_max_depths(yarf_mod)get_tree_num_nodes_leaves_max_depths(yarf_mod)
yarf_mod |
The YARF model in which to query |
A list with three integer vector components all of which have length equal to the
number of trees completes: (1) num_nodes has entries equal to the number of nodes
in each tree, (2) num_leaves has entries equal to the number of leaves (terminal
nodes) in each tree and (3) max_depths has entries equal to the maximum depth of
each tree.
Adam Kapelner
Illustrates trees to PNG files in the local directory.
illustrate_trees( yarf_mod, trees = NULL, max_depth = -1, background_rgb_color = c(255, 255, 255), line_rgb_color = c(100, 100, 100), text_rgb_color = c(0, 0, 0), font_family = "Arial", font_size = 12, margin_in_px = 70, character_width_in_px = 4.2, length_in_px_per_half_split = 20, depth_in_px_per_split = 100, print_at_split_node_script = NULL, print_at_leaf_script = NULL, file_format = "png", use_real_names = TRUE, title = "yarf_mod_tree", open_file = FALSE )illustrate_trees( yarf_mod, trees = NULL, max_depth = -1, background_rgb_color = c(255, 255, 255), line_rgb_color = c(100, 100, 100), text_rgb_color = c(0, 0, 0), font_family = "Arial", font_size = 12, margin_in_px = 70, character_width_in_px = 4.2, length_in_px_per_half_split = 20, depth_in_px_per_split = 100, print_at_split_node_script = NULL, print_at_leaf_script = NULL, file_format = "png", use_real_names = TRUE, title = "yarf_mod_tree", open_file = FALSE )
yarf_mod |
The YARF model whose trees are to be illustrated |
trees |
A vector of the tree indices (a subset of 1...num_trees) |
max_depth |
Illustrate the tree only up to this depth. Default is |
background_rgb_color |
A vector with 3 elements in |
line_rgb_color |
A vector with 3 elements in |
text_rgb_color |
A vector with 3 elements in |
font_family |
The font family of the text in the illustration. This string is passed in
as the first argument in a new |
font_size |
The font size in the illustrations in pixels. Default is |
margin_in_px |
The margin in pixels of the entire illustration (equal on all sides). Default
is |
character_width_in_px |
Estimate as to the size of the character width in pixels. Differs based on fonts. |
length_in_px_per_half_split |
The length of half of a split in pixels. Default is |
depth_in_px_per_split |
The length of the depth of a split in pixels. Default is |
print_at_split_node_script |
A custom javascript function which creates a special custom message to be printed
at split nodes (see below). The default is function printAtSplitNode(node){ //node is of type YARF.YARFNode ... return message; //a string } |
print_at_leaf_script |
A custom javascript function which creates a special custom message to be printed
at leaf nodes (see below). The default is function printAtLeaf(node){ //node is of type YARF.YARFNode ... return message; //a string } |
file_format |
The image format of the resulting illustration. Legal values are "jpg", "gif", "png". Default is "png". |
use_real_names |
In the illustrations of split rules, set this to |
title |
The name of the file. Note that "_00t.png" will be added where "t" is the tree index and "00" is an appropriate number of leading zeroes. Default is "yarf_mod_tree". |
open_file |
Should the first file be opened by the operating system? Default is |
Adam Kapelner
Convenience method for testing a model's fit. The strategy is simple: one builds a YARF model from the residuals of the model you wish to fit and performs an omnibus test of all covariates against those residuals. This effecticely answers the question: "is there out-of-sample predictive information left over after this model was fit?" The p-val is determined by a permutation-like test.
model_fit_test( X, y, model_fit = NULL, num_permutation_samples = 100, plot = TRUE, ... )model_fit_test( X, y, model_fit = NULL, num_permutation_samples = 100, plot = TRUE, ... )
X |
The training data frame |
y |
The vector of training responses |
model_fit |
An already-fit model that you wish to test if there
is any predictive power left over. This model's class must respond
to the "predict" S3 method. The default is |
num_permutation_samples |
The resolution of the test. See cov_importance_test. The
default is |
plot |
Plot the result as a histogram. Default is |
... |
Additional parameters to be passed to YARF. |
The same as cov_importance_test.
Adam Kapelner
Predictions are an aggregation from each of the
individual trees' predictions based on the function passed into the model creation step. The default
aggregation is the sample average for regression and the modal class for classification. You can set
a custom aggregation method via the YARF_set_aggregation_method function.
## S3 method for class 'YARF' predict(object, new_data, ...)## S3 method for class 'YARF' predict(object, new_data, ...)
object |
A YARF model object. |
new_data |
A n* x p matrix where each row is an observation you wish to predict for. Here, p is the same length as the original training data. |
... |
Other methods passed to predict |
The yhat prediction for the new_data record(s).
Kapelner
Returns the leaf nodes that are responsible for predictions for a given dataset.
prediction_nodes(yarf_mod, X = NULL, oob_only = TRUE)prediction_nodes(yarf_mod, X = NULL, oob_only = TRUE)
yarf_mod |
A YARF model object. |
X |
A n* x p data frame where each row is an observation you wish to obtain prediction information. Here,
p is the same length as the original training data and the column names match the original training data.
If a matrix object is passed in, it is automatically cast to a data frame. This may not always give you what
you want. If |
oob_only |
If |
A list where values are rJava YARFNode objects for all the prediction nodes. The first key
indexes the row number in X and the second, the tree number.
Kapelner
Alias for summary.
## S3 method for class 'YARF' print(x, ...)## S3 method for class 'YARF' print(x, ...)
x |
A YARF model object. |
... |
Other methods passed to print |
Adam Kapelner
Computes information about the "proximity" of observations within the YARF model. Given two datasets, information is computed for all pairs of observation.
proximity_info(yarf_mod, X, prox_single_node_calc_script = NULL)proximity_info(yarf_mod, X, prox_single_node_calc_script = NULL)
yarf_mod |
A YARF model object. |
X |
A n* x p data frame where each row is an observation. If |
prox_single_node_calc_script |
Script ... |
(For now, a list of raw information for rows in each dataset)
Matt Olson
Prunes the trees in a YARF mode. This is an irreversible modification to the tree structure.
prune_YARF_model(yarf_mod, prune_if_script, trees_to_prune = NULL)prune_YARF_model(yarf_mod, prune_if_script, trees_to_prune = NULL)
yarf_mod |
The YARF model whose trees are to be pruned. |
prune_if_script |
A custom javascript function (see below) which determines the pruning of a node (i.e. deletes the node's children and sets the node to a leaf and then sets a y_hat using the assign function). function pruneIf(node){ //node is of type YARF.YARFNode ... return boolean //true if this node is to be pruned } |
trees_to_prune |
An array of indices which specify which of the trees to prune. The default is |
Adam Kapelner
Function which queries the YARF model about how many times the variables are used when splitting
query_variable_counts(yarf_mod, type = "splits")query_variable_counts(yarf_mod, type = "splits")
yarf_mod |
The model in which to query |
type |
Either " |
A vector with length p, the number of variables considered in this YARF model, with either a binary
value for type = "trees" or a count value for type = "splits".
Adam Kapelner
query_variable_counts.Function which queries the YARF model about the proportion of times the variables are used when splitting.
This is a convenience method which merely normalizes the result of query_variable_counts.
query_variable_proportions_across_trees(yarf_mod, type = "splits")query_variable_proportions_across_trees(yarf_mod, type = "splits")
yarf_mod |
The model in which to query |
type |
Either " |
A vector with length p, the number of variables considered in this YARF model, with values between
0 and 1.
Adam Kapelner
Sets the number of cores for YARF to use for all operations which are multithreaded (e.g. model construction, prediction, etc)
set_YARF_num_cores(num_cores)set_YARF_num_cores(num_cores)
num_cores |
The number of cores to use |
Adam Kapelner
Prints a summary of the model to the console
## S3 method for class 'YARF' summary(object, ...)## S3 method for class 'YARF' summary(object, ...)
object |
A YARF model object. |
... |
Other methods passed to summary |
Adam Kapelner
Computes information about the "proximity" of observations within the YARF model. Given two datasets, information is computed for all pairs of observations. Information is averaged over the trees in the model.
tree_average_proximity_info(raw)tree_average_proximity_info(raw)
raw |
The information returned from |
A list indexed by the row number of X1 whose elements are a list indexed by
the row number of X2 whose elements are an array consisting of the following
information: (a) proportion of trees these two observations were in the same node,
(b) average node 1/2/shared depth (c) average node 1/2/shared cost (d) average node
1/2/shared parent cost (e) average node 1/2/shared size (f) average depth difference
between node 1/2 and shared
Kapelner
Builds a YARF Model. There are many customizations available.
YARF( X = NULL, y = NULL, Xy = NULL, Xother = NULL, allow_missingness_in_y = FALSE, num_trees = NULL, bootstrap_indices = NULL, n_max_per_tree = NULL, other_indices = NULL, mtry = NULL, nodesize = NULL, mtry_script = NULL, split_vals_script = NULL, make_node_to_leaf_script = NULL, cost_single_node_calc_script = NULL, cost_both_children_calc_script = NULL, node_assign_script = NULL, after_node_birth_function_script = NULL, shared_scripts = NULL, use_missing_data = TRUE, replace_missing_data_with_x_j_bar = FALSE, no_missing_data_split_rule = "RANDOM", serialize = FALSE, seed = NULL, wait = TRUE, calculate_oob_error = TRUE, fit_until_convergence = FALSE, oob_cost_calculation_script = NULL, tolerance = 0.01, verbose = TRUE, debug_log = FALSE )YARF( X = NULL, y = NULL, Xy = NULL, Xother = NULL, allow_missingness_in_y = FALSE, num_trees = NULL, bootstrap_indices = NULL, n_max_per_tree = NULL, other_indices = NULL, mtry = NULL, nodesize = NULL, mtry_script = NULL, split_vals_script = NULL, make_node_to_leaf_script = NULL, cost_single_node_calc_script = NULL, cost_both_children_calc_script = NULL, node_assign_script = NULL, after_node_birth_function_script = NULL, shared_scripts = NULL, use_missing_data = TRUE, replace_missing_data_with_x_j_bar = FALSE, no_missing_data_split_rule = "RANDOM", serialize = FALSE, seed = NULL, wait = TRUE, calculate_oob_error = TRUE, fit_until_convergence = FALSE, oob_cost_calculation_script = NULL, tolerance = 0.01, verbose = TRUE, debug_log = FALSE )
X |
The data frame of training data |
y |
The vector of training responses which is either numeric (for regression or factor (for classification). |
Xy |
The data frame of training data where the last column is responses |
Xother |
Other data that is used in the training but the RF doesn't split on it |
allow_missingness_in_y |
If |
num_trees |
The number of trees in the RF. Default is If the model is fit asynchronously via the |
bootstrap_indices |
A list with keys 1, 2, ..., num_trees where each value is the indices of the training data
you wish to use for each tree. The default is |
n_max_per_tree |
An upper limit on the number of observations used to build each tree. If this parameter is specified,
there is no need to specify |
other_indices |
An optional list with keys 1,2,..., num_trees where each value is indices of the training data
you wish to use in some custom way for each tree using a custom function. If not custom function
is specified which makes use of this, it will be ignored. However, indices specified here will not be part of
the out-of-bag collection of indices. If this is not your wish, the elements in the vectors
specified in this list's values should be a subset of those in the values of |
mtry |
The number of variables tried at every split. The default is |
nodesize |
The minimum number of observations in a node. YARF will stop splitting at this point.
If |
mtry_script |
A custom javascript function which selects the variables to be greedily searched (see below)
The default is function tryVars(node){ //node is of type YARF.YARFNode ... return int_array //a subset of 0,...,p-1, indices indicating the variables to perform the exhaustive search on } |
split_vals_script |
A custom javascript function which selects the split values to be greedily searched in feature j.
The default is function tryVals(node, j){ //node is of type YARF.YARFNode and j is the feature number in 0,...,p-1 ... return double_array //a vector of split vals to greedily assess } |
make_node_to_leaf_script |
A custom javascript function to be used to calculate nodesize (see below).
The default is function makeNodeIntoLeaf(node){ //node is of type YARF.YARFNode ... return boolean //where true makes this node into a leaf } |
cost_single_node_calc_script |
A custom cost calculation for a potential node (when considering a split) in Javascript (see below).
The default is function nodeCost(node){ //node is of type YARF.YARFNode ... return double //where a higher number indicates a higher cost } |
cost_both_children_calc_script |
A custom cost calculation in Javascript for an entire split considering both the putative left and right children
nodes (see below). The default is function totalChildrenCost(leftNode, rightNode){ //both nodes are of type YARF.YARFNode ... return double //where a higher number indicates a higher cost } |
node_assign_script |
A custom node assignment function in Javascript (see below). This function is run after RF greedily finds the
"lowest cost" split. The default is function assignYhatToNode(node){ //node is of type YARF.YARFNode ... return double //assigned as this node's predicted value ("y_hat") } |
after_node_birth_function_script |
A custom function in Javascript which is executed after a node is given birth to. The default is
function nodeAfterNodeBirth(node){ //node is of type YARF.YARFNode ... } |
shared_scripts |
Custom Javascript code that are always in scope when running all your custom methods.
The default is |
use_missing_data |
Use the "missing-incorporated-in-attributes" strategy to fit data with missingness. The
default is |
replace_missing_data_with_x_j_bar |
Replace missing predictor values with their training-column means before fitting. |
no_missing_data_split_rule |
What rule should be used for missing data seen in testing for nodes that did not
see missing data in training? Choices: |
serialize |
Should the YARF model be saved? The default is |
seed |
Set a random seed for reproducibility. |
wait |
Should we hang R to wait for the YARF model to complete? The default is |
calculate_oob_error |
Should we also calculate the OOB error? Default is |
fit_until_convergence |
Default is After each tree is completed, the OOB cost is computed. If the cost
decreases the algorithm does not engage. However, upon the first
increase in cost, the model checks for convergence by calculating the average change in cost
and the standard deviation of change in cost. If a 95% confidence interval of the average cost is
within the window created by plus or minus It is strongly recommended to set the If the user wishes to view the convergence in real time, we recommend the |
oob_cost_calculation_script |
This parameter will determine the out-of-bag cost of the RF forest model. Default is If non-null, an optional custom Javascript function which calculates the cost of a prediction given the true
value of the prediction (see below). If is likely similar to function oobCost(y_hat, y){ //y_hat is the predicted value and y is the true value (both are of type double) ... return double //where a larger number indicates a higher cost to the error between y and y_hat. } If this script is specified (i.e. the argument is non- |
tolerance |
This parameter is ignored unless |
verbose |
Should we print out messages verbosely during construction? Default is |
debug_log |
Should we print out messages from Java? Default is |
A list that (a) reiterates all ob the above arguments passed in and (b) pred_type is the
guess as to modeling type: regression or classification (c) java_YARF is the Java object
(d) y_levels is the unique values of the response variable (e) model_matrix_training_data
is the data matrix with factor variables converted to dummies that is fed into YARF as the X variables
(f) training_data_features is the names of the features (g) predictors_which_are_factors
is a list of the variables that are factors and (h) n and p which are the dimensions of
the training data model matrix and (i) various other convenient data and internal data of use to
other functions in this package.
Adam Kapelner
Computes the out-of-bag (OOB) predictions for all observations in training data by tree, returning a matrix. This is useful for debugging or other analyses
YARF_all_oob_results_matrix(yarf_mod)YARF_all_oob_results_matrix(yarf_mod)
yarf_mod |
The yarf model object such that |
Adam Kapelner
Prints out a messages reflecting the progress of the YARF model construction until completion.
YARF_convergence(yarf_mod, time_delay_in_seconds = 5, trail_pts = 50)YARF_convergence(yarf_mod, time_delay_in_seconds = 5, trail_pts = 50)
yarf_mod |
The yarf model object |
time_delay_in_seconds |
Frequency of messages in seconds. Default is |
trail_pts |
If non-null, it will plot a secondary window with only the trees in the last |
If time_delay_in_seconds is not NULL, running this function once will slow down your model construction since it will forever
be calculating OOB statistics after each tree (until completion or until the user stops the function).
Adam Kapelner
TRUE, all verbose logging is turned
off and serializing is turned off as well.Duplicates the settings of YARF model. Wait is TRUE, all verbose logging is turned
off and serializing is turned off as well.
yarf_duplicate(yarf_mod, X = NULL, y = NULL)yarf_duplicate(yarf_mod, X = NULL, y = NULL)
yarf_mod |
The YARF model with settings you wish to duplicate |
X |
The training data as a data frame. Default is |
y |
The training responses as numeric / integer / factor vector. Default is |
A newly fitted YARF model with the original settings and the supplied training data.
Adam Kapelner
Computes a response prediction for new data observation(s) for all trees (unaggregated).
YARF_predict_all_trees(yarf_mod, new_data)YARF_predict_all_trees(yarf_mod, new_data)
yarf_mod |
A YARF model object. |
new_data |
A n* x p data frame where each row is an observation you wish to predict for. Here,
p is the same length as the original training data and |
A matrix of yhat predictions for the new_data record(s) for all trees
(size n* x num_trees).
Adam Kapelner
Prints out a message reflecting the progress of the YARF model construction
YARF_progress(yarf_mod, console_message = TRUE)YARF_progress(yarf_mod, console_message = TRUE)
yarf_mod |
The yarf model object |
console_message |
Should we print a message to console? Default is |
The yarf object with new progress data
Adam Kapelner
save and save.image
to write it to a file that can be then loaded into another and/or future R session.Serializes the model so the user can use save and save.image
to write it to a file that can be then loaded into another and/or future R session.
YARF_serialize(yarf_mod)YARF_serialize(yarf_mod)
yarf_mod |
The yarf model object |
Adam Kapelner
This function sets custom code to be run when the Forest is making a decision from its many trees. This function should be run
before you run predict or YARF_update_with_oob_results so that your custom aggregation can be employed. This function
is optional as the default aggregation method corresponds to (a) the sample average for regression and (b) the modal category
for classification. One further note of warning: once this script has been set, it will be retained in the YARF model until
this function is run again for this YARF model object with a new aggregation script. Set aggregation_script = NULL if you
wish to reset.
YARF_set_aggregation_method(yarf_mod, aggregation_script)YARF_set_aggregation_method(yarf_mod, aggregation_script)
yarf_mod |
The yarf model object |
aggregation_script |
A custom javascript function which aggregates the predictions in the trees for one observations into one scalar prediction (see below). function aggregateYhatsIntoOneYhat(y_hats, yarf){ //y_hats is an array of doubles of size num_trees //and yarf provides access to the entire random forest object if needed (of type YARF.YARF) ... return double //this is the final predicted value aggregated from all tree predictions } |
The yarf model object (invisibly)
Adam Kapelner
Halts the model building.
YARF_stop(yarf_mod)YARF_stop(yarf_mod)
yarf_mod |
The yarf model object |
Adam Kapelner
Computes the out-of-bag (OOB) predictions for the training data. This gives a good sense of out-of-sample performance in the future.
YARF_update_with_oob_results( yarf_mod, oob_cost_calculation_script = NULL, indices = NULL )YARF_update_with_oob_results( yarf_mod, oob_cost_calculation_script = NULL, indices = NULL )
yarf_mod |
The yarf model object |
oob_cost_calculation_script |
An optional custom Javascript function which calculates the cost of a prediction given the true
value of the prediction (see below). If is likely similar to function oobCost(y_hat, y){ //y_hat is the predicted value and y is the true value (both are of type double) ... return double //where a larger number indicates a higher cost to the error between y and y_hat. } |
indices |
The indices to compute OOB performance. This is generally a private argument for use by the validation / test functions. |
Adam Kapelner
Computes the out-of-bag (OOB) predictions for the test subset of the training data. This function should only be run once. If it run again you risk adapting your model to the test data which will lead to overfitting and hence a biased out-of-sample performance estimate.
YARF_update_with_oob_test_results(yarf_mod)YARF_update_with_oob_test_results(yarf_mod)
yarf_mod |
The yarf model object such that |
Adam Kapelner
Computes the out-of-bag (OOB) predictions for the validation subset of the training data. This allows the user to try many different models while still leaving test data for true out-of-sample performance estimation.
YARF_update_with_oob_validation_results(yarf_mod, validation_proportion = 0.8)YARF_update_with_oob_validation_results(yarf_mod, validation_proportion = 0.8)
yarf_mod |
The yarf model object |
validation_proportion |
The percentage of the data to use for validation data (and the remaining is test data). The default is 80%. |
Adam Kapelner
You can conceptualize this as a random forest without the feature that de-correlates the trees, namely trying random subsets of variables at each split. Here, all splits are considered. You can also think about this as model averaging many CART trees built from bootstrap samples (or specific samples you choose).
YARFBAG( X = NULL, y = NULL, Xy = NULL, Xother = NULL, allow_missingness_in_y = FALSE, num_trees = NULL, bootstrap_indices = NULL, n_max_per_tree = NULL, other_indices = NULL, nodesize = NULL, mtry_script = NULL, split_vals_script = NULL, make_node_to_leaf_script = NULL, cost_single_node_calc_script = NULL, cost_both_children_calc_script = NULL, node_assign_script = NULL, after_node_birth_function_script = NULL, shared_scripts = NULL, use_missing_data = TRUE, replace_missing_data_with_x_j_bar = FALSE, serialize = FALSE, seed = NULL, wait = TRUE, calculate_oob_error = TRUE, fit_until_convergence = FALSE, oob_cost_calculation_script = NULL, tolerance = 0.01, verbose = TRUE, debug_log = FALSE )YARFBAG( X = NULL, y = NULL, Xy = NULL, Xother = NULL, allow_missingness_in_y = FALSE, num_trees = NULL, bootstrap_indices = NULL, n_max_per_tree = NULL, other_indices = NULL, nodesize = NULL, mtry_script = NULL, split_vals_script = NULL, make_node_to_leaf_script = NULL, cost_single_node_calc_script = NULL, cost_both_children_calc_script = NULL, node_assign_script = NULL, after_node_birth_function_script = NULL, shared_scripts = NULL, use_missing_data = TRUE, replace_missing_data_with_x_j_bar = FALSE, serialize = FALSE, seed = NULL, wait = TRUE, calculate_oob_error = TRUE, fit_until_convergence = FALSE, oob_cost_calculation_script = NULL, tolerance = 0.01, verbose = TRUE, debug_log = FALSE )
X |
The data frame of training data |
y |
The vector of training responses |
Xy |
The data frame of training data where the last column is responses |
Xother |
Other data that is used in the training but the RF doesn't split on it |
allow_missingness_in_y |
If |
num_trees |
The number of trees in the RF. Default is If the model is fit asynchronously via the |
bootstrap_indices |
A list with keys 1, 2, ..., num_trees where each value is the indices of the training data
you wish to use for each tree. The default is |
n_max_per_tree |
An upper limit on the number of observations used to build each tree. If this parameter is specified,
there is no need to specify |
other_indices |
An optional list with keys 1,2,..., num_trees where each value is indices of the training data
you wish to use in some custom way for each tree using a custom function. If not custom function
is specified which makes use of this, it will be ignored. However, indices specified here will not be part of
the out-of-bag collection of indices. If this is not your wish, the elements in the vectors
specified in this list's values should be a subset of those in the values of |
nodesize |
The minimum number of observations in a node. YARF will stop splitting at this point.
If |
mtry_script |
A custom JavaScript function that selects candidate variables at each split. |
split_vals_script |
A custom javascript function which selects the split values to be greedily searched in feature j.
The default is function tryVals(node, j){ //node is of type YARF.YARFNode and j is the feature number in 0,...,p-1 ... return double_array //a vector of split vals to greedily assess } |
make_node_to_leaf_script |
A custom javascript function to be used to calculate nodesize (see below).
The default is function makeNodeIntoLeaf(node){ //node is of type YARF.YARFNode ... return boolean //where true makes this node into a leaf } |
cost_single_node_calc_script |
A custom cost calculation for a potential node (when considering a split) in Javascript (see below).
The default is function nodeCost(node){ //node is of type YARF.YARFNode ... return double //where a higher number indicates a higher cost } |
cost_both_children_calc_script |
A custom cost calculation in Javascript for an entire split considering both the putative left and right children
nodes (see below). The default is function totalChildrenCost(leftNode, rightNode){ //both nodes are of type YARF.YARFNode ... return double //where a higher number indicates a higher cost } |
node_assign_script |
A custom node assignment function in Javascript (see below). This function is run after RF greedily finds the
"lowest cost" split. The default is function assignYhatToNode(node){ //node is of type YARF.YARFNode ... return double //assigned as this node's predicted value ("y_hat") } |
after_node_birth_function_script |
A custom function in Javascript which is executed after a node is given birth to. The default is
function nodeAfterNodeBirth(node){ //node is of type YARF.YARFNode ... } |
shared_scripts |
Custom Javascript code that are always in scope when running all your custom methods.
The default is |
use_missing_data |
Use the "missing-incorporated-in-attributes" strategy to fit data with missingness. The
default is |
replace_missing_data_with_x_j_bar |
Replace missing predictor values with their training-column means before fitting. |
serialize |
Should the YARF model be saved? The default is |
seed |
Set a random seed for reproducibility. |
wait |
Should we hang R to wait for the YARF model to complete? The default is |
calculate_oob_error |
Should we also calculate the OOB error? Default is |
fit_until_convergence |
Fit trees until the out-of-bag convergence criterion is met. |
oob_cost_calculation_script |
This parameter will determine the out-of-bag cost of the RF forest model. Default is If non-null, an optional custom Javascript function which calculates the cost of a prediction given the true
value of the prediction (see below). If is likely similar to function oobCost(y_hat, y){ //y_hat is the predicted value and y is the true value (both are of type double) ... return double //where a larger number indicates a higher cost to the error between y and y_hat. } If this script is specified (i.e. the argument is non- |
tolerance |
This parameter is ignored unless |
verbose |
Should we print out messages verbosely during construction? Default is |
debug_log |
Should we print out messages from Java? Default is |
A list that (a) reiterates all ob the above arguments passed in and (b) pred_type is the
guess as to modeling type: regression or classification (c) java_YARF is the Java object
(d) y_levels is the unique values of the response variable (e) model_matrix_training_data
is the data matrix with factor variables converted to dummies that is fed into YARF as the X variables
(f) training_data_features is the names of the features (g) predictors_which_are_factors
is a list of the variables that are factors and (h) n and p which are the dimensions of
the training data model matrix and (i) various other convenient data and internal data of use to
other functions in this package.
Adam Kapelner
A convenience method to build a CART model via YARF. There are many customizations available.
YARFCART( X = NULL, y = NULL, Xy = NULL, Xother = NULL, allow_missingness_in_y = FALSE, bootstrap_indices = NULL, other_indices = NULL, mtry = NULL, nodesize = NULL, mtry_script = NULL, split_vals_script = NULL, make_node_to_leaf_script = NULL, cost_single_node_calc_script = NULL, cost_both_children_calc_script = NULL, node_assign_script = NULL, after_node_birth_function_script = NULL, shared_scripts = NULL, use_missing_data = TRUE, replace_missing_data_with_x_j_bar = FALSE, serialize = FALSE, seed = NULL, calculate_oob_error = TRUE, oob_cost_calculation_script = NULL, verbose = TRUE, debug_log = FALSE )YARFCART( X = NULL, y = NULL, Xy = NULL, Xother = NULL, allow_missingness_in_y = FALSE, bootstrap_indices = NULL, other_indices = NULL, mtry = NULL, nodesize = NULL, mtry_script = NULL, split_vals_script = NULL, make_node_to_leaf_script = NULL, cost_single_node_calc_script = NULL, cost_both_children_calc_script = NULL, node_assign_script = NULL, after_node_birth_function_script = NULL, shared_scripts = NULL, use_missing_data = TRUE, replace_missing_data_with_x_j_bar = FALSE, serialize = FALSE, seed = NULL, calculate_oob_error = TRUE, oob_cost_calculation_script = NULL, verbose = TRUE, debug_log = FALSE )
X |
The data frame of training data |
y |
The vector of training responses |
Xy |
The data frame of training data where the last column is responses |
Xother |
Other data that is used in the training but the RF doesn't split on it |
allow_missingness_in_y |
If |
bootstrap_indices |
A list with keys 1, 2, ..., num_trees where each value is the indices of the training data
you wish to use for each tree. The default is |
other_indices |
An optional list with keys 1,2,..., num_trees where each value is indices of the training data
you wish to use in some custom way for each tree using a custom function. If not custom function
is specified which makes use of this, it will be ignored. However, indices specified here will not be part of
the out-of-bag collection of indices. If this is not your wish, the elements in the vectors
specified in this list's values should be a subset of those in the values of |
mtry |
The number of variables tried at every split. The default is |
nodesize |
The minimum number of observations in a node. YARF will stop splitting at this point.
If |
mtry_script |
A custom javascript function which selects the variables to be greedily searched (see below)
The default is function tryVars(node){ //node is of type YARF.YARFNode ... return int_array //a subset of 0,...,p-1, indices indicating the variables to perform the exhaustive search on } |
split_vals_script |
A custom javascript function which selects the split values to be greedily searched in feature j.
The default is function tryVals(node, j){ //node is of type YARF.YARFNode and j is the feature number in 0,...,p-1 ... return double_array //a vector of split vals to greedily assess } |
make_node_to_leaf_script |
A custom javascript function to be used to calculate nodesize (see below).
The default is function makeNodeIntoLeaf(node){ //node is of type YARF.YARFNode ... return boolean //where true makes this node into a leaf } |
cost_single_node_calc_script |
A custom cost calculation for a potential node (when considering a split) in Javascript (see below).
The default is function nodeCost(node){ //node is of type YARF.YARFNode ... return double //where a higher number indicates a higher cost } |
cost_both_children_calc_script |
A custom cost calculation in Javascript for an entire split considering both the putative left and right children
nodes (see below). The default is function totalChildrenCost(leftNode, rightNode){ //both nodes are of type YARF.YARFNode ... return double //where a higher number indicates a higher cost } |
node_assign_script |
A custom node assignment function in Javascript (see below). This function is run after RF greedily finds the
"lowest cost" split. The default is function assignYhatToNode(node){ //node is of type YARF.YARFNode ... return double //assigned as this node's predicted value ("y_hat") } |
after_node_birth_function_script |
A custom function in Javascript which is executed after a node is given birth to. The default is
function nodeAfterNodeBirth(node){ //node is of type YARF.YARFNode ... } |
shared_scripts |
Custom Javascript code that are always in scope when running all your custom methods.
The default is |
use_missing_data |
Use the "missing-incorporated-in-attributes" strategy to fit data with missingness. The
default is |
replace_missing_data_with_x_j_bar |
Replace missing predictor values with their training-column means before fitting. |
serialize |
Should the YARF model be saved? The default is |
seed |
Set a random seed for reproducibility. |
calculate_oob_error |
Should we also calculate the OOB error? Default is |
oob_cost_calculation_script |
This parameter will determine the out-of-bag cost of the RF forest model. Default is If non-null, an optional custom Javascript function which calculates the cost of a prediction given the true
value of the prediction (see below). If is likely similar to function oobCost(y_hat, y){ //y_hat is the predicted value and y is the true value (both are of type double) ... return double //where a larger number indicates a higher cost to the error between y and y_hat. } If this script is specified (i.e. the argument is non- |
verbose |
Should we print out messages verbosely during construction? Default is |
debug_log |
Should we print out messages from Java? Default is |
A list that (a) reiterates all ob the above arguments passed in and (b) pred_type is the
guess as to modeling type: regression or classification (c) java_YARF is the Java object
(d) y_levels is the unique values of the response variable (e) model_matrix_training_data
is the data matrix with factor variables converted to dummies that is fed into YARF as the X variables
(f) training_data_features is the names of the features (g) predictors_which_are_factors
is a list of the variables that are factors and (h) n and p which are the dimensions of
the training data model matrix and (i) various other convenient data and internal data of use to
other functions in this package.
Adam Kapelner
This function imputes missing data using the missForest algorithm which essentially iteratively runs the RF algorithm on each column with missingness until the predictions do not change significantly. We provide a means to run this imputation on both training data (with responses) and test data (without responses). The user can also enter algorithm hyperparameters like convergence settings.
YARFMissForest( Xtrain, ytrain, Xtest = NULL, maxiter = 10, converge_after_increasing = TRUE, verbose_missForest = TRUE, ... )YARFMissForest( Xtrain, ytrain, Xtest = NULL, maxiter = 10, converge_after_increasing = TRUE, verbose_missForest = TRUE, ... )
Xtrain |
The training data as a data frame. |
ytrain |
The training responses as numeric / integer / factor vector. |
Xtest |
The test data as a data frame (optional, default |
maxiter |
The maximum number of iterations before stopping (default is 10, as in package |
converge_after_increasing |
The algorithm quits after imputation errors increase once. Default is |
verbose_missForest |
Prints out messages by iteration. Default is |
... |
Settings to pass to YARF during the iterations. |
After each iteration the difference between the previous and the new imputed data matrix is assessed for the continuous and categorical parts. The stopping criterion is defined such that the imputation process is stopped as soon as both differences have become larger once. In case of only one type of variable the computation stops as soon as the corresponding difference goes up for the first time. However, the imputation last performed where both differences went up is generally less accurate than the previous one. Therefore, whenever the computation stops due to the stopping criterion (and not due to 'maxiter') the before last imputation matrix is returned.
A list with objects: Xtrain_imp (and Xtest_imp if Xtest was specified) as well as nrmses which is a record of the sum of the normalized rmse's over all numeric features by iteration and npfcs which is a record of the sum of the misclassification rates over all categorical features by iteration.
Adam Kapelner
This generates data with a plot of an ROC curve or an ROC like curve for binary response
YARFROC( X, y, x_axis = "FPR", y_axis = "TPR", use_prop_data = 1, minimum_class_proportion = 0.02, desired_interval = 0.05, tolerance = 0.001, y_axis_fine_resolution = FALSE, plot = TRUE, verbose = TRUE, ... )YARFROC( X, y, x_axis = "FPR", y_axis = "TPR", use_prop_data = 1, minimum_class_proportion = 0.02, desired_interval = 0.05, tolerance = 0.001, y_axis_fine_resolution = FALSE, plot = TRUE, verbose = TRUE, ... )
X |
The covariate data frame |
y |
The binary response vector |
x_axis |
The rate to plot on the x-axis of the ROC like plot as a 3-character case-insensitive string. The default is "FPR" but it canbe any of the following: "tpr" - true positive rate (AKA "sensitivity" or "recall") i.e the number predicted positive among those truly positive "fnr" - false negative rate (AKA "miss rate") i.e the number predicted negative among those truly positive "fpr" - false positive rate (AKA "fall-out") i.e. the number predicted positive among those truly negative "tnr" - true negative rate (AKA "specificity") i.e. the number predicted negative among those truly negative "ppv" - positive predictive value i.e. the number truly positive among those predicted positive "fdr" - false discovery rate i.e. the number truly negative among those predicted positive "fomr" - false omission rate i.e. the number truly positive among those predicted negative "npv" - negative predictive value i.e. the number truly negative among those predicted negative |
y_axis |
The rate to plot on the y-axis of the ROC like plot as a 3-character string.
The default is "TPR" but it can be any of those
listed in the documentation for the |
use_prop_data |
Relative multiplier controlling the resampled class proportions. |
minimum_class_proportion |
Smallest positive-class proportion considered while tracing the curve. |
desired_interval |
Target spacing between adjacent metric values. |
tolerance |
Recursion stops when candidate class proportions differ by less than this value. |
y_axis_fine_resolution |
Whether to make an additional pass emphasizing resolution on the y-axis. |
plot |
Whether to draw intermediate and final ROC-like plots. |
verbose |
Whether to print progress information. |
... |
Additional arguments passed to YARF construction. This cannot include |
A data frame containing all
models with the percent of first class in levels(y) for the bootstrap indices to replicate
any point on the plot.
Adam