ultra::src::holdout_validation Class Reference

#include <holdout_validation.h>

Inheritance diagram for ultra::src::holdout_validation:

Classes
struct	params

Public Member Functions
	holdout_validation (src::problem &, params={})
evaluation_context	training_setup (unsigned) noexcept override
evaluation_context	shake (unsigned) noexcept override
bool	validation_setup (unsigned) noexcept override
std::unique_ptr< validation_strategy >	clone () const override

Detailed Description

Holdout validation, aka one round cross-validation or conventional validation.

Holdout validation involves partitioning a sample of data into complementary subsets, performing the analysis on one subset (called the training set) and validating the analysis on the other subset (called the validation set).

See also

https://en.wikipedia.org/wiki/Training,_validation,_and_test_data_sets

Examples

sonar.cc.

Constructor & Destructor Documentation

holdout_validation()

ultra::src::holdout_validation::holdout_validation ( src::problem & prob, params par = {} )

explicit

Sets up a hold-out validator.

Parameters

[in]	prob	current problem
[in]	par	parameters for controlling the splitting / subsampling

Examples from prob.data(training) are randomly partitioned into training, validation and test set according to parameters contained in par.

Note

Behaviour summary (nothing should "jump" unexpectedly):

User input Result (train / val / test) 0 / -1 70 / 30 / 0 0 / 10 70 / 10 / 20 80 / -1 80 / 20 / 0 80 / 50 80 / 20 / 0 100 / anything 100 / 0 / 0 -5 / 5 70 / 5 / 25

Member Function Documentation

clone()

std::unique_ptr< validation_strategy > ultra::src::holdout_validation::clone ( ) const

nodiscardoverridevirtual

Creates a polymorphic copy of the validation strategy.

Returns

a std::unique_ptr owning a copy of this validation strategy

Remarks

The cloned object shares the same underlying problem instance as the original, since the problem is stored by reference. This is intentional: validation strategies are lightweight configuration objects and do not own the problem data.

Note

The caller is responsible for ensuring that the referenced problem outlives all clones created via this method.

See also

validation_strategy::clone

Implements ultra::validation_strategy.

shake()

evaluation_context ultra::src::holdout_validation::shake ( unsigned )

inlinenodiscardoverridevirtualnoexcept

Changes the training environment during evolution.

Returns

true if the training environment has changed

Note

Called at the beginning of every generation (multiple times per run).

Implements ultra::validation_strategy.

training_setup()

evaluation_context ultra::src::holdout_validation::training_setup ( unsigned )

nodiscardoverridevirtualnoexcept

Prepares the data structures / environment needed for training.

Note

Called at the beginning of the evolution (one time per run).

Implements ultra::validation_strategy.

validation_setup()

bool ultra::src::holdout_validation::validation_setup ( unsigned )

overridevirtualnoexcept

Prepares the data structures / environment needed for validation

Returns

true if a validation environment can be set up; false otherwise

Note

Called at the end of the evolution (one time per run).

Implements ultra::validation_strategy.

---

The documentation for this class was generated from the following files:

• holdout_validation.h
• holdout_validation.cc