This article performs a study on correlation between pairs of variables in dependence on the problem dimensionality. Two tests, based on Pearson and Spearman coefficients, have been designed and used in this work. In total, 86 test problems ranging between 10 and 1000 variables have been studied. If the most commonly used experimental conditions are used, the correlation between pairs of variables appears, from the perspective of the search algorithm, to consistently decrease. This effect is not due to the fact that the dimensionality modifies the nature of the problem but is a consequence of the experimental conditions: the computational feasibility of the experiments imposes an extremely shallow search in case of high dimensions. An exponential increase of budget and population with the dimensionality is still practically impossible. Nonetheless, since real-world application may require that large scale problems are tackled despite of the limited budget, an algorithm can quickly improve upon initial guesses if it integrates the knowledge that an apparent weak correlation between pairs of variables occurs, regardless the nature of the problem.
Large scale problems in practice: The effect of dimensionality on the interaction among variables / Caraffini, Fabio; Neri, Ferrante; Iacca, Giovanni. - 10199:(2017), pp. 636-652. (Intervento presentato al convegno EvoApplications 2017 tenutosi a Amsterdam nel 19th-21st April 2017) [10.1007/978-3-319-55849-3_41].
Large scale problems in practice: The effect of dimensionality on the interaction among variables
Iacca, Giovanni
2017-01-01
Abstract
This article performs a study on correlation between pairs of variables in dependence on the problem dimensionality. Two tests, based on Pearson and Spearman coefficients, have been designed and used in this work. In total, 86 test problems ranging between 10 and 1000 variables have been studied. If the most commonly used experimental conditions are used, the correlation between pairs of variables appears, from the perspective of the search algorithm, to consistently decrease. This effect is not due to the fact that the dimensionality modifies the nature of the problem but is a consequence of the experimental conditions: the computational feasibility of the experiments imposes an extremely shallow search in case of high dimensions. An exponential increase of budget and population with the dimensionality is still practically impossible. Nonetheless, since real-world application may require that large scale problems are tackled despite of the limited budget, an algorithm can quickly improve upon initial guesses if it integrates the knowledge that an apparent weak correlation between pairs of variables occurs, regardless the nature of the problem.File | Dimensione | Formato | |
---|---|---|---|
Chapter_Author.pdf
Open Access dal 26/03/2019
Tipologia:
Post-print referato (Refereed author’s manuscript)
Licenza:
Tutti i diritti riservati (All rights reserved)
Dimensione
455.29 kB
Formato
Adobe PDF
|
455.29 kB | Adobe PDF | Visualizza/Apri |
I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione