Overcoming the linearity of Ordinal Logistic Regression adding non-linear covariates from Evolutionary Hybrid Neural Network models

Research areas:

• Methodology - Machine learning

Year:

2015

Type of Publication:

In Proceedings

Authors:

• Dorado-Moreno, Manuel
• Gutiérrez, Pedro Antonio
• Sánchez-Monedero, Javier
• Hervás-Martínez, César

Volume:

9422

Book title:

16th Conference of the Spanish Association for Artificial Intelligence (CAEPIA 2015)

Series:

Lecture Notes in Computer Science (LNCS)

Pages:

301-311

Organization:

Albacete (Spain)

Month:

09th-12th November

ISBN:

978-3-319-24597-3

BibTex:

@conference{Dorado15c, author = "Manuel Dorado-Moreno and Pedro Antonio Guti{\'e}rrez and Javier S{\'a}nchez-Monedero and C{\'e}sar Herv{\'a}s-Mart{\'i}nez", abstract = " This paper proposes a non-linear ordinal logistic regression method based on the combination of a linear regression model and an evolutionary neural network with hybrid basis functions, combining Sigmoidal Unit and Radial Basis Functions neural networks. The process for obtaining the coefficients is carried out in several steps. Firstly we use an evolutionary algorithm to determine the structure of the hybrid neural network model, in a second step we augment the initial feature space (covariate space) adding the non-linear transformations of the input variables given by the hybrid hidden layer of the best individual of the evolutionary algorithm. Finally, we apply an ordinal logistic regression in the new feature space. This methodology is tested using 10 benchmark problems from the UCI repository. The hybrid model outperforms both the RBF and the SU pure models obtaining a good compromise between them and better results in terms of accuracy and ordinal classification error.", booktitle = "16th Conference of the Spanish Association for Artificial Intelligence (CAEPIA 2015)", doi = "10.1007/978-3-319-24598-0_27", isbn = "978-3-319-24597-3", month = "09th-12th November", organization = "Albacete (Spain)", pages = "301--311", publisher = "Springer International Publishing", series = " Lecture Notes in Computer Science (LNCS)", title = "{O}vercoming the linearity of {O}rdinal {L}ogistic {R}egression adding non-linear covariates from {E}volutionary {H}ybrid {N}eural {N}etwork models", url = "doi.org/10.1007/978-3-319-24598-0_27", volume = "9422", year = "2015", }

Abstract:

This paper proposes a non-linear ordinal logistic regression method based on the combination of a linear regression model and an evolutionary neural network with hybrid basis functions, combining Sigmoidal Unit and Radial Basis Functions neural networks. The process for obtaining the coefficients is carried out in several steps. Firstly we use an evolutionary algorithm to determine the structure of the hybrid neural network model, in a second step we augment the initial feature space (covariate space) adding the non-linear transformations of the input variables given by the hybrid hidden layer of the best individual of the evolutionary algorithm. Finally, we apply an ordinal logistic regression in the new feature space. This methodology is tested using 10 benchmark problems from the UCI repository. The hybrid model outperforms both the RBF and the SU pure models obtaining a good compromise between them and better results in terms of accuracy and ordinal classification error.