Publication:
On mathematical optimization for clustering categories in contingency tables

Simple item page
dc.affiliation.dptoUC3M. Departamento de Estadísticaes
dc.contributor.authorCarrizosa, Emilio
dc.contributor.authorGuerrero Lozano, Vanesa
dc.contributor.authorRomero Morales, Dolores
dc.contributor.funderEuropean Commissionen
dc.contributor.funderAgencia Estatal de Investigación (España)es
dc.date.accessioned2022-07-13T10:17:30Z
dc.date.available2022-07-13T10:17:30Z
dc.date.issued2022-06-28
dc.description.abstractMany applications in data analysis study whether two categorical variables are independent using a function of the entries of their contingency table. Often, the categories of the variables, associated with the rows and columns of the table, are grouped, yielding a less granular representation of the categorical variables. The purpose of this is to attain reasonable sample sizes in the cells of the table and, more importantly, to incorporate expert knowledge on the allowable groupings. However, it is known that the conclusions on independence depend, in general, on the chosen granularity, as in the Simpson paradox. In this paper we propose a methodology to, for a given contingency table and a fixed granularity, find a clustered table with the highest χ2 statistic. Repeating this procedure for different values of the granularity, we can either identify an extreme grouping, namely the largest granularity for which the statistical dependence is still detected, or conclude that it does not exist and that the two variables are dependent regardless of the size of the clustered table. For this problem, we propose an assignment mathematical formulation and a set partitioning one. Our approach is flexible enough to include constraints on the desirable structure of the clusters, such as must-link or cannot-link constraints on the categories that can, or cannot, be merged together, and ensure reasonable sample sizes in the cells of the clustered table from which trustful statistical conclusions can be derived. We illustrate the usefulness of our methodology using a dataset of a medical study.en
dc.description.sponsorshipOpen Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature. This research has been financed in part by research projects EC H2020 MSCA RISE NeEDS (Grant agreement ID: 822214), FQM-329, P18-FR-2369 and US-1381178 (Junta de Andalucía, with FEDER Funds), PID2019-110886RB-I00 and PID2019-104901RB-I00 (funded by MCIN/AEI/10.13039/501100011033). This support is gratefully acknowledged.en
dc.identifier.bibliographicCitationCarrizosa, E., Guerrero, V., & Romero Morales, D. (2022). On mathematical optimization for clustering categories in contingency tables. Advances in Data Analysis and Classification.en
dc.identifier.doihttps://doi.org/10.1007/s11634-022-00508-4
dc.identifier.issn1862-5347
dc.identifier.publicationfirstpage1es
dc.identifier.publicationlastpage23es
dc.identifier.publicationtitleAdvances in Data Analysis and Classificationen
dc.identifier.urihttps://hdl.handle.net/10016/35451
dc.identifier.uxxiAR/0000030950
dc.language.isoenges
dc.publisherSpringeres
dc.relation.projectIDinfo:eu-repo/grantAgreement/EC/H2020/822214es
dc.relation.projectIDGobierno de España. PID2019-110886RB-I00es
dc.relation.projectIDGobierno de España. PID2019-104901RB-I00es
dc.relation.projectIDAT-2022es
dc.rights© The Author(s) 2022es
dc.rightsAtribución 3.0 España*
dc.rights.accessRightsopen accessen
dc.rights.urihttp://creativecommons.org/licenses/by/3.0/es/*
dc.subject.ecienciaEstadísticaes
dc.subject.otherContingency Tablesen
dc.subject.otherMathematical optimizationes
dc.subject.otherRelational constraintsen
dc.subject.otherClusteringen
dc.titleOn mathematical optimization for clustering categories in contingency tablesen
dc.typeresearch article*
dc.type.hasVersionVoR*
dspace.entity.typePublication
Files
Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
mathematical_ADAC_2022.pdf
Size:
681.15 KB
Format:
Adobe Portable Document Format
Download
Collections
DES - Artículos de Revistas