Ricardo Menchaca Méndez
Investigador
Profesor titular en el Laboratorio de Comunicaciones y Redes de computadoras del CIC-IPN. En 2013 obtuvo su Doctorado en Ciencias de la Computación por parte de la Universidad de California en Santa Cruz bajo la dirección del Profesor Dimitris Achlioptas. Ha publicado múltiples artículos científicos en revistas y conferencias que han sido citados más de 100 veces hasta la fecha. Sus intereses de investigación actual son estructuras aleatorias, optimización combinatorial, análisis y diseño de algoritmos y aprendizaje de máquina.
Transformers for Tabular Data
Tabular data is a widely used data format in machine learning because it can naturally describe many
different phenomena. However, deep learning has not been as successful with tabular data as with other
applications such as natural language processing or vision. In this paper, we demonstrate that Transformers
can perform better than other methods for tabular data. We argue that this is because Transformers can
function as an efficient dynamic feature selection algorithm. We introduce a measure that captures the
cumulative attention that a feature gets across all the layers of the Transformer. Our experiments revealed
that Transformers learn to give more cumulative attention to relevant features, which is important for
high-dimensional datasets where many features may be irrelevant or the relevant features may change
depending on the input. Moreover, Transformers can handle a variable number of features with the same number
of parameters. This contrasts with other machine learning models whose parameters increase with the number
of features, requiring larger datasets to deduce relationships among features effectively.