{"id":730,"date":"2022-11-16T13:37:55","date_gmt":"2022-11-16T13:37:55","guid":{"rendered":"https:\/\/igormatias.com\/?p=730"},"modified":"2026-02-02T13:05:37","modified_gmt":"2026-02-02T13:05:37","slug":"behavioral-data-categorization-for-transformers-based-models-in-digital-health","status":"publish","type":"post","link":"https:\/\/igormatias.com\/pt\/2022\/11\/16\/behavioral-data-categorization-for-transformers-based-models-in-digital-health\/","title":{"rendered":"Behavioral Data Categorization for Transformers-based Models in Digital Health"},"content":{"rendered":"<p><span id=\"ContentPlaceHolder1_LinkPaperPage_LinkPaperContent_LabelAbstractTitle\" class=\"LabelSearchRepeater\">Abstract: Transformers are recent deep learning (DL) models used to capture the dependence between parts of sequential data. While their potential was already demonstrated in the natural language processing (NLP) domain, emerging research shows transformers can also be an adequate modeling approach to relate longitudinal multi-featured continuous behavioral data to future health outcomes. As transformers-based predictions are based on a domain lexicon, the use of categories, commonly used in specialized areas to cluster values, is the likely way to compose lexica. However, the number of categories may influence the transformer prediction accuracy, mainly when the categorization process creates imbalanced datasets, or the search space is very restricted to generate optimal feasible solutions. This paper analyzes the relationship between models&#8217; accuracy and the sparsity of behavioral data categories that compose the lexicon. This analysis relies on a case example that uses mQoL-Transformer to model the influence of physical activity behavior on sleep health. Results show that the number of categories shall be treated as a further transformer&#8217;s hyperparameter, which can balance the literature-based categorization and optimization aspects. Thus, DL processes could also obtain similar accuracies compared to traditional approaches, such as long short-term memory, when used to process short behavioral data sequences.<\/span><\/p>\n<p>Clauirton Siebra<sup>1<\/sup>, Igor Matias<sup>1<\/sup>, and Katarzyna Wac<sup>1<\/sup><\/p>\n<p><sup>1<\/sup>Quality of Life Technologies Lab, University of Geneva, Switzerland<\/p>\n<p><span id=\"ContentPlaceHolder1_LinkPaperPage_LinkPaperContent_AuxInProceeding\" class=\"PublicationsDetailNormal\">In 2022 IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI).<\/span><\/p>\n<p><a href=\"https:\/\/doi.org\/10.1109\/BHI56158.2022.9926938\" target=\"_blank\" rel=\"noopener noreferrer\">https:\/\/doi.org\/10.1109\/BHI56158.2022.9926938<\/a><\/p>\n\n\n<p><\/p>","protected":false},"excerpt":{"rendered":"<p>Abstract: Transformers are recent deep learning (DL) models used to capture the dependence between parts of sequential data. While&#8230;<\/p>","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":"","_links_to":"","_links_to_target":""},"categories":[11],"tags":[],"class_list":["post-730","post","type-post","status-publish","format-standard","hentry","category-conference"],"gutentor_comment":0,"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v25.1 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Behavioral Data Categorization for Transformers-based Models in Digital Health<\/title>\n<meta name=\"description\" content=\"Abstract: Transformers are recent deep learning (DL) models used to capture the dependence between parts of sequential data. While...\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/igormatias.com\/pt\/2022\/11\/16\/behavioral-data-categorization-for-transformers-based-models-in-digital-health\/\" \/>\n<meta property=\"og:locale\" content=\"pt_PT\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Behavioral Data Categorization for Transformers-based Models in Digital Health\" \/>\n<meta property=\"og:description\" content=\"Abstract: Transformers are recent deep learning (DL) models used to capture the dependence between parts of sequential data. While...\" \/>\n<meta property=\"og:url\" content=\"https:\/\/igormatias.com\/pt\/2022\/11\/16\/behavioral-data-categorization-for-transformers-based-models-in-digital-health\/\" \/>\n<meta property=\"og:site_name\" content=\"Igor Matias &#x1f9e0;&#x1f916;\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/igormatias16\" \/>\n<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/igormatias16\" \/>\n<meta property=\"article:published_time\" content=\"2022-11-16T13:37:55+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2026-02-02T13:05:37+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/igormatias.com\/wp-content\/uploads\/2026\/04\/Matias_2026_square_933.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"933\" \/>\n\t<meta property=\"og:image:height\" content=\"933\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Igor Matias\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@IgorMatias3\" \/>\n<meta name=\"twitter:site\" content=\"@IgorMatias3\" \/>\n<meta name=\"twitter:label1\" content=\"Escrito por\" \/>\n\t<meta name=\"twitter:data1\" content=\"Igor Matias\" \/>\n\t<meta name=\"twitter:label2\" content=\"Tempo estimado de leitura\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutos\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/igormatias.com\/2022\/11\/16\/behavioral-data-categorization-for-transformers-based-models-in-digital-health\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/igormatias.com\/2022\/11\/16\/behavioral-data-categorization-for-transformers-based-models-in-digital-health\/\"},\"author\":{\"name\":\"Igor Matias\",\"@id\":\"https:\/\/igormatias.com\/#\/schema\/person\/da6ca85cd791ef8f7a14e9bd3714f519\"},\"headline\":\"Behavioral Data Categorization for Transformers-based Models in Digital Health\",\"datePublished\":\"2022-11-16T13:37:55+00:00\",\"dateModified\":\"2026-02-02T13:05:37+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/igormatias.com\/2022\/11\/16\/behavioral-data-categorization-for-transformers-based-models-in-digital-health\/\"},\"wordCount\":241,\"publisher\":{\"@id\":\"https:\/\/igormatias.com\/#\/schema\/person\/da6ca85cd791ef8f7a14e9bd3714f519\"},\"articleSection\":[\"Conference\"],\"inLanguage\":\"pt-PT\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/igormatias.com\/2022\/11\/16\/behavioral-data-categorization-for-transformers-based-models-in-digital-health\/\",\"url\":\"https:\/\/igormatias.com\/2022\/11\/16\/behavioral-data-categorization-for-transformers-based-models-in-digital-health\/\",\"name\":\"Behavioral Data Categorization for Transformers-based Models in Digital Health\",\"isPartOf\":{\"@id\":\"https:\/\/igormatias.com\/#website\"},\"datePublished\":\"2022-11-16T13:37:55+00:00\",\"dateModified\":\"2026-02-02T13:05:37+00:00\",\"description\":\"Abstract: Transformers are recent deep learning (DL) models used to capture the dependence between parts of sequential data. While...\",\"breadcrumb\":{\"@id\":\"https:\/\/igormatias.com\/2022\/11\/16\/behavioral-data-categorization-for-transformers-based-models-in-digital-health\/#breadcrumb\"},\"inLanguage\":\"pt-PT\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/igormatias.com\/2022\/11\/16\/behavioral-data-categorization-for-transformers-based-models-in-digital-health\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/igormatias.com\/2022\/11\/16\/behavioral-data-categorization-for-transformers-based-models-in-digital-health\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/igormatias.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Behavioral Data Categorization for Transformers-based Models in Digital Health\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/igormatias.com\/#website\",\"url\":\"https:\/\/igormatias.com\/\",\"name\":\"Igor Matias\",\"description\":\"(Computer Science, Health, AI)\",\"publisher\":{\"@id\":\"https:\/\/igormatias.com\/#\/schema\/person\/da6ca85cd791ef8f7a14e9bd3714f519\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/igormatias.com\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"pt-PT\"},{\"@type\":[\"Person\",\"Organization\"],\"@id\":\"https:\/\/igormatias.com\/#\/schema\/person\/da6ca85cd791ef8f7a14e9bd3714f519\",\"name\":\"Igor Matias\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"pt-PT\",\"@id\":\"https:\/\/igormatias.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/igormatias.com\/wp-content\/uploads\/2026\/04\/Matias_2026_square_933.jpg\",\"contentUrl\":\"https:\/\/igormatias.com\/wp-content\/uploads\/2026\/04\/Matias_2026_square_933.jpg\",\"width\":933,\"height\":933,\"caption\":\"Igor Matias\"},\"logo\":{\"@id\":\"https:\/\/igormatias.com\/#\/schema\/person\/image\/\"},\"description\":\"Ph.D. student in Information Systems at the University of Geneva, Switzerland, focusing Alzheimer\u2019s Disease diagnosis. Master of Computer Science and Engineering by University of Beira Interior, Portugal.\",\"sameAs\":[\"https:\/\/igormatias.com\/\",\"https:\/\/www.facebook.com\/igormatias16\",\"https:\/\/instagram.com\/igormatias3\",\"https:\/\/linkedin.com\/in\/igormatias3\",\"https:\/\/x.com\/IgorMatias3\"]}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Behavioral Data Categorization for Transformers-based Models in Digital Health","description":"Abstract: Transformers are recent deep learning (DL) models used to capture the dependence between parts of sequential data. While...","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/igormatias.com\/pt\/2022\/11\/16\/behavioral-data-categorization-for-transformers-based-models-in-digital-health\/","og_locale":"pt_PT","og_type":"article","og_title":"Behavioral Data Categorization for Transformers-based Models in Digital Health","og_description":"Abstract: Transformers are recent deep learning (DL) models used to capture the dependence between parts of sequential data. While...","og_url":"https:\/\/igormatias.com\/pt\/2022\/11\/16\/behavioral-data-categorization-for-transformers-based-models-in-digital-health\/","og_site_name":"Igor Matias &#x1f9e0;&#x1f916;","article_publisher":"https:\/\/www.facebook.com\/igormatias16","article_author":"https:\/\/www.facebook.com\/igormatias16","article_published_time":"2022-11-16T13:37:55+00:00","article_modified_time":"2026-02-02T13:05:37+00:00","og_image":[{"width":933,"height":933,"url":"https:\/\/igormatias.com\/wp-content\/uploads\/2026\/04\/Matias_2026_square_933.jpg","type":"image\/jpeg"}],"author":"Igor Matias","twitter_card":"summary_large_image","twitter_creator":"@IgorMatias3","twitter_site":"@IgorMatias3","twitter_misc":{"Escrito por":"Igor Matias","Tempo estimado de leitura":"2 minutos"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/igormatias.com\/2022\/11\/16\/behavioral-data-categorization-for-transformers-based-models-in-digital-health\/#article","isPartOf":{"@id":"https:\/\/igormatias.com\/2022\/11\/16\/behavioral-data-categorization-for-transformers-based-models-in-digital-health\/"},"author":{"name":"Igor Matias","@id":"https:\/\/igormatias.com\/#\/schema\/person\/da6ca85cd791ef8f7a14e9bd3714f519"},"headline":"Behavioral Data Categorization for Transformers-based Models in Digital Health","datePublished":"2022-11-16T13:37:55+00:00","dateModified":"2026-02-02T13:05:37+00:00","mainEntityOfPage":{"@id":"https:\/\/igormatias.com\/2022\/11\/16\/behavioral-data-categorization-for-transformers-based-models-in-digital-health\/"},"wordCount":241,"publisher":{"@id":"https:\/\/igormatias.com\/#\/schema\/person\/da6ca85cd791ef8f7a14e9bd3714f519"},"articleSection":["Conference"],"inLanguage":"pt-PT"},{"@type":"WebPage","@id":"https:\/\/igormatias.com\/2022\/11\/16\/behavioral-data-categorization-for-transformers-based-models-in-digital-health\/","url":"https:\/\/igormatias.com\/2022\/11\/16\/behavioral-data-categorization-for-transformers-based-models-in-digital-health\/","name":"Behavioral Data Categorization for Transformers-based Models in Digital Health","isPartOf":{"@id":"https:\/\/igormatias.com\/#website"},"datePublished":"2022-11-16T13:37:55+00:00","dateModified":"2026-02-02T13:05:37+00:00","description":"Abstract: Transformers are recent deep learning (DL) models used to capture the dependence between parts of sequential data. While...","breadcrumb":{"@id":"https:\/\/igormatias.com\/2022\/11\/16\/behavioral-data-categorization-for-transformers-based-models-in-digital-health\/#breadcrumb"},"inLanguage":"pt-PT","potentialAction":[{"@type":"ReadAction","target":["https:\/\/igormatias.com\/2022\/11\/16\/behavioral-data-categorization-for-transformers-based-models-in-digital-health\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/igormatias.com\/2022\/11\/16\/behavioral-data-categorization-for-transformers-based-models-in-digital-health\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/igormatias.com\/"},{"@type":"ListItem","position":2,"name":"Behavioral Data Categorization for Transformers-based Models in Digital Health"}]},{"@type":"WebSite","@id":"https:\/\/igormatias.com\/#website","url":"https:\/\/igormatias.com\/","name":"Igor Matias","description":"(Computer Science, Health, AI)","publisher":{"@id":"https:\/\/igormatias.com\/#\/schema\/person\/da6ca85cd791ef8f7a14e9bd3714f519"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/igormatias.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"pt-PT"},{"@type":["Person","Organization"],"@id":"https:\/\/igormatias.com\/#\/schema\/person\/da6ca85cd791ef8f7a14e9bd3714f519","name":"Igor Matias","image":{"@type":"ImageObject","inLanguage":"pt-PT","@id":"https:\/\/igormatias.com\/#\/schema\/person\/image\/","url":"https:\/\/igormatias.com\/wp-content\/uploads\/2026\/04\/Matias_2026_square_933.jpg","contentUrl":"https:\/\/igormatias.com\/wp-content\/uploads\/2026\/04\/Matias_2026_square_933.jpg","width":933,"height":933,"caption":"Igor Matias"},"logo":{"@id":"https:\/\/igormatias.com\/#\/schema\/person\/image\/"},"description":"Ph.D. student in Information Systems at the University of Geneva, Switzerland, focusing Alzheimer\u2019s Disease diagnosis. Master of Computer Science and Engineering by University of Beira Interior, Portugal.","sameAs":["https:\/\/igormatias.com\/","https:\/\/www.facebook.com\/igormatias16","https:\/\/instagram.com\/igormatias3","https:\/\/linkedin.com\/in\/igormatias3","https:\/\/x.com\/IgorMatias3"]}]}},"_links":{"self":[{"href":"https:\/\/igormatias.com\/pt\/wp-json\/wp\/v2\/posts\/730"}],"collection":[{"href":"https:\/\/igormatias.com\/pt\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/igormatias.com\/pt\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/igormatias.com\/pt\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/igormatias.com\/pt\/wp-json\/wp\/v2\/comments?post=730"}],"version-history":[{"count":4,"href":"https:\/\/igormatias.com\/pt\/wp-json\/wp\/v2\/posts\/730\/revisions"}],"predecessor-version":[{"id":1069,"href":"https:\/\/igormatias.com\/pt\/wp-json\/wp\/v2\/posts\/730\/revisions\/1069"}],"wp:attachment":[{"href":"https:\/\/igormatias.com\/pt\/wp-json\/wp\/v2\/media?parent=730"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/igormatias.com\/pt\/wp-json\/wp\/v2\/categories?post=730"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/igormatias.com\/pt\/wp-json\/wp\/v2\/tags?post=730"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}