Approaches To Using Digital Technology To Address Diversity Issues In Neuropsychological Assessment

Authors: Sunderaraman, P. & Porrselvi, A.P.

Synopsis: The adoption of digital technology in neuropsychological assessment can help address the need for culturally appropriate tools that have sound psychometric properties and can help neuropsychology reach underserved populations around the world.

Author Disclosures: Dr. Porrselvi is the author of the TAM Battery and holds the copyright to this test battery.

Overall Description:

Most of the traditional tests commonly used for assessment of cognition were developed in the United States incorporating stimuli from a western perspective. Unfortunately, the adoption of these tests inadvertently led to cultural unfairness for multiple groups both within the United States (e.g., immigrants, diverse racial/ethnic groups) and globally (e.g., the use of these traditional tests in Asian and other international countries). The use of digital technology has been explored since the 1970’s and different approaches to conducting assessments using digital technology have been adopted.

Case Study Examples:

1. The TAM Battery (Porrselvi, 2022) developed for Tamil speakers, consists of neuropsychological tests which are inspired from the traditional paper and pencil tests but have been adapted to be administered digitally based on the test construct. The battery includes pre-recorded test stimuli for enhanced standardization and semi-automated scoring. It was developed to include features that tap into the advantages digital testing offers for resource optimization in underserved populations where there are inadequate financial resources, few trained professionals, and geographical barriers to serving these populations. When the test battery is enabled with dynamic norming it may not require a single large study involving collaboration between researchers and clinicians from multiple centers to develop the normative dataset. Conducting a large-scale study is challenging in a target population, such as those from India, where financial resources and availability of trained professionals are both limited. Dynamic norming overcomes these barriers by providing updated norms wherein each test data is compared against all previous tests taken in the normative dataset. Hence the normative data has a reduced risk of becoming outdated for the population (Trahan et al, 2014). The battery also includes the possibility of testing in multiple languages once those translations become available, after their equivalence is established, and after they are validated. This will be useful for conducting pan-India assessments and assessments across languages within the same region given the heterogeneity in the population.
2. A novel avenue for technology to be culturally adapted is their use in the development of and/or monitoring of functional abilities. Assessments such as the Oxford digital multiple errands test (OxMET) involve a series of shopping-based tasks where participants use a tablet to complete the task (Sam et al., 2021). The Virtual Supermarket Program, developed for Chinese participants in sync with their cultural habits, is another computer-based task (Yan et al., 2021). Similar tasks have been developed in other countries such as in Greece and Singapore (Boz et al., 2020; Chua at el., 2019). While these tasks were designed to examine cognitive functioning (learning and memory, executive functioning), another task designed to measure a functional ability is the Virtual Kitchen Challenge which relies on a touchscreen (Giovannetti et al., 2019) and initial findings from this task are promising. Other researchers have used driving simulators to examine driving as a functional ability (Raphail, Vickers, Leist, & Schultheis, 2019). The potential of these tasks to be adapted to diverse populations and cultures is feasible given that the stimuli can be manipulated, and test administration occurs using digital modalities. Such functional assessments could help us bridge the gap between different cultures and people from different backgrounds for the purpose of assessing cognition and function with ecological validity.

Limitations:

The most obvious approach to adopting technology in neuropsychological assessment has been to recreate a digital version of commonly used cognitive tests. There are a number of problems with this approach such as the continued use of the “biased” test stimuli, the dearth of psychometric information about the digital version of the tests, and factors such as background distraction and lack of reliable internet services impacting digital test performance (Fox-Fuller, Rizer, Andersen, & Sunderaraman, 2022). While these seem like general considerations, performance on digital tests might be hindered by unexpected cultural challenges encountered during data collection. For example, Porrselvi & Shankar (2018) observed that, in their participants from India, the novelty of using a stylus to draw on a tablet screen may have interfered with the learning process on a figural memory test. They found that, despite being technologically proficient, cognitively normal participants performed worse on the digital version of the test when compared to their performance on the paper and pencil version.

Despite these challenges, digital tests offer unique advantages. There might be ways in which digital tests can be tailored to meet expectations of different cultures. Technology has the potential to take neuropsychology to regions of the world where access to the benefits that our field offers have been limited merely due to the lack of proper assessment tools as we have been relying on traditional tests for too long. The comparisons drawn between the traditional tests and their computerized version have slowed progress in the development of new, efficient tasks that use the complete potential of computerization of assessment to yield data about the functioning of an individual.

Further Reading:

Porrselvi, A.P & Shankar, V (2018). Limitations of Integrating Technology in Cognitive Testing of Non-Verbal Memory. International Journal of Indian Psychology, 6(3), 167-173. DIP:18.01.098/20180603, DOI:10.25215/0603.098

Fox-Fuller, J. T., Rizer, S., Andersen, S. L., & Sunderaraman, P. (2022). Survey findings about the experiences, challenges, and practical advice/solutions regarding teleneuropsychological assessment in adults. Archives of Clinical Neuropsychology, 37(2), 274-291.

Porrselvi A. P. (2022) TAM battery: Development and pilot testing of a Tamil computer-assisted cognitive test battery for older adults, The Clinical Neuropsychologist, DOI: 10.1080/13854046.2022.2156396

Sam S. Webb, Anders Jespersen, Evangeline G. Chiu, Francesca Payne, Romina Basting, Mihaela D. Duta & Nele Demeyere (2021) The Oxford digital multiple errands test (OxMET): Validation of a simplified computer tablet based multiple errands test, Neuropsychological Rehabilitation, DOI: 10.1080/09602011.2020.1862679)

Yan M, Yin H, Meng Q, Wang S, Ding Y, Li G, Wang C, Chen L, A Virtual Supermarket Program for the Screening of Mild Cognitive Impairment in Older Adults: Diagnostic Accuracy Study JMIR Serious Games 2021;9(4):e30919 doi: 10.2196/30919 PMID: 34870610 PMCID: 8686451

Trahan LH, Stuebing KK, Fletcher JM, Hiscock M. The Flynn effect: a meta-analysis. Psychol Bull. 2014 Sep;140(5):1332-60. doi: 10.1037/a0037173.

Boz HS, Limoncu H, Zygouris S, Tsolaki M, Giakoumis D, Votis K, Tzovaras D, Öztürk V, & Yener GG (2020) A new tool to assess amnestic mild cognitive impairment in Turkish older adults: virtual supermarket (VSM), Aging, Neuropsychology, and Cognition, 27:5, 639-653, DOI: 10.1080/13825585.2019.1663146

Chua SIL, Tan NC, Wong WT, Allen JC, Quah JHM, Malhotra R, et al. Virtual Reality for Screening of Cognitive Function in Older Persons: Comparative Study. J Med Internet Res 2019 Aug 01;21(8):e14821

Giovannetti, T., Yamaguchi, T., Roll, E., Harada, T., Rycroft, S. S., Divers, R., … & Mis, R. (2019). The Virtual Kitchen Challenge: preliminary data from a novel virtual reality test of mild difficulties in everyday functioning. Aging, Neuropsychology, and Cognition, 26(6), 823-841.

Raphail, A. M. A., Vickers, K. L., Leist, T. P., & Schultheis, M. T. (2019). The relationship between MS symptom severity measures and performance on driving variability metrics in a virtual reality simulator. American journal of physical medicine and rehabilitation.