Institutional-Industry Collaboration for Data Sharing and Research

Author: Delis, D.C., Bilder, R.M.

Author Disclosures:

• Delis has received royalty income from Pearson.
• Bilder has received funding from the National Institutes of Health for research with for-profit organizations, including Pearson and ThinkNow, Inc. He also has received honoraria or consulting fees within the last two years from Atai Life Sciences and Karuna Therapeutics. 

Synopsis: In the past, few neuropsychologists outside of university or other large medical centers had the luxury of conducting sophisticated clinical research studies. The digital age is making it possible for exciting new research partnerships to form between test publishers and neuropsychologists working in all types of settings. 

Overall Description: In the past, neuropsychologists who did not work at university medical centers or other large hospital systems (e.g., VA medical centers) were at a tremendous disadvantage in conducting research with homogeneous clinical populations. This barrier has limited the type of clinical research that neuropsychologists working in academic settings without medical schools or in private practice can conduct. The digital age promises to break down these barriers and greatly expand research opportunities for neuropsychologists working in all settings, even in solo private practices. For example, with cloud scoring and test-data storage (which must be HIPAA compliant and have appropriate informed consent from participants), the potential exists to create databases of neuropsychological test findings in numerous, homogeneous clinical populations, which can then be provided to researchers regardless of the setting in which they work. One example of this approach is the collaboration that has been formed between the National Neuropsychology Network (NNN; supported by NIH [R01MH118514], Robert M. Bilder, Ph.D., Overall Principal Investigator) and Pearson Assessments. In this program, Pearson has created an application programming interface (API) that connects their portals for digital data collection through Q-Interactive or Q-Global with the NNN database, which in turn prepares these and other data collected at NNN sites and shares these data publicly via the NIMH Data Archive.    As part of this collaboration with the NNN, Pearson created a “Research ID” field within Q-interactive that enables research data to be shared after removal of individually identifying information.  Another example is a new program that has been spearheaded by Lisa W. Drozdick, Ph.D., Principal Research Director at Pearson, in which researchers are given free access and use of forthcoming digital tests (e.g., WAIS-V or DKEFS-2) as soon as they are “locked in” at the beginning of the national standardization study. That is, researchers can begin to use those instruments in their studies while the tests are undergoing national standardization. Then, once the national norming studies have been completed for those tests, the normative data can be provided to the researchers. This research opportunity is part of a program called the Research Engagement Initiative, which is striving to provide neuropsychologists with different research options. One option available now allows neuropsychologists to establish their own private database within a Q-Interactive or Q-Global account, where they collect test data on one or more clinical groups, and they keep those data private for their own use. A future option would allow researchers to establish a shared research data portal, where, with informed consent from participants, the data can be provided to other researchers who have submitted appropriate applications to use those data. An exciting feature about the shared research portal is that even neuropsychologists working in settings without access to homogeneous clinical populations would then be able to conduct sophisticated clinical research studies, including comparative studies across clinical groups and in-depth analyses of the process data provided by the assessment instruments. Researchers working at large medical centers could also benefit from this program by having access to test results from a greater number of different clinical groups. If a researcher using either the private or shared portals needs normative participants matched on selective demographics, Pearson, following review and approval, could provide those data from their national normative samples. This model was followed by the NNN to examine measurement invariance, comparing the factor structures of WAIS-IV, WMS-IV, CVLT-3 and D-KEFS data from the NNN clinical samples with the original standardization sample results (see Loring et al., 2021; Bilder et al., 2022; Reise et al., 2023).  Importantly, even if a researcher elects to use the private research portal but would like access to test results from other clinical groups that are in the shared database, those data could be provided to the researcher.

Helpful LInks/References:

Bilder, R. M., Widaman, K. F., Bauer, R. M., Drane, D., Loring, D. W., Umfleet, L. G., … & Shih, S. (2022). Construct identification in the neuropsychological battery: What are we measuring? Neuropsychology.

Loring, D. W., Bauer, R. M., Cavanagh, L., Drane, D. L., Enriquez, K. D., Reise, S. P., … & NNN Study Group. (2022). Rationale and design of the National Neuropsychology Network. Journal of the International Neuropsychological Society, 28(1), 1-11.

Reise, S. P., Wong, E., Block, J., Widaman, K. F., Gullett, J. M., Bauer, R. M., … & Bilder, R. M. (2023). Computerized adaptive test strategies for the matrix reasoning subtest of the Wechsler Adult Intelligence Scale, (WAIS-IV). Journal of the International Neuropsychological Society, 1-10.

Limitations: There are some administrative and technical hurdles to address to use the methods described above.  Administrative issues include obtaining your local institutional approvals (from institutional review boards and/or compliance teams), and creating an agreement with the test vendor.  Technical issues involve possessing the technology required for test administration (e.g., iPads and/or computers) and knowledge of data organization methods, along with basic computing infrastructure.

Justice, Equity, Diversity, Inclusion Issues: On the positive side, these methods offer hope through aggregating data on diverse samples that information important to determining the effects of social and structural determinants of health on neuropsychological functioning will be improved. Further, if methods are more efficient, there may be greater access to neuropsychological assessments.  On the negative side, the “digital divide” continues to have an impact on minoritized groups, and some individuals may have less familiarity or comfort using technology and this may be confounded with other characteristics.