ICAT Validation Study Published

The Internet Based Cognitive Assessment Tool (ICAT) (Hafiz et al., 2019) is a web-based self-administered objective screening tool for remote testing of cognitive functioning. ICAT is implemented as a web application using CARP as the study manager (incl. participant management) and for data storage. ICAT is an implementation of the paper-based Screen for Cognitive Impairment in Psychiatry (SCIP) assessment approach (Miskowiak et al., 2018) and includes real-time assessment of verbal learning and recall using Automatic Speech Recognition (ASR) technology.

A recent study (Miskowiak et al., 2021) has been investigating the validity of ICAT, i.e., whether ICAT is able to perform cognitive assessment in patients with Bipolar Disorder (BD) correctly as compared to the “golden standard” (the SCIP test). This was done by investigating (i) the sensitivity of ICAT to assess cognitive impairments, and (ii) the concurrent validity of the ICAT as compared to SCIP.

Figure 1 – Scatterplot of the correlation between the ICAT and SCIP total scores across the entire sample (r = .72). Patients in the Bipolar Disorder (BD) group are represented by red dots. Participants in the Healthy Control (HC) group are represented by blue dots.

Across the entire sample, we found a strong positive correlation between the total scores of the ICAT and SCIP (r(61)=.72, p<.001). This is shown in Figure 1. Analyses also revealed moderate to strong positive correlations between performance on individual ICAT subtests and their corresponding SCIP subtests.

We also investigated the usability of ICAT and found that participants reported high usability of the ICAT system as measured by their perceived overall usability on the PSSUQ scale (M=4.1, SD=0.5 of max 5).

In conclusion, ICAT showed adequate concurrent validity, suggesting that this tool can be used for remote, self-administered assessments of objective cognitive functioning in patients with BD. The ICAT was even more sensitive to cognitive impairments than the SCIP and may be more sensitive in capturing perceived real-life functional and cognitive difficulties. The use of real-time ASR for assessment of verbal memory, as well as the close resemblance to a standard neuropsychological screening tool, represents important advantages of the ICAT system as compared to existing web-based cognitive screening tools.