The importance of driver safety is a significant issue. As life expectancy increases and there are medical advances improving recovery from illness, there are more people requiring reassessment of their driving skill [1]. The driver safety issue in the elderly demographic has been reviewed at length in the literature with older age causing decline in cognition that impacts on driving performance, however far less than perceived in the wider community [2]. Driving has been linked with feelings of normality, a marker of recovery from disability or illness and is often considered critical to someone’s own identity [3],[4]. Driving is an important consideration post-illness and during the ageing process. In view of this information, there has been a recent emphasis on the assessment of individual driver capacity.

In Queensland, Australia, medical fitness to drive is ultimately determined by the treating medical team. There are national guidelines of minimal medical standards required to drive a vehicle within Australia [5] in which the medical officer utilizes to aid in this decision making. Challenges arise for those clients who meet these medical standards, however may have a functional concern that impacts on their ability to drive. In these situations, occupational therapists (OTs), particularly those with specialized driver training, often provide assessment and opinion to medical officers to aid with the decision-making process.

It is well recognized that the “gold standard” for driving review post-illness or during the ageing process is via a driving-focused off-road screening process, as well as a practical behind the wheel on-road assessment [6]. In Australia, these real-world, driving assessments are typically completed within the field of occupational therapy, to comprehensively examine the impact of disability, ageing or medical event on driving performance and overall driving safety [7]. Local processes and reporting requirements regarding driving assessment vary within the different Australian States and Territories, although the actual targeted driving assessment process has some standardized components and consistency within occupational therapy around the country.

The on-road assessment is conducted with the client, qualified driving instructor and OT within a dual controlled car [7]. This on-road assessment uses a standardized route with consistent road conditions such as a set number of roundabouts, dual carriage ways, traffic lights, shopping center parking, and transiting highways if deemed appropriate. There is always a mid-way feedback session and a consensus that two critical errors result in an end of the on-road assessment. The off-road driving assessment component involves a review of a number of components including driving history, driving needs and goals, medical history and medications, visual screen (through a government subsided rebated eye test through an optometrist), sensory-motor function (including tests of coordination, active range of motion, strength, kinesthesia, and sensation), cognitive-behavioral function (including DriveSafe DriveAware [8], Montreal Cognitive Assessment [9], and Useful Field of View [10]), and basic road law knowledge [11].

There are obvious and significant pragmatic difficulties with resource management and costs associated with this “gold standard” process to determine fitness to drive. There are limited resources to offer these highly specialized comprehensive service and services that do, often have significant client associated costs. The spread of the population across metropolitan, rural, and regional areas also poses a problem with most OT driving services located in metro areas, usually in tertiary hospital settings or private practice. As such, there has been interest in the development of off-road assessment screening processes to better predict on-road driving performance to either eliminate the need the comprehensive, time consuming and costly driving assessment and/or assist with the prioritization and timing on these assessments [11],[12],[13]. Despite previous research attempts, to date, there has been no available off-road screening processes that has been sufficiently robust to fully predict on-road performance and therefore eliminate the need for a practical on-road driving assessment [13],[14]. As such, research is now focusing on how these screening processes give “insight” into the potential on-road performance to help with decision making, timely referral, and best use of scare driving specific resources [15].

One of these screening tests of particular interest is the TMT B, a widely available neuropsychology screening tool. The TMT B has been utilized in the driving field as it screens cognitive skills including speed of attention, sequencing, mental flexibility, visual scanning, and motor function, similar to those cognitive skills being needed during the driving process [1],[16],[17]. Trails Making Test B was originally developed in 1944 for use by the United States Military to examine the cognitive performance of new recruits [18]. During the 1990s, the tool started to be considered to determine the relationship with brain injury and driving [14] and further research in 2004 [19] suggested a potential relevance in assessment of driver capacity and safety. More recently, Roy and Molnar [20] specifically investigated TMT B cut off score in reference to driving ability, suggesting that a “rule of three and three” as a cut off for driving safety. This “rule” outlined that the completion of TMT B in 3 minutes or less, with three errors or less, would give an indication of a person’s cognitive ability to return to drive safely. Further research in 2015 [21] suggests that an inability to complete the TMT B in a timeframe of less than 108 seconds may differentiate safe and unsafe drivers and recommend if further on-road testing may be required.

Research context

The local state funded health service operated an on-road driving assessment service within its metropolitan community rehabilitation services in Brisbane until 2019. This OT-based on-road driving service conducted the “gold standard” on-road driving assessment for a wide variety of clients within the community, with a wide variety of diagnosis and ageing adults with cognitive decline. This service utilized the TMT B as part of the off-road assessment battery.

The aim of this study was to explore the trends observed between TMT B and the outcomes of the on-road driving assessment within a wide range of diagnostic categories within this service. In particular, the specific objectives were to:

• Contribute to the existing research regarding the relationship of the TMT B and on-road driving performance.
• Provide clinicians with additional information regarding the use of this screening tool to guide recommendations as to the likely functional impact of their conditions in relation to driving. This may also influence the need and timing of the “gold standard” on-road assessment and also potentially assist in providing further screening information in situations where on-road assessments are unavailable or impractical to be conducted.

Overview of trial design

This research utilized a retrospect review of client data within a community health service over a nine-year period (2010–2019), comparing the results from the TMT B scores and the final outcome of driving safety after a medical event, ageing or disability.

Given the retrospective design of the study, attempts were made to minimize the effect of confounding variables such as:

• All participants included in the sample met a strict inclusion criteria, i.e.
  • medical clearance to undertake the assessment,
  • had a goal to drive, o had a stated disease or illness which was classified within clearly defined internal coding system,
  • completed the whole assessment including off-road screening and on-road driving assessment
• Differences between participants such as gender and age were analyzed.
• All driving OTs had completed their required accredited driver training assessment. The same driving instructor was used on all on-road driving assessments. Participants were marked in all assessments using the same criteria for both off-road and on-road components. The on-road component had a standardized set route. Adjustments were made for traffic and weather conditions in the assessment as per the guidelines within the OT driving assessment training.

Ethics

Ethics approval was sourced through the Metro South Human Research Ethics Committee (EC00167) Centres for Health Research with a HREC Reference number: LNR/2019/QMS/49294. The ethics approval was for a research project of low or negligible risk.

Context and timeline

The community rehabilitation setting is situated in an urban setting in Brisbane, Queensland, Australia. The service provided funding for OT driving assessments once a week, with some limited Allied Health Assistant support for service coordination tasks and client bookings. Clients within the service are from a range of diagnostic groups, including mental health clients.

The comprehensive driving assessments were completed by a total of five OTs over the nine-year period, with the majority completed by two main therapists. All OTs involved in the data collection met OT registration requirements and have completed appropriate post-graduate specialized training at the University of Sydney.

The research used a retrospect review of the data over a nine-year period with data collected from 2010 to 2019.

Participants

Participants in the research were all clients referred to the service between 2010 and 2019, who undertook both the TMT B and an on-road driving assessment within the service. All participants came from a wide variety of diagnostic groups determined by a local coding system and were allocated by the treating OT. As such, all participants were from wide range of ages.

Clients were excluded from the research if they did not participate in both the TMT B and an on-road driving assessment due to individual client circumstances, such as self-ceasing driving, financial constraints, health concerns or medical clearance revoked by the client’s treating medical team.

There were also operational limitations to the number of driving assessments completed in this timeframe, such as staff leave and availability, driving instructor availability, and service management prioritization.

Driving assessment process

All participants were taken for a formal practical on-road assessment with an OT and qualified driving instructor in a dual controlled car, on a set route. This was completed in a dual controlled vehicle with the capacity to accommodate vehicle modifications if required.

A pre-determined set route with a variety of road conditions were included. This was based on competency standards [7] and were applied consistently by the OT according to set clinical standards and protocols used by Queensland Health locally in the community rehabilitation service.

The OT recorded driving behaviors, skills, and road conditions while the qualified driving instructor maintained vehicle safety and provided instruction and directions of travel.

At the conclusion of the assessment, the clients were allocated to one of two outcome groups following their on-road driving assessment. The two outcome groups were determined after the on-road assessment was completed and defined as the following:

1. Pass:
   1. Medically safe to drive with no driving restrictions.Medically safe to drive with license conditions applied.
   2. This may occur after the initial OT on-road driving assessment, after completion of a remediation program and followed by an OT on-road reassessment to deem the remediation program a success (i.e., now demonstrating safe driving skills).
2. Fail: a combination of the results of the off-road assessment and observations from the on-road assessment indicates unsafe driving. This may occur on:
   1. The first on-road assessment with the OT, with a clear inability to improve performance and therefore inappropriate for remediation/rehabilitation.
   2. Alternatively, the client may progress through a remediation program and be unsuccessful in demonstrating driving safety after the  completion of this process and a second OT on-road assessment.

These groups were determined by clinical reasoning of the OT in conjunction with the qualified driving instructor.

A pass/fail outcome is based on:

1. Number and type of “critical incidents” are defined as intervention of the driving instructor on the steering wheel or foot pedals for safety reasons.
2. Safe driving performance as per a QLD Transport on-road state-based licensing test (https://www.qld.gov.au/transport/licensing/getting/practical-tests).

Secondary to the above is whether the driver is deemed appropriate for remediation. This is when the OT will consider cognition, learning and insight, but a pass/fail result is purely based on critical incidents and meeting QLD Transport criteria.

Off-road assessment tool

The Trail Making Test is a cognitive assessment of speed of attention, sequencing, mental flexibility, visual scanning, and motor function [17]. Although the Trail Making Test has two parts (A and B), TMT B was utilized solely for the research purposes as it requires more complex cognitive ability akin to on-road driving performance [22]. This part of the assessment includes 25 encircled numbers and letters, and the client is required to draw a line connecting these encircled numbers and letters in alternating order (e.g., 1A, 2B, 3C, 4D, and so on). It requires the client to complete the connecting line as quickly as possible and time is taken in whole seconds [17],[23]. The number of errors were not specifically recorded as part of the assessment result, however the additional time taken to correct error was accounted for in the final time score. The version of the TMT B used in the research was sourced from the University of South Dakota Internet Psychology Lab. Participants were provided with verbal instructions as per the instructions outlined in the Compendium of Neuropsychological Tests [17]. All participants were shown the practice page before commencing the test. If the client could not successfully complete the practice page, the test was abandoned at this stage.

Data analysis

Post hoc power calculations of the total group results indicated that minimum total sample size of 40 (G* Power: alpha = 0.05, power = 0.8) was required.

Descriptive statistics were reported as mean and standard deviation (SD) for normally distributed continuous data, median and interquartile range (IQR) for non-normally distributed data, and frequencies and percentages for categorical data.

The population was reviewed and results for the TMT B and outcome groups were analyzed. The distribution was not normal; therefore a Mann–Whitney test was performed. The Mann–Whitney test is used to compare differences between two independent groups when the dependent variable is either ordinal or continuous, but not normally distributed [24]. The Mann–Whitney test was used for the analysis of potential age difference between pass and fail on road driving test. The Fisher Test is utilized to determine if a statistically significant relationship is present between two categorical variables and was used in this study for a gender comparison analysis [25]. All statistical analyses were carried out using the R statistical software [26] and p-values < 0.05 were considered statistically significant.

Logistic regression was performed to explore if the TMT B assessment tool could confidently predict the on-road driving performance. Logistic regression is a statistical method used for predicting binary outcomes (e.g., yes/no, success/fail), focused on one of more independent variables. Logistic regression models “relationships” between one set variable and the other independent variable [27]. Logistic regression outcomes were reported at the TMT B time cut off scores of 100, 120, and 180 seconds when age was held constant at the median age of the total group, median age of the pass group and median age of the fail group. All p values were two sided, and statistical significance was set at = 0.05.

The specificity and sensitivity statistical analysis was completed to determine the ability of the test to correctly identify clients who will pass an on-road driving test, and who will not. These were specifically examined at the cut off points of the TMT B of 100, 120, and 180 seconds. These cut off points were chosen based on previous research cut off points of around 100 and 180 seconds [20],[21] and trends noted within this data set around the 120 seconds mark. This test is important to determine the presence and level of false positives and false negatives, which can be viewed by the positive predictive value (PPV) and the negative predictive value (NPV).

In total, 157 participants were included in the study, from various diagnostic categories, which included 64 participants who failed the on-road driving test, and 93 who passed the on-road driving test. Table 1 provides the sample demographic details of all participants.

Table 2 outlines the descriptive statistics, including median scores, interquartile ranges, and p-values for the sample population. Statistical analysis indicated an overall significant effect between the pass and fail groups.

Results from the Fisher test indicate that gender had no effect between pass and fail groups (p=0.09).

Results of the Man–Whitney test indicate age was significantly different between pass and fail on-road performance groups (p=3.5e^-07).

Furthermore, logistic regression analysis indicated TMT B was found to be statistically significantly related to on-road driving performance within the total group even when adjusted for age (p=0.016). In addition, results indicate that age was also statistically significant (p=0.02), indicating that older patients were more likely to fail.

The predicted probability for the TMT B cut off scores at 180, 120, and 100 seconds, when age was held constant, is represented in Table 3. This table outlines the median age score of the total sample group, pass group, and fail group. At the 120 seconds time score for the TMT B, there was a 71% confidence level of the TMT B accurately predicting on-road driving assessment outcomes for the age of 63 years. This drops to 64% for 69 years and 57% for 75 years. Predictive probability values decrease within age groups, with a slower TMT B completion time of 180 seconds. Conversely the predictive probability score increases to 75% with the faster TMT B completion time of 100 seconds. Figure 1 is a visual representation of the predicted probabilities as mentioned above (Table 3).

The specificity and sensitivity statistical analysis indicated that the NPV was 72% and the PPV was 68% when reviewed at 120 seconds. When the statistical analysis looked at increased cut off times (180 seconds), the NPV improved to 88%, compared to a reduction in the PPV to 49%. Conversely, the cut off time of 100 seconds resulted in the NPV result of 61% with the PPV improving to 72%.

This study examined the relationship between the TMT B tool and on-road driving performance for participant populations with various medical conditions within a wide cross section of ages. It was anticipated that this research would add to the confidence and accuracy for the use of this tool within clinical settings to assist with clinical decision making regarding on-road performance and guide the potential need for further assessment if required. While this research is similar to previous research conducted in the area [11],[12],[13], this study focus was exclusively on the TMT B and the correlation with the “gold standard” outcome; an OT driving on-road assessment to give further exploration of the relationship between the two.

Generally, the outcome of this research indicates that the TMT B test is able to predict on-road performance, particularly those of younger age (under 63 years), with statistical significance noted between the pass and fail scores of the sample group as well as when the group is adjusted for age. In addition, similar to research conducted by Selander et al. [28], the results indicate a significant relationship between age, TMT B score as well as on-road driving performance, with older clients more likely to fail. Clinically, this suggests that the TMT B may be a useful pre-screening tool for clinicians who have clients considering returning to driving, particularly for the younger, disabled cohort. It may also allow therapists to gain a better indication of possible driving performance/safety before taking participants on road, therefore reducing risk to all stakeholders and utilizing scarcely available driving assessment services in a more targeted way. It may also provide earlier predictive recommendations of cessation of driving, reducing resource demands within on-road services.

As with previous research [20], there is an obvious interest in a specific “cut off” score for the various clinical groups to help guide clinicians in their recommendations around driving. The results of the specificity and sensitivity analysis in this research suggests that a 120 second time cut off may be a clinically relevant marker, as both the NPV and PPV score around 70% (NPV is 72% and PPV 68%), which is a clinically accepted level for these scores [29],[30]. This is similar to research presented by Ma’u and Cheung [31] within a dementia diagnosis population. Given these statistics, this proposed cut off time reduces the risk of false positives in using the TMT B to predict on-road performance (i.e., clients would perform well on the TMT B, however would have poor on-road driving performance). Given the potential public safety risk associated with returning to driving post-injury or illness, minimizing the potential false positives (i.e., participants who score well on the TMT B but fail an on-road driving assessment), with use of TMT B as a predictive tool is imperative. Furthermore, the cut off score is within acceptable limits to avoid a false negative (i.e., participants who would pass on on-road driving but perform poorly on the TMT B), but this would negatively affect the participant on an individual and personal level.

Logistic regression scores further support the results presented and recommendations for a cut off score of 120 seconds, particularly for the younger age groups, 63 years and below. Logistic regression aids clinicians by providing a percentage score of confidence in use of the TMT B time scores and the strong association with on-road driving safety. As with specificity and sensitivity, predictive values over 70% would be considered favorable compared to the previous research. Within this research, this target was met within the younger population (<63 years) at 71%. The confidence percentage score for the older population groups of 69 years and 75 years was 64% and 57% respectively, suggesting a lower confidence in the predictability at this cut off time.

Interestingly, this proposed cut off time of 120 seconds differs from previous research by Roy and Molnar [20] who recommended a 180 second TMT B cut off to predict on-road performance in a generic and demographically diverse population. However, if this cut off time was used within this research data set, the PPV would be 49%, suggesting that the previous cut off time score is too lenient and would yield a higher rate of false positives. Conversely, research conducted by Papandonatos et al. 2015 [21] indicated 108 seconds may be the more reliable cut off for safe and unsafe driver, which is more similar to the findings in this study.

The overall outcome of the research suggests that the TMT B is a cost effective and statistically significant tool to assist in the decision making around returning to driving. It helps clinicians in the timing and need for an on-road driving assessment and to potentially assist with decision making in situations where these assessments are not practically available, or funding access is an issue for the participant.

Specifically, the TMT B appears to demonstrate robust psychometric properties to predict on-road performance within the younger population (63 years and younger), at a cut off time of 120 seconds. This cut off time is not as clearly robust within the older population. Despite this, the evidence suggests that the TMT B is still a useful tool to aid clinicians in decision making about returning to driving. Clinicians can consider the risk versus the benefits of the test as a predictive tool given their specific contextual environment and access to on-road driving assessments.

Limitations of this study include a large sample size including a variety of diagnostic areas and age ranges, which facilitated a large sample size for statistical purposes to gain adequate power calculations for comparison. However, the significant participant diversity may also have impacted on the outcomes. The retrospective study design is a limitation but as mentioned previously, known confounders have been considered and controlled to the best of the authors abilities. The assessments were completed by five different OT which may have impacted outcomes. However, all OTs followed the required QLD Health guidelines, university training program curriculum, and OT Australia competency standards. These OTs were also not blinded to the TMT B results when proceeding forward with the OT on-road driving test and ultimately the pass/fail result.

Further research in the area should focus on larger sample population to give further clarity to the links between TMT B time and on-road driving performance, as well as age. This research could also focus on the relationship between TMT B outcome and on-road driving performance within various diagnostic groups, such as neurological conditions and congenital concerns, especially within the younger demographic.