Validating the Military Service Identification Tool in Its Ability to Correctly Identify Civilians and Those Who Have Served in the Military

Currently, there is no military service marker in electronic healthcare records to identify who is a veteran in England and Wales [3]. This makes it difficult to evaluate the unique healthcare needs of those who have served in the UK military. In a previous study, the investigators set out to investigate whether it was feasible to identify veterans who accessed secondary mental healthcare services via their anonymized electronic healthcare records using a manual, text search based, method. To do this, the investigators used the South London and Maudsley (SLaM) Biomedical Research Centre Case Register. This is a novel data resource, derived directly from the routine electronic healthcare records of the SLaM NHS Foundation Trust. SLaM is one of Europe's largest mental health providers, serving over 1.2 million residents in four South London boroughs. This Case Register holds patient's electronic healthcare records for all secondary mental healthcare provisions within SLaM. All these electronic healthcare records have been anonymized and a system has been developed to enable researchers to access and search through these records called the clinical record interactive system (CRIS).

As part of the first study the investigators developed a manual, text search based, method using commonly used terms and phrases found in the free text clinical notes to identify veterans. Running this manual method resulted in the identification of n = 6,039 potential veteran records, of which n = 1,600 were selected to scrutinize in more detail, whilst considering time and manpower restrictions. This resulted in n = 693 veteran records, suggesting an identification rate of 43% when using the manual method. Therefore, it was concluded that it was feasible to identify veterans in electronic healthcare records, but time consuming. Each potential veteran record was manually verified by the research team by reading through each patient's notes. This took on average 11 minutes per record.

To address the time it took to identify veterans and improve sensitivity and reach, he investigators developed the Military Service Identification Tool (MSIT), and it proved to be quicker, more accurate and cheaper than the manual method. The Tool was designed to detect military service, not just veteran status. 1the investigators took a systematic approach to developing and testing the MSIT. Then a different subsets of all the electronic healthcare records available to us, to ensure the MSIT was developed and trained on a different set of electronic healthcare records (named the training dataset). Subsequently the MSIT was verified on another subset of the data (named the gold standard dataset) to ensure it would still be able to identify veterans if a different set of electronic healthcare records would be used.

The investigators managed to identify 2,922 veterans in the electronic healthcare records when applying MSIT to inspect 150,000 individual records. The MSIT took only 20 minutes to go through all these records and had an identification rate of 88% when compared to the manual approach. The investigators also matched these veterans to 2,922 non-veterans based on age and gender to compare their mental health treatment pathways.

Despite the success of our initial developments, the investigators have faced a significant barrier that may prevent the widespread, dissemination, roll-out and implementation of the MSIT. Various academics and stakeholders (e.g. NHS England, Cobseo) are interested in the MSIT. However, it has been highlighted an important limitation, namely it is still unclear whether the veterans identified by the MSIT are 'actual' veterans. MSIT identifies veterans based on the notes provided by the clinician in the electronic healthcare record. It is also important to understand if higher rates of physical and mental health are represented in a veteran population compared to civilians. By including both samples - civilian and veteran - this ensures that fair comparisons are able to be made. This There is currently no opportunity to verify their veteran status. For example, no service number is provided in their medical records.