Cybermedicine for the Patient and Physician

We developed a computer-based medical history for patients to take in their homes via the Internet. The history is divided into 24 modules- family history, social history, cardiac history, pulmonary history, and the like. So far as possible, it is designed to model the comprehensive, inclusive, general medical history traditionally taken, when time permits, by a primary care doctor seeing a patient for the first time. It contains 232 primary questions asked of all patients about the presence or absence of medical problems. Of these, 215 have the preformatted mutually exclusive responses "Yes," "No," "Uncertain (Don't Know, Maybe)," "Don't understand," and "I'd rather not answer;" 10 have other sets of multiple choices, one response permitted; five have multiple choices with more than one response permitted, and two have numerical responses. In addition, more than 6000 questions, explanations, suggestions, and recommendations are available for presentation, as determined by the patient's responses and the branching logic of the program. These questions are available to explore in detail medical problems elicited by one or more of the primary questions. If for example, a patient responds with "Yes" to the question about chest pain, the program branches to multiple qualifying questions about characteristics of the pain, such as onset, location, quality, severity, relationship to exertion, and course. Once we had completed the interview in preliminary form, we made it available to members of our medical advisory board for their criticisms and suggestions. We then conducted a formal read-aloud assessment in which 10 volunteer patients read each primary question aloud to an investigator in attendance and offered their understanding and general assessment of the questions. We revised our program based on comments from the advisory board and the patients

We then conducted a test/retest reliability study of the 215 of the 232 primary questions that have the preformatted, allowable response set of "Yes," "No," "Uncertain (Don't know, Maybe)," "Don't understand," and "I'd rather not answer, the 10 questions that have other response sets with one answer permitted, and the 5 questions with more than one response permitted. Email messages were sent via PatientSite (our patients' portal to their electronic medical record) to inform patients of the study and how to sign on to the informed consent form, and for those that had consented to the study to remind them to take interview for the first and then the second time.)

From randomly selected patients of doctors affiliated with Beth Israel Deaconess Medical Center in Boston, 48 patients took the history twice with intervals between sessions ranging from one to 35 days (mean seven days; median five days). When we analyzed the inconsistency between first and second interviews with which the 48 patients responded to each of the primary questions. We found that the 215 questions with response options of "Yes," "No," "Uncertain," "Don't understand," and "I'd rather not answer" had the lowest incidence (6 percent); the 10 other multiple choice questions with one response permitted had a 13 percent incidence, and the five multiple choice questions with more than one response permitted had a 14 percent incidence. Whenever an inconsistency was detected with the repeat interview, the patient was asked to choose when appropriate from four possible reasons. Reasons chosen were: "clicked on the wrong choice" (23 percent), "not sure about the answer" (23 percent), "medical situation changed" (6 percent), and "didn't understand the question (less than 1 percent). With the remaining 47 percent of the inconsistencies, no reason was given.

We then computed the percentage of agreement for each of the primary questions together with Cohen's Kappa Index of Reliability. Of the 215 "Yes," "No," "Uncertain (Don't know, Maybe)," "Don't understand," and "I'd rather not answer" questions, 96 (45 percent) had kappa values greater than .75 (excellent agreement by the criteria of Landis and Koch, and of these, 38 had kappa values of one (perfect agreement); an additional 24 primary questions (12 percent), to which all patients had made identical responses both times (perfect consistency), had no Kappa values. Sixty-eight of these questions (32 percent) had kappa values between .40 and .75 (fair to good agreement); and 26 (13 percent) had kappa values less than .40 (poor agreement). Of the 27 questions with poor kappa values, 15 had percentages of agreement greater than 90 percent, and we deemed these to be sufficiently reliable within their clinical context to remain unrevised. We selected the 12 questions with poor kappa values and percentages of agreement less than 90 percent for rewording. Of the 15 primary questions with varying sets of responses, half had kappa values in the excellent range and half had kappa values in the fair to good range, and we kept these in place unrevised. Fifteen of the primary questions (7 percent) received a "don't understand" response. Although there was but a single "don't understand" response for each of these questions, we were able to isolate seven with which the possibility of confusion seemed to be evident, and we revised these accordingly.

With the first of the two interviews-with a mean of 545 frames presented and a completion time of 45 to 90 minutes (based on an estimated 7 seconds per frame -the volunteers were for the most part favorable in their assessment of the interview when asked a set of 10, 10-point Likert-scale questions.