Does the method of administration influence the UK GMC patient questionnaire ratings?

AIM To examine whether different methods of administration of a patient questionnaire (PQ) assessing a doctor's professional performance can influence the quality of data collected. BACKGROUND Patient surveys are important information sources regarding a doctor's professional performance. Previous research across a range of patient outcomes suggests that the method of administration can influence response rates and ratings, although no data that explore how this might influence patients' assessments of a doctor's professional performance are available. METHODS A volunteer sample of 13 UK doctors from primary care settings undertook a cross-sectional patient survey. Seven doctors distributed patient surveys using exit and touch-tone telephone administration, and six doctors used exit and postal administration. A consecutive sample of up to 40 patients per doctor completed each method of administration; postal and touch-tone surveys were administered retrospectively, whereas the exit surveys were administered prospectively. The GMC (General Medical Council) PQ included nine performance evaluation questions. We examined the response rates, item completion rates and response profiles of exit survey responses with those obtained from either postal or touch-tone telephone methods. RESULTS The administration methods influence the quality and comparability of data obtained. The exit survey response rate was higher than touch-tone (82.1% versus 37.5%; P < 0.0001) or postal survey (no reminder) (76.7% versus 60.8%; P < 0.0001) administration, but comparable to the postal survey after one reminder (78.3% versus 75.0%; P = 0.39). Item completion rates were comparable for exit and touch-tone surveys (highest rates of missing data 4.1% and 4.6%, respectively), but exit surveys resulted in more missing values than postal surveys (10.3 to 11.7% versus 1.1 to 3.9%). Response profiles varied. Both touch-tone and postal surveys yielded significantly more critical ratings than the exit survey (three of nine items). CONCLUSION Mixing administration methods requires caution when the data are used to create benchmarks against which a doctor's performance is assessed.