Can we reliably measure social work communication skills? Development of a scale to measure child and family social work direct practice

Abstract

Few attempts have been made to define and measure the effectiveness of social work communication skills. This paper describes a coding scheme for rating seven dimensions of skilled communication in child and family social work practice and presents an empirical evaluation of whether the dimensions can be coded for reliably. Four dimensions of skill were adapted from the Motivational Interviewing Treatment Integrity (MITI) code. A further three dimensions, primarily related to appropriate use of authority, were developed in consultation with key stakeholders. The seven dimensions were used to score 133 audio recordings of direct practice. Of these, 28 (21%) were scored by three independent raters in order to test inter-rater reliability (IRR). IRR was assessed using Krippendorff's α and Intra-Class Correlation (ICC). Results indicate that it is possible to reliably measure key elements of skilled communication, with Krippendorff's α scores ranging from .461 (good) to .937 (excellent) and ICC ranging from .731 (good) to .967 (excellent). Establishing reliability provides a foundation for exploring the validity of the measure and the relationship between these skills and outcomes, as well as for further research looking at the impact of training, supervision or other methods of professional development on skills in practice. The problems and potential contribution of using such an approach are discussed.