A Comparison of Observer Presence and Instructed Tasks on Parent Praise Statements
Poster Number
12
Format
Poster Presentation
Faculty Mentor Name
Scott Jensen
Faculty Mentor Department
Psychology
Abstract/Artist Statement
Inter-Observer Agreement (IOA) refers to the correspondence between independent observers when scoring the same behavior. To improve the reliability of an observational measure, observers are trained to consistently discriminate the target behavior. Few studies have systematically evaluated IOA training procedures. A multiplebaseline across-subjects design was used to evaluate the effects of various training procedures on the IOA of undergraduate students scoring parent praise statements. Undergraduate students (N = 4) from University of the Pacific collected data on the occurrences and qualities of parent praise statements from video observations. During each training phase, repeated measures of the IOA scores between each participant and the trained observer were collected. After receiving the operational definition of praise during baseline, participants demonstrated low IOA for praise occurrences (M = 9.2%) and praise qualities (M = 4.0%). After taking a quiz regarding the operational definition of praise, participants demonstrated improvement in IOA for praise occurrences (M = 74.7%) and praise qualities (M = 43.5%). During the feedback phases, IOA remained near its previous level for praise occurrences (M = 73.6%) but slightly improved for praise qualities (M = 63.9%). Research teams that rely on direct observation must establish high IOA before they may begin assessment. A variety of IOA training procedures are available, but no standard practice has been established. The findings of this study are important because they may suggest procedures that reliably produce high IOA, and thus better inform future observer training protocols.
Location
DeRosa University Center, Ballroom
Start Date
20-4-2013 10:00 AM
End Date
20-4-2013 12:00 PM
A Comparison of Observer Presence and Instructed Tasks on Parent Praise Statements
DeRosa University Center, Ballroom
Inter-Observer Agreement (IOA) refers to the correspondence between independent observers when scoring the same behavior. To improve the reliability of an observational measure, observers are trained to consistently discriminate the target behavior. Few studies have systematically evaluated IOA training procedures. A multiplebaseline across-subjects design was used to evaluate the effects of various training procedures on the IOA of undergraduate students scoring parent praise statements. Undergraduate students (N = 4) from University of the Pacific collected data on the occurrences and qualities of parent praise statements from video observations. During each training phase, repeated measures of the IOA scores between each participant and the trained observer were collected. After receiving the operational definition of praise during baseline, participants demonstrated low IOA for praise occurrences (M = 9.2%) and praise qualities (M = 4.0%). After taking a quiz regarding the operational definition of praise, participants demonstrated improvement in IOA for praise occurrences (M = 74.7%) and praise qualities (M = 43.5%). During the feedback phases, IOA remained near its previous level for praise occurrences (M = 73.6%) but slightly improved for praise qualities (M = 63.9%). Research teams that rely on direct observation must establish high IOA before they may begin assessment. A variety of IOA training procedures are available, but no standard practice has been established. The findings of this study are important because they may suggest procedures that reliably produce high IOA, and thus better inform future observer training protocols.