In the aviation safety research domain, the problem of cause identification refers to the task in which we have to identify the possible causes, or shaping factors, responsible for a safety incident. Given incident report narratives, this task of cause identification suffers from a number of challenges including scarcity of labeled data and difficulties in finding relevant portions of the text. In this research we use annotator rationales, which are the text fragments that motivates an annotator to assign a particular label, to overcome both these challenges. We propose several new ways of utilizing the rationales in this problem and show that through judicious use of the rationales, including using them as features as well as generating different kinds of pseudo-instances, it is possible to achieve significant improvement on a supervised SVM baseline.