Effects of Presentation of Pretrial Risk Probabilities on Pretrial Release Decisions

As jurisdictions across the United States are using pretrial risk assessments as a component of pretrial reform, one overlooked aspect of risk assessments is risk communication. Risk communication is the examination of how different communication methods (e.g., probabilities, confidence intervals, categories, etc.) effectively relay a defendant’s risk to the community to pretrial decision-makers who decide whether to release or detain a defendant while balancing community safety. Many judicial decisions-makers use risk assessment scores in their pretrial decisions. Accordingly, it is important for risk assessments to effectively communicate what is meant by “risk”. Traditionally, risk assessment tools present fixed probabilities that a defendant assigned a given risk level will engage in misconduct if released pretrial. However, this presentation may overstate the certainty involved in risk judgments. The goal of the current study was to assess whether presentation of risk assessment information as fixed or variable probabilities affects judges’ and pretrial officers’ pretrial decisions, and prosecutors’ and defense attorneys’ pretrial recommendations. We issued online vignettes to 298 United States judges, pretrial officers, prosecutors, and defense attorneys with random assignment to a vignette with a one-value probability or a range of probabilities to accompany the risk level. We quantitatively assessed differences in decisions to order unconditional release and we asked pretrial actors to explain the rationale behind their decisions. This mixed-methods approach provided much needed quantitative and contextual data on pretrial judicial decision-making. Findings may inform changes to risk communication methods to best communicate risk in current and future risk assessment tool development. Findings indicate that variable risk presentation did not affect risk appraisal nor pretrial release decisions. Despite risk appraisal being the second most frequently cited theme in decision-making, it did not affect decision-making. Our findings beg the question of whether court actors heavily consider risk assessment values in their decision-making.

My name is Ashley Rodriguez, and my project is the Effects of Presentation of Pretrial Risk Probabilities on Pretrial Release Decisions. I conducted this project under the supervision of Dr. Evan Lowder.
Risk assessments are being widely implemented across the U.S. as a tool to help decision-makers standardize a defendant’s risk to the community relative to other defendants. This is because numerous studies have found that judges make inherently biased decisions. Risk assessment tools remove some of the bias in decision-making by ensuring that defendants with the same offense and background receive the same decision. Thus far, research on how risk assessment information is presented is often overlooked. There are only two studies that examine risk assessment presentation in the judicial decision-making setting, but both studies suffer from a lack of statistical power. Studies on forensic clinicians demonstrate that risk communication is important for decision-making.
The purpose of this study was two-fold. First, I wanted to examine whether fixed or variable presentation of risk probabilities affected pretrial decision-making along a variety of points such as the decision to release, and the likelihood of release. The second purpose was to examine whether the effect of risk presentation was moderated by whether the offense was violent or nonviolent.
Participants were surveyed via Qualtrics. They were randomly assigned to one of eight vignettes based on a 2 X 4 design of the two independent variables. Following the vignettes, all participants answered the same questionnaire. I recruited participants via snowball sampling. I used publicly available contact information to email government offices and professional associations. I requested that they forward the email to people they knew who would be eligible to participate in this survey.
My sample consisted of 298 participants. The average participant age was 45 years old. Approximately 50.6% of the participants were female, and 78.5% were White. 52% of the participants indicated that they encountered risk assessment information daily. The average participant had 16.3 years of experience in the criminal justice system and 11.2 years of experience handling pretrial cases. Most of the participants were defense attorneys, followed by pretrial officers, prosecutors, judges, other, and unknown role participants.
I used two independent variables to create a 2 x 4 design. First, I varied vignettes by risk probability. Vignettes had either a 30%, 40%, 50%, or 30-50% risk value. I then varied vignettes by whether the offense was violent or nonviolent. The combination of these two independent variables led to eight vignettes.
My first dependent variable examined whether participants would release the defendant on their own recognizance or on bond. My second dependent variable examined the likelihood the participants would release the defendant on a scale of 1% to 100%.
The first research question examined how risk assessment value and offense type interacted to affect release decisions. To answer this question, I conducted logistic regression which contained interaction terms between risk value and offense type. I found no significant evidence that interactions between risk values and offense types affected release decisions.
The second main question asked participants to rate the likelihood they would release the defendant with additional conditions. I used linear regression to examine this question, with interaction terms between risk value and offense type. Similar to the first question, there was no significant evidence that risk values interacted with offense type on the likelihood of release with conditions.
Preliminary qualitative results indicate that the biggest factors in decision-making were case facts, risk appraisal, community stability, and criminal history. In other words, legal factors appeared to matter more to decision-makers than risk assessment values. Interestingly, considerable portions of each criminal justice role considered risk appraisal in their decision-making. However, the actual risk values were infrequently referenced in the open response questions pertaining to the participants’ decision-making rationale.
The purpose of this study was to examine how risk assessment presentation affected pretrial release decisions. My findings did not support any of my hypotheses. This study demonstrates that risk presentation did not have a significant effect on decision-making even when I manipulated the seriousness of the offense. These results were somewhat contradicted in qualitative findings. Despite risk appraisal being the second most frequently cited theme in decision-making, it did not affect decision-making. I expected an interaction between risk value and offense seriousness because people who commit more serious offenses are perceived to be more of a risk to the community than people who commit less serious offenses. My findings beg the question of whether court actors heavily consider risk assessment values in their decision-making.
It is important to note that despite adding in various manipulation check questions, my study has a few limitations. One limitation of my study is that only willing contacts connected the email invitation to participants which may have biased results. Another limitation of this study is that my analysis may have been too underpowered to detect small effects.
Future studies using a similar design should consider varying risk assessment presentation with the defendant’s case facts.
Here are my references.
I would like to thank OSCAR Undergraduate Research Scholars Program for funding this project. I would also like to thank Dr. Sarah Desmarais, Dr. Samantha Zottola, Troy Hatfield, and Honorable Mark Spitzer for providing feedback on vignette drafts. Finally, I would like to thank Dr. Allison Redlich and Samantha Luna for providing example IRB protocols.
Here is my contact information for any questions. Thank you for listening to my presentation.