What is risk appetite and how does it affect testing?

Psychologists have spent quite a lot of time looking at game shows as ideal controlled environments to analyse human decision making under pressure. Mountains of free data, that would be expensive & difficult to replicate in a lab.

One thing that has been noticed in shows like Who Wants to be a Millionaire and Jeopardy is the differing proclivity the genders have to guess. In short, men are more likely to do it, and it’s not isolated, dozens of studies have indicated men are, on average, more likely to take risks of all sorts.[1]

This is not without contention, either on the nature vs nurture split (much scholarly discussion about the importance of testosterone vs social constructs), and more recently suggestions that the effect is exaggerated due to how we define risk-taking (for example, more emphasis on riding bicycles without helmets rather than donating kidneys).[2]

However, what is undoubtedly the case is that, within the realms of recruitment and assessment, for whatever reason, there is a guessing/risk-taking gender gap, and it has consequences.

What does this risk gap look like?

Take the SAT’s (the ‘Scholastic Aptitude Tests’), a supposedly meritocratic screener (a kind of IQ test) used in the USA for college applications. In the mathematics section, there is a persistent attainment gap between the genders. The reasons for this are complex and controversial (take a look at stereotype threat), however, one to consider is how the test rewards risk-taking.

 

 

Up until 2016 the SAT’s would give you 1 mark for a correct and answer and deduct a ¼ of a mark for incorrect answers. In any given question there were 5 possible answers. The system was designed to discourage wild guessing and, at a first glance, it does. The problem is, in reality, you are still rewarded if you do guess.

So, why is this? Well, with 5 possible answers if you can eliminate one , you had a ¼ chance of getting an answer right. If you guessed on 20 questions you would get 5 correct answers and only lose 3.75 marks. You come ahead 1.25 marks.

Superficially discouraging risky behaviour whilst still rewarding it, is a system rigged to reward risk takers. So if we take that as an average and with what we know about risk gap in mind, that is a system rigged against women. In an analysis of 2001 SAT scores, up to 40% of the gender gap can be explained by this tendency to skip questions.

It’s for this very reason that as part of an overall overhaul in 2016 SAT’s no longer penalised wrong answers. This is likely to have had some effect, although, even where there is no overt penalisation, it still doesn’t close the guessing gap entirely. [3]  In our own studies Mapped have found that women tended to skip more questions even without any penalty, and as a result, we have taken unique steps to reverse this trend.

Risk appetite vs ability

To the sceptic, there might still be the question that maybe women skip more questions not down to risk tolerance but instead because they know less. Why do we think this is an indication of risk appetite rather than ability?

On one point accuracy (as opposed to total score) is identical, and secondly when you force everyone to answer questions the gap largely disappears.[4] These results mirrored those according to Iris Bohnet in “multiple choice tests in South Africa, microeconomics tests in the United States, and Hadassah aptitude tests in Israel”.

The implications for business are significant. Multiple choice tests of various stripes are used as the gateways to most of the most prestigious graduate schemes. If you are trying to test analytical ability but are instead testing willingness to guess and risk appetite, not only are you failing to assess the former, you are quite likely encouraging negative behaviours. Moreover, you are shooting yourself in the foot if you are aiming to get a diverse outcome.

Mapped are tackling this through a series of strategic interventions and have halved the gender gap compared to other providers so far. If you are interested to find out more then get in touch to book your demo today.

 

REQUEST A DEMO TO FIND OUT MORE ABOUT THE MAPPED DIFFERENCE

 

[1] https://psmag.com/social-justice/on-jeopardy-women-take-fewer-risks-vs-men-31797

[2] https://www.sciencedaily.com/releases/2017/10/171005102626.htm

[3] https://princetonreview.blog/2017/12/06/the-good-and-bad-news-about-gender-gaps-on-the-psat

[4] Katherine Baldiga, “Gender Differences in Willingness to Guess,” Management Science 60 (2013): 434-447.