IAM survey fatally flawed, study implies

Follow-up survey heaps doubt upon doubt on last month’s press release

Cyclists say “survey data should not be wilfully misused by any charity”

A big thank you to everyone who filled out the follow-up survey after the IAM’s spectacle last month.

There were hundreds of responses, and I’m pleased to say that I feel the data I can now present to you is probably not significantly less valid than the IAM ‘research’, although it shares many of the same caveats. Anyone who’d like to discuss the specifics of what I’ve done is welcome to get in touch.

The following indicative findings may make for interesting reading:

Who responded?

Pleasingly, this sample is largely made up of regular, experienced cyclists, with a massive 94% cycling every day, or at least more often than not. Just 1% of the sample were “not a cyclist”.

Do they jump red lights?

Since this was the focus of the IAM’s spin, let’s address this right up front. 97% of cyclists do not frequently run red lights!

This survey completely failed to support the conclusions of the IAM survey.

The real issues for vulnerable road users

A large proportion of the survey was focused on asking what vulnerable road users feel endangers them on Britain’s roads.

The following proportion of the sample felt that the below activities posed a danger to them as vulnerable road users:

  • 98% – speeding motorists
  • 98% – motorists using laptops or mobile phones
  • 98% – motorists ignoring bus/bike lanes, ASLs or parking/waiting restrictions
  • 97% – motorists jumping red or amber lights

Hopefully this sends a clear message to the IAM (and other interested parties) that negligent, reckless and dangerous behaviour by motorists must be addressed as a priority before fiddling around with diversions like cyclists jumping lights.

To reinforce this, the survey also asked what cyclists found to be the most significant issues to their safety. The proportions significantly impacted were as follows:

  • 89% – aggressive / wilfully dangerous driving
  • 88% – driving while using a mobile phone
  • 88% – drivers partly overtaking and immediately turning left
  • 85% – drivers travelling at excessive speed
  • 83% – drivers failing to observe lane markings, parking restrictions and ASLs
  • 80% – oncoming drivers turning right
  • 77% – HGV drivers: stopping so close to ASLs that the cyclists occupying them are no longer visible through the windscreen
  • 74% – HGV drivers: fitting blind spot mirrors and checking their nearside is clear before turning left

In contrast, approximately 3/4 of cyclists feel red light jumping by other cyclists is a non-issue, while around 4/5 feel that pavement cycling is a non-issue.

It’s hard to interpret results like these as a call for anything other than robust action on negligent, reckless and dangerous behaviour by motorists – and quite difficult to justify focusing attention on fringe issues like red light jumping.

How do cyclists rate the performance of the IAM?

Following the widespread condemnation of the IAM last month, the survey also asked the following two questions:

Do you believe road safety groups such as the IAM have a strict responsibility not to distort or misrepresent their website polls to the media?

Do you believe road safety groups such as the IAM have a strict responsibility not to incite hatred against vulnerable road users by ‘spinning’ discredited or unreliable survey data?

94% of respondents said that survey data “should not be wilfully misused by any charity”, and 93% said that “vulnerable road users need to be protected, not victimised”.

If representative of the nation’s cyclists as a whole, this would be a brutal slap to the face of certain individuals and their organisation.

We can at least hope that the future will be brighter, providing that so-called safety groups wake up to what these cyclists identify as the real issues on our streets.

The caveats

I’d encourage anyone with a real interest in the results to commission a professional survey from a company who know how to ask the questions that get the answers you desire in the most defensible way possible.

This survey is not that. Despite having a large number of respondents it suffers from many of the same issues as the IAM poll (plus probably a few more), results are indicative only, and so on and so forth.

3 Comments

  1. Rob

    Dusting off my statistics hat, I’m going to take issue with your concluding statement: “I’d encourage anyone with a real interest in the results to commission a professional survey from a company who know how to ask the questions that get the answers you desire in the most defensible way possible.”

    This is absolutely not the way to conduct research, although I suspect I’m reading slightly more into the wording than you intended. Agree that professional surveys should be non-leading, open and ideally attempt to remove selection bias from their samples (as well as a host of other considerations), but phrasing questions to elicit the ‘desired answer’ invalidates the result. To pick an example from yours ‘Do you believe road safety groups such as the IAM have a strict responsibility not to incite hatred against vulnerable road users by ‘spinning’ discredited or unreliable survey data?’ has to be answered ‘yes’, and therefore doesn’t provide anything interesting as a result.

    My interpretation of both your own and IAM’s results is that we have a reasonable idea of the personal opinions of two polarised groups of individuals. This is of itself exceptionally useful, for example, the fact that a sample of motorists believe that 50% of cyclists occassionally jump red lights is worryingly high. This is not saying that 50% of cyclists actually do (which is where IAM had their foot-in-mouth conclusion moment), but it’s still interesting and worthwhile.

    If someone actually wants to find an accurate estimate of the % that jump red lights I don’t think a personal survey is the right way to do it – it’s too emotive and self-incriminating. I suspect it would need someone to stick some hidden cameras at various lights at various times in various urban/rural conditions and does some counting…

  2. lionfish

    Rob: I think this survey was tongue in cheek: It was a parody on the feeble nature of the IAM survey, and shows how easy it is to get the result you want.

    The really important difference between the two is that one is being reported in the media as fact. The result is people read articles reinforcing their anti-cyclist view, which means they’ll have even less respect for cyclists. When overtaking or manoeuvring a few will give cyclists less space. The practical upshot is probably a few more cyclists will be seriously injured.

    For a charity allegedly interested in road safety, to have put out such a press release is pretty appalling.

  3. Dave

    Hi Rob:

    Thanks for a thoughtful response!

    lionfish has mostly covered it, but yeah, the point of this was really to highlight the deficiencies in the IAM’s “research” methodology – “polls are a game anyone can play” rather than an attempt to do serious research.

    To address your opening point: you’re right that it’s not the way to do *research*, but I was being very deliberate (and very cynical) when I wrote “commission a professional survey from a company who know how to ask the questions that get the answers you desire in the most defensible way possible.” :)

    cheers,

    Dave

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>