Some noise has been made recently about the results of a survey conducted in the UK to determine how many residents of the country identify as vegans. According to the much publicised conclusions, there are now roughly half a million vegans the UK, around 1% of the population, up from 350,000 ten years ago.
Vegans and animal rights advocates may look upon these numbers with delight, but whenever presented with numbers, its good and healthy to ask some questions about it, rather than taking it all at face value.
We have seen some people get a bit confused with the results, so we thought we would explain a bit of the methods behind surveying, and hopefully help make sense of what is being widely heralded as proof of the inexorable rise of veganism.
Where did these graphs come from and what do they mean?
Surveys and statistics are common in the news media, but it is not always obvious how numbers were obtained, why the purpose of the survey was, who paid for the survey and what we should make of the results that we are now presented with.
The results come from surveys which are usually conducted over the phone: whereby a commercial company is hired by an organisation to contact a portion of a specific population in order to answer a specific question.
The population in question can be either the entire population of, for example, a country, or it can be a specific group within an entire population, for example people over the age of 60.
When surveying the chosen population, the biggest obstacle is the size of the population. Surveys take time to complete, produce a potentially huge amount of data, and are not mandatory, so completing any survey will require a lot of time and work to ask the questions, make sense of the results, and perhaps most importantly, find people willing to give up their time to answer the questions, even though doing so is unlikely to benefit them directly.
The first, and most important, way to work around this obstacle is by sampling. Sampling means that instead of surveying the whole population, you survey a portion of the population which you assume to be representative of the population as a whole.
Once you have the results for the sample, you scale them up and use them to describe the whole population.
For example, if you wanted to determine how many people in believe the earth is flat, you could call and ask 10000 people the question. If 100 people said yes, they do believe that, then that represents 1% of the sample. Assuming this is representative of all Norwegians, it could then be claimed “1% of Norwegians believe the earth is flat”.
The total population of Norway is around 5 million, and 1% of that is 50000, a larger number than people who were surveyed. This means that we found 100 people who think the world is flat, and by using simple maths we assume that this allows us to predict that there are 50000 such people in Norway.
This is how the UK vegan survey is able to predict half a million vegans in the UK despite only having a sample size of 10,000 people.
The data we don’t see
There are three main ways to avoid data you don't want getting to the public.
The first is by only including people in the survey you expect to give the "right" answers. To use the flat earth example from earlier, if when choosing a sample the members list of the flat earth society is used to select names of people to question then it could be reasonably assumed that you might get a higher response of people believing the earth is flat than if that membership list wasn't included. This is an issue of how representative a sample is, and often a single survey is not enough to make a concrete judgement on an issue, and it is only when other surveys have been conducted with different samples sizes and levels of representation, that you can begin to form a conclusion based on the aggregate of results (this kind of thing is called a meta-analysis)
The second is by choosing questions and the way they are asked so as to guide towards a certain result. Concepts such as priming or loaded questions are applicable here. How people respond to seemingly simple questions is governed by complex psychology and there are many people who study just that. One example of this is that people are more likely to respond in the positive to a given question, and therefore during the Scottish independence referendum, there was some debate over whether the question asked should be "Should Scotland be independent?" or "Should Scotland remain in Great Britain?" as our inherent bias towards responding with a "yes" skews the responses in favour of one answer or another.
The third is selective reporting of results. When a company pays for a survey, they have no obligation to make the questions of the survey or the raw data publicly available. This differs from a scientific study where details about the methods and results are expected to be meticulously detailed. This means that we don't know exactly what questions were used. This is an issue of transparency, and most people discover the results of a survey via mainstream media, once the polling organisation has chosen the data it wants to share, formatted it into a press release, and distributed it to media outlets, where the person who writes about the results may or may not have an understanding of the topic at hand or how research is conducted or should be interpreted.
Veganism in the UK & beyond: what do we do now?
Of course we think it is massively important to have an understanding of the number of people who are vegan, and the trends of its growth over time. First of all, knowledge for its own sake is a good thing: vegans and non-vegans alike tend to feel as if veganism is growing: more vegan restaurants are springing up, we tend to meet more vegans in every day life, and we hear more about vegans and vegan issues in the media. But how much of that is "selective attention" (that is, now we are vegan, we tend to spend more time with other vegans, searching out vegan restaurants etc.) and how much is the result of actual growth of the the numbers of vegans out there?
It is also important for representation and recognition by government. Government has a responsibility to represent the population it governs, and if the beliefs of that population are changing, it stands to reason that their voice should be heard, and they should be taken seriously as a group. This is perhaps even more important for the vegan movement, as it does not have the kind of financial backing that, for example, the meat and dairy industries do. The money from these industries goes towards lobbying government, positive representation in the media, and generally furthering their own interests. The vegan community does not have the same financial advantage as those industries, so in order to compete for attention, it may be necessary to emphasise the growing number of people who are vegans.
What we need to keep in mind is this survey and most like it are not scientific in either their methods or the presentation of results. Science is of course still subject to biases, errors and later revision, but the general consensus is that efforts should be made to find a more accurate description of "the way things are", rather than "the way we would like things to be".
It is important to get numbers as accurate as possible from the stand point of animal advocates so we can best assess how effective we are. We will naturally be happy to hear that veganism is on the rise, as it suggests our efforts are successful, however we need to know where we are successful, how successful we are, and where we fail. Accurate numbers will help us do better forms of activism, spend our time more effectively, and give us an accurate picture of reality rather than a picture that validates our preexisting biases.
We want to do work which maximises the benefit to animals, rather than our own egos, an accurate statistics will help with that.
If you want to learn more about data interpretation, statistics, and effective activism, we have provided some links below for further reading.
Survey Monkey Blog - Survey Monkey is an online company which allows you create your own online surveys. Their blog covers a range of issues from data collection techniques and the psychology of surveys, to results surveys of current interest and social media deployment of surveys and their results.
Effective Altruism - If you have ever wondered which charity is the best to donate to to promote the things you believe in,then this is for you. They encourage the use of evidence to guide activism, charity donations and altruistic actions.
A New Kind of Wrong - A nice article by doctor, epidemiologist and writer Ben Goldacre in which he looks at the media representation of the results of a survey and explore the problems, both with the survey and how it was interpreted.