How we view our role and relationships within our communities shapes how we behave – . When widespread, these views constitute ‘social norms’, which people tend to follow because they believe that others do and because they think it’s expected of them. Media Action’s programmes aim to challenge social norms that can lead to people risking their health and reinforce those that support people to make healthier choices.

Donors increasingly want proof that we’re changing norms at scale. But it’s not easy either to or to identify exactly how media influences desired outcomes, such as children getting vaccinated or .

These are some of the challenges we face as Media Action’s research team. How do we design research to measure social norms and track whether they’re changing? We came up against this question when evaluating whether our had changed social norms in Ethiopia.

We set out to understand whether our programmes had convinced families to by, for example, saving money and planning how to travel to the hospital. In Ethiopia, preparing for a baby’s arrival typically means planning the important post-birth thanksgiving ceremonies. Women are judged negatively if they don’t put on a good spread but the costs of hosting this celebration mean that there’s less money to get a woman to a facility when the time comes.

We primarily used qualitative methods to understand social norms but we also undertook large-scale surveys to understand how we were shaping social norms at a population level.

To highlight challenges around researching social norms with surveys – and how they might be overcome – here are five common responses heard in the field in Ethiopia: 

1. ‘How would I know what other people think?’

Though intuitive in English, it’s not always obvious in other languages that asking someone ‘what would your neighbour think?’ really means ‘what do you think your neighbour would think?’. Respondents sometimes take the question very literally and reply that they ‘didn’t know the minds of other people’.   

, where interviewees reflect on what they understand by a question, and careful piloting of questions can reveal when something isn’t interpreted as intended. To get the wording of a question right, it’s essential to take the time to test and refine translations into local languages and carefully train field interviewers.

2. ‘Which people?’

When asked about ‘other people in the community’, interviewees often weren’t sure who they were meant to be thinking about. We consequently explained what we meant to respondents, to help them move from an amorphous sense of ‘other people’ to something more meaningful. For example: ‘If you think of five women you know, how many of them would start going for check-ups in the first three months of pregnancy?’  

3. ‘It’s none of my business!’

Our team found people in Amhara (a region in northern Ethiopia where the research took place) to be quite reserved. It’s an area where it seemed to be impolite to speculate on the lives of others, not least to an outsider with a clipboard.

Surveying was as much an art as a science, with proving to be as important as good survey design.

When it comes to rapport the ordering of questions is important. First of all, you can turn people off a survey by leading with potentially sensitive questions. How willing would you be to tell someone what you earn or weigh when you just met them?

Second, it’s easier to get good answers to tricky questions (like those around social norms) if they’re asked some way into the survey after rapport has been built. But these questions can’t be left too late. People often find talking about things like community dynamics tiring, so you have to broach these topics while people still have enough energy to discuss them.

Finally, it’s important to brief interviewers well so they’re confident of the research aims. They’ll also be better at reassuring participants they’re not trying to probe into their private lives, but rather uncover the practices and perceptions of their community as a whole.

4. ’I don’t know’   

Respondents are often stumped by questions – particularly by those that ask to what extent they agree with a given statement. However, presenting them with a range of simple and concrete options can help elicit answers.

For example, to determine whether respondents believe their community sees something as appropriate or expected, we presented them with options such as: ‘they would think it’s OK’, ‘they would think it’s not OK’ and ‘they wouldn’t care’.

To determine what respondents think people in the community are actually doing, we presented them with choices like ‘none or a few do it’ and ‘almost all do it’.

We found that listing these options helped increase understanding and made the surveys less demanding of participants, resulting in fewer ‘don’t knows’.

5. ‘Let me tell you what happened to me during my last pregnancy…’

When a respondent has to choose from a limited number of options, there’s no way of recording the rich detail someone might provide through sharing their experiences in a more narrative form. Within the context of a survey, personal stories become conundrums (how do I classify this?) rather than goldmines of information. This tension demonstrates why qualitative research remains invaluable.

Illustrating this, we recently used a to allow mothers to tell us the journey of their pregnancy. Asking questions like ‘Would you say that your story of pregnancy is the same as most of the mothers in your community?’ helped us understand whether or not interviewees were conforming to norms.

We’re continually exploring ways to improve our surveys to pinpoint what social norms are and how they’re changing, always with the aim of truly reflecting people’s perceptions and helping them improve their health.

Hilina Assefa is a Senior Research Officer with 成人论坛 Media Action’s Ethiopia team; Lois Aspinall is a Research Manager with the UK office.

Related content:

Film:

Blog: 

Blog: 

There's a lack of data on what ordinary people think, feel and want in developing countries. Our new aims to help fix that. Sonia Whitehead runs through five questions the portal can help answer on governance, media and resilience.

The development world is all aflutter about data. There’s much talk of a , the sector’s hiring and the World Bank just launched a to ‘data crunch the world’.  

Not to dampen all this excitement but we need a lot more data about people in the Global South before it can become a transformative force there. Addressing this lack of data will speed up progress on everything from to .  

Enter our new  (view on desktop), which brings together data, reports and visualisations from surveys conducted in 13 developing countries that there aren't a lot of statistics about. Over five years, we asked more than 75,000 (rarely polled) people about what they think, feel and want. The portal covers a range of issues from what they’re most worried about to how interested they are in politics.  

We want these insights to help development leaders, practitioners and researchers better understand ordinary people in the developing world so they can produce more effective strategies, projects and communications.

To mark the launch of the portal, we run through five questions that it can help answer on , and , while also showcasing the different types of content available on the portal.

1. What sources of information do people trust?

Being a media organisation, we wanted to know whether people believe what they hear on the airwaves, see on TV and read online. We found that trust levels in radio are universally high, at over 80% in , and , and reaching 90% in .

However, people are more circumspect about the truthfulness of the internet, with the , which is concerning given that say they go online in order to read the news.

To illustrate these (and other) insights into what media people think of different sources of information, we produced a series of visualisations – some of the ones for Kenya are previewed below (media visual available , governance one ): 

Head of Research Programmes Sonia Whitehead shares her three top takeaways from the first-ever UN World Data Forum.

This week, 2,000 statisticians, researchers and policymakers from across the world gathered in Cape Town for the first-ever . Focused on data’s role in achieving the (SDGs), the conference looked at how the sector can better understand the poorest 20% (P20) of the global population. Delegates puzzled over how to better understand and engage these people in order to uphold the commitment to , which underpins the SDGs.

To do this, the discussions centred on the need for well-communicated, high quality data. The collective view was that – at a minimum – national statistics bureaus should be producing data that breaks down statistics by gender, geography, age and disability. But what more needs to be done to improve our understanding of the P20 to inform policy decisions?

There is an abundance of data being collected today by companies seeking to map consumer preferences. The rise in social media usage is also helping improve our understanding of people, supplementing traditional sources of data. But does this data support development? And how can we improve the communication of data to ensure that everyone engages with and understands it?

Along these themes, there were three points discussed at this conference that gave me particular food for thought.

1) Data guardians are changing

Anne Jellema () talked about how new players are collecting and understanding the kind of data that used to be monopolised by the state. National statistics bureaus have traditionally collected data from censuses and held administrative data on healthcare, education and employment – among much else. Governments have long been held accountable for how they use this data.

Technology, however, has shifted this state of affairs. Skype, Google Maps and Uber collect colossal amounts of user data, which doesn’t belong to the public and which their parent companies aren’t as accountable for as governments are.

This growing abundance of data could lead to a situation where data is increasingly in the hands of the few, seeking to utilise it for commercial rather than civic aims. Jellema raised this as a challenge that the data revolution needs to address in order to improve transparency, accountability and the accessibility of data.

Fortunately, we are already seeing how the proliferation of data can also empower citizens. In China, for example, the allows users to check the government’s real-time pollution data on factories near them, equipping them with the targeted information they need to push for better environmental practices locally. More than 1,800 factories are said to have taken steps to reduce pollution due to public pressure facilitated by the app, which enables users to tag violations of pollution standards on social media.

2) It's possible to collect data on people’s views – without asking one question

Collecting data on people’s views and perceptions has long been the preserve of researchers posing questions to respondents sampled in a systematic way. But there are now newer, faster ways of understanding views at scale, which don’t require a survey.  

At the time of the Zika outbreak, to analyse social media content to inform the development of a digital campaign for Brazilians. Around 100 million people in the country use Facebook and Zika was a popular topic of discussion there. Anonymised Facebook posts were analysed to reveal that 58% of posts on Zika were by men.

Equipped with these insights, UNICEF went on to produce ads featuring a man kissing his daughter born with microcephaly, in a concerted effort to engage men around its Zika campaign. The analysis also showed that people didn’t want to just hear about Zika but also wanted to know find out more about other mosquito-borne diseases.

This example highlights that in an emergency where a high proportion of the population use a particular social media platform, content analysis of posts is a good way of providing audience insights quickly. 成人论坛 Media Action also used Facebook to run short online surveys in Nepal to inform , but the Zika example shows that data on what people are concerned about can be collected without posing a single question.  

3) People don’t know Global Development 101

Has the proportion of the world’s population living in extreme poverty changed over the last twenty years? Presented with the options of ‘almost doubled’, ‘remained more or less the same’ and ‘almost halved’, select the final of the three, which is in fact the correct one.

This pessimism about the true state of global development is not limited to the US; and got the right answer. Similar questions were posed to conference delegates by Ola Rosling (). And we were similarly downbeat about how many children are vaccinated against measles and how many years women have spent in the formal education sector. Even a bunch of statisticians fully versed in development didn’t get the answers right.  

Through its , Gapminder seeks to tackle these and other misconceptions about international development. The project is driven by a concern that the ‘actual facts’ about global progress aren’t getting through the clutter of people’s preconceptions, their tendency to generalise based their own experiences and the (likely outdated) facts they vaguely remembered having learned once in school.

Data clearly needs to be conveyed in a more engaging way so that people around the world can know the facts about topics ranging from global health to migration.

These were just some of my top takeaways; the conference also stressed the importance of breathing life into statistics with a narrative, improving everyone’s data literacy so they can engage with statistics better and giving greater weight to qualitative data about people's perceptions and values. I look forward to the next one in Dubai in two years' time!

is 成人论坛 Media Action's Head of Research Programmes, overseeing research across Africa, Asia and the Middle East.