Media Action Insight Blog Feed

The art of designing surveys about social norms: insights from Ethiopia

2017-05-17T14:16:48+00:00

How we view our role and relationships within our communities shapes how we behave – with important consequences for development. When widespread, these views constitute ‘social norms’, which people tend to follow because they believe that others do and because they think it’s expected of them. Media Action’s programmes aim to challenge social norms that can lead to people risking their health and reinforce those that support people to make healthier choices.

Donors increasingly want proof that we’re changing norms at scale. But it’s not easy either to define social norms or to identify exactly how media influences desired outcomes, such as children getting vaccinated or mothers going to antenatal check-ups.

These are some of the challenges we face as Media Action’s research team. How do we design research to measure social norms and track whether they’re changing? We came up against this question when evaluating whether our maternal health radio shows had changed social norms in Ethiopia.

We set out to understand whether our programmes had convinced families to prepare for a birth by, for example, saving money and planning how to travel to the hospital. In Ethiopia, preparing for a baby’s arrival typically means planning the important post-birth thanksgiving ceremonies. Women are judged negatively if they don’t put on a good spread but the costs of hosting this celebration mean that there’s less money to get a woman to a facility when the time comes.

We primarily used qualitative methods to understand social norms but we also undertook large-scale surveys to understand how we were shaping social norms at a population level.

To highlight challenges around researching social norms with surveys – and how they might be overcome – here are five common responses heard in the field in Ethiopia:

1. ‘How would I know what other people think?’

Though intuitive in English, it’s not always obvious in other languages that asking someone ‘what would your neighbour think?’ really means ‘what do you think your neighbour would think?’. Respondents sometimes take the question very literally and reply that they ‘didn’t know the minds of other people’.

Cognitive interviewing, where interviewees reflect on what they understand by a question, and careful piloting of questions can reveal when something isn’t interpreted as intended. To get the wording of a question right, it’s essential to take the time to test and refine translations into local languages and carefully train field interviewers.

2. ‘Which people?’

When asked about ‘other people in the community’, interviewees often weren’t sure who they were meant to be thinking about. We consequently explained what we meant to respondents, to help them move from an amorphous sense of ‘other people’ to something more meaningful. For example: ‘If you think of five women you know, how many of them would start going for check-ups in the first three months of pregnancy?’

3. ‘It’s none of my business!’

Our team found people in Amhara (a region in northern Ethiopia where the research took place) to be quite reserved. It’s an area where it seemed to be impolite to speculate on the lives of others, not least to an outsider with a clipboard.

Surveying was as much an art as a science, with interviewer rapport proving to be as important as good survey design.

When it comes to rapport the ordering of questions is important. First of all, you can turn people off a survey by leading with potentially sensitive questions. How willing would you be to tell someone what you earn or weigh when you just met them?

Second, it’s easier to get good answers to tricky questions (like those around social norms) if they’re asked some way into the survey after rapport has been built. But these questions can’t be left too late. People often find talking about things like community dynamics tiring, so you have to broach these topics while people still have enough energy to discuss them.

Finally, it’s important to brief interviewers well so they’re confident of the research aims. They’ll also be better at reassuring participants they’re not trying to probe into their private lives, but rather uncover the practices and perceptions of their community as a whole.

4. ’I don’t know’

Respondents are often stumped by questions – particularly by those that ask to what extent they agree with a given statement. However, presenting them with a range of simple and concrete options can help elicit answers.

For example, to determine whether respondents believe their community sees something as appropriate or expected, we presented them with options such as: ‘they would think it’s OK’, ‘they would think it’s not OK’ and ‘they wouldn’t care’.

To determine what respondents think people in the community are actually doing, we presented them with choices like ‘none or a few do it’ and ‘almost all do it’.

We found that listing these options helped increase understanding and made the surveys less demanding of participants, resulting in fewer ‘don’t knows’.

5. ‘Let me tell you what happened to me during my last pregnancy…’

When a respondent has to choose from a limited number of options, there’s no way of recording the rich detail someone might provide through sharing their experiences in a more narrative form. Within the context of a survey, personal stories become conundrums (how do I classify this?) rather than goldmines of information. This tension demonstrates why qualitative research remains invaluable.

Illustrating this, we recently used a personal narrative approach to allow mothers to tell us the journey of their pregnancy. Asking questions like ‘Would you say that your story of pregnancy is the same as most of the mothers in your community?’ helped us understand whether or not interviewees were conforming to norms.

We’re continually exploring ways to improve our surveys to pinpoint what social norms are and how they’re changing, always with the aim of truly reflecting people’s perceptions and helping them improve their health.

Hilina Assefa is a Senior Research Officer with ��̳ Media Action’s Ethiopia team; Lois Aspinall is a Research Manager with the UK office.

Blog: Design thinking and health communication: learning from failure

Blog: Health communication changes lives by reaching millions

Five questions our data portal can help answer

2017-04-04T06:00:00+00:00

There's a lack of data on what ordinary people think, feel and want in developing countries. Our new Data Portal aims to help fix that. Sonia Whitehead runs through five questions the portal can help answer on governance, media and resilience.

The development world is all aflutter about data. There’s much talk of a data revolution, the sector’s hiring data scientists and the World Bank just launched a new podcast to ‘data crunch the world’.

Not to dampen all this excitement but we need a lot more data about people in the Global South before it can become a transformative force there. Addressing this lack of data will speed up progress on everything from gender to alleviating poverty.

Enter our new Data Portal (view on desktop), which brings together data, reports and visualisations from surveys conducted in 13 developing countries that there aren't a lot of statistics about. Over five years, we asked more than 75,000 (rarely polled) people about what they think, feel and want. The portal covers a range of issues from what they’re most worried about to how interested they are in politics.

We want these insights to help development leaders, practitioners and researchers better understand ordinary people in the developing world so they can produce more effective strategies, projects and communications.

To mark the launch of the portal, we run through five questions that it can help answer on governance, media and resilience, while also showcasing the different types of content available on the portal.

1. What sources of information do people trust?

Being a media organisation, we wanted to know whether people believe what they hear on the airwaves, see on TV and read online. We found that trust levels in radio are universally high, at over 80% in Kenya, Nigeria and Myanmar, and reaching 90% in Nepal.

However, people are more circumspect about the truthfulness of the internet, with the Burmese especially sceptical, which is concerning given that half of Myanmar's internet users say they go online in order to read the news.

To illustrate these (and other) insights into what media people think of different sources of information, we produced a series of visualisations – some of the ones for Kenya are previewed below (media visual available here, governance one here):

2. How free do people feel to speak their minds?

We asked people in three Asian countries (Bangladesh, Myanmar, Nepal), four African countries (Kenya, Nigeria, Sierra Leone, Tanzania) and the Palestinian territories whether they could 'say what they think'. A majority felt at least somewhat free to speak their minds in all but one of the countries: Myanmar.

But across the eight countries in our governance data set, we found that many people don’t feel they can criticise those in charge. Around a third of Kenyans, Sierra Leoneans, Bangladeshis and Palestinians ‘feel people like them are free to talk negatively about the government in public’; in Tanzania and Myanmar this drops to under one in four.

Nepal is the only place we looked at where a majority (65%) feel at least somewhat able to criticise those in charge. Nigerians are the next most comfortable with openly complaining about their leaders, a little over 40% say they could – though only half that number felt very liberated to do so.

3. Who are the keenest voters?

In six countries, we also asked whether people had voted in the last general election: Bangladesh, Myanmar, Palestine, Sierra Leone, Tanzania and Nigeria.

Sierra Leone came out top, with 90% reporting having voted in the last general election, closely followed by Myanmar at 87% and Tanzania at 86%. (A quick note – we conducted our Burmese survey in 2016, after the victory of Aung San Suu Kyi's National League for Democracy in 2015.)

Turnout was lowest in Bangladesh and the Palestinian territories, where 42% and 45% respectively said they’d cast a ballot in the last national election.

Of course, people don’t just get involved with public life through voting. Meetings, protests, and various forms of communication are all types of political participation. The Sierra Leone visualisation previewed below shows that while only a small proportion of people have been in touch with government officials, nearly two thirds have teamed up with others in their community to solve a problem.

4. How do people feel about those who are different to them?

Historically, Kenya has been the site of ethnicity-based politics and violence. Yet even against this background, well over 80% of Kenyans want to live with people from other ethnic groups and think it's important to know about different cultures and what life is like for those who are different to them (see below).

In a country with over 100 ethnic groups, 97% of people in Myanmar believe that peace relies on mutual respect between people from different ethnic, religious and social groups. Similarly in Nigeria – which has wrestled with religious divides – nine tenths of the population believe that people from different backgrounds have ‘more in common than they think’.

However, a large majority of both Nigerians and Burmese feel that some differences between groups are ‘just too difficult to overcome’.

5. How are people adapting to environmental change?

Building on our Climate Asia project – which examined 33,500 people’s everyday experiences of climate change – we’ve more recently asked Tanzanians and Bangladeshis about how they’re coping with changes to the world around them.

In the drought-ridden areas of Dodoma and Morogoro in Tanzania, more people think agricultural productivity has decreased than increased over the past ten years. They are split as to whether rainfall is higher or lower than it was a decade ago.

Getting their information predominantly from the radio, Tanzanians are making some – though not a lot – of changes in light of the environmental challenges they face. Popular responses include keeping food for longer and finding new water supplies.

In Bangladesh, rainfall, temperatures and extreme weather are all commonly seen to have increased in the past decade. TV is the country's go-to medium for getting information about water, food, energy and extreme weather, considerably more so than the radio, newspaper, friends and family.

Around a third of the population have changed how they live in response to environmental changes; swapping jobs is the most common way of shaking things up.

In addition to all of the data, the portal also hosts a number of other resources:

For extra guidance on navigating the portal, take a look at our ‘How-to’ section, as well as our ‘walkthrough’ and ‘running a query’ videos.

Those interested in how we collected the data should refer to the methodologies and questionnaires available on the right-hand sidebar of each of the thematic pages (media, governance, resilience).

The portal is also home to reports which summarise and analyse data available on the portal. For example, we've produced a report exploring how to better connect with the least politically engaged Kenyans. This is just a flavour of what’s to come, similar reports analysing our governance data in other countries will follow in the coming months.

On each thematic page, there are reports and tools to support practitioners to use media for development. For example, we’ve featured the communication toolkit from our Climate Asia project, which includes a guide on how to talk about climate change in an accessible and engaging way, as well as a set of cards (with instructions) for co-creating a communication strategy with partners and your target audience.

Sonia Whitehead is ��̳ Media Action's Head of Research Programmes, overseeing research across Africa, Asia and the Middle East.

Dispatch from the UN World Data Forum

2017-01-19T11:57:21+00:00

Head of Research Programmes Sonia Whitehead shares her three top takeaways from the first-ever UN World Data Forum.

This week, 2,000 statisticians, researchers and policymakers from across the world gathered in Cape Town for the first-ever UN World Data Forum. Focused on data’s role in achieving the Sustainable Development Goals (SDGs), the conference looked at how the sector can better understand the poorest 20% (P20) of the global population. Delegates puzzled over how to better understand and engage these people in order to uphold the commitment to ‘leave no one behind’, which underpins the SDGs.

To do this, the discussions centred on the need for well-communicated, high quality data. The collective view was that – at a minimum – national statistics bureaus should be producing data that breaks down statistics by gender, geography, age and disability. But what more needs to be done to improve our understanding of the P20 to inform policy decisions?

There is an abundance of data being collected today by companies seeking to map consumer preferences. The rise in social media usage is also helping improve our understanding of people, supplementing traditional sources of data. But does this data support development? And how can we improve the communication of data to ensure that everyone engages with and understands it?

Along these themes, there were three points discussed at this conference that gave me particular food for thought.

1) Data guardians are changing

Anne Jellema (World Wide Web Foundation) talked about how new players are collecting and understanding the kind of data that used to be monopolised by the state. National statistics bureaus have traditionally collected data from censuses and held administrative data on healthcare, education and employment – among much else. Governments have long been held accountable for how they use this data.

Technology, however, has shifted this state of affairs. Skype, Google Maps and Uber collect colossal amounts of user data, which doesn’t belong to the public and which their parent companies aren’t as accountable for as governments are.

This growing abundance of data could lead to a situation where data is increasingly in the hands of the few, seeking to utilise it for commercial rather than civic aims. Jellema raised this as a challenge that the data revolution needs to address in order to improve transparency, accountability and the accessibility of data.

Fortunately, we are already seeing how the proliferation of data can also empower citizens. In China, for example, the Bluesky Map App allows users to check the government’s real-time pollution data on factories near them, equipping them with the targeted information they need to push for better environmental practices locally. More than 1,800 factories are said to have taken steps to reduce pollution due to public pressure facilitated by the app, which enables users to tag violations of pollution standards on social media.

2) It's possible to collect data on people’s views – without asking one question

Collecting data on people’s views and perceptions has long been the preserve of researchers posing questions to respondents sampled in a systematic way. But there are now newer, faster ways of understanding views at scale, which don’t require a survey.

At the time of the Zika outbreak, Facebook and UNICEF teamed up to analyse social media content to inform the development of a digital campaign for Brazilians. Around 100 million people in the country use Facebook and Zika was a popular topic of discussion there. Anonymised Facebook posts were analysed to reveal that 58% of posts on Zika were by men.

Equipped with these insights, UNICEF went on to produce ads featuring a man kissing his daughter born with microcephaly, in a concerted effort to engage men around its Zika campaign. The analysis also showed that people didn’t want to just hear about Zika but also wanted to know find out more about other mosquito-borne diseases.

This example highlights that in an emergency where a high proportion of the population use a particular social media platform, content analysis of posts is a good way of providing audience insights quickly. ��̳ Media Action also used Facebook to run short online surveys in Nepal to inform post-earthquake broadcasts, but the Zika example shows that data on what people are concerned about can be collected without posing a single question.

3) People don’t know Global Development 101

Has the proportion of the world’s population living in extreme poverty changed over the last twenty years? Presented with the options of ‘almost doubled’, ‘remained more or less the same’ and ‘almost halved’, only 5% of Americans select the final of the three, which is in fact the correct one.

This pessimism about the true state of global development is not limited to the US; only 14% of South Africans and 23% of Swedes got the right answer. Similar questions were posed to conference delegates by Ola Rosling (Gapminder Foundation). And we were similarly downbeat about how many children are vaccinated against measles and how many years women have spent in the formal education sector. Even a bunch of statisticians fully versed in development didn’t get the answers right.

Through its Ignorance Project, Gapminder seeks to tackle these and other misconceptions about international development. The project is driven by a concern that the ‘actual facts’ about global progress aren’t getting through the clutter of people’s preconceptions, their tendency to generalise based their own experiences and the (likely outdated) facts they vaguely remembered having learned once in school.

Data clearly needs to be conveyed in a more engaging way so that people around the world can know the facts about topics ranging from global health to migration.

These were just some of my top takeaways; the conference also stressed the importance of breathing life into statistics with a narrative, improving everyone’s data literacy so they can engage with statistics better and giving greater weight to qualitative data about people's perceptions and values. I look forward to the next one in Dubai in two years' time!

Sonia Whitehead is ��̳ Media Action's Head of Research Programmes, overseeing research across Africa, Asia and the Middle East.