Data is a commodity. Do your survey respondents feel data-rich?
One of the things that made me uncomfortable when I first started working as a data scientist was the power imbalance in the data collection methods I was being trained in. One of my very first jobs was to help design and administer a survey to rural women in Southeast Asia. I felt uncomfortable asking survey questions to these women while they put their lives on hold for almost two hours per survey. These women were mostly living on less than $2 per day.
I knew that the data we were collecting through these surveys was important and useful. Including to the women who were answering the questions. The analysis of the data was helping us understand which parts of our project were working, and for who, and sometimes even why.
I still struggled with the fact that many of the North American women I knew would not consider spending 2 hours with me answering detailed questions about their lives and jobs and incomes. Particularly not during their actual working day.
This discomfort helped found the We All Count project for equity in data. The Data Collection step in the Data Lifecycle has always been a core focus for me.
One of the things my team experimented with was to pay the women for the time it took them to complete the survey. We also offered to provide child care during that time. I’m proud of this step, however, donors and funders were concerned that this would skew the results of the data. And some donors forbid us outright from using any of their money to do this saying that it simply wasn’t allowed in their funding guidelines.
As the saying goes, “data is the new oil” and wealthy people are starting to get worried about the value of data and who owns their data; leading to a new focus on equity and data collection. This is a great opportunity to move the ball forward in how the social sector collects data and to rebalance the power dynamics in Data Collection.
Forward Thinking
Jaron Lanier has just published a series of videos in the New York Times addressing the issue of data ownership and the economic imbalance between “data creators” and the people making money off of the data. He’s talking specifically about e-commerce and social media and similar stuff – but the very same argument applies to data in the social sector.
He’s come up with a plan called “Data Dignity.”
The fundamental idea is that “You should have the moral rights to every bit of data that exists because you exist, now and forever.”
What if every person who contributed data to a social sector project retained ownership of that data? What if they received a small amount of money from the research reports, the impact reports, the journal articles, and the social media posts that included any result from their data? These products generate income for many people including foundation staff, non-profit staff, researchers, academics, and social media companies. That income could be shared with the people who are actually generating the data. It would be a step in the direction of rebalancing the equity in the Data Collection Life Cycle.
Jaron has a specific plan about how to implement this in the real world. And the steps he suggests would work in the social sector. They would work on the ground in most places I’ve worked.
What about our donor’s concerns that it would change the answers to the questions? There isn’t a lot of good research out there on this topic. Which is an important signal. The research that is out there seems to indicate that it would not be a problem. If anything, they found that many answers were more honest if the survey participants were compensated. This piece of research did find some changes in the reported level of income – but it’s inconclusive as to where these changes were towards accuracy or not. One difference with this research and what we’re thinking about is that these projects frame the payment to participants as ‘a gift’. This creates a further imbalance in terms of equity. In the new model, the compensation is not framed as a ‘gift’ but rather as an actual legitimate compensation for the creation of a valuable commodity – data.
If You Pay your Survey Respondents You Just Might Get a Different Answer
Can Incentives Improve Survey Data Quality?
The Social Sector is Different
It’s true that the social sector and nonprofit organizations have a different mandate than say, Facebook and Google. And paying participants for data may, in fact, be outside of your reach for the time being. We can still change the lens through which we see the participants giving their data.
In most of the existing literature about compensating for time spent generating survey data, it is framed as ‘a gift’, but if we adjust our perspective we see that the data being produced by our social surveys is an extremely valuable commodity; a source of fuel that runs an entire section of our industry. The people providing data are actually a large and significant donor base. These people should be thought of as donating their data to the nonprofit sector. What if we gave them tax receipts in appropriate countries for their donation? What if we treated these people as a donor class and were forced to invest in our relationships, pitch our methodologies, and defend our spending? What if we ensured that they were getting appropriate reports and updates in the same way our “top-tier” donor base is? This would potentially add a lot of equity to the data collection and ownership process.
What I Ask Myself:
Data collection equity can be improved through direct compensation, changes in respondent treatment, and a paradigm shift in who is considered a donor. There’s a touchstone thought I want all of us in the social sector to start thinking: “Would I take this survey? For free?”.
