July 2011
THE DISTRIBUTION BULLETIN ISSUE #16
07/Jul/2011
SPECIAL REPORT: HOW FILMS CAN CHANGE THE WORLD
Three new studies assessing the impacts of AN INCONVENIENT TRUTH, THE END OF THE LINE, and WAITING FOR “SUPERMAN” finally prove Sam Goldwyn wrong. The Hollywood mogul famously declared, “If you want to send a message, use Western Union.” These reports highlight the real world results these films sparked and provide a new framework for evaluating the impacts of documentaries and features.
In the past, there was little research or rigorous analysis of powerful films such as FAHRENHEIT 9/11, SICKO, SUPER SIZE ME, and FOOD, INC. Instead they were evaluated primarily on anecdotal information and subjective impressions. The appearance of these three new studies finally provides the research and analysis filmmakers need to better understand how to ignite social change.
AN INCONVENIENT TRUTH, THE END OF THE LINE, and WAITING FOR “SUPERMAN” were each made to avert a looming crisis: global warming, the collapse of the world’s fisheries, and the failure of America’s public education system.
This Special Report includes exclusive coverage of the studies of AN INCONVENIENT TRUTH and WAITING FOR “SUPERMAN,” along with a concise analysis of THE END OF THE LINE report.
THE END OF THE LINE – A Social Impact Evaluation
This exemplary report documents the significant changes THE END OF THE LINE produced, highlights the importance of brand partnerships, and provides useful lessons concerning social media and coordination with partners.
The film was described by The Economist as “the Inconvenient Truth about the impact of overfishing on the world’s oceans.” Produced in the UK by the invaluable Channel 4 BRITDOC Foundation and financed by the Esmee Fairbairn Foundation, this beautifully designed report is the product of an 18-month study, which used qualitative and quantitative analysis, focus groups, and media analysis. It concludes that the film had a major impact on public awareness of overfishing—directly on viewers and indirectly on nonviewers through the huge amount of press it generated. The report estimates that the PR value of this media coverage was £4,186,710, more than four times the budget of the film.
The study also concludes that the film helped create “a tipping point in corporate policy” that spurred a number of corporations to switch to sustainable sources of fish. The upscale grocery chain Waitrose sponsored the film’s release and promoted it in their stores, giving customers postcards about film and the importance of buying sustainable fish. The classy Prêt A Manger chain of sandwich shops totally changed its fish buying policy after its founder saw the film.
When I interviewed the visionary Jess Search (CEO of BRITDOC and co-creator of the report with her colleague Beadie Finzi) about the report, she shared her belief that businesses are “engines of change.” Top-down change (requiring legislation and/or elections) and bottom-up change (requiring widespread grassroots involvement) are very difficult to achieve, but if you can persuade corporate decision-makers that the change you are seeking is in their interest, hundreds of thousands of consumers can be affected.
The study features a brilliant graphic that illustrates the complementary and interlocking partnerships filmmakers need to build with foundations and philanthropists, NGOs and advocates, policymakers, the media and brands. The report shows how much difference a film can make – expanding public awareness of an urgent issue, changing consumer behavior, altering corporate policy, and providing advocates with an effective tool.
BEYOND THE BOX OFFICE – New Documentary Valuations
This pioneering study assesses the true value of AN INCONVENIENT TRUTH in the UK. Jess Search conducted research concerning AN INCONVENIENT TRUTH over a two-year period for her MBA dissertation. After calculating the film’s financial returns, Jess tackled the complex task of examining and quantifying the social good the film created.
Her study begins by noting that “over the past seven years documentaries have proven to be effective drivers of public awareness, setting press agendas, influencing politicians, companies, and campaigners.” Given the limited funding available for documentaries, the paper states that it is essential to “capture and measure the public good films deliver.” Documentary investors, from individual philanthropists to representatives of foundations, charities, corporations, and government “need hard data to show to colleagues, bosses, and boards when it comes to media funding decisions.”
The dissertation explores alternative ways of assessing the social impact of AN INCONVENIENT TRUTH. One approach would be to “measure and then value the effects of the film” from swaying public opinion to changing behavior and reducing carbon emissions. The film generated extensive press coverage, which reached people who saw the film in theaters, on TV, or on DVD as well as many more people who never saw it, but learned about global warming through this coverage. The dissertation estimates the advertising value of this media at £3,732,000. AN INCONVENIENT TRUTH dramatically increased the press coverage of global warming, which continues to this day.
Seeing the film inspired the CEO of the Marks and Spencer chain to radically cut the company’s electricity usage while increasing its use of renewable electric power. Around £200 million is being invested over five years to transform the core business, which is on-track “to become carbon neutral by 2012.” The M&S website highlights the company’s goal of “becoming the world’s most sustainable major retailer.”
While it would be possible “to interview a selection of UK companies to discover if corporate policy had changed as a result of the film, and, if so, gather facts and figures on the carbon impact of these changes,” it would be very time consuming, labor intensive, and expensive.
Instead the dissertation uses an alternative “willingness to pay” technique for determining social value. As the study notes, contingent valuation surveys are used by environmental economists “to ascribe a dollar value to things like clean air and biodiversity in forests which had previously been attributed with no economic value.” In 1990 the US government used a major “willingness to pay” study to calculate damages in the Exxon Valdez case. Since then libraries, museums, and other subsidized institutions have used this approach to demonstrate their worth.
Jess applied this technique to film for the first time. She did a “willingness to pay” study to determine the intrinsic value placed on AN INCONVENIENT TRUTH by UK citizens. People who agreed with the film’s message were asked how much they would have been willing to pay to ensure the film was released in cinemas and on TV. Of those surveyed by YouGov, 54% would have been willing to pay something to ensure that others had the opportunity to see the film: 5% would have contributed £82.75 | 2% - £28 | 10% - £14 | 26% - £7 | 9% - £3.50 | 2% - £0.89. The results were similar for men and women, and also across economic classes “perhaps contradicting ideas that the environment is a more middle class pre-occupation.”
Extrapolating from those surveyed to the adult British population, the dissertation gives AN INCONVENIENT TRUTH an intrinsic valuation of £73,411,565. When this value is combined with the estimated advertising value (£3,732,000) and the film’s estimated UK profits (£1,258,972), the social return on investment in the UK for AN INCONVENIENT TRUTH is a whopping 57:1.
The paper demonstrates that the “willingness to pay” approach can be applied usefully to film. The dissertation makes a compelling case that “willingness to pay” studies may be the most practical method for evaluating the impacts of many social issue documentaries. Using a “willingness to pay” approach could provide a way to compare the impact of different films. Filmmakers seeking funding could also do preliminary research to show there’s significant interest in the topic of their film.
WAITING FOR “SUPERMAN”
This substantial study analyzes the impact of WAITING FOR “SUPERMAN” on viewers’ perceptions and attitudes. It examines how successful the film was in changing their views of the US education system. The study also highlights which content was most effective and which was least effective.
Funded in part by The Ford Foundation, the report was written by the Harmony Institute, a nonprofit research center established in 2007 to assess the influence of entertainment on social and environmental issues. Research began just before the US theatrical release of WAITING FOR “SUPERMAN” in September 2010. It included an analysis of press coverage, focus groups, an online survey, online trending reports, and in-depth interviews. This six-month study focused on the immediate effects of the film on viewers’ beliefs rather than on the film’s long-term impacts. The primary author was Eleanor M. Cleverly.
The paper concludes that “WAITING FOR ‘SUPERMAN” had a notable effect on audience perceptions of education in the US” and that “the film increased general understanding and elevated concerns over a number of problems plaguing public education.” The report described the film as “one of the most expansive communication campaigns concerning education in America to date.”
However the research showed that many viewers complained that the film didn’t make clear what they could do to improve things. Directives such as “get involved in your child’s education” were vague, and “offered no actionable items for individuals.” Audiences also felt that the film “failed to discuss many of the larger social issues that contribute to low-performing students and schools.” As one respondent noted “bad teachers and bad unions are not the only impediment to educational success… poverty, parenting, resources, and curriculum are just as important.” Viewers also felt that “there was a general overemphasis on charter schools.”
Viewers did respond quite well to the characters in the film and found the charter school lottery a compelling metaphor for the state of US public education. Reactions to the film differed sharply between educators and the general public. Overall general audiences and the press “reviewed the film favorably, giving the film an average rating of four stars out of a possible five.” Teachers disagreed sharply, giving the film an average of 2 stars and challenging the film’s “depiction of teachers and unions as simplistic.”
The report also looked at the ripple effect on organizations that were affiliated with the film’s outreach effort. DonorsChoose.org increased its individual users by 75,000 in 2010 and generated $2.1 million in pledges to fund classroom projects across the country. Being affiliated with WAITING FOR “SUPERMAN” also helped the United Way establish itself as a central player in the education world and changed the way it works within local communities. The lesson for other filmmakers is that “garnering early support from reputable affiliates currently working on a social issue can greatly assist both parties.”
The report notes that although the film played in US theaters for 13 weeks grossing $6.4 million and had 149,000 followers on Facebook as of March 18, 2011, it was “unable to foster a national conversation among those not previously invested in the education reform debate.” However, the study also noted that WAITING FOR “SUPERMAN” was “successful at reinforcing a commitment to teaching among those already in the profession.” One teaching student said, “the film illuminated for her how important teachers are to the success of their students.”
Debika Shome, the Harmony Institute’s Deputy Director, explained that the report had enabled the Institute to refine its methodology for measuring the success of media.
While the report is an internal document, the strikingly-designed highlights are available here.
THE FUTURE
These three studies mark the beginning of a new era of impact evaluation. They expand our understanding of how films can create real change. They include concrete examples of the significant effects of these films on corporations, consumers, and nonprofits. The reports also explore ways of tracking, measuring, and valuing impacts. Their methodologies will be used and refined by other researchers.
More foundations will follow Esmee Fairburn and The Ford Foundation in funding impact studies. The Wolfensohn Family Foundation and the Pacific Foundation both provided additional funds for the WAITING FOR “SUPERMAN” study. The Fledgling Fund will continue highlighting the importance of assessing the social impact of media that began with the publication of its working paper, which includes this chart.
More cutting-edge research on documentaries and features will enable independents to prove that films can make a difference. Filmmakers who learn how others have achieved social impact will be empowered to make films that can truly change the world.
Three new studies assessing the impacts of AN INCONVENIENT TRUTH, THE END OF THE LINE, and WAITING FOR “SUPERMAN” finally prove Sam Goldwyn wrong. The Hollywood mogul famously declared, “If you want to send a message, use Western Union.” These reports highlight the real world results these films sparked and provide a new framework for evaluating the impacts of documentaries and features.
In the past, there was little research or rigorous analysis of powerful films such as FAHRENHEIT 9/11, SICKO, SUPER SIZE ME, and FOOD, INC. Instead they were evaluated primarily on anecdotal information and subjective impressions. The appearance of these three new studies finally provides the research and analysis filmmakers need to better understand how to ignite social change.
AN INCONVENIENT TRUTH, THE END OF THE LINE, and WAITING FOR “SUPERMAN” were each made to avert a looming crisis: global warming, the collapse of the world’s fisheries, and the failure of America’s public education system.
This Special Report includes exclusive coverage of the studies of AN INCONVENIENT TRUTH and WAITING FOR “SUPERMAN,” along with a concise analysis of THE END OF THE LINE report.
THE END OF THE LINE – A Social Impact Evaluation
This exemplary report documents the significant changes THE END OF THE LINE produced, highlights the importance of brand partnerships, and provides useful lessons concerning social media and coordination with partners.
The film was described by The Economist as “the Inconvenient Truth about the impact of overfishing on the world’s oceans.” Produced in the UK by the invaluable Channel 4 BRITDOC Foundation and financed by the Esmee Fairbairn Foundation, this beautifully designed report is the product of an 18-month study, which used qualitative and quantitative analysis, focus groups, and media analysis. It concludes that the film had a major impact on public awareness of overfishing—directly on viewers and indirectly on nonviewers through the huge amount of press it generated. The report estimates that the PR value of this media coverage was £4,186,710, more than four times the budget of the film.
The study also concludes that the film helped create “a tipping point in corporate policy” that spurred a number of corporations to switch to sustainable sources of fish. The upscale grocery chain Waitrose sponsored the film’s release and promoted it in their stores, giving customers postcards about film and the importance of buying sustainable fish. The classy Prêt A Manger chain of sandwich shops totally changed its fish buying policy after its founder saw the film.
When I interviewed the visionary Jess Search (CEO of BRITDOC and co-creator of the report with her colleague Beadie Finzi) about the report, she shared her belief that businesses are “engines of change.” Top-down change (requiring legislation and/or elections) and bottom-up change (requiring widespread grassroots involvement) are very difficult to achieve, but if you can persuade corporate decision-makers that the change you are seeking is in their interest, hundreds of thousands of consumers can be affected.
BEYOND THE BOX OFFICE – New Documentary Valuations
This pioneering study assesses the true value of AN INCONVENIENT TRUTH in the UK. Jess Search conducted research concerning AN INCONVENIENT TRUTH over a two-year period for her MBA dissertation. After calculating the film’s financial returns, Jess tackled the complex task of examining and quantifying the social good the film created.
The dissertation explores alternative ways of assessing the social impact of AN INCONVENIENT TRUTH. One approach would be to “measure and then value the effects of the film” from swaying public opinion to changing behavior and reducing carbon emissions. The film generated extensive press coverage, which reached people who saw the film in theaters, on TV, or on DVD as well as many more people who never saw it, but learned about global warming through this coverage. The dissertation estimates the advertising value of this media at £3,732,000. AN INCONVENIENT TRUTH dramatically increased the press coverage of global warming, which continues to this day.
Seeing the film inspired the CEO of the Marks and Spencer chain to radically cut the company’s electricity usage while increasing its use of renewable electric power. Around £200 million is being invested over five years to transform the core business, which is on-track “to become carbon neutral by 2012.” The M&S website highlights the company’s goal of “becoming the world’s most sustainable major retailer.”
While it would be possible “to interview a selection of UK companies to discover if corporate policy had changed as a result of the film, and, if so, gather facts and figures on the carbon impact of these changes,” it would be very time consuming, labor intensive, and expensive.
Instead the dissertation uses an alternative “willingness to pay” technique for determining social value. As the study notes, contingent valuation surveys are used by environmental economists “to ascribe a dollar value to things like clean air and biodiversity in forests which had previously been attributed with no economic value.” In 1990 the US government used a major “willingness to pay” study to calculate damages in the Exxon Valdez case. Since then libraries, museums, and other subsidized institutions have used this approach to demonstrate their worth.
Jess applied this technique to film for the first time. She did a “willingness to pay” study to determine the intrinsic value placed on AN INCONVENIENT TRUTH by UK citizens. People who agreed with the film’s message were asked how much they would have been willing to pay to ensure the film was released in cinemas and on TV. Of those surveyed by YouGov, 54% would have been willing to pay something to ensure that others had the opportunity to see the film: 5% would have contributed £82.75 | 2% - £28 | 10% - £14 | 26% - £7 | 9% - £3.50 | 2% - £0.89. The results were similar for men and women, and also across economic classes “perhaps contradicting ideas that the environment is a more middle class pre-occupation.”
Extrapolating from those surveyed to the adult British population, the dissertation gives AN INCONVENIENT TRUTH an intrinsic valuation of £73,411,565. When this value is combined with the estimated advertising value (£3,732,000) and the film’s estimated UK profits (£1,258,972), the social return on investment in the UK for AN INCONVENIENT TRUTH is a whopping 57:1.
The paper demonstrates that the “willingness to pay” approach can be applied usefully to film. The dissertation makes a compelling case that “willingness to pay” studies may be the most practical method for evaluating the impacts of many social issue documentaries. Using a “willingness to pay” approach could provide a way to compare the impact of different films. Filmmakers seeking funding could also do preliminary research to show there’s significant interest in the topic of their film.
WAITING FOR “SUPERMAN”
This substantial study analyzes the impact of WAITING FOR “SUPERMAN” on viewers’ perceptions and attitudes. It examines how successful the film was in changing their views of the US education system. The study also highlights which content was most effective and which was least effective.
The paper concludes that “WAITING FOR ‘SUPERMAN” had a notable effect on audience perceptions of education in the US” and that “the film increased general understanding and elevated concerns over a number of problems plaguing public education.” The report described the film as “one of the most expansive communication campaigns concerning education in America to date.”
However the research showed that many viewers complained that the film didn’t make clear what they could do to improve things. Directives such as “get involved in your child’s education” were vague, and “offered no actionable items for individuals.” Audiences also felt that the film “failed to discuss many of the larger social issues that contribute to low-performing students and schools.” As one respondent noted “bad teachers and bad unions are not the only impediment to educational success… poverty, parenting, resources, and curriculum are just as important.” Viewers also felt that “there was a general overemphasis on charter schools.”
Viewers did respond quite well to the characters in the film and found the charter school lottery a compelling metaphor for the state of US public education. Reactions to the film differed sharply between educators and the general public. Overall general audiences and the press “reviewed the film favorably, giving the film an average rating of four stars out of a possible five.” Teachers disagreed sharply, giving the film an average of 2 stars and challenging the film’s “depiction of teachers and unions as simplistic.”
The report also looked at the ripple effect on organizations that were affiliated with the film’s outreach effort. DonorsChoose.org increased its individual users by 75,000 in 2010 and generated $2.1 million in pledges to fund classroom projects across the country. Being affiliated with WAITING FOR “SUPERMAN” also helped the United Way establish itself as a central player in the education world and changed the way it works within local communities. The lesson for other filmmakers is that “garnering early support from reputable affiliates currently working on a social issue can greatly assist both parties.”
The report notes that although the film played in US theaters for 13 weeks grossing $6.4 million and had 149,000 followers on Facebook as of March 18, 2011, it was “unable to foster a national conversation among those not previously invested in the education reform debate.” However, the study also noted that WAITING FOR “SUPERMAN” was “successful at reinforcing a commitment to teaching among those already in the profession.” One teaching student said, “the film illuminated for her how important teachers are to the success of their students.”
Debika Shome, the Harmony Institute’s Deputy Director, explained that the report had enabled the Institute to refine its methodology for measuring the success of media.
While the report is an internal document, the strikingly-designed highlights are available here.
THE FUTURE
These three studies mark the beginning of a new era of impact evaluation. They expand our understanding of how films can create real change. They include concrete examples of the significant effects of these films on corporations, consumers, and nonprofits. The reports also explore ways of tracking, measuring, and valuing impacts. Their methodologies will be used and refined by other researchers.
More cutting-edge research on documentaries and features will enable independents to prove that films can make a difference. Filmmakers who learn how others have achieved social impact will be empowered to make films that can truly change the world.