main article image
(golubovy/iStock)

A Reporter Went Undercover as a Facebook Moderator And Discovered Appalling Practices

Is this deliberate profiteering from extreme content?

ISOBEL ASHER HAMILTON & JAKE KANTER, BUSINESS INSIDER
18 JUL 2018
 

A journalist from the British broadcaster Channel 4 went undercover as a Facebook moderator and found a stream of toxic content that was intentionally left on the site.

The reporter posed as an employee of CPL Resources - a Dublin-based content-moderation contractor that has worked with Facebook since 2010 - for the documentary "Inside Facebook: Secrets of the Social Network."

 

The journalist undertook CPL Resources' training, in which new staff members are brought up to speed with Facebook's community standards and set to work reviewing content including images of graphic violence, child abuse, and hate speech.

Moderators were given three options when reviewing material: ignore, delete, or mark as disturbing. Content marked as disturbing remains on Facebook but has restrictions on who is able to view it.

The reporter found instances in which images of child abuse, racism, and violence were allowed to remain on Facebook.

In some cases, the findings also exposed wild inconsistencies between the way moderators were being trained and Facebook's standards.

A video of a little boy being beaten by an adult man

During the training session, the reporter was shown an example of content that should be marked as disturbing: a video of an adult man beating a small boy.

The video was reported to Facebook in December 2012 by Nicci Astin, who campaigns online against child abuse, but she was told at the time that the video did not violate Facebook's terms.

 

In its first two days on Facebook, the video was shared 44,000 times, and it was still up years later when Channel 4 investigated.

 

Richard Allan, Facebook's vice president of public policy, told Channel 4's Krishnan Guru-Murthy that the video "should have been taken down."

A week after Channel 4 brought the video to Facebook's attention, it was still online. As of Monday, Business Insider was still able to find a version of the video on the platform.

A racist meme of a girl being drowned

In Facebook's community standards on hate speech, it says "we do not allow hate speech on Facebook because it creates an environment of intimidation and exclusion and in some cases may promote real-world violence."

CPL Resources trainees were shown a meme of a little girl having her head held underwater with the caption "when your daughter's first crush is a little negro boy."

The reporter was told that the image was an "ignore," because "it implies a lot, but to reach the actual violation, you have to jump through a lot of hoops to get there."

 

Facebook told Channel 4 that the image did, in fact, violate its hate-speech policy and that it was "reviewing what went wrong to prevent it happening again."

The undercover reporter also found that certain instances of hate speech were permitted.

A comment aimed at Muslim immigrants that said "f**k off back to your own countries" was allowed to remain on the site. Had the comment been aimed solely at Muslims, rather than Muslim immigrants, it apparently would have been deleted.

"People are debating very sensitive issues on Facebook, including issues like immigration. And that debate can be entirely legitimate," Allan said in response to the comment. When pressed about whether it constituted hate speech, he said it's "right on that line."

In a statement to Business Insider, Allan said Facebook had made mistakes. The company has reviewed training materials at contractors like CPL and provided refresher training courses for moderators.

"It's clear that some of what is shown in the program does not reflect Facebook's policies or values, and falls short of the high standards we expect," Allan said.

"We take these mistakes in some of our training processes and enforcement incredibly seriously and are grateful to the journalists who brought them to our attention. Where we know we have made mistakes, we have taken action immediately. We are providing additional training and are working to understand exactly what happened so we can rectify it."

 

Does Facebook profit from extreme content?

Channel 4 spoke with Roger McNamee, an early Facebook investor who has become a critic of the company over issues including the Cambridge Analytica data scandal. He said Facebook stood to benefit from extreme content.

"It's the really extreme, really dangerous form of content that attracts the most highly engaged people on the platform," he said.

"Facebook understood that it was desirable to have people spend more time on site if you're going to have an advertising-based business."

This was backed up by a CPL Resources staff member, who told the undercover reporter that violent content was left on Facebook because "if you start censoring too much, then people lose interest in the platform."

The person added: "It's all about making money at the end of the day."

Facebook's Allan strongly disagreed. "Shocking content does not make us more money – that's just a misunderstanding of how the system works," he said.

He added: "There is a minority who are prepared to abuse our systems and other internet platforms to share the most offensive kind of material. But I just don't agree that that is the experience that most people want, and that's not the experience we're trying to deliver."

The documentary also tackles problems with moderating images of self-harm, underage users, and far-right pages with large followings.

"Inside Facebook: Secrets of the Social Network" will air on Channel 4 on Tuesday evening in the UK. It is produced by Firecrest Films.

/Beyond is ScienceAlert's new section covering the wider world of gadgets, games, and digital culture.

This article was originally published by Business Insider.

More from Business Insider:

 

More From ScienceAlert

The date of the legendary volcano explosion of Thera has finally been traced

One of the largest volcanic eruptions in recorded history.

9 hours ago