EARLIER this month, Twitch announced it would start banning users for behavior away from its site.
The move by Amazon Inc's live-streaming platform involved hiring a law firm to conduct investigations into users' misconduct, a new twist in the latest prominent example of tech companies acting on "off-service" behavior.
How platforms enforce against activities conducted not just on their services but on other sites and offline, is often only described vaguely in their rules. But as lawmakers and researchers examine tech's relationship with real-world violence or harm, this moderation is gaining attention.
While some groups have praised platforms for being proactive in protecting users, others criticize them for infringing on civil liberties.
"This isn't content moderation, this is conduct moderation," said Corynne McSherry, legal director at the digital rights group Electronic Frontier Foundation, who said she was concerned about platforms that struggle to effectively moderate content on their own sites extending their reach.
In interviews, platform policy chiefs described how they drew different lines around off-service actions that could impact their sites, acknowledging a minefield of challenges.
"Our team is looking across the web at a number of different platforms and channels where we know that our creators have a presence...to understand as best as possible the activities that they're engaging in there," said Laurent Crenshaw, policy head at Patreon, a site where fans pay subscriptions for creators' content.
Facebook Inc's rules ban users they deem dangerous, including those involved in terrorist activity, organized hate or criminal groups, convicted sex offenders and mass murderers. People who have murdered one person are mostly allowed, a spokeswoman said, due to the crime's volume. Last year, Facebook expanded the list to include "militarized social movements" and "violence-inducing conspiracy networks" like QAnon.
Twitch's new rules say it may ban users for "deliberately acting as an accomplice to non-consensual sexual activities" or actions that would "directly and explicitly compromise the physical safety of the Twitch community," categories which a spokeswoman said were intentionally broad.
Twitch's change in policy https://reut.rs/3ewoCgf largely stemmed from the gaming industry's #MeToo moment in summer 2020 when the site saw harassment at real-life gaming events and on sites like Twitter and Discord, Chief Operating Officer Sara Clemens told Reuters.
Looking beyond their own sites has helped companies remove extremists and others who have "learned the hairline cracks" in site rules to stay online, said Dave Sifry, vice president of the Anti-Defamation League's Center for Technology and Society, which has pushed for major platforms to incorporate this behavior into decisions.
Self-publishing site Medium established off-service behavior rules in 2018, after realizing attendees of the August 2017 white nationalist rally in Charlottesville who had not broken rules on specific sites appeared to be "bad actors on the internet in general," it said.
Last summer's protests over the murder of George Floyd prompted Snap Inc to talk publicly about off-platform rules: CEO Evan Spiegel announced Snapchat would not promote accounts of people who incite racial violence, including off the app. In December 2020, TikTok updated its community guidelines to say it would use information available on other sites and offline in its decisions, a change that a spokeswoman said helped it act against militia groups and violent extremists.
Notably, this year, sites like Facebook, Twitter Inc and Twitch took into account former U.S. President Donald Trump's off-service actions that led to his supporters storming the U.S. Capitol on Jan. 6 when they banned him.
FROM MURDER TO MONEY LAUNDERING
Tech companies differ in approaches to off-platform behavior and how they apply their rules can be opaque and inconsistent, say researchers and rights groups.
Twitter, a site where white nationalists like Richard Spencer continue to operate, focuses its off-service rules on violent organizations, global director of public policy strategy and development Nick Pickles said in an interview.
Other platforms described specific red-flag activities: Pinterest, which took a hardline approach to health misinformation, might remove someone who spreads false claims outside the platform, policy head Sarah Bromma said. Patreon's Crenshaw said while the subscription site wanted to support rehabilitated offenders, it might prohibit or have restrictions around convicted money launderers or embezzlers using its platform to raise money.
Sites also diverge on whether to ban users solely for off-service activity or if on-site content has to be linked to the offense.
Alphabet Inc's YouTube says it requires users' content to be closely linked to a real-world offense, but it may remove users' ability to make money from their channel based on off-service behavior. It recently did this to beauty influencer James Charles for allegedly sending sexually explicit messages to minors.
Charles' representatives did not respond to requests for comment. In a statement posted on Twitter this month he said he had taken accountability for conversations with individuals who he said he thought were over 18 and said his legal team was taking action against people who spread misinformation.
Deciding which real-life actions or allegations require online punishments is a thorny area, say online law and privacy experts.
Linking the activity of users across multiple sites is also difficult for reasons including data privacy and the ability to attribute actions to individuals with any measure of certainty, say experts.
But that has not deterred many companies from expanding the practice. Twitch's Clemens said the site was initially focusing on violence and sexual exploitation, but it planned to add other off-site activities to the list: "It's incremental by design," she said.
Reuters
Tue Apr 27 2021
Facebook Inc's rules ban users they deem dangerous, including those involved in terrorist activity, organized hate or criminal groups, convicted sex offenders and mass murderers. - REUTERSpic
Who is Prabowo Subianto, incoming president of Indonesia?
A wealthy ex-general with ties to Indonesia's popular outgoing president and its dictatorial past, looks set to be its next leader.
Iran's supreme leader says Hamas leader's death will not halt 'Axis of Resistance'
The "Axis of Resistance", built up with years of Iranian support, includes Hamas, the Lebanese Hezbollah group, the Houthi movement in Yemen, and various Shi'ite groups in Iraq and Syria.
Putin says Russia willing to seek compromises between Iran and Israel
Russia is ready to help seek compromises between arch-foes Israel and Iran, President Vladimir Putin said on Friday, saying these would be difficult but possible.
What proposals will Russia push at the BRICS summit?
The proposal is also to establish a BRICS reinsurance company to allow uninterrupted shipment of goods and key commodities between members.
Indonesia's free meals plan in the spotlight as Prabowo readies for office
Prabowo calls the programme one of the main drivers of economic growth, eventually set to add an estimated 2.5 million jobs.
Astro AWANI's revamped English news website, AWANI International, launches on Oct 21
Astro AWANI's revamped English platform delivers in-depth global news and expert analysis to keep you informed on key developments.
Israeli strikes kill 33 people in Jabalia refugee camp in Gaza, medics say
Residents of Jabalia said Israeli tanks had reached the heart of the camp after pushing through suburbs and residential districts.
Liam Payne's ex-partner calls for media restraint after 'painful' death
Cheryl Tweedy used her statement to urge the media to remember they had a seven-year-old son, Bear, who could read the reports.
Analysts: Indonesia's strong MoF leadership team to boost investor confidence
Sri Mulyani Indrawati as head of Indonesia's Ministry of Finance is expected to instil confidence among investors.
Biden offers both a carrot and a stick to Israel as his term nears an end
Israel has frequently resisted US advice and has caused political difficulties for the Biden administration.
Putin says BRICS will generate most of global economic growth
Russian President Vladimir Putin will host a summit of the group in the city of Kazan on Oct. 22-24.
ISIS Malaysia's perspective of Budget 2025
An excellent rakyat-centric budget under the overarching principle of a caring and humane economy.
Budget 2025: Record increase in STR, SARA aid initiatives
The government will provide a significant boost to the Sumbangan Tunai Rahmah (STR) and Sumbangan Asas Rahmah (SARA) initiatives next year.
Budget 2025: EPF contributions to be made mandatory for foreign workers – PM Anwar
The government plans to make it compulsory for all non-citizen workers to contribute to the Employees Provident Fund (EPF).
What policies to expect from Indonesia's new President Prabowo
Prabowo will be open to foreign investment, his aide has said, such as by offering investors management of airports and sea ports.
Budget 2025: Govt allocates RM470 mil to empower women's participation in PMKS
The Women's Leadership Apprenticeship Program will be intensified as an effort to produce more female corporate personalities.
Israel sends more troops into north Gaza, deepens raid
Residents of Jabalia in northern Gaza said Israeli tanks had reached the heart of the camp, using heavy air and ground fire.
Indonesia ramps up security ahead of Prabowo's inauguration
Prabowo Subianto will be sworn in as Indonesia's president on Sunday with Vice President-elect, Gibran Rakabuming Raka, also taking office.
Immediate allocation of RM150 mil for local authorities, DID to tackle flash floods
Datuk Seri Anwar Ibrahim said this allocation is intended to address the recent flash floods that hit the capital and several major towns.
Budget 2025: Sabah, Sarawak to continue receiving among highest allocations - PM
Sabah and Sarawak continues to be prioritised under Budget 2025, with allocations of RM6.7 billion and RM5.9 billion respectively.