This is the initial report from the Social CG Taskforce on Trust and Safety for ActivityPub, ActivityStreams 2.0 and the Fediverse. We've structured this document to outline why we need to think about trust and safety, the current trust and safety features in ActivityPub, the current areas of work we're undertaking to improve trust and safety in ActivityPub, and advice for people building ActivityPub enabled software.

Introduction

ActivityPub and ActivityStreams are inherently social protocols, as such, they must adequately address trust and safety for the success of the protocol and the platforms that built on top.

As an implementer of an ActivityPub service, if you were to only read the ActivityPub and ActivityStreams 2.0 specifications, you would not have enough knowledge to ensure trust and safety for your service. This report aims to provide additional information to ensure you reach a baseline of trust and safety in your ActivityPub software.

As noted by the Atlantic Council in their Scaling Trust on the Web paper:

Risk and harm are set to scale exponentially and may strangle the opportunities generational technologies create. We have a narrow window and opportunity to leverage decades of hard won lessons and invest in reinforcing human dignity and societal resilience globally.

Atlantic Council - Scaling Trust on the Web

This applies directly to ActivityPub software: we are in a pivotal moment of early growth and adoption, where we can leverage decades of learning on trust and safety for online social platforms to ensure that protocol meets the challenges that will face us as adoption grows.

We have already seen issues with moderation reports for harmful behaviour being dropped due to incompatibilities between various ActivityPub software, we regularly face issues with spam and abuse, and have seen services taken down by online harms related to user generated content. These issues result in people not trusting platforms that are powered by ActivityPub.

Trust and safety isn't just about the needs of specific groups that use platforms, but about ensuring all those that use platforms built on ActivityPub have experiences that align with their expectations to be free from harassment, abuse, and inauthentic activity.

What is the ActivityPub Trust and Safety Taskforce?

The taskforce brings together a variety of contributors from varying projects, platforms and organisations, along with experts in trust and safety, researchers and moderators. We work to reach consensus with the documents that we produce through regular meetings that are open for anyone to participate in.

We have agreed on an initial scope of work that focuses on improving core protocol features that already exist and are in use, before working on new features.

You can view our current scope of work on the Taskforce GitHub Repository, where you can also find out information about the leadership of the taskforce and other policies and information.

What is the Initial Report?

This report from the ActivityPub Trust & Safety Taskforce, that you are currently reading, covers the following topics:

• Existing ActivityPub features and considerations for Trust & Safety
• Guidance for implementers building ActivityPub software to ensure Trust & Safety, particularly of moderators reviewing reported content.

ActivityPub features and considerations for Trust & Safety

TODO

Reporting Activities and Actors

TODO: something about reports and flag activities

Security considerations about Spam and Abuse

TODO: something about the spam and abuse considerations at protocol level

Spam is a problem in any network, perhaps especially so in federated networks. While no specific mechanism for combating spam is provided in ActivityPub, it is recommended that servers filter incoming content both by local untrusted users and any remote users through some sort of spam filter.

[[[ActivityPub]]] Specification, section B.6 Spam

Blocking Users

TODO: something about how whilst there is a block activity, it's not widely used due to safety issues

Federation Content

TODO: Something about how content is distributed, and how it can leak to unintended audiences through the use of embedding of foreign objects (e.g., announce by reference vs announce by embedding)

Guidance for implementers building ActivityPub software

TODO, some ideas:

• Federation Management (being able to block domains / actors)
• General report management
• Show the classification clearly, so the moderator is aware of the type of content they are about to review
• Blur all media until the moderator hovers to view greyscale version (re-blur when hover not detected or mouseleave event)
• Grayscale all media until the moderator clicks to toggle greyscale (allow toggle state back to greyscale)
• Mute all audio until the moderator requests audio
• Allow the moderator to reclassify the report
• Allow the service operator to choose from a list of harms or rules they want to receive reports about
• Offer the end user a path to report an actor, behaviour, or content, e.g. “report this account” or “report this post”
• Condense the labels by type and classification, and label each report. Use standard metadata to classify and present reported content.
• Use standard language to describe the reporting context.
• Consider a multi-step report submission process that allows fine-grained reporting
• Ensuring users can delete media
• Automatically closing open registration upon moderator inactivity
• Open registration and the moderation responsibility that it entails

Federation Management

Explain the different approaches to federation management: open federation, consent based federation and closed federation. Explain how the implementation of federation management can directly impact your users experiences.

Open Registration Considerations

Explain that whilst open registration can be desirable, it comes with the responsibility to be able to moderate around the clock, and also should be implemented such that if a server becomes unmaintained, it automatically closes, as to prevent that server from being used as a vector for abuse and spam