This is the initial report from the Social CG Task Force on Trust and Safety for ActivityPub, Activity Streams 2.0, and the Fediverse. We've structured this document to outline why we need to think about trust and safety, the current trust and safety features in ActivityPub, the current areas of work we're undertaking to improve trust and safety in ActivityPub, and advice for people building ActivityPub-enabled software.
This report is still in the process of being written. Please see the Initial Report issue on GitHub for current status.
ActivityPub is an inherently social protocol, and as such, it must adequately address trust and safety for the success of the protocol and the platforms that are built with it.
As an implementer of an ActivityPub-enabled social platform, if you were to only read the ActivityPub, Activity Streams 2.0 Core, and Activity Vocabulary specifications, you would not have enough knowledge to ensure trust and safety for your service. This report provides additional information to ensure you can reach a baseline of trust and safety in your ActivityPub-enabled software.
As noted by the Atlantic Council in their [[[scalingtrust]]] paper:
Risk and harm are set to scale exponentially and may strangle the opportunities generational technologies create. We have a narrow window and opportunity to leverage decades of hard won lessons and invest in reinforcing human dignity and societal resilience globally.
Atlantic Council — [[[scalingtrust]]]
This applies directly to ActivityPub software: we are in a pivotal moment of early growth and adoption, where we can leverage decades of learning on trust and safety for online social platforms to ensure that the protocol meets the challenges that will face us as adoption grows.
We regularly face issues with spam and abuse, and we have seen services taken down by online harms related to user generated content. We have already seen issues with moderation reports for harmful behaviour being dropped due to incompatibilities between various ActivityPub software. These issues result in people not trusting platforms that are powered by ActivityPub.
Trust and safety isn't just about the needs of specific groups that use platforms, but about ensuring all those that use platforms built on ActivityPub have experiences that align with their expectations to be free from harassment, abuse, and inauthentic activity.
The task force brings together a variety of contributors from varying projects, platforms, and organisations, along with experts in trust and safety, researchers, and moderators. We work to reach consensus on the documents that we produce through regular meetings that are open for anyone to join.
We have agreed on an initial scope of work that focuses on improving core protocol features that already exist and are in use, before working on new features.
You can view our current scope of work on the task force's GitHub repository, where you can also find information about the leadership of the taskforce and other policies and information.
This initial report from the ActivityPub Trust & Safety Task Force covers the following topics:
TODO
The ActivityPub specification contains the following language regarding spam:
Spam is a problem in any network, perhaps especially so in federated networks. While no specific mechanism for combating spam is provided in ActivityPub, it is recommended that servers filter incoming content both by local untrusted users and any remote users through some sort of spam filter.
[[[ActivityPub]]], B.6 Spam
TODO: expand on why this isn't enough
The [[[activitystreams-vocabulary]]] defines the Block and Ignore activity types, which can be useful for people to control what they see and who can interact with them.
ActivityPub defines side effects for the Block activity when POSTed to the outbox. Per [[[ActivityPub]]]:
The Block activity is used to indicate that the posting actor does not want another actor (defined in the object property) to be able to interact with objects posted by the actor posting the Block activity. The server SHOULD prevent the blocked user from interacting with any object posted by the actor.
Servers SHOULD NOT deliver Block Activities to their object.
[[[ActivityPub]]], 6.9 Block Activity
In practice, despite the normative recommendation for outbox servers to not deliver Block activities to their object, existing softwares currently deliver Block activities to their object. Philosophically, this is done under the assumption that the remote inbox is being managed by a server instead of being visible to any users directly; the motivation to deliver Block activities is so that trustworthy servers can enforce the Block remotely and extend the definition of "interact with" to include "viewing".
The Ignore activity is not defined to have any side effects in [[[ActivityPub]]].
Relying solely on Block and Ignore activities does not provide people with adequate control of their experiences. See [[[#flexible-control]]] for more information.
The [[[activitystreams-vocabulary]]] defines the Flag activity type, which can be useful for reporting problematic content to moderators.
At present, there is no formal definition of how to process Flag activities, which properties one can expect or require to be included, or where to send these activities. Existing implementations have differing requirements for Flag activities, and they generally send these activities to standard inboxes. Philosophically, this is done under the assumption that the remote inbox is being managed by a server instead of being visible to any users directly; in cases where this is not true, a report may end up being delivered as an activity directly visible to the user being reported.
One goal of the Trust & Safety Task Force is to define a common profile for Flag activities and where to send them. For more information, see this document: TODO
TODO: Something about how content is distributed, and how it can leak to unintended audiences through the use of embedding of foreign objects (e.g., announce by reference vs announce by embedding)
TODO, some ideas:
Explain the different approaches to federation management: open federation, consent-based federation, and closed federation. Explain how the implementation of federation management can directly impact your users and their experiences.
Explain that whilst open registration can be desirable, it comes with the responsibility to be able to moderate around the clock, and also should be implemented such that if a server becomes unmaintained, it automatically closes, as to prevent that server from being used as a vector for abuse and spam
When implementing ActivityPub, it can make sense to implement support for the Block and Ignore activities as described in [[[#block-and-ignore]]]. However, it is better to go beyond this limited functionality and allow people to more fully control what they see and how others can interact with them, outside of simple binary actions.
The functionality of an Ignore activity, commonly called a "mute" or sometimes "hide", is better served by a filter system. For compatibility mapping, ignoring or muting a user can be thought of as a filter whose context is scoped to the author of a post being that user: when an Object is `attributedTo` a certain actor, or when an Activity has a certain `actor`, it should not be shown to the user.
Other filter contexts are possible. The following properties are particularly useful for filtering:
Recursive filters are also possible, such as filtering when the object is `inReplyTo` some other object whose `attributedTo` or `actor` has a particular value to be filtered; however, be wary of performance concerns when recursively fetching linked resources. Your software can avoid this problem by setting a maximum recursion limit, which might be 1.
The functionality of a Block applies to all interactions, but your users may reasonably want to only limit some interactions, or they may want to set conditions on certain usage of their content. This can be done with policies.
At this time, there is no broad consensus on a specific mechanism for signaling these policies, but some vocabularies are being developed or could be applied for this purpose. For example, [[[ODRL-Vocab]]] allows describing policies that explicitly grant permission for certain actions, although there is no enforcement attached — it is up to consumers to respect these policies. For social proof, publishers can provide acknowledgement of other people's resources, although there is no broad consensus here either. It is possible to respect some authorities by fetching canonical representations of certain collections defined by [[[ActivityPub]]]:
Furthermore, some other properties defined by the [[[activitystreams-vocabulary]]] are commonly used in social situations, with values that can be Collections (or might have canonical collections assoicated with them):