Overview
ZEGOCLOUD In-app Chat (ZIM) supports content moderation for messages. Currently, there are two ways to meet your moderation needs:
-
(Recommended) You can enable and use the built-in content moderation capability in ZIM. This allows you to quickly moderate text messages, image messages, voice messages, and video messages without additional development. Additionally, ZIM supports custom sensitive word lists to customize the text message moderation.
-
If you prefer to moderate messages by yourself or integrate with a third-party content moderation service, you can use the Callback on message not sent yet.
Built-in content moderation capability
After contacting ZEGOCLOUD technical support to enable the content moderation service, you can obtain the content moderation capability without additional configuration. ZIM will return the moderation results based on the preset moderation level, and you only need to implement relevant business logic based on the results.
The following table provides to an introduction to content moderation capability:
| Message Type | Moderation Method | Moderation Channel | Moderation Tags |
|---|---|---|---|
| Text Message | Moderate before sending |
| Default detection of the following content: politics, terrorism, prohibition, erotic, abuse, advertisement, privacy, advertising law violations, etc. If adjustments are needed, please contact ZEGOCLOUD technical support. |
| Image Message |
|
| Default detection of the following content: politics, pornography, terrorism, QR code, advertisement, etc. Additionally, it can support identifying whether the text in the image is in violation. If adjustments are needed, please contact ZEGOCLOUD technical support. |
| Audio Message | Moderate before sending |
| Default detection of the following content: politics, pornography, terrorism, etc. Additionally, it can support identifying the following content: statesman voiceprints, advertisement, prohibition, abuse, etc. If adjustments are needed, please contact ZEGOCLOUD technical support. |
| Video Message | Moderate before sending |
| Video moderation includes the moderation of images and audio in the video.
|
-
Explanation of Moderation Methods:
- Moderate before sending: ZIM moderates the message before sending it. If the moderation passes, the message is sent successfully; if the moderation fails, the message fails to be sent.
- Send before moderation: ZIM sends the message first and then moderates it. If the moderation passes, the message is displayed normally; if the moderation fails, the message is withdrawn.
-
When using the content moderation service, you can configure corresponding moderation tags and thresholds based on the moderation channels. For example, in some social scenarios, stricter thresholds and tags are used for group chats, while looser thresholds and tags are used for one-on-one chats.
-
Different moderation level can be configured based on the moderation results. The moderation results are divided into "pass," "uncertain," and "fail."
- If you set the moderation intensity to "Weak Moderation": When the result is "uncertain," ZIM will still send the message.
- If you set the moderation intensity to "Strong Moderation": When the result is "uncertain," ZIM will stop sending the message.
To understand the full process including enabling and using the content moderation service, please refer to the instruction.
Customize sensitive word lists
If the default sensitive words used for the built-in text message moderation capability do not satisfy your demands, you can customize word lists.
To use this feature, please provide ZEGOCLOUD technical support with custom lists of sensitive words and explain the processing strategy.
After the configuration is completed by ZEGOCLOUD technical support, ZIM will process the content in user messages that matches the list. Currently, ZIM provides three processing strategies:
| Processing Strategy | Description | Examples | ||
|---|---|---|---|---|
| Sensitive Words | Original Text | Result | ||
| Block Sensitive Words | If a message contains sensitive words, the message will be blocked and the sending will fail. | Hello | Hello, welcome. | Send failed. |
| Replace Sensitive Words | If a message contains sensitive words, ZIM will replace the sensitive words with other content and send the message. The receiving user will see the replaced content. | Hello | Hello, welcome. | **, welcome. |
| Send Original Text | Even if a message contains sensitive words, ZIM will still send the message. | Hello | Hello, welcome. | Hello, welcome. |
When providing lists of sensitive words, please follow these practices:
- The list files should be in Excel format.
- Classify the sensitive words according to the processing strategy, and then place them in separate list files.
- A sensitive word can only appear in one list file, which means only one processing strategy can be configured.
- If a sensitive word is configured with multiple strategies, ZIM will prioritize the processing of the word according to the following order: block > replace > send original text.
- If the sensitive words contain vocabulary from different languages, please classify them according to the language and place them in separate list files.
For example, if your custom sensitive words include both Chinese and English languages, and you need to use all the 3 processing strategies, you need to provide ZEGOCLOUD technical support with 6 sensitive word list files in Excel. 3 files should only contain Chinese vocabulary (file names: project_name_block_zh, project_name_replace_zh, project_name_send_zh), and the other 3 files should only contain English vocabulary (file names: project_name_block_en, project_name_replace_en, project_name_send_en).
Pricing
Please contact ZEGOCLOUD business personnel for pricing details.