Content Moderation

Complete guide to Concordia’s content moderation system and admin tools

Concordia implements a comprehensive moderation system to maintain community standards and manage user-generated content across the platform.

Overview

The moderation system provides:

• Moderation queue: Centralized view of flagged content
• Status-based workflows: Content goes through pending → published → moderated states
• Role-based access: Moderators and admins can take moderation actions
• Audit logging: All moderation actions are logged
• User notifications: Authors are notified when content is moderated
• Self-moderation prevention: Users cannot moderate their own content

Moderation Permissions

Two permissions control moderation capabilities:

// From permissions.ts:136-138
| "moderation.queue"     // View moderation queue
| "moderation.action"    // Take moderation actions

Who has moderation permissions:

• Moderators: Both moderation.queue and moderation.action
• Administrators: Both permissions plus additional content deletion rights
• Regular users: No moderation permissions

Content Status Model

Most user-generated content follows a status-based lifecycle:

Status Values

// Common statuses across content types
type ContentStatus = 
  | "published"      // Live and visible to users
  | "moderated"      // Hidden due to moderation action
  | "deleted"        // Soft-deleted by user or admin
  | "pending_review" // Awaiting approval (for some content types)

Content Types with Status

Content Type	Statuses	Default
Reviews	published, moderated, deleted	published
Comments	published, moderated, deleted	published
Forum Posts	published, moderated, deleted	published
Forum Threads	published, closed, moderated, deleted	published
Classifieds	pending_review, active, sold, expired, archived, rejected	pending_review
Places	pending_review, published, archived, rejected	pending_review
Articles	draft, pending_review, published, archived, rejected	draft

Moderation API

The moderation API is located at /api/admin/moderate.ts.

Endpoint

POST /api/admin/moderate

Request Body

{
  postType: string;    // Type of content to moderate
  postId: string;      // UUID of the content
  reason: string;      // Reason for moderation (min 5 chars)
}

Supported Content Types

// From moderate.ts:15-17
const entityMap = {
  blogComments: { 
    table: blogComments, 
    statusField: "status", 
    authorField: "authorId" 
  },
};

Currently supported:

• blogComments - Blog post comments

Extensible: The entity map can easily be extended to support:

• reviews - Place reviews
• forumPosts - Forum posts
• classifieds - Classified ads
• comments - General comments

Authentication & Authorization

// From moderate.ts:22-39
const auth = await getAuth();
const session = await auth.api.getSession({ headers: request.headers });

if (!session?.user?.id) {
  return new Response(JSON.stringify({ error: "AUTH_UNAUTHORIZED" }), {
    status: 401,
  });
}

const roles = await getUserRoles(session.user.id);
if (!hasPermission(roles, "moderation.action")) {
  return new Response(JSON.stringify({ error: "AUTH_FORBIDDEN" }), {
    status: 403,
  });
}

Security checks:

1. User must be authenticated
2. User must have moderation.action permission
3. User cannot moderate their own content
4. Content must exist and not already be moderated

Validation

// From moderate.ts:44-66
if (!postType || !postId || !reason) {
  return new Response(
    JSON.stringify({
      error: "VAL_REQUIRED_FIELD",
      message: "postType, postId, reason required",
    }),
    { status: 400 }
  );
}

if (!Object.keys(entityMap).includes(postType)) {
  return new Response(
    JSON.stringify({ error: "VAL_INVALID_ENUM", field: "postType" }),
    { status: 400 }
  );
}

if (reason.trim().length < 5) {
  return new Response(
    JSON.stringify({ 
      error: "VAL_TOO_SHORT", 
      message: "reason must be at least 5 chars" 
    }),
    { status: 400 }
  );
}

Validation rules:

• All fields are required
• postType must be a valid entity type
• reason must be at least 5 characters

Self-Moderation Prevention

// From moderate.ts:95-102
const authorId = entity[config.authorField] as string;
if (authorId === session.user.id) {
  return new Response(
    JSON.stringify({ error: "BIZ_SELF_MODERATION" }),
    { status: 403 }
  );
}

Users cannot moderate content they created - prevents abuse.

Moderation Action

// From moderate.ts:104-124
// Set status to moderated
await db
  .update(config.table)
  .set({ status: "moderated" })
  .where(eq((config.table as any).id, postId));

// Notify the author
await createNotification({
  userId: authorId,
  type: "moderation",
  title: "Contenu modéré",
  body: `Votre contenu a été modéré. Raison : ${reason}`,
  targetType: postType,
  targetId: postId,
});

return new Response(
  JSON.stringify({ success: true, postType, postId, status: "moderated" }),
  { status: 200 }
);

Actions taken:

1. Update content status to "moderated"
2. Send notification to content author with reason
3. Return success response

Moderation Workflow

1. Content flagged or reported

   User reports inappropriate content or automated system flags it

2. Appears in moderation queue

   Moderators see flagged content in their moderation dashboard

3. Moderator reviews content

   Moderator examines the content and context

4. Moderation decision

   Moderator takes action:

   • Approve: Keep published (remove from queue)
   • Hide: Set status to "moderated"
   • Delete: Soft-delete the content
5. Author notified

   If content is moderated, author receives notification with reason

6. Audit logged

   Moderation action is recorded in audit log

Notification System

When content is moderated, the author receives a notification:

// From moderate.ts:111-118
await createNotification({
  userId: authorId,              // Content author
  type: "moderation",            // Notification type
  title: "Contenu modéré",       // French: "Content moderated"
  body: `Votre contenu a été modéré. Raison : ${reason}`,
  targetType: postType,          // Type of content (blogComments, etc.)
  targetId: postId,              // UUID of moderated content
});

Notification includes:

• Clear title indicating moderation
• Full reason provided by moderator
• Link to the moderated content (via targetType and targetId)
• Type marker for filtering/display

Moderator Permissions

Moderators have extensive permissions to manage community content:

// From permissions.ts:267-280
moderator: new Set([
  "moderation.queue",        // View moderation queue
  "moderation.action",       // Take moderation actions
  "review.delete_any",       // Delete any review
  "comment.delete_any",      // Delete any comment
  "forum.delete_any_post",   // Delete any forum post
  "forum.pin_thread",        // Pin important threads
  "forum.lock_thread",       // Lock threads to prevent replies
  "classified.delete_any",   // Delete any classified ad
  "event.delete_any",        // Delete any event
  "group.delete_any",        // Delete any group
  "gallery.delete_any",      // Delete any gallery
  "product.delete_any",      // Delete any product
])

Content Deletion Rights

Moderators can delete:

• Reviews (any)
• Comments (any)
• Forum posts (any)
• Classifieds (any)
• Events (any)
• Groups (any)
• Galleries (any)
• Products (any)

Additional forum powers:

• Pin threads (highlight important discussions)
• Lock threads (prevent further replies)

Admin vs Moderator

Capability	Moderator	Admin
View moderation queue	✅	✅
Moderate content	✅	✅
Delete user content	✅	✅
Pin/lock forum threads	✅	✅
Approve places	❌	✅
Approve articles	❌	✅
Manage users	❌	✅
Assign roles	❌	✅
Access audit logs	❌	✅
Manage taxonomy	❌	✅
Publish transparency reports	❌	✅

Error Codes

The moderation API returns specific error codes:

Code	Status	Description
AUTH_UNAUTHORIZED	401	User not authenticated
AUTH_FORBIDDEN	403	User lacks moderation permission
VAL_REQUIRED_FIELD	400	Missing required field
VAL_INVALID_ENUM	400	Invalid postType value
VAL_TOO_SHORT	400	Reason too short (< 5 chars)
BIZ_NOT_FOUND	404	Content not found
BIZ_ALREADY_MODERATED	400	Content already moderated
BIZ_SELF_MODERATION	403	Cannot moderate own content

Usage Example

Client-Side Request

async function moderateComment(commentId: string, reason: string) {
  const response = await fetch('/api/admin/moderate', {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
    },
    credentials: 'include',
    body: JSON.stringify({
      postType: 'blogComments',
      postId: commentId,
      reason: reason,
    }),
  });
  
  if (!response.ok) {
    const error = await response.json();
    throw new Error(error.error);
  }
  
  return await response.json();
}

// Usage
try {
  await moderateComment(
    'comment-uuid-123',
    'Contains spam links and inappropriate language'
  );
  console.log('Comment moderated successfully');
} catch (error) {
  console.error('Moderation failed:', error.message);
}

Success Response

{
  "success": true,
  "postType": "blogComments",
  "postId": "comment-uuid-123",
  "status": "moderated"
}

Error Response

{
  "error": "BIZ_SELF_MODERATION"
}

Audit Logging

All moderation actions should be logged to the audit log:

await db.insert(auditLog).values({
  id: randomUUID(),
  action: "content_moderated",
  userId: session.user.id,      // Moderator who took action
  targetId: postId,              // Content that was moderated
  data: {
    postType,
    reason,
    authorId,
    timestamp: new Date().toISOString(),
  },
});

Audit log captures:

• Who performed the moderation (userId)
• What was moderated (targetId, postType)
• Why it was moderated (reason)
• When it occurred (createdAt)

Future Enhancements

Planned moderation features:

Automated Moderation

• AI-based content filtering
• Automatic flagging of suspicious content
• Spam detection
• Profanity filters

Advanced Queue Management

• Priority levels for moderation queue
• Assignment of items to specific moderators
• SLA tracking for moderation response times
• Bulk moderation actions

Appeals Process

• Users can appeal moderation decisions
• Appeal review by senior moderators
• Audit trail of appeals and outcomes

Moderation Templates

• Pre-defined moderation reasons
• Quick action buttons
• Custom response templates

Reporting System

• User-initiated content reports
• Report categorization
• Report status tracking
• Reporter notifications

Best Practices

1. Always provide clear reasons

   When moderating content, provide specific, actionable reasons so authors understand what violated community standards.

2. Be consistent

   Apply moderation standards consistently across all users and content types to maintain fairness.

3. Review context

   Before moderating, review the full context of the content, including surrounding discussion and user history.

4. Document decisions

   Use the audit log to track patterns and ensure moderation actions are defensible.

5. Respond promptly

   Process moderation queue items quickly to maintain community trust and content quality.

6. Educate users

   Use moderation as an opportunity to educate users about community standards.

See Also

• User Roles & Permissions
• Authentication System
• API Overview