> ## Documentation Index
> Fetch the complete documentation index at: https://docs.arcbeam.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Review Traces Together

> Collaborative workflows for team-based trace evaluation

Collections in Arcbeam enable collaborative review workflows, allowing teams to work together on evaluating AI/LLM outputs, data quality, and application traces. Team members can be assigned as reviewers, submit approvals or rejections, and discuss findings through comments.

## Overview

Collection reviews provide a structured workflow for quality assurance and peer review:

* **Assign reviewers** to collections
* **Submit reviews** with approval, rejection, or pending status
* **Add review comments** to provide context and feedback
* **Track review progress** with visual status indicators
* **Discuss findings** through collection-level comments

This feature is ideal for:

* Evaluating LLM output quality before production deployment
* Peer reviewing data quality improvements
* Collaborative debugging of RAG pipeline issues
* Team-based validation of trace collections

## Creating a Collection

Before you can review traces together, you need to create a collection. Collections are curated sets of traces organized around a specific theme or purpose.

### From the Traces Page

<Steps>
  <Step title="Navigate to traces">
    Go to your project's traces page to see all available traces.
  </Step>

  <Step title="Select traces">
    Use filters to find relevant traces, then select the ones you want to group together. You can filter by:

    * Date range
    * Search terms (e.g., "refund", "error")
    * Status codes
    * Model used
  </Step>

  <Step title="Create collection">
    Click **Create Collection** button with your selected traces.
  </Step>

  <Step title="Name and describe">
    Give your collection a descriptive name and explanation:

    * **Good**: "GPT-4 Response Quality - Week 12"
    * **Good**: "Refund Policy Errors - Dec 2024"
    * **Avoid**: "Collection 1" or "Test"

    Add a description explaining the purpose:

    ```
    "Evaluating production traces from the new checkout flow to ensure
    AI responses are accurate and helpful. Focus areas: refund policy
    clarity and return process guidance."
    ```
  </Step>

  <Step title="Save collection">
    Click **Save** to create the collection. It will now appear in your project's collections list.
  </Step>
</Steps>

### From Individual Traces

While viewing a specific trace:

1. Click **Add to Collection** button
2. Choose an existing collection or create a new one
3. The trace is immediately added to that collection

### Using Filters to Auto-Create Collections

Save time by creating collections directly from filter results:

<Steps>
  <Step title="Apply filters">
    Use the traces filter panel to find specific patterns. For example:

    * Date range: Last 7 days
    * Search: "refund"
    * Status: 200
  </Step>

  <Step title="Save as collection">
    Click **Save as Collection** to turn all matching traces into a collection.
  </Step>

  <Step title="Optional auto-sync">
    Choose whether new traces matching these filters should automatically be added to the collection.
  </Step>
</Steps>

### Collection Types and Use Cases

**Bug Investigation**

* Group all traces related to a specific bug
* Example: "Checkout calculation errors - Issue #234"

**Feature Testing**

* Collect traces showing a new feature in action
* Example: "Multi-language support beta testing"

**User Feedback Review**

* Traces where users gave negative feedback
* Example: "Thumbs down responses - December"

**Topic Analysis**

* All traces about a specific subject
* Example: "Pricing and billing questions"

**Model Comparison**

* Compare different models or prompts
* Example: "GPT-4o vs GPT-4o-mini - Customer Support"

## Adding Reviewers to a Collection

<Steps>
  <Step title="Navigate to the collection">
    Open the collection you want to add reviewers to from your project's collections list.
  </Step>

  <Step title="Click the assignees section">
    In the collection header, you'll see an assignees area showing current members. Click the "+ Add" button or assignee avatars to manage members.
  </Step>

  <Step title="Select team members">
    Choose one or more team members from your organization to add as reviewers.
  </Step>

  <Step title="Members are added">
    Added members will appear with their status set to "Pending" until they submit their review.
  </Step>
</Steps>

<Note>
  Only collection creators can add or remove members from collections.
</Note>

## Submitting a Review

Once assigned to a collection, you can submit your review:

1. **Review the traces**: Navigate through the collection's traces to evaluate the quality, accuracy, and performance

2. **Add comments**: Use the Comments tab to discuss specific findings or ask questions

3. **Submit your review**: Click your avatar in the assignees section and select one of:
   * **Approve**: The collection meets quality standards
   * **Request changes**: Issues found that need addressing
   * **Mark as pending**: Still reviewing or awaiting additional information

4. **Add a review comment**: Provide context for your decision to help the team understand your reasoning

### Review Status Indicators

Collection members are displayed with visual indicators:

* 🟢 **Green checkmark**: Review approved
* 🔴 **Red X**: Changes requested or rejected

## Review Workflow Example

Here's a typical review workflow for evaluating LLM outputs:

<Steps>
  <Step title="Create collection">
    A data scientist creates a collection called "GPT-4 Response Quality - Week 12" containing 50 production traces.
  </Step>

  <Step title="Assign reviewers">
    Two team members are added as reviewers: a senior engineer and a domain expert.
  </Step>

  <Step title="Review traces">
    Each reviewer examines the traces, looking for hallucinations, off-topic responses, or formatting issues.
  </Step>

  <Step title="Add feedback">
    Reviewers add comments on specific traces they find problematic:

    > "Trace #127 shows hallucination - the model cited a non-existent research paper"

    > "Overall quality looks good, but 3 traces had formatting inconsistencies"
  </Step>

  <Step title="Submit reviews">
    The senior engineer approves with a comment: "Minor issues, but acceptable for production"

    The domain expert requests changes: "Need to address the hallucination in trace #127 before deploying"
  </Step>

  <Step title="Address feedback">
    The data scientist filters out the problematic trace, adjusts the prompt, and updates the collection.
  </Step>

  <Step title="Re-review">
    After fixes, the domain expert changes their review to approved.
  </Step>
</Steps>

## Managing Collection Members

### View All Members

To see all members assigned to a collection:

1. Open the collection detail page
2. The assignees section shows all current members with their review status
3. Hover over any member to see their role and review comment (if provided)

### Remove a Member

To remove a member from a collection:

1. Click on the member's avatar in the assignees section
2. Select "Remove from collection"
3. Confirm the removal

## Using Comments for Discussion

The Comments tab in a collection provides a threaded discussion space for reviewers:

### Adding a Comment

1. Navigate to the **Comments** tab in the collection
2. Type your comment in the text field (max 1000 characters)
3. Press **Cmd+Enter** (Mac) or **Ctrl+Enter** (Windows) to submit
4. Your comment appears with your name and avatar

### Comment Best Practices

* **Be specific**: Reference trace IDs when discussing specific issues
* **Provide context**: Explain why something is problematic or noteworthy
* **Ask questions**: Use comments to clarify requirements or get additional input
* **Link to traces**: Mention trace IDs so team members can easily find what you're discussing

Example good comment:

```
Reviewing traces 45-67, I noticed the retrieval quality dropped significantly.
The similarity scores are < 0.5 compared to our usual 0.7+ range.

@sarah Should we adjust the embedding model parameters or is this expected
for this data source?
```

## Collection Status Management

Collections have four status levels that help teams track progress:

* **Open**: Collection is actively being reviewed
* **In Progress**: Review is underway but not yet complete
* **Completed**: All reviews submitted and any issues resolved
* **Closed**: Final status, collection archived

Change the collection status from the collection detail page to reflect the current stage of review.

## Tips for Effective Reviews

<Tip>
  **Batch similar traces**: Group traces with similar patterns into collections for more efficient review. Instead of reviewing 500 random traces, create focused collections like "Error Cases - Week 12" or "High-Cost Queries - Dec 2024".
</Tip>

<Tip>
  **Set review deadlines**: Communicate expected review turnaround times to your team, especially for production-critical evaluations.
</Tip>

<Tip>
  **Review incrementally**: For large collections, assign different sections to different reviewers to parallelize the work.
</Tip>

## Next Steps

* Learn how to [organize with projects](/v0/setup/organize-with-projects)
* Explore [observability features](/v0/core-concepts/observability)
* View all your [project traces](/v0/debugging/find-problematic-traces)