Testing Your Integration

This guide covers what you should test when integrating Pillar into your application.

What You Control vs. What Pillar Controls

Before diving in, it's important to understand the testing boundary:

You should test your code. Pillar tests ours.

The agent's behavior—which tools it suggests, how it responds to queries—is Pillar's responsibility. We continuously test and improve it. Your job is to ensure your tools and handlers work correctly.

Testing Tool Handlers

Your handlers are the main thing to test. They should:

1. Return the correct format: { success: true, data: {...} } or { success: false, error: "..." }
2. Call the right functions in your app
3. Handle errors gracefully

Simple Handler Test

// tools.test.ts
import { describe, it, expect, vi } from 'vitest';
import { inviteUser } from '@/lib/team';  // Your existing function
import { handlers } from './tools';

vi.mock('@/lib/team');

describe('invite_team_member handler', () => {
  it('returns success when invite works', async () => {
    vi.mocked(inviteUser).mockResolvedValue({ id: 'invite-123' });
    
    const result = await handlers.invite_team_member({ 
      email: 'new@example.com',
      role: 'member',
    });
    
    expect(result).toEqual({
      success: true,
      data: { inviteId: 'invite-123' },
    });
    expect(inviteUser).toHaveBeenCalledWith('new@example.com', 'member');
  });

  it('returns error when invite fails', async () => {
    vi.mocked(inviteUser).mockRejectedValue(new Error('Invalid email'));
    
    const result = await handlers.invite_team_member({ 
      email: 'bad-email',
      role: 'member',
    });
    
    expect(result).toEqual({
      success: false,
      error: 'Invalid email',
    });
  });
});

What to Test

Validating Tool Definitions

Before syncing, verify your tools are well-formed:

// tools.validation.test.ts
import { describe, it, expect } from 'vitest';
import { tools } from './tools';

describe('tool definitions', () => {
  it('all tools have required fields', () => {
    tools.forEach((tool) => {
      expect(tool.name).toBeTruthy();
      expect(tool.description).toBeTruthy();
      expect(tool.type).toMatch(/^(navigate|trigger_tool|inline_ui|external_link|copy_text)$/);
    });
  });

  it('descriptions are specific enough', () => {
    tools.forEach((tool) => {
      // Descriptions should be at least 20 characters
      expect(tool.description.length).toBeGreaterThan(20);
      // Avoid vague descriptions
      expect(tool.description).not.toMatch(/^(Do something|Handle|Process)/i);
    });
  });

  it('navigate tools have paths', () => {
    const navigateTools = tools.filter((a) => a.type === 'navigate');
    navigateTools.forEach((tool) => {
      expect(tool.path).toBeTruthy();
    });
  });
});

Manual QA Checklist

The best way to test your Pillar integration is to use it. Run through this checklist after making changes:

Basic Functionality

• Panel opens and closes correctly
• Chat input accepts messages
• Responses appear without errors

Tool Testing

For each tool you've defined:

• Ask a question that should trigger the tool
• Verify the tool appears as a suggestion
• Click the tool and verify your handler runs
• Check that success/error states display correctly

Edge Cases

• Ask something unrelated to your tools—agent should answer from knowledge base or say it doesn't know
• Test with missing context (e.g., ask about a project when none is selected)
• Test permission-gated tools with different user roles

Example Test Queries

Using the Pillar Dashboard

After your integration is live, use the Pillar dashboard to monitor and improve:

Review Conversations

1. Go to Conversations in your Pillar dashboard
2. Filter by:
   • Negative feedback (thumbs down)
   • Abandoned conversations
   • No tool triggered

Identify Gaps

Look for patterns like:

• Users asking for tools that don't exist → define new tools
• Wrong tools being suggested → improve descriptions or add example phrases
• Users confused by responses → add to your knowledge base

Create Corrections

When you find a bad response:

1. Open the conversation detail
2. Click "Create Correction"
3. Explain what the right answer should be

Pillar uses corrections to improve future responses.

CI/CD Integration

Add a simple test step to catch obvious issues:

# .github/workflows/test.yml
name: Test Pillar Integration

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - name: Setup Node
        uses: actions/setup-node@v4
        with:
          node-version: '20'
      
      - name: Install dependencies
        run: npm ci
      
      - name: Run handler tests
        run: npm test -- --grep "tools"

Summary

Focus your testing effort on your handlers and tool definitions. Let Pillar handle the AI behavior—that's what you're paying us for.