Code Execution API

Overview

The Canvas MCP Code Execution API enables you to perform bulk operations on Canvas data with minimal token usage. Instead of loading all submissions into Claude's context, the code executes locally and returns only summaries.

Token Savings Comparison

Scenario: Grading 90 Jupyter notebook submissions

Traditional Approach (Inefficient)

The traditional approach loads all submissions into Claude's context:

// Load ALL submissions into context
const submissions = await list_submissions({
  courseIdentifier: "60366",
  assignmentId: "123"
});
// → 90 submissions × 15K tokens each = 1.35M tokens!

// Process each one (more tokens!)
for (const sub of submissions) {
  await grade_with_rubric({
    courseIdentifier: "60366",
    assignmentId: "123",
    userId: sub.userId,
    rubricAssessment: { ... }
  });
}

Why This Is Inefficient

• ✗ All 90 submissions loaded into Claude's context
• ✗ ~1.35M tokens consumed
• ✗ Slow execution (sequential processing)
• ✗ Risk of hitting token limits
• ✗ Expensive for large classes

Code Execution Approach (Efficient)

The code execution API processes data locally:

import { bulkGrade } from './canvas/grading/bulkGrade';

await bulkGrade({
  courseIdentifier: "60366",
  assignmentId: "123",
  gradingFunction: (submission) => {
    // ⭐ This function runs LOCALLY
    // ⭐ Submissions never enter Claude's context!

    const notebook = submission.attachments?.find(
      f => f.filename.endsWith('.ipynb')
    );

    if (!notebook) {
      console.log(`No notebook for user ${submission.userId}`);
      return null; // Skip this submission
    }

    // Download and analyze notebook (locally!)
    const analysis = analyzeNotebook(notebook.url);

    if (analysis.hasErrors) {
      return {
        points: 0,
        rubricAssessment: {
          "_8027": {
            points: 0,
            comments: `Found errors: ${analysis.errors.join(', ')}`
          }
        },
        comment: "Please fix errors and resubmit."
      };
    }

    return {
      points: 100,
      rubricAssessment: {
        "_8027": { points: 100, comments: "Excellent work!" }
      },
      comment: "Great submission!"
    };
  }
});

Why This Is Efficient

• ✓ Only ~3.5K tokens total (99.7% reduction!)
• ✓ Data processed locally in execution environment
• ✓ Faster execution (can process concurrently)
• ✓ No token limit concerns
• ✓ Scales to 1000+ submissions easily

Bulk Grading Example

Output Format

Starting bulk grading for assignment 123...
Found 90 submissions to process

✓ Graded submission for user 12345
✓ Graded submission for user 12346
Skipped submission for user 12347 (no notebook)
✓ Graded submission for user 12348
✗ Failed to grade user 12349: Network timeout
...

Bulk grading complete:
  Total: 90
  Graded: 87
  Skipped: 2
  Failed: 1

Advanced Custom Analysis

await bulkGrade({
  courseIdentifier: "60366",
  assignmentId: "123",
  gradingFunction: (submission) => {
    const notebook = submission.attachments?.find(
      f => f.filename.endsWith('.ipynb')
    );

    if (!notebook) return null;

    // Custom analysis logic
    const analysis = {
      cellCount: countCells(notebook),
      hasDocstrings: checkDocstrings(notebook),
      passesTests: runTests(notebook),
      codeQuality: analyzeCodeQuality(notebook)
    };

    // Complex grading rubric
    let points = 0;
    const rubricComments = {};

    // Criterion 1: Functionality (50 points)
    if (analysis.passesTests) {
      points += 50;
      rubricComments["_8027"] = {
        points: 50,
        comments: "All tests pass!"
      };
    }

    // Criterion 2: Documentation (30 points)
    const docPoints = analysis.hasDocstrings ? 30 : 15;
    points += docPoints;

    // Criterion 3: Code Quality (20 points)
    const qualityPoints = Math.min(20, analysis.codeQuality * 20);
    points += qualityPoints;

    return { points, rubricAssessment: rubricComments };
  }
});

Bulk Discussion Grading

Grade discussion posts with initial post + peer review requirements:

import { bulkGradeDiscussion } from './canvas/discussions/bulkGradeDiscussion';

// Preview grades first (dry run)
await bulkGradeDiscussion({
  courseIdentifier: "60365",
  topicId: "990001",
  criteria: {
    initialPostPoints: 10,      // Points for initial post
    peerReviewPointsEach: 5,    // Points per peer review
    requiredPeerReviews: 2,     // Must review 2 peers
    maxPeerReviewPoints: 10     // Cap at 10 pts for reviews
  },
  dryRun: true  // Preview first!
});

// Then apply grades
await bulkGradeDiscussion({
  courseIdentifier: "60365",
  topicId: "990001",
  assignmentId: "1234567",  // Required to write grades
  criteria: { ... },
  dryRun: false
});

Features

• Automatically analyzes initial posts vs peer reviews
• Configurable grading criteria with point allocation
• Optional late penalties with customizable deadline
• Dry run mode to preview grades before applying
• Concurrent processing with rate limiting

Discovering Available Tools

Use the search_canvas_tools MCP tool to discover available operations:

// Search for grading-related tools
search_canvas_tools("grading", "signatures")

// List all available tools
search_canvas_tools("", "names")

// Get full implementation details
search_canvas_tools("bulk", "full")

Natural Language Discovery

Ask Claude:

• "Search for grading tools in the code API"
• "What bulk operations are available?"
• "Show me all code API tools"

Code API File Structure

src/canvas_mcp/code_api/
├── client.ts              # Base MCP client bridge
├── index.ts               # Main entry point
└── canvas/
    ├── assignments/       # Assignment operations
    ├── grading/          # Grading operations
    │   ├── gradeWithRubric.ts
    │   └── bulkGrade.ts  # ⭐ Bulk grading
    ├── discussions/      # Discussion operations
    │   └── bulkGradeDiscussion.ts
    ├── courses/          # Course operations
    └── communications/   # Messaging operations

Dry Run Mode (Testing)

Always test your grading logic before actually grading:

await bulkGrade({
  courseIdentifier: "60366",
  assignmentId: "123",
  dryRun: true,  // ⭐ Test mode - doesn't actually grade
  gradingFunction: (submission) => {
    console.log(`Would grade: ${submission.userId}`);
    return { points: 100, ... };
  }
});

Best Practices

1. Always test with dry run first before grading for real
2. Handle errors gracefully - return null to skip problematic submissions
3. Provide detailed rubric comments to help students understand their grades
4. Log progress using console.log() to track grading status
5. Validate rubric criterion IDs before grading

Common Rubric Criterion ID Patterns

Canvas rubric criterion IDs typically start with underscore:

• "_8027" - Common format
• "criterion_123" - Alternative format
• "8027" - Without underscore (rare)

Troubleshooting

"No exported function found"

• Check that your TypeScript files have export async function declarations
• Verify file paths are correct

"Criterion ID not found"

• Use list_assignment_rubrics to get correct criterion IDs
• Remember: IDs often start with underscore ("_8027")

"Rate limit exceeded"

• Add delays between grading operations
• Reduce maxConcurrent parameter (default: 5)

"Submission not found"

• Check that courseIdentifier and assignmentId are correct
• Verify students have actually submitted