New in Claude Code: Plugin Evaluation

New in Claude Code: Plugin Evaluation

Now you can see what actual benefits your plugin brings and whether it needs improvements.

How It Works

1. Create Test Cases: Run the plugin or skill, evaluate the results, and then rerun each test without the plugin and compare the difference.

2. Initialization: Start in your plugin folder and execute:

   BASH

   Скопіювати

   claude plugin eval init
   

   You explain to Claude what a good and bad result looks like and provide several real prompts. Claude generates test cases and checks on its own, runs a trial set, and reports how much a full run will cost.

3. Evaluation: Then execute:

   BASH

   Скопіювати

   claude plugin eval
   

   In the terminal, you will see the evaluation of each test case with your plugin and without it, as well as receive an HTML report with all the details. If your account supports it, the report is also published as a private artifact.

Important Points

• Checks invoke the model, so they consume tokens, and results may vary from run to run. Before a full run, first do a trial run with:

  BASH

  Скопіювати

  --runs 1
  

• Hooks and MCP servers of your plugin run on your behalf, so only check plugins you trust.

Try the New Feature

To try the feature, execute:

claude update

Documentation