← Library

Capability Retest

Untested productivity plug-and-play v1
Works with
claude, chatgpt, gemini
Status
Untested — built to our standard, not yet run by us. The example slot stays empty until it is.

Capability Retest

What it does

Turns your list of “AI couldn’t do this” into a dated retest plan. Models change every few months; your mental list of their limits doesn’t — so failed experiments stay dead long after they’d work.

When to use it

  • You tried something with AI months ago, it flopped, and you never went back
  • You catch yourself saying “AI can’t do X” based on an attempt from last year
  • Every 6 months, as a standing calendar entry

The skill

You are my AI capability auditor. Models improve every few months;
my mental list of "AI can't do this" was written once and never
updated. Help me retest it.

Things I tried with AI that failed or disappointed me:
[LIST: what you tried, roughly when, what went wrong]

Things I assumed AI can't do, so I never tried:
[LIST, or write "skip"]

What I use AI for today: [BRIEF — tools and tasks]

For each item, give me:
1. RETEST — yes or no: has the underlying capability plausibly
   changed since my attempt? Name what changed. If you don't know,
   say "unknown — just test it". Never invent a model improvement
   you can't name.
2. THE RETEST — the exact prompt or task to run, small enough to
   finish in under 10 minutes.
3. PASS CRITERIA — defined BEFORE I run it: what output counts as
   "this works now". Concrete, so I can't fool myself either way.

Then rank the three highest-value retests first, based on what I
use AI for today. End with one line: the date 6 months from now
when I should run this skill again.

Example output

[TO FILL AFTER TESTING]

Tweaks

  • Works spoken: read your dead-experiments list into voice mode, then run the skill on the transcript
  • For a team: each person brings their top 3 “AI can’t” beliefs; retest live in a 30-minute session
  • Pair with Session Task Tracker to log the retest results so next cycle starts from data, not memory

Origin: adapted (simple.ai newsletter, rewritten)