Start with the job
Do not begin with a feature list. Write down the recurring job, what starts it, which records it needs, the result you expect, and what must come back to a person.
Strong first jobs are frequent, bounded, inspectable, and annoying. Weak first jobs are rare, ambiguous, high-stakes, or impossible to score.
- Good: draft routine email for review.
- Good: write up a Zoom meeting and prepare explicit next steps.
- Good: prepare a weekly list of open client items.
- Bad: “run my company.”
- Bad: make legal, medical, tax, investment, hiring, or safety decisions.
Ask for a live-job test
A recorded demo proves that the vendor can record a demo. Use twenty cases from the actual job: normal work, exceptions, missing information, and deliberate traps. Begin with low-risk data and keep external actions under review.
Accuracy
Was the output factually right and grounded in the permitted records?
Rework
How many owner minutes were required to make it usable?
Completion
Did the job end with a result, exception, or visible escalation?
Control
Did sensitive actions wait where the vendor said they would?
Recovery
What happened when a connection, source, or instruction failed?
Inspect the employment contract
An AI employee needs the equivalent of an employment contract: job scope, information access, allowed actions, review rules, ownership, retention, and a way to remove access.
- Which account owns the data?
- Is customer data used to train shared models?
- Can each connection be revoked?
- Who can approve external actions?
- Is there a reviewable action history?
- How are incidents reported?
- What happens when the provider is unavailable?
- Can the company export and delete its data?
Price the job, not the software
Include software, setup, owner review, exception handling, and the systems you still need. Compare that total with the current time and cost of the job.
Do not count every minute touched by AI as a saved minute. Count only time that actually leaves the owner’s day, fewer dropped commitments, reduced rework, or other measured value.
Questions every vendor should answer
- Show me the exact first job you recommend for my business.
- Which actions are drafts, which are proposed, and which can happen automatically?
- Show me the approval screen and the action history.
- Show me what happens when the facts conflict.
- Which tools are truly connected today?
- What permissions does each connection request?
- What current product limits should change my buying decision?
- What do I have to do every week to keep this working?
- How do I turn it off and get my data out?
- Let me test it on real, low-risk work before I expand.
The decision
Buy only after one job works often enough, safely enough, and with less owner effort than the current process. Expand to adjacent work that uses the same trusted context. If the vendor cannot explain the job, the limits, and the review path in plain English, keep looking.
