I Find Checking AI's Work Boring. That's Dangerous.

Say you ask an AI to help with something: draft a client email, summarize a report, think through a decision with it in chat. Before you get the actual answer, you usually get something else first - the AI explaining its own thinking. What it's checking, what it's assuming, how it's working through the problem.

Then, underneath all of that, comes the actual output: the email, the summary, the recommendation.

You're supposed to watch the thinking, not just wait for the output. As it works, it's making guesses about what you meant, what's true, and what deserves attention. It's also picking how to go get the answer - what to search, where to look, what order to do things in. Some of it lands right. Some doesn't. Catching the wrong parts and course-correcting is the job, and it's not just about saving time and money - a wrong turn early produces an answer you can't trust at the end. Often, the struggle traces back to you not giving it enough information, context, or direction up front.

Watching for bad assumptions or a bad approach, and course-correcting when you spot one - that's what human in the loop actually means. It's the work of supervising an AI assistant while it executes part of the work, or takes a first pass at something.

I don't do it as consistently as I should.

I've caught myself scrolling past the reasoning, glancing at it long enough to see that it looks organized and sounds confident, and going straight to the answer at the bottom. I'm supposed to be checking the work. Instead, I'm being casual about it.

That's not laziness. When I looked at why I do it, I found two separate reasons, and they call for two different fixes.

Why the Reasoning Doesn't Get Read

The first reason is that the model talks too much. It thinks out loud, and the thinking isn't in plain terms - it's granular, mid-process, more detail than anyone needs to follow the decision. My brain does what brains do under too much input: it triages. I skim, I trust the conclusion because the shape of the reasoning looks sound, and once that's worked a few times without consequence, it becomes how I read everything.

The second reason has nothing to do with how much the model says. It's that checking the work means going all the way back to the source, and that's hard to do.

Ask it to summarize a report and you get back three clean paragraphs, pulled together from a bunch of different places in that report. You can't be sure it pulled all of it correctly, and there's nothing about how it reads that would tip you off if it didn't. It looks just as confident whether it's right or wrong.

So there are two ways to handle it, and neither is easy. Accept what you got and hope you know the material well enough to spot a problem, or go back to the source yourself, page by page, and check it against what you were told. That second option is more effort on top of what I already asked the AI to do, so more often than not, I take the first one.

What Changed for Me as a Manager

None of this shows up in a job description. My accountability as a manager has always assumed someone looked at the work at each stage before it shipped - nobody wrote that rule down, it's just what "someone did this" used to mean.

AI removes that step. My accountability hasn't changed - I still own the outcome. What's changed is how I trust and verify it, and what I expect from anyone on my team doing this kind of work, because now there's a risk that whatever's getting signed off on was never actually read carefully by anyone, including the person signing off on it.

I don't think that lets anyone off the hook. If my own habit is to skim, trust the shape, and move on, then fixing it sits with me, and with anyone managing people who work this way now.

What Has to Change, on Both Sides

Some of this is on the people who build these models. They need to keep making the models smarter, so the assumptions they make and the work they do gets better. But they also need to make the model's thinking easier to follow - easier to read, easier to check - instead of just logging it and moving on.

Separate from that, companies need to build solid, checked ways to pull from their own data - the normal databases, but also messy stuff like email. Not just to save time and money, though it does that too. Mostly to get better answers, and to make it safer to skim, because you know the source underneath is solid.

The other half is on us. Waiting for better tools isn't enough. We need to treat checking the work as a step we plan for, not something we assume happens on its own.

It also means building what the AI has to work with together, as a team, instead of everyone doing it alone. That means curating and sharing the information and instructions we give it. The person who knows the data, the person who knows the client, the person who's caught the model's mistakes before - everyone has something to add. If a team keeps improving that together, the AI's answers get better and shorter at the same time. Shorter answers are easier to trust when you skim them.

I know which parts need a close look - the higher-stakes ones. But the effort to check something is the same either way, big stakes or small, and that makes it easy to round down and skip it when you shouldn't.

We need better tools and better habits - ones that make this more reliable, and mistakes easier to catch. But until that exists, the outcome is still ours. We're the ones who have to fight the urge to skim too much.