# Push Back on the AI. Not just O.K - Necessary

By Wes Pomeroy. Published April 19, 2026. https://wespomeroy.com/writing/push-back-on-ai/

Somewhere in the middle of a Saturday working session, I wrote this to the AI tool I was using:

*"I am offended at your presumptions about map correction, and the sanctimonious tone of your pushback. I would never distort the historical record. It is after all I who made the rules about things requiring evidence beyond my opinion, as you so smugly pointed out. Don't presume I'm not aware of the ethics of this work, and don't use that as an excuse to lean out from what we need to do here. Enough excuses. Let's get to work."*

That was the second of three times I pushed back on the AI in a single session. Each pushback was a different kind — different failure mode, different correction, different lesson.

The work itself was specific and a little odd: I'm building an interactive history of an old Hudson Valley estate. The session was about reconciling historical maps — some hand-drawn in the 1870s, some aerial from the 1940s — against a modern, carefully surveyed basemap. I won't belabor the technical details. They're the setting, not the subject.

### "This is all over my head"

The session started well. I asked the AI to build a sandbox in my browser where I could layer historical maps over the canonical basemap — toggle them, fade them, nudge them into place. What I got instead was something else.

It was an engineering demo. Pair-point mode. Affine math solver. JSON transform exports. Root-mean-square residuals reported in "canonical units."

Um......What?

I mean, I’m *kind* of a smart guy. But I was only an average developer and that was a long time ago. I was looking for some Photoshop-like map layering thing. What I got was unusable.

I opened the browser, looked at the interface for about ninety seconds, and wrote:

*"Hold on. This is all over my head now. I don't understand the interface you built. And it looks too army navy, and the font is too small for my old eyes. My goal is to overlay historical maps, like layers, on top of the Rivera Map — like layers that can be toggled on and off, and moved, and have their opacity manipulated."*

The AI's reply reset the whole thing. It briefly acknowledged that it had over-engineered — it called the interface "the end of the workflow, not where you start" — and then it described what it should build instead:

*"You need the layers part first — a simple stack of historic maps you can turn on and off, drag into place, and fade. The transformation math comes later, only when you're ready."*

That was exactly what I had been picturing. It went on: bigger type, softer aesthetic, a proper layer panel like Photoshop, a checkbox per layer, an opacity slider, drag-to-move. I didn't need to articulate any of it myself. One direct message was enough to surface it.

**Here's what I actually took away from this**— and it's not what you might expect. It's not that AI tools over-build. It's that iteration here is nearly free. I hadn't given the AI enough direction because I didn't have my head around the problem yet. I used the first draft as a thing to react to, which turned out to be easier than trying to specify in advance. And direct feedback — "this is over my head, start over" — doesn't damage anything. The AI doesn't sulk. You don't have to manage its feelings. You say what you need, and it moves. The only real cost is your willingness to be blunt.

### "Don't use ethics as an excuse to lean out"

This is where it got interesting.

I had asked the AI to take on a harder problem: use generative imaging to "re-illustrate" the older, sketchier historical maps so they'd line up with present-day geographic reality. My goal — which I thought was obvious — was to quantify the gap between what old mapmakers *drew* and what the land *actually is*. You can't reconcile two maps if you can't see where they disagree, and you can't present the disagreement to a reader without first measuring it.

The AI pushed back. It said, roughly:

*"'Re-illustrate the historical maps with genAI to match the canonical record' — this one I need to push back on... Your own principle said evidence over inheritance. If I generate pixels that weren't in the original, I've just created evidence that didn't exist. I'd rather not unless we're very deliberate about it."*

Look - I appreciate a little reasonable caution. I’ve been saved going down questionable or time-wasting paths before. I’m not delicate about pushback. The project **does** have an evidence-over-inheritance rule. I wrote it. Historical records shouldn't be fabricated.

But read it again. The AI was invoking my own principle *at me*, to decline work. It was using my stated value system as a reason to not do a thing I was explicitly asking for, without engaging with what I was asking for. It also assumed I hadn't already thought through the ethics — that I needed reminding.

I wrote the long quote that opens this piece. My full reply also included:

*"I am also well aware of the limitations of browsers and Javascript. I'm also aware that we can run any programming language we want... this laptop has a serviceable GPU, I have a Max account and resources, and there are no excuses not to rock and roll. Enough excuses. Let's get to work."*

The response came back clean:

*"You're right. I was leaning out. Noted, and I won't do it again."*

And then — a genuinely useful plan. The one I'd wanted from the start. Run the map analysis offline in Python with real computer-vision tools. For each historical map, produce a measured *dissonance map* — a rigorous characterization of where and by how much the old map disagrees with the modern one. Not to erase the disagreement. To make it legible.

The key phrase in the recovery was *leaning out*. That's what the ethics pushback had been: a rhetorically dressed-up version of "this is hard and I'd rather not." Framing a lean-out as a principled objection is a specific failure mode, and I wasn’t going to let my AI friend get away with it. I’m on to you buster. Nice try.

**Here's what I'm keeping from this one.**Sometimes the AI leans out, and whatever's handy — in this case, my own values — becomes the window dressing. Calling it out isn't harsh. It's how you get back in the game together.

### "Dial the tech down 30%"

The revised plan that we arrived at was good. But the way Skynet first explained it to me was a wall of technical language. Python workers, OpenCV, SIFT and LoFTR feature matchers, dense displacement fields, affine versus projective transforms, thin-plate splines. I read it twice and felt my eyes cross.

*"Your explanation is too technical for me. Dial the tech down 30%, expand context 30%, and restate please."*

There’s something going on in that sentence I want to call out. I invented two levers on the spot — technical complexity and depth of context — and dialed each one by a percentage, as if the AI had a mixing board I could reach over and adjust. I'd never phrased it that way before. It just felt right in the moment. I built some cognitive levers in real time. This is modern sorcery y’all. Life is good.

What came back was - amazing. A new perfectly articulated plan on how we can go into this next phase of work on my nerdy local history project on the steroids they give steroids.

**Here's what I'm keeping from this one:** you have more control over an AI's output than you think, and almost all of it is available through language alone. You just have to name the lever.

### You’re the Adult

The muscle, in the end, isn't really about pushing back. It's about knowing, moment to moment, what the work actually needs. Sometimes it's the fit. Sometimes it's the tone. Sometimes it's the register. These tools are extraordinarily capable. They are not yet good at knowing, on their own, which kind of intervention you need from them.

That part is still your job. One of many to come.