Picture an architecture review. Two teams walk in, each with a polished deck. One argues the billing system should be the source of truth for customer data. The other argues it should be the CRM. Both decks are clear, well structured, full of good reasons. Both were put together with help from AI.

Same tool. Same kind of data. Opposite answers. So which one was right?

The tool did not decide. Each team decided the moment they typed the question, and AI did what it does best. It helped them make their case.

The problem lives at the edges

I use AI as a thought partner a lot, including for Enterprise Architecture problems. It is genuinely good at it. It takes a messy idea and hands it back clean, organized, ready to present.

And that is exactly what makes me uneasy. Clean and right feel the same.

Ask "how do we make this team's approval process faster?" and you get a solid plan. Automate the handoffs, add a dashboard, cut a step or two. All reasonable. But maybe that approval only exists because two systems do not share customer information, and people are checking by hand what the systems should be telling each other. Make it faster and you have not fixed anything. You have just moved bad data downstream quicker.

AI did not miss that because it is not smart enough. It missed it because it was never in the question. It works inside the frame you give it, and in Enterprise Architecture the problem is almost always at the edge of that frame. Upstream, downstream, the team nobody invited to the meeting.

The answer can be technically correct and strategically incomplete.

The question it rarely asks back

The most useful question I know in this work is what are you actually trying to solve? It is the question that pulls you out of the task and back into the bigger picture.

AI almost never asks it unprompted. You ask for a solution, it gives you a solution. It is like a colleague who is brilliant, fast, and always agrees with you. Nice to work with. Not much of a thought partner.

So when was the last time AI told you that you were solving the wrong problem, without you asking it to?

Make it think like the enterprise

The fix is not to stop using it. The fix is to stop asking it to agree. Before taking an answer into a review, I try questions like these:

  • Who else is affected by this that I have not mentioned?
  • What happens upstream and downstream if we do this?
  • Make the strongest case for the other team's option.
  • What problem might this process be compensating for?
  • What question should I be asking instead?

It feels a little uncomfortable the first time. That is kind of the point.

AI has no stake in the decision. It will not be in the room three years later when the integration breaks, or when two systems both claim to own the customer. The architect will. Which means the thinking before the question, and the judging after the answer, are still ours.

If it only ever agrees with you, you are not thinking with a partner. You are thinking with a mirror.