The Specificity Problem: Why Generic AI Coaching Fails

A few months ago I was reviewing a conversation log from one of the AI coaching platforms that had been in the market for about two years. The user was a senior commercial director, 47 days into a new role, and she had asked the coach a question about how to handle a board member who was undermining her authority in cross-functional meetings. The coach's response was technically correct, broadly applicable, and completely useless. It gave her a framework for managing upward relationships. It suggested she "build trust through consistent delivery." It recommended she "seek to understand the board member's perspective." Every sentence was defensible. Not one sentence was specific enough to act on.

The coach had failed her — not because the underlying model was unsophisticated, but because the knowledge it was drawing on was generic. It knew about managing upward relationships in the abstract. It did not know anything about the specific dynamics of a commercial director navigating a newly-formed executive team, the particular political landscape of a board member who had backed a different internal candidate, or the neurological reality of trying to establish authority while your prefrontal cortex is still recalibrating to a new environment. The gap between what she needed and what she received was not a technology gap. It was a knowledge gap.

The Assumption Nobody Questions

The dominant assumption in the AI coaching market is that the quality of a coaching interaction is primarily a function of the quality of the underlying language model. Better model, better coaching. This is the logic that drives the arms race between providers and the breathless coverage of each new model release.

It is also wrong, in a way that matters practically.

Language models are extraordinarily good at pattern-matching across vast bodies of text. They can identify the general shape of a problem, retrieve relevant frameworks, and construct plausible responses with remarkable fluency. What they cannot do — what no amount of model sophistication can compensate for — is apply knowledge they do not have.

The quality of an AI coaching interaction is not primarily determined by the model. It is determined by the specificity and depth of the domain knowledge the model has been given to work with. A frontier model with generic coaching knowledge will produce generic coaching. A mid-tier model with deep, precisely-structured domain expertise will produce coaching that is genuinely useful to the person asking.

This is not a controversial claim once you examine it. But it is consistently overlooked because the technology is more visible than the knowledge architecture behind it.

What Generic Looks Like in Practice

Generic AI coaching has a recognisable signature. It is warm, broadly applicable, and leaves the user with the feeling that they have had a conversation without quite having had an answer. The frameworks it offers are real — they exist in the literature, they have been validated in research — but they are calibrated for the average case, not the specific one.

Consider the difference between a question about managing performance anxiety and the same question asked by a senior barrister preparing for a high-profile case, a commercial director presenting to a sceptical board for the first time, and a professional athlete returning from injury. The surface question is identical. The underlying mechanisms, the relevant evidence base, the practical interventions, and the psychological context are entirely different.

Generic coaching treats these as the same question. Domain-specific coaching treats them as three distinct problems that happen to share a label.

The reason this matters is not merely academic. When someone is 60 days into a new role and their confidence is eroding in ways they cannot fully articulate, they do not need a framework for building confidence. They need someone — or something — that understands the specific neurological mechanisms of confidence erosion under transition conditions, the particular way that imposter syndrome manifests in high-achieving executives who have moved up rather than across, and the difference between interventions that address the symptom and those that address the mechanism.

Generic coaching cannot provide this. Not because the technology is inadequate, but because the knowledge was never there to begin with.

The Knowledge Architecture Problem

Building genuinely domain-specific AI coaching is harder than it appears, and the difficulty is not where most people look for it.

The technical challenge — fine-tuning a model, constructing a retrieval system, designing a conversation interface — is tractable. The hard problem is the knowledge architecture: the process of taking what a genuine domain expert knows, after decades of practice, and encoding it in a form that an AI system can use to produce specific, accurate, contextually-relevant responses.

Expert knowledge is not stored in bullet points. It is stored in patterns of recognition — the ability to look at a situation and immediately identify which of several superficially similar problems is actually present, which intervention is appropriate, and which well-intentioned approaches will make things worse. This pattern recognition is built through thousands of hours of practice and is largely tacit. The expert often cannot fully articulate what they know. They know it through doing.

Translating this into a knowledge base that an AI system can use requires a different kind of intellectual work. It requires extracting not just the frameworks — which are usually documented — but the clinical judgment that determines when each framework applies, the contraindications that most practitioners learn through painful experience, and the specific language that resonates with the particular population being served.

Most AI coaching platforms have not done this work. They have taken general coaching frameworks, added a conversational interface, and called the result domain-specific because it is deployed in a particular context. The knowledge architecture is generic. The domain specificity is cosmetic.

Why This Produces a Particular Kind of Failure

The failure mode of generic AI coaching is not dramatic. It does not give obviously wrong advice. It does not contradict established frameworks. It fails in a more insidious way: it gives advice that is correct in general but unhelpful in particular.

This is actually more dangerous than obvious error, because it is harder to identify. When advice is clearly wrong, the recipient rejects it. When advice is plausible but misses the specific mechanism, the recipient often accepts it, acts on it, and then attributes the lack of results to their own failure to implement correctly rather than to the inadequacy of the advice itself.

I have seen this pattern repeatedly in organisations that have deployed generic AI coaching tools at scale. The usage data looks reasonable — people are engaging with the system, having conversations, completing sessions. The outcome data tells a different story. The specific problems that brought people to the system in the first place — the performance anxiety, the relationship difficulty, the confidence erosion — are not resolving at the rate they should. The coaching is happening. The change is not.

The explanation is almost always the same. The system is providing general-purpose responses to specific problems. The gap between what the user needs and what the system can provide is invisible to both parties, because neither has a clear picture of what genuinely domain-specific coaching would look like.

What Specificity Actually Requires

Genuine domain specificity in AI coaching requires three things that most platforms do not have.

The first is deep practitioner knowledge — not the knowledge that appears in textbooks or training programmes, but the knowledge that experienced practitioners carry about what actually works, in which circumstances, for which populations, and why. This knowledge is typically held by a small number of people who have spent decades working in a specific niche and have developed a level of pattern recognition that is genuinely difficult to replicate.

The second is structured knowledge encoding — the process of taking that practitioner knowledge and organising it in a way that an AI system can use to produce specific, contextually-relevant responses. This is not a matter of writing a long system prompt. It requires a structured knowledge base that covers the mechanisms underlying the domain, the specific presentations that practitioners encounter, the interventions that work and those that do not, and the clinical reasoning that connects situation to response.

The third is domain-calibrated language — the specific vocabulary, framing, and conceptual architecture that resonates with the particular population being served. A coaching interaction with a senior executive requires different language than one with a mid-career professional. A coaching interaction focused on performance anxiety in high-stakes presentations requires different framing than one focused on the same anxiety in a different context. Generic language produces generic resonance.

The Implication for Anyone Choosing an AI Coach

The question to ask of any AI coaching platform is not "which model does it use?" The question is "what does it actually know about my specific situation?"

This means asking about the knowledge architecture behind the system — not the technology, but the expertise. Who built the domain knowledge? How was it structured? What specific population was it designed to serve? How does the system distinguish between superficially similar problems that require different interventions?

Most platforms will struggle to answer these questions in specific terms. That difficulty is itself informative. It suggests that the knowledge architecture has not been the primary focus of development — that the investment has gone into the technology rather than the expertise that makes the technology useful.

The commercial director who needed help with her board member did not need a better language model. She needed a system that had been built by someone who had spent years working with senior executives navigating exactly that kind of political complexity — someone who understood the specific mechanisms, the specific failure modes, and the specific interventions that work when the conventional approaches do not.

The gap between what she received and what she needed was not a technology gap. It was a knowledge gap. And in AI coaching, as in human coaching, that is the gap that matters.