Your Junk Drawer of Knowledge Isn't Ready for AI

Most banks and credit unions are about to spend a year and a budget rolling out AI that confidently gives wrong answers. Not because the AI is flawed, but because of what they're feeding it.

Adopting AI is like shining a giant spotlight on the state of knowledge management in your organization. Procedures that were outdated, unclear, or incorrect used to go by relatively unnoticed. But with AI, that knowledge starts getting used. What some call hallucinations is simply a case of garbage in, garbage out.

Imagine your employees are in a desert looking for water. There is an underground water source, but they can't reach it. The water represents your company's policies, procedures, and job aids – your Operational Knowledge. The chatbot is a pipe that they can put in the ground to access the water.

The pipe (the chatbot) doesn't clean the water. It just gives employees access to it. If the water is dirty or contaminated it will taste bad or make the employees sick, and they will stop using it. They may blame the pipe (the AI), but the pipe can't change the water it is accessing.

If we really want to quench their thirst, we need to make sure the water source is clean and free of contaminants.

Too many organizations point the AI pipe at their junk drawer of knowledge (the dirty water) and then say the AI is flawed or hallucinating. The problem isn't the pipe.

The Challenge With Complex Procedures

But not every task is a quick drink. An open tap is perfect when an employee just needs a fast answer. A complex, multi-step procedure is different. Point them at the chatbot and it's like drinking from a fire hose: a wall of endless text to read, interpret, and sequence while a customer sits in front of them. What they need is the opposite – just the right amount of information at the exact moment they need it, so they can stay focused on the customer instead of the screen. That's not a job for an open pipe. It's a job for a guided system that releases the right step at the right time.

Chatbot owners spend all their time trying to adjust prompts to get the right balance between detail and usability for complex procedures without ever asking if a chatbot is the right delivery vehicle for this level of complexity.

Moving Beyond Answers to Guidance

To move forward, financial institutions need to stop focusing on what AI "can do" and more on the end results they want to achieve with AI. In our sphere, this comes down to employee performance – can frontline employees answer questions, perform tasks, and solve problems confidently and independently? Can they trust the water source and drink the water without being blasted by a fire hose?

We call this the Moment of Performance. If AI can't empower the moment of performance for frontline employees, it will fail to impact employee ramp times, compliance, and productivity.

Empowering the "Moment of Performance"

In order to empower employees in this moment, we need two things:

A validated source of truth for policies, procedures, and job aids

A translation layer that converts that knowledge into performance-ready resources

The validated source of truth is the clean water. The translation layer makes sure that we get just the right amount of water for the situation, that we are not blasting people with a fire hose of information.

For example, consider the procedure that financial institutions need to follow when a customer passes away and the family wants to transfer ownership of the account. We've seen these run 12 to 35 pages. If we present this in a chatbot, then the employee who's trying to follow it is going to be overwhelmed with information. We've seen teams that try to simplify this, but the problem isn't just the length. It's the format.

Creating a Validated Source of Truth

Here's what surprises people: cleaning the water doesn't have to take a year. Most of the institutions we talk to assume it will, and for most of them it does, but not because the work is inherently slow. It's because they start without a framework. With no system for deciding what to keep, spotting what's missing, and preparing each procedure for AI, every document turns into its own debate. The teams that move fast aren't working harder. They're working from a repeatable process (we wrote a book on this called, "Find and Follow"). With one in place, we've taken teams through this in as little as 90 days.

The essential concepts are that you separate out foundational and actionable knowledge. Foundational knowledge is definitions, the what and the why behind something, while actionable knowledge is the how. When you separate these two things, you end up with content that is much easier for subject-matter experts to write, much easier for them to review and approve, and much easier to maintain. Foundational knowledge will rarely change, while actionable knowledge will change all the time. When organizations mix these two types of information, they create a content-maintenance nightmare and decrease the effectiveness of their documentation.

By using a proven framework and AI to separate foundational from actionable knowledge and turn actionable knowledge into recipes instead of encyclopedias, we've helped teams dramatically accelerate the time it takes to complete a project like this.

Translating Knowledge Into Performance-Ready Resources

When employees are interacting with customers, they are not working in an ideal environment. They have:

The stress of their normal lives

The pressure of someone either in front of them or on the phone

The cognitive overload of trying to remember everything they have been trained to do

This all gets worse when it's a situation they don't deal with often. In this moment, we need to think of what they are experiencing and how we can set them up for success.

Lengthy chatbot responses won't help here. What we really need is something that is findable, followable, and scannable.

Employees need to be able to find the resource in five seconds or less.

They need to be able to follow it without turning to a coworker or supervisor for assistance.

They need to be able to scan it while they are interacting with the customer without making the customer wait while they read lengthy instructions.

When it comes to complex procedures, such as IRAs, deceased customers, wire transfers, and beneficiary changes, a decision tree is the ideal format to enable this type of independence and performance.

When we design a decision tree, we can design it such that the employee never needs to read too much information at once. They have visuals that will guide them through and simple answers that they can respond to, prompting them with what to say and what to do.

Instead of using AI to create these decision trees on the fly, which can lead to compliance issues when hallucinations happen, we can use AI to assist subject matter experts in designing decision trees. Now, when an employee brings up the decision tree, they know that they're using something that is human-approved and that will be compliant with their procedures. They don't have to worry about trusting the information.

The Impact – Ramp Time, Compliance, and Supervisor Bandwidth

When you get a validated source of truth and have a translation layer that turns knowledge into performance-ready resources, you can have a massive impact on the key metrics in your organization. We have seen groups take new teller Time to Independence and reduce it from six to 12 months down to as little as one week. Other groups have reduced internal escalations by as much as 80%, freeing up massive amounts of supervisor time. This creates an environment where operations can have confidence that employees are following the right procedures, and customers can have confidence that they're getting the best service possible.

The institutions that win with AI won't be the ones with the most advanced chatbot. They'll be the ones who cleaned the water and built a way to deliver it, just enough, at the exact moment their people need it. That's what turns a year-long risk into a week-long ramp, lets you push a corrected procedure to every employee the day a regulation changes, and keeps your people focused on the customer instead of the screen. The technology is ready. The real question is whether your Operational Knowledge is.

Greg DeVore is an author, co-founder, and CEO of ScreenSteps, a software platform that helps regulated industries turn knowledge into performance. He studied at the Berklee College of Music in Boston and at Harvard, though the Harvard in question was his junior high school.

He is the co-author of Find & Follow: Reduce Supervisor Burnout & Improve Employee Performance by Transferring Knowledge Faster. The Find & Follow Framework helps organizations improve employee performance by reducing their reliance on tribal knowledge.