What We Learned Deploying AI Knowledge Infrastructure at a Fortune 500 Company
We recently deployed Phyvant at one of the world's largest private companies—160,000+ employees across 70+ countries, operations spanning agriculture, food processing, financial services, and industrial products.
I'm going to share what we learned. Some of it confirmed our assumptions. Some of it surprised us completely. Some things broke in ways we didn't anticipate.
The Setup
The company had already invested heavily in AI. They had ChatGPT Enterprise deployed to thousands of employees. They had custom AI assistants for specific workflows. They had RAG pipelines connected to their document repositories.
Each tool worked fine on general knowledge. Ask about market trends, industry dynamics, or common business processes—solid answers. Ask about anything company-specific—product codes, internal processes, cross-division workflows, regional compliance requirements—and the AI failed.
Not catastrophically failed. Quietly failed. The AI would give confident-sounding answers that were subtly wrong, or technically correct but missing critical context, or outdated by two years. Employees learned not to trust it for internal questions.
That's what brought us in: could we give their existing AI tools the business context they needed to actually work?
Surprise #1: The Knowledge Was Everywhere and Nowhere
We assumed the biggest challenge would be technical—building pipelines, integrating systems, scaling infrastructure. We were wrong.
The biggest challenge was that institutional knowledge was fragmented across hundreds of people. There was no single document, database, or system that captured the full picture of how things worked.
Here's an example: they had a product that was sold under different names in different regions, with different formulations based on local regulations, processed at different facilities depending on capacity and logistics. To fully understand this product, you'd need to synthesize information from:
- The product master database in North America
- A separate system in Europe (with different ID conventions)
- Compliance documents in three different SharePoint sites
- A spreadsheet that someone in Supply Chain maintained
- The tribal knowledge of two people who'd been there for 20 years
No single person knew the complete picture. The knowledge was distributed across the organization, and much of it had never been explicitly documented. It lived in people's heads, in informal processes, in the way things were "just done."
Surprise #2: The Same Words Meant Different Things
"Product review" meant something completely different in R&D versus Sales versus Compliance.
In R&D, product review was a technical evaluation of formulation and performance data. In Sales, it was a quarterly analysis of revenue and market positioning. In Compliance, it was an audit process for regulatory requirements.
Same term. Three completely different processes. Three different sets of stakeholders, documents, and workflows.
This wasn't an isolated case. The company had grown through acquisitions over decades. Each acquired unit brought its own terminology. Efforts to standardize had only partially succeeded. The result was a patchwork of overlapping and sometimes contradictory conventions.
When AI tools retrieved documents containing "product review," they had no way to know which meaning applied. The answers they gave depended on which documents happened to be retrieved—essentially random from the user's perspective.
What We Built
We developed a three-layer approach:
Layer 1: Automated extraction from existing sources. We built pipelines to pull structured data from their ERP systems, CRM, HR databases, and other authoritative systems. This gave us the baseline: entities, relationships, and attributes that were already formally documented.
Layer 2: Structured knowledge capture from domain experts. This was the harder part. We worked with subject matter experts across divisions to capture the knowledge that wasn't written down. What do these terms actually mean in your context? How do these processes actually work? What are the exceptions that everyone knows but nobody documented?
We developed interview protocols to make this efficient—not trying to capture everything, but focusing on the 20% of institutional knowledge that drove 80% of the AI failures we'd observed.
Layer 3: A knowledge graph that modeled relationships. All of this knowledge fed into a graph structure that captured relationships between entities—products, people, processes, systems, regulations. The graph knew that Product-A in System-1 was the same as Product-B in System-2. It knew which policies applied to which regions. It knew who the domain experts were for different topics.
On top of this, we built an API layer that any AI tool in their stack could query. Before generating a response about something company-specific, the AI would check with Phyvant: "What do you know about this entity? What context is relevant? What relationships should I consider?"
The Non-Negotiable: On-Prem Deployment
I should mention this because it shaped every technical decision: everything had to run on their infrastructure. Zero data could leave their network.
This wasn't negotiable from day one. They're handling proprietary processes, confidential financial data, and trade secrets worth billions. The idea of sending any of this to a third-party cloud was a non-starter.
This constraint made things harder. We couldn't use cloud-hosted services for any part of the stack. We had to work within their existing infrastructure—which varied significantly by region. We had to deal with internal IT policies, security reviews, and deployment processes that were designed for very different kinds of software.
But it was the right constraint. The knowledge we were capturing was literally their competitive advantage. It should stay within their walls.
Early Results
I want to be careful here. We're still early. I'm not going to make up impressive-sounding metrics.
What we can say:
Consistency improved. AI tools started giving the same answers regardless of which documents happened to be retrieved. When someone asked about Product-A, the AI knew it was the same as Product-B and Product-C in other systems. The random variability decreased.
New employee onboarding accelerated. New hires could ask questions about company-specific topics and get reliable answers. They weren't relying on finding the right person to ask or hoping they found the right document. The institutional knowledge was accessible.
Expert time was freed up. The senior people who used to field dozens of questions per week about "how things work" reported fewer interruptions. The routine questions were getting handled by AI. They could focus on questions that actually required their judgment.
Trust began rebuilding. This might be the most important result. Employees who had given up on AI tools for internal questions started using them again. They tested them, found they got reasonable answers, and gradually increased reliance.
Lessons Learned
If you're considering something similar, here's what we'd suggest:
Start with one division. We began with a single business unit, got it working well, and then expanded. Trying to capture institutional knowledge across a 160,000-person organization at once would have been impossible.
Focus on the high-impact knowledge gaps. Not all institutional knowledge is equally valuable for AI accuracy. Identify which missing knowledge is causing the most failures, and start there. For us, it was cross-system entity resolution—knowing that different identifiers referred to the same real-world thing.
Plan for the long tail. The first 80% of value came relatively quickly. The remaining 20% has been harder—edge cases, exceptions, knowledge that only one or two people have. It's ongoing work, not a one-time project.
Make knowledge capture part of the workflow. The system improves when people contribute corrections and additions. We built mechanisms to capture these passively—when someone corrects an AI answer, that correction feeds back into the knowledge graph.
We're still learning. But we're confident in the core thesis: AI tools can't understand your business without institutional knowledge. Building that layer is hard work, but it's the work that makes AI actually useful.