From unstructured data to useful metadata: how to apply generative AI agents

Business decisions rely mostly on structured data: sales tables, orders, invoices. Yet much of what an organisation knows lives elsewhere: call recordings, PDF contracts, emails, meeting minutes, customer comments on social media. In practice, that unstructured content is invisible to automation: no report summarises it and no process reacts to what it says.

What is missing to make use of it is metadata: specific fields that describe the content and that a system can actually use. "Customer intent", "reason for leaving", "non-standard penalty clause". Until now, creating that metadata meant someone reading or listening to every item, which is slow, expensive and impossible to scale. Generative AI agents change that equation.

The problem: metadata that cannot be created by hand

Source	What it contains	Metadata of interest
Customer service transcripts	Complaints, questions, intent to buy or cancel	Reason for contact, frustration level, churn risk
Contracts and legal documents	Clauses, deadlines, obligations	Clause type, deviation from the organisation's template, risk level
Emails and tickets	Requests and incidents	Category, urgency, responsible team
Minutes and technical reports	Decisions and commitments	Agreements, owners, deadlines, discrepancies
Undocumented tables in the data platform	Columns with cryptic names	Purpose of the column, process that feeds it, likely owner

When metadata creation does not scale, the business reacts late: the sign that a customer was about to leave was in last week's call, and the risk in the contract was on page 14.

What a generative AI agent is (and is not)

An agent is not a chatbot. It is a system that turns a business task into a sequence of actions and carries it out end to end. Powered by a language model, an agent can:

• Reason and plan: break the task down into logical steps.
• Act: use tools and APIs from business systems such as the CRM, the ERP or the document management system.
• Iterate: review intermediate results and adjust the plan.

The practical difference is clear. A chatbot replies "this call looks like a billing complaint". An agent analyses every recording from the week, extracts the defined fields, updates the CRM and records which passage each value is based on.

The technical stack has two layers:

• The model (the brain): a language model with reasoning capability and, where needed, multimodal support for audio and images.
• Orchestration (the body): platforms such as LangChain or n8n, or equivalent services from the major cloud providers, which define the workflow and connect the agent to the target systems.

RAG: accuracy with your own information

A language model does not know an organisation's contracts, policies or procedures, and if asked about them it may invent a plausible answer. The technique that solves this is retrieval-augmented generation (RAG):

1. Query. The agent receives a specific task: "find the indemnity clause in this contract".
2. Retrieval. The system searches the organisation's knowledge base for relevant passages: documents, databases, the data lake.
3. Augmentation. The model receives the request together with those passages.
4. Generation. The model returns the answer or the structured metadata, citing the internal sources it relies on.

RAG has become the reference approach for using generative AI with proprietary information: it keeps data under the organisation's control, avoids relying on what the model learnt during training and makes every result verifiable. It does not eliminate errors, but it makes them detectable.

Four high-value use cases

1. Customer service and sales

From transcripts, the agent extracts fields such as reason for contact, frustration level on a defined scale, reason for cancellation or lead qualification. With those fields in the CRM, customers at risk of churning can be spotted in time and team training can draw on real cases.

2. Procurement and legal

Working on PDF contracts, the agent identifies clauses, classifies them and flags deviations from the organisation's own template or risky clauses. Document reviews such as due diligence become much faster, and every finding is linked to the exact passage in the document thanks to RAG.

3. Documenting the data platform

Many tables in a data lake or warehouse have no documentation. An agent can propose the purpose of each column, the business process that feeds it and a likely owner, based on the schema, the load code and whatever documentation exists. These are proposals for the data steward to validate, but they remove the most tedious part of the work and speed up both the catalogue and GDPR compliance.

4. Analysing several sources at once

The agent compares, for example, a technical report, the applicable regulations and the notes from a meeting, and extracts key differences and potential non-compliance. A person can do this, but not across dozens of documents at once, nor every time one of them changes.

How to ensure metadata quality

Metadata that nobody trusts is useless. A few practices make the difference:

• A closed schema. Define the fields and their permitted values in advance (controlled lists, scales) and require the model to return exactly that format.
• Cited source. Each value carries a reference to the source passage, so review is quick.
• A validated sample. Measure accuracy against a set reviewed by experts before automating, and repeat the measurement periodically.
• Human review in proportion to risk. Systematic validation for legal matters or anything affecting people's rights; sampling for operational ones.
• Personal data under control. Transcripts and emails contain personal data: minimise what is sent to the model and control where it is processed and who sees the results.

Adoption roadmap

Phase	Focus	Who is involved
1. Discovery	Prove value in a low-risk use case, for example classifying a batch of documents	Innovation and the business area concerned
2. Industrialising RAG	Build the secure knowledge base and a RAG architecture with traceability	Data architecture, security
3. Pilot agents	Deploy agents in the priority use cases and measure efficiency and quality indicators	Business, data, security
4. Scale	Embed agents in end-to-end processes, with more autonomy, auditing and AI governance	The whole organisation, within a stable governance framework

Each phase should close with measured results (accuracy, time saved, adoption) before moving on to the next.

Where to start

The useful question is not "what can we do with AI?" but "which metadata are we missing, and what is it worth to have it?". A good starting point is to choose that metadata in the area where it hurts most (usually customer service or contracts), define its schema, prepare a validated sample and build a proof of concept that measures accuracy on real data. At Bidatia we support this kind of project, from choosing the use case to integrating it with the data platform and data governance.