The AI you already have without knowing it: AI features in your applications and what to ask the vendor

Many organisations believe they do not use artificial intelligence yet, and yet they already have it in-house: in the word processor, in the ERP, in the case management system, in the phone system or in the cameras. It often arrives in an update, switched on by default, as “just another feature”. And for the EU AI Act that changes nothing: the manufacturer is the provider and the organisation using it is the deployer, with the same obligations as if it had procured it separately.

You classify the feature, not the product

The same product can have minimal-risk features and high-risk ones. Some examples:

Product	AI feature	How it is classified
Word processor with Copilot	Draft, summarise, rewrite	Permitted use with public or internal information; conditional with personal data. The risk lies in permissions
ERP: accounting and purchasing	Detect duplicate invoices or anomalies	Minimal risk; conditional if it processes data on natural persons
ERP: human resources	Propose shifts, evaluate performance, screen applications, predict absenteeism	High risk (employment), with information to staff and their representatives. This is the case most often overlooked
Case management system	Classify, summarise or propose content	Conditional; if it proposes decisions, it is assessed as an output with legal effects
Phone system	Transcribe calls, voice bot	Conditional: disclose that it is an AI, personal data and a human channel
Cameras with analytics	Count occupancy or detect objects without identifying people	Not high risk, but if the images identify people they are personal data
Cameras: number plate recognition	Identify vehicles	Personal data: impact assessment and a specific legal basis
Cameras: facial recognition	Identify or categorise people	High risk at the very least; real-time remote identification in publicly accessible spaces is prohibited, with exceptions
Any product	Detect employees’ emotions	Prohibited (Art. 5 of the EU AI Act)

Watch out: if the organisation substantially modifies a high-risk system or gives it a purpose that makes it high-risk, it becomes a provider and takes on the provider’s obligations (Art. 25 of the EU AI Act).

Risks specific to third-party AI

• Features switched on by default in an update, without assessment.
• Lack of information from the manufacturer about what the AI does, with what data and where.
• Data flowing out to the manufacturer’s AI services, sometimes outside the EU.
• Lock-in: it is not always possible to switch off the feature without losing the product.
• In the public sector, certifications under Spain’s National Security Framework (ENS) whose scope does not cover the AI features, which are sometimes provided by another service or a subcontractor.

What to do

1. Initial inventory: send a short questionnaire to the vendor of each application in use and record the answers in the register of AI systems.
2. Switch off by default new AI features until they are assessed, reviewing the release notes of each version.
3. Classify each feature and, if it processes personal data or is involved in decisions, treat it as a new use case with its full assessment.
4. Require in tender specifications that the vendor declare its AI features, give notice before enabling new ones and allow them to be switched off.

A typical example: an ERP update enables, in the HR module, an “automatic shift proposal” based on absence history. IT spots it when reviewing the release notes and leaves it switched off. The assessment concludes that it is high-risk (it affects working conditions) and that it uses absence data that may reveal health information. Enabling it would require an impact assessment, information to staff and authorisation; the department decides not to do so for now.

How to build the inventory step by step

1. Start from the application inventory that the IT department already has: office software, ERP, CRM, document management, phone system, video surveillance, training or recruitment platforms.
2. Review the release notes of each product for the last twelve months, looking for “smart”, “assisted” or “AI-powered” features.
3. Send the questionnaire to each vendor and give a reasonable deadline.
4. Record each feature in the AI systems inventory: product, feature, status (active or not), data used, where it is processed and risk classification.
5. Cross-check the AI inventory against the application inventory periodically: every new application or significant update is a review.
6. Prioritise high-risk features and those that process personal data.

How to assess the vendor’s answers

Answer	Reading
“Data is not used for training”, and this is stated in the contract	Correct
“We don’t use your data for training”, with no contractual backing	Insufficient: it must be included in the contract
“We process in the EU”, but support accesses from outside	There is an international transfer: safeguards are needed
“It can’t be switched off”	Lock-in risk: consider alternatives or conditions
“We don’t have that information”	This is in itself a risk that should weigh in the assessment
Security certification without the AI features in its scope	The feature is not used until this is resolved

Clauses worth including in contracts

• Declaration of the product’s AI features and prior notice before enabling new ones.
• The ability to switch off each feature without losing the rest of the product.
• A ban on using the data, prompts and responses to train or improve models.
• Location of storage and processing, preferably in the EU, and a list of sub-processors.
• Retention period for conversations and logs, and access to those logs.
• Notification of incidents and vulnerabilities within a specific timeframe (for example, 24 hours).
• Technical documentation and sufficient explainability to justify decisions.
• Export of data and logs at the end of the contract, and certified deletion.
• In the public sector, security certification (ENS) that expressly covers the AI features.

The case of office software

Productivity suites are the clearest example of embedded AI: email summaries, meeting transcription, document drafts or search across all the organisation’s information. Here the main risk is not the model but permissions: the assistant sees everything each user can see. Before turning it on, you need to review permissions, classify information and set usage rules, as we explain in how to prepare your organisation before turning on Microsoft 365 Copilot.

Minimum questionnaire for the AI vendor

If the vendor cannot answer any of these questions, that in itself is a risk to be weighed:

1. Is the product really an AI system under the EU AI Act? Which AI features does it include and which are active?
2. Which model does it use, in which version, and who developed it?
3. Where does the model run: country or region, own cloud or third-party cloud?
4. What data from the organisation does it receive?
5. Does it use that data to train or improve the model?
6. Which sub-processors are involved?
7. Where is the data stored and processed?
8. How does it log its activity (inputs, outputs and actions), and how long are the logs kept?
9. How does it enable human oversight?
10. What measures does it have against manipulation and prompt injection?
11. How does it manage and communicate model version changes?
12. Can the AI feature be switched off without losing the rest of the product?
13. How are data and logs exported at the end of the contract?
14. What security certification does it hold (in the public sector, ENS), and does it cover the AI features exactly?
15. Can it provide an inventory of models, dependencies and data (AI BOM)?
16. How, and within what timeframe, does it notify vulnerabilities and incidents?

At Bidatia we carry out AI inventories of our clients’ applications and help draft contract clauses. For the full framework, see which regulations apply to AI and data and the lifecycle of an AI project.