The lifecycle of an AI project: from idea to retirement

Many AI projects fail because of the path they take, not because of the technology: they start from the tool instead of the problem, they are tested with real data without a legal basis, the pilot drags on without a decision, or human oversight is never made concrete. A lifecycle with gates (points at which you check whether you can move forward) avoids almost all of these mistakes.

The seven phases

Phase	What is done	Who decides	Output
1. Idea	Describe the need and the purpose	Sponsor	One-page proposal
2. Assessment	Classify the use; review data, risks and feasibility	Business owner, with data protection and security	Register entry and minimum questions answered
3. Proof of concept or pilot	Test in a limited environment, with minimal or synthetic data	Technical lead	Results measured against the test bench
4. Validation	Check against the acceptance criteria; review errors and biases	Business owner	Validation report
5. Authorisation	Review the complete file and authorise or reject	AI governance body	Authorisation and publication
6. Operation	Use with human oversight, monitoring and periodic review	Business owner	Review report, at least annually
7. Retirement	Decommission the system, recover or delete data and close access	Business owner	Register entry with status “retired”

Ways back: if validation does not meet the criteria, you go back to the pilot. If there is a substantial change during operation (a new model or provider, new data sources, a new purpose, new users or greater capacity to act, such as moving from an assistant that answers to an agent that carries out actions), you go back to assessment.

Not every use goes through the full cycle: simple uses without personal data only need an authorised tool and basic training. The full cycle is for uses that process personal data, interact with people or support decisions.

Each phase in detail

Phase 1. Idea (1–2 weeks). Describe the problem, who it affects and what you expect to achieve, and ask whether AI is really needed or whether improving the process or automating it with rules would be enough. Deliverable: a one-page proposal with the purpose, the expected benefit and the area affected. Typical mistake: starting from the tool (“we want to use AI”) instead of the problem.

Phase 2. Assessment (2–4 weeks). Classify the use according to its risk, rule out that it is a prohibited practice, identify the data and the legal basis, carry out an initial risk analysis with data protection and security, weigh up alternatives and providers with a minimum questionnaire, and set the acceptance criteria. Open the register entry for the use case. Typical mistake: consulting data protection at the end, or not estimating the recurring cost.

Phase 3. Proof of concept or pilot (1–3 months). A limited environment (one area, few users, limited duration), preferably with synthetic, anonymised or minimal data. Build and run the test bench and gather user feedback. The EU AI Act does not apply to testing before the system is put into service, but data protection and security always do. Typical mistake: testing only the easy cases, or extending the pilot without making a decision.

Phase 4. Validation (2–4 weeks). Compare the results with the acceptance criteria, review errors and biases, complete the impact assessment, define human oversight (who reviews, what, with which sample and how decisions are reversed), categorise security, sign the data processing agreement and prepare the training.

Phase 5. Authorisation (2–4 weeks). The AI governance body reviews the file, may set conditions, and authorises or rejects. If authorised, the system is registered and published where appropriate. Typical mistake: starting to use it “in production” while approval is still pending.

Phase 6. Operation. Human oversight, tracking of indicators, incident management, re-running the test bench after every change, and a review at least once a year.

Phase 7. Retirement. When it fails to meet its indicators, the regulations change, the contract ends, an unacceptable risk appears or it is no longer needed. Exit plan: communicate, export whatever must be retained, require deletion with a certificate, revoke access and connectors, and update the register.

The evidence traffic light

Before authorising the move to production, the use case record must provide all of the following evidence. If any item is missing, the file is not approved:

Evidence	✓
Purpose and legal competence or business justification
Named business owner
Classification under the EU AI Act
Security category
Legal basis, if there is personal data
Impact assessment and, if high-risk, the fundamental rights impact assessment
Signed data processing agreement
Provider and sub-providers identified
Location of processing
Test bench passed
Human oversight defined
Incident plan
Exit or reversibility plan
Register updated

Who does what

Task	Accountable	Responsible	Consulted
Propose the use case	Sponsor	Sponsor	Business owner
Classify and open the register entry	Business owner	Business owner	Technical, security, data protection
Legal basis and impact assessment	Business owner	Business owner	Data protection, security
Security and risk analysis	Security officer	Security officer	Technical
Pilot and test bench	Business owner	Technical lead and supervisor	Security, data protection
Authorise production	AI governance body	AI governance body	All of the above
Oversight during operation	Business owner	Human supervisor	—
Incident management	Security officer	Technical and supervisor	Data protection

Two rules: data protection advises and monitors, but does not approve or execute, to avoid conflicts of interest; and whoever oversees does not approve what they oversee.

Minimum questions before starting

If any of them remains unanswered, the use case does not move forward:

1. What specific problem does it solve, and for whom?
2. Is AI needed, or would improving the process or automating it with rules be enough?
3. Which risk category does it fall into? Could it be high-risk or a prohibited practice?
4. What data does it need, from which sources, and on what legal basis?
5. Is the data of sufficient quality and representative?
6. Will it interact with people or influence decisions?
7. Who is the business owner and who will oversee the results?
8. Which provider, where is the data processed and does the provider use it for training?
9. How will success be measured and what will count as a failure?
10. Have data protection and security been consulted?
11. Has a model of the right size been chosen?

The test bench

This is the objective, repeatable check that the system works. A set of cases with known correct answers, prepared by the area that will use the system, including:

• common cases, in sufficient numbers;
• edge and ambiguous cases;
• questions the AI must not answer;
• manipulation attempts;
• cases from different groups of people, to detect bias.

The acceptance criteria are set before the pilot, not afterwards. For example: at least 90 % correct answers, no invented regulatory references and zero answers containing another person’s data. The bench is run during validation, after every change of model or instructions and periodically during operation. A typical case: the provider updates the model, the bench is re-run and date extraction drops from 96 % to 88 %; the update is suspended until it is fixed.

Guardrails

These are the technical controls that keep the system within its purpose:

• Input filters: block data that must not get in and hidden malicious instructions.
• Output filters: detect offensive content, data the user must not see or answers without a source.
• Topic limits: the system only answers within its scope and refers everything else to a person.
• Action limits for agents: a closed list of actions, human confirmation before sending, modifying, deleting or paying, and caps on volume and cost.
• System instructions, documented and versioned.

They are applied in three layers (model provider, platform and your own application), tested before production, every activation is logged, and a balance is sought: if they are too strict, people are pushed towards unauthorised tools.

Minimum questions before production

If any answer is “no” or “we don’t know”, the system does not go into production:

• Have the test bench criteria been met and are the errors documented?
• Have extreme values and out-of-scope uses been tested, and does the system refuse, warn or refer rather than make things up?
• Are the guardrails configured and tested against manipulation?
• Have the impact assessment and, if high-risk, the fundamental rights impact assessment been completed?
• Has security been categorised and the data processing agreement signed?
• Is human oversight defined and tested: who, when and how decisions are reversed?
• Are people informed that they are interacting with an AI?
• Can you explain in plain language how the system influences the outcome?
• Are logs kept so that you can reconstruct which input produced which output?
• Is there training, a monitoring plan, an incident channel and retirement criteria?

Operation and retirement

During operation, measure the accuracy rate in sampling, the supervisor correction rate (if it drops to zero, oversight has become a box-ticking exercise), incidents, usage and cost. Retirement is planned from the outset: export whatever must be retained, require the provider to delete data with a certificate, and revoke connectors and technical accounts.

At Bidatia we support AI projects throughout their lifecycle, from assessment to operation. For the regulatory framework, see which regulations apply to AI and data.