Smart Xchange
Overview
A four-month thought leadership engagement into how people decide whether to let an AI service act on their behalf during financial moments they cannot control. The work produced a three-part principle model for designing trust into AI-enabled financial services, and a service concept built to test it.
Categories
Service Design
AI in Financial Services
Thought Leadership
Date
Nov 2024 - Feb 2025
Client
EY Seren (EY Studio+)
Royal College of Art
Role
Service designer. Research synthesis, systems mapping, narrative and delivery.
Key signal:
The feature with the most technical sophistication tested worst. Shown real-time AI insight on market rates, participants said they would verify the figure with their bank or with Google before acting on it.
The features they accepted were the ones where they had set the parameters themselves. On this evidence, participants assessed the AI service on whether anyone could be held responsible for it, before they assessed how well it performed.
Key signal:
The feature with the most technical sophistication tested worst. Shown real-time AI insight on market rates, participants said they would verify the figure with their bank or with Google before acting on it. The features they accepted were the ones where they had set the parameters themselves. On this evidence, participants assessed the AI service on whether anyone could be held responsible for it, before they assessed how well it performed.
Methods:
20 depth interviews with women aged 25 to 39 who had relocated to the UK.
Root cause analysis, use case and journey mapping, emotion mapping, value exchange system mapping, hypothesis-led prototyping, a mixed-gender co-design and testing workshop, and Jobs to Be Done.
Validation:
Three hypotheses were built into discrete feature prototypes and tested in one co-design workshop. One was supported strongly, one partially, one was not supported.
The unsupported result is what shaped the model.
20
women interviewed, aged 25 to 39, relocating or recently relocated to the UK
3
hypotheses built into testable feature prototypes
1 of 3
not supported in testing, including the most technically advanced feature
3
conditions for trust in AI services, argued to hold beyond financial services
Read details below
Findings
A three-part model for trust in AI services
EY Seren set an open brief. In regulated sectors where trust and empathy carry commercial and reputational weight, how can AI provide human-centred support while retaining efficiency? We answered with a model, then built a service concept to test whether it held.
Across interviews and testing, trust did not present as a single property that a service either has or lacks. It presented as three conditions that depend on each other, and the order mattered. Participants responded to the warmth of the language only once they could see who was responsible for the service and what they still controlled themselves. Where those two conditions were missing, the same warm language was read as a sales technique.
Hover to reveal
The building blocks for trust
3 things services need to ensure they can attest to before launching another* AI app that "changes your life"
[Drag to view]
1. Accountability
2. User Agency
3. Empathetic language
Trust is extended in instalments
The model describes the structure. It does not describe the pace. Testing suggested that participants extended trust in small amounts and withdrew it easily, and that each condition had to be visible before the next one registered with them. For a service team this becomes a sequencing decision. Begin with the smallest reversible action, then widen the scope of delegated authority over time, rather than requesting that authority at onboarding.
The sequencing view presented to EY Seren. Human in the loop sits between control and tone.
Visible access to a person is what allowed an automated action to feel reversible to participants.
Why the model may travel. It was derived in currency exchange. The conditions that produced it are a high value decision, an outcome the user cannot influence, low domain literacy, and time pressure.
Those same conditions describe a claims assessment, a benefits application, a diagnosis, and a redundancy consultation. This transfer is argued here rather than tested, and the limits section sets out what that means in practice.
The moment
Compounded high emotion
Studying trust in AI requires a moment where trust is genuinely at stake. General financial anxiety is too diffuse to design against, so the first task was to locate a specific point where emotion, money and powerlessness meet.
We interviewed 20 women aged 25 to 39 who had recently moved to the UK, and used 5 Whys to work past the first answers. Budgeting and adapting to a new cultural context came up in almost every conversation. Both are symptoms rather than design problems.
Underneath them sat something more specific. Money is the mechanism through which the social, functional and emotional needs of a relocation get met. The exchange rate is the one variable in that system the user has no influence over. It is the point at which a person's plan for their new life meets a number they cannot negotiate with.
2 personas, 2 Journeys, 1 Pattern
Recently arrived. Regret and fear, driven by mistimed transfers.
Settled longer. Anxiety and exposure, driven by unpredictable outgoings.
|
Imagine if you had…
A six month rent payment due in a week. Assignments due in two days. Utility bills in five. A friend's birthday in six.
And you're reliant on the FX rate that is moving throughout.
The working definition of compounded high emotion we took into design:
We named this "as-is" state: "compounded high emotion",
as it is not a single acute crisis but several ordinary pressures stacking inside one window, with a large uncontrollable financial decision sitting on top of them.
It matters because it changes what the user is optimising for. Several participants described wanting the decision to be over more than wanting it to be right.
That is the condition in which a poorly designed AI intervention does the most damage, and in which a well designed one carries the most value.
Value exchange map. The user holds a fixed sum in their home currency and needs the largest possible variable amount in the new one. The exchange rate sits between the two as an uncontrollable variable, and the pain points recorded in interviews route back through it.
Who are we designing for?
Why this segment
The narrowing was deliberate. Women make up 49.6 per cent of the global population and around 30 per cent are assessed as financially literate (UN DESA; GFLEC). The gap follows from historical exclusion from financial decision making rather than from aptitude, and financial products have been designed around a male default user for long enough that the deficit is structural. The 25 to 39 band concentrates the effect, with high financial pressure, high emotional intensity and limited available time occurring together.
Concluding that designing at the sharp end of a literacy and trust gap tends to produce principles that also hold for less constrained users. Working the other way round is less reliable.
Segment rationale, showing the overlap of financial pressure and emotional intensity.
Hover to reveal
Defining the Problem
Hover to reveal
HMW Statement
Smart Xchange
Overview
A four-month thought leadership engagement into how people decide whether to let an AI service act on their behalf during financial moments they cannot control. The work produced a three-part principle model for designing trust into AI-enabled financial services, and a service concept built to test it.
Categories
Service Design
AI in Financial Services
Thought Leadership
Date
Nov 2024 - Feb 2025
Client
EY Seren (EY Studio+)
Royal College of Art
Role
Service designer. Research synthesis,
systems mapping, narrative and delivery.
Key signal:
The feature with the most technical sophistication tested worst. Shown real-time AI insight on market rates, participants said they would verify the figure with their bank or with Google before acting on it.
The features they accepted were the ones where they had set the parameters themselves. On this evidence, participants assessed the AI service on whether anyone could be held responsible for it, before they assessed how well it performed.
Key signal:
The feature with the most technical sophistication tested worst. Shown real-time AI insight on market rates, participants said they would verify the figure with their bank or with Google before acting on it. The features they accepted were the ones where they had set the parameters themselves. On this evidence, participants assessed the AI service on whether anyone could be held responsible for it, before they assessed how well it performed.
Methods:
20 depth interviews with women aged 25 to 39 who had relocated to the UK.
Root cause analysis, use case and journey mapping, emotion mapping, value exchange system mapping, hypothesis-led prototyping, a mixed-gender co-design and testing workshop, and Jobs to Be Done.
Validation:
Three hypotheses were built into discrete feature prototypes and tested in one co-design workshop. One was supported strongly, one partially, one was not supported.
The unsupported result is what shaped the model.
20
women interviewed, aged 25 to 39, relocating or recently relocated to the UK
3
hypotheses built into testable feature prototypes
1 of 3
not supported in testing, including the most technically advanced feature
3
conditions for trust in AI services, argued to hold beyond financial services
Read details below
Findings
A three-part model for trust in AI services
EY Seren set an open brief. In regulated sectors where trust and empathy carry commercial and reputational weight, how can AI provide human-centred support while retaining efficiency? We answered with a model, then built a service concept to test whether it held.
Across interviews and testing, trust did not present as a single property that a service either has or lacks. It presented as three conditions that depend on each other, and the order mattered. Participants responded to the warmth of the language only once they could see who was responsible for the service and what they still controlled themselves. Where those two conditions were missing, the same warm language was read as a sales technique.
Hover to reveal
The building blocks for trust
3 things services need to ensure they can attest to before launching another* AI app that "changes your life"
[Drag to view]
1. Accountability
The foundation of all to come. Participants wanted to know who is responsible for the information and who they can go to when something goes wrong. This is a question about liability and recourse rather than about model explainability. No participant asked how the system reached its recommendation. Every group asked who was answerable for it.
2. User Agency
Control, or at least, a visible and credible version of it. Automation was welcomed where the user had set the boundary conditions themselves. The useful design question is what the user still decides, and whether they can see that decision holding after they walk away from the screen.
3. Empathetic language
The third condition, applied on top of the other two. Participants responded to communication that was warm, specific, and carried a clear action. The same language applied on its own reduced trust rather than building it.
Trust is extended in instalments
The model describes the structure. It does not describe the pace. Testing suggested that participants extended trust in small amounts and withdrew it easily, and that each condition had to be visible before the next one registered with them. For a service team this becomes a sequencing decision. Begin with the smallest reversible action, then widen the scope of delegated authority over time, rather than requesting that authority at onboarding.
The sequencing view presented to EY Seren. Human in the loop sits between control and tone.
Visible access to a person is what allowed an automated action to feel reversible to participants.
Why the model may travel. It was derived in currency exchange. The conditions that produced it are a high value decision, an outcome the user cannot influence, low domain literacy, and time pressure.
Those same conditions describe a claims assessment, a benefits application, a diagnosis, and a redundancy consultation. This transfer is argued here rather than tested, and the limits section sets out what that means in practice.
The moment
Compounded high emotion
Studying trust in AI requires a moment where trust is genuinely at stake. General financial anxiety is too diffuse to design against, so the first task was to locate a specific point where emotion, money and powerlessness meet.
We interviewed 20 women aged 25 to 39 who had recently moved to the UK, and used 5 Whys to work past the first answers. Budgeting and adapting to a new cultural context came up in almost every conversation. Both are symptoms rather than design problems.
Underneath them sat something more specific. Money is the mechanism through which the social, functional and emotional needs of a relocation get met. The exchange rate is the one variable in that system the user has no influence over. It is the point at which a person's plan for their new life meets a number they cannot negotiate with.
2 personas, 2 Journeys, 1 Pattern
Recently arrived. Regret and fear, driven by mistimed transfers.
Settled longer. Anxiety and exposure, driven by unpredictable outgoings.
|
Imagine if you had…
A six month rent payment due in a week. Assignments due in two days. Utility bills in five. A friend's birthday in six. And you're reliant on the FX rate that is moving throughout.
The working definition of compounded high emotion we took into design:
We named this "as-is" state: "compounded high emotion",
as it is not a single acute crisis but several ordinary pressures stacking inside one window, with a large uncontrollable financial decision sitting on top of them.
It matters because it changes what the user is optimising for. Several participants described wanting the decision to be over more than wanting it to be right.
That is the condition in which a poorly designed AI intervention does the most damage, and in which a well designed one carries the most value.
Value exchange map (Above).
The user holds a fixed sum in their home currency and needs the largest possible variable amount in the new one. The exchange rate sits between the two as an uncontrollable variable, and the pain points recorded in interviews route back through it.
Who are we designing for?
Why this segment
The narrowing was deliberate. Women make up 49.6 per cent of the global population and around 30 per cent are assessed as financially literate (UN DESA; GFLEC). The gap follows from historical exclusion from financial decision making rather than from aptitude, and financial products have been designed around a male default user for long enough that the deficit is structural. The 25 to 39 band concentrates the effect, with high financial pressure, high emotional intensity and limited available time occurring together.
Concluding that designing at the sharp end of a literacy and trust gap tends to produce principles that also hold for less constrained users. Working the other way round is less reliable.
Value exchange map (Above).
The user holds a fixed sum in their home currency and needs the largest possible variable amount in the new one. The exchange rate sits between the two as an uncontrollable variable, and the pain points recorded in interviews route back through it.
Hover to reveal
Defining the Problem
Hover to reveal
HMW Statement
Smart Xchange
Overview
A four-month thought leadership engagement into how people decide whether to let an AI service act on their behalf during financial moments they cannot control. The work produced a three-part principle model for designing trust into AI-enabled financial services, and a service concept built to test it.
Categories
Service Design
AI in Financial Services
Thought Leadership
Date
Nov 2024 - Feb 2025
Client
EY Seren (EY Studio+)
Royal College of Art
Role
Service designer. Research synthesis, systems mapping, narrative and delivery.
Key signal:
The feature with the most technical sophistication tested worst. Shown real-time AI insight on market rates, participants said they would verify the figure with their bank or with Google before acting on it.
The features they accepted were the ones where they had set the parameters themselves. On this evidence, participants assessed the AI service on whether anyone could be held responsible for it, before they assessed how well it performed.
Methods:
20 depth interviews with women aged 25 to 39 who had relocated to the UK.
Root cause analysis, use case and journey mapping, emotion mapping, value exchange system mapping, hypothesis-led prototyping, a mixed-gender co-design and testing workshop, and Jobs to Be Done.
Validation:
Three hypotheses were built into discrete feature prototypes and tested in one co-design workshop. One was supported strongly, one partially, one was not supported.
The unsupported result is what shaped the model.
20
women interviewed, aged 25 to 39, relocating or recently relocated to the UK
3
hypotheses built into testable feature prototypes
1 of 3
not supported in testing, including the most technically advanced feature
3
conditions for trust in AI services, argued to hold beyond financial services
Read details below
Findings
A three-part model for trust in AI services
EY Seren set an open brief. In regulated sectors where trust and empathy carry commercial and reputational weight, how can AI provide human-centred support while retaining efficiency?
We answered with a model, then built a service concept to test whether it held.
Across interviews and testing, trust did not present as a single property that a service either has or lacks. It presented as three conditions that depend on each other, and the order mattered. Participants responded to the warmth of the language only once they could see who was responsible for the service and what they still controlled themselves. Where those two conditions were missing, the same warm language was read as a sales technique.
Hover to reveal
The building blocks for trust
3 things services need to ensure they can attest to before launching another* AI app that "changes your life"
[Tap above to view]
1. Accountability
The foundation of all to come. Participants wanted to know who is responsible for the information and who they can go to when something goes wrong. This is a question about liability and recourse rather than about model explainability. No participant asked how the system reached its recommendation. Every group asked who was answerable for it.
2. User Agency
Control, or at least, a visible and credible version of it. Automation was welcomed where the user had set the boundary conditions themselves. The useful design question is what the user still decides, and whether they can see that decision holding after they walk away from the screen.
3. Empathetic language
The third condition, applied on top of the other two. Participants responded to communication that was warm, specific, and carried a clear action. The same language applied on its own reduced trust rather than building it.
Trust is extended in instalments
The model describes the structure. It does not describe the pace. Testing suggested that participants extended trust in small amounts and withdrew it easily, and that each condition had to be visible before the next one registered with them. For a service team this becomes a sequencing decision. Begin with the smallest reversible action, then widen the scope of delegated authority over time, rather than requesting that authority at onboarding.
The sequencing view presented to EY Seren. Human in the loop sits between control and tone.
Visible access to a person is what allowed an automated action to feel reversible to participants.
Why the model may travel. It was derived in currency exchange. The conditions that produced it are a high value decision, an outcome the user cannot influence, low domain literacy, and time pressure.
Those same conditions describe a claims assessment, a benefits application, a diagnosis, and a redundancy consultation. This transfer is argued here rather than tested, and the limits section sets out what that means in practice.
The moment
Compounded high emotion
Studying trust in AI requires a moment where trust is genuinely at stake. General financial anxiety is too diffuse to design against, so the first task was to locate a specific point where emotion, money and powerlessness meet.
We interviewed 20 women aged 25 to 39 who had recently moved to the UK, and used 5 Whys to work past the first answers. Budgeting and adapting to a new cultural context came up in almost every conversation. Both are symptoms rather than design problems.
Underneath them sat something more specific. Money is the mechanism through which the social, functional and emotional needs of a relocation get met. The exchange rate is the one variable in that system the user has no influence over. It is the point at which a person's plan for their new life meets a number they cannot negotiate with.
2 personas, 2 Journeys, 1 Pattern
Recently arrived. Regret and fear, driven by mistimed transfers.
Settled longer. Anxiety and exposure, driven by unpredictable outgoings.
|
Imagine if you had…
A six month rent payment due in a week. Assignments due in two days. Utility bills in five. A friend's birthday in six.
And you're reliant on the FX rate that is moving throughout.
The working definition of compounded high emotion we took into design:
We named this "as-is" state: "compounded high emotion",
as it is not a single acute crisis but several ordinary pressures stacking inside one window, with a large uncontrollable financial decision sitting on top of them.
It matters because it changes what the user is optimising for. Several participants described wanting the decision to be over more than wanting it to be right.
That is the condition in which a poorly designed AI intervention does the most damage, and in which a well designed one carries the most value.
Value exchange map. The user holds a fixed sum in their home currency and needs the largest possible variable amount in the new one. The exchange rate sits between the two as an uncontrollable variable, and the pain points recorded in interviews route back through it.
Who are we designing for?
Why this segment
The narrowing was deliberate. Women make up 49.6 per cent of the global population and around 30 per cent are assessed as financially literate (UN DESA; GFLEC). The gap follows from historical exclusion from financial decision making rather than from aptitude, and financial products have been designed around a male default user for long enough that the deficit is structural. The 25 to 39 band concentrates the effect, with high financial pressure, high emotional intensity and limited available time occurring together.
Concluding that designing at the sharp end of a literacy and trust gap tends to produce principles that also hold for less constrained users. Working the other way round is less reliable.
Segment rationale, showing the overlap of financial pressure and emotional intensity.
Hover to reveal
Defining the Problem
Hover to reveal
HMW Statement
Testing
The feature we were most confident in was not supported
Three hypotheses were converted into three discrete feature prototypes and tested in a co-design workshop. We recruited men alongside women, because a trust finding that only holds for one group is a preference rather than a principle, and we wanted to know which of the two we had.
20+
Particpants across age range, life-stage, with a women dominant gender split of around 3:1
9+
Sessions rans with seperate groups
Recording of various co-design and testing workshops.
Recording of various co-design and testing workshops.
How the sessions ran
The three prototypes were not shown together. One was presented as a standalone product and two were embedded inside an existing trusted platform, so that we could separate a participant's response to the feature from their response to the party offering it.
That split is what allowed the accountability finding to surface, since the same capability drew different reactions depending on whose name was on it.
Each hypothesis was translated into a separate testable feature,
so that a negative result could be attributed to a specific mechanism rather than to the concept as a whole.
Each hypothesis was translated into a separate testable feature, so that a negative result could be attributed to a specific mechanism rather than to the concept as a whole.
Each hypothesis was translated into a separate testable feature, so that a negative result could be attributed to a specific mechanism rather than to the concept as a whole.
LOW SUPPORT
Real-time AI insight on market rates
We expected that AI notifying users of financial risk would improve their ability to react to currency movement. Participants disagreed. Told that the rate had moved, they said they would verify the figure with their bank or with Google before acting on it.
What we took from it. The issue was standing rather than accuracy. Information from a source with no accountable party creates a second task instead of removing the first. This result produced the accountability condition in the model.
HIGH SUPPORT
Personalised alerts anchored to the user's own goals
The strongest result of the workshop. Participants wanted the service to know what they spend and to tell them how much to move against a target rate they had set themselves. One participant described wanting it to know her monthly spending and to tell her how much to transfer against her targeted rate. Being modelled accurately was described in terms of being looked after.
What we took from it. The same underlying capability, reading the market, becomes acceptable once the user supplies the threshold. Agency is what makes continuous monitoring legible as a service rather than as surveillance.
PARTIAL SUPPORT
Empathetic nudges
Supported with conditions. Participants wanted subtlety, and they wanted the message to carry an action. Warm language with no next step was described as something they would ignore.
What we took from it. Empathetic language works as a delivery mechanism rather than as a value proposition, which is why it sits third in the model.
Validation levels against workshop evidence.
The lowest score belongs to the feature that was, on paper, the most sophisticated.
Validation levels against workshop evidence.
The lowest score belongs to the feature that was, on paper, the most sophisticated.
Key Insights gained
What did we change and adjust in response
We kept the prediction feature and stopped asking users to accept it.
The screen states the recommendation, shows the data it was derived from, tested and validated having a"Verify" function as the primary action rather than a confirmation where the interface assumes that the user will cross-check and builds that step into the flow.
This is the accountability condition expressed as a single component.
It revealed itself during the testing of one interaction and it changes the service from something that asks to be believed into something that expects to be checked.
Prediction AI.
The recommendation is anchored to the user's own expenditure rather than to the market in general, and the primary action is Verify rather than Confirm.
The 4 themes underneath the results
01
Social validation
01
Social validation
02
Personalisation
02
Personalisation
03
Automation
03
Automation
04
Accountability
the Core Insight*
04
Accountability
the Core Insight*
The service; our vision for it
Smart Xchange: An automated currency management inside limits the user sets
The concept is deliberately modest, as the journey moves the user from daily manual monitoring to bounded automated control, and it is designed to sit inside an existing trusted platform rather than to establish trust of its own.
That last decision follows directly from the accountability finding.
A standalone product would have to manufacture credibility it has not earned. Embedding the service in an established financial relationship gives it an answerable party from the first interaction.
That last decision follows directly from the accountability finding.
A standalone product would have to manufacture credibility it has not earned. Embedding the service in an established financial relationship gives it an answerable party from the first interaction.
[Drag to view features]
1. Controlled automations
1. Controlled automations
Agency made explicit. The user sets the target rate, the amount, and the ceiling and floor. The service notifies when the target is reached, or executes the exchange if the user has authorised that in advance. The user keeps the judgement and hands over the monitoring. That division is what made automation acceptable in testing.
1. Controlled automations
Agency made explicit. The user sets the target rate, the amount, and the ceiling and floor. The service notifies when the target is reached, or executes the exchange if the user has authorised that in advance. The user keeps the judgement and hands over the monitoring. That division is what made automation acceptable in testing.
2. Smart predictions
2. Smart predictions
Prediction analysis paired with consumption analysis. The service reads spending patterns against rate movement, then recommends when to exchange and how much. The recommendation is made against the user's stated goal rather than against the market in general. It is the same capability that failed in testing, rebuilt around agency and verification.
2. Smart predictions
Prediction analysis paired with consumption analysis. The service reads spending patterns against rate movement, then recommends when to exchange and how much. The recommendation is made against the user's stated goal rather than against the market in general. It is the same capability that failed in testing, rebuilt around agency and verification.
3. Empathetic nudges
3. Empathetic nudges
Tone carrying an action. Alerts written around the user's spending behaviour and financial objectives, in a warm and direct register, each carrying a specific action. Tone is applied last and on top of the two conditions below it, which is the sequence the model requires.
3. Empathetic nudges
The third condition, applied on top of the other two. Participants responded to communication that was warm, specific, and carried a clear action. The same language applied on its own reduced trust rather than building it.
Jobs to Be Done:
used to make the consequences of each feature explicit before build. The emotional and social jobs, meaning trust, relief and confidence in financial conversations, carried more weight in testing than the functional ones.
Honest limitations + What could be done next
A proposal that measures Smart Xchange
Rather than adopt generic engagement metrics, we inverted the needs surfaced in research and assessed the concept against them.
Each feature was evaluated on a functional and an emotional axis. Notifications were assessed on whether they prompted action in the user's own interest and left the user feeling in control. Personalisation was assessed on whether it reduced guesswork and produced a sense of being understood. Consumption analysis was assessed on whether it removed the repeated balance checking that dominated the interviews.
I strongly believe the emotional axis is capable doing real work when scaled.
In a compounded high emotion moment, a service that improves the outcome while worsening the experience is unlikely to be used a second time.
This was a four month thought leadership engagement rather than a product build. This is a strong foundational model/framework to assist in developing any AI-Native product/software for the financial services and beyond.
This was a four month thought leadership engagement rather than a product build.
This is a strong foundational model/framework
to assist in developing any AI-Native product/software for the financial services and beyond.
Limitations of this project and its process
Limitations of this project and its process
Limitations of this project and its process
• Sample Size. 20 depth interviews and multiple co-design workshops concentrated on women (at a postgraduate level)
in London. Sufficient to generate a model, insufficient to claim prevalence. The literacy and trust patterns would need retesting with lower income and more a diverse range of participants before generalising.
• Self-reported trust. Participants told us what they would trust. No real money moved. The gap between stated and revealed trust is where AI adoption research most often fails, and this study sits on the stated side of it.
• No live deployment. The concept was prototyped and workshopped rather than shipped. Accountability in particular is straightforward to draw and difficult to operate. It requires a real complaints route, a named liable party, and a regulatory view on who owns an automated transfer that goes wrong.
• The transfer is argued. The ordering of accountability, agency and language was derived in one domain. Its application to health or to public services is reasoned here rather than demonstrated.
• Sample Size. 20 depth interviews and multiple co-design workshops concentrated on women (at a postgraduate level)
in London. Sufficient to generate a model, insufficient to claim prevalence. The literacy and trust patterns would need retesting with lower income and more a diverse range of participants before generalising.
• Self-reported trust. Participants told us what they would trust. No real money moved. The gap between stated and revealed trust is where AI adoption research most often fails, and this study sits on the stated side of it.
• No live deployment. The concept was prototyped and workshopped rather than shipped. Accountability in particular is straightforward to draw and difficult to operate. It requires a real complaints route, a named liable party, and a regulatory view on who owns an automated transfer that goes wrong.
• The transfer is argued. The ordering of accountability, agency and language was derived in one domain. Its application to health or to public services is reasoned here rather than demonstrated.
An open question:
In moments of high emotion, people may not trust their own judgement.
They tend to look instead to someone they already trust. That leaves the question we returned to EY Seren and did not answer.
Should an AI service work to earn trust of its own, or should it carry the trust that already exists between a person and the people they rely on?
They tend to look instead to someone they already trust. That leaves the question we returned to EY Seren and did not answer.
Should an AI service work to earn trust of its own, or should it carry the trust that already exists between a person and the people they rely on?
