The Hidden Security Cost of "Free" AI

[slapper]

In January of 2024, an employee gave twenty-five million dollars to scammers in the midst of a video call. The employee saw his/her boss and colleagues on screen. Everyone looked real. But all of the individuals were AI-generated.

Verification once used to be cheap. You see somebody's face, hear their voice, you know it's them. That verification cost was essentially nothing. Now? The same verification is worthless. To actually verify identity you need multiple channels, code phrases, callback protocols; all of which cost time and money. The cost of trust just got higher, and higher, for all people.

The financial industry is beginning to price this in. Companies without appropriate security protocols are seeing their insurance costs go up. The average cost of one of these attacks is around six hundred thousand dollars, not counting the big ones like that twenty-five million loss. But individuals? We're taking in this cost in a different way, through the time we waste being suspicious of everything, through money lost to scams we can't do anything about, through opportunities we miss because we can't trust what we see.

I've begun to suspect every single financial request unless otherwise given. That's a tax on my time and my mental energy, that did not exist two years ago. And I'm realising that this is just the beginning. As this technology gets cheaper and more available, the cost of maintaining financial security is going to keep getting higher for all of us.

Who pays for this? Right now, it's individuals and small business that can't afford sophisticated security systems. The big players will adapt and transfer the costs to customers. But what about everyone else? What happens to people that can't afford the new cost of trust? Has anybody here changed the way they deal with financial verification as a result?

[justinlamode]

In January of 2024, an employee gave twenty-five million dollars to scammers in the midst of a video call. The employee saw his/her boss and colleagues on screen. Everyone looked real. But all of the individuals were AI-generated.

This is a serious issue but in a formal setting, there is a mix of careless in this story because twenty-five million is not a small amount to give out without enough supporting authorization and verification. Authorization and verification via email and phone call is necessary to get anything of this magnitude moved, at least I'm in the system so I know how some of these things work. Maybe the employee have some questions to answer regarding the whole thing.

Who pays for this? Right now, it's individuals and small business that can't afford sophisticated security systems. The big players will adapt and transfer the costs to customers. But what about everyone else? What happens to people that can't afford the new cost of trust? Has anybody here changed the way they deal with financial verification as a result?

AI is definitely changing a lot of things and even increasing the doubt of people regarding what is real and what is not. It brought a lot of advantages but those did not come at no cost. Eventually we will adapt to this new way of life where everything in the digital world have to be questioned and verified.

[Findingnemo]

Transferred 25 million dollars over the video call? For that big amount there must be a legal process involved with all the paper and documentation so it is not that easy unless someone is stupid enough and given that much money to manage.

But for the smaller amounts, it is possible, I heard a few stories that someone over the phonecall sounded exactly like a father or husband asked for an emergency need and asked to send money to an bank account but only later the person realized that they have been scammed.

SO keep the numbers saved in your contact and don't asnwer unless they call from that number.

[abhiseshakana]

When it comes to financial or asset security, I always believe that thieves are always one step ahead of the legal institutions that regulate them. It can be concluded that criminal innovations always outpace innovations in protection or security. This is an undeniable phenomenon, a manifestation of our world's daily advancements. As users and innovators, we cannot simply focus on the positive aspects (efficiency, convenience, and the "wow" effect of technology) without experiencing the real impact of our adoption of technology in our lives.

When security becomes a top priority, and thieves constantly find new methods, the only thing left is for system owners to react by adding layers of security to protect users. Users are left in the back seat, having to comply with the system by adapting to new rules with all their complexities. Money, effort, and time are what we pay, but value and security are what we get.

Problems arise for small and medium-sized businesses when their capital is limited to invest in additional layers amidst the demand for zero-fault security. Where experiments can't be conducted or capital is exhausted to keep up with the thieves, those who are unwilling and unable to adapt are the ones left behind. So, if asked who bears the greatest cost, it's users who refuse to adapt to multilayer security standards. In an era where proving honesty is more important than honesty itself, we (users, law enforcement, institutions, and business entities) must start thinking about how to not just react but take the initiative to counter thieves' moves.

[yhiaali3]

This is indeed becoming very serious. With artificial intelligence becoming ubiquitous and potentially misused, there will be more problems like this in the future.

I remember reading a news article about two years ago about Turkish security forces arresting someone who used  AI to impersonate President Recep Tayyip Erdoğan's voice, attempting to deceive select companies and high-ranking government officials.
https://www.trtworld.com/article/14515799

That was two years ago. Now, the situation is much more complex and dangerous, as it's now possible to create realistic videos of people who look real, not just voices.

[Furball808]

In January of 2024, an employee gave twenty-five million dollars to scammers in the midst of a video call. The employee saw his/her boss and colleagues on screen. Everyone looked real. But all of the individuals were AI-generated.

Fast forward to today, we are now more aware of AI and the signs accompanied.
 that something might be AI-generated. AI might become more sophisticated in the future so we really have to be always suspicious of everything. Better to look like you are overreacting than regret being too trusting.

[goro_chief]

 I think they take advantage of adults who are in a hurry to solve problems right away. Nowadays, you have to be careful with phone numbers, which is why I don't trust anyone until I see them in person and they give me their phone number. For online transactions, I understand that there are secure ways outside of social media to make it safer. However, before making any transaction, it is always better to use p2p on secure platforms that hold the money until the corresponding service or person verifies their identity. Cheer up, friend, the world is for those who prepare themselves

[Zlantann]

When it comes to financial or asset security, I always believe that thieves are always one step ahead of the legal institutions that regulate them. It can be concluded that criminal innovations always outpace innovations in protection or security. This is an undeniable phenomenon, a manifestation of our world's daily advancements. As users and innovators, we cannot simply focus on the positive aspects (efficiency, convenience, and the "wow" effect of technology) without experiencing the real impact of our adoption of technology in our lives.

Criminals are not always ahead of some key organisations like Central Banks and other top financial institutions. These institutions have some of the best security systems and have a large budget for security. Sometimes these security breaches are enabled by insiders who are driven by financial and other gains.

Problems arise for small and medium-sized businesses when their capital is limited to invest in additional layers amidst the demand for zero-fault security. Where experiments can't be conducted or capital is exhausted to keep up with the thieves, those who are unwilling and unable to adapt are the ones left behind. So, if asked who bears the greatest cost, it's users who refuse to adapt to multilayer security standards. In an era where proving honesty is more important than honesty itself, we (users, law enforcement, institutions, and business entities) must start thinking about how to not just react but take the initiative to counter thieves' moves.

There will never be a perfect security system. Every institution would have to use available funds to invest in research and development of improved security systems. I still believe that there are affordable but still sound security systems that small and medium-scale businesses could adopt.

Who pays for this? Right now, it's individuals and small business that can't afford sophisticated security systems. The big players will adapt and transfer the costs to customers.

Every institution, including the government, transfers costs to individuals, so it's not strange. Organisations that cannot afford the cost will not kill themselves or go out of business. They will seek the security systems they can afford.

[WillyAp]

He who falls for those tricks deserves the fate.
The one trusting 25million to an employee without control mechanism is also to blame.

Another kind of AI trick: Self-delusions

I am the CEO of a $6.7 billion fintech company.
Last year I replaced 700 customer service workers with AI.
I called it "the future of work."
The future arrived.
It's worse.
In 2023, we stopped hiring entirely.
I announced it on stage.
People applauded.
Applause is how you know you've made a mistake.
We partnered with OpenAI.
I said "AI can already do all of the jobs that we, as humans, do."
I said this publicly.
Into a microphone.
With my whole chest.
We saved $10 million.
I put that in the press release.
Press releases are how you celebrate before the consequences arrive.
Our employee count went from 5,500 to 3,400.
I called it "efficiency."
Efficiency is when you fire people and the stock goes up.

For a while.

The AI handled customer complaints.
Customers complained about the AI.
It couldn't do nuance.
It couldn't do empathy.
It couldn't do angry customers yelling about missed payments at 2 AM.
Turns out those are the only customers who call.
I started getting emails.
From customers.
About the AI.
The AI was hallucinating payment plans that don't exist.
It told one customer their refund was "processing in the astral realm."
I don't know what that means.
Neither did the customer.
They posted it on Twitter.
It went viral.
Not the good kind of viral.
The kind where Congress starts asking questions.
My VP of Customer Experience scheduled a meeting.
She asked if we could "reintroduce human elements."
Human elements means people.
People I fired.
I said we'd "explore hybrid solutions."
Hybrid solutions means admitting we were wrong.
Without using the word "wrong."
I did an interview with Bloomberg.
I said "there will always be a human if you want."
A human if you want.
Like it's a topping.
Like empathy is extra cheese.
I announced a new hiring initiative.

We're bringing back customer service workers.
But not as employees.
As gig workers.
From home.
No benefits.
No stability.
Like Uber.

But for apologizing.
I called it "flexible human infrastructure."
That's not a real thing.
But it sounds like one.
The workers we fired are now contractors.
Doing the same job.
For less money.
With no healthcare.
I called it "the evolution of the customer experience."
Evolution means we broke something and fixed it worse.
But the word sounds forward-thinking.
We spent $10 million to save $10 million.
And ended up with angry customers, viral tweets, and a gig economy call center.

I'm doing a keynote at Davos next month.
The topic is "AI Transformation: Lessons in Leadership."
I haven't learned any lessons.
But I have learned to call them lessons.
That's the same thing. In business.

source: https://x.com/i/status/2003239718478033147

[Shishir99]

I find it extremely hard to believe.
I see a lot of AI-generated videos on Social media, and I can detect them within a few seconds. We are talking about a video conference here where AI had to match the voice, face, and everything, which is nearly impossible. Even if AI can do this, it should not be that hard to notice the difference. If this is a real news, I would expect some referance link so I can have a read. Even if your AI knows a lot of information about you, it should be between you and the AI. If a 3rd person can get the data, that means you didn't care about your privacy and you are liable for that.

[Justbillywitt]

Transferred 25 million dollars over the video call? For that big amount there must be a legal process involved with all the paper and documentation so it is not that easy unless someone is stupid enough and given that much money to manage.

But for the smaller amounts, it is possible, I heard a few stories that someone over the phonecall sounded exactly like a father or husband asked for an emergency need and asked to send money to an bank account but only later the person realized that they have been scammed.

SO keep the numbers saved in your contact and don't asnwer unless they call from that number.

But someone can be in trouble and call with another line, maybe they got involved in an accident and lost their phone or they were attacked and lost their phone and they needed to call and inform their family about the incident so at that point they can't call from the phone numbers that are being saved on our phone. This thing is a problem and I we have to be careful, but sometimes we don't have to neglect picking calls from unknown numbers.

 It could actually be someone that knows us that's calling and they need our help or want to pass urgent information that could save life. So my suggestion is this, if someone calls and pretend to be another person by using AI, and request for financial help, ask them few personal questions that only the real person can actually provide the answer, questions that can't be found online.

[goldkingcoiner]

In January of 2024, an employee gave twenty-five million dollars to scammers in the midst of a video call. The employee saw his/her boss and colleagues on screen. Everyone looked real. But all of the individuals were AI-generated.

Verification once used to be cheap. You see somebody's face, hear their voice, you know it's them. That verification cost was essentially nothing. Now? The same verification is worthless. To actually verify identity you need multiple channels, code phrases, callback protocols; all of which cost time and money. The cost of trust just got higher, and higher, for all people.

The financial industry is beginning to price this in. Companies without appropriate security protocols are seeing their insurance costs go up. The average cost of one of these attacks is around six hundred thousand dollars, not counting the big ones like that twenty-five million loss. But individuals? We're taking in this cost in a different way, through the time we waste being suspicious of everything, through money lost to scams we can't do anything about, through opportunities we miss because we can't trust what we see.

I've begun to suspect every single financial request unless otherwise given. That's a tax on my time and my mental energy, that did not exist two years ago. And I'm realising that this is just the beginning. As this technology gets cheaper and more available, the cost of maintaining financial security is going to keep getting higher for all of us.

Who pays for this? Right now, it's individuals and small business that can't afford sophisticated security systems. The big players will adapt and transfer the costs to customers. But what about everyone else? What happens to people that can't afford the new cost of trust? Has anybody here changed the way they deal with financial verification as a result?

Anti-AI software is going to become big, I think.

I am quite tired of the AI slop and advanced scamming.
We need a way to filter these things.

AI has its limitations and those limitations should be poked and prodded at until something gives way.

[Lida93]

Honestly, security cost is quite alarming that's why individuals and big businesses are replacing human technology with AI Applications, since security check up is another vent of making the poor look helpless in every financial system, verification process with difficult task has made many loss opportunities and means of surviving because average cost is not affordable but we just hope there's is simplest means in future.

[Findingnemo]

Transferred 25 million dollars over the video call? For that big amount there must be a legal process involved with all the paper and documentation so it is not that easy unless someone is stupid enough and given that much money to manage.

But for the smaller amounts, it is possible, I heard a few stories that someone over the phonecall sounded exactly like a father or husband asked for an emergency need and asked to send money to an bank account but only later the person realized that they have been scammed.

SO keep the numbers saved in your contact and don't asnwer unless they call from that number.

But someone can be in trouble and call with another line, maybe they got involved in an accident and lost their phone or they were attacked and lost their phone and they needed to call and inform their family about the incident so at that point they can't call from the phone numbers that are being saved on our phone. This thing is a problem and I we have to be careful, but sometimes we don't have to neglect picking calls from unknown numbers.

 It could actually be someone that knows us that's calling and they need our help or want to pass urgent information that could save life. So my suggestion is this, if someone calls and pretend to be another person by using AI, and request for financial help, ask them few personal questions that only the real person can actually provide the answer, questions that can't be found online.

That is why I said for smaller amounts, it can be very danger but the amount in that situation might be less than $1000 or even smaller than that. Keep a secondary number too and your family member might know that number in case of such incidents because we get caught in those situation with very closed ones not just every family relative right?

[KiaKia]

Transferred 25 million dollars over the video call? For that big amount there must be a legal process involved with all the paper and documentation so it is not that easy unless someone is stupid enough and given that much money to manage.

But for the smaller amounts, it is possible, I heard a few stories that someone over the phonecall sounded exactly like a father or husband asked for an emergency need and asked to send money to an bank account but only later the person realized that they have been scammed.

SO keep the numbers saved in your contact and don't asnwer unless they call from that number.

Unfortunately the world is still full of stupid people, there are still many people who haven't fallen for scams like this and they have no reason to raise their guides up until they become a victim.

Ordinary home invasion looks like something that is so unreal in the face of many people today because they are living in a country where they can leave their doors open at night and nothing will happen.

About the number, may we not become a victim to evil situations, what if the person whom the phone number belongs to is already under captivity? Depending on the situation it will be very hard to tell, because they can still use the phone number to make that call, if that's your pass sign you still fail.

[Findingnemo]

~
About the number, may we not become a victim to evil situations, what if the person whom the phone number belongs to is already under captivity? Depending on the situation it will be very hard to tell, because they can still use the phone number to make that call, if that's your pass sign you still fail.

This is totally different situation, the person under the control of the one who is asking for money then we mght not have option other than paying the ransom that they were asking for because the other option is to contact the police which may work or not but we can't take chances with a closed one's life. And if we are capable of paying the money then we could do and secure the one that should be the priority because we can earn back the money later but not the life once it's lost.

[justdimin]

This is totally different situation, the person under the control of the one who is asking for money then we mght not have option other than paying the ransom that they were asking for because the other option is to contact the police which may work or not but we can't take chances with a closed one's life. And if we are capable of paying the money then we could do and secure the one that should be the priority because we can earn back the money later but not the life once it's lost.

What you are talking here is pure kidnapping. There is not much we can do instead of sending the money in this situation because no matter what, we can't take the risk of our loved ones getting hurt. For that, we will do anything possible and people mostly even prefer not to involve cops and resolve the matter quickly by giving the money. This is a much safer option but sometimes even the police can be of much help here depending on their resolution time. These kinds of kidnappings were quite common back then but as the rules and regulations have tightened, we rarely see any such scenario.

Scammers can still use AI to fake the kidnapping by spoofing the cell phone or by creating fake pictures and videos. Most people will be trapped here and will think this is for real and will send the money without having any extra thoughts. This is quite dangerous to be honest, so I believe in whatever OP said.