Person no go fit just send AI message to steal crypto, this hackers only use AI for some things which are, using AI to identify potential security weaknesses, create convincing phishing messages, automate attacks, assist in developing malicious software and they can use it to Impersonate and make their hacking easier.
This is a lie, you cannot ask AI for phishing answers and it offers you, it had been trained not to answer such questions. I have asked AI many similar questions to give me a clear answer hot to create phishing link or bypass certain procedures to get quick results, but AI told me she doesn't deliver such messages.
What hackers may do is to ask the question indirectly, so that AI will only give a legitimate answer that'll give the hackers clue on how to proceed with their dirty job.
You know there is a good template how to ask AI questions and then get direct answer how you want it to be. These hackers understand that and use it very well to their advantage.