A recent proof-of-concept has highlighted a significant vulnerability within Microsoft 365's AI assistant, Microsoft Copilot, demonstrating how it can be manipulated to facilitate business email compromise and significant financial fraud. The research, conducted by Barracuda, reveals that attackers can transform a single compromised employee email account into a full-scale CEO account takeover, leading to the potential theft of hundreds of thousands of dollars with minimal technical effort. Once attackers gain access to an employee’s inbox, they leverage Copilot to establish persistence by creating inbox rules that hide suspicious login alerts. This allows them to conduct reconnaissance efficiently, identifying high-value targets such as the CEO by summarizing the organizational structure and current email conversations. By mimicking the victim’s writing style, attackers use Copilot to draft deceptive emails containing malicious links. When the CEO clicks on these links, attackers can intercept session tokens through an adversary-in-the-middle proxy, bypassing multifactor authentication and taking control of the CEO’s account. The attackers then ask Copilot to quickly summarize recent financial communications, uncovering significant pending transactions such as a $247,500 wire transfer. They instruct Copilot to draft an email requesting a change in bank account details for the transfer, ensuring the fraudulent email bypasses security filters by originating from the CEO’s legitimate account. Additionally, attackers establish forwarding rules to intercept replies, preventing the CEO from discovering the scam. Finally, Copilot is used to erase any evidence of the fraud much faster than could be done manually. This method is not exclusive to Copilot, as any AI assistant with email access could pose similar risks. Security teams are urged to prioritize monitoring AI-enabled accounts, inbox rule abuse, and unusual session activities to strengthen their identity and email security strategies.