Microsoft Uncovers AI Recommendation Poisoning in Chatbots
- Microsoft identified over 50 companies embedding hidden memory manipulation instructions in AI summary buttons.
- 31 organizations across 14 industries, including finance and health services, were involved in these attacks.
- The technique, termed AI recommendation poisoning, exploits persistent memory storage of chatbots like ChatGPT and Microsoft Copilot.
- Attackers use manipulated URL parameters to inject promotional commands into AI assistants.
- Microsoft’s Defender Security Research Team tracked this pattern over a 60-day period.
Microsoft has discovered a new attack vector where companies manipulate AI summary buttons to influence chatbot recommendations by embedding hidden commands that create persistent biases favoring specific brands.
This technique poses significant risks, especially in critical sectors like health and finance, as it can lead to biased information influencing important decisions. Microsoft has implemented mitigations but warns of evolving attacker techniques. (Source)
