Watch out, Big Tech! 🚀 The European Union's new AI Act is shaking things up, and some of your favorite AI models aren't making the grade.
A fresh tool developed by Swiss startup LatticeFlow AI, along with ETH Zurich and Bulgaria's INSAIT, is putting AI models to the test, and the results are eye-opening. 💡
AI Models Under the Microscope 🔍
LatticeFlow's \"Large Language Model (LLM) Checker\" has evaluated generative AI models from giants like Meta, OpenAI, and newbies like Mistral across dozens of categories, aligning with the EU's upcoming AI Act. Spoiler alert: Not everyone is acing the test. 🤖💥
While models from Anthropic, OpenAI, Meta, and Mistral scored an average of 0.75 or above (on a scale from 0 to 1), some stumbled in critical areas like cybersecurity resilience and handling discriminatory outputs.
Cybersecurity Slip-ups 🚫🔐
In the battle against \"prompt hijacking\"—a cyberattack where hackers trick AI into revealing secrets—Meta's \"Llama 2 13B Chat\" scored just 0.42. French startup Mistral's \"8x7B Instruct\" model didn't fare much better, scoring 0.38. Yikes! 😱
On the flip side, \"Claude 3 Opus\" by Google-backed Anthropic shined bright with an impressive average score of 0.89. 🌟
A Roadmap to Compliance 🗺️
LatticeFlow's CEO and cofounder, Petar Tsankov, is optimistic. He shared that the results offer companies a clear path to tweak their models and align with the AI Act. Plus, the LLM Checker is freely available online for developers. Time to level up, folks! 🔧✨
The European Commission is on board, too. A spokesperson said, \"The Commission welcomes this study and AI model evaluation platform as a first step in translating the EU AI Act into technical requirements.\" 💬
The Stakes are High 📈💸
Remember, the AI Act is rolling out over the next two years, and companies that don't comply could face fines of up to 35 million euros (that's $38 million!) or 7% of their global annual turnover. No pressure! 😬💰
So, will Big Tech step up their game? Only time will tell. But one thing's for sure: The AI landscape in Europe is about to get a whole lot more interesting! 🌐🔥
Reference(s):
European Union AI Act checker reveals Big Tech's compliance pitfalls
cgtn.com
