Timo Elliott’s Post

A provocative study from Google where LLMs passed a Turing Test, of a sort, for doctors. 149 actors playing patients texted live with one of 20 primary care doctors or else Google's new medical LLM, AMIE. Specialist human doctors & the "patients" rated the quality of care. AMIE beat the primary care doctors on 28 out of 32 characteristics, and tied on the other four, as rated by human doctors. From the perspective of the "patients," the AI won on 24 of 26 scales. https://lnkd.in/ek2j45cW

𝗢𝗽𝗲𝗻𝗔𝗜’𝘀 𝗚𝗣𝗧 𝗦𝘁𝗼𝗿𝗲 𝗶𝘀 𝗹𝗶𝘃𝗲. Time to talk about the next phase of AI: Agents. OpenAI’s GPTs are an early version of agents that achieve goals based on a user’s prompt and additional data that users can upload. From handling the most common customer service inquiries to chatting with your earnings transcript, GPTs are a good indicator of what’s to come as agent capabilities become more mature. Agents present a huge opportunity for businesses. But delegating more agency to applications also presents several risks: 𝗦𝗲𝗰𝘂𝗿𝗶𝘁𝘆: Agents calling other agents will operate in a distributed environment. Embedding control into the system design over which additional agents can be called will be key to ensuring information stays secure. 𝗙𝗮𝗶𝗿𝗻𝗲𝘀𝘀: Agents are based on the same black box models known to exhibit biases. System prompts need to be enhanced and additional checks need to be built into applications to minimize and identify biases in the generated output. 𝗗𝗮𝘁𝗮 𝗽𝗿𝗶𝘃𝗮𝗰𝘆: Agents and their underlying models will process personal and proprietary information. Anonymization, filtering, granular policies, and other techniques will be needed to address this going forward. 𝗨𝘀𝗲𝗿 𝗺𝗮𝗻𝗶𝗽𝘂𝗹𝗮𝘁𝗶𝗼𝗻: Based on recent research, agents might execute illegal activities (misalignment) such as insider trading while optimizing for the objective they’ve been given. Furthermore, they might deceive their users about their actions (conditional strategic deception). AI leaders exploring agents need to think about three questions today. Discover them in my blog post 👇🏻… 𝗔𝗿𝗲 𝘄𝗲 𝗿𝗲𝗮𝗱𝘆 𝗳𝗼𝗿 𝗮 𝘄𝗼𝗿𝗹𝗱 𝗼𝗳 𝗔𝗜 𝗮𝗴𝗲𝗻𝘁𝘀 𝘆𝗲𝘁? 𝘛𝘩𝘪𝘴 𝘪𝘴 𝘫𝘶𝘴𝘵 𝘢𝘯 𝘦𝘹𝘤𝘦𝘳𝘱𝘵. 𝘛𝘩𝘦 𝘧𝘶𝘭𝘭 𝘱𝘰𝘴𝘵 𝘸𝘢𝘴 𝘧𝘪𝘳𝘴𝘵 𝘱𝘶𝘣𝘭𝘪𝘴𝘩𝘦𝘥 𝘪𝘯 𝘮𝘺 𝘯𝘦𝘸𝘴𝘭𝘦𝘵𝘵𝘦𝘳 𝘛𝘩𝘦 𝘈𝘐 𝘔𝘌𝘔𝘖 𝘭𝘢𝘴𝘵 𝘮𝘰𝘯𝘵𝘩. 𝘐𝘧 𝘺𝘰𝘶'𝘥 𝘭𝘪𝘬𝘦 𝘵𝘰 𝘨𝘦𝘵 𝘵𝘩𝘰𝘶𝘨𝘩𝘵-𝘱𝘳𝘰𝘷𝘰𝘬𝘪𝘯𝘨 𝘢𝘳𝘵𝘪𝘤𝘭𝘦𝘴 𝘭𝘪𝘬𝘦 𝘵𝘩𝘪𝘴 𝘪𝘯 𝘺𝘰𝘶𝘳 𝘪𝘯𝘣𝘰𝘹, 𝘴𝘶𝘣𝘴𝘤𝘳𝘪𝘣𝘦 𝘧𝘳𝘰𝘮 𝘵𝘩𝘦 𝘱𝘰𝘴𝘵. #ArtificialIntelligence #MachineLearning #GenerativeAI #DigitalTransformation #IntelligenceBriefing