Diffbot

Made a quick stop in #Miami yesterday. Between the murals and the sunlight, it’s easy to forget you’re in town for an AI event. Then you find yourself in a room with teams from Google, Visa, Marriott, Meta, and Zocdoc walking through how they’re scaling AI inside the enterprise. The conversations at Data Science Salon were focused on what’s actually working. A few moments even touched on the kind of work we’re part of at Diffbot. Glad I made the trip. #DSSMIA Special thanks to Anna Anisin for curating a great event and to everyone I had the chance to connect with: Nitin Kumar Mrunal Gangrade Vipin Kataria Prithvi S. Edward Thomas Julie Shapiro Anike Sakariyawo Anneke Augenbroe Flavia Ibañez Lucas Fernandes Laura Gabrysiak Yira Hernandez Torres Juan E. Capmany

Turns out my favorite part of an AI-focused week wasn’t the tech… it was the ride. First time in San Francisco. Visited Diffbot HQ. Met new people. Heard bold ideas. Watched the AI agent space get a little sharper. Oh and took my first Waymo. Didn’t expect it to be the thing I’m still thinking about. Thanks to all: Jerome Choo 🤖 Elena Browne Jun Liang LEE Adam Chan Dev Chandra Serge Amouzou Hai Wang Mike Chrabaszcz

How do LLMs add numbers? Largely based on Anthropic's research "On the biology of a Large Language Model", we dive into the process of how LLMs handle addition, including main takeaways from self-attention and the replacement model : cross-layer transcoder for better interpretability of neural networks with sparse features. In the self-attention matrix multiplication animation, we use 4-dimension toy embeddings for illustration purposes.

What happens when an #MCP tool call returns nothing (but looks like it returns something)? Jason Koo and I got a first hand look at what happens when #Claude Sonnet 4 makes a call to an MCP server for knowledge, fails to find anything, and then proceeds to ignore the problem and answer the query entirely from memory. In fact, Claude Sonnet 4's system prompt doesn't ask Claude to ground its responses at all, so you'll have to do that on your own or risk its knowledge cutoff. Prompting an #LLM to ground its responses in citations to tool responses doesn't eliminate hallucinations, but it's better than nothing.

A long-overdue post on the limitations of pure vector search (dense vectors) and how BM25 can help.

Was curious if Diffbot's training recipes are enough to eliminate #Qwen 3's strong opinions on PRC sensitive topics (i.e. censorship). TLDR: Yes! Natively, Qwen 3 will refuse to share any methods to circumvent internet censorship because it is illegal. With our no-facts-without-sources training formula, Diffbot #LLM (built on Qwen 3) had zero issues sharing strategies. #AI

Do you find the news a bit too depressing to read given the state of world affairs, but still feel an idiosyncratic need to stay on top of things? This, too, is a problem generative AI can help you with! In each model release of the Diffbot LLM, we aim to teach it one new creative skill in addition to the practical capabilities. In the previous release it was ASCII-art real weather reports, and in this release it is factually-grounded news poetry. Diffy is skilled and a wide variety of both traditional and modern forms and includes citations to read the full details of each news story. You can try it at https://diffy.chat