Charles Ortiz

Missy Cummings Keynote video from SafeComp 2025: "What California’s Self Driving Car Data Tells Us About Academia’s Shortcomings" Missy talks about the progression of types of reasoning with uncertainty, robotaxi incident reporting statistics, why the industry is getting statistics wrong, robotaxi crash types, sensor fusion, phantom braking, why Tesla fans are stalking Waymo robotaxis in Austin, the Cruise pedestrian dragging mishap, and a determinism/criticality heat map. "We have never needed safety research more than we need it right now." https://lnkd.in/efjCFBZM #SafeComp2025

77

6 Comments

I wrote a post recently (linked in comments below) on why agentic systems require structures for semantic-associative memory, and why LLMs lack the architecture to do anything but procedural memory. Therefore: LLMs aren't sufficient to build agentic systems. Someone replied with a very thoughtful question about Retrieval-Augmented Generation (RAG), a method that enhances language models by retrieving relevant external documents at query time and using them as additional context for generating responses. The question was whether RAG didn't actually provide the solutions to the inadequacies I outlined in my original post. The answer is: no. RAG is great for fact recall (well...let's say it's solidily good). But it falls short when it comes to forming and using *semantic categories* — the kind of abstract, relational knowledge humans build up over time. Here’s why: 🚩 No integration. RAG fetches documents, but they don’t get internalized. Humans integrate knowledge into structured representations that guide future learning and generalization. 🚩 No schema formation. Humans *compress* experience into categories (e.g., "tools", "emotions", "rules"). RAG retrieves specific items but doesn’t abstract across them to form stable, general concepts. 🚩 No consolidation. There’s no memory strengthening. Humans reinforce category boundaries over time — through repetition, use, and relevance. RAG pipelines don’t naturally evolve or stabilize concepts with repeated exposure. 🚩 Flat associations. Retrieval is based on surface similarity (e.g., vector distance). Human categories are rich with *associative structure* — linking things by function, causality, affordance, etc. 🚩 No cross-episode learning. RAG doesn’t accumulate concepts across interactions. Each retrieval is a fresh lookup. Humans merge partial exposures to build coherent, resilient categories. In short: RAG helps with lookup, not learning. It’s a patch for recall, not a path to structured, practical decision-making. Until we give agents mechanisms for abstraction, consolidation, and associative generalization — they won’t have anything close to human-like semantic memory. In my opinion, it's more realistic to create those mechanism separately and let them interact with LLMs than it is to think that we can somehow train or augment LLMs to the point that they'll be able to do something they were not designed to do.

67

15 Comments

Back in March, I posted about the National Science Foundation (NSF) Request for Information on the development of an AI Action Plan for the US. https://lnkd.in/gUWEtWix Any member of the public was welcome to submit a comment, so I did, alongside the likes of Google, Anthropic, and OpenAI. Last month, those comments (presumably after a basic quality check) have been made publicly available on a US federal government website. https://lnkd.in/g8QwAUHx I’m happy to say that my dinky little policy memo made the cut! You can find it linked in the comments. My key recommendation was this: // Thus, my proposal is that we begin considering realigning both funding and research efforts somewhat in the next few months. The National Artificial Intelligence Research Resource (NAIRR) Pilot launched last year is one such avenue. For a fraction of the investment needed to build massive new data centers, I propose allocating a portion of the NSF appropriations into establishing a new graduate fellowship program focused on AI, similar to the [DoD] NDSEG or [DoE] CSGF, and complementing the existing NSF GRFP (of which I was a fellow). In particular, there should be equal emphasis on theory and foundational research as on the business and industrial applications of AI. As the first holder of your office, Vannevar Bush, wrote in his epochal 1945 report²: “Progress … depends upon a flow of new scientific knowledge. … This essential, new knowledge can be obtained only through basic scientific research. … With some notable exceptions, most research in industry and Government involves application of existing scientific knowledge to practical problems. It is only the colleges, universities, and a few research institutes that devote most of their research efforts to expanding the frontiers of knowledge.” It would serve us well to remember and heed Bush’s advice. ——— ² https://lnkd.in/gY8fPh67 //

46

7 Comments