Xiyang Yeh

What can AI researchers learn from TPU and LPU chip design? There's a common misconception in chip design: that a specialized chip for a specific use case will always outperform a general one. Time and time again, we've seen that this isn't true. When we started the TPU project, we faced pushback from experienced engineers: "You won’t beat GPUs. They’re already optimized to a level you can’t surpass. Instead, we should build chips tailored to specific models: one for speech recognition, one for visual object detection, etc." Despite this advice, we had a strong instinct that specialization was the wrong path. We couldn’t articulate why at the time, but every successful chip before us had one thing in common: It was programmable. What ultimately convinced us was the math. When we ran the numbers, our general-purpose design outperformed even the best estimates for specialized chips. We took the same approach with the LPU. It was designed before attention models, before diffusion models, before many of today’s breakthroughs. Yet it performs well across all of them because we focused on making it general and optimizing every aspect of its architecture. The only thing we specialized for was inference. What are the lessons in this experience for AI researchers?

502

34 Comments

Fascinating insights from Dr. Vijay Narayanan's Distinguished Lecture, "Semiconductor Innovations for the AI Era," today at Rensselaer Polytechnic Institute. He illustrated the critical connections of research developments on materials, devices, front-end/back-end-of-line, chiplet heterogeneous integration, architecture, large language models (LLMs), etc., in shaping the future of AI. Equally important, his effort is fostering vital connections between academia and industry, i.e., university-industry engagement for the CHIPS R&D for the future of AI. Thank you, Dr. Narayanan! #semiconductors #AI #chiplets #research #partnership

166

Navigating the AI/ML revolution as an Electrical Engineer? With so much information out there, finding quality resources can be challenging. I recently completed UCSD Extension's "Embedded Machine Learning" course (link in comments), and it truly delivered. What impressed me most was the comprehensive curriculum, covering the entire pipeline from dataset preparation to model deployment on edge devices – a topic I'm particularly interested in. The course seamlessly blended theoretical understanding with practical, hands-on exercises. A huge shoutout to instructor Norman McEntire for his clear and engaging teaching style. If you're an EE or anyone interested in AI on the edge, I highly recommend checking it out. Have you discovered any valuable AI/ML learning resources? Please share your recommendations in the comments! Disclaimer: Just sharing a resource I found helpful, no affiliation with UCSD. #EdgeAI #ElectricalEngineers #ContinuousLearning

119

7 Comments

25,000,000,000 bits per second… That’s the promise of PoX, the new non-volatile memory developed at Fudan University. It is 10,000 times faster than current flash. They achieved this by replacing silicon with graphene, a single layer of carbon atoms known for fast, low-resistance electron flow. A technique called superinjection floods charge into the memory cell almost instantly. Together, these push past the old speed limits of flash and approach the raw speed of SRAM while keeping data persistent. It writes data in 400 picoseconds and keeps it even when the power is off. No more juggling between fast but forgetful SRAM and slow but stable flash. If this scales, it could flip the memory stack for AI hardware, remove the need for SRAM caches, and cut energy use across the board. That shift could benefit accelerators, edge devices, laptops, and data centers. They all need fast, persistent, low power memory. Daily #electronics insights from Asia. Follow me, Keesjan, and never miss a post by ringing my 🔔. #technology #innovation

301

54 Comments

The rapid disbandment of Tesla's Dojo project by August 2025, just two years after its production launch, illustrates the fundamental flaw in treating autonomous driving challenges as primarily a compute scaling problem rather than an algorithmic optimization challenge. While Tesla invested enormous resources into building custom silicon and massive data processing capabilities to churn through millions of terabytes of video data, companies focusing on algorithmic efficiency like DeepSeek have demonstrated that intelligent model architectures, better training methodologies, and optimized inference pipelines can achieve superior results with dramatically less computational overhead. The reasoning is straightforward: throwing more hardware at an inefficient algorithm is like trying to empty a swimming pool with a larger bucket rather than finding a drain—you're scaling the wrong variable. For example, DeepSeek's approach of architectural innovations and training optimizations has shown that you can achieve comparable or better performance in language models (and by extension, other AI tasks) while using a fraction of the compute resources that brute-force scaling approaches demand. Tesla's pivot away from Dojo suggests they might have ultimately recognized that the path to reliable autonomous driving lies not in building bigger video-processing monsters, but in developing smarter algorithms that can extract meaningful patterns from data more efficiently and targeting specialized operation domain (e.g., public transit, freight).

23

1 Comment