Unreal Engine 5.1 ships with an updated ML Deformer which can greatly improve mesh deformation in real-time. Basically you can train it on any deformation. In the example below it is trained to approximate the results of a FEM solver that tries to preserve volume. However, you can use anything (for now without dynamics). We have built a framework that allows people to build custom ML mesh deformation models. The framework gives you an editor to train, inspect and test your deformation on a given character. It also gives some ML Deformer asset type and ML Deformer component, to have a unified user experience. UE 5.1 ships with a model that we developed and call Neural Morph Model. The great thing about this is that it is relatively light weight in both memory usage and performance. The ML Deformer for the model below uses around 1 megabyte of memory on the GPU and it evaluates on latest consoles below 100 microseconds per character on CPU and below 50 microseconds on the GPU. We hope to still reduce the CPU time significantly. While the Neural Morph Model is good for body deformations, and can also handle certain types of clothing, we also have another model in production that performs better on clothing. The goal of this feature is to bring VFX quality deformation to real-time. Or at least, close to that quality. The video below shows the difference between a character that is linear skinned (including helper bones), and with ML Deformer enabled. I am enabling and disabling it throughout the video. Notice the chest area. I know the character in this video might not be the best example, as it is an older Paragon character, but I hope I can show more later on! :) #ue5 #mldeformer #machinelearning #unrealengine5
John van der Burg
Principal Animation Programmer at Epic Games/Unreal Engine