Kakao Brain, the AI research unit of South Korea's web service company Kakao, has developed a new technology that would expand the capabilities of smooth and fast face-swapping through enhanced identity embedding and simplified architecture. The company predicts its new technology is expected to develop various kinds of digital humans.
Face swapping refers to the task of swapping faces between images or in a video, while maintaining the rest of the body and environment context. It is commonly used to change faces taken on social networking services to other faces. Kakao Brain said its new technology called " Smooth-Swap" would significantly reduce the complexity of face-swapping's architecture and have great potential for commercialization.
The innovative model has replaced complex external modules of existing systems with an integrated smooth identity embedder to facilitate faster, more stable face swapping, the company said, adding that the simple architecture and enhanced performance of Smooth-Swap enable face-swapping during video playback.
"I strongly believe this technology will accelerate innovation in the face-swapping sphere, bringing us another step closer to the incredibly immersive metaverse we always dreamed of as wells as the digital human services of the future," Kakao Brain CEO Kim Il-doo said in a statement on May 9.
An accurate and consistent identity gradient is essential to seamlessly changing a person’s identity without sacrificing the image’s high quality. Trained via supervised contrastive loss, Smooth-Swap acquires its stable identity gradient by learning embedding with a higher smoothness, addressing the earlier model’s weakness of adding handcrafted components and 3D face modeling which ultimately complicated its design and entailed sophisticated hyperparameter tuning.
Smooth-Swap relies on a simple U-Net-based architecture with an integrated smooth identity embedder, suggests a differentiated identity embedding approach, and empowers the generator to create higher-quality images, especially when changing a subject’s face shape, according to Kakao Brain.
Kakao Brain is trying to develop an artificial brain and a digital human that combines artificial characters with virtual human figures and travels around metaverse spaces. The metaverse world is a collective virtual shared space, created by the convergence of virtually enhanced physical reality and physically persistent virtual space.
