Comparisons with Previous Methods on LRS3

[1] Lee, Jiyoung, Joon Son Chung, and Soo-Whan Chung. "Imaginary voice: Face-styled diffusion model for text-to-speech." ICASSP. IEEE, 2023.
[2] Lee, Minyoung, Eunil Park, and Sungeun Hong. "Fvtts: Face based voice synthesis for text-to-speech." Interspeech 2024, 4953-4957.

x

Input Face	FaceTTS [1]	FVTTS [2]	Ours
	I'M NOT A SOCIAL PROBLEM GUY
	AND THE WAY TO DO IT IS THROUGH THE BLOODSTREAM
	THAT'S THE WORK THAT STANDS BEFORE US
	I WAS TERRIFIED
	AND WHO DOES THAT BENEFIT
	I HAVE TO REPORT IT
	THIS IS WHEN I WAS IN TEXAS
	BUT THINGS HAVE CHANGED

Comparisons with Previous Methods on Artistic Portraits

Input Face	FaceTTS [1]	FVTTS [2]	Ours
	Our prime purpose in this life is to help others. And if you can't help them, at least don't hurt them.
	Remember your dreams and fight for them. You must know what you want from life. There is just one thing that makes your dream become impossible: the fear of failure.
	Life is not a problem to be solved, but a reality to be experienced.
	In three words I can sum up everything I've learned about life: it goes on.
	Transformation is a process, and as life happens there are tons of ups and downs. It's a journey of discovery, there are moments on mountaintops and moments in deep valleys of despair.
	Life is not about how fast you run or how high you climb, but how well you bounce.
	Our prime purpose in this life is to help others. And if you can't help them, at least don't hurt them.
	Transformation is a process, and as life happens there are tons of ups and downs. It's a journey of discovery, there are moments on mountaintops and moments in deep valleys of despair.
	Life is ten percent what happens to you and ninety percent how you respond to it.
	Life isn't about finding yourself. Life is about creating yourself.
	Literature adds to reality, it does not simply describe it. It enriches the necessary competencies that daily life requires and provides; and in this respect, it irrigates the deserts that our lives have already become.
	Transformation is a process, and as life happens there are tons of ups and downs. It's a journey of discovery, there are moments on mountaintops and moments in deep valleys of despair.

Consistent Voice Generation

Input Face	Prediction 1	Prediction 2	Prediction 3

By using different words in descriptive text