Production Idea & Process

Experienced consecutive perishing of my grandparents. I failed to know it on time since my parents didn't want to disturb me during my academic periods. I felt regretful not able to talk to them lastly before they go.

I gathered a descent quantity of the past videos that included their talking, trained a text-to-sound AI model that produces my grandma’s voice, and used HeyGen to animate her based on the trained voice-over. Finally I made a memorizing video using clips made by AI, and let grandparents “chat” gain to each of my family members. In the video I made them greeted and blessed each of the family members.

The Challenging Part

The hard part was to conquer the text-sound conversion process. Although in recent days, there are plenty of tools and platforms to easily do text-speech conversion, the obtaining of descent quality, usable training data(sound clips) is a matter of searching and gathering. I spent a week searched and gathered as much video & audio based assets for my grandparents, some of them came from the obsolete recorded videos, then I denoise, filter, categorize and trim them as training clips.

The second part is to have the exported sound capable of speacking the accents of Henan, which is very different from how normal Chinese accent sounds like. I found a straightforward but effective method on solving this, I referenced the dictionary of rare word, and found the words that prononce like the same pronunciation of the wanted word in Henan accent, and I used the text script composed by rare words and fed to the model, and then got the desired result that sounds like a Henan accent.

Outcome

The completion of this video made me felt of meeting again with them, I found a lot of interesting stories when parsing the former video clips. The making of this video looks like a test of using the generative AI tool to mimic someone’s voice, but also to let us “talk” to them again.