IF YOU’VE NOT YET discovered what AI has to offer you as a creative then it’s time to find out, and to jump on board fast. This is an area that’s exploding, and those who are fully tuned in, such as award-winning director Karen X Cheng, are taking full advantage of its fast expanding and cutting-edge possibilities.
With a host of hugely inventive and eyepopping productions on platforms such as Tik Tok to her name, Karen can boast over 500 million views to date and counting. Aided by the technical wizardry of experts such as top software designer James Perlman, she’s inspired by the diverse range of programs that AI is powering, and eager to come at them from her often quirky left field standpoint to produce razor-sharp snippets of dynamic motion that is adored by her growing fan base.
Of course, AI packages, clever as they might be, are only made possible by the power that lies under the bonnet of your PC, and NVIDIA products are fuelling the revolution, and driving this sector. Karen’s Asus Zenbook Pro Duo, for example, which features at its heart an NVIDIA GeForce RTX 3080 GPU, comes primed and ready for all kinds of exciting AI adventures. One of the most impressive of these is provided by the NVIDIA Canvas app, a free download that’s available to anyone using the company’s GeForce RTX GPUs.
The app uses AI to interpret basic lines and shapes, translating these into realistic landscape images and textures, and seeing is believing as simple brushstrokes are instantly turned into incredibly lifelike images. It’s a stunningly simple process that opens up all kinds of creative possibilities, while also making it easy to provide awesome visuals for those looking to sketch out ideas to share with the likes of art directors or location scouts.
“The first time I tried it out it was amazing,” says Karen. “Seeing in real time how this software reacts to your brush strokes is really incredible. There are a lot of AI programs where you can type a prompt and then have to wait for ten seconds for something to happen, but with Canvas everything happens instantaneously: you can touch it, you can feel it. And I was working with a computer that has a stylus, and so it was actually like drawing directly on the screen. It was like magic was coming out of my fingers.”
Like so many true artists, Karen takes her inspiration from the things she finds around her, and AI comes with the potential to constantly enthral and entertain her. It leads her imagination into fresh areas as she delights in breaking the rules and seeing what comes out the other end. “I find it so exciting when new technology comes out,” she confides. “I’m always afraid I’m going to run out of ideas, so whenever there’s anything new to take a look at, I just can’t wait to go and try it out.”
James is her co-conspirator and key AI trouble-shooter, and he’s now well used to Karen enthusing about fresh programs that need to be checked out and played with. He’s likewise been blown away by Canvas and the possibilities it presents – “it’s beautiful and cool” – and he’s continually trying out new software packages to look closer at what they have to offer.
“The overarching theme is trying to figure out how to use this technology,” he says. “We’re looking out for cutting edge research, and sometimes the programs we’re working with are very much in the early development phase, and representing a new concept. And when we get word of them and think there’s some kind of artistic purpose we can put this technology to, we try to figure out how to use and apply it.”
THE AI PLAYGROUND
With NVIDIA technology providing the power, the world of AI is opening up to those like Karen and James who are keen to explore. Take the program known as NeRF for example: this incredible piece of software enables a three-dimensional capture of reality that, in the right hands, can create SFX that wouldn’t look out of place in a high-tech sci-fi movie.
See here how Karen used DAIN to create ultra-smooth slow motion.
“It’s a huge improvement over the previous technology, which was called photogrammetry,” says James. “Using that process it was very difficult to get a decent result. You couldn’t get nice surface reflections, while handling something transparent, like water, was near on impossible. NeRF technology came out around two years ago and it uses neural
“It’s a huge improvement over the previous technology, which was called photogrammetry,” says James. “Using that process it was very difficult to get a decent result. You couldn’t get nice surface reflections, while handling something transparent, like water, was near on impossible. NeRF technology came out around two years ago and it uses neural radiance fields to learn and understand the scene from multiple perspectives. I’ve been playing with it since December last year, and initially it was very slow to work with, and results could take several days to come together. And then NVIDIA came out with a project called Instant NGP, which Karen discovered, and this just changed everything. It reduced a process that could take four days to something like 20 minutes.”
This massive speeding up of the entire procedure not only saved time but it also opened a host of new creative doors. “If it takes four days to render something there’s not a whole lot of experimentation you can do,” says Karen. “With instant NGP we could test out different rooms, reflections, poses and human faces. We could ask questions like how still does a human need to be? How long would they need to stand still? What kind of clothing and fabric looks good?
“We fed something like 100 iterations into the system until we found out what worked, and we ended up with Hollywood level effects that were really empowering. We even simulated a drone taking off without leaving the ground, impossible if everything we tried out took several days.”
Another spectacular SFX enabled by AI was DAIN, which enables ultra-smooth stop-motion, transforming a process that can often be notoriously jerky. “Again, this really transformed what we had been able to do previously,” says Karen. “Stop motion is traditionally used to create artificial slow motion, but I wondered if we could use DAIN to create impossible movements, such as my hat spinning around on my head or myself gliding across the floor. And we did so many tests, using so many different body positions, until we were able to zero in on the very specific cases where everything actually did look quite good, and then we posted those.”
While AI looks like an unstoppable force right now, it’s NVIDIA that’s doing the enabling, and supplying the momentum that’s required for the full potential of cutting-edge software to be realised. “I can’t say enough good things about NVIDIA,” says James. “The company is currently the premier hardware supplier in the entire AI world: there’s nothing else that comes close. You seriously wouldn’t be able to think about running most of the AI programs we’re using without a good NVIDIA chip. Right now, I wouldn’t even consider another piece of hardware to be honest: it really is that good.”
© 2022, Professional Photo Magazine and respective content owners.. All rights reserved.