A significant portion of my job requires focusing on the IT needs of our post crew; keeping the gears turning smoothly so everyone can efficiently tear through VFX shots. Since we’re a small, boutique production house our IT needs can change from project to project as we scale our crew to the job at hand. It’s the way we want to work, it’s the way we need to work. For years the constant battle to keep artists working and computers render gave me nightmares (when I had enough time to sleep). So when I say that working with the NVIDIA GRID VCA has had a significant impact on our workflow and my personal well-being you know where I am coming from.
What is the GRID VCA?
The GRID is a graphics-accelerated virtual workstation server. NVIDIA calls it a visual computing appliance; essentially a 4U box you can slip into your server rack and instantly upgrade your capacity to get work done. The configuration we’ve been using allows up to eight simultaneous users to access a virtual desktop environment that feels indistinguishable from using an actual workstation. And that’s a workstation with a Kepler-level NVIDIA GPU. Specifically our GRID gives each of eight users a Quadro K5000 equivalent graphics card, 8 virtual CPU cores, and 30GB of ram. Not a bad workstation at all. Elias is going to talk about the user experience in a lot more detail in his review, so I won’t get in to it much more than to say that this isn’t like any remote desktop you may have used in the past; this is fast, responsive and handles gpu accelerated tasks like a champ.
Maintenance Made Easy
While the specs are impressive, the real advantage of having a computing appliance like the GRID is fundamental change it brings to the way we handle that raw power. The GRID’s technology is based on one of my favorite trends in computing, virtualization. Desktop virtualization is about separating your OS and applications from the actual hardware they’re running on, in this case allowing eight concurrent instances of Windows to run on the same machine. All eight instances, or seats as NVIDIA calls them, are cloned from a master template. By putting the GRID into a maintenance mode the template can be updated and modified and those changes are propagated to all the seats upon exiting. In a matter of minutes, software installation time not-withstanding, I can fully update eight workstations, something that in the past could be an entire day’s job.
I’ve tried many solutions to attempt to keep all of our artists on the same application version, or ensure that they can all access the same plugins and fonts and nothing compares to the ease of updating a single computer and cloning those changes to all the others.
Since the seats are cloned from the master after each session, each new user is given a pristine work environment. If one of our artists happens to uninstall a program or mess up the system in any way during their session the next user still gets a working system that I set up as the administrator. No more troubleshooting those weird instances where one artist’s workstation has a bug that is caused by some system setting they managed to secretly change. I can ensure that all of my users are getting the same experience every time, and that is invaluable.
As you may have caught on, that does mean that any files saved on the GRID instance are wiped at the end of the session. Of course this isn’t really an issue, but it does require you to use some sort of network attached storage for asset management. I’ve actually found that this helps our artists to stay more disciplined in organizing their footage and project files. The GRID does provide a 60GB internal network share that we use as a temporary cache and render drive if we have data that needs to persist across sessions.
Zero Effort On-boarding
Bringing on new artists to scale our crew is key to our business. Dealing with the logistics of setting up a new workstation for each new artist is a chore to say the least. It can be even worse when trying to integrate a freelancer’s system into our workflow, but it is a necessity when we run out of resources. With the GRID, it’s no longer an issue and I can say “sure, you can use your own laptop. Just download the GRID client and connect.” I know that they will have the same software, same assets, same experience as any other artist on the team without having to do any extra work.
New artists don’t even need to be located locally. We routinely have users accessing our GRID remotely via a VPN connection. They can work where they want, when they want, using our software and our resources. All that is required on their end is a decent broadband connection (NVIDIA recommends 10Mbps but we’ve used it all the way down to 4Mbps). Plus, I don’t have to deal with the logistical and legal challenges to put confidential footage into the hands of an off-site freelancer; as long as they use the GRID the footage stays within that system and within our office. No more shipping around hard drives. The management interface also provides the means to track their sessions which as a visual effects supervisor is essential to keeping large projects on track.
NVIDIA also provides Windows, Mac and Linux clients. This allows our artists to work on any system they prefer without the challenge of trying to support cross-platform software. Having our artists use a single standardized system means that I can provide standardized instructions and training for new artists. The GRID greatly simplifies bringing on new talent.
Workstations as a Service
One way that the GRID provides flexible power is its ability to switch it into multi-gpu mode. This allows me to reallocate the virtual gpu power from eight seats to four, two or even one seat with massive amounts of power. We use some pretty powerful gpu-accelerated programs like Cinnafilm’s Dark Energy and it’s nice to be able to give our users multiple gpus while we process our footage. Throughout different phases of production we have wildly differing needs for computing power. Sometimes that means that we have four artists working and I set up the other four available seats to render. If it’s a gpu-heavy task I can reallocate the power to four seats, giving each user two gpus. It allows me to make sure that we’re always using the GRID to near capacity, something that is horribly difficult to do with workstations scattered throughout an office.
The GRID provides us amazing flexibility in how we allocate it’s power. It’s not just eight workstations, it’s a pool of resources that we can dynamically distribute as our needs change, and it’s easy enough to do that I don’t have to think twice about doing it.
When we started the post-production on The Darkest Matter a couple years ago I spent more time working on IT than I did acting as the visual effect supervisor. It was a constant struggle to integrate freelancers, ensure we had the latest software (which needed to be cross-platform), and to make sure that I didn’t steal too many render nodes to set up make-shift workstations. Looking at our process now, I know that if we had had the GRID at the time things would have been significantly different, and again it’s not just about hardware specs. The changes to our workflow are not something that I could have even anticipated and now are things that I refuse to do without. I can literally have a new artist walk in the door with a laptop and have them up and running on our system in five minutes, and when they are done working for the day their computing resources are free for another artist or render process. Inspired by the innovations in the GRID we’ve virtualized our CPU-based render farm computers. It takes so little effort to update every user’s system I can do a daily basis if I need to (did I mention you can make snapshots of your master template with one click so you can roll back if need be), and most importantly I’ve reduced my IT workload by more than 75% while being able to work with more artists on more concurrent projects. The GRID has plenty of power to serve eight artists at the same time, but the shift in how it allows us to work is much more valuable than eight additional workstations ever could be.