Virtual production has rapidly expanded in the film, television, and event industry; it’s clearly a must-have skill for future media professionals. With this in mind, schools and universities are investing in virtual production infrastructure to get students the hands-on experience they need.
In a sign of the times, one of the most important virtual production programs in the country will debut later this year. At the state-of-the-art Martin Scorsese Virtual Production Center at Industry City, NYU Tisch School of the Arts will offer a one-year advanced degree program starting in the fall. The new Master’s of Professional Studies degree in Virtual Production blends cutting-edge technology, science, and creativity to advance the art of storytelling.
NYU’s Martin Scorsese Virtual Production Center at Industry City encompasses a 45,000-square foot facility with a new Virtual Production LED Volume Stage designed by industry pioneer Lux Machina Consulting. The center will have two double-height, column-free stages, two television studios, and cutting-edge broadcast control rooms.
The new degree and facility will further establish Tisch as a global leader in cinematic arts.
“Tisch is excited to be a pioneer in training students on groundbreaking technology that is transforming the film and television industry. Our new master’s program keeps Tisch at the forefront of cinematic education by preparing students to enter the rapidly evolving field of virtual production,” said Allyson Green, Dean of NYU Tisch School of the Arts.
The degree requires 36 credits of coursework, including introduction to virtual production, technical direction, art direction and production design, resume and reel building, and a master’s seminar that covers writing, directing, visual storytelling and postproduction. Classes will include lectures, discussions, screenings, critical and creative writing workshops and collaborative exercises in production techniques.
To support the program, NYU turned to Lux Machina Consulting, a leader in virtual production, display technology, and in-camera visual effects, to create the production stages that are integral to the program. Working with the company that developed the technology allows NYU Tisch to train students in a real-world setting.
“It is a privilege to partner with a leading film school like New York University Tisch School of the Arts to give the next generation of filmmakers a hands-on, immersive experience in redefining the frontiers of storytelling,” Lux Machina Consulting President of Operations Zach Alexander said. “To set a gold standard on an accredited program created by two educational media powerhouses like NYU and LuxMC is a legacy we are proud of.”
Virtual production combines physical filmmaking with computer technology. Using gaming software, graphic cards, and augmented reality, it allows directors and actors to work with visual effects in real time, rather than in post-production. Actors can respond naturally to their surroundings and directors are able to plan out scenes and shots with precision. Virtual production reduces postproduction costs and the need to film on location, making projects more sustainable. This technology has been used in a range of films and television shows, including such projects that Lux Machina Consulting worked on as The Mandalorian, House of the Dragon, Bullet Train, and Barbie.
Meet the Designer
Lux Machina Consulting develops and engineers leading-edge technical video solutions, specializing in virtual production, in-camera visual effects, display technologies, and creative screen control. The company’s designers, producers, and engineers are uniquely positioned to help guide projects from ideation to implementation, tailoring solutions to individual project needs and creative goals. LuxMC works globally to help produce some of the most technically sophisticated projects, including the blockbuster films Top Gun: Maverick, Bullet Train, and Barbie, TV shows like House of the Dragon and Hijack, and live broadcast events such as the televised Academy Awards ceremony.
Louisiana State University (LSU) was awarded a five-year $1.25 million grant from Louisiana Economic Development to establish the LSU Virtual Production Program, a dedicated curriculum and production environment to cultivate a new generation of students fluent in the technologies and approaches of emerging media filmmaking. Following the grant, the university opened a new virtual production (VP) studio. Located at LSU’s Digital Media Center, the studio provides students with the opportunity for hands-on learning experiences using VP methods.
Several LSU programs are involved with the new studio including Digital Art, Film & TV, Experimental Music & Digital Media, Screen Arts, Digital Media Arts & Engineering, Electrical Engineering and Computer Science, as well as the LSU Center for Computation & Technology. The studio itself is outfitted with the latest equipment and technology in VP. The setup includes an LED volume consisting of a 20-foot-wide by nearly 10-foot-high LED video wall that is powered by Unreal, in addition to other technologies.
For camera tracking and motion capture, LSU selected an OptiTrack camera tracking system with eight PrimeX 22 cameras. Featuring edge-to-edge coverage across any capture area, the OptiTrack system delivers spot on camera-tomarker range and 3D accuracy.
Also included is OptiTrack’s CinePuck—a highend, optical-fused inertial measurement unit (IMU) tracking tool for VP studios that can be seamlessly integrated into any production workflow, even crowded rigging configurations, using a variety of mounts.
“The grant allows us to not only build out the hardware, but also plan more long-term,” said Derick Ostrenko, associate professor at the LSU College of Art & Design. “We are developing degree programs and curriculums, and have been teaching foundational classes in Unreal Engine, filmmaking and production. More recently, we’ve created classes covering different techniques of VP as well as extended reality (XR) performance using motion capture.”
Since opening, interest in the virtual studio has grown across campus. “It really has broad appeal,” said Marc Aubanel, director of Digital Media Arts & Engineering at the LSU College of Art & Design. “We’ve had several programs inquire about the studio, from the English department to Athletics which is looking at potentially using it for recruiting.” Jason Jamerson, LSU assistant professor of Virtual Production and Immersive Media who holds a joint appointment in the School of Art and the School of Theatre, has been using the virtual studio to teach film production classes.
“We’ve created music videos, documentary segments and have shot tests for weather, commercials and infomercials,” Jamerson said. “The ability to track the camera and get parallax offers so many advantages over a green screen. I’m planning on moving my film production class to the studio so we can work with the equipment full time.”
The new studio has also opened the door to exciting collaborations. NASA awarded LSU $5 million to create a digital twin of the main factory at Michoud Assembly Facility which provides vital support to NASA exploration and discovery missions. Using OptiTrack, Ostrenko, Jamerson and Aubanel lead the development of a real-time digital representation connecting the real location to the digital one and enabling space industry professionals to test designs at the VP studio from anywhere in the world. The experience also led to the development of a class on digital twinning.
Prior to building the studio, LSU evaluated a range of VP equipment and technologies as part of their selection process. For a tracking system, the team was intent on acquiring a robust solution that could handle a lot of use cases, according to Aubanel. “We needed to track cameras, but we also wanted the ability to track people and objects,” he said.
They talked to multiple vendors and looked closely at two options, both of which were singlesolution products. “They worked great, but each system could only solve one problem,” he said. “For example, you could track a camera but nothing else.”
With OptiTrack, they found a solution with the flexibility for multiple applications. They also get the benefit of a tracking system that is optimized for low latency throughput into Unreal. “Obviously, for live and virtual production, latency is very important,” Aubanel said.
Prior to building the studio, LSU evaluated a range of VP equipment and technologies as part of their selection process. For a tracking system, the team was intent on acquiring a robust solution that could handle a lot of use cases, according to Aubanel. “We needed to track cameras, but we also wanted the ability to track people and objects,” he said.
Outside the realm of VP, LSU has developed classes in XR performance using OptiTrack for motion capture in unison with live productions such as musical concerts, theater and dance. PhD students from the Experimental Music & Digital Media program partnered with students from Digital Media Arts & Engineering to use the system for live events. This included tracking dancers and recreating the choreography with avatars using Unreal Engine in real-time. “We did everything from aerial silks to standup comedy to interpretive dance and generative visuals,” Ostrenko said.
OptiTrack was appealing to LSU in that they could acquire a system at a reasonable cost, then build it out depending on the directions they decide to take. “If we want to expand our volume, we can just add more cameras,” Ostrenko said. “It’s repositionable and we can track many different types of objects.” The system also gives them the optionality to experiment and try new things—ideal for a university environment. “For instance, we are interested in doing measurements in virtual space by placing markers on the corners of the LED video wall or on top of lights,” Ostrenko said. “There is a long history of researchers and scientists working with motion capture. It’s a closed system, but it has a lot of ways of interacting with the data that comes out. From a research institution point of view, it’s appealing to explore things that people haven’t really done before or figured out yet.”
In January, Webster University in St. Louis, Missouri opened a brand-new virtual production stage—the first university virtual production facility in the Midwest. The $7 million renovation and technology refresh of its School of Communications, including the school’s entire 66,000-square-foot Sverdrup Building. The refresh included a new API Audio AXS Legacy console along with a pair of 1234A Smart Active Monitors (SAM) as the main L-R array, a 7382A Smart Active Subwoofer and a pair of 1032A Studio Monitors as nearfields, and a new marquee virtual studio setup. ARwall, in partnership with LED manufacturer Absen delivered a new 100-panel 20×12 ft. 1.5mm pitch Absen AX Pro LED wall and ARFX Pro Server System as the centerpieces of the refresh; the house cameras are ARRI. Integrated and driven by ARwall servers and software, the LED wall is one of the few available to students interested in virtual production nationwide.
Leveraging Unreal Engine, students in 3D animation, computer gaming, film and video production and other disciplines in the School of Communications have access to the virtual studio and LED wall. ARwall’s ARFX Pro Plugin technology requires no coding or 3D design knowledge, which greatly increases its usability. ARwall’s on-set workstation includes its ARFX Pro Plugin, as well as the ARFX Launcher. The tools are powered by Unreal Engine 5, a real-time 3D creation platform. The ARFX Pro Plugin combines pre-visualization, production, and post in a single tool set.
Webster University, ranked Top 50 Best Colleges in Film and Photography in the U.S. by Niche.com, commits to a hands-on approach when it comes to student programs. The school recognized the talent crisis in the production industry and is taking action by fueling its students’ educational and creative projects.
“Virtual production is now an integral part of the overall production landscape, so future professionals should learn these techniques in the same ways as they learn about photography, lighting, editing and other filmmaking processes,” says Eric Rothenbuhler, Dean of the School of Communications, Webster University. “The industry is always evolving and we want to make sure our students have access to cutting-edge, innovative tech that will set them up for success in their careers.
“Up until about a year ago, our plans were for a great big cyc wall, a bigger, better cyc wall than we’d ever had before,” Rothenbuhler shares. “We were in a faculty meeting in December [2021] and a faculty member raised his hand and said, “We don’t need a cyc wall, we need an LED wall. We need virtual cinema.” I thought, there’s no way we can afford that, but then another faculty member spoke up, and I started thinking about it. We talked with the architects, and we started doing some research and began to think maybe we could afford this. The more we looked into it, the more we thought we just about have to do it. It was an opportunity to be the first in the region.
The more we looked around and called around the more special we realized we could be. We found only one other school that had one already. I’m sure there are a few. But there can’t be more than a handful of schools or educational institutions across the country that have the virtual cinema system. More will one day have it, but right now, we’re first in the region and one of very few in the country. That’s a very exciting thing.”
Brian Macauto, director of virtual production at Absen, said the company’s AX Pro LED wall includes several features that are well-suited for VP, including a high base refresh rate, accurate brightness, and precise color temperature levels. “In a normal LED application, none of these things really matter,” he said. “In the cinema world, when you’re editing your footage and you’ve gone from Shot A to Shot B and the red brick wall—the background—changes from red to orange, then you have to change it all in post-production.”
Rene Amador, CEO and Co-Founder of ARwall said, “Webster University is full of people eager to take advantage of this tech and move production control further into the hands of creatives and filmmakers. Students can finally realize the projects they couldn’t create before. It’s so important to get hands-on experience when you’re studying. Having access to the same tools used on major productions will no doubt open many doors for these students.”
Frank Ren, Chief Technical Officer, Absen: “The mission of Absen is to make the world shine more, and this project reflects this mission. The hands-on experience that students will receive while learning how to create digital films on virtual studio volumes is something they will take into their future careers. We are glad to work with great partners like ARwall and Webster University for solutions that impact the industry. We will continue to offer the best product for the virtual production industry and help the ecosystem grow.”
