Why asset-centric workflows are the future of our pipelines

The media and entertainment industry is constantly changing. Each year, we’re presented with new technologies and processes, all of them promising to change the way we work and give us more streamlined pipelines. It can be confusing, especially with everything developing at such a rapid pace; it can be hard to know what’s here to stay and what is a passing fad. 

Over the past decade, we’ve seen a shift from the more traditional, centralized way visual effects (VFX) studios work to cloud and hybrid workflows. Traditionally, studios kept all their data and equipment on-site. As studios took on more work to keep up with the rise in content, problems began to arise. Studios had to upscale, getting more equipment and space to handle the sheer volume of work they were inundated with. 

Studios have long been looking to find answers to established bottlenecks of hybrid and cloud setups For example, how do you manage work that crosses countries when all your data is stored in one place? This process was accelerated over the past few years due to the effects of the pandemic and much of the world working remotely.

A data problem

While the scalability of cloud resource presents a golden opportunity to accelerate VFX pipelines, there has been some hesitance in adopting the cloud as part of the solution. This is, in part, due to data. 

Studios use and are reliant on extreme amounts of data which, in turn, requires lots of hardware to store it. Having all this data then makes it difficult to transfer or upload files they need—it’s not as simple as moving it from one place to another as we do on Google Drive. It can take several hours or even days to transfer a single render. This slows down the process as it means decisions can’t be made and implemented on the spot, they have to wait until they have the render. This means if studios want to upload anything to the cloud, it takes up time and resources to do so and many will still need to scale up to counteract this.

To negate the bottlenecks, studios have begun to evolve their own asset management systems and incorporate them into hybrid cloud setups. This creates dynamic and distributed workflows which allow assets to exist in different places whilst also taking advantage of the cloud. A challenge here is that the storage systems and APIs used by commercially available cloud solutions don't work in the same way as traditional on-site file storage systems. Studios have to spend a lot of time trying to connect all these different pieces of software together so they can get the right data in the right place as it’s needed. This has created a great deal of complexity and hasn’t had the positive effect on efficiency that many studios hoped it would. 

What we’re also seeing as an industry is pipelines becoming increasingly non-linear with productions no longer following the traditional structure so many of us are used to. Instead, technology like real-time and machine learning is allowing for post-production to happen a lot sooner in the pipeline. Studios need workflows that can support multiple artists from different disciplines and let them work concurrently and collaboratively with ease. 

Enter: asset-centric workflows as first-class citizens in content creation tools.

All your assets, whenever you need them

First-class asset-centric workflows are designed to help solve problems of data locality more effectively. Pipelines exist to try and help people manage data. DCC tools and applications should treat the pipeline as a first-class concept, and by extension, treat assets the same way. As software creators, we should be doing everything we can to facilitate asset-centric workflows in our products.

The result of this would be enabling artists to work faster, more efficiently, and with less pipeline friction, whilst also building new workflows and having the ability to collaborate with studios and artists worldwide. It also means that artists can share assets and renders more easily, allowing for quicker turnaround times on decisions and feedback. 

Asset-centric workflows are an essential prerequisite to building data management systems that enable a studio to take advantage of scalable cloud-based compute power. Studios won’t need the equivalent of a machine per artist, but rather a pool of resources that can be shared as and when needed. This way, studios can ensure their resources are continually optimized to what their artists need. Getting renders done quicker using on-demand cloud resources.

Here at Foundry, we’ve been tracking the evolution of asset-centric workflows and exploring the challenge of scale and how to securely move data at the different stages of production. OpenAssetIO is one of those solutions. It is open source and aims to simplify how DCC tools integrate into production pipelines by reducing the ‘glue code’ that studios have to write. This makes for a more consistent and robust artist experience across tools and aims to make relocating data between sites easier. 

OpenAssetIO will help product engineering teams to design and build new asset-centric workflows. Adoption of common open standards will allow for the flexible interchange and editing of content by a wider variety of stakeholders across a broader spectrum of the production pipeline. 

Data set management APIs are another area that Foundry and our Research team are exploring. We’re aiming to make it easier to integrate our products and use the cloud for fast artist feedback. Open Data Manifests is a project in the pipeline for next year and will help determine the data needed for any particular task to simplify the packaging of work to take advantage of on-demand resources such as the cloud. We are working with key industry partners from both studios and ISVs, as well as institutions like the Academy Software Foundation and MovieLabs to validate our direction and inform new projects. 

Hybrid working environments and asset-centric workflows present a promising future for studios. They are set to change the way we work for good, promoting easier ways of work for artists with less pipeline friction and allowing studios to efficiently manage their data and assets without the worry of scale or lack of resources. 

We want to hear from you! If you’re keen to get involved and are on ASWF Slack look us up here or, alternatively, join the project on GitHub.