I've been doing some R&D on a new workflow that converts 2D images (real or AI-generated) into a 2.5D environment in Unreal Engine for Virtual Production.

The image used in this experiment was generated in Adobe Firefly, converted to a Gaussian Splat using Apple's new SHARP model in ComfyUI, and imported into Unreal Engine via the XGrids SDK just a few minutes before shooting.

This approach is particularly beneficial for pick-up shots or rapid environment turnarounds. As a proof of concept it serves as a potential cost-effective middle-tier solution for productions that require dynamic backgrounds without the need to create full Unreal Engine environments.

Resources:

- Apple SHARP: https://lnkd.in/gwDKfXtv

- ComfyUI SHARP: https://lnkd.in/gKP-wX7b

- XGrids LCC Unreal SDK: https://lnkd.in/g6RnhzH9