FFmpeg Subsystem for Screen Recording in Unreal Engine, Part I

14th December 2024 • 5 min read

In the first of this two parts series, I’ll guide you through the initial steps of creating a plugin for Unreal Engine that after the second part will provide a screen recording subsystem. This subsystem will be a GameInstance subsystem with a convenient, Blueprint-exposed API including StartRecording and StopRecording methods, as well as a struct for passing basic options such as video size, offset position, output file name, framerate, and a few other parameters. Screen recording itself will be implemented using FFmpeg libraries: avdevice, avformat, avcodec, and swscale. This approach utilizes what is known as the Facade pattern.

Instead of writing a wrapper around the FFmpeg executable, which, while works, I find somewhat sloppy, we can do better. We’re going to statically link the abovementioned libraries to our plugin, include external header files and we’ll ensure that the DLL files required for runtime are provided correctly. This means we won’t write any custom copy functions in the Build.cs file, as you might have seen in some other tutorials.

I’ve tested the implementation in Unreal Engine 5.4.4 and 5.5.0. While it might work in other versions too, I can’t guarantee it will without any changes. However, if any changes would be required, I don’t think they would be significant.

Third Party Library Plugin Sample

In your project, first go to Edit → Plugins, click the Add button in the top-left corner, and a window for creating a new plugin from a plugin template will appear.

Although we can create this plugin using a blank template, I recommend starting with the Third-Party plugin template especially if you’re creating your first plugin of this type. It provides an example of how a plugin that is meant to be a wrapper for a set of third-party libraries should typically look. Skim through the example implementation, examine the file structure and pay extra attention to the Build.cs file.

If this is really your very first encounter with Unreal Engine plugins, I strongly recommend reading the Plugins section and I also recommend to read section about integration of third-party libraries in the Unreal Engine documentation before you move on. You might also find useful to read about modules and how they differ from plugins.

FFmpeg Libraries

Now, if you have a basic understanding of how a typical plugin in UE is structured, it's time to get FFmpeg libraries. You could either build your own x64 variants for Windows from source codes or you can get them from BtbN/FFmpeg-Builds GitHub repository.

By the time of building this plugin, I used ffmpeg-n7.1-57-g364f6a5f11-win64-gpl-shared-7.1.zip from BtbN/FFmpeg-Builds/releases which was the latest build.

You can extract this archive into any location within your plugin structure, but a good practice is to have a directory called ThirdParty. I’d recommend a plugin structure like this, which I’m going to assume later in the Build.cs file.

Module Rules

In order to use FFmpeg libraries wrapped in a plugin, or any other third-party libraries that are not header-only, you need to set the module rules in Build.cs, which means:

  • Adding the path to the directory that contains all header files, typically named includes, to the PrivateIncludePaths or PublicIncludePaths list. Since there’s no need to expose these includes outside the plugin, we’re going to add them to the private list.
  • Adding paths to .lib files to the PublicAdditionalLibraries list (there’s no PrivateAdditionalLibraries list).
  • Linking .dll file paths with matching paths in the binary output directory via RuntimeDependencies list, which causes the Unreal Build Tool (UBT) to properly copy them for runtime usage.

Let’s start with the first step, which is the most straightforward. It can be done with one line of C# code.

PrivateIncludePaths.Add(Path.Combine(ModuleDirectory, "ThirdParty", "ffmpeg", "include"));

Notice to which location the ModuleDirectory variable, inherited from the parent class ModuleRules, points, by comparing the rest of the path with the image of the project structure above. Now, let's add static libraries:

string LibrariesPath = Path.Combine(ModuleDirectory, "ThirdParty", "ffmpeg", "lib");

string[] Libs =
{
    "avcodec.lib",
    "avdevice.lib",
    "avfilter.lib",
    "avformat.lib",
    "avutil.lib",
    "swresample.lib",
    "swscale.lib",
};

foreach (string Lib in Libs)
{
    PublicAdditionalLibraries.Add(Path.Combine(LibrariesPath, Lib));
}

And finally, tell UBT where to find and where to copy .dll files for runtime usage in a similar way:

string[] Dlls =
{
    "avutil-59.dll",
    "swresample-5.dll",
    "avcodec-61.dll",
    "avformat-61.dll",
    "swscale-8.dll",
    "postproc-58.dll",
    "avfilter-10.dll",
    "avdevice-61.dll",
};

string BinariesPath = Path.Combine(ModuleDirectory, "ThirdParty", "ffmpeg", "bin");

foreach (string Dll in Dlls)
{
    string DllPath = Path.Combine(BinariesPath, Dll);
    PublicDelayLoadDLLs.Add(Dll);
    RuntimeDependencies.Add($"$(BinaryOutputDir)/{Dll}", DllPath);
}

If you want to use more recent versions, don’t forget to update the filenames. For example, avcodec version 61 was the latest at the time of writing this post, but now it could be 62 or even higher. However, be aware that the API of these libraries tend to change from time to time, and the second part of this series, where we’ll be talking about screen recorder implementation, might not match newer versions.

Before we wrap this part up, I’d like to point out the line with PublicDelayLoadDLLs.Add(Dll). What are we doing here? If you know the answer to this question, you’ve probably read about integration of third-party libraries in Unreal Engine documentation, which I recommended earlier. If not, I’m providing the link again, as Delay Loading DLLs is well explained in that specific section.

You might have noticed that I’m only covering the implementation for the Windows platform. If you want your implementation to support more platforms, you’d need to wrap the Windows-specific parts with an if (Target.Platform == UnrealTargetPlatform.Win64) block and extend Build.cs with logic for other platforms, like Linux or macOS, which I’m not going to cover in this series.

And that’s all for today. Next time, we’ll continue by going through the implementation of the UScreenRecorder class with the FScreenRecorderOptions struct and the UScreenRecorderSubsystem class, which will be derived from UGameInstanceSubsystem.