If you are involved in game audio in any capacity then you’ve definitely had to deal with capturing, importing and working with videos in your DAW! There are many reasons why game audio sound designers and composers deal with a variety of video formats: game trailers, pre-rendered cinematics, in-game cinematics, animation exports from tools, capturing gameplay moments to create SFX or score a scene, video exports from the game engines tools, and many more.
There are a dizzying multitude of video containers and codec formats to deal with! It can be really confusing for audio professionals to figure out which codec formats and file types are best to work with in their audio DAWs.
I know I had a difficult time with this when I first got into professional audio. I would continually capture and get videos delivered to me in all sorts of formats. Some were compatible and worked perfectly in ProTools and others would stutter, have timing issues with frames, or be completely blank in the video preview window of my DAW!
This post will show you the BEST video formats and settings to use for videos in your audio DAW! I’ll also discuss some free and paid software options you can use to capture videos and convert your videos into these formats. Let’s get started 🙂
BEST SETTINGS FOR VIDEOS IN YOUR AUDIO DAW
Here is the extremely short version:
**THE BEST VIDEO SETTINGS TO USE IN YOUR AUDIO DAWS**
Use .avi or .mov video container files encoded with intra-frame codecs like Photo Jpeg, Motion JPEG (MJPEG) or Pro Res.
These settings have better DAW performance because they require less decoding by your CPU, ensure your videos have frame by frame accuracy for detailed scrubbing, and are compatible with every major DAW! Sounds simple enough right? Ok, if you don’t know much about video editing that probably doesn’t make any sense…
Want to know all the technical details??? Well then, let’s journey further down the rabbit hole to discover the whole process of capturing and converting videos into the above formats.
In a perfect world, every games tools would allow video exports from within the tool. However, at some point you will inevitably have to capture gameplay footage of your game so you can add sound to it. This isn’t just for cinematics. I capture gameplay footage all the time for things like animations, vfx, scripted moments, evaluating a mixes dynamic range and all sorts of other reasons.
This is a critical skill for any game audio professional. Depending on what platforms you are working on you will have lots of options to capture your gameplay.
PC GAMING CAPTURE
Pretty much every game audio professional I know has used Fraps before. It’s simple to use and it’s even free with some limitations (30 second limit and watermark). You can also pay $37 to buy it and remove the limits which I recommend. However, many sound design tasks can be accomplished with the free version and it’s limits.
If you are playing a game on consoles you will need a game capture device to record gameplay onto your computer. Devices like Roxio Capture allow you to do just that! Find the right device for your consoles video output and you are pretty much ready to go! It comes with free software to record your video in case you don’t have video editing software like Final Cut Pro or Sony Vegas.
iOS CAPTURE – iPad and iPhone
Reflector allows you to record gameplay from your iOS games! It streams the video from your iPad or iPhone to your computer. From there you just record the footage and save the video! It’s a life saver for people working on mobile games on the iOS platform!
GAME ENGINE CAPTURE
Many game tools have options for outputting videos. This is commonly used for things like animations so audio people can have video renders of animations to work with. These are often in the .mp4 format, but can be any format based on the tools and teams workflows. This is actually fairly rare to find, but I mentioned it because I would LOVE to see more tools incorporating video output in MJPEG, .avi format directly from the toolset! This would save us time in converting movies and make workflows better for everyone.
CONTAINERS VS. FORMATS
This is a tricky concept to wrap your head around at first, but I assure you it’s actually quite simple! Think of video files as a jar with contents inside.
The container type, .mov and .avi, is simply the name of the jar. The contents of the jar and how they are organized and formatted is known as the codec formats. This contents typically includes audio, metadata and other data such as subtitles.
The containers are simply what hold all the information of a video file! Containers DO NOT decode or encode a files contents. Instead, containers just deal with the organization of the contents of a video file. These containers merely hold and organize the contents of the file in different ways. These containers contain the codec and metadata.
Containers are also the files extension. The more common video containers are .mp4, .mov, .avi, and .mpg. I have found it best to use .mov or .avi for audio DAWs as previously mentioned.
Formats control how the video files have been encoded and how they will be decoded. The codec of a video is COMPLETELY SEPARATE from the container that holds it. For example, a .AVI container can “contain” ANY type of codec!
In order to play a file that uses a particular CODEC you need to have that CODEC installed in your video player. Otherwise it will not display or show you an error when you try to play it. This is because your player can’t decode the video contents without the proper decoding algorithm.
Many ProTools users complain that .mp4 files don’t work in their DAW… At this point you might understand how illogical this statement really is! .mp4 files are only the CONTAINER for the video file. When people complain that .mp4 files don’t work in ProTools they are ACTUALLY complaining that the H.264 quicktime codec doesn’t work in protools! Most issues involve the codecs because they control how videos were encoded, and playback once decoded!
INTRA-FRAME CODECS VS. INTER-FRAME CODECS
Audio professionals should ONLY work with Intra-Frame codecs! These retain frame by frame information and allow accurate timing and better scrubbing in your DAW.
Intra Frame Codecs- frame by frame. Least amount of CPU burden and frame by frame accuracy. COMMON INTRA-FRAME CODECS: MJPEG – (JPEG frames bunched together), Pro Res
Inter Frame Codecs – Should be avoided for use in daw because of loss of frame by frame information and taxing the CPU while decoding these codecs.
COMMON INTER-FRAME CODECS: H.264, MPEG4.
I recommend reading Wikipedia and the following links to learn more about codecs and formats: Codecs Explained
BEST FORMATS (CODECS) FOR DAWS
Short version: Only use movie files in Motion JPEG or Pro Res formats in your DAWs! Despite their larger size these formats are less taxing on your CPU because they require less decoding, and also provide the benefit of frame by frame accuracy.
Some codecs are EXTREMELY lossy, lose frame information, and are a HUGE PAIN to work with as a game audio professional! I’ve always had issues with .mp4 container files using H.264 codec.
By using Motion JPEG (MJPEG) or Pro Res formats you retain frame information, get high quality images, and don’t tax the CPU with decoding the video. Codecs like MJPEG do result in larger file sizes, but this is because there is less decoding for the CPU to do on the images. This is a GOOD thing! You want your cpu to be free from video decoding while you are working on game audio in your DAW!
So, now you know what formats you want to use, but how do you get videos into the proper format? Conversion software does just that for you! Many of them are free as well. Here are some of your options:
CONVERSION SOFTWARE PC
Pazera (Free)- .MP4 to .AVI Simple .mp4 conversion, converts FRAPS .avi files into MJPEG .avi, and does much more than just .mp4 to .avi conversion. I use it ALL the time for converting .mp4, .avi, and .mov container videos into the proper .avi container and MJPEG codec format!
MPEG Streamclip (Free) Free converter that works with multiple container types and codec formats. I haven’t used this, but some coworkers recommended it and said they use it all the time and it’s simple to use.
Quicktime Pro (- Although this still allows exporting to .avi container type and the MJPEG codec format, I don’t recommend this program anymore. It isn’t compatible with enough video files to be worth the money in my opinion.
VIDEO EDITORS –Compile and Convert
Video Editing Software is more useful than just having conversion software. Some DAWs, such as ProTools LE, limit how many videos you can have in a session. Using video editing software allows you to put multiple videos into one video to bypass this limitation. These programs also allow you to import videos to get exact frame timing for events.
Let’s say you captured some gameplay footage of a sound event you created because your sound effect plays back a few seconds late. Instead of guessing how much you need to adjust the timing of your event, you can import this video into your video editor and find out EXACTLY how late your event is being triggered.
Here are some options for video editing software.
Sony Vegas Pro This is the one I use all the time. I will be showing examples from this program in this post. I think it’s a very simple video editor for audio people to learn.
Premiere Elements I haven’t used this, but know a few game audio pros who use this software for video edits.
Final Cut Pro This is pretty much the standard for MAC users.
STEPS TO CONVERT VIDEOS TO .AVI CONTAINERS AND MJPEG CODEC FORMAT
In this example I’ll show you how to convert many video types to the proper .avi MJPEG format.
OPTION #1: Use Pazera Free .MP4 to .AVI converter
- Open Pazera Free .MP4 to .AVI converter
- Drag in your video files
- Change the output file format to the Motion JPEG Video Codec
- Change the audio settings to PCM 16-Bit or 320 kbit/s MP3.
- Choose your output directory.
- Change the AVI Video bitrate to 20,000 if you want the highest quality videos!
- Click Convert
THAT’S IT! Thanks to free programs like Pazera a very complex process is now extremely simple! If this doesn’t work for your file you may have to use something like MPPEG Streamclip or Sony Vegas.
OPTION #2: Use Sony Vegas to compile and export the best format for an Audio DAW.
- Open Sony Vegas
- Create a new session
- Click and drag your media files into Sony Vegas
- Click Yes on the prompt that asks if you want to set your project video settings to match this media. This retains the frame rate of the original video.
- Right click at the top of the timeline and choose “Set Selection to Project”
- Go into the File Menu and select Render As
- Choose the correct output folder and name
- Make a new Output Format Template to match the following settings
- .mov, NTSC DV (720X40), Frame Rate should match your initial video frame rate, Video format Photo – JPEG or Motion JPEG A
- Click Render
All done! This will output a .mov container file with a JPEG codec. Now you just have to import this video into your DAW and start getting creative.
This post wouldn’t be complete without discussing media players. I am a big fan of VLC Media Player! It can view pretty much every type of codec. This helps you view files delivered from a wide variety of game engines so you can see what you are dealing with before encoding the videos into the proper formats 🙂 It’s even FREE! FREEEEE!!!!!!! If you don’t already have this media player it’s time to jump on board!
CODEC PACKS – Why you should generally avoid using them
There are many options for codec packs out there. These provide you with the codecs to play any type of video file. These have their uses, but for pro audio I don’t recommend them in most cases. Codec packs allow you to view pretty much any video format out there. By installing them you make sure you can view videos from sources that use unusual codec formats.
However, as discussed above, you should only use intra-frame codecs in your DAWs for compatibility reasons, CPU usage reduction, and to ensure proper frame-by-frame accuracy while scrubbing video in your audio DAW.
While it seems ok to just install a codec pack to view any type of video file in your audio DAW you would be losing all the benefits I just described. So, although these are a very tempting short term solution, it’s a far better solution to follow the directions in this post to use videos in the proper format!
The only reason I think it’s ok to install CODEC packs is if you can’t view video files that have been delivered to you in VLC Media Player. This is highly unlikely considering VLC media player comes with it’s own codecs.
The best video container type is .mov or .avi and the best codec format is Motion JPEG. By converting your videos using these settings you are ensuring intra-frame accurate videos that minimize CPU usage and provide maximum compatibility in your DAW!
Although this is all very complex, by understanding the basics and knowing the proper settings to use you can prevent all sorts of headaches and workflow issues with improper video types.
I hope you found this useful! If you have anything to add, or questions you’d like to ask, don’t hesitate to leave a comment!.
Special shout out to Mark Kilborn, Stephen Schappler and Matt Bauer for inspiring me to learn a bit more about codecs and their terminology 🙂