VIDEO FOR GAME AUDIO - Capturing and Converting Video Files for Use Within Your Audio DAW

VIDEO FOR GAME AUDIO – Capturing and Converting Video Files for Use Within Your DAW.

INTRODUCTION

If you are involved in game audio in any capacity then you’ve definitely had to deal with capturing, importing and working with videos in your DAW! There are many reasons why game audio sound designers and composers deal with a variety of video formats: game trailers, pre-rendered cinematics, in-game cinematics, animation exports from tools, capturing gameplay moments to create SFX or score a scene, video exports from the game engines tools, and many more.

There are a dizzying multitude of video containers and codec formats to deal with! It can be really confusing for audio professionals to figure out which codec formats and file types are best to work with in their audio DAWs.

I know I had a difficult time with this when I first got into professional audio. I would continually capture and get videos delivered to me in all sorts of formats. Some were compatible and worked perfectly in ProTools and others would stutter, have timing issues with frames, or be completely blank in the video preview window of my DAW!

This post will show you the BEST video formats and settings to use for videos in your audio DAW! I’ll also discuss some free and paid software options you can use to capture videos and convert your videos into these formats. Let’s get started ūüôā

BEST SETTINGS FOR VIDEOS IN YOUR AUDIO DAW

Here is the extremely short version:

Drumroll please……

**THE BEST VIDEO SETTINGS TO USE IN YOUR AUDIO DAWS**

Use .avi or .mov video container files encoded with intra-frame codecs like Photo Jpeg, Motion JPEG (MJPEG) or Pro Res.

These settings have better DAW performance because they require less decoding by your CPU, ensure your videos have¬†frame by frame accuracy for detailed scrubbing, and are compatible with every major DAW! Sounds simple enough right? Ok, if you don’t know much about video editing that probably doesn’t make any sense…

Want to know all the technical details??? Well then, let’s journey¬†further down the rabbit hole to discover¬†the whole process of capturing and converting videos into the above formats.

VideoCodecRabbitHole

 

CAPTURING

In a perfect world, every games tools would allow video exports from within the tool. However, at some point you will inevitably have to capture gameplay footage of your game so you can add sound to it. This isn’t just for cinematics. I capture gameplay footage all the time for things like animations, vfx, scripted moments, evaluating a mixes dynamic range and all sorts of other reasons.

This is a critical skill for any game audio professional. Depending on what platforms you are working on you will have lots of options to capture your gameplay.

PC GAMING CAPTURE

Fraps – Free

Pretty much every game audio professional I know has used Fraps before. It’s simple to use and it’s even free with some limitations (30 second limit and watermark). You can also pay $37 to buy it and remove the limits which I recommend. However, many sound design tasks can be accomplished with the free version and it’s limits.

CONSOLE CAPTURING

Roxio Capture

If you are playing a game on consoles you will need a game capture device to record gameplay onto your computer. Devices like Roxio Capture¬†allow you to do just that! Find the right device for your consoles video output and you are pretty much ready to go! It comes with free software to record your video in case you don’t have video editing software like Final Cut Pro or Sony Vegas.

iOS CAPTURE – iPad and iPhone

Reflector

Reflector_iOS_Capture

 

Reflector allows you to record gameplay from your iOS games! It streams the video from your iPad or iPhone to your computer. From there you just record the footage and save the video! It’s a life saver for people working on mobile games on the iOS platform!

GAME ENGINE CAPTURE

Many game tools have options for outputting videos. This is commonly used for things like animations so audio people can have video renders of animations to work with. These are often in the .mp4 format, but can be any format based on the tools and teams workflows. This is actually fairly rare to find, but I mentioned it because I would LOVE to see more tools incorporating video output in MJPEG, .avi format directly from the toolset! This would save us time in converting movies and make workflows better for everyone.

 

CONTAINERS VS. FORMATS

This is a tricky concept to wrap your head around at first, but I assure you it’s actually quite simple! Think of video files as a jar with contents inside.

Video_Containers_And_Codecs

The container type, .mov and .avi, is simply the name of the jar. The contents of the jar and how they are organized and formatted is known as the codec formats. This contents typically includes audio, metadata and other data such as subtitles.

 

CONTAINERS

The containers are simply what hold all the information of a video file! Containers DO NOT decode or encode a files contents. Instead, containers just deal with the organization of the contents of a video file. These containers merely hold and organize the contents of the file in different ways. These containers contain the codec and metadata.

Containers are also the files extension. The more common video containers are .mp4, .mov, .avi, and .mpg. I have found it best to use .mov or .avi for audio DAWs as previously mentioned.

 

FORMATS (CODECS)

Formats¬†control how the video files have been encoded and how they will be decoded. The codec of a video is COMPLETELY SEPARATE from the container that holds it. For example, a .AVI container can “contain” ANY type of codec!

In order to play a file that uses a particular CODEC you need to have that CODEC installed in your video player. Otherwise it will not display or show you an error when you try to play it. This is because your player can’t decode the video contents without the proper decoding algorithm.

Many ProTools users complain that .mp4 files don’t work in their DAW… At this point you might understand how illogical this statement really is! .mp4 files are only the CONTAINER for the video file. When people complain that .mp4 files don’t work in ProTools they are ACTUALLY¬†complaining that the H.264 quicktime codec doesn’t work in protools! Most issues involve the codecs because they control how videos were encoded, and playback once decoded!

 

 

INTRA-FRAME CODECS VS. INTER-FRAME CODECS

Audio professionals should ONLY work with Intra-Frame codecs! These retain frame by frame information and allow accurate timing and better scrubbing in your DAW.

 

Intra Frame Codecs- frame by frame. Least amount of CPU burden and frame by frame accuracy. COMMON¬†INTRA-FRAME CODECS:¬†MJPEG ‚Äď (JPEG frames bunched together), Pro Res

 

Inter Frame Codecs Should be avoided for use in daw because of loss of frame by frame information and taxing the CPU while decoding these codecs.

COMMON INTER-FRAME CODECS: H.264, MPEG4.

 

I recommend reading Wikipedia and the following links to learn more about codecs and formats: Codecs Explained

Wikipedia Video Codec

Wikipedia Container Format

Wikipedia Codec

 

BEST FORMATS (CODECS) FOR DAWS

Video_Containers_And_Codecs

Short version: Only use movie files in Motion JPEG or Pro Res formats in your DAWs! Despite their larger size these formats are less taxing on your CPU because they require less decoding, and also provide the benefit of frame by frame accuracy.

Some codecs are EXTREMELY lossy, lose frame information, and are a HUGE PAIN¬†to work with as a game audio professional! I’ve always had issues with .mp4 container files using H.264 codec.

By using Motion JPEG (MJPEG) or Pro Res formats you retain frame information, get high quality images, and don’t tax the CPU with decoding the video. Codecs like MJPEG do result in larger file sizes, but this is because there is less decoding for the CPU to do on the images. This is a GOOD thing! You want your cpu to be free from video decoding while you are working on game audio in your DAW!

CONVERTING OPTIONS

So, now you know what formats you want to use, but how do you get videos into the proper format? Conversion software does just that for you! Many of them are free as well. Here are some of your options:

CONVERSION SOFTWARE PC

Pazera (Free)- .MP4 to .AVI Simple .mp4 conversion, converts FRAPS .avi files into MJPEG .avi, and does much more than just .mp4 to .avi conversion. I use it ALL the time for converting .mp4, .avi, and .mov container videos into the proper .avi container and MJPEG codec format!

MAC

MPEG Streamclip¬†(Free) Free converter that works with multiple container types and codec formats. I haven’t used this, but some coworkers recommended it and said they use it all the time and it’s simple to use.

Quicktime Pro (- Although this still allows exporting to .avi container type and the MJPEG codec format, I don’t recommend this program anymore. It isn’t compatible with enough video files to be worth the money in my opinion.

 

VIDEO EDITORS –Compile and Convert

Video Editing Software is more useful than just having conversion software. Some DAWs, such as ProTools LE, limit how many videos you can have in a session. Using video editing software allows you to put multiple videos into one video to bypass this limitation. These programs also allow you to import videos to get exact frame timing for events.

Let’s say you captured some gameplay footage of a sound event you created¬†because your sound effect¬†plays back a few seconds late. Instead of guessing how much you need to adjust the timing of your event, you can import this video into your video editor and find out EXACTLY how late your event is being triggered.

Here are some options for video editing software.

Sony Vegas Pro This is the one I use all the time. I will be showing examples from this program in this post. I think it’s a very simple video editor for audio people to learn.

Premiere Elements I haven’t used this, but know a few game audio pros who use this software for video edits.

Final Cut Pro This is pretty much the standard for MAC users.

STEPS TO CONVERT VIDEOS TO .AVI CONTAINERS AND MJPEG CODEC FORMAT

In this example I’ll show you how to convert many video types to the proper .avi MJPEG format.

OPTION #1: Use Pazera Free .MP4 to .AVI converter

Pazera_MP4_to_AVI_Converter_Settings

  1. Open Pazera Free .MP4 to .AVI converter
  2. Drag in your video files
  3. Change the output file format to the Motion JPEG Video Codec
  4. Change the audio settings to PCM 16-Bit or 320 kbit/s MP3.
  5. Choose your output directory.
  6. Change the AVI Video bitrate to 20,000 if you want the highest quality videos!
  7. Click Convert

THAT’S IT! Thanks to free programs like Pazera¬†¬†a very complex process is now extremely simple! ¬† If this doesn’t work for your file you may have to use something like MPPEG Streamclip or Sony Vegas.

OPTION #2: Use Sony Vegas to compile and export the best format for an Audio DAW.

Here is how you can convert files to the proper container and format using Sony Vegas. Sony-Vegas-Export-Render-As-Settings-for-ProTools-Video-DAW

  1. Open Sony Vegas
  2. Create a new session
  3. Click and drag your media files into Sony Vegas
  4. Click Yes on the prompt that asks if you want to set your project video settings to match this media. This retains the frame rate of the original video.
  5. Right click at the top of the timeline and¬†choose “Set Selection to Project”
  6. Go into the File Menu and select Render As
  7. Choose the correct output folder and name
  8. Make a new Output Format Template to match the following settings
    1. .mov, NTSC DV (720X40), Frame Rate should match your initial video frame rate, Video format Photo – JPEG or Motion JPEG A
  9. Click Render

All done! This will output a .mov container file with a JPEG codec. Now you just have to import this video into your DAW and start getting creative.

MEDIA PLAYER

This post wouldn’t be complete without discussing media players. I am a big fan of VLC Media Player! It can view pretty much every type of codec. This helps you view files delivered from a wide variety of game engines so you can see what you are dealing with before encoding the videos into the proper formats ūüôā It’s even FREE! FREEEEE!!!!!!! If you don’t already have this media player it’s time to jump on board!

VLC MEDIA PLAYER

CODEC PACKS – Why you should generally avoid using them

There are many options for codec packs out there. These provide you with the codecs to play any type of video file. These have their uses, but for pro audio I don’t recommend them in most cases. Codec packs allow you to view pretty much any video format out there. By installing them you make sure you can view videos from sources that use unusual codec formats.

However, as discussed above, you should only use intra-frame codecs in your DAWs for compatibility reasons, CPU usage reduction, and to ensure proper frame-by-frame accuracy while scrubbing video in your audio DAW.

While it seems ok to just install a codec pack to view any type of video file in your audio DAW you would be losing all the benefits I just described. So, although these are a very tempting short term solution, it’s a far better solution to follow the directions in this post to use videos in the proper format!

The only reason I think it’s ok to install¬†CODEC packs is if you can’t view video files that have been delivered to you in VLC Media Player. This is highly unlikely considering VLC media player comes with it’s own codecs.

REVIEW

If you’ve come this far then congratulations! You now understand video codecs and which settings to use in your audio DAWs!

The best video container type is .mov or .avi and the best codec format is Motion JPEG. By converting your videos using these settings you are ensuring intra-frame accurate videos that minimize CPU usage and provide maximum compatibility in your DAW!

Although this is all very complex, by understanding the basics and knowing the proper settings to use you can prevent all sorts of headaches and workflow issues with improper video types.

I hope you found this useful! If you have anything to add, or questions you’d like to ask, don’t hesitate to leave a comment!.

 

Special shout out to Mark Kilborn, Stephen Schappler and Matt Bauer for inspiring me¬†to learn a bit more about codecs and their terminology ūüôā

-Aaron Brown

 

Thanks for visiting PlayDotSound! Your best resource for learning about video game sound!
Come back soon for more posts about game audio, sound design, composition, integration using engines like Unreal and Unity, and much more.
 
Play Dot Sound is created by multiple award winning sound designers and composers in the video game audio industry. Our purpose is to educate, connect and inspire both beginners and professionals to make better sounds for their games.
 
Aaron Brown –¬†http://www.AaronBrownSound.com
Brad Fotsch –¬†http://www.bradfotsch.com/
Matt Piersall –¬†http://www.Gl33k.com

 

  • http://www.playdotsound.com PlayDotSound

    Test