Matching sound to video

OK, Chris, but that is just a recorder like mine. In your report you just say " Import mp3 audio files" and there is no video-voice sync being done. I guess you must either start the video recording and the sound recording together (which is what I do normally) or spend time lining them up on the timeline in the video editor (which takes ages especially as one has to edit out personal conversations). Does your editor have a facility for dropping sound clips onto the timeline based on the mp3-encoded time?

Having done more digging, the key to all this is SMPTE timecode.

You need an audio recorder which embeds the timecode in the audio. This is one popular model

and there are smaller ones, though nothing I can see below a good few hundred quid.

The resulting audio cannot be played with a normal player because the digital data is embedded in the audio track, but there are players which work.

There seems to be a basic issue however: you need split-second sync between the sound recorder clock (which generates the timecode) and the camera clock.

And this is hard to achieve. One could easily achieve it over say a 10 hour project (fairly cheap quartz crystal oscillator tolerance is say 5ppm, which is c. 0.1 sec over 10 hours) but how do you sync the two devices to start with? Pressing the two Start buttons simultaneously (as I have been doing with my past projects) is OK for aviation movies but won’t give you lip sync which is what the wider industry needs.

That is presumably why the DR-701 takes the video signal from the camera (HDMI), extracts the time from that, and uses that to generate the timecode recorded onto the audio. Then nobody needs to sync anything.

Video doesn’t need timecoding because the date/time is already embedded in every frame.

But clearly this won’t work with a remotely mounted camera to which there is no access before the recording session. Also I can’t practically run an HDMI cable from the camera. Nor does any action cam support timecode output over say bluetooth. I think there are some which support sound input via bluetooth and obviously that avoids the need for timecoding the sound (because the two are recorded together, in the camera) but that leaves you with an inaccessible camera radiating at 2.4GHz (which most people will say is fine ). Another approach is to extract a timecode from the wifi-streamed video which all current higher-end action cams offer… no idea how to do that since the wifi protocols are secret.

So, the nearest one will get will be able to do is to set two two clocks as accurately as possible. I don’t know if that option is available i.e. whether there is a totally standalone SMPTE sound recorder with a simple “Start Clock” button.

The idea is that one can record discontinuous video footage, discontinuous audio too, and when the two are dropped onto the timeline in the video editor (I use Vegas Pro 13 now) the editor automatically discards any audio for which there is no video.

Video doesn’t need timecoding because the date/time is already embedded in every frame.

Surprised none of the pros here picked this up (I know there are at least two here) but apparently the above is not true for common recording formats. It may be true for Apple .mov (which is the format generated by DSLRs in video mode) but isn’t for .mp4. The only way to timecode an mp4 file is to hope the start date/time is right and then frame-count from there on.

So I have given up on trying to make this work and will continue to run the camera and the sound recorder continuously. That in turn limits the usability, not least because not everybody in the cockpit wants to be recorded even if you tell them you will delete all the stuff in th editor

There are various boxes (timecode generators) for adding timecode to sound, but it is another step to do, and it won’t be of any use unless the video also contains a timecode.

But if the video is continuous and the sound is continuous, and both were started at the same moment, you don’t need a timecode anywhere. You just drop the video onto the timeline, drop the sound onto the timeline below it, and they will line up.

That’s unless you need lip sync and then you need great accuracy (better than 1 frame) between the video and the sound. Obviously that can’t be achieved by pressing the two Start Record buttons at the same time. Moreover, even with near state of the art electronics (temperature compensated crystal oscillators, accurate to a few ppm) this sync accuracy can’t be held for more than a few hours, so solutions (jam sync) have been developed for that too. Fortunately in GA one is rarely filming somebody’s lips so this is not an issue.

Tascam (TEAC) never replied to my questions and their customer service is an answering machine, but I think the above Tascam box works by picking up the time from the start of the video (i.e. from the camera’s clock) and then it counts frames arriving over the HDMI cable and uses that to make sure the timecode in the audio is frame-accurate. That was the opinion of one user in the USA, but it might be picking up a real timecode which might be in the .mov file which is the format most/all DSLRs use for video. Quite cunning really but not suitable for GA shooting because – short of some weird solutions, or a cable – you can’t get HDMI from the externally mounted camera into the sound recorder box.

I simply decided life is too short to mess with this stuff so I didn’t go into it any more.

The other approach which sidesteps all of this is to record the cockpit sound in the camera itself. Since no action cam implements the bluetooth headset profile (stupidly!), that means running a cable to its external mike input. Or other radio solutions…

What video format are you using, what camera, what software? What is the actual workflow, for a remotely mounted camera?