[vlc-devel] qtcapture and audio

[Feurstein, Michael]

Hi,

I've been trying to add audio capturing to the qtcapture module. So far, I can successfully initialize my audio device, add it to the QTCaptureSession (p_sys->session), add it to the elementary stream (es_out_Add) and control/send data (es_out_Control / es_out_Send). I've created another audio interface just like the working VLCDecompressedVideoOutput (@interface VLCDecompressedAudioOutput : QTCaptureDecompressedAudioOutput) with the QTKit delegate method:  - (void)outputAudioSampleBuffer:(QTSampleBuffer *)sampleBuffer fromConnection:(QTCaptureConnection *)connection;. It gets called and I read the AudioBufferList from the passed (QTSampleBuffer *)sampleBuffer . From this point on I am having problems correctly passing the raw audio Bytes to the Buffer with (- (mtime_t)copyCurrentAudioBytesToBuffer:(void *)buffer) . I created a new AudioBuffer which I fill with the data given by the sampleBuffer and work with that.

However when listening to the audio data it's all scrambled artefacts. Am I passing the correct data; using the correct acodec?

I've looked at other modules such as (dc1394.c; auhal.c and araw.c) to try and understand how audio data is being processed there.
The QTFormatDescriptionAudioStreamBasicDescriptionAttribute returns the following data for the QTSampleBuffer:

2011-07-29 14:52:39.983 VLC[11466:ee0b]   Format ID:                 lpcm
2011-07-29 14:52:39.983 VLC[11466:ee0b]   Format Flags:                29
2011-07-29 14:52:39.984 VLC[11466:ee0b]   Bytes per Packet:             4
2011-07-29 14:52:39.984 VLC[11466:ee0b]   Frames per Packet:            1
2011-07-29 14:52:39.984 VLC[11466:ee0b]   Bytes per Frame:              4
2011-07-29 14:52:39.985 VLC[11466:ee0b]   Channels per Frame:           2
2011-07-29 14:52:39.985 VLC[11466:ee0b]   Bits per Channel:            32

I've also looked into Apple example projects such as: http://developer.apple.com/library/ios/#samplecode/MixerHost/Introduction/Intro.html to see how they work with the AudioBuffer and so.
As I am no C expert (actually I have no experience in C) and only have experience with basic Objective-C and Java, I am kind of stuck here. I've attached my qtcapture.m file (I know it's probably really bad code - I've been shamelessly trying to hack till it works).

If someone could point me in the right direction or help me out with the last step of copying the audio data to the buffer I would be really grateful.

To my understanding the workflow would work as follows:
- in the delegate method from QTKit somehow get the correct audioBytes (from Core Audio AudioBuffer)
- calculate/get the correct i_pts
- copy those bytes to buffer (copyCurrentAudioBytesToBuffer)
- in Demux send i_pts and buffer

Thanks for your time!

Regards
Michael Feurstein
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.videolan.org/pipermail/vlc-devel/attachments/20110729/1ef8761d/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: qtcapture.m
Type: application/octet-stream
Size: 33953 bytes
Desc: qtcapture.m
URL: <http://mailman.videolan.org/pipermail/vlc-devel/attachments/20110729/1ef8761d/attachment.obj>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.videolan.org/pipermail/vlc-devel/attachments/20110729/1ef8761d/attachment.htm>