举个例子来看下，codec是如何和OpenMAX IL沟通的，这其实应该是Stagefright和codec的交互这篇博客当中的，想想比较独立，又比较长，干脆单独拿出来写一下。
通常来说一个codec(encoder或者decoder)无论怎么复杂，它总会提供一个接口来是我们输入一段数据，并且提供一个接口我们来获取一段数据，因为它的功能就是这样的。
所以我们要把一个codec使用OpenMAX IL的方式集成进来，对我们来说工作量不算很大。这里以FLAC encoder来举个例子(因为它比较简单)，FLAC本身这个codec相关的信息请参见http://xiph.org/flac/，在Android当中源码位于/path/to/aosp/external/flac/，在Android当中OpenMAX IL与它交互是通过libstagefright_soft_flacenc.so这个东西，我们今天就重点关注这个，它的源码位于/path/to/aosp/frameworks/av/media/libstagefright/codecs/flac/enc/。
对Android当中Stagefright和OpenMAX IL不熟悉的话，还是建议先看文章开始提到的那篇。
首先SoftFlacEncoder继承自SimpleSoftOMXComponent，重写了4个方法，分别是
virtual OMX_ERRORTYPE initCheck() const; // 在SoftOMXComponent当中是空实现，自己的codec需要实现来让他人通过这个方法来探测你的codec是否正常
virtual OMX_ERRORTYPE internalGetParameter(
OMX_INDEXTYPE index, OMX_PTR params); // 就是OpenMax IL组件的getParameter
virtual OMX_ERRORTYPE internalSetParameter(
OMX_INDEXTYPE index, const OMX_PTR params); // 就是OpenMax IL组件的setParameter
virtual void onQueueFilled(OMX_U32 portIndex); // 就是OpenMax IL组件的emptyThisBuffer和fillThisBuffer，如果你不清楚，并且还在看这篇文章的话，那你一定是在当散文看 ^_^
这四个方法如果你不明白，就先翻翻OpenMax IL/Stagefright相关代码或者规范。
另外它还有几个私有方法
void initPorts(); // OpenMAX IL通信需要port
OMX_ERRORTYPE configureEncoder(); // FLAC本身也需要些配置，比如把下面的callback设置进去
// FLAC encoder callbacks
// maps to encoderEncodeFlac()
static FLAC__StreamEncoderWriteStatus flacEncoderWriteCallback(
const FLAC__StreamEncoder *encoder, const FLAC__byte buffer[],
size_t bytes, unsigned samples, unsigned current_frame, void *client_data); // 这个是实际丢下去的callback
FLAC__StreamEncoderWriteStatus onEncodedFlacAvailable(
const FLAC__byte buffer[],
size_t bytes, unsigned samples, unsigned current_frame); // 这个只是做了个C向C++的转换
上面这两个方法是callback函数，其实就一个callback，因为如下
// static
FLAC__StreamEncoderWriteStatus SoftFlacEncoder::flacEncoderWriteCallback(
const FLAC__StreamEncoder *encoder, const FLAC__byte buffer[],
size_t bytes, unsigned samples, unsigned current_frame, void *client_data) {
return ((SoftFlacEncoder*) client_data)->onEncodedFlacAvailable(
buffer, bytes, samples, current_frame);
}
为什么要有callback？之前有讲过一个codec总有一个输入和输出，在这里FLAC有给我们一个函数FLAC__stream_encoder_process_interleaved(…)来输入数据，提供了一个callback来输出数据，仅仅这样！
可以参照下FLAC头文件的说明
/** Submit data for encoding. * This version allows you to supply the input data where the channels * are interleaved into a single array (i.e. channel0_sample0, * channel1_sample0, ... , channelN_sample0, channel0_sample1, ...). * The samples need not be block-aligned but they must be * sample-aligned, i.e. the first value should be channel0_sample0 * and the last value channelN_sampleM. Each sample should be a signed * integer, right-justified to the resolution set by * FLAC__stream_encoder_set_bits_per_sample(). For example, if the * resolution is 16 bits per sample, the samples should all be in the * range [-32768,32767]. * * For applications where channel order is important, channels must * follow the order as described in the * <A HREF="../format.html#frame_header">frame header</A>. * * \param encoder An initialized encoder instance in the OK state. * \param buffer An array of channel-interleaved data (see above). * \param samples The number of samples in one channel, the same as for * FLAC__stream_encoder_process(). For example, if * encoding two channels, \c 1000 \a samples corresponds * to a \a buffer of 2000 values. * \assert * \code encoder != NULL \endcode * \code FLAC__stream_encoder_get_state(encoder) == FLAC__STREAM_ENCODER_OK \endcode * \retval FLAC__bool * \c true if successful, else \c false; in this case, check the * encoder state with FLAC__stream_encoder_get_state() to see what * went wrong. */ FLAC_API FLAC__bool FLAC__stream_encoder_process_interleaved(FLAC__StreamEncoder *encoder, const FLAC__int32 buffer[], unsigned samples);
这个就是注册callback的地方，我们注册的是FLAC__StreamEncoderWriteCallback这个callback。
/** Initialize the encoder instance to encode native FLAC streams. * * This flavor of initialization sets up the encoder to encode to a * native FLAC stream. I/O is performed via callbacks to the client. * For encoding to a plain file via filename or open \c FILE*, * FLAC__stream_encoder_init_file() and FLAC__stream_encoder_init_FILE() * provide a simpler interface. * * This function should be called after FLAC__stream_encoder_new() and * FLAC__stream_encoder_set_*() but before FLAC__stream_encoder_process() * or FLAC__stream_encoder_process_interleaved(). * initialization succeeded. * * The call to FLAC__stream_encoder_init_stream() currently will also * immediately call the write callback several times, once with the \c fLaC * signature, and once for each encoded metadata block. * * \param encoder An uninitialized encoder instance. * \param write_callback See FLAC__StreamEncoderWriteCallback. This * pointer must not be \c NULL. * \param seek_callback See FLAC__StreamEncoderSeekCallback. This * pointer may be \c NULL if seeking is not * supported. The encoder uses seeking to go back * and write some some stream statistics to the * STREAMINFO block; this is recommended but not * necessary to create a valid FLAC stream. If * \a seek_callback is not \c NULL then a * \a tell_callback must also be supplied. * Alternatively, a dummy seek callback that just * returns \c FLAC__STREAM_ENCODER_SEEK_STATUS_UNSUPPORTED * may also be supplied, all though this is slightly * less efficient for the encoder. * \param tell_callback See FLAC__StreamEncoderTellCallback. This * pointer may be \c NULL if seeking is not * supported. If \a seek_callback is \c NULL then * this argument will be ignored. If * \a seek_callback is not \c NULL then a * \a tell_callback must also be supplied. * Alternatively, a dummy tell callback that just * returns \c FLAC__STREAM_ENCODER_TELL_STATUS_UNSUPPORTED * may also be supplied, all though this is slightly * less efficient for the encoder. * \param metadata_callback See FLAC__StreamEncoderMetadataCallback. This * pointer may be \c NULL if the callback is not * desired. If the client provides a seek callback, * this function is not necessary as the encoder * will automatically seek back and update the * STREAMINFO block. It may also be \c NULL if the * client does not support seeking, since it will * have no way of going back to update the * STREAMINFO. However the client can still supply * a callback if it would like to know the details * from the STREAMINFO. * \param client_data This value will be supplied to callbacks in their * \a client_data argument. * \assert * \code encoder != NULL \endcode * \retval FLAC__StreamEncoderInitStatus * \c FLAC__STREAM_ENCODER_INIT_STATUS_OK if initialization was successful; * see FLAC__StreamEncoderInitStatus for the meanings of other return values. */ FLAC_API FLAC__StreamEncoderInitStatus FLAC__stream_encoder_init_stream(FLAC__StreamEncoder *encoder, FLAC__StreamEncoderWriteCallback write_callback, FLAC__StreamEncoderSeekCallback seek_callback, FLAC__StreamEncoderTellCallback tell_callback, FLAC__StreamEncoderMetadataCallback metadata_callback, void *client_data);
简单介绍下这代码的意思
initPorts()初始化两个ports，0为输入，1为输出，记住一点很重要，谁申请的内存，谁释放，这点对于理解层次复杂的这套系统来说很重要。
最重要的是下面这个方法，上层把buffer满之后会call到这里，上次需要获取处理好的buffer也会call到这里
void SoftFlacEncoder::onQueueFilled(OMX_U32 portIndex) {
ALOGV("SoftFlacEncoder::onQueueFilled(portIndex=%ld)", portIndex);
if (mSignalledError) { // 出错了
return;
}
List<BufferInfo *> &inQueue = getPortQueue(0); // 输入口，好奇这里一次会不会有多个buffer，理论上来同一次就一个
List<BufferInfo *> &outQueue = getPortQueue(1); // 输出的
while (!inQueue.empty() && !outQueue.empty()) {
BufferInfo *inInfo = *inQueue.begin();
OMX_BUFFERHEADERTYPE *inHeader = inInfo->mHeader;
BufferInfo *outInfo = *outQueue.begin();
OMX_BUFFERHEADERTYPE *outHeader = outInfo->mHeader;
if (inHeader->nFlags & OMX_BUFFERFLAG_EOS) { // 编解码结束的时候，都清空标记，notifyEmptyBufferDone和notifyFillBufferDone都返回
inQueue.erase(inQueue.begin()); // 这是我们自己申请的内存，自己释放
inInfo->mOwnedByUs = false;
notifyEmptyBufferDone(inHeader);
outHeader->nFilledLen = 0;
outHeader->nFlags = OMX_BUFFERFLAG_EOS;
outQueue.erase(outQueue.begin()); // 这是我们自己申请的内存，自己释放
outInfo->mOwnedByUs = false;
notifyFillBufferDone(outHeader);
return;
}
if (inHeader->nFilledLen > kMaxNumSamplesPerFrame * sizeof(FLAC__int32) * 2) {
ALOGE("input buffer too large (%ld).", inHeader->nFilledLen);
mSignalledError = true;
notify(OMX_EventError, OMX_ErrorUndefined, 0, NULL);
return;
}
assert(mNumChannels != 0);
mEncoderWriteData = true;
mEncoderReturnedEncodedData = false;
mEncoderReturnedNbBytes = 0;
mCurrentInputTimeStamp = inHeader->nTimeStamp;
const unsigned nbInputFrames = inHeader->nFilledLen / (2 * mNumChannels);
const unsigned nbInputSamples = inHeader->nFilledLen / 2;
const OMX_S16 * const pcm16 = reinterpret_cast<OMX_S16 *>(inHeader->pBuffer); // OMX_BUFFERHEADERTYPE当中pBuffer默认为8位，这里强制按16为重新分组，所以一个buffer当中samples的数量就是nFilledLen/2
// 可是为什么要按照16位重新分组？因为设置给codec的FLAC__stream_encoder_set_bits_per_sample(mFlacStreamEncoder, 16);
for (unsigned i=0 ; i < nbInputSamples ; i++) {
mInputBufferPcm32[i] = (FLAC__int32) pcm16[i]; // 按32位解析，因为FLAC需要
}
ALOGV(" about to encode %u samples per channel", nbInputFrames);
FLAC__bool ok = FLAC__stream_encoder_process_interleaved(
mFlacStreamEncoder,
mInputBufferPcm32,
nbInputFrames /*samples per channel*/ ); // 在这里编码，因为FLAC对齐的关系，对输入的buffer需要做一定的转换，也就是上面标记出来的两处
if (ok) {
// 之前注册的callback应该是block的，也就是在FLAC__stream_encoder_process_interleaved返回callback因该也返回了
if (mEncoderReturnedEncodedData && (mEncoderReturnedNbBytes != 0)) {
ALOGV(" dequeueing buffer on output port after writing data");
outInfo->mOwnedByUs = false;
outQueue.erase(outQueue.begin());
outInfo = NULL;
notifyFillBufferDone(outHeader); // 编码好的数据通过这里就回到了上层
outHeader = NULL;
mEncoderReturnedEncodedData = false;
} else {
ALOGV(" encoder process_interleaved returned without data to write");
}
} else {
ALOGE(" error encountered during encoding");
mSignalledError = true;
notify(OMX_EventError, OMX_ErrorUndefined, 0, NULL);
return;
}
inInfo->mOwnedByUs = false;
inQueue.erase(inQueue.begin());
inInfo = NULL;
notifyEmptyBufferDone(inHeader); // 告诉上层，FLAC codec已经取走了in buffer里面的数据，这样看来这里就是上层送数据，codec编码，codec送数据，然后开始第二轮
// 不知道有没有那种实现，就是上层送数据，只要codec告诉上层已经取走了数据，上层也可以继续送数据，即使codec这个时候还没有编码完毕，还没有把上一次的数据传送给上层，也许这样在某些情况下可以加快互等拷贝/准备数据的时间，不过这是soft编码，都会占用CPU的时间，所以多CPU上也许可以改进，但那样应该会复杂些
inHeader = NULL;
}
}
接下来看callback里面
FLAC__StreamEncoderWriteStatus SoftFlacEncoder::onEncodedFlacAvailable(
const FLAC__byte buffer[],
size_t bytes, unsigned samples, unsigned current_frame) {
ALOGV("SoftFlacEncoder::onEncodedFlacAvailable(bytes=%d, samples=%d, curr_frame=%d)",
bytes, samples, current_frame);
#ifdef WRITE_FLAC_HEADER_IN_FIRST_BUFFER
if (samples == 0) {
ALOGI(" saving %d bytes of header", bytes);
memcpy(mHeader + mHeaderOffset, buffer, bytes);
mHeaderOffset += bytes;// will contain header size when finished receiving header
return FLAC__STREAM_ENCODER_WRITE_STATUS_OK;
}
#endif
if ((samples == 0) || !mEncoderWriteData) {
// called by the encoder because there's header data to save, but it's not the role
// of this component (unless WRITE_FLAC_HEADER_IN_FIRST_BUFFER is defined)
ALOGV("ignoring %d bytes of header data (samples=%d)", bytes, samples);
return FLAC__STREAM_ENCODER_WRITE_STATUS_OK;
}
List<BufferInfo *> &outQueue = getPortQueue(1);
CHECK(!outQueue.empty());
BufferInfo *outInfo = *outQueue.begin();
OMX_BUFFERHEADERTYPE *outHeader = outInfo->mHeader;
#ifdef WRITE_FLAC_HEADER_IN_FIRST_BUFFER
if (!mWroteHeader) {
ALOGI(" writing %d bytes of header on output port", mHeaderOffset);
memcpy(outHeader->pBuffer + outHeader->nOffset + outHeader->nFilledLen,
mHeader, mHeaderOffset);
outHeader->nFilledLen += mHeaderOffset;
outHeader->nOffset += mHeaderOffset;
mWroteHeader = true;
}
#endif
// write encoded data
ALOGV(" writing %d bytes of encoded data on output port", bytes);
if (bytes > outHeader->nAllocLen - outHeader->nOffset - outHeader->nFilledLen) {
ALOGE(" not enough space left to write encoded data, dropping %u bytes", bytes);
// a fatal error would stop the encoding
return FLAC__STREAM_ENCODER_WRITE_STATUS_OK;
}
memcpy(outHeader->pBuffer + outHeader->nOffset, buffer, bytes); // 拷贝数据到OpenMAX IL的buffer里面
outHeader->nTimeStamp = mCurrentInputTimeStamp;
outHeader->nOffset = 0;
outHeader->nFilledLen += bytes;
outHeader->nFlags = 0;
mEncoderReturnedEncodedData = true;
mEncoderReturnedNbBytes += bytes;
return FLAC__STREAM_ENCODER_WRITE_STATUS_OK;
}
需要注意的地方
// FLAC takes samples aligned on 32bit boundaries, use this buffer for the conversion // before passing the input data to the encoder FLAC__int32* mInputBufferPcm32; // 这个问题上面已经分析过了
短短几百行代码，这个算简单！