Month: May 2013

Android Camera系统结构

说到架构/构架就觉得心虚，感觉这两个字太高端了，所以我只敢用结构来表达。

以下表述如果存在错误，请指教！

之前有在一篇博客中提到Camera相关的东西
http://guoh.org/lifelog/2012/09/asop-camera-source-code-reading-video-snapshot/

比如

$ANDROID_SRC_HOME/frameworks/base/core/java/android/hardware/Camera.java
$ANDROID_SRC_HOME/frameworks/base/core/jni/android_hardware_Camera.cpp

// 注：目前以下这两个文件在JB当中应该被移动位置了，由此可见Android也是个活跃的项目

$ANDROID_SRC_HOME/frameworks/base/include/camera/Camera.h
$ANDROID_SRC_HOME/frameworks/base/libs/camera/Camera.cpp

这篇博客是阅读http://source.android.com/devices/camera.html，结合在JB代码的基础上写的笔记，可能不是那么详细和有逻辑，只关注重点和要注意的地方，所以如果你刚好读到，但是你又没有什么Android Camera的基础的话，建议还是先看看http://developer.android.com/guide/topics/media/camera.html和Reference

正式开始
大体来说Camera分为这么多部分
APP — JNI — NATIVE — BINDER — CAMERA SERVICE — HAL — DRIVER

下面列出这每一部分对应于AOSP当中的代码

APP($ANDROID_SRC_HOME/frameworks/base/core/java/android/hardware/Camera.java)

JNI($ANDROID_SRC_HOME/frameworks/base/core/jni/android_hardware_Camera.cpp)

NATIVE($ANDROID_SRC_HOME/frameworks/av/camera/Camera.cpp)

BINDER($ANDROID_SRC_HOME/frameworks/av/camera/ICameraService.cpp
       $ANDROID_SRC_HOME/frameworks/av/camera/ICameraClient.cpp)

CAMERA SERVICE($ANDROID_SRC_HOME/frameworks/av/services/camera/libcameraservice/CameraService.cpp)

HAL($ANDROID_SRC_HOME/hardware/libhardware/include/hardware/camera.h
    $ANDROID_SRC_HOME/hardware/libhardware/include/hardware/camera_common.h
    $ANDROID_SRC_HOME/frameworks/av/services/camera/libcameraservice/CameraHardwareInterface.h)

对于Galaxy Nexus来讲，HAL的具体实现位于

$ANDROID_SRC_HOME/hardware/ti/omap4xxx/camera/CameraHal.cpp
$ANDROID_SRC_HOME/hardware/ti/omap4xxx/camera/CameraHalCommon.cpp

头文件位于

$ANDROID_SRC_HOME/hardware/ti/omap4xxx/camera/inc/

对于Galaxy Nexus来讲

DRIVER(https://developers.google.com/android/nexus/drivers)

不知道看了对Camera有没有个大体的认识？
大致了解了的话，就翻代码出来看看自己感兴趣的部分吧

你可以通过repo拉取所有代码到本地看，也可以在线查看。
android/hardware/Camera.java

public static Camera open(int cameraId) {
    return new Camera(cameraId);
}
Camera(int cameraId) {
    mShutterCallback = null;
    mRawImageCallback = null;
    mJpegCallback = null;
    mPreviewCallback = null;
    mPostviewCallback = null;
    mZoomListener = null;

    Looper looper;
    if ((looper = Looper.myLooper()) != null) {
        mEventHandler = new EventHandler(this, looper);
    } else if ((looper = Looper.getMainLooper()) != null) {
        mEventHandler = new EventHandler(this, looper);
    } else {
        mEventHandler = null;
    }

    native_setup(new WeakReference<Camera>(this), cameraId);
}

以上代码描述了我们所要研究的东西的基础，开启camera，跟所有的硬件一样，使用之前需要先开启。
这里有个需要注意的地方就是mEventHandler，这个变量是做什么用的，文档有解释的比较清楚，如下：

Callbacks from other methods are delivered to the event loop of the
thread which called open(). If this thread has no event loop, then
callbacks are delivered to the main application event loop. If there
is no main application event loop, callbacks are not delivered.

如果你不是写一个玩具程序的话，你就应当注意这里的描述。

随着native_setup(new WeakReference(this), cameraId);，代码走到了
android_hardware_Camera.cpp

// connect to camera service
static void android_hardware_Camera_native_setup(JNIEnv *env, jobject thiz,
    jobject weak_this, jint cameraId)
{
    sp<Camera> camera = Camera::connect(cameraId);

    if (camera == NULL) {
        jniThrowRuntimeException(env, "Fail to connect to camera service");
        return;
    }

    // make sure camera hardware is alive
    if (camera->getStatus() != NO_ERROR) {
        jniThrowRuntimeException(env, "Camera initialization failed");
        return;
    }

    jclass clazz = env->GetObjectClass(thiz);
    if (clazz == NULL) {
        jniThrowRuntimeException(env, "Can't find android/hardware/Camera");
        return;
    }

    // We use a weak reference so the Camera object can be garbage collected.
    // The reference is only used as a proxy for callbacks.
    sp<JNICameraContext> context = new JNICameraContext(env, weak_this, clazz, camera);
    context->incStrong(thiz);
    camera->setListener(context);

    // save context in opaque field
    env->SetIntField(thiz, fields.context, (int)context.get());
}

随着sp camera = Camera::connect(cameraId);，代码走到了
av/include/camera/Camera.h，当然实现都在av/camera/Camera.cpp

sp<Camera> Camera::connect(int cameraId)
{
    ALOGV("connect");
    sp<Camera> c = new Camera();
    const sp<ICameraService>& cs = getCameraService();
    if (cs != 0) {
        c->mCamera = cs->connect(c, cameraId);
    }
    if (c->mCamera != 0) {
        c->mCamera->asBinder()->linkToDeath(c);
        c->mStatus = NO_ERROR;
    } else {
        c.clear();
    }
    return c;
}

这里getCameraService()是一个比较关键的东西，需要仔细看看。

// establish binder interface to camera service
const sp<ICameraService>& Camera::getCameraService()
{
    Mutex::Autolock _l(mLock);
    if (mCameraService.get() == 0) {
        sp<IServiceManager> sm = defaultServiceManager();
        sp<IBinder> binder;
        do {
            binder = sm->getService(String16("media.camera"));
            if (binder != 0)
                break;
            ALOGW("CameraService not published, waiting...");
            usleep(500000); // 0.5 s
        } while(true);
        if (mDeathNotifier == NULL) {
            mDeathNotifier = new DeathNotifier();
        }
        binder->linkToDeath(mDeathNotifier);
        mCameraService = interface_cast<ICameraService>(binder);
    }
    ALOGE_IF(mCameraService==0, "no CameraService!?");
    return mCameraService;
}

然后就是av/camera/ICameraService.cpp当中代理类BpCameraService的

// connect to camera service
virtual sp<ICamera> connect(const sp<ICameraClient>& cameraClient, int cameraId)
{
    Parcel data, reply;
    data.writeInterfaceToken(ICameraService::getInterfaceDescriptor());
    data.writeStrongBinder(cameraClient->asBinder());
    data.writeInt32(cameraId);
    remote()->transact(BnCameraService::CONNECT, data, &reply);
    return interface_cast<ICamera>(reply.readStrongBinder());
}

最终转移到BnCameraService的connect方法，但是这是个纯虚函数，而且CameraService继承了BnCameraService，所以请看
av/services/camera/libcameraservice/CameraService.h，当然具体实现还是在av/services/camera/libcameraservice/CameraService.cpp当中。

sp<ICamera> CameraService::connect(
        const sp<ICameraClient>& cameraClient, int cameraId) {
    int callingPid = getCallingPid();

    LOG1("CameraService::connect E (pid %d, id %d)", callingPid, cameraId);

    if (!mModule) {
        ALOGE("Camera HAL module not loaded");
        return NULL;
    }

    sp<Client> client;
    if (cameraId < 0 || cameraId >= mNumberOfCameras) {
        ALOGE("CameraService::connect X (pid %d) rejected (invalid cameraId %d).",
            callingPid, cameraId);
        return NULL;
    }

    char value[PROPERTY_VALUE_MAX];
    property_get("sys.secpolicy.camera.disabled", value, "0");
    if (strcmp(value, "1") == 0) {
        // Camera is disabled by DevicePolicyManager.
        ALOGI("Camera is disabled. connect X (pid %d) rejected", callingPid);
        return NULL;
    }

    Mutex::Autolock lock(mServiceLock);
    if (mClient[cameraId] != 0) {
        client = mClient[cameraId].promote();
        if (client != 0) {
            if (cameraClient->asBinder() == client->getCameraClient()->asBinder()) {
                LOG1("CameraService::connect X (pid %d) (the same client)",
                     callingPid);
                return client;
            } else {
                ALOGW("CameraService::connect X (pid %d) rejected (existing client).",
                      callingPid);
                return NULL;
            }
        }
        mClient[cameraId].clear();
    }

    if (mBusy[cameraId]) {
        ALOGW("CameraService::connect X (pid %d) rejected"
                " (camera %d is still busy).", callingPid, cameraId);
        return NULL;
    }

    struct camera_info info;
    if (mModule->get_camera_info(cameraId, &info) != OK) {
        ALOGE("Invalid camera id %d", cameraId);
        return NULL;
    }

    int deviceVersion;
    if (mModule->common.module_api_version == CAMERA_MODULE_API_VERSION_2_0) {
        deviceVersion = info.device_version;
    } else {
        deviceVersion = CAMERA_DEVICE_API_VERSION_1_0;
    }

    switch(deviceVersion) {
      case CAMERA_DEVICE_API_VERSION_1_0:
        client = new CameraClient(this, cameraClient, cameraId,
                info.facing, callingPid, getpid());
        break;
      case CAMERA_DEVICE_API_VERSION_2_0:
        client = new Camera2Client(this, cameraClient, cameraId,
                info.facing, callingPid, getpid());
        break;
      default:
        ALOGE("Unknown camera device HAL version: %d", deviceVersion);
        return NULL;
    }

    if (client->initialize(mModule) != OK) {
        return NULL;
    }

    cameraClient->asBinder()->linkToDeath(this);

    mClient[cameraId] = client;
    LOG1("CameraService::connect X (id %d, this pid is %d)", cameraId, getpid());
    return client;
}

比如我们这里以HAL 1.0来看，参见
av/services/camera/libcameraservice/CameraClient.h
和
av/services/camera/libcameraservice/CameraClient.cpp

CameraClient::CameraClient(const sp<CameraService>& cameraService,
        const sp<ICameraClient>& cameraClient,
        int cameraId, int cameraFacing, int clientPid, int servicePid):
        Client(cameraService, cameraClient,
                cameraId, cameraFacing, clientPid, servicePid)
{
    int callingPid = getCallingPid();
    LOG1("CameraClient::CameraClient E (pid %d, id %d)", callingPid, cameraId);

    mHardware = NULL;
    mMsgEnabled = 0;
    mSurface = 0;
    mPreviewWindow = 0;
    mDestructionStarted = false;

    // Callback is disabled by default
    mPreviewCallbackFlag = CAMERA_FRAME_CALLBACK_FLAG_NOOP;
    mOrientation = getOrientation(0, mCameraFacing == CAMERA_FACING_FRONT);
    mPlayShutterSound = true;
    LOG1("CameraClient::CameraClient X (pid %d, id %d)", callingPid, cameraId);
}

status_t CameraClient::initialize(camera_module_t *module) {
    int callingPid = getCallingPid();
    LOG1("CameraClient::initialize E (pid %d, id %d)", callingPid, mCameraId);

    char camera_device_name[10];
    status_t res;
    snprintf(camera_device_name, sizeof(camera_device_name), "%d", mCameraId);

    mHardware = new CameraHardwareInterface(camera_device_name);
    res = mHardware->initialize(&module->common);
    if (res != OK) {
        ALOGE("%s: Camera %d: unable to initialize device: %s (%d)",
                __FUNCTION__, mCameraId, strerror(-res), res);
        mHardware.clear();
        return NO_INIT;
    }

    mHardware->setCallbacks(notifyCallback,
            dataCallback,
            dataCallbackTimestamp,
            (void *)mCameraId);

    // Enable zoom, error, focus, and metadata messages by default
    enableMsgType(CAMERA_MSG_ERROR | CAMERA_MSG_ZOOM | CAMERA_MSG_FOCUS |
                  CAMERA_MSG_PREVIEW_METADATA | CAMERA_MSG_FOCUS_MOVE);

    LOG1("CameraClient::initialize X (pid %d, id %d)", callingPid, mCameraId);
    return OK;
}

当然还会用到
libhardware//include/hardware/camera.h
和
libhardware//include/hardware/camera_common.h
和
av/services/camera/libcameraservice/CameraHardwareInterface.h

status_t initialize(hw_module_t *module)
{
    ALOGI("Opening camera %s", mName.string());
    int rc = module->methods->open(module, mName.string(),
                                   (hw_device_t **)&mDevice);
    if (rc != OK) {
        ALOGE("Could not open camera %s: %d", mName.string(), rc);
        return rc;
    }
    initHalPreviewWindow();
    return rc;
}

这只是一个接口，具体的实现是厂商来做的啦

以上是从上层调用到底层，接着我们还会看一个从底层调用到上层的情况。

从代码看出有绑定3个callback到driver

mHardware->setCallbacks(notifyCallback,
        dataCallback,
        dataCallbackTimestamp,
        (void *)mCameraId);

所以这三个callback会有机会被call到
以dataCallback为例子，其中有一个是

// picture callback - compressed picture ready
void CameraClient::handleCompressedPicture(const sp<IMemory>& mem) {
    disableMsgType(CAMERA_MSG_COMPRESSED_IMAGE);

    sp<ICameraClient> c = mCameraClient;
    mLock.unlock();
    if (c != 0) {
        c->dataCallback(CAMERA_MSG_COMPRESSED_IMAGE, mem, NULL);
    }
}

通过ICameraClient调用会app，这里的ICameraClient是BpCameraClient(为什么是Bp，后面会稍加解释)

// generic data callback from camera service to app with image data
void BpCameraClient::dataCallback(int32_t msgType, const sp<IMemory>& imageData,
                  camera_frame_metadata_t *metadata)
{
    ALOGV("dataCallback");
    Parcel data, reply;
    data.writeInterfaceToken(ICameraClient::getInterfaceDescriptor());
    data.writeInt32(msgType);
    data.writeStrongBinder(imageData->asBinder());
    if (metadata) {
        data.writeInt32(metadata->number_of_faces);
        data.write(metadata->faces, sizeof(camera_face_t) * metadata->number_of_faces);
    }
    remote()->transact(DATA_CALLBACK, data, &reply, IBinder::FLAG_ONEWAY);
}

于是

status_t BnCameraClient::onTransact(
    uint32_t code, const Parcel& data, Parcel* reply, uint32_t flags)
{
    switch(code) {
        case NOTIFY_CALLBACK: {
            ...
        } break;
        case DATA_CALLBACK: {
            ALOGV("DATA_CALLBACK");
            CHECK_INTERFACE(ICameraClient, data, reply);
            int32_t msgType = data.readInt32();
            sp<IMemory> imageData = interface_cast<IMemory>(data.readStrongBinder());
            camera_frame_metadata_t *metadata = NULL;
            if (data.dataAvail() > 0) {
                metadata = new camera_frame_metadata_t;
                metadata->number_of_faces = data.readInt32();
                metadata->faces = (camera_face_t *) data.readInplace(
                        sizeof(camera_face_t) * metadata->number_of_faces);
            }
            dataCallback(msgType, imageData, metadata); // 因为pure virtual function的关系，所以会call到Camera当中的实现
            if (metadata) delete metadata;
            return NO_ERROR;
        } break;
        case DATA_CALLBACK_TIMESTAMP: {
            ...
        } break;
        default:
            return BBinder::onTransact(code, data, reply, flags);
    }
}

于是

// callback from camera service when frame or image is ready
void Camera::dataCallback(int32_t msgType, const sp<IMemory>& dataPtr,
                          camera_frame_metadata_t *metadata)
{
    sp<CameraListener> listener;
    {
        Mutex::Autolock _l(mLock);
        listener = mListener;
    }
    if (listener != NULL) {
        listener->postData(msgType, dataPtr, metadata);
    }
}

从app call到service是BnCameraClient(client是实体，service是代理，从new的代码就可以观察出来)
从service call到app是BpCameraClient(service是实体，client是代理)
而这个Bn到Bp的转换是通过

sp<ICameraClient> cameraClient = interface_cast<ICameraClient>(data.readStrongBinder());

完成的。

后面我们会列举些需要注意的点：

ICameraClient继承自IInterface
BnCameraClient继承自BnInterface，然而BnInterface又继承自ICameraClient，BBinder

Camera继承自BnCameraClient

ICameraService继承自IInterface
BnCameraService继承自BnInterface，然而BnInterface又继承自ICameraService，BBinder

ICamera继承自IInterface
BnCamera继承自BnInterface，然而BnInterface又继承自ICamera，BBinder

CameraService继承自BinderService，BnCameraService，IBinder::DeathRecipient
而BinderService主要是用来实例化service的统一接口

CameraService有个嵌套类Client继承自BnCamera
真正的实现大部分在CameraClient当中(Camera2Client是Google提供的调用新的HAL的一种方式，部分厂商会用这样的方式来实现自己的HAL，比如Qualcomm)
这才是真正的client，直接调用HW或者被HW调用。

sendCommand往driver送参数的时候会先检验是不是CameraService这层可以处理的东西(有些东西不需要到driver的)

cmd == CAMERA_CMD_SET_DISPLAY_ORIENTATION
cmd == CAMERA_CMD_ENABLE_SHUTTER_SOUND
cmd == CAMERA_CMD_PLAY_RECORDING_SOUND
cmd == CAMERA_CMD_SET_VIDEO_BUFFER_COUNT
cmd == CAMERA_CMD_PING
以上这些都CameraService自己处理了
其他均丢给driver(mHardware->sendCommand(cmd, arg1, arg2);)

接下来是一些疑问点：

// these are initialized in the constructor.
sp mHardware; // cleared after disconnect()
mHardware在disconnect当中被clear

// Make sure disconnect() is done once and once only, whether it is called
// from the user directly, or called by the destructor.
if (mHardware == 0) return;
这是防止disconnect调用一次之后，析构函数又调用一次。

在Linux上开启Core dump来调试

这是一篇原来使用Core dump的记录，整理资料的时候看到的，没有深入的分析，只是用法

基本知识不清楚的话，请在网络上搜寻查阅
http://en.wikipedia.org/wiki/Core_dump
http://en.wikipedia.org/wiki/GNU_Debugger

guohai@KNIGHT:~$ ulimit -c
0
guohai@KNIGHT:~$ ulimit -c unlimited
guohai@KNIGHT:~$ ulimit -c
unlimited
guohai@KNIGHT:~$ ./a.out 
Floating point exception (core dumped)

a.out是需要分析的程序，以上命令就是Linux上使用方法，很简单

更多情况请参考
http://www.cppblog.com/kongque/archive/2011/03/07/141262.aspx

那么在Android上怎么开启呢(首先得有root权限)？

$ adb remount
$ adb shell
root@android:/ # ulimit -c                                                     
unlimited

更改Core dump档案存储的路径(这个存储的路径可以根据需要定制)

root@android:/ # echo "/data/coredump/%p_%e" > /proc/sys/kernel/core_pattern

这样当有native crash存在的时候就会出现对应的Core dump档案了
(有时候执行没有生成Core dump，因为没有/data/coredump文件夹也可能导致无法生成Core dump，大概是没有权限创建)

然后就把档案拷贝到宿主机上，用GDB去载入档案，分析出错的原因

———–EOF———–